Hey, I'm Eliahu Horwitz

Currently, I am a Computer Science PhD student at the Hebrew University of Jerusalem, studying under the supervision of Prof. Yedid Hoshen.

However, research has not always been my priority. Beginning as a self-taught software developer, I have worked with a variety of technologies across the tech stack. More recently, I have pursued a career in research. My passion is demystifying complex ideas and developing simpler, intuitive alternatives.

Email Scholar Semantic Scholar Twitter Github Linkedin Resume

Publications

An image showing an overview of ProbeX

Learning on Model Weights using Tree Experts

Arxiv Preprint
Eliahu Horwitz*, Bar Cavia*, Jonathan Kahana*, Yedid Hoshen

We identify a key property of real-world models: most public models belong to a small set of Model Trees, where all models within a tree are fine-tuned from a common ancestor (e.g., a foundation model). Importantly, we find that within each tree there is less nuisance variation between models. We introduce Probing Experts (ProbeX), a theoretically motivated, lightweight probing method. Notably, ProbeX is the first probing method designed to learn from the weights of just a single model layer. Our results show that ProbeX can effectively map the weights of large models into a shared weight-language embedding space. Furthermore, we demonstrate the impressive generalization of our method, achieving zero-shot model classification and retrieval.

Project Page arXiv Code - Coming soon... 🤗 Dataset - Coming Soon...

An image showing an overview of ProbeGen

Deep Linear Probe Generators for Weight Space Learning

Arxiv Preprint
Jonathan Kahana, Eliahu Horwitz, Imri Shuval, Yedid Hoshen

We conduct a study of weight space analysis methods and observe that probing is a promising approach for such tasks. However, we find that a vanilla probing approach performs no better than probing a neural network with random data. To address this, we propose "Deep Linear Probe Generators" (ProbeGen), a simple and effective modification to probing-based methods of weight space analysis. ProbeGen introduces a shared generator module with a deep linear architecture, providing an inductive bias toward structured probes. ProbeGen significantly outperforms the state-of-the-art and is highly efficient, requiring 30 to 1,000 times fewer FLOPs than other leading approache.

Project Page arXiv Code

Data Size Recovery from Lora Weights

Arxiv Preprint
Mohammad Salama, Jonathan Kahana, Eliahu Horwitz, Yedid Hoshen

We introduce the task of dataset size recovery that aims to determine the number of samples used to train a model based on its weights. We then propose DSiRe, a method for recovering the number of images used to fine-tune a model, in the common case where fine-tuning uses LoRA. We discover that both the norm and the spectrum of the LoRA matrices are closely linked to the fine-tuning dataset size. To evaluate dataset size recovery of LoRA weights, we develop and release a new benchmark, LoRA-WiSE, consisting of over 25000 weight snapshots.

Project Page arXiv Code 🤗 Dataset

Real-Time Deepfake Detection in the Real-World

Arxiv Preprint
Bar Cavia, Eliahu Horwitz, Tal Reiss, Yedid Hoshen

We introduce "Locally Aware Deepfake Detection Algorithm" (LaDeDa), using merely patch-level information LaDeDa significantly improves over current SoTA, achieving around 99% mAP on current benchmarks. We further distill LaDeDa into Tiny-LaDeDa which has 375x fewer FLOPs and is 10,000x more parameter-efficient than LaDeDa. These almost-perfect scores raise the question: is the task of deepfake detection close to being solved? We find that current training protocols prevent methods from generalizing to real-world deepfakes, we therefore introduce WildRF, a new deepfake detection dataset curated from several popular social networks.

Project Page arXiv Code

Unsupervised Model Tree Heritage Recovery

Arxiv Preprint
Eliahu Horwitz, Asaf Shul, Yedid Hoshen

Inspired by Darwin's tree of life, we define the Model Tree which describes the origin of models i.e., the parent model that was used to fine-tune the target model. Similarly to the natural world, the tree structure is unknown. We therefore introduce the task of Model Tree Heritage Recovery (MoTHer Recovery). Our hypothesis is that model weights encode this, we find that certain distributional properties of the weights evolve monotonically during training, which enables us to classify the relationship between two given models. MoTHer recovery reconstructs entire model hierarchies, represented by a directed tree.

Project Page arXiv Code

Distilling Datasets Into Less Than One Image

Arxiv Preprint
Asaf Shul*, Eliahu Horwitz*, Yedid Hoshen

In this paper, we push the boundaries of dataset distillation, compressing the dataset into less than an image-per-class. We therefore propose Poster Dataset Distillation (PoDD), a new approach that distills the entire original dataset into a single poster. The poster approach motivates new technical solutions for creating training images and learnable labels. Our method can achieve comparable or better performance with less than an image-per-class compared to existing methods that use one image-per-class. Our method establishes a new state-of-the-art performance on CIFAR-10, CIFAR-100, and CUB200 using as little as 0.3 images-per-class.

Project Page arXiv Code

Recovering the Pre-Fine-Tuning Weights of Generative Models

ICML, 2024
Eliahu Horwitz, Jonathan Kahana, Yedid Hoshen

The dominant paradigm in generative modeling consists of two steps: i) pre-training on a large-scale but unsafe dataset, ii) aligning the pre-trained model with human values via fine-tuning. This practice is considered safe, as no current method can recover the unsafe, pre-fine-tuning model weights. In this paper, we demonstrate that this assumption is often false. Concretely, we present Spectral DeTuning, a method that can recover the weights of the pre-fine-tuning model using a few low-rank (LoRA) fine-tuned models. In contrast to previous attacks that attempt to recover pre-fine-tuning capabilities, our method aims to recover the exact pre-fine-tuning weights. Our approach exploits this new vulnerability against large-scale models such as a personalized Stable Diffusion and an aligned Mistral.

Project Page arXiv Code 🤗 Dataset

Dreamix: Video Diffusion Models are General Video Editors

Arxiv Preprint
Eyal Molad*, Eliahu Horwitz*, Dani Valevski*, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan†, Yedid Hoshen†

We present the first diffusion-based method that is able to perform text-based motion and appearance editing of general, real-world videos. Our approach uses a video diffusion model to combine, at inference time, the low-resolution spatio-temporal information from the original video with new, high resolution information that it synthesized to align with the guiding text prompt. We extend our method for animating images, bringing them to life by adding motion to existing or new objects, and camera movements.

Project Page arXiv

Conffusion: Confidence Intervals for Diffusion Models

Arxiv Preprint
Eliahu Horwitz, Yedid Hoshen

We construct a confidence interval around each generated pixel such that the true value of the pixel is guaranteed to fall within the interval with a probability set by the user. Since diffusion models parametrize the data distribution, a straightforward way of constructing such intervals is by drawing multiple samples and calculating their bounds. However, this method has several drawbacks: i) slow sampling speeds ii) suboptimal bounds iii) requires training a diffusion model per task. To mitigate these shortcomings we propose Conffusion, wherein we fine-tune a pre-trained diffusion model to predict interval bounds in a single forward pass.

Project Page arXiv Code

Anomaly Detection Requires Better Representations

SSLWIN Workshop - ECCV 2022
Tal Reiss, Niv Cohen, Eliahu Horwitz, Ron Abutbul, Yedid Hoshen

In this position paper, we first explain how self-supervised representations can be easily used to achieve state-of-the-art performance in commonly reported anomaly detection benchmarks.
We then argue that tackling the next-generation of anomaly detection tasks requires new technical and conceptual improvements in representation learning.

Project Page arXiv

Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection

VAND Workshop - CVPR 2023
Eliahu Horwitz, Yedid Hoshen

We conduct a careful study seeking answers to several questions: questions:
(1) Do current 3D AD&S methods truly outperform state-of-the-art 2D methods on 3D data?
(2) Is 3D information potentially useful for AD&S?
(3) What are the key properties of successful 3D AD&S representations?
(4) Are there complementary benefits from using 3D shape and color modalities?

Project Page arXiv Code

DeepSIM: Image Shape Manipulation from a Single Augmented Training Sample

ICCV, 2021 (Oral)
Yael Vinker*, Eliahu Horwitz*, Nir Zabari, Yedid Hoshen

We present DeepSIM, a generative model for conditional image manipulation based on a single image. We find that extensive augmentation is key for enabling single image training, and incorporate the use of thin-plate-spline (TPS) as an effective augmentation. Our network learns to map between a primitive representation of the image to the image itself. At manipulation time, our generator allows for making complex image changes by modifying the primitive input representation and mapping it through the network.

Project Page arXiv Code