New paper proposes end-to-end training for generative models via explorative modeling

In 2012, AlexNet ended an era with an overwhelming victory. Before then, image recognition relied on humans manually designing layered feature extraction pipelines; AlexNet demonstrated—later confirmed repeatedly—that handing the entire task end-to-end to a model to learn on its own almost always outperformed human-crafted, multi-stage pipelines.

From image classification to object detection and then to image segmentation, each leap in deep learning boils down to the same idea: let it learn it all in one go.

There is only one area that remains an exception: generative models.

Today's most powerful and scalable generative models, whether autoregressive or diffusion-based, are not end-to-end.

During training, they learn only to predict a single step, but during inference, they must repeatedly unfold this step hundreds or even thousands of times, like a recurrent network.

The sampling method used for training differs from that used for inference. This discrepancy leads to a longstanding issue: errors at each step are fed into the next, causing the input to gradually drift away from the distribution seen during training, with errors accumulating over time. In academic terms, this is known as "exposure bias."

In other words, the most fundamental lesson in deep learning—that end-to-end is better—has never truly been realized in generative models over the past decade.

Recently, a paper from UIUC and Harvard University attempted to fill in this final piece of the puzzle.

The author named this new paradigm Explorative Modeling (XM), whose concept is simple to the point of naivety, yet points to a bold conclusion: generative models have, beyond parameters and data, a third axis that can be scaled.

Project website: https://explorative-modeling.github.io

Paper URL: https://arxiv.org/abs/2607.27372

Code repository: https://github.com/alexiglad/XM

To understand what this paper is addressing, you first need to understand why generation is so challenging.

In ordinary supervised learning (such as classification), each input typically has only one correct answer, so the model only needs to learn a deterministic mapping.

But the generation is different. When you ask the model to "generate a dog," the correct answer could have infinitely many possibilities. These valid outputs are individual modes within the data distribution—each representing a distinct peak in the distribution. Generation is difficult precisely because it requires capturing all of these modes simultaneously.

The issue is that mainstream generative models are trained using reconstruction losses (such as mean squared error). When a single input is paired with many different valid targets, the optimal solution provided by the reconstruction loss is the average of those targets. For most data, this average does not lie on the data manifold but instead falls between several modes, resembling none of them.

The figure in the paper is very intuitive: without any tricks, if the model directly performs end-to-end regression, three clusters of scatter points will be predicted as a single point in the center, a photo of a dog will blur into a mess, and a sentence will degenerate into repeating "the" endlessly. This is called "mode blurring"—where the optimal solution is precisely the one that looks least like the true data.

How do existing models bypass this? By breaking down the act of “generation” into smaller steps. Autoregressive models predict only one element at a time; diffusion models remove only a small amount of noise at each step. By reducing each tiny step to nearly a single mode, the reconstruction loss no longer needs to take an average.

This "split generation process" is precisely why diffusion and autoregressive models can produce high-quality samples, but it is also what prevents them from being end-to-end.

The author thus raises a key question: a generative model has only two aspects that can be broken down—how it generates and how it is trained.

If breaking this path destroys end-to-end, why not break the training instead?

Explorative Modeling breaks down the training loop itself.

Its mechanism can be explained in one sentence: at each training step, instead of generating only one sample to force-fit the target, the model generates K candidates and selects only the one closest to the true data for training and gradient backpropagation. The paper implements this as a simple 3- to 5-line for loop, so straightforward it’s almost suspicious (Algorithm 1).

Why does this solve the mode ambiguity?

Consider a real-life scenario: guessing where darts will land. If you’re only allowed one guess, your best strategy is to guess the average position of all darts—but that’s often a spot on the dartboard where very few darts actually hit. However, if you’re allowed K guesses and only your closest guess counts, your optimal strategy changes immediately: you’ll spread out your guesses so that each one covers a different cluster of possible landing points.

The same applies to the model. When allowed to explore K candidates, different input noises each "claim" a distinct mode, rather than all clustering around the mean. The more exploration steps, the more modes the model can reliably capture.

The author named this long-neglected capability "generative expressivity" and noted that it is determined by the training objective itself—it will not increase on its own, no matter how much you scale up parameters or data.

This also explains a long-observed anomaly in the industry: why today’s best models are so heavily reliant on guidance techniques.

Classifier-free guidance essentially involves pushing the prediction away from that vague average. But if the model itself isn’t fuzzy, why push it at all? The reason guidance works is precisely because of the ambiguity left by the mode.

The paper also presents two exploration directions: Forward and Reverse. The Forward direction fixes a real target and searches for the closest match within its own generation, favoring "recall" (covering all modes); the Reverse direction fixes a generated sample and searches for the closest match in the real data, favoring "precision" with nearly no additional computational cost, at the expense of potentially collapsing into a few modes. The two are complementary and can be used together.

The paper's most significant conclusion is that it has validated "exploration" as a genuine scaling axis.

The author applied exploration to diffusion/flow models, Jumpy models, and masked diffusion language models, consistently observing monotonic performance improvements across image, video, and language modalities. More importantly, the gains scale dramatically with size—the larger the model, the greater the improvement.

The paper reports the following figures: as data scale increases, the benefit from exploration rises from 7% to 36%; as model size grows, it increases from 13% to 23%; when computational power triples, the efficiency gain more than doubles.

In terms of efficiency, it achieved a 4.1x improvement in FLOP efficiency, a 6.2x improvement in sample efficiency, and a 47% improvement in parameter efficiency. For image generation, it pushed the current state-of-the-art RAE recipe to an unconditioned FID of 1.43 on ImageNet, nearing the industry’s best performance.

A Large model that explores five modes can even outperform an XLarge model with 47% more parameters but no exploration.

The implication of this trend is significant. The author explains that at smaller scales, models are primarily constrained by parameters and data, and generation expressiveness is not yet the bottleneck; but once parameters and data are scaled up to the point where they are no longer limiting, generation expressiveness becomes increasingly the true bottleneck—and it is precisely what can be directly scaled.

Given that current real-world foundation model training requires computing power approximately four orders of magnitude greater than the largest experiment in this paper, the authors believe that the numbers reported in the paper are likely only a lower bound of the returns achievable at larger scales.

What happens if we push exploration to its limit? The answer returns to the original mystery: generative models can now be end-to-end.

The authors treat XM as an independent end-to-end model for robotic control tasks. In behavior cloning, their exploratory policy matches or even surpasses the performance of a diffusion policy requiring 100 forward passes, using only a single forward pass. In goal-oriented world modeling, their exploratory world model achieves better average performance using 16 to 256 times less inference compute than the diffusion model.

The source of this gap is clear: diffusion models trade off expressiveness by generating outputs over hundreds of steps during inference, while end-to-end XM shifts this cost to exploration during training, allowing inference to require only a single forward pass. Handling multiple modes—this same task—is either slowly decomposed during inference or fully explored in one go during training; this paper chooses the latter.

The first author of this paper, Alexi Gladstone, is no stranger. In July 2025, his work on Energy-Based Transformers (EBT) sparked considerable discussion on social media.

The work claims to have outperformed the scaling curve of the standard feedforward Transformer across multiple dimensions for the first time, and aims to extend "System 2 thinking" to any modality. See the Jiqizhixin article: "A New Paradigm Has Arrived! The New Energy Model Breaks the Transformer++ Scaling Limit, Achieving 35% Faster Training Scaling."

Explorative Modeling is consistent with his一贯 thinking: both question whether models can achieve what a single forward pass cannot, through some form of search or exploration. This paper builds further on his earlier theoretical framework, Mode Forcing, developed with his collaborators Yilun Du and Heng Ji. The paper openly acknowledges that, thanks to this prior theory, most of XM’s results were first predicted theoretically and then verified experimentally—an approach uncommon in the deep learning community, where the norm is to run experiments first and then interpret the results.

The authors also candidly acknowledge the limitations: the idea of best-of-K is not new—it has been done many times before; their true contribution is in clearly understanding what this simple loop actually does: it directly amplifies generative expressiveness without splitting the generation process.

In addition, autoregressive language models remain the most challenging problem, and purely end-to-end Forward XM is still too costly for highly multimodal distributions (such as image generation), leaving these for future work.

For over a decade, we’ve been accustomed to tuning generative models with two knobs: making them larger and feeding them more data.

The third knob proposed in this paper is quite simple: have the model make multiple guesses and keep only the best one. But if the trend it reveals holds true, then as scale continues to grow, the first two knobs will eventually reach their limits, while the third has only just begun to turn.

Reference link

https://x.com/AlexiGlad/status/2083230922196107288

This article is from the WeChat public account "Machine Heart" (ID: almosthuman2014), authored by Panda.