Feed
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Generate K guesses during training, learn only from the closest, and blurry averages turn into sharp samples.
4.1×
fewer FLOPs
FLOP efficiency from exploration
Reaches the strongest image recipe's best score with far less compute
How it works
Averages blur modes
When one input could map to many valid outputs, squared-error training predicts their mean, which matches no real sample.
Best-of-K training
Each step, generate K candidates from different noise, compare to the data, and backpropagate only through the closest one.
New scaling knob
Raising K improves image, video and language models, and lets one-pass policies match 100-step diffusion.
From the paper