New Harvard paper shows generative models may be missing a third scaling axis: how much they explore during training.

What if the next scaling axis for generative models is not a bigger model or more data, but more candidate generations per training step?

Added to a strong Representation Autoencoder (RAE) image-generation recipe, exploration reaches the baseline's final performance with 6.2× fewer training samples processed and 4.1× fewer FLOPs.

This paper treats best-of-K training as something bigger: a way to scale how many modes a generative model can learn without adding inference steps.

Today, diffusion, flow, and autoregressive models handle multimodal targets largely by splitting generation into many easier steps, which also creates a mismatch between how they train and how they sample.

Explorative Modeling moves that burden into training instead.

At each update, the model tries K candidate generations and learns only from the candidate closest to the target, letting different latents specialize to different modes rather than being pulled toward an average.

If the scaling trend survives larger runs, compute-optimal generative training may need to budget for exploration alongside parameters and data.

• arxiv. org/abs/2607.27372

Title: "Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation"