The Meta/Oxford study finds, a multimodal model may need surprisingly little image-generation data if language and visual understanding are trained with it from the start.

So, you probably don't need to spend that much training compute teaching a multimodal model to generate images.

Language training already helps the model with vision, and learning to understand images also makes it better at generating them. But training it to generate images does very little for language or image understanding.

That leads to a very uneven training mix.

In their 1T-token experiments, the best overall split was 70% language, 25% image understanding, and only 5% image generation.

They then tested this at 13.5B scale over 2T tokens. Even with 5x fewer image-generation tokens than the balanced setup, GenEval improved from 0.467 to 0.482, while language and image understanding improved too.

There's another lesson: don't bolt vision on too late.

The longer the model trains only on language, the more it starts ignoring the image and relying on language shortcuts. The authors call this "vision laziness."

• arxiv. org/abs/2608.05000

Title: "Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes"