# Meta/Oxford 研究：多模态预训练仅需 5% 图像生成数据

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-14 11:21
- AIHOT 分数：36
- AIHOT 链接：https://aihot.virxact.com/items/cmssdsxrc04ksrod05kfkw5xr
- 原文链接：https://x.com/rohanpaul_ai/status/2088103654519968190

## AI 摘要

Meta 与牛津大学研究发现，若语言与视觉理解同步训练，多模态模型仅需极少量图像生成数据。1T token 实验中最佳配比为 70% 语言、25% 图像理解、5% 图像生成；13.5B 规模 2T token 测试中，图像生成 token 减少 5 倍，GenEval 仍从 0.467 提升至 0.482。研究还指出过晚引入视觉会导致“视觉惰性”，模型倾向依赖语言捷径。

## 正文

The Meta/Oxford study finds, a multimodal model may need surprisingly little image-generation data if language and visual understanding are trained with it from the start.

So, you probably don't need to spend that much training compute teaching a multimodal model to generate images.

Language training already helps the model with vision, and learning to understand images also makes it better at generating them. But training it to generate images does very little for language or image understanding.

That leads to a very uneven training mix.

In their 1T-token experiments, the best overall split was 70% language, 25% image understanding, and only 5% image generation.

They then tested this at 13.5B scale over 2T tokens. Even with 5x fewer image-generation tokens than the balanced setup, GenEval improved from 0.467 to 0.482, while language and image understanding improved too.

There's another lesson: don't bolt vision on too late.

The longer the model trains only on language, the more it starts ignoring the image and relying on language shortcuts. The authors call this "vision laziness."

- arxiv. org/abs/2608.05000

Title: "Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes"