Abra:扩散图像训练的规模化定律研究

HuggingFace Daily Papers(社区热门论文)·2026-08-18 08:00·6天前
AI 导读

Abra 研究团队对文生图扩散模型进行了系统性规模化定律分析,训练计算量横跨三个数量级(10¹⁹ 至 10²² FLOPs),远超此前工作。研究发现扩散模型与语言模型一样可预测,但需要约每参数 200 个图像 token 才能达到计算最优,是 LLM 的 Chinchilla 最优配方的十倍。此外,扩散模型对过度训练具有鲁棒性,且预测性可延伸至生成质量、最优 CFG 设置及训练曲线形状。

HuggingFace Daily Papers(社区热门论文)
56AI 编辑部评分,满分 100

Abra:扩散图像训练的规模化定律研究

2026-08-18 08:00· 6天前
AI 导读

Abra 研究团队对文生图扩散模型进行了系统性规模化定律分析,训练计算量横跨三个数量级(10¹⁹ 至 10²² FLOPs),远超此前工作。研究发现扩散模型与语言模型一样可预测,但需要约每参数 200 个图像 token 才能达到计算最优,是 LLM 的 Chinchilla 最优配方的十倍。此外,扩散模型对过度训练具有鲁棒性,且预测性可延伸至生成质量、最优 CFG 设置及训练曲线形状。

Compute-optimal scaling laws guide the training of frontier language models yet remain largely unexplored for visual generation. We present a systematic scaling law study for text-to-image diffusion models using Abra, a controlled family of flow-matching transformers trained across three orders of magnitude worth of compute (10^{19} to 10^{22} FLOPs), reaching significantly larger compute budgets than previous works. We demonstrate that diffusion models scale just as predictably as language models but require far more data to train optimally: compute optimality occurs at approximately 200 image tokens per parameter, ten times the Chinchilla compute-optimal prescription for LLMs. We show that unlike language models, diffusion models are robust to overtraining and that practitioners should err on the side of more data rather than a larger model. Finally, we show that this predictability extends beyond training loss to generative quality metrics, optimal CFG settings, representation quality, and even the shape of the training curves, which collapse onto a universal form.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org