This paper investigates an increasingly important topic in generative modeling: pixel-space diffusion models. Although numerous studies have explored this topic, most focus on small-scale or class-conditional settings. Consequently, a practical recipe for training pixel-space models that rival or exceed well-established latent-space counterparts remains elusive. Through a comprehensive empirical study, we first observe that direct large-scale pre-training in pixel space converges substantially more slowly than in latent space. This observation motivates a latent-to-pixel strategy that acquires generative priors efficiently in latent space and transitions to pixel space during post-training. We then systematically investigate the key design choices governing this transition, including weight initialization, data composition, prediction target, decoder architecture, and noise schedule, and identify a practical recipe that makes the resulting pixel-space models match or outperform their latent-space counterparts while delivering 3.18 to 4.75 times end-to-end inference speedups. We hope that our findings provide useful empirical insights and practical guidelines for future research on pixel-space generation.
像素空间文生图扩散模型训练实证研究:潜空间到像素空间策略带来 3.18-4.75 倍推理加速
AI 导读
一项实证研究发现,像素空间扩散模型直接大规模预训练收敛速度远慢于潜空间模型,由此提出“潜空间到像素空间”策略:先在潜空间高效获取生成先验,再在后训练阶段转向像素空间。该策略结合权重初始化、数据构成、预测目标、解码器架构与噪声调度等设计选择,使像素空间模型达到或超越潜空间模型,并实现 3.18 至 4.75 倍的端到端推理加速。
HuggingFace Daily Papers(社区热门论文)
53
AI 编辑部评分,满分 100像素空间文生图扩散模型训练实证研究:潜空间到像素空间策略带来 3.18-4.75 倍推理加速
一项实证研究发现,像素空间扩散模型直接大规模预训练收敛速度远慢于潜空间模型,由此提出“潜空间到像素空间”策略:先在潜空间高效获取生成先验,再在后训练阶段转向像素空间。该策略结合权重初始化、数据构成、预测目标、解码器架构与噪声调度等设计选择,使像素空间模型达到或超越潜空间模型,并实现 3.18 至 4.75 倍的端到端推理加速。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org