Meta TBD 实验室发布 Avocado(内部代号 Muse Spark),未开源。团队仅用 9 个月重建预训练技术栈,以不到 Llama 4 Maverick 十分之一的算力达到相近能力。作者认为,基础设施才是 AI 实验室的真正护城河,决定模型训练速度和实验迭代效率。
Meta released Avocado, they call it Muse Spark.
It's not open source (a bit sad).
Meta TBD lab rebuilt the entire pretraining stack in 9 months and reached similar capability with >10x less compute than Llama 4 Maverick.
I still think infra is the real moat in AI labs. You can train models much faster with a good infra, and it allows researchers to experiment with many more ideas much more quickly.