SemiAnalysis@SemiAnalysis_ · XAI 摘要
SemiAnalysis 指出推理正被多轮“切分”以降低成本。第一步按阶段拆分：prefill 与 decode 用不同芯片；第二步按层拆分：attention 用 HBM 富裕的 GPU，前馈网络用 SRAM 基芯片；第三步按时间拆分：工作负载切片为执行窗口，在集群中交错调度。每次切分回收闲置利用率，从而降低每 token 成本。更便宜的 token 不会压缩需求，反而刺激增长——这是 MLSys 2026 的核心叙事。
Inference keeps getting carved up， and every cut makes intelligence cheaper.
First we split by phase： prefill on one set of chips， decode on another. Then by layer： attention on HBM-rich GPUs， the feed-forward network on SRAM-based silicon. Now by time itself： workloads sliced into execution windows and interleaved across the cluster.
Each split recovers wasted utilization. Recovered utilization lowers the cost per token. We think cheaper tokens don't shrink demand， they grow it.
That was the real story of MLSys 2026. （1/2）🧵