elvis@omarsar0 · XNVIDIA 将混合 MoE 大模型 Nemotron-3-Super 压缩为 Puzzle-75B-A9B。通过联合结构搜索同时优化异构 MoE 剪枝、活跃参数预算和 Mamba 剪枝，配合蒸馏、强化学习、量化及多 token 预测头的迭代流程，在保持质量的同时将交互式服务器吞吐量提升约 2 倍。在单台 8×B200 节点上达约 2 倍父模型吞吐量，单个 H100 上 1M token 并发从 1 请求提升至 8 请求。推理、编码、长上下文和智能体基准准确率不变。论文：arxiv.org/abs/2607.04371。
Banger compression paper from NVIDIA.
（bookmark it）
Bigger MoE models keep winning on quality， but serving them at interactive latency is still hard.
NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality.
Pay attention to the joint structural search. Heterogeneous MoE pruning， active-parameter budget， and Mamba pruning get optimized together rather than one at a time， wrapped in an iterative pipeline with distillation， RL， quantization， and a Multi-Token Prediction head.