Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past its peak. We adopted Quantization-Aware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B to 60B to MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 times less weight memory and half the teacher's parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 times faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyper-parameter search.
量化感知修复:恢复压缩 4 位 LLM 的实用方案
AI 导读
研究团队提出量化感知修复(QAH)替代量化感知训练(QAT),以恢复压缩并量化至 4 位的 LLM。在 GPT-OSS 120B 压缩至 60B 并量化为 MXFP4 的流程中,QAH 学生模型在 9 项基准的 7 项上匹敌或超越其 bfloat16 来源,权重内存减少约 4 倍,并以 Hypernova-60B 开源发布。
HuggingFace Daily Papers(社区热门论文)
52
AI 编辑部评分,满分 100量化感知修复:恢复压缩 4 位 LLM 的实用方案
研究团队提出量化感知修复(QAH)替代量化感知训练(QAT),以恢复压缩并量化至 4 位的 LLM。在 GPT-OSS 120B 压缩至 60B 并量化为 MXFP4 的流程中,QAH 学生模型在 9 项基准的 7 项上匹敌或超越其 bfloat16 来源,权重内存减少约 4 倍,并以 Hypernova-60B 开源发布。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org