量化感知修复:恢复压缩 4 位 LLM 的实用方案

HuggingFace Daily Papers(社区热门论文)·2026-08-21 08:00·4天前
AI 导读

研究团队提出量化感知修复(QAH)替代量化感知训练(QAT),以恢复压缩并量化至 4 位的 LLM。在 GPT-OSS 120B 压缩至 60B 并量化为 MXFP4 的流程中,QAH 学生模型在 9 项基准的 7 项上匹敌或超越其 bfloat16 来源,权重内存减少约 4 倍,并以 Hypernova-60B 开源发布。

HuggingFace Daily Papers(社区热门论文)
52AI 编辑部评分,满分 100

量化感知修复:恢复压缩 4 位 LLM 的实用方案

2026-08-21 08:00· 4天前
AI 导读

研究团队提出量化感知修复(QAH)替代量化感知训练(QAT),以恢复压缩并量化至 4 位的 LLM。在 GPT-OSS 120B 压缩至 60B 并量化为 MXFP4 的流程中,QAH 学生模型在 9 项基准的 7 项上匹敌或超越其 bfloat16 来源,权重内存减少约 4 倍,并以 Hypernova-60B 开源发布。

Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past its peak. We adopted Quantization-Aware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B to 60B to MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 times less weight memory and half the teacher's parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 times faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyper-parameter search.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org