Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H100 roofline almost perfectly. Its trained weights, however, expose only one decoding path - an absorbed MQA form - which ties efficient inference to H100-class compute-bandwidth ratios, forfeits tensor parallelism along the head axis, and yields no Multi-Token Prediction (MTP) gain on commodity inference GPUs such as the export-restricted H20. We propose Group-Query Latent Attention (GQLA), a minimal modification of MLA whose trained weights expose two algebraically equivalent decoding paths over the same parameters: an MQA-absorb path identical to MLA's, and a GQA path with a per-group expanded cache. The runtime picks the path that matches the target hardware - no retraining, no custom kernels - so a single set of GQLA weights pins the rooflines of both H100 (MQA-absorb, s_q=1) and H20 (GQA + MTP, s_q=2), while supporting up to 8-way zero-redundancy tensor parallelism on the GQA path. To avoid pretraining from scratch we extend TransMLA into TransGQLA, which converts a pretrained GQA checkpoint into a GQLA model; on LLaMA-3-8B it compresses the per-token KV cache to 28.125% of the GQA baseline on the MQA-absorb path while structurally preserving GQA-level traffic on the per-group path.
GQLA:面向硬件自适应的大语言模型解码的分组查询潜在注意力
AI 导读
本研究提出分组查询潜在注意力(GQLA),对DeepSeek-V2/V3中的多头潜在注意力(MLA)进行最小修改,使其在一套权重上暴露两种等效解码路径:与MLA相同的MQA吸收路径,以及带有每组扩展缓存的GQA路径。运行时可根据硬件自动选择路径,无需重新训练。单一的GQLA权重能同时适配H100(采用MQA吸收)与H20(采用GQA及多令牌预测)的硬件性能上限,并在GQA路径上支持高达8路的零冗余张量并行。通过扩展TransMLA为TransGQLA,可将预训练的GQA模型转换为GQLA模型。在LLaMA-3-8B上的实验表明,其MQA吸收路径将每令牌的KV缓存压缩至GQA基线的28.125%,同时在分组路径上结构性保留了GQA级别的流量效率。
HuggingFace Daily Papers(社区热门论文)
57
AI 编辑部评分,满分 100GQLA:面向硬件自适应的大语言模型解码的分组查询潜在注意力
本研究提出分组查询潜在注意力(GQLA),对DeepSeek-V2/V3中的多头潜在注意力(MLA)进行最小修改,使其在一套权重上暴露两种等效解码路径:与MLA相同的MQA吸收路径,以及带有每组扩展缓存的GQA路径。运行时可根据硬件自动选择路径,无需重新训练。单一的GQLA权重能同时适配H100(采用MQA吸收)与H20(采用GQA及多令牌预测)的硬件性能上限,并在GQA路径上支持高达8路的零冗余张量并行。通过扩展TransMLA为TransGQLA,可将预训练的GQA模型转换为GQLA模型。在LLaMA-3-8B上的实验表明,其MQA吸收路径将每令牌的KV缓存压缩至GQA基线的28.125%,同时在分组路径上结构性保留了GQA级别的流量效率。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org