# 亚马逊论文：按KV缓存策略微调可防长上下文失效

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-31 20:24
- AIHOT 分数：45
- AIHOT 链接：https://aihot.virxact.com/items/cmth8i1sh06wsrodm3g8pgahq
- 原文链接：https://x.com/rohanpaul_ai/status/2094400884109410340

## AI 摘要

亚马逊新论文表明，KV缓存策略不仅是推理优化，还能改变模型所需的训练方式。若模型推理时会遗忘部分上下文，训练时也应模拟这种遗忘——将微调与KV缓存策略匹配，可防止长上下文失效。在128k token测试中，全注意力微调模型在稀疏推理下常产生冗长无意义回答，而匹配缓存策略微调的模型能正常作答并停止；该方法支持任意缓存策略，可在40 GB A100上计算4B模型梯度。

## 正文

New Amazon paper shows KV-cache policy is not just an inference optimization; but it can change the training regime the model needs.

If your LLM will forget parts of its context at inference, train it to forget that way too: this paper shows matching fine-tuning to the KV-cache policy can prevent long-context failures.

Sparse attention lets long-context inference use a fixed-size KV cache by keeping only part of the model’s past context. But models are often fine-tuned with full attention, then asked to work with missing memory at inference.

That mismatch can break behavior. In 128k-token tests, models trained with full attention often produced long, nonsensical answers under sparse inference, while models fine-tuned with the same cache policy learned to answer and stop normally.

The method makes this policy-matched training practical for arbitrary cache policies, including computing gradients for a 4B model on a 40 GB A100.

– arxiv. org/abs/2608.19920

Title: "Learning how to Forget: Fine-tuning for Long-Context Sparse Attention"