多阶段适配器感知定向遗忘
阅读原文· arxiv.org当前机器遗忘评估存在结构性偏差,测试“为什么”类因果知识的问题在主流基准中占比极低(不足1.3%)。为此,研究提出5WBENCH基准,其包含5,000个样本,均匀覆盖“5W”类别,使因果遗忘失败首次变得可量化。分析表明,现有方法无法在“为什么”类问题上同时实现高遗忘与高保留,因为这类问题涉及多跳推理链(44%)和超过40个token的答案跨度。研究提出MAAT框架,该方法在LoRA适配器权重上进行三阶段操作,结合梯度投影上升、SVD秩维剪枝、任务向量取反和混合KL-隐藏状态保留修复,是首个在因果知识上实现高遗忘与高保留平衡的方法。
Machine unlearning evaluation is structurally skewed: Why-type questions, which probe causal and relational knowledge, comprise less than 0.06% of CounterFact, 0.6% of ZSRE, and less than 1.3% of TOFU, MUSE, and WMDP-Cyber. This near-zero representation means that methods that fail on causal knowledge can score highly in aggregate, and this failure is undetectable without balanced evaluation. We present 5WBENCH, a balanced 5,000-sample benchmark with 1,000 examples per 5W category (Who, What, When, Where, Why), making causal unlearning failures quantifiable for the first time. Using 5WBENCH, we show that no existing baseline simultaneously achieves high forgetting and high retention on Why-type questions: aggressive forgetting degrades retained knowledge, while conservative methods fail to forget causal facts. Why-type difficulty stems from multi-hop reasoning chains (44% of Why entries vs. less than or equal to 2% for others) and gradient dilution over 40.1-token answer spans. We present MAAT (Multi-phase Adapter-Aware Targeted Unlearning), a three-phase framework operating on LoRA adapter weights, combining gradient-projected ascent, SVD rank-dimension pruning, task vector negation, and hybrid KL-hidden-state retain repair. MAAT is the first method to simultaneously achieve high forgetting and high retention on Why-type causal knowledge, reaching a new operating point on the forget-retain Pareto frontier. We make our code publicly available.