# PEFT-Arena：从稳定性-可塑性视角理解参数高效微调

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-05-27 08:00
- AIHOT 分数：65
- AIHOT 链接：https://aihot.virxact.com/items/cmpp1ycod0b4zslv44a7bssz0
- 原文链接：https://arxiv.org/abs/2605.28819

## AI 摘要

PEFT-Arena是一个新的参数高效微调评估基准，它同时衡量下游任务性能和大语言模型预训练通用能力的保留情况。研究发现，不同微调方法展现出不同的稳定性-可塑性特征；在相似参数预算下，正交微调取得了最佳的性能-保留权衡帕累托前沿。通过权重空间（谱分析）和激活空间（表示失真度量）两个几何视角的分析，研究解释了这些差异，并指出最终的SFT检查点常常会越过一个更优的目标-保留操作点。基于此，研究通过路径回溯案例展示了一种改进方法。

## 正文

Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstream accuracy while overlooking the retention of pretrained capabilities. We argue that PEFT should be assessed through the stability-plasticity dilemma: the trade-off between target-task adaptation and resistance to forgetting. We introduce PEFT-Arena, a benchmark that jointly measures downstream performance and general capability retention. Across methods, we find distinct stability-plasticity profiles; under comparable parameter budgets, orthogonal finetuning achieves the most favorable Pareto frontier. To explain these differences, we analyze PEFT updates from two geometric perspectives. In weight space, spectral analysis reveals how parameterizations interact with the pretrained singular-value structure. In activation space, retention metrics show whether finetuning preserves or distorts general-capability representations, with forgetting linked to non-isometric representation distortion. Finally, an analysis shows that final SFT checkpoints often overshoot a better target-retention operating point. Inspired by this, we present case studies of a post-hoc improvement with path-wise rewinding.