# RDP LoRA：大语言模型参数高效适应的几何驱动识别方法

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-21 08:00
- AIHOT 链接：https://aihot.virxact.com/items/cmoa6zlx700h0sl1yfpi99ks7
- 原文链接：https://arxiv.org/abs/2604.19321

## AI 摘要

研究团队提出RDP LoRA方法，将大语言模型隐藏状态演化建模为高维几何轨迹，利用Ramer-Douglas-Peucker算法无训练地识别表征路径关键断点，并直接作为层选择信号。在Qwen3-8B-Base的LoRA微调实验中，仅需适配13个RDP选择的层，便在MMLU-Math上达到81.67%准确率，显著优于全层适配的79.32%和随机选择的75.56%，证明几何轨迹分析可优化参数高效微调中的层选择决策。

## 正文

Fine-tuning Large Language Models (LLMs) remains structurally uncertain despite parameter-efficient methods such as Low-Rank Adaptation (LoRA), as the layer-specific roles of internal representations are poorly understood, leading to heuristic decisions about where adaptation should be applied. We model the evolution of hidden states as a high-dimensional geometric trajectory and propose using the Ramer-Douglas-Peucker (RDP) algorithm, a parameter-free and training-free polygon simplification method that preserves global structural transitions while eliminating locally redundant changes, to identify critical breakpoints along the representation path. Crucially, we use these geometric pivots not merely for analysis, but as a direct decision signal for determining which layers should be adapted during parameter-efficient fine-tuning. By integrating this geometry-aware layer selection strategy into LoRA fine-tuning of Qwen3-8B-Base, we achieve superior performance on MMLU-Math using only 13 RDP-selected layers (81.67%), significantly outperforming both full 36-layer adaptation (79.32%) and random 13-layer selection (75.56%), as well as the baseline Qwen3-8B-Base model (74.25%). These results demonstrate that leveraging the intrinsic geometry of representation trajectories provides a robust, interpretable, and training-free signal for optimizing layer selection during model adaptation.