# 理解与实现任务算术中的权重解耦

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-19 01:34
- AIHOT 链接：https://aihot.virxact.com/items/cmo9u46qk04lisls2cnsbme61
- 原文链接：https://arxiv.org/abs/2604.17078

## AI 摘要

研究人员提出"任务特征专业化"（TFS）作为任务算术（Task Arithmetic）中权重解耦的根本原理，证明TFS不仅是权重解耦的充分条件，还会导致权重向量正交性这一可观测的几何特征。基于该理论发现，团队开发OrthoReg正则化方法，在微调过程中对任务向量的权重更新矩阵强制实施正交结构，以间接促进解耦。大量实验表明，OrthoReg能持续显著提升多种任务算术方法的性能。

## 正文

Task arithmetic provides an efficient, training-free way to edit pre-trained models, yet lacks a fundamental theoretical explanation for its success. The existing concept of ``weight disentanglement" describes the ideal outcome of non-interfering task composition but does not reveal its underlying cause. Crucially, what intrinsic properties of the pre-trained model (θ_0) or the task vectors (τ_t) enable this disentanglement remains underexplored. In this paper, we introduce Task-Feature Specialization (TFS), a model's ability to allocate distinct internal features to different tasks, as the fundamental principle. We first prove that TFS is a sufficient condition for weight disentanglement. More importantly, we find that TFS also gives rise to an observable geometric consequence: weight vector orthogonality. This positions TFS as the common cause for both the desired functional outcome (disentanglement) and a measurable geometric property (orthogonality). This relationship provides the key insight for our method: since the abstract TFS property is intractable to enforce directly, we can instead promote weight disentanglement by shaping its concrete geometric consequence, orthogonality. Therefore, we propose OrthoReg, a simple and effective regularization method that actively enforces an internal orthogonal structure on weight updates (ΔW) that constitute τ_t during fine-tuning. And we theoretically prove that OrthoReg promotes disentanglement. Extensive experiments demonstrate that OrthoReg consistently and significantly enhances the performance of various task arithmetic methods. Code is available at https://github.com/RL-MIND/OrthoReg{https://github.com/RL-MIND/OrthoReg}.