PEFT的扩展性研究:迈向由万亿参数模型支撑的百万个人模型
阅读原文· arxiv.org论文提出,参数高效微调(PEFT)不仅是全参数微调的低成本替代,更是强大共享基座模型上承载个性化行为的紧凑基底。研究围绕三个扩展维度展开:Scale Up(更强共享先验提升小适配器效用)、Scale Down(探索适配器可靠性的最小化边界)、Scale Out(大规模持久化适配实例共存)。以MinT为例,展示了管理适配器身份、版本、溯源、评估与服务驻留的基础设施。结论表明,PEFT有潜力成为持久化个人模型的紧凑底座,而不仅仅是预算替代方案。
Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.