# 基于神经元激活图的目标导向预训练数据选择

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-17 08:00
- AIHOT 链接：https://aihot.virxact.com/items/cmo9h6xfu02h7sls29xvza673
- 原文链接：https://arxiv.org/abs/2604.15706

## AI 摘要

研究人员提出神经元激活图排序（NAG-based Ranking）框架，用于目标导向的语言模型预训练数据选择。该方法无需训练且可解释，通过量化神经元影响构建跨层NAG，并依据NAG相似度排序候选数据。在六个基准测试中平均比随机采样提升4.9%，在HellaSwag上比SOTA基线提升5.3%。多目标场景下分别超过两个基线1.1%和4.1%。分析表明，仅停用0.12%的NAG选择神经元就会导致23.5%性能崩溃，证明NAG捕捉了学习目标特征的稀疏"功能骨干"。

## 正文

Everyday tasks come with a target, and pretraining models around this target is what turns them into experts. In this paper, we study target-oriented language model (LM) pretraining by introducing Neuron-Activated Graph Ranking (NAG-based Ranking), a training-free and interpretable framework for target pretraining data selection. Rather than using black-box representations, our approach directly characterizes each target input by a sparse set of high-impact neurons in any off-the-shelf LLMs. Concretely, we quantify neuron impact and select the most influential neurons across layers into a compact Neuron-Activated Graph (NAG), and rank candidate data by NAG similarity to target examples. We conduct experiments across six benchmarks, where our NAG-based Ranking improves target-oriented pretraining by 4.9% on average over random sampling, and also outperforms state-of-the-art baselines by 5.3% accuracy on HellaSwag. It also remains effective under a more applicable multi-target setting, where our best setup surpasses two baselines by 1.1% and 4.1%, respectively. Furthermore, we provide a comprehensive analysis on why and how our NAG works, e.g., deactivating NAG-selected neurons (only 0.12% of all) causes a 23.5% performance collapse, and restricting NAG to the final layer incurs a 4.1% average drop, indicating that NAG captures a sparse "functional backbone" for learning target features. We release the code at https://github.com/asillycat/NAG.