# 仅一层就够？训练单个Transformer层可媲美全参数RL训练

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-07-03 12:36
- AIHOT 分数：61
- AIHOT 链接：https://aihot.virxact.com/items/cmr4g28d201iqsll565znjq3o
- 原文链接：https://x.com/rohanpaul_ai/status/2072902174712156458

## AI 摘要

一项新研究发现，RL后训练的大部分收益可能仅来自一个Transformer层。论文冻结几乎所有层，每次只训练一层，检查单层能恢复多少完整RL提升（“层贡献”）。在7个模型、3种RL方法和数学、编码、智能体任务中，最佳层通常位于网络中间，早期和后期层贡献很小。实际结果是，仅训练中间层就能超越完整RL训练，例如Qwen3-8B数学准确率达69.1对66.4。

## 正文

What if most RL gains come from 1 transformer layer?

They test a strange idea and find it often works: train 1 layer, freeze the rest.

This paper finds that reward training is less like whole-model learning and more like layer selection.

The problem is that RL post-training usually updates the whole model, so it hides which parts actually learn the useful behavior.

The paper freezes almost every layer, trains only 1 transformer layer at a time, then checks how much of full RL improvement that single layer recovers.

This score, called layer contribution, reveals that the useful RL changes are not spread evenly through the network.

Across 7 models, 3 RL methods, and math, code, and agent tasks, the best layers usually sit near the middle, while early and late layers help much less.

The practical result is that training only the best middle layers can beat full RL, such as 69.1 math accuracy versus 66.4 on Qwen3-8B.

----

Link – arxiv. org/abs/2607.01232v1

Title: "Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training"