# RLM论文揭示：Harness可让模型"作弊"式泛化

- 来源：swyx (@swyx)
- 发布时间：2026-07-21 11:43
- AIHOT 分数：41
- AIHOT 链接：https://aihot.virxact.com/items/cmru4b3b255rrbihz5koquf16
- 原文链接：https://x.com/swyx/status/2079411861150429402

## AI 摘要

RLM论文指出，前沿模型训练可通过训练“测试集仿冒数据”来目标化任何benchmark分数，而开源模型因不公开数据集/RL环境而难以被验证。Alex和Omar提出用标准NLP距离指标检测隐藏轨迹。研究发现，设计良好的harness能让RLM在仅训练短任务后，完全泛化到8-32倍长的相似任务，甚至跨领域（如作文归属与数学解题）实现泛化。

## 正文

very notable trajectory comparison writeup here buried in the RLM paper from @a1zhang and @lateinteraction.

an open secret of "frontier" model training is that even without training on test， you can basically cheat by training on test lookalikes， enabling you to goalseek almost any benchmark number you want.

however when they are released open weights， 99% of the time the norm is that you do not get the datasets/rlenvs that would easily show you if someone was training on Temu Tbench， so there is plausible deniability. Alex and Omar discuss applying standard NLP distance metrics on hidden trajectories. There's no ultimate solution here， but they have some prelim explorations. It happens to support the finding that RLMs can generalize to unseen tasks that share latent structure observed in training.

### 引用推文

> alex zhang：Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize th...