# ECHO：终端智能体免费学习世界模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-05-23 08:00
- AIHOT 分数：54
- AIHOT 链接：https://aihot.virxact.com/items/cmpmylyfn0szcsl01xhxsmjss
- 原文链接：https://arxiv.org/abs/2605.24517

## AI 摘要

ECHO（环境交叉熵混合目标）是一种针对CLI智能体的新训练方法。它在标准的GRPO策略梯度损失之上，增加了一个辅助损失项，训练模型预测自身动作产生的环境观察token。这将稀疏的结果奖励转化为每一轮rollout的密集监督。实验表明，在TerminalBench-2.0上，ECHO使Qwen3-8B的pass@1从2.70%提升至5.17%，Qwen3-14B从5.17%提升至10.79%。该方法无需专家示范，能使基础模型匹配专家SFT后GRPO的性能，并在部分场景下可能实现无验证器的自我改进。

## 正文

CLI agents are the closest thing language models have to an embodied setting: the model emits commands, the terminal executes them, and the returned stream -- stdout, errors, files, logs, and traces -- records the consequences. We argue that this stream is a supervision signal, but standard agent RL discards it: GRPO-style training updates action tokens with sparse outcome-level rewards while ignoring environment responses already in the rollout. Failed rollouts provide little policy-gradient signal despite containing rich evidence about how the environment responds. We introduce ECHO (Environment Cross-entropy Hybrid Objective), a hybrid objective that combines the standard policy-gradient loss on action tokens with an auxiliary loss that trains the policy to predict environment observation tokens resulting from its own actions. ECHO reuses the same forward pass as GRPO, requires no additional rollouts, and turns terminal feedback into dense supervision for all rollouts. ECHO doubles GRPO pass@1 on TerminalBench-2.0: Qwen3-8B improves from 2.70% to 5.17%, and Qwen3-14B from 5.17% to 10.79%. ECHO also produces policies that better predict terminal dynamics, even on trajectories they did not generate: across held-out rollouts, it sharply reduces environment-token cross-entropy while GRPO alone barely changes it. From base Qwen3-8B, ECHO matches expert-SFT-then-GRPO performance on held-out terminal tasks without expert demonstrations, and recovers roughly half of the expert-SFT initialization benefit on TerminalBench-2.0. In some settings, the environment prediction loss alone enables verifier-free self-improvement, allowing policies to improve on unseen OOD tasks by learning only from environment interactions. Together, these results suggest that environment observations are not merely context for future actions, but a dense, on-policy supervision signal already present in every rollout.