# EnvACE：通过世界预演内化环境动态的智能体强化学习方法

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-06 08:00
- AIHOT 分数：56
- AIHOT 链接：https://aihot.virxact.com/items/cmsiewqd61c31ronkwvnqmzsp
- 原文链接：https://arxiv.org/abs/2608.06197

## AI 摘要

EnvACE 提出一种智能体强化学习方法，用世界预演替代训练中的外部环境交互：策略先生成工具调用，再扮演环境生成响应，并基于预演结果继续决策，两端通过任务成功奖励端到端联合优化。

## 正文

Training large language model agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments, whose construction and verification are costly, or on external simulators that are difficult to ground. We introduce EnvACE, an agentic reinforcement learning method that replaces external environment interaction during training with world rehearsal. The policy alternates between acting and rehearsal: it first generates a tool call, then plays the role of the environment to produce the response induced by that action, and conditions subsequent decisions on the rehearsed response. Both roles are jointly optimized end-to-end using task-success rewards. Through world rehearsal, the policy internalizes the relationship between actions and their environment responses in its parameters, yielding an agent world model that directly supports decision making. Across BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, EnvACE achieves strong and transferable performance, outperforming environment-scaling baselines in the overall evaluation. Controlled studies further show that world rehearsal consistently improves policy learning across model scales. At test time, the internalized world model enables private rehearsal before committed execution, yielding further gains under a moderate rehearsal budget without additional external interaction. Our findings establish world rehearsal as a new path toward scaling LLM agent training beyond the constraints of external environments. Our code is publicly available at https://github.com/Within-yao/EnvACE.