# LEGO-RL：面向编码智能体的原生 Harness 强化学习框架

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-18 08:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmt0cf1z718terodptlfxqeb7
- 原文链接：https://arxiv.org/abs/2608.17393

## AI 摘要

LEGO-RL 提出一种在不修改内部控制流的前提下，将原生编码智能体 harness 与可扩展策略梯度优化相结合的框架，解决环境崩溃与奖励黑客污染结果信号、训练-推理不一致等问题。

## 正文

Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.