# TRON：面向视觉推理强化学习的可控在线环境框架

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-01 10:52
- AIHOT 分数：64
- AIHOT 链接：https://aihot.virxact.com/items/cmpxn3gho05iaslckuyaqswik
- 原文链接：https://arxiv.org/abs/2606.01599

## AI 摘要

TRON 是一个面向视觉推理强化学习（RL）的在线环境框架。它通过可控的生成器-验证器程序，按需生成全新的视觉状态、图像和问答实例。当前 TRON 套件包含 520 个环境，按能力分为五个类别。该框架支持单一全模型训练和按桶训练专家模型，无需额外数据采集，并提供了生成可靠性、多样性等分析。基于 TRON 进行 RL 后训练，能持续提升 Qwen3-VL-4B、Qwen2.5-VL-7B 与 MiMo-VL-7B-SFT 在多个外部多模态推理基准上的性能。

## 正文

Reinforcement learning (RL) for visual reasoning needs scalable, verifiable, and controllable training signals. Existing visual RL post-training trains on static curated datasets, with fixed image-question-answer samples bounded by their collection budget. In this work, we introduce TRON (Targeted, Rule-verifiable Online eNvironments), an online environment substrate: a training rollout is generated on demand by a controllable generator-verifier program that samples a fresh latent visual state, renders an image, asks a question, and exactly verifies the answer. A single run can therefore draw an unbounded stream of fresh instances at the difficulty level required by the current curriculum. The current TRON suite contains 520 environments organized into five ability buckets (spatial, mathematical, diagram, pattern/logic, and counting); the same substrate supports both a single full model trained on all buckets and per-bucket ability-specialist models, with no additional data collection. We also introduce a substrate analysis covering generation reliability, instance and level diversity, cross-environment near-duplicates, and base-model pass rate by difficulty level. RL post-training with METHOD consistently improves performance on ten external multimodal reasoning benchmarks across Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.