MineExplorer:评估多模态大语言模型智能体在 Minecraft 中的开放世界探索能力
阅读原文· arxiv.org研究提出了 MineExplorer 基准测试,用于评估多模态大语言模型智能体在 Minecraft 开放世界中的探索能力。该基准采用 ReAct 式能力表述,将原子任务组合为隐式多跳任务,并利用多智能体合成工作流共同设计任务图、沙盒场景和基于规则的里程碑评估器。实验表明,开放世界探索仍具挑战性,强模型能处理许多单跳任务,但在需要协调更长轨迹中隐藏先决条件时性能急剧下降。代码与数据集已开源。
Multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and action generation. However, their ability to sustain exploration in dynamic open worlds remains unclear. Existing embodied and game-based benchmarks often compress interaction into short-horizon tasks or entangle success with domain-specific game mechanics. In this paper, we introduce MineExplorer benchmark for evaluating open-world exploration capabilities of MLLM agents in Minecraft. We first filter atomic tasks whose solutions rely heavily on Minecraft-specific knowledge to better reflect general open-world reasoning. Then we organize the benchmark around a ReAct-style capability formulation and compose atomic tasks into implicit multi-hop tasks. To further construct reliable instances, MineExplorer uses a multi-agent synthesis workflow that jointly designs task graphs, sandbox scenes, and rule-based milestone evaluators. Human evaluation shows that the multi-agent synthesis workflow produces significantly more reliable instances than a single-agent baseline. Experiments with advanced MLLM agents show that open-world exploration remains challenging, as strong models can handle many single-hop tasks but degrade sharply when hidden prerequisites must be coordinated over longer trajectories. Further analysis finds that task difficulty tracks agent completion, and larger models or thinking modes do not consistently translate into better performance. Code and dataset are available at https://github.com/Jometeorie/MineExplorer.