# MobilePA-Bench：面向复杂真实任务的移动端规划智能体评测基准

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-24 08:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmt89ixa72xy8ro73wqugms3o
- 原文链接：https://arxiv.org/abs/2608.23035

## AI 摘要

MobilePA-Bench 是一个交互式、有状态且以工具为中心的评测基准，用于评估移动端规划智能体的工具调用与规划能力。它运行在可执行沙箱中，覆盖 13 个功能域和 212 个真实移动工具，并额外评测子智能体协作、记忆使用和技能调用三个高级维度。实验显示，当前前沿 LLM 在严格工具排序、权限限制和意外运行时错误下性能显著下降。

## 正文

As on-device LLM agents evolve into personal copilots, the mobile operating system has become a key testbed for this paradigm, making rigorous capability evaluation essential. Yet existing benchmarks fall into two camps, each with a critical blind spot: GUI-centric benchmarks test surface-level screen manipulation while overlooking background tool use and long-horizon planning, whereas static function-calling benchmarks rely on offline API matching that is detached from real runtime constraints. To close this gap, we present MobilePA-Bench, an interactive, stateful, and tool-centric benchmark for evaluating the tool-calling and planning abilities of mobile planning agents. MobilePA-Bench runs on an executable sandbox that maintains live application databases and returns structured feedback, spanning 13 functional domains and 212 realistic mobile tools. Beyond basic tool use, it evaluates a central planning agent along three advanced dimensions: (1)Sub-agent Collaboration---decomposing a complex task and delegating specialized work to capable sub-agents; (2)Memory Usage---recalling stored memories, user profiles, and past preferences to resolve implicit requests; and (3)~Skill Usage---invoking pre-packaged composite skills instead of planning every step from scratch. Extensive experiments show that current frontier LLMs remain unreliable in mobile settings: performance drops sharply under strict tool ordering, permission limits, and unexpected runtime errors. By pairing an interactive function-calling sandbox with evidence-based verification, MobilePA-Bench serves as both a practical diagnostic benchmark and an interactive foundation for agentic reinforcement learning---accelerating the development of dependable mobile agents.