GUI vs. CLI:屏幕仅限与技能中介的计算机使用智能体的执行瓶颈
阅读原文· arxiv.org该研究引入一项匹配执行层基准测试(440个桌面任务、18个应用、12个工作流类别),对屏幕仅限的GUI智能体与技能中介的CLI智能体进行控制对比,两者接受相同目标、状态和最终状态验证器,但仅限使用模态原生操作。最强GUI智能体全通过率59.1%,高于最强原始技能CLI智能体的48.2%;经验证器引导的技能增强后,CLI成功率升至69.3%,表明CLI缺陷主要来自技能覆盖不完整。结果揭示两类智能体不同的执行瓶颈:GUI受限于长时程工作流中的可靠接地交互,CLI受限于技能接口的覆盖率和可扩展性。
Computer-use agents can execute software tasks through either graphical interfaces or programmatic command interfaces, but existing evaluations confound interaction modality with differences in tasks, initial states, verifiers, and permitted actions. We introduce a matched execution-layer benchmark of 440 desktop tasks across 18 applications and 12 workflow categories, where screen-only GUI agents and skill-mediated CLI agents receive identical goals, states, and final-state verifiers while being restricted to modality-native actions. In this controlled setting, the strongest GUI agent reaches a 59.1% full pass rate, outperforming the strongest original-skill CLI agent at 48.2%; however, verifier-guided skill augmentation raises CLI success to 69.3%, showing that much of the CLI deficit comes from incomplete skill coverage rather than model capability alone. These results suggest that GUI and CLI expose different execution bottlenecks: GUI agents are limited by reliable grounded interaction over long-horizon workflows, whereas CLI agents are limited by the coverage and scalability of their skill interfaces.