EnterpriseClawBench:从真实工作会话构建的企业智能体基准
阅读原文· arxiv.orgEnterpriseClawBench 是一个从专有真实工作会话档案构建的企业智能体基准,产出 852 个可复现任务,配有恢复的夹具、重写的提示词、角色类、技能子类、硬规则和语义评估细则。因含企业内部内容,不公开基准数据。在 EnterpriseClawBench 上,最佳配置(Codex with GPT-5.5)仅达 0.663。结果表明企业智能体评估必须报告多种维度(如 harness-模型组合、产物交付、视觉质量、成本、运行时和技能迁移行为),而非单一分数。代码已公开。
Enterprise agents increasingly operate inside workspaces: they read heterogeneous files, invoke tools, and deliver business artifacts. We introduce EnterpriseClawBench, an enterprise agent benchmark constructed from proprietary, real-world agent sessions. Starting from a large archive of workplace sessions, the EnterpriseClawBench produces 852 reproducible tasks, each paired with recovered fixtures, rewritten prompts, role classes, skill subclasses, hard rules, and semantic rubrics. Because the sessions contain internal enterprise content, we do not release the benchmark data; instead, our reusable contribution is the construction and evaluation protocol. On EnterpriseClawBench, the best configuration reaches only 0.663 (Codex with GPT-5.5). These results show that enterprise agent evaluation must report harness--model combinations, artifact delivery, visual quality, cost, runtime, and skill-transfer behavior, rather than collapsing performance into a single score. Code: https://github.com/FrontisAI/EnterpriseClawBench