# 微软发布Thinkingbox：智能体可靠性基准

- 来源：DAIR.AI (@dair_ai)
- 发布时间：2026-08-23 01:03
- AIHOT 分数：47
- AIHOT 链接：https://aihot.virxact.com/items/cmt4o47ft1q6wro6tmve0fwkf
- 原文链接：https://x.com/dair_ai/status/2091209556122808532

## AI 摘要

微软发布Thinkingbox，一个用于真实业务流程中智能体可靠性的沙盒环境，配备MCP兼容工具会话，以及涵盖零售、酒店、汽车保险等领域的507个策略条件工作流基准。每项尝试根据智能体留下的后端状态评分，最强模型pass@1达65.36%，pass^20为25.25%。许多失败尝试以有效的状态变更工具调用干净终止，仅观察响应或工具调用难以判断任务是否真正完成。

## 正文

Banger paper from Microsoft.

It's on agent reliability in real business workflows.

(bookmark it)

Thinkingbox is a sandbox with isolated MCP-compatible tool sessions, plus a benchmark of 507 policy-conditioned workflows across retail, hospitality, auto insurance, neobank IT, and consulting support.

Every attempt is graded on the backend state the agent leaves behind. Executable checks accept valid trajectories and reject wrong, missing, or extra effects, so collateral damage counts against you.

The strongest model reaches 65.36% pass@1 and 25.25% pass^20.

Many failed trials terminate cleanly with valid state-changing tool calls. Watching the response or the tool call tells you very little about whether the task actually completed.

Paper: https://arxiv.org/abs/2608.19741

Track more trending AI papers in our academy: https://academy.dair.ai/