# τ^τ-Bench 论文：Claude Opus 5 在 Claude Code 下构建客服智能体仅通过 23.9% 评估，专家人类参考为 82.2%

- 来源：DAIR.AI (@dair_ai)
- 发布时间：2026-09-08 05:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmtrr6wam0d3trotnu78zhl34
- 原文链接：https://x.com/dair_ai/status/2097067454883328053

## AI 摘要

Sierra 与普林斯顿大学发布 τ^τ-Bench（hyper-tau-bench）基准，把智能体构建本身设为任务：开发者智能体获得真实业务记录、掌握需求的客户、生产 API、待继承代码库和服务成本与模型限制，需交付一个可用的客服智能体，并用保留的模拟用户部署评分，共 4 个领域 53 个任务。

## 正文

Really strong benchmark paper on coding agents.

Claude Opus 5 running under Claude Code passes 23.9% of the evaluations. An expert human reference scores 82.2%.

Here is what the task actually is.

The developer agent is dropped into a client engagement. It gets the records a business really keeps, a client who holds the requirements and will answer questions, a production API that operations has to run through, an inherited codebase, and hard limits on serving cost and model choice. From that it has to deliver a working customer service agent.

The score comes from deploying the agent it built against held-out simulated users. 53 tasks across four domains.

The failures look familiar to anyone who has done this work. The models query the business records shallowly instead of reading them properly. They tell the client almost nothing. They experiment very little with agent architecture or serving spend, and ship the first design that runs.

Existing coding benchmarks score a patch. This one scores whether an agent can do the cooperative part of the job.

Paper: https://academy.dair.ai/papers/bench-an-environment-for-end-to-end-realistic-agent-construction-2609.04611