# Artificial Analysis 发布 Harvey LAB-AA 法律智能体基准

- 来源：Artificial Analysis (@ArtificialAnlys)
- 发布时间：2026-07-08 01:12
- AIHOT 分数：61
- AIHOT 链接：https://aihot.virxact.com/items/cmrax1uf001u0ihog32p5v2rh
- 原文链接：https://x.com/ArtificialAnlys/status/2074541975186165887

## AI 摘要

Artificial Analysis 推出 Harvey LAB-AA，基于 Harvey 构建的 120 项任务（覆盖 24 个法律实践领域），以全部通过率评估模型质量。Claude Fable 5（max）以 14.2% 领先，Claude Opus 4.8（max）与 GLM-5.2（max）并列 7.5%，MiniMax-M3 6.7%，Claude Sonnet 5 5.0%，GPT-5.5（xhigh）和 Claude Sonnet 4.6（max）均为 4.2%。成本上，Fable 5 约 $19/任务，Gemini 3.1 Flash-Lite 约 $0.02/任务，跨度 950 倍。28 个模型中 13 个全通过率为 0，顶尖法律智能体仍有很大提升空间。

## 正文

After our announcement last month, Artificial Analysis is now launching Harvey LAB-AA (Legal Agent Benchmark), our implementation of Harvey's new agentic legal benchmark that evaluates language models on real-world legal work across 24 practice areas

Harvey LAB-AA tests models on a private set of 120 legal tasks built by the team at @harvey. The tasks span 24 practice areas from corporate M&A and capital markets to tax, litigation, and bankruptcy. Models work to create the legal outputs specified in the tasks, and each task is graded against a rubric of binary criteria. The primary metric we present is the all-pass rate: the share of tasks where all criteria in the rubric are satisfied, reflecting the high standard of real-world professional legal deliverables.

Claude Fable 5 (max, with fallback) from @AnthropicAI leads Harvey LAB-AA with a 14.2% all-pass rate, after falling back to Opus 4.8 in only 1 task. This is almost double the scores of the next best models Claude Opus 4.8 (max) and GLM-5.2 (max) from @Zai_org, which tie at 7.5%.

Key takeaways from Harvey LAB-AA:

➤ Frontier legal work is far from solved: At launch, most models pass a majority of individual criteria but very few fully satisfy the requirements of any given task. The best model, Claude Fable 5, fully satisfies rubrics on just 14.2% of tasks, leaving ~86% of professional legal deliverables incomplete. Claude Opus 4.8 (max) and GLM-5.2 (max) follow at 7.5%, MiniMax-M3 at 6.7%, and Claude Sonnet 5 at 5.0%, ahead of GPT-5.5 (xhigh) from @OpenAI and Claude Sonnet 4.6 (max), which both score 4.2%.

➤ Models can pass many requirements of legal tasks, but rarely all of them: the leading models pass >90% of individual rubric criteria, but 13 of the 28 evaluated models fully pass 0 tasks.

➤ The top open weights model scores just over half the frontier leader: GLM-5.2 (max) ties Claude Opus 4.8 for second with a 7.5% all-pass rate and criteria pass of 91.0% vs. 91.1% respectively, both now behind Claude Fable 5 (14.2%). GLM-5.2 reaches that at ~6% of Fable 5's cost per task (~$1 vs. ~$19).

➤ Cost per task spans ~950x: the most expensive model, Claude Fable 5, costs ~$19 per task, while Gemini 3.1 Flash-Lite passes 31.1% of criteria for ~$0.02 per task.