# NVIDIA 新论文提出 ACES 技能评估法

- 来源：elvis (@omarsar0)
- 发布时间：2026-08-24 20:47
- AIHOT 分数：39
- AIHOT 链接：https://aihot.virxact.com/items/cmt78f2k21ygsro73eor2m7xl
- 原文链接：https://x.com/omarsar0/status/2091869893339812222

## AI 摘要

NVIDIA 新论文提出 ACES 方法评估智能体技能：在同一模型、沙箱、工作区和评分器下，分别在有/无技能加载时运行同一任务，对比智能体完成度的差异。基于 58 个生产技能、947 组配对案例、4 个 harness 的测试显示，技能执行、行为检查和技能效率的过程指标提升最大。传统结构扫描评分与 LLM 评判质量的相关性仅 Spearman rho 0.14。

## 正文

Very interesting new paper from NVIDIA.

(bookmark it)

It takes a closer look at evaluating agent skills.

Enterprise teams are starting to leverage shared skill libraries, and the review gate is typically a scanner that checks structure, style, and security.

NVIDIA measured whether that gate predicts anything.

Across 145 real skills from internal and public catalogs, structural scan scores correlate with LLM-judge quality at a Spearman rho of 0.14.

ACES proposes Skill Lift instead.

In other words, run the same task twice under the same model, sandbox, workspace, and scorer, once with the skill loaded and once without. Then you measure the difference in what the agent completed.

They scored 947 paired cases from 58 production skills across four harnesses, normalizing trajectories into a shared Agent Trajectory Interchange Format, so results compare across harnesses.

They fins that the largest process-metric gains appear in skill execution, behavior check, and skill efficiency.

Paper: https://arxiv.org/abs/2608.20614

Track more trending AI papers in our academy: https://academy.dair.ai/