Very interesting new paper from NVIDIA.

(bookmark it)

It takes a closer look at evaluating agent skills.

Enterprise teams are starting to leverage shared skill libraries, and the review gate is typically a scanner that checks structure, style, and security.

NVIDIA measured whether that gate predicts anything.

Across 145 real skills from internal and public catalogs, structural scan scores correlate with LLM-judge quality at a Spearman rho of 0.14.

ACES proposes Skill Lift instead.

In other words, run the same task twice under the same model, sandbox, workspace, and scorer, once with the skill loaded and once without. Then you measure the difference in what the agent completed.

They scored 947 paired cases from 58 production skills across four harnesses, normalizing trajectories into a shared Agent Trajectory Interchange Format, so results compare across harnesses.

They fins that the largest process-metric gains appear in skill execution, behavior check, and skill efficiency.

Paper: https://arxiv.org/abs/2608.20614

Track more trending AI papers in our academy: https://academy.dair.ai/