# Skill Following 论文提出 RAE 度量，揭示 LLM Agent 检索技能的真实效果被聚合指标掩盖

- 来源：DAIR.AI (@dair_ai)
- 发布时间：2026-09-03 10:00
- AIHOT 分数：62
- AIHOT 链接：https://aihot.virxact.com/items/cmtkwpg5308boroal8dnot2sx
- 原文链接：https://x.com/dair_ai/status/2095330956823629995

## AI 摘要

DAIR.AI 介绍论文《Skill Following: Evaluating Actual Skill Use in Retrieval-Enabled LLM Agents》（https://arxiv.org/abs/2609.00549）。

## 正文

Good measurement work on whether retrieved agent skills actually help.

They report that agent skills that lift your aggregate score can be hurting every task they touch.

The usual way of checking compares tasks where a skill was retrieved against tasks where none was.

Those are different tasks, so the comparison mixes the effect of retrieval with the effect of which tasks trigger it.

The fix presented in the paper is a matched comparison. Retrieval-Invoked Actual-Use Effect runs the same task twice, once with skills enabled and once disabled, and counts only tasks where the agent actually retrieved something.

Across 17 LLMs on coding and math, models frequently show positive aggregate retrieval lift alongside a negative same-task effect.

On MBPP+, several models that look beneficial system-wide are hurting themselves on exactly the tasks where retrieval fired.

Anyone maintaining a skills directory can run this against their own stack today.

Paper: https://arxiv.org/abs/2609.00549

Chat with Paper: https://academy.dair.ai/papers/skill-following-evaluating-actual-skill-use-in-retrieval-enabled-llm-agents-2609.00549