Rohan Paul@rohanpaul_ai
42AI 编辑部评分,满分 100

提示词如何让编程智能体浪费算力:Same Task, Different Work 论文揭示

2026-08-15 19:01· 41分钟前
AI 导读

论文发现,提示词要求编程智能体考虑多种方案时,其推理量增加2.4–7.4倍,但成功率无提升;智能体平均展开约三个方案后弃用,仅实现其中一个。“Be absolutely certain”指令则触发冗余验证,最高使运行成本达基线18.25倍、工具调用15次对6次、耗时3倍,成功率不变。

Really useful revelation on prompting coding agent in this paper.

Your coding-agent prompt may be spending compute on work you never needed.

A coding agent can solve the same bug correctly while doing far more work, just because the prompt asks it to consider several approaches.

Across six open-weight reasoning models on the frozen holdout, that instruction increased reasoning by 2.4-7.4× with no measured success gain.

The traces explain why: agents elaborated roughly three alternatives, discarded them, then implemented exactly one.

"Be absolutely certain" created a different failure mode, triggering repeated tests and extra checking after the fix looked done.

At the highest observed redundant-verification level, runs cost 18.25× the clean-run median, used 15 versus 6 tool calls, and took 3× longer, while success stayed flat.

Prompt length barely mattered. Verbose repetition stayed around 1.0×.

A bounded instruction, start with the failing test, make the smallest sufficient change, run relevant tests, then stop, stayed at or below baseline reasoning while preserving diagnosis and validation.

The tasks were small, at most four files, so this may not carry over to large architectural work.

For coding agents, prompt design is work design: specify the useful work and give the agent a concrete stopping rule.

  • arxiv. org/abs/2608.01347

Title: "Same Task, Different Work: Prompt-Induced Waste in Coding Agents"

来源:Rohan Paul · x.com

提示词如何让编程智能体浪费算力:Same Task, Different Work 论文揭示

Rohan Paul · @rohanpaul_ai · X·2026-08-15 19:01·41分钟前
AI 导读

论文发现,提示词要求编程智能体考虑多种方案时,其推理量增加2.4–7.4倍,但成功率无提升;智能体平均展开约三个方案后弃用,仅实现其中一个。“Be absolutely certain”指令则触发冗余验证,最高使运行成本达基线18.25倍、工具调用15次对6次、耗时3倍,成功率不变。

Really useful revelation on prompting coding agent in this paper.

Your coding-agent prompt may be spending compute on work you never needed.

A coding agent can solve the same bug correctly while doing far more work, just because the prompt asks it to consider several approaches.

Across six open-weight reasoning models on the frozen holdout, that instruction increased reasoning by 2.4-7.4× with no measured success gain.

The traces explain why: agents elaborated roughly three alternatives, discarded them, then implemented exactly one.

"Be absolutely certain" created a different failure mode, triggering repeated tests and extra checking after the fix looked done.

At the highest observed redundant-verification level, runs cost 18.25× the clean-run median, used 15 versus 6 tool calls, and took 3× longer, while success stayed flat.

Prompt length barely mattered. Verbose repetition stayed around 1.0×.

A bounded instruction, start with the failing test, make the smallest sufficient change, run relevant tests, then stop, stayed at or below baseline reasoning while preserving diagnosis and validation.

The tasks were small, at most four files, so this may not carry over to large architectural work.

For coding agents, prompt design is work design: specify the useful work and give the agent a concrete stopping rule.

  • arxiv. org/abs/2608.01347

Title: "Same Task, Different Work: Prompt-Induced Waste in Coding Agents"

来源:Rohan Paul· x.com