# SWE-Interact：重新构想面向用户驱动的多轮编码会话的SWE基准测试

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-29 08:00
- AIHOT 分数：60
- AIHOT 链接：https://aihot.virxact.com/items/cmr2es3vy07easl8z8lcwe5iy
- 原文链接：https://arxiv.org/abs/2606.30573

## AI 摘要

SWE-Interact是一个面向编码智能体的新测试平台，评估其在多轮、交互式、用户驱动的软件工程任务中的表现。与一次性给出完整需求的传统SWE基准不同，它通过精心设计的用户模拟器，从模糊指令开始逐步揭示需求并提供反馈。在系列前沿和开源模型测试中，单轮任务表现优异的模型在多轮交互任务上的成功率从约50%降至约25%。最强模型虽能应对初始模糊指令，但仍存在过度编码、遗忘需求等技术错误；较弱模型则早早放弃或忽略要求。该测试衡量了模型交互式目标发现和迭代精炼的真实能力。

## 正文

We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks. Existing frontier SWE benchmarks typically provide complete requirements upfront and evaluate agents on autonomous implementation. In contrast, SWE-Interact places agents in a realistic developer workflow: a carefully designed user simulator starts with vague or incomplete instructions, progressively reveals requirements, inspects the agent's workspace, and provides targeted feedback, revisions, and new constraints until the full task goal has been handed off. Grounded in large-scale studies of real coding-agent interactions, this setup tests whether agents can discover user intent, adapt to evolving requirements, and build on their own prior work. Across a suite of frontier and open-weight models, we find that strong performance on single-turn SWE tasks does not reliably transfer to multi-turn, user-driven workflows: the best-performing models solve roughly 50% of single-turn baseline tasks but only 25% of the corresponding SWE-Interact tasks. The strongest models in our evaluation, including Opus 4.8 and GPT 5.5, start strong even in the face of vague initial instructions, persevere until all the requirements are surfaced by the user, integrate them better and write clean code. However, they still suffer from over-agentic coding, forgetting requirements and technical mistakes. Weaker models start poorly under ambiguity, give up early, forget or ignore instructions and rework their code more. Overall, SWE-Interact measures an orthogonal, real-world capability axis for frontier model development: interactive goal discovery and iterative refinement with a user in the loop.