"我没做微决策":在协作中衡量、引导与揭示目标层面的AI贡献
阅读原文· arxiv.org本研究提出了CoTrace目标级归因框架,用于分解协作目标并追踪AI的贡献。对638份真实对话的分析发现,大语言模型在目标塑造中的直接贡献为11%-26%,但在引入具体实践需求方面作用显著,并存在多种间接影响。控制实验表明交互设计会影响AI的目标行为。用户研究显示,向用户展示目标级分析后,其对AI贡献的感知评分在5分制中变化了近2分,揭示了用户对自身AI协作成果存在系统性的校准偏差。
As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.