# SoCRATES：面向跨领域与社会认知变化的主动式LLM调解评估基准

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-04 08:00
- AIHOT 分数：51
- AIHOT 链接：https://aihot.virxact.com/items/cmq4wvmt402g9slt2h490zprq
- 原文链接：https://arxiv.org/abs/2606.05563

## AI 摘要

SoCRATES是一个评估主动式LLM调解员在真实多领域场景中表现的基准。它通过智能体流水线从真实冲突构建8个领域场景，探测战略姿态、参与方构成、历史长度、情绪反应和文化身份5个社交认知适应轴，并使用主题局部评估器仅对推进该主题的轮次打分。该评估器与人类专家的对齐度达0.82，是每轮基线得分的两倍以上。对8个前沿LLM的测试显示，即使最强的调解员在多样化现实测试床下也仅能弥合约三分之一的未调解共识差距，且性能随社交认知轴剧烈变化，表明进步关键在于社会适应能力。

## 正文

Evaluating LLM mediators remains challenging, as mediation unfolds as a real-time trajectory shaped by disputants' shifting emotions, intentions, and context. Existing testbeds rely on a few expert-authored domains, vary mainly strategic posture, and score every turn against every topic, introducing off-topic noise. We introduce SoCRATES, a benchmark for evaluating proactive LLM mediators in realistic, multi-domain testbeds. It constructs scenarios from real conflicts through an agentic pipeline across eight domains, probes five socio-cognitive adaptation axes (strategic posture, party composition, history length, emotional reactivity, and cultural identity), and scores each topic only on the turns that advance it via a topic-localized evaluator. The evaluator reaches 0.82 alignment with human experts, more than doubling a per-turn baseline. Benchmarking eight frontier LLMs, we find that even the strongest mediator closes only about a third of the unmediated consensus gap under diverse and realistic testbeds, with performance varying sharply by socio-cognitive axis, highlighting that progress lies in social adaptation to diverse conditions.