# MedSP1000：标准化病人驱动的临床智能体交互式基准

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-03 08:00
- AIHOT 分数：62
- AIHOT 链接：https://aihot.virxact.com/items/cmpzjq2v8045fslkpfl4jkvgz
- 原文链接：https://arxiv.org/abs/2606.05112

## AI 摘要

MedSP1000是一个包含1,638个标准化病人（SP）案例和24,602条经同行评审的轨迹级评分标准的交互式基准，用于评估临床智能体动态决策能力。在闭环模拟中，智能体行为依据专家标准逐项打分。测试通用及医学专用大语言模型发现，静态基准表现无法可靠迁移到该场景。最佳模型GPT-5.5仅完成60.4%的专家评分项，最强医学专用模型达40.0%，增加测试时计算量未带来可测量提升。当前大语言模型尚不足以安全整合到临床实践。

## 正文

Large language models (LLMs) are increasingly proposed as clinical agents, yet static, single-turn benchmarks cannot capture how a model dynamically delivers care across an encounter: gathering information, planning treatment, and adapting longitudinal management across successive patient states. Medical education has long addressed an analogous challenge through standardized patients (SPs): trained actors who consistently portray clinical cases, enabling realistic practice and objective, scripted assessment. Here we introduce MedSP1000, an SP-derived interactive benchmark for clinical-agent evaluation, including 1,638 SP cases with 24,602 trajectory-level peer-reviewed rubrics. MedSP1000 converts peer-reviewed SP teaching cases into executable scenarios with defined SP case scripts, clinical environment contexts, and human-validated structured rubric. In each simulation evaluation run, a clinical agent interacts in closed loop with a patient agent and an environment controller, and its behaviour is scored throughout the encounter against expert criteria specified in the original materials. Applying MedSP1000 to a range of general-purpose and medically specialized LLMs, we find that performance on static benchmarks does not reliably translate to such educational scenarios. The best-performing model, GPT-5.5, completes only 60.4% of expert-defined rubric items, whereas the strongest medically specialized model reaches 40.0%; increasing test-time compute produces no measurable gain. These results suggest that current LLMs, including agentic systems tuned for medicine, are not yet reliable enough to be safely integrated into actual clinical practice. More broadly, MedSP1000 shows how process-level, SP-style evaluation can reveal clinically relevant failure modes that single-turn benchmarks miss.