AtomiMed:层次化原子事实检查实现通用临床感知的医学报告评估
阅读原文· arxiv.org现有医学报告生成评估指标依赖表层n-gram重叠,无法捕捉临床事实准确性且易忽略灾难性诊断错误。AtomiMed是一种通用、跨模态框架,将医学叙述分解为标准化多层次原子临床事实(疾病级实体与位置、形态、严重程度等属性级描述),并通过在地面真实与预测报告间执行智能体交叉验证循环模拟多放射科医生同行评审,实现诊断检测与描述准确性的解耦评估。配套开源工具包MRGEvalKit与多模态基准OmniMRG-Bench(覆盖X光、CT、MRI、超声)。实验表明,AtomiMed与人类判断相关性显著高于传统及基于模型的指标。代码已开源。
Traditional metrics for Medical Report Generation (MRG) predominantly rely on surface-level n-gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing AtomiMed, a universal, modality-agnostic evaluation framework that decomposes complex medical narratives into a standardized, multi-level hierarchy of Atomic Clinical Facts, encompassing Disease-level entities and Attribute-level descriptors, including location, morphology, and severity. By implementing an Agentic Cross-Verification loop between ground-truth and predicted reports, AtomiMed simulates a multi-radiologist peer-review process to verify clinical consistency, thus enabling the decoupled assessment of diagnostic detection and descriptive accuracy. To facilitate standardized evaluation, we introduce MRGEvalKit, an open-source toolkit for automated hierarchical extraction, and curate OmniMRG-Bench, a comprehensive multi-modal benchmark covering X-ray, CT, MRI, and Ultrasound. Extensive experiments on multiple expert-annotated reader studies demonstrate that AtomiMed achieves significantly higher correlation with human radiologist judgment compared to traditional and model-based metrics. Our code are release at https://github.com/Venn2336/MRGEvalkit