PRL-Bench:评估 LLM 前沿物理研究能力的综合基准测试
阅读原文· arxiv.org研究团队发布 PRL-Bench 基准测试,用于系统评估 LLM 执行端到端物理研究的能力边界。该基准基于 2025 年 8 月以来《物理评论快报》100 篇精选论文构建,涵盖天体物理、凝聚态物理、高能物理、量子信息和统计物理五大领域,任务设计模拟真实科研的探索性、长周期工作流和客观可验证性。评估显示前沿模型最佳总体得分不足 50 分,揭示当前 LLM 能力与自主科学发现需求间仍存在显著差距。
The paradigm of agentic science requires AI systems to conduct robust reasoning and engage in long-horizon, autonomous exploration. However, current scientific benchmarks remain confined to domain knowledge comprehension and complex reasoning, failing to evaluate the exploratory nature and procedural complexity of real-world research. In this work, we present research-oriented evaluations in theoretical and computational physics, a natural testbed with comprehensive domain knowledge, complex reasoning, and verifiable end-to-end workflows without reliance on experiments. Here we introduce PRL-Bench (Physics Research by LLMs), a benchmark designed to systematically map the capability boundaries of LLMs in executing end-to-end physics research. Constructed from 100 curated papers from the latest issues of Physical Review Letters since August 2025 and validated by domain experts, PRL-Bench covers five major theory- and computation-intensive subfields of modern physics: astrophysics, condensed matter physics, high-energy physics, quantum information, and statistical physics. Each task in the benchmark is designed to replicate the core properties of authentic scientific research, including exploration-oriented formulation, long-horizon workflows, and objective verifiability, thereby reconstructing the essential reasoning processes and research workflows of real physics research. Evaluation across frontier models shows that performance remains limited, with the best overall score below 50, revealing a pronounced gap between current LLM capabilities and the demands of real scientific research. PRL-Bench serves a reliable testbed for accessing next generation AI scientists advancing AI systems toward autonomous scientific discovery.