HRBench:混合推理大语言模型思考模式切换策略的评测与理解
阅读原文· arxiv.org本研究提出了HRBench,一个统一的评测框架,用于研究混合推理大语言模型的思考模式切换策略。框架从三类切换策略家族(基于提示的选择、外部路由、推测执行)与四种训练方式(免训练、SFT、离线RL、在线RL)两个维度设计空间,形成多种受控评测设置。研究在涵盖从Qwen3.5-2B到Kimi-K2.5-1.1T的6个模型,以及数学、科学、代码等5个推理基准上,重新实现了12余种现有方法并进行评估。分析表明,不同策略在有效性与效率的权衡中特点各异,且策略偏好随模型规模和任务领域变化。
Hybrid-reasoning large language models (LLMs) expose explicit controls over reasoning effort, allowing users or systems to trade off answer quality against inference cost. However, existing methods for adaptive thinking-mode selection are typically evaluated under different models, datasets, and implementation assumptions, making it difficult to compare their practical behavior. We introduce HRBench, a unified evaluation framework for studying thinking-mode switching in hybrid-reasoning LLMs. HRBench organizes the design space along two axes: three switching strategy families, prompt-based selection, external routing, and speculative execution, and four training regimes, training-free, SFT, offline and online RL, yielding 12 controlled evaluation settings. We evaluate these settings across 6 LLMs, from Qwen3.5-2B to Kimi-K2.5-1.1T, and 5 reasoning benchmarks covering mathematics, science, and code, while reimplementing 12+ representative prior methods within the same pipeline. Our analysis characterizes how different switching strategies occupy distinct effectiveness-efficiency trade-off regions: prompt-based methods often provide favorable token-accuracy trade-offs, routing methods offer more stable cost reduction, and speculative methods tend to improve accuracy at higher token cost. We further find that training affects strategies differently, and that the preferred strategy varies with model scale and task domain. HRBench provides reference implementations and a unified evaluation platform to support more controlled research on efficient reasoning in hybrid-reasoning LLMs. Our data, code and repository are available at https://github.com/usail-hkust/HRBench.