ChLogic:中文逻辑推理鲁棒性评估基准
阅读原文· arxiv.org大语言模型在逻辑推理基准上表现良好,但中文环境下的鲁棒性未知。研究团队提出中英文对齐基准ChLogic,包含通用对齐集(60条命题)、困难对齐集(40道难题)及纯中文集(15类语言现象),每个对齐项含一条英文参考和五种中文实现。在Qwen3、Ministral和GLM上的实验显示中英文性能持续存在差距。将中文回译成英文可提升通用集表现,但在困难集上效果混杂,Qwen3-32B和GLM-5.1翻译后性能反而下降。这表明中文实现、翻译伪影和模型特定行为共同影响多语言逻辑推理。
Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.