XBCP:跨语言深度研究基准测试
阅读原文· arxiv.org研究团队推出XBCP基准测试,用于评估深度研究AI智能体在证据语言与用户查询不一致时的表现。XBCP保留BrowseComp-Plus的英文问答空间,将支持文档改为跨语言(单语言证据)和多语言(12种语言均匀分布)两种设置。评估四种AI智能体使用稀疏和密集多语言检索器。结果显示,证据翻译后准确率、证据召回率和引用可靠性显著下降,且即使直接提供所有黄金证据,准确率仍然较低。这表明跨语言深度研究不仅存在检索失败,智能体在整合语言不匹配的证据时还有独立困难。
Deep research agents are increasingly evaluated on their ability to search for evidence, reason over retrieved sources, and produce grounded answers. Existing browsing benchmarks, however, largely assume that the user's query and the supporting evidence are written in the same language, leaving open whether agentic search systems can operate when relevant evidence appears in another language. We introduce XBCP (Cross-lingual BrowseComp-Plus), a controlled benchmark that preserves the English question-and-answer space of BrowseComp-Plus but varies the languages of the supporting documents. XBCP instantiates two complementary settings: in the cross-lingual setting, each query is paired with evidence in a single assigned language. In the multilingual setting, the full evidence corpus is distributed equally and randomly across 12 languages spanning high-resource and low-resource regimes. We evaluate four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval. Results reveal substantial degradation when evidence is translated. Even strong, dense retrievers lose evidence recall, and agents become less calibrated and cite evidence less reliably. Notably, accuracy remains lower even when all gold evidence is supplied directly. These findings suggest that cross-lingual deep research exposes both retrieval failures and an independent, agent-side difficulty in integrating language-mismatched evidence.