Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/ZhenghuaBao/spoken-multihop-rag .
多跳 RAG 如何放大上游 ASR 错误:实体图谱链接与迭代改写反而加剧鲁棒性下降
AI 导读
研究测试了实体图谱链接与迭代改写两种 RAG 扩展在 ASR 输入下的表现。在 HotpotQA、2WikiMultiHopQA 和 MuSiQue 三个多跳问答基准上,两种扩展均放大错误:干净文本与最高 WER 口音间的 F1 差距比朴素稠密检索大 36-67%。查询实体损坏是主要失败模式,占 2WikiMultiHopQA 上 87-96% 的退化案例。
HuggingFace Daily Papers(社区热门论文)
66
AI 编辑部评分,满分 100多跳 RAG 如何放大上游 ASR 错误:实体图谱链接与迭代改写反而加剧鲁棒性下降
研究测试了实体图谱链接与迭代改写两种 RAG 扩展在 ASR 输入下的表现。在 HotpotQA、2WikiMultiHopQA 和 MuSiQue 三个多跳问答基准上,两种扩展均放大错误:干净文本与最高 WER 口音间的 F1 差距比朴素稠密检索大 36-67%。查询实体损坏是主要失败模式,占 2WikiMultiHopQA 上 87-96% 的退化案例。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org