Apple 提出专用小型 seq2seq 模型用于 ASR 纠错

2026-07-06 08:00·2天前

AI 摘要

Apple 机器学习研究团队采用紧凑的 seq2seq 模型进行 ASR 纠错，训练数据来自真实和合成音频的 ASR 错误。通过级联 TTS 和 ASR 构建合成语料，关键在于匹配真实错误分布的多样性。模型采用 correction-first 解码，生成候选后利用 ASR 声学分数重新排序。与 LLM 相比，该模型参数少 15 倍，在 LibriSpeech test-clean/other 上分别达到 1.5% 和 3.3% 的词错误率（WER），优于 LLM，并能泛化至 CTC、Seq2seq、Transducer 等多种 ASR 架构，在低错误率场景中提供精确纠错。

原文 · 未翻译

research area Methods and Algorithms, research area Speech and Natural Language Processing

content type paperpublished July 2026

Revisiting ASR Error Correction with Specialized Models

AuthorsZijin Gu, Tatiana Likhomanenko, Richard He Bai, Erik McDermott, Ronan Collobert, Navdeep Jaitly†**

View publication

Language models play a central role in automatic speech recognition (ASR), yet most methods rely on text-only models unaware of ASR error patterns. Recently, large language models (LLMs) have been applied to ASR correction, but introduce latency and hallucination concerns. We revisit ASR error correction with compact seq2seq models, trained on ASR errors from real and synthetic audio. To scale training, we construct synthetic corpora via cascaded TTS and ASR, finding that matching the diversity of realistic error distributions is key. We propose correction-first decoding, where the correction model generates candidates rescored using ASR acoustic scores. With 15x fewer parameters than LLMs, our model achieves 1.5/3.3% WER on LibriSpeech test-clean/other, outperforms LLMs, generalizes across ASR architectures (CTC, Seq2seq, Transducer) and diverse domains, and provides precise corrections in the low-error regime where LLMs struggle.

† Google
** Work done while at Apple

Related readings and updates.

Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition

January 18, 2025research area Speech and Natural Language Processingconference ICASSP

This paper presents an efficient decoding approach for end-to-end automatic speech recognition (E2E-ASR) with large language models (LLMs). Although shallow fusion is the most common approach to incorporate language models into E2E-ASR decoding, we face two practical problems with LLMs. (1) LLM inference is computationally costly. (2) There may be a vocabulary mismatch between the ASR model and the LLM. To resolve this mismatch, we need to…

Retrieval-Augmented Correction of Named Entity Speech Recognition Errors

September 11, 2024research area Knowledge Bases and Search, research area Speech and Natural Language Processing

In recent years, end-to-end automatic speech recognition (ASR) systems have proven themselves remarkably accurate and performant, but these systems still have a significant error rate for entity names which appear infrequently in their training data. In parallel to the rise of end-to-end ASR systems, large language models (LLMs) have proven to be a versatile tool for various natural language processing (NLP) tasks. In NLP tasks where a database…