音频智能涉及对音频和语音的理解、推理与生成。在这项工作中,我们推出了 Nemotron-Labs-Audex-30B-A3B(Audex),这是一个基于 Nemotron-Cascade-2-30B-A3B(一个强大的纯文本 MoE 大语言模型)构建的统一音频-文本大语言模型。Audex 采用简洁的统一设计,仅使用单个 Transformer 解码器:音频输入被编码并投影到文本嵌入空间,而文本 token 和量化后的音频输出 token 在生成过程中被统一处理。这种架构实现了强大的音频-文本融合、无缝的多模态生成,以及与标准大语言模型训练和推理基础设施的兼容性。在训练方面,我们精心整理了包含 1574 亿个音频 token 和 3205 亿个文本 token 的音频-文本数据集。我们对这些数据集进行了多阶段监督训练,随后进行了纯文本 Cascade RL 和多领域在线策略蒸馏。Audex 在音频理解、语音识别与翻译、文本转语音、音频生成以及语音到语音生成方面达到了业界领先水平,同时在其纯文本大语言模型骨干网络的基础上,以极小或无性能回退的方式,保留了非常出色的推理、对齐、知识、长上下文和智能体能力。我们发布了模型检查点,以促进开放研究。
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.