Nemotron 3 Nano Omni:高效开放的多模态智能模型
阅读原文· arxiv.orgNemotron 3 Nano Omni是该系列最新模型,首次原生支持音频、文本、图像和视频输入。通过架构、训练数据及方法的改进,其在所有模态上的准确性均超越前代Nemotron Nano V2 VL,尤其在真实文档理解、长音视频理解和智能体计算机使用方面表现领先。该模型基于高效的Nemotron 3 Nano 30B-A3B骨干网络,并融入创新的多模态令牌缩减技术,实现了比同类规模模型更低的推理延迟和更高的吞吐量。为促进后续研发,团队发布了BF16、FP8和FP4格式的模型检查点,以及部分训练数据和代码库。
We introduce Nemotron 3 Nano Omni, the latest model in the Nemotron multimodal series and the first to natively support audio inputs alongside text, images, and video. Nemotron 3 Nano Omni delivers consistent accuracy improvements over its predecessor, Nemotron Nano V2 VL, across all modalities, enabled by advances in architecture, training data and recipes. In particular, Nemotron 3 delivers leading results in real-world document understanding, long audio-video comprehension, and agentic computer use. Built on the highly efficient Nemotron 3 Nano 30B-A3B backbone, Nemotron 3 Nano Omni further incorporates innovative multimodal token-reduction techniques to deliver substantially lower inference latency and higher throughput than other models of similar size. We are releasing model checkpoints in BF16, FP8, and FP4 formats, along with portions of the training data and codebase to facilitate further research and development.