# PolyAI 发布 Dialog-RSN-1：融合话轮转换、语音识别、函数调用与回复生成的音频原生对话模型

- 来源：MarkTechPost（RSS）
- 作者：Michal Sutter
- 发布时间：2026-07-31 13:06
- AIHOT 分数：53
- AIHOT 链接：https://aihot.virxact.com/items/cms8i5t6i0eqnrot0evbc0gpi
- 原文链接：https://www.marktechpost.com/2026/07/30/polyai-releases-dialog-rsn-1-an-audio-native-dialog-model-that-fuses-turn-taking-speech-recognition-function-calling-and-response

## AI 摘要

PolyAI 推出 Dialog-RSN-1，一款直接感知来电者音频而非读取转写的对话模型，将话轮转换、语音识别、函数调用和回复生成融合为单一音频原生模型，已用于实时生产通话。

## 正文

PolyAI has introduced Dialog-RSN-1, a dialog model that perceives the caller’s audio directly instead of reading a transcript. It fuses turn-taking, speech recognition, function calling and response generation into one audio-native model, and is already handling live production calls.

Key Takeaways

Dialog-RSN-1 is audio-aware on the input side only; TTS stays separate, so the output voice remains controllable.

It runs as a request-based LLM probed on demand, not an always-on stream that pins a GPU.

Turn-taking is the model’s first output token: EMPTY, ONGOING or COMPLETE.

PolyAI reports sub-300ms responses, +11% relative containment at a restaurant group, and −37% latency at an insurer.

English only at launch, delivered through PolyAI’s platform rather than open weights or a public API.

Is it deployable, and by whom?

Yes, but only through PolyAI: no open weights, no public API yet. Existing customers can enable it today; new customers can request early access.

Company level: large, high-call-volume enterprises. PolyAI reports 100+ enterprise customers and 2,000+ live deployments at its $86M Series D in December 2025. Self-serve developers and SMBs are not the target.

Industries: restaurants, insurance, financial services, healthcare, hotels, retail, telecom, travel and utilities.

Applications: booking and reservations, billing and payments, authentication, call routing, order management and troubleshooting.

The architecture

Two architectures dominate, and each concedes something. A cascaded stack sends only the ASR’s best guess to the LLM, so tone, hesitation and recognition uncertainty are gone before the LLM sees anything. Tuning means hand-adjusting end-pointing parameters and ASR biasing that rarely generalize across use cases. Speech-to-speech models such as GPT Realtime and Gemini Live keep the audio but bake the voice into the model, limiting pronunciation control, and always-on full-duplex variants pin a GPU for the entire call.

Dialog-RSN-1 is audio-aware on input only: one model reasons over raw audio and hands generation to a separate, promptable TTS system. It is probed on demand rather than streamed: a high-recall VAD plus a few timers decide when to run it, and the first token of the reply settles whether the agent should speak. Cheap acoustic cues only choose when to ask; the model, with full context, makes the actual turn-taking call.

How it was built

PolyAI post-trained open-weight multimodal models with supervised and reinforcement finetuning on in-house data. The pipeline is broadly base-model agnostic; PolyAI evaluated Gemma, GPT-OSS, Qwen and Mistral. Targeting sub-300ms on A100 GPUs puts candidates in the 8B dense to 30B sparse range. Latency work includes prefilling the attention cache while the user speaks, an append-only prompt template to minimize cache invalidation, routing each caller to the same GPU, a finetuned speculative drafter with mean acceptance of 3.9 tokens, and auto-reasoning learned during RFT. Transcription runs last, after the response or tool call, in parallel with speech generation.

Results

PolyAI evaluated on Dialog-Eval, an internal benchmark it plans to open-source. Each example is a call truncated at one decision point, scoring a single atomic next step rather than a full rollout. PolyAI reports Dialog-RSN-1 as the highest-scoring real-time capable model, puts the cascaded Audio-score ceiling near 77, and notes GPT Realtime 2.1 scoring on par with cascades on audio-aware examples. On transcription, gpt-4o-transcribe’s WER improved from 7.8% to 6.9% once given the same context, with Dialog-RSN-1 lower still.

For this release PolyAI focused on English; Raven 3.5 remains its recommendation for non-English and rich web chat. A technical report and a Dialog-Eval paper are planned.

Check out the Technical details here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.