MaineCoon:首个实时音频-视频社交世界模型
阅读原文· arxiv.orgMaineCoon 是一个 22B 参数的实时音频‑视频自回归模型,在单 GPU 上实现最高 47.5 FPS 的流式生成与亚秒级交互,是首个专为社交交互场景优化的实时音视频生成模型。训练中引入自重采样、跨模态表示对齐、领域偏好优化和 Reinforced Online‑Policy Distillation(ROPD)。同时设计了首个智能体流推理框架,通过智能缓存管理与提示规划支持千秒级以上的长序列生成并缓解漂移。该模型在高质量、低延迟、长时域音视频自回归建模上确立了新 SOTA 基准。
As an increasing majority of global video content is consumed on social platforms for interactive social purposes, video generation models built for social worlds are important but largely overlooked by previous studies. In this work, we define the position of social world models and build a prototype model as the first step towards this goal. While previous world models successfully simulate physical environments or gaming world exploration, they remain fundamentally detached from human-centric social dynamics. To bridge this gap as the first step to social world models, we present MaineCoon, the first real-time audio-visual autoregressive model that has 22B parameters and is capable of real-time streaming generation and sub-second interaction, with a record-breaking frame rate of up to 47.5 FPS, on a single GPU. To the best of our knowledge, MaineCoon is also the first real-time audio-visual generation model specifically optimized for social-interactive applications. To enable efficient and stable training, we introduce several novel techniques into MaineCoon, including self-resampling, cross-modal representation alignment, domain-aware preference optimization, and reinforced online-policy distillation (ROPD). We also design the first agentic streaming inference framework that supports thousand-second-scale or even longer generation while mitigating drift with agentic cache management and prompt planing. These innovations significantly accelerate training while optimizing real-time inference performance. We believe this work not only sets a new state-of-the-art (SOTA) performance benchmark for high-quality, low-latency, and long-horizon audio-visual autoregressive models, but also points out the paradigm shift desired for next-generation AI-native social platforms.