本次发布中，我们开放了 Qwen3.8-Flash-Next 的权重，这是一个多模态 MoE 模型，同时也是 Qwen4 所采用架构的早期预览。它所扮演的角色与 Qwen3-Next 之于 Qwen3.5 相同：当时引入的混合 Gated DeltaNet + Gated Attention 设计，此后已被广泛应用于 Qwen3.5、Qwen3.6、Qwen3.7 和 Qwen3.8 系列。我们再次提前发布架构变更，以便社区在完整的 Qwen4 模型家族基于此构建之前，能够先行审视这些改动。
Qwen3.8-Flash-Next 从四个方面系统性地升级了模型——注意力、残差、嵌入和优化——在提升模型能力的同时，进一步优化了计算效率、模型容量和训练稳定性：
注意力：采用 GDN + QSA 混合架构。Gated DeltaNet（GDN）高效压缩历史信息；Qwen Sparse Attention（QSA）使用压缩的轻量级索引器，在微块粒度上选择重要上下文，大幅降低长序列上的注意力计算成本。
残差：Gated Residual（GR）将残差流拓宽为 4 个分支，并通过动态门控控制读写，强化跨层信息流动和训练稳定性。
嵌入：N-gram Embedding 利用局部上下文进行查表，以极少的额外计算扩展模型容量；嵌入表可卸载至主机内存，并通过异步预取与模型计算重叠执行。
优化：采用 Muon 优化器，围绕正交化精度、Muon 与 AdamW 的分工、融合参数的分割进行了精细化调整，并为新架构重新拟合了缩放定律。
Qwen3.8-Flash-Next 拥有 125B 参数的主模型，另配有 51B 的 N-gram 嵌入，每个 token 激活 6B 参数。与 Qwen3.7-Plus 相比，Qwen3.8-Flash-Next 大幅降低了训练和推理成本——训练量仅约为其 1/9，同时在编程和办公任务上展现出更优的能力。
它原生支持 262,144 个 token 的上下文，并可通过 YaRN 扩展至 1,000,000 个 token。关于 Qwen3.8-Flash-Next 的架构、训练方法和实验分析的更多技术细节，请参阅我们 GitHub 仓库中的技术报告。
Qwen3.8-Flash-Next 的权重现已在 Hugging Face 和 ModelScope 上提供。生产版本默认支持 1M 上下文，并内置官方工具，以 Qwen3.8-Flash 的名称在 QwenCloud 上提供服务，定价为每百万输入 token 0.16 美元，每百万输出 token 0.47 美元（API 即将推出）。
语言#
Qwen3.8-Flash-NextQwen3.8-27BQwen3.7-PlusDeepSeek-V4-Flash-0731Claude-Opus-4.6 (Max)
参数量
125B27B397B284B--
激活参数量
6B27B17B13B--
N-gram 嵌入参数量
51B--------
编码
智能体编码
DeepSWE 1.1
58.742.216.554.4--
智能体编码
SWE-bench Pro
62.561.755.856.053.4
多语言软件工程
SWE-bench Multilingual
81.073.875.8--77.5
仓库级代码生成
NL2Repo-Bench
48.142.341.154.247.6
智能体
长周期办公任务
CoWorkBench
73.970.765.145.168.2
专业工作任务
JobBench
55.733.427.641.336.6
前沿智能体任务
Agents' Last Exam
Pass@1
得分
Pass@1
得分
Pass@1
得分
Pass@1
得分
真实世界工具使用
Toolathlon Verified (Pass@1)
73.567.150.670.3--
通用
指令遵循
IFBench
81.379.579.179.262.5
科学推理
GPQA Diamond
91.789.290.390.891.3
多学科推理
HLE
35.930.834.733.840.0
竞赛级编码
LiveCodeBench v6
91.990.389.690.688.8
- DeepSWE 1.1：使用 Claude Code 和 mini-SWE-agent 测试框架进行评估，temp=1.0，top_p=0.95，256K 上下文窗口。我们报告两个测试框架中的最高得分；值得注意的是，Qwen3.8-Flash-Next 在 mini-SWE-agent 上表现最佳。
- SWE-bench Pro：除 Claude-Opus-4.6 (Max) 报告其官方发布得分外，所有模型均使用 Claude Code 测试框架进行评估，temp=1.0，top_p=0.95，256K 上下文窗口。有问题的任务已修正，所有基线模型均在改进后的基准上重新评估。
- SWE-bench Multilingual：使用 mini-SWE-agent 测试框架进行评估，temp=1.0，top_p=0.95，256K 上下文窗口。
- NL2Repo-Bench：使用 Claude Code 测试框架进行评估。为防止奖励作弊，我们禁用了试图访问特定仓库的 Bash 命令，例如 pip download、pip install 和 git clone。
- CoWorkBench：一个内部协同工作基准，用于评估跨计算机科学、金融、法律、医疗及其他生产力领域的长期办公与生产力智能体任务。
- HLE：由 GPT-4o 进行评判。
- 每行最佳结果以粗体显示。
- 空单元格（--）：分数尚未公布或暂不适用。
视觉语言#
Qwen3.8-Flash-NextQwen3.8-27BQwen3.7-PlusClaude-Opus-4.6 (Max)
智能体多模态智能
多模态工具调用
ClawEval-MM
Pass@3
平均分
Pass@3
平均分
Pass@3
平均分
Pass@3
平均分
应用复刻
RecreationBench
49.947.130.2--
移动端使用
AndroidWorld
84.581.981.062.0
电脑端使用
OSWorld 2.0
可视化网页开发
Vision2Web
64.062.942.1--
通用多模态智能
具身智能
ERQA
72.365.569.840.8
长视频理解
LVBench
76.672.476.263.0
真实世界感知
RealWorldQA
88.585.986.973.9
视觉数学问题求解
MathVision
无CI
有CI
无CI
有CI
无CI
有CI
无CI
科学图表分析
CharXiv (RQ)
无CI
有CI
无CI
有CI
无CI
有CI
无CI
- ClawEval-MM：分数以“pass@3 / 平均分”形式报告。Pass@3 衡量在三次尝试中至少一次通过的百分比，平均分是三次尝试的平均得分。
- RecreationBench：一个内部长期应用复刻基准，用于评估跨五个平台（桌面端——Ubuntu、macOS、Windows，移动端——Android，以及网页端）的混合智能体能力。
- OSWorld 2.0：分数以“二元 / 部分”形式报告。二元分数是获得完整任务奖励的任务百分比，部分分数则汇总所有任务中获得的部分奖励。
- Vision2Web：分数以前端、网页和网站类别的平均值报告，使用 Claude Code 测试框架，并由 gpt-5.4-2026-03-05 进行评判。
- MathVision、CharXiv（RQ）：分数以“无置信区间 / 有置信区间”两种形式报告。MathVision 中少量错误的地面真值标注经人工核验后已修正。我们模型的分数使用固定提示词评估，例如“请逐步推理，并将最终答案放入 \boxed{} 中。”对于其他模型，我们报告在带与不带 \boxed{} 格式的两次运行中取得的较高分数。
- 每行最佳结果以粗体显示。
- 空单元格（--）表示分数尚未公布或暂不适用。
模型架构#
注意力机制：GDN + QSA，实现高效内存与精准检索#
传统全注意力机制可直接访问所有历史 token，但随着上下文不断变长，计算成本和 KV Cache 内存访问成本都会大幅增加。
沿袭 Qwen3.5 引入的架构设计，Qwen3.8-Flash-Next 采用 GDN [1] + 注意力混合架构：每四层中有三层使用 Gated DeltaNet（GDN）将历史信息持续压缩为固定大小的状态，其余一层使用全局注意力对完整上下文进行精准信息检索。
在全局注意力方面，我们进一步引入了 Qwen 稀疏注意力（QSA）。稀疏注意力通过仅关注重要上下文来降低长序列计算量。然而，DSA [2] 等现有方法仍依赖 token 级索引器来识别重要位置；随着上下文增长，索引器本身也会成为不可忽视的计算来源。
QSA 进一步压缩了这一过程：轻量级索引器先将序列聚合为微块，在块级别评估上下文重要性，再选择最相关的区域进行注意力计算。这不仅降低了注意力本身的计算成本，也减少了识别重要上下文所需的索引开销。与跨层共享索引的方法 [3] 相比，QSA 在每一层内独立进行序列压缩，降低了对跨层注意力相似度的依赖，因此特别适合 GDN 与注意力层交错排列的混合架构。
简而言之：GDN 擅长高效“记忆”，而 QSA 擅长精准“检索”。
在 1M token 规模下，QSA 的 Attention Kernel 在 Prefill 和 Decode 阶段分别可实现最高 7.6 倍和 4.9 倍的加速。在模拟高缓存复用在线服务场景（90% Prefix Cache 命中率）的实验设置中，Qwen3.8-Flash-Next 在 1M token 上下文长度下，其 Prefill 吞吐量达到 Qwen3.7-Plus 的 8.6 倍。
门控残差：为信息流动提供更多路径#
在传统 Transformer 架构中，所有层都持续从同一个残差流（Residual Stream）中读取和写入信息。随着网络不断加深，早期特征会反复与后期信息混合，导致重要信号更易被逐渐稀释。
门控残差（GR）可视为两种思想的结合：它借鉴 Hyper-Connection [4] 将残差流扩展为多个分支，同时将 GatedNorm [5] 的逐元素动态门控机制引入残差读取过程。原本单一的残差流被扩展为四个并行分支，使模型能够根据当前内容动态决定从每个分支读取多少信息、以及向每个分支写回多少信息。
这可以理解为将单一信息通道扩展为多条并行通路：部分分支负责局部信息流动，而另一些分支则将早期信息直接保留至网络深层。实证分析还发现，其中一个分支会自然涌现为一条长程通路，将第一个 Attention 层与大部分中间层及后续层连接起来。
GR 还进一步简化了 Hyper-Connection。一旦读取和写入操作具备足够的表达能力，额外的分支混合便不再带来显著收益，可以直接移除，从而降低内存访问开销和训练不稳定性来源。Gate 机制还能有效抑制激活值离群点，提升训练稳定性。此外，Residual State 支持 FP8 存储，进一步减少了内存访问开销。
N-gram 嵌入：以低成本扩展模型容量#
受 Gemma 3n 中的逐层嵌入（Per-Layer Embedding）以及 DeepSeek Engram [6] 等工作的启发，我们进一步引入了 N-gram 嵌入（N-gram Embedding），以在 Transformer 主干参数之外扩展模型容量。
标准嵌入层仅基于单个 token 进行查找。而 N-gram 嵌入则利用当前 token 与其前若干个 token 构成的局部上下文进行查找，为常见短语和局部模式提供额外的表示。
其关键优势在于，它可以在几乎不增加每个 token 额外计算量的情况下，引入大量新增参数。
Qwen3.8-Flash-Next 额外引入了 51B 的 N-gram 嵌入参数。由于查找位置可以预先确定，这些参数可以存储在主机内存中，并在模型计算的同时进行异步预取，而不会永久占用 GPU 内存。
最终模型仅在网络起始位置附近使用单一的 N-gram 嵌入层，以相对较低的额外成本，有效增加了一个大规模的“局部模式记忆”。
优化：架构与优化的协同设计#
Qwen3.8-Flash-Next 使用 Muon 优化器 [7] 进行训练，并围绕将 Muon 应用于大规模模型训练的三个关键方面进行了进一步改进：正交化精度、Muon 与 AdamW 之间的参数分配，以及融合参数矩阵的拆分。
对于真正充当二维线性映射的参数，例如 Attention、GDN 和 MoE 专家中的主要权重，我们使用 Muon。嵌入层、MoE 路由器以及 GR 中的低秩参数则继续使用 AdamW。对于在实现中融合的 QKV、SwiGLU 和 GDN 投影，我们首先根据它们所代表的独立线性变换进行拆分，然后分别进行正交化。
针对新的架构和优化器，我们重新拟合了缩放定律（Scaling Law）。结果表明，该模型可以稳定地使用更大的学习率和批大小，进一步提升了收敛效率和大规模并行训练吞吐量。
我们还发现，大规模模型训练中的常见做法——Batch Size Warmup（批大小预热）已不再必要：从小 Batch 逐步增大到目标 Batch 并不会改善最终结果，反而需要多消耗 18.8% 的优化器步数。因此，在最终的训练方案中，我们直接以目标 Batch Size 开始训练。
其他架构优化#
其余组件沿用了 Qwen3-Next 中确立、并在 Qwen3.5–Qwen3.8 系列中持续打磨的设计。
超稀疏 MoE：通过全局负载均衡 [8]，在保持激活专家数量不变的同时增加专家总参数量，可以稳定降低训练损失。因此，Qwen3.8-Flash-Next 采用大型专家池，每个 token 仅路由少量专家，并搭配一个共享专家。
多 token 预测：MTP 模块采用多步训练，保持训练与推理的一致性，从而提升真实场景中投机解码的接受率，同时增强主干模型的性能。其全注意力层也替换为 QSA。
训练稳定性：保留了零中心 RMSNorm（对归一化权重施加权重衰减）、注意力输出门控机制 [9] 以及归一化的 MoE 路由器初始化。这些设计使小规模消融实验更可靠，并有助于大规模训练平稳进行。
基座模型性能#
我们将 Qwen3.8-Flash-Next-Base 与 Qwen3.8-27B 和 Qwen3.7-Plus 的基座模型进行了对比。
Qwen3.8-Flash-Next-BaseQwen3.8-27B-BaseQwen3.7-Plus-Base
参数量
125B27B397B
激活参数量
6B27B17B
N-gram 嵌入参数量
51B----
通用任务
MMLU
90.3687.5190.43
MMLU-Redux
90.6887.2691.47
MMLU-Pro
73.2368.6070.90
SuperGPQA
51.3644.8648.42
BBH
90.8789.5689.41
数学与 STEM 任务
GPQA
51.4245.0151.52
GSM8K
93.2993.1892.95
MATH
72.7860.5474.38
编程任务
EvalPlus
78.7676.0578.06
MultiPL-E
79.0974.5081.68
SWEBench-Pretrain
50.9941.6649.24
多语言任务
MGSM
89.3386.3785.42
MMMLU
84.8679.7484.53
INCLUDE
78.4074.3778.90
- 每行最佳结果以粗体显示。
- 空单元格（--）：表示分数尚未公布或不适用。
在激活参数为 6B 的情况下，Qwen3.8-Flash-Next-Base 在 14 项基准测试中的 8 项上取得了最佳成绩，包括 MMLU-Pro、SuperGPQA、BBH、GSM8K、EvalPlus、SWEBench-Pretrain、MGSM 和 MMMLU，并且在 MMLU、MMLU-Redux、GPQA、MATH 和 MultiPL-E 上保持与 Qwen3.7-Plus-Base 接近的水平。51B 的 N-gram 嵌入参数通过确定性寻址方式处理，不占用每个 token 的矩阵乘法预算。
使用 Qwen3.8-Flash-Next 进行开发#
Qwen3.8-Flash-Next 作为开放权重模型在 HuggingFace 和 ModelScope 上提供，并在 QwenCloud 上提供官方托管 API。该模型旨在平衡能力、延迟和成本，非常适合高吞吐量应用、工具驱动的工作流程以及编程与协作助手。下面，您可以了解如何调用 QwenCloud API，并将 Qwen3.8-Flash-Next 集成到智能体系统和编程助手中。
即将推出。该 API 尚未上线——将在本文发布后不久启用。本节中的示例届时即可使用。
Qwen3.8-Flash-Next 可通过 API 使用：
QwenCloud#
在 QwenCloud 上，该模型以名称 qwen3.8-flash 提供服务。QwenCloud 支持行业标准协议，包括兼容 OpenAI 的 Chat Completions 和 Responses API，以及兼容 Anthropic 的接口。
""" 环境变量： DASHSCOPE_API_KEY：您的 API 密钥，来自 https://home.qwencloud.com/ DASHSCOPE_BASE_URL：（可选）兼容模式 API 的基础 URL。
- 北京：https://dashscope.aliyuncs.com/compatible-mode/v1
- 新加坡：https://dashscope-intl.aliyuncs.com/compatible-mode/v1
- 美国（弗吉尼亚州）：https://dashscope-us.aliyuncs.com/compatible-mode/v1 """ from openai import OpenAI import os api_key = os.environ.get("DASHSCOPE_API_KEY") if not api_key: raise ValueError( "DASHSCOPE_API_KEY is required. " "Set it via: export DASHSCOPE_API_KEY='your-api-key'" ) client = OpenAI( api_key=api_key, base_url=os.environ.get( "DASHSCOPE_BASE_URL", "https://dashscope-intl.aliyuncs.com/compatible-mode/v1", ), ) messages = [{"role": "user", "content": "编写一个 Python 函数来合并两个已排序的链表。"}] completion = client.chat.completions.create( model="qwen3.8-flash", messages=messages, extra_body={ "enable_thinking": True,
"preserve_thinking": True,
}, reasoning_effort="xhigh", # 支持的级别为 xhigh、medium 和 low stream=True, ) reasoning_content = "" answer_content = "" is_answering = False print("\n" + "=" * 20 + "推理过程" + "=" * 20 + "\n") for chunk in completion: if not chunk.choices: print("\n用量：") print(chunk.usage) continue delta = chunk.choices[0].delta if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None: if not is_answering: print(delta.reasoning_content, end="", flush=True) reasoning_content += delta.reasoning_content if hasattr(delta, "content") and delta.content: if not is_answering: print("\n" + "=" * 20 + "回答" + "=" * 20 + "\n") is_answering = True print(delta.content, end="", flush=True) answer_content += delta.content
智能体框架与编程助手#
Qwen3.8-Flash-Next 与主流智能体框架和编程助手无缝集成：
QwenWork（即将推出）#
QwenWork 是阿里巴巴旗舰级 AI 生产力平台，旨在帮助个人和企业自动化日常任务并提升运营效率。
我们很高兴地宣布，QwenWork 已集成 Qwen3.8-Flash-Next，为其新推出的“标准”模式提供支持，充分利用该模型的尖端能力，提供流畅且高性价比的体验，为职场 AI 智能体树立新标准。
更多详情请参阅官方文档！
Claude Code#
Qwen API 支持 Anthropic API 协议，可直接与 Claude Code 配合使用：
npm install -g @anthropic-ai/claude-code export ANTHROPIC_MODEL="qwen3.8-flash" export ANTHROPIC_SMALL_FAST_MODEL="qwen3.8-flash" export ANTHROPIC_BASE_URL=https://dashscope-intl.aliyuncs.com/apps/anthropic export ANTHROPIC_AUTH_TOKEN= claude
Codex#
Qwen API 支持 OpenAI Responses 协议，因此可以与 Codex 配合使用：
在 ~/.codex/model-catalog.local.json 中
{ "models": [ { "slug": "qwen3.8-flash", "display_name": "qwen3.8-flash", "description": "QwenCloud: Qwen3.8-Flash", "default_reasoning_level": "xhigh", "supported_reasoning_levels": [ { "effort": "low", "description": "快速响应，推理较轻" }, { "effort": "medium", "description": "针对复杂问题提供更深入的推理" }, { "effort": "xhigh", "description": "针对复杂问题提供超高深度推理" } ], "context_window": 1000000, "effective_context_window_percent": 95, "supports_parallel_tool_calls": true, "supports_image_detail_original": true, "input_modalities": ["text", "image"], "shell_type": "default", "visibility": "list", "supported_in_api": true, "priority": 1, "base_instructions": "", "support_verbosity": false, "supports_reasoning_summaries": false, "experimental_supported_tools": [], "truncation_policy": { "mode": "bytes", "limit": 10000 } } ] }
在 ~/.codex/config.toml 中
model_catalog_json = "~/.codex/model-catalog.local.json" model_provider = "QwenCloud" model = "qwen3.8-flash" [model_providers.QwenCloud] name = "QwenCloud" base_url = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1" env_key = "OPENAI_API_KEY" wire_api = "responses"
npm install -g @openai/codex export OPENAI_API_KEY= codex
Qoder CLI#
Qoder 与 Qwen 协同进化，专为智能体编程而生：
curl -fsSL https://qoder.com/install | bash qoder
Qwen Code#
Qwen Code 针对 Qwen 系列进行了深度优化：
npm install -g @qwen-code/qwen-code@latest qwen
OpenClaw#
通过 QwenCloud 连接 OpenClaw：
curl -fsSL https://openclaw.ai/install.sh | bash export DASHSCOPE_API_KEY= openclaw dashboard
配置 ~/.openclaw/openclaw.json：
{ "models": { "mode": "merge", "providers": { "qwencloud": { "baseUrl": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1", "apiKey": "DASHSCOPE_API_KEY", "api": "openai-completions", "models": [ { "id": "qwen3.8-flash", "name": "qwen3.8-flash", "reasoning": true, "input": ["text", "image"], "contextWindow": 1000000, "maxTokens": 65536 } ] } } }, "agents": { "defaults": { "model": { "primary": "qwencloud/qwen3.8-flash" } } } }
Qwen3.8-Flash-Next 在 Qwen3-Next 引入的混合架构基础上，沿四个方向进行了扩展：注意力机制、残差连接、嵌入向量和优化。QSA 将序列压缩到每一层内的微块中，在长上下文场景下同时降低了注意力计算成本和索引成本，同时保持了精确的检索能力。Gated Residual 将残差流拓宽为多个并行分支，并通过逐元素、依赖数据的门控机制控制读写操作，以可忽略不计的算术开销改善了跨层信息流动和训练稳定性；残差状态还可以额外以 FP8 格式保存，进一步减少内存流量。N-gram 嵌入通过确定性寻址的查找内存扩展容量，这种内存可以以可忽略不计的每 token 计算开销进行扩展，并可卸载到主机内存。在优化方面，Muon 被用作主要优化器，其中正交化精度、参数分配和融合矩阵切分是关键实现选择，同时针对新架构重新拟合了缩放定律。
我们提前发布这些权重，以便社区能够独立评估该架构，正如我们对 Qwen3-Next 所做的那样，我们将继续对其进行优化，朝着 Qwen4 的方向推进。
@techreport{qwen2026design, title = {On the Design of {Qwen3.8-Next} Architecture: Evaluation, Efficiency, and Training Stability}, author = {{Qwen Team}}, institution = {Alibaba Group}, month = {August}, year = {2026} } @misc{qwen3.8flashnext, title = {{Qwen3.8-Flash-Next}: A New Architecture, Towards Ultimate Cost-Efficiency}, author = {{Qwen Team}}, month = {August}, year = {2026}, url = {https://qwen.ai/blog?id=qwen3.8-flash-next} }
参考文献#
[1] Gated Delta Networks: Improving Mamba2 with Delta Rule
[2] DeepSeek-V3.2：推动开放大语言模型前沿发展
[3] IndexCache：通过跨层索引复用加速稀疏注意力
[4] 超连接（Hyper-Connections）
[5] 注意力与残差汇（Residual Sinks）的统一视角：异常值驱动的重缩放对 Transformer 训练至关重要
[6] 通过可扩展查找实现条件记忆：大语言模型稀疏性的新维度
[7] Muon：面向神经网络隐藏层的优化器
[8] 细节中的魔鬼：论训练专业化混合专家模型时负载均衡损失的实现
In this release we are opening the weights of Qwen3.8-Flash-Next, a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4. It plays the same role that Qwen3-Next played for Qwen3.5: the hybrid Gated DeltaNet + Gated Attention design introduced at that time has since been used across the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series. We are again releasing the architectural changes early, so that the community can examine them before the full Qwen4 model family is built on top of them.
Qwen3.8-Flash-Next upgrades the model systematically along four aspects — attention, residual, embedding and optimization — improving model capability while further optimizing computational efficiency, model capacity and training stability:
Attention: A GDN + QSA hybrid architecture. Gated DeltaNet (GDN) compresses the history efficiently; Qwen Sparse Attention (QSA) uses a compressed lightweight indexer to select the important context at micro-block granularity, substantially reducing the cost of attention on long sequences.
Residual: Gated Residual (GR) widens the residual stream into 4 branches and controls reads and writes with a dynamic gate, strengthening cross-layer information flow and training stability.
Embedding: N-gram Embedding looks up a table using the local context to scale model capacity with very little extra computation; the embedding table can be offloaded to host memory and overlapped with model computation through asynchronous prefetching.
Optimization: The Muon optimizer is used, refined around orthogonalization accuracy, the division of labour between Muon and AdamW, and the splitting of fused parameters, with the scaling law refitted for the new architecture.
Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. Compared with Qwen3.7-Plus, Qwen3.8-Flash-Next substantially reduces both training and inference cost — training takes only about 1/9 as much, yet it delivers superior capabilities in coding and office tasks.
It natively supports 262,144 tokens of context and is extensible to 1,000,000 tokens with YaRN. For more technical details on the architecture, training methodology, and experimental analysis of Qwen3.8-Flash-Next, please refer to the technical report in our GitHub repository.
Qwen3.8-Flash-Next weights are now available on Hugging Face and ModelScope. The production version, with 1M context by default and official built-in tools, is served as Qwen3.8-Flash on QwenCloud, priced at 0.16 USD per million input tokens and 0.47 USD per million output tokens (API coming soon).
Language#
Qwen3.8-Flash-NextQwen3.8-27BQwen3.7-PlusDeepSeek-V4-Flash-0731Claude-Opus-4.6 (Max)
Params
125B27B397B284B--
Activated params
6B27B17B13B--
N-gram embedding params
51B--------
Coding
Agentic coding
DeepSWE 1.1
58.742.216.554.4--
Agentic coding
SWE-bench Pro
62.561.755.856.053.4
Multilingual software engineering
SWE-bench Multilingual
81.073.875.8--77.5
Repo-level code generation
NL2Repo-Bench
48.142.341.154.247.6
Agent
Long-horizon office work
CoWorkBench
73.970.765.145.168.2
Professional job tasks
JobBench
55.733.427.641.336.6
Frontier agentic tasks
Agents' Last Exam
Pass@1
Score
Pass@1
Score
Pass@1
Score
Pass@1
Score
Real-world tool use
Toolathlon Verified (Pass@1)
73.567.150.670.3--
General
Instruction following
IFBench
81.379.579.179.262.5
Scientific reasoning
GPQA Diamond
91.789.290.390.891.3
Multidisciplinary reasoning
HLE
35.930.834.733.840.0
Competitive coding
LiveCodeBench v6
91.990.389.690.688.8
- DeepSWE 1.1: evaluated with the Claude Code and mini-SWE-agent harnesses, temp=1.0, top_p=0.95, 256K context window. We report the highest score across the two harnesses; notably, Qwen3.8-Flash-Next performs best on mini-SWE-agent.
- SWE-bench Pro: except for Claude-Opus-4.6 (Max), for which we report the officially published score, all models are evaluated with the Claude Code harness, temp=1.0, top_p=0.95, 256K context window. Problematic tasks were corrected and all baseline models were re-evaluated on the refined benchmark.
- SWE-bench Multilingual: evaluated with the mini-SWE-agent harness, temp=1.0, top_p=0.95, 256K context window.
- NL2Repo-Bench: evaluated with the Claude Code harness. To prevent reward hacking, we disable Bash commands that attempt to access the specific repository, such as pip download, pip install and git clone.
- CoWorkBench: an in-house cowork benchmark for evaluating long-horizon office and productivity agent tasks across computer science, finance, law, medical and other productivity domains.
- HLE: judged by GPT-4o.
- The best result in each row is shown in bold.
- Empty cells (--): scores are not yet available or are not applicable.
Vision Language#
Qwen3.8-Flash-NextQwen3.8-27BQwen3.7-PlusClaude-Opus-4.6 (Max)
Agentic Multimodal Intelligence
Multimodal tool use
ClawEval-MM
Pass@3
Average
Pass@3
Average
Pass@3
Average
Pass@3
Average
Application recreation
RecreationBench
49.947.130.2--
Mobile use
AndroidWorld
84.581.981.062.0
Computer use
OSWorld 2.0
Visual web development
Vision2Web
64.062.942.1--
General Multimodal Intelligence
Embodied intelligence
ERQA
72.365.569.840.8
Long video understanding
LVBench
76.672.476.263.0
Real-world perception
RealWorldQA
88.585.986.973.9
Visual math problem solving
MathVision
Without CI
With CI
Without CI
With CI
Without CI
With CI
Without CI
Scientific chart analysis
CharXiv (RQ)
Without CI
With CI
Without CI
With CI
Without CI
With CI
Without CI
- ClawEval-MM: scores are reported as "pass@3 / average score". Pass@3 measures the percentage passed in at least one of three trials, and the average score is the mean score across the three trials.
- RecreationBench: an in-house long-horizon application-recreation benchmark for evaluating hybrid-agent abilities spanning five platforms — desktop (Ubuntu, macOS, Windows), mobile (Android) and web.
- OSWorld 2.0: scores are reported as "binary / partial". The binary score is the percentage of tasks that receive the full task reward, while the partial score aggregates the partial rewards obtained across all tasks.
- Vision2Web: scores are reported as the average over the frontend, webpage and website categories, using the Claude Code harness and judged by gpt-5.4-2026-03-05.
- MathVision, CharXiv (RQ): scores are reported as "without CI / with CI". A small number of incorrect ground-truth annotations in MathVision were corrected after manual verification. Our model's score is evaluated using a fixed prompt, e.g. "Please reason step by step, and put your final answer within \boxed{}." For other models, we report the higher score between runs with and without the \boxed{} formatting.
- The best result in each row is shown in bold.
- Empty cells (--) indicate scores not yet available or not applicable.
Model Architecture#
Attention: GDN + QSA for Efficient Memory and Precise Retrieval#
Traditional Full Attention provides direct access to all previous tokens, but as the context grows longer, both computation and KV Cache memory-access costs increase substantially.
Following the architecture design introduced in Qwen3.5, Qwen3.8-Flash-Next adopts a GDN [1] + Attention Hybrid architecture: three out of every four layers use Gated DeltaNet (GDN) to continuously compress historical information into a fixed-size state, while the remaining layer uses global Attention for precise retrieval of information across the full context.
For global Attention, we further introduce Qwen Sparse Attention (QSA). Sparse Attention reduces long-sequence computation by attending only to important context. However, existing approaches such as DSA [2] still rely on a token-level indexer to identify important positions; as the context grows, the indexer itself becomes a non-negligible source of computation.
QSA further compresses this process: a lightweight indexer first aggregates the sequence into micro-blocks, estimates context importance at the block level, and then selects the most relevant regions for Attention. This reduces not only the cost of Attention itself, but also the indexing overhead required to identify important context. Compared with approaches that share indices across layers [3], QSA performs sequence compression independently within each layer, reducing its dependence on cross-layer Attention similarity and making it particularly well suited to Hybrid architectures where GDN and Attention layers are interleaved.
Put simply: GDN efficiently “remembers,” while QSA precisely “retrieves.”
At 1M tokens, QSA’s Attention Kernel achieves up to 7.6× and 4.9× speedups in Prefill and Decode, respectively. In an experimental setup representative of online serving scenarios with high cache reuse (a 90% Prefix Cache hit rate), Qwen3.8-Flash-Next achieves 8.6× the Prefill throughput of Qwen3.7-Plus at a 1M-token context length.
Gated Residual: More Paths for Information Flow#
In a traditional Transformer, all layers continuously read from and write to the same Residual Stream. As the network becomes deeper, early features are repeatedly mixed with later information, making important signals more likely to be gradually diluted.
Gated Residual (GR) can be viewed as a combination of two ideas: it follows Hyper-Connection [4] in widening the residual stream into multiple branches, while incorporating the element-wise dynamic gating of GatedNorm [5] into the residual read. The original single residual stream is expanded into four parallel branches, allowing the model to dynamically determine how much information to read from each branch and how much to write back to each branch based on the current content.
This can be conceptualized as expanding a single information channel into multiple parallel pathways: some branches handle local information flow, while others preserve early information directly deep into the network layers. Empirical analysis also reveals that one of these branches naturally emerges as a long-range pathway connecting the first Attention layer to most of the middle and subsequent layers.
GR also further simplifies Hyper-Connection. Once the read and write operations are expressive enough, additional branch mixing yields no significant benefits and can thus be directly removed, thereby reducing memory access overhead and sources of instability. The Gate also effectively suppresses activation outliers and improves training stability. In addition, the Residual State supports FP8 storage, further reducing memory-access overhead.
N-gram Embedding: Expanding Model Capacity at Low Cost#
Inspired by Per-Layer Embedding in Gemma 3n and works such as DeepSeek Engram [6], we further introduce N-gram Embedding to scale model capacity beyond the parameters of the Transformer backbone.
A standard Embedding performs a lookup based on a single token. N-gram Embedding instead performs lookups using the local context formed by the current token and several preceding tokens, providing additional representations for common phrases and local patterns.
Its key advantage is that it can add a large number of parameters with almost no additional computation per token.
Qwen3.8-Flash-Next introduces an additional 51B N-gram Embedding parameters. Because lookup locations can be determined in advance, these parameters can be stored in Host Memory and asynchronously prefetched in parallel with model computation, without permanently occupying GPU memory.
The final model uses only a single N-gram Embedding layer near the beginning of the network, effectively adding a large-scale “local-pattern memory” at relatively low additional cost.
Optimization: Co-designing Architecture and Optimization#
Qwen3.8-Flash-Next is trained with the Muon Optimizer [7], with further improvements around three key aspects of applying Muon to large-scale model training: orthogonalization accuracy, parameter assignment between Muon and AdamW, and splitting fused parameter matrices.
For parameters that genuinely act as two-dimensional linear maps, such as the main weights in Attention, GDN, and MoE Experts, we use Muon. Embeddings, the MoE Router, and the low-rank parameters in GR continue to use AdamW. For QKV, SwiGLU, and GDN projections that are fused in the implementation, we first split them according to the independent linear transformations they represent, and then perform orthogonalization separately.
For the new architecture and Optimizer, we refit the Scaling Law. The results show that the model can stably use larger Learning Rates and Batch Sizes, further improving convergence efficiency and large-scale parallel training throughput.
We also find that Batch Size Warmup, a common practice in large-scale model training, is no longer necessary: gradually increasing from a small Batch to the target Batch does not improve the final result, but instead requires 18.8% more optimizer steps. In the final training Recipe, we therefore start directly with the target Batch Size.
Other Architecture Optimizations#
The remaining components follow the design established in Qwen3-Next and refined through the Qwen3.5–Qwen3.8 series.
Ultra-sparse MoE: With global load balancing [8], increasing total expert parameters while keeping the number of activated experts fixed steadily reduces training loss. Qwen3.8-Flash-Next therefore uses a large expert pool with a small number of routed experts per token, together with one shared expert.
Multi-Token Prediction: The MTP module is trained with multiple steps, maintaining consistency between training and inference and thereby improving the acceptance rate of speculative decoding in real scenarios, while also enhancing the performance of the backbone. Its full-attention layers are replaced with QSA as well.
Training stability: Zero-centered RMSNorm with weight decay applied to norm weights, the attention output gating mechanism [9], and normalized MoE router initialization are retained. These designs make small-scale ablations more reliable and help large-scale training run smoothly.
Base Model Performance#
We compare Qwen3.8-Flash-Next-Base with the base models of Qwen3.8-27B and Qwen3.7-Plus.
Qwen3.8-Flash-Next-BaseQwen3.8-27B-BaseQwen3.7-Plus-Base
Params
125B27B397B
Activated params
6B27B17B
N-gram embedding params
51B----
General tasks
MMLU
90.3687.5190.43
MMLU-Redux
90.6887.2691.47
MMLU-Pro
73.2368.6070.90
SuperGPQA
51.3644.8648.42
BBH
90.8789.5689.41
Math & STEM tasks
GPQA
51.4245.0151.52
GSM8K
93.2993.1892.95
MATH
72.7860.5474.38
Coding tasks
EvalPlus
78.7676.0578.06
MultiPL-E
79.0974.5081.68
SWEBench-Pretrain
50.9941.6649.24
Multilingual tasks
MGSM
89.3386.3785.42
MMMLU
84.8679.7484.53
INCLUDE
78.4074.3778.90
- The best result in each row is shown in bold.
- Empty cells (--): scores are not yet available or not applicable.
With 6B activated parameters, Qwen3.8-Flash-Next-Base achieves the best result on 8 of the 14 benchmarks, including MMLU-Pro, SuperGPQA, BBH, GSM8K, EvalPlus, SWEBench-Pretrain, MGSM and MMMLU, and remains close to Qwen3.7-Plus-Base on MMLU, MMLU-Redux, GPQA, MATH and MultiPL-E. The 51B N-gram embedding parameters are deterministically addressed and do not enter the per-token matrix-multiplication budget.
Develop with Qwen3.8-Flash-Next#
Qwen3.8-Flash-Next is available as an open-weight model on HuggingFace and ModelScope, with official managed APIs on QwenCloud. Designed to balance capability, latency, and cost, it is well suited for high-volume applications, tool-driven workflows, and coding & coworking assistants. In the following, you can explore how to call the QwenCloud API and integrate Qwen3.8-Flash-Next into agentic systems and coding assistants.
Coming soon. The API is not live yet — it will be enabled shortly after this post goes out. The examples in this section will work from then on.
Qwen3.8-Flash-Next is available via API:
QwenCloud#
On QwenCloud, the model is served under the name qwen3.8-flash. QwenCloud supports industry-standard protocols, including OpenAI-compatible Chat Completions and Responses APIs, alongside an Anthropic-compatible interface.
""" Environment variables: DASHSCOPE_API_KEY: Your API Key from https://home.qwencloud.com/ DASHSCOPE_BASE_URL: (optional) Base URL for compatible-mode API.
- Beijing: https://dashscope.aliyuncs.com/compatible-mode/v1
- Singapore: https://dashscope-intl.aliyuncs.com/compatible-mode/v1
- US (Virginia): https://dashscope-us.aliyuncs.com/compatible-mode/v1 """ from openai import OpenAI import os api_key = os.environ.get("DASHSCOPE_API_KEY") if not api_key: raise ValueError( "DASHSCOPE_API_KEY is required. " "Set it via: export DASHSCOPE_API_KEY='your-api-key'" ) client = OpenAI( api_key=api_key, base_url=os.environ.get( "DASHSCOPE_BASE_URL", "https://dashscope-intl.aliyuncs.com/compatible-mode/v1", ), ) messages = [{"role": "user", "content": "Write a Python function to merge two sorted linked lists."}] completion = client.chat.completions.create( model="qwen3.8-flash", messages=messages, extra_body={ "enable_thinking": True,
"preserve_thinking": True,
}, reasoning_effort="xhigh", # supported levels are xhigh, medium, and low stream=True, ) reasoning_content = "" answer_content = "" is_answering = False print("\n" + "=" * 20 + "Reasoning" + "=" * 20 + "\n") for chunk in completion: if not chunk.choices: print("\nUsage:") print(chunk.usage) continue delta = chunk.choices[0].delta if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None: if not is_answering: print(delta.reasoning_content, end="", flush=True) reasoning_content += delta.reasoning_content if hasattr(delta, "content") and delta.content: if not is_answering: print("\n" + "=" * 20 + "Answer" + "=" * 20 + "\n") is_answering = True print(delta.content, end="", flush=True) answer_content += delta.content
Agent Frameworks & Coding Assistants#
Qwen3.8-Flash-Next integrates seamlessly with popular agent frameworks and coding assistants:
QwenWork (Coming Soon) #
QwenWork is Alibaba’s flagship AI productivity platform, designed to help individuals and enterprises automate daily tasks and accelerate operational efficiency.
We are excited to share that QwenWork has integrated Qwen3.8-Flash-Next to power its newly launched “Standard” mode, leveraging the model’s cutting-edge capabilities to deliver a seamless, cost-effective experience that sets a new standard for AI agents in the workplace.
Learn more in the official documentation!
Claude Code#
Qwen APIs support the Anthropic API protocol, enabling direct use with Claude Code:
npm install -g @anthropic-ai/claude-code export ANTHROPIC_MODEL="qwen3.8-flash" export ANTHROPIC_SMALL_FAST_MODEL="qwen3.8-flash" export ANTHROPIC_BASE_URL=https://dashscope-intl.aliyuncs.com/apps/anthropic export ANTHROPIC_AUTH_TOKEN= claude
Codex#
Qwen APIs support the OpenAI Responses protocol, enabling use with Codex:
In ~/.codex/model-catalog.local.json
{ "models": [ { "slug": "qwen3.8-flash", "display_name": "qwen3.8-flash", "description": "QwenCloud: Qwen3.8-Flash", "default_reasoning_level": "xhigh", "supported_reasoning_levels": [ { "effort": "low", "description": "Fast responses with lighter reasoning" }, { "effort": "medium", "description": "Greater reasoning depth for complex problems" }, { "effort": "xhigh", "description": "Extra high reasoning depth for complex problems" } ], "context_window": 1000000, "effective_context_window_percent": 95, "supports_parallel_tool_calls": true, "supports_image_detail_original": true, "input_modalities": ["text", "image"], "shell_type": "default", "visibility": "list", "supported_in_api": true, "priority": 1, "base_instructions": "", "support_verbosity": false, "supports_reasoning_summaries": false, "experimental_supported_tools": [], "truncation_policy": { "mode": "bytes", "limit": 10000 } } ] }
In ~/.codex/config.toml
model_catalog_json = "~/.codex/model-catalog.local.json" model_provider = "QwenCloud" model = "qwen3.8-flash" [model_providers.QwenCloud] name = "QwenCloud" base_url = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1" env_key = "OPENAI_API_KEY" wire_api = "responses"
npm install -g @openai/codex export OPENAI_API_KEY= codex
Qoder CLI#
Qoder co-evolves with Qwen for agentic coding:
curl -fsSL https://qoder.com/install | bash qoder
Qwen Code#
Qwen Code is deeply optimized for the Qwen series:
npm install -g @qwen-code/qwen-code@latest qwen
OpenClaw#
Connect to OpenClaw via QwenCloud:
curl -fsSL https://openclaw.ai/install.sh | bash export DASHSCOPE_API_KEY= openclaw dashboard
Configure ~/.openclaw/openclaw.json:
{ "models": { "mode": "merge", "providers": { "qwencloud": { "baseUrl": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1", "apiKey": "DASHSCOPE_API_KEY", "api": "openai-completions", "models": [ { "id": "qwen3.8-flash", "name": "qwen3.8-flash", "reasoning": true, "input": ["text", "image"], "contextWindow": 1000000, "maxTokens": 65536 } ] } } }, "agents": { "defaults": { "model": { "primary": "qwencloud/qwen3.8-flash" } } } }
Qwen3.8-Flash-Next extends the hybrid architecture introduced in Qwen3-Next along four directions: attention, residual, embedding and optimization. QSA compresses the sequence into micro-blocks within each layer, reducing both the attention cost and the indexing cost at long context while keeping precise retrieval. Gated Residual widens the residual stream into several parallel branches and controls reads and writes with an elementwise, data-dependent gate, improving cross-layer information flow and training stability at negligible arithmetic cost; the residual state can additionally be kept in FP8, which further reduces memory traffic. N-gram embedding scales capacity through deterministically addressed lookup memory, which can be scaled with negligible per-token computation and offloaded to host memory. On the optimization side, Muon is used as the main optimizer, with orthogonalization accuracy, parameter assignment and fused-matrix splitting as the decisive implementation choices, and the scaling law refitted for the new architecture.
We release these weights early so that the architecture can be evaluated independently by the community, as we did with Qwen3-Next, and we will continue to refine it towards Qwen4.
@techreport{qwen2026design, title = {On the Design of {Qwen3.8-Next} Architecture: Evaluation, Efficiency, and Training Stability}, author = {{Qwen Team}}, institution = {Alibaba Group}, month = {August}, year = {2026} } @misc{qwen3.8flashnext, title = {{Qwen3.8-Flash-Next}: A New Architecture, Towards Ultimate Cost-Efficiency}, author = {{Qwen Team}}, month = {August}, year = {2026}, url = {https://qwen.ai/blog?id=qwen3.8-flash-next} }
References#
[1] Gated Delta Networks: Improving Mamba2 with Delta Rule
[2] DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
[3] IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
[4] Hyper-Connections
[5] A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
[6] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
[7] Muon: An Optimizer for Hidden Layers in Neural Networks
[8] Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models