摘要
我们在 8× NVIDIA H200 上，使用 SGLang Diffusion 对 MiniMax-H3 视频生成进行了基准测试，在六个工作负载中固定了提示词、随机种子、分辨率、帧率和去噪步数。
- SGLang 的密集无损路径比 Diffusers 快 1.85–1.95 倍，且无任何近似：相同的去噪工作量，在更快的运行时上完成。
- 叠加步数复用与稀疏注意力最高可达 6.24 倍加速，平均 SSIM 为 0.76–0.91。测试中最快的配置 SubBlock 0.80 + Cache-DiT stride，在 5 秒/10 秒 T2VA 上实现 5.06 倍/5.72 倍加速，在 FL2VA 上实现 5.86 倍/6.24 倍加速。代价并不均匀：FL2VA 在该配置下保持 0.85–0.91 的 SSIM，而 T2VA 降至 0.76–0.78。
- 若以质量为先作为默认，可单独使用 Cache-DiT（最高 2.99 倍加速，平均 SSIM 0.90–0.92）。若追求均衡取舍，SubBlock 0.75 + Cache-DiT stride 可在 SSIM 0.79–0.90 下实现 4.90–5.93 倍加速。
- 加速收益来自三个可组合的层面：融合内核（单独测试时，非 GEMM 站点各自提速 2.00–12.16 倍——这是微基准测试，并非端到端累加）、步数复用（Cache-DiT 跳过冗余去噪步骤），以及 SubBlock 稀疏注意力（NVIDIA 的块稀疏前向计算，可降低实际运行步骤的成本）。
范围说明。本次对比覆盖了 SGLang Diffusion 的三个加速旋钮。它还支持更多有损路径，但不在本次测试范围内——包括量化和渐进式分辨率——因此这里的数字只是可用加速空间的一部分，而非其上限。以下所有数据均为实测结果，而非预估；文末的视频片段可供您自行判断质量代价。
| 硬件 | 8× NVIDIA H200（141 GB） |
| 工作负载 | MiniMax-H3 · 1344×768 · 24 FPS · 50 去噪步 · 5 秒和 10 秒输出 |
| 并行方式 | 所有模式均使用 8 块 GPU；Diffusers 使用 CP8，SGLang 使用 SP/Ulysses 度 8 |
| 版本 | SGLang v0.5.18 (d90318b3e2) |
| 测试日期 | 2026-08-18 |
背景
尽管 SGLang Diffusion 已为 MiniMax-H3 提供了快速的无损路径，但社区一直期待更快的、有损的高质量视频生成。基于其长期积累的丰富有损加速旋钮栈，SGLang Diffusion 在过去几周一直在积极推进这项工作；本文是这些旋钮实际效果的首次实测报告。
视频扩散模型的开销主要由两部分构成：去噪循环需要将同一个 Transformer 反复运行数十次，而每一步的大部分计算预算都消耗在超长 token 序列的注意力计算上。一段 5 秒、1344×768 分辨率、24 FPS、50 步去噪的片段，早已远超单张 GPU 能够实际处理的极限，因此问题不在于是否要并行化，而在于剩余计算中有多少是可以被省去的。
三种加速方案从不同方向切入，并且可以叠加组合：
- 融合内核（Fused kernels）在不改变数学计算的前提下，削减每一步的固定开销。
- Cache-DiT 在去噪步骤之间复用计算结果，因此部分步骤根本无需执行。
- SubBlock 稀疏注意力通过跳过贡献低于阈值的注意力块，降低实际执行步骤的计算成本。
第一种是无损的。另外两种则是在相似度与速度之间做权衡，这也是本文中所有数字都以 SSIM（结构相似性指标）对照无损基线来报告的原因。
概览
答案取决于基线。与匹配的 Diffusers 方案相比，SGLang 的稠密无损路径在两种任务和两种时长下都已经快了约 2 倍。Cache-DiT 在去噪步骤之间复用计算结果，而 SubBlock 稀疏注意力则降低仍须执行步骤的成本。两者组合构成了本矩阵中最快的路径。
若追求以质量为先的加速默认配置，可使用 Cache-DiT conservative 或 Cache-DiT stride（不叠加 SubBlock）。若追求速度与质量的均衡取舍，可使用 SubBlock 0.75 + Cache-DiT stride，在 5 秒任务上实现 4.90–5.64 倍加速，在 10 秒任务上实现 5.44–5.93 倍加速。
下图汇总了整体基准测试结果表。
详细结果
本文中的每一种配置都可以通过 SGLang 的 MiniMax-H3 cookbook 页面复现，该页面提供了每种模式的确切启动参数。
我们报告的是生成侧的推理时间；服务器启动、预热、HTTP 轮询和 MP4 下载时间均不包含在内。对于每个任务和时长，延迟和 SSIM 均在三个不同的提示词上评估。加速倍数以匹配的 Diffusers 方案为基准。SSIM 在 YUV420 色彩空间下对所有帧计算，并以匹配的 SGLang 无损视频为参照。
T2VA
| 模式 | 5 秒中位数 / 加速倍数 | 10 秒中位数 / 加速倍数 | 5 秒平均 SSIM | 10 秒平均 SSIM |
|---|---|---|---|---|
| Diffusers | 74.34 秒 / 1.00× | 207.71 秒 / 1.00× | — | — |
| SGLang 无损模式 | 39.67 秒 / 1.87× | 112.44 秒 / 1.85× | 1.0000 | 1.0000 |
| Cache-DiT 保守模式 | 28.02 秒 / 2.65× | 78.28 秒 / 2.65× | 0.8986 | 0.9179 |
| SubBlock 0.75 | 30.90 秒 / 2.41× | 77.12 秒 / 2.69× | 0.8006 | 0.8301 |
| SubBlock 0.75 + Cache-DiT 保守模式 | 21.41 秒 / 3.47× | 57.48 秒 / 3.61× | 0.7936 | 0.8288 |
| Cache-DiT 步进模式 | 18.13 秒 / 4.10× | 52.07 秒 / 3.99× | 0.8037 | 0.8078 |
| SubBlock 0.75 + Cache-DiT 步进模式 | 15.16 秒 / 4.90× | 38.21 秒 / 5.44× | 0.7713 | 0.7834 |
| SubBlock 0.80 | 29.49 秒 / 2.52× | 72.85 秒 / 2.85× | 0.7858 | 0.8193 |
| SubBlock 0.80 + Cache-DiT 步进模式 | 14.68 秒 / 5.06× | 36.29 秒 / 5.72× | 0.7584 | 0.7765 |
FL2VA
| 模式 | 5 秒中位数 / 加速比 | 10 秒中位数 / 加速比 | 5 秒平均 SSIM | 10 秒平均 SSIM |
|---|---|---|---|---|
| Diffusers | 80.44 秒 / 1.00× | 217.31 秒 / 1.00× | — | — |
| SGLang 无损模式 | 41.31 秒 / 1.95× | 114.02 秒 / 1.91× | 1.0000 | 1.0000 |
| Cache-DiT 保守模式 | 26.90 秒 / 2.99× | 78.24 秒 / 2.78× | 0.9389 | 0.9771 |
| SubBlock 0.75 | 31.27 秒 / 2.57× | 76.95 秒 / 2.82× | 0.8946 | 0.9385 |
| SubBlock 0.75 + Cache-DiT 保守模式 | 20.64 秒 / 3.90× | 56.39 秒 / 3.85× | 0.8924 | 0.9414 |
| SubBlock 0.75 + SageAttention | 30.64 秒 / 2.63× | 74.42 秒 / 2.92× | 0.8827 | 0.9219 |
| Cache-DiT 步进模式 | 18.02 秒 / 4.46× | 51.31 秒 / 4.24× | 0.8903 | 0.9248 |
| SubBlock 0.75 + Cache-DiT 步进模式 | 14.27 秒 / 5.64× | 36.62 秒 / 5.93× | 0.8629 | 0.9202 |
| SubBlock 0.80 | 29.74 秒 / 2.71× | 72.44 秒 / 3.00× | 0.8837 | 0.9350 |
| SubBlock 0.80 + Cache-DiT 步进模式 | 13.73 秒 / 5.86× | 34.80 秒 / 6.24× | 0.8498 | 0.9144 |
关键要点
- SGLang 的密集路径是最容易获得的提升。在保持工作负载不变的情况下，它在两个任务和两种时长下都比 Diffusers 实现了 1.85–1.95× 的加速。
- SubBlock 0.75 + Cache-DiT 步进模式是均衡的方案。它在 5 秒时长下实现 4.90–5.64× 加速、10 秒时长下实现 5.44–5.93× 加速的同时，保持了良好的输出质量。
- 步进缓存带来的吞吐量提升最大。它单独使用即可达到 3.99–4.46× 的加速，而保守模式只有 2.65–2.99×。
- FL2VA 从缓存 + 稀疏组合中获益略多。FL2VA 最快的组合达到 5.86×/6.24×，而 T2VA 为 5.06×/5.72×。
- 速度与质量的权衡关系很清晰。保守的 Cache-DiT 保留了 0.8986–0.9771 的 SSIM；激进的 0.80 + 步进组合则牺牲了部分质量余量，换取最低延迟。
加速来自哪里
三个机制驱动了这些方案层面的提升。
融合内核降低了仍在执行的每一步的成本。H3 路径将索引式 AdaLN 更新、门控残差、SwiGLU 激活以及带 3D RoPE 的 QK RMSNorm 融合在一起，减少了中间张量、内存流量和内核启动次数。下一节报告了这些独立的核心里程碑式测量结果；它们是逐步实现的一部分，而 Cache-DiT 和 SubBlock 则决定了该实现中有多少部分会被实际执行。
Cache-DiT 将一个 DBCache 上下文附加到 MiniMax-H3 的共享 DiT 块堆栈上。在预热步骤之后，它会评估配置的边界块，并将归一化残差变化与之前的缓存状态进行比较。如果变化保持在阈值以下，并且连续缓存限制允许，则中间块会复用其缓存结果；否则，堆栈会被重新计算并刷新缓存。所有缓存模式均使用 Fn=1、Bn=0 和四个预热步骤：
- 保守模式：共享打包堆栈 RDT 0.04，最大连续缓存步数 1；
- 步进模式：共享打包堆栈 RDT 0.08，最大连续缓存步数 3。
MiniMax-H3 有一个 MiniMaxH3DiTModel，其块堆栈承载打包的视频和音频 token。因此，Cache-DiT 对整个打包堆栈做出一个共享决策；它不会维护独立的视频和音频缓存。工作进程记录一个合并的 Cache-DiT 步骤列表，跟踪图例遵循该执行模型。
SubBlock 稀疏注意力减少了在计算步骤中读取的 KV 块数量。它使用 n_k=n_q=4；前十个去噪步骤使用密集注意力，之后启用 SubBlock。最小序列长度为 4096。矩阵测试了 0.75 和 0.80 的稀疏度；后者更快，但在多个 T2VA 案例中 SSIM 较低。
聚合性能分析结果展示了这些性能分析端到端的行为方式；它们并未隔离内核时间，也未提供逐步成本分解。下面的跟踪是请求级别的执行跟踪，而非算子计时测量。
一个实测的 49 步跟踪
该工作负载配置了 50 个推理步骤。由于 sigma 调度包含两个区间端点，去噪循环执行 49 次模型评估（len(sigmas) - 1）；“49 步跟踪”即指这些模型评估。
为了让执行模式更具体，我们对一个 5 秒的 T2VA 请求运行了六种配置：无损、Cache-DiT 保守模式、SubBlock 0.75、SubBlock 0.75 + 保守 Cache-DiT、Cache-DiT 步进模式，以及 SubBlock 0.80 + 步进模式。工作节点记录了每个请求的实际 cached_steps 列表。由于视频和音频 token 共享一个打包的 H3 块堆栈，缓存命中会复用组合后的输出；在此路径中不存在独立的“视频已缓存、音频已计算”状态。SubBlock 行中的蓝色单元格标记了在前十步去噪步骤之后使用稀疏注意力进行计算的计算步骤。
跟踪运行耗时分别为：无损 37.78 秒、Cache-DiT 保守模式 26.82 秒、SubBlock 0.75 为 29.97 秒、SubBlock 0.75 + 保守 Cache-DiT 为 22.18 秒、Cache-DiT 步进模式 17.23 秒，以及 SubBlock 0.80 + 步进模式 14.34 秒。这些数字标识的是跟踪运行本身；它们不能替代三提示词聚合中位数。
内核层
缓存决定了执行多少步去噪步骤；内核则决定了每一步计算的速度。MiniMax-H3 将视频和音频 token 打包到一个序列中，因此非 GEMM 路径在整个过程中都受益于同一个基本原则：更少的内存流量、更少的中间张量以及更少的内核启动次数。AdaLN 调制和门控残差通过 token 索引查找参数，并在一次遍历中更新激活值。SwiGLU 直接对融合后的 gate_up 缓冲区进行操作。QK RMSNorm 和 3D RoPE 被融合到单个内核中，而不是作为独立的即时操作运行。
下表使用了 5 秒 T2VA 请求在 1344×768×124 帧下的真实每秩形状：SP/Ulysses-8 填充后为 4,722 行，隐藏大小为 5,376，56 个注意力头，头维度为 128，RoPE 维度为 96，输入为 BF16。每个数字是 10 轮、每轮 20 次调用的每次调用 CUDA 事件时间的中位数。基线是相应的即时组合。
| 算子 | 即时组合 | SGLang 内核 | 加速比 |
|---|---|---|---|
| AdaLN 调制（索引缩放平移） | 136.7 微秒 | 38.2 微秒 | 3.58× |
| AdaLN 门控残差（索引） | 93.2 微秒 | 46.6 微秒 | 2.00× |
| SwiGLU 激活（原地） | 364.5 微秒 | 105.2 微秒 | 3.46× |
| QK RMSNorm | 334.0 微秒 | 76.9 微秒 | 4.35× |
| QK RMSNorm + 3D RoPE，单内核 | 1335.6 微秒 | 109.8 微秒 | 12.16× |
这些是针对各独立环节的微基准测试结果，并非可叠加的端到端延迟节省。融合后的 QK-Norm + RoPE 结果使用了 main 分支上可用的精确舍入路径（round_norm_before_rope=True）。
SubBlock 稀疏注意力如何工作
SubBlock 是一种免训练的块稀疏注意力路由机制。它将序列划分为 64 个 token 的查询块和键块，然后在两侧将每个块进一步拆分为四个 16 token 的子块（n_q=n_k=4）。一个轻量级的池化与 log-sum-exp 评分机制，为每个查询块和注意力头估算每个键块未归一化的 softmax 质量。路由机制保留得分最高的键块，并将其索引传递给块稀疏注意力内核；完整的注意力矩阵永远不会被显式构建。
稀疏度数值表示允许被丢弃的键块比例，而非保留的比例。因此，sparsity=0.75 意味着每个查询块大约保留 25% 的键块。更激进的 0.80 设置速度更快，但近似误差预算也更大，这与最激进配置行中观察到的较低 SSIM 值是一致的。
下图展示了得分分布；竖线表示所展示的两种预算下，按行路由截断值的中位数。此处，sparsity=0.50 作为诊断参考被纳入；基准测试配置使用的是 0.75 和 0.80。由于路由机制对每个查询块和注意力头独立地对键块进行排序，sparsity=0.50 和 0.75 分别大致保留该行可用键块的前一半和前四分之一，并受 8 块预算取整的约束。在这些工作负载中，0.75 的预算保留了行内局部中位数以上的大部分得分质量，同时将选择集中在高得分尾部。
稀疏路径仅对内核支持的长序列、非因果 DiT 注意力调用启用：BF16 输入、头维度为 128，且序列长度至少为 4096 个 token。前十个去噪步骤使用稠密注意力；短序列段、token 精炼器以及不支持的调用则使用稠密回退路径。在 H200/SM90 上，所选的 64×64 路由方案由 SGLang 的 CuTe 块稀疏 FlashAttention 内核执行。
演示
演示集为每个选定的提示词包含四种模式：
- 提示词 1 · T2VA · 5 秒 · 三只猫拿着铜管乐器，在一个熟睡的主人身边演奏；
- 提示词 2 · T2VA · 10 秒 · 夜晚雨中的赛博朋克城市；
- 提示词 3 · FL2VA · 5 秒 · 黏土狐狸的延续镜头。
四种模式分别是 SGLang 无损模式、Cache-DiT 保守模式、SubBlock 0.75 + Cache-DiT 步进模式，以及 SubBlock 0.80 + Cache-DiT 步进模式。文件名编码了提示词、任务、模式和时长；SVG 图表位于同一文件夹中。
提示词 1 · T2VA · 5 秒
SGLang 无损模式
Cache-DiT 保守模式
SubBlock 0.75 + Cache-DiT 步进模式
SubBlock 0.80 + Cache-DiT 步进模式
提示词 2 · T2VA · 10 秒
SGLang 无损模式
Cache-DiT 保守模式
SubBlock 0.75 + Cache-DiT 步进模式
SubBlock 0.80 + Cache-DiT 步进模式
提示词 3 · FL2VA · 5 秒
SGLang 无损模式
Cache-DiT 保守模式
SubBlock 0.75 + Cache-DiT 步进模式
SubBlock 0.80 + Cache-DiT 步进模式
提示词 1 · 完整提示词
integrated_multimodal_description: [Shot 1] Live-action, whimsical cinematic, a medium-wide shot frames a dim bedroom at night where the owner sleeps under the covers. A bedroom door opens and three cats enter in single file, each carrying a tiny brass instrument. The camera tracks sideways with small amplitude at slow speed as the cats march beside the bed and play a short, lively diegetic brass tune in synchrony; the sleeping owner shifts slightly but does not wake. The cats finish with one crisp flourish, pivot together, and abruptly file back out through the doorway, with the last cat's tail disappearing from frame. No character speaks and no human voice is heard.
overall_soundscape: Quiet nighttime room tone, the owner's steady breathing, soft pawsteps on the floor, a faint door creak, and light bedding rustle as the procession passes.
non_diegetic_music: N/A
提示词 2 · 完整提示词
integrated_multimodal_description: [Shot 1] Live-action, cinematic, a wide establishing shot frames a futuristic cyberpunk city at night as rain falls across dense towers, elevated transit lines, and a crowded street lined with vivid neon light. The camera pushes forward with small amplitude at slow speed above the wet pavement while pedestrians in reflective coats pass beneath transparent umbrellas, a compact hovering vehicle glides through the intersection, and saturated magenta, cyan, and amber reflections ripple across puddles. Steam drifts from a street vent and briefly catches the neon glow as the vehicle recedes between the towers. No dialogue or voiceover is heard.
overall_soundscape: Steady rainfall, distant traffic, the low hum of elevated transit, electrical buzzing from signs, soft footsteps through shallow water, and a brief rush of air as the hovering vehicle passes.
non_diegetic_music: A slow electronic pulse with deep analog bass, sparse metallic percussion, and sustained synthesizer tones that gradually increase in volume before fading.
提示词 3 · 完整提示词
For the target video, at 0.00 seconds into the target video, <Picture 1> is fully referenced.
integrated_multimodal_description:
[Shot 1] A handcrafted stop-motion clay animation begins from <Picture 1>. A small orange clay fox with large expressive eyes trots along a mossy path through a warm, richly detailed miniature forest. The camera tracks the fox smoothly at eye level while layered clay trees and shrubs create gentle parallax. The fox looks curiously toward the camera, slows near the middle of the path, flicks its tail, then continues toward the small wooden cabin in the distance. Preserve the exact clay textures, warm amber lighting, forest layout, fox proportions, and family-friendly whimsical tone established by <Picture 1>. Motion remains coherent and physically plausible for stop-motion animation.
overall_soundscape:
Soft clay footsteps, rustling leaves, distant birds, and a light forest breeze accompany the fox's movement.
non_diegetic_music:
A gentle playful score with pizzicato strings, wooden percussion, and soft flute.
致谢
本基准测试是多个团队共同努力的成果，我们对此深表感谢。
- SGLang Diffusion 团队——撰写了本篇文章的初稿，推动了这些成果所依托的 SGLang 内核开发工作，并提供了本文所测的扩散模型运行时、融合内核及并行方案。
- 黄吉（@IPostYellow），蚂蚁集团——运行了 H200 基准测试，将 SubBlock 稀疏注意力引入 SGLang Diffusion，并修订了本文。
- Cache-DiT 团队——@DefTruth 和 vipshop.com 团队，感谢他们开发了 Cache-DiT，并支持将其缓存配置集成到 SGLang Diffusion 中。
- MiniMax——感谢开源 MiniMax-H3，本文所有测量均在该模型上运行。
- NVIDIA——感谢其提供的底层 SubBlock 稀疏注意力支持，包括本文结果所依赖的块稀疏注意力前向计算。
测量日期为 2026-08-18，环境为 8× NVIDIA H200。复现细节及每条提示词的原始数据见基准测试仓库。
TL;DR
We benchmarked MiniMax-H3 video generation on 8× NVIDIA H200 with SGLang Diffusion, holding prompts, seeds, resolution, frame rate, and denoising steps fixed across six workloads.
- SGLang's dense, lossless path is 1.85–1.95× faster than Diffusers with no approximation: the same denoising work, on a faster runtime.
- Stacking step reuse and sparse attention reaches up to 6.24×, at 0.76–0.91 mean SSIM. The fastest tested profile, SubBlock 0.80 + Cache-DiT stride, delivers 5.06×/5.72× on 5 s/10 s T2VA and 5.86×/6.24× on FL2VA. The cost is not uniform: FL2VA holds 0.85–0.91 SSIM there, while T2VA drops to 0.76–0.78.
- For a quality-first default, use Cache-DiT alone (up to 2.99×, mean SSIM 0.90–0.92). For a balanced trade-off, SubBlock 0.75 + Cache-DiT stride gives 4.90–5.93× at SSIM 0.79–0.90.
- The gains come from three layers that compose: fused kernels (2.00–12.16× on individual non-GEMM sites in isolation — microbenchmarks, not additive end-to-end), step reuse (Cache-DiT skips redundant denoising steps), and SubBlock sparse attention (NVIDIA's block-sparse forward, which cuts the cost of the steps that do run).
Scope. This comparison covers three of SGLang Diffusion's acceleration knobs. It supports more lossy paths that are not part of this run — quantization and progressive resolution among them — so the numbers here are a slice of the available envelope, not its ceiling. Everything below is measured, not projected; the clips at the end let you judge the quality cost yourself.
| Hardware | 8× NVIDIA H200 (141 GB) |
| Workload | MiniMax-H3 · 1344×768 · 24 FPS · 50 denoising steps · 5 s and 10 s outputs |
| Parallelism | All modes use 8 GPUs; Diffusers uses CP8, and SGLang uses SP/Ulysses degree 8 |
| Version | SGLang v0.5.18 (d90318b3e2) |
| Measured | 2026-08-18 |
Background
While SGLang Diffusion already delivers a fast lossless path for MiniMax-H3, faster lossy generation of high-quality video has long been sought after by the community. Building on its long-standing stack of versatile knobs for lossy acceleration, SGLang Diffusion has been actively working on this over the past weeks; this post is the first measured account of where those knobs land.
Video diffusion is dominated by two costs: the denoising loop runs the same transformer dozens of times, and each step spends most of its budget in attention over a very long token sequence. A 5-second 1344×768 clip at 24 FPS with 50 denoising steps is far past the point where a single GPU is practical, so the question is not whether to parallelize but how much of the remaining work can be avoided.
Three accelerations attack that from different directions, and they compose:
- Fused kernels cut the fixed cost of every step without changing its math.
- Cache-DiT reuses results between denoising steps, so some steps never run.
- SubBlock sparse attention reduces the cost of the steps that do run, by skipping attention blocks whose contribution is below a threshold.
The first is lossless. The other two trade similarity against speed, which is why every number in this post is reported with SSIM against the lossless baseline.
At a Glance
The answer depends on the baseline. Against the matched Diffusers case, SGLang's dense, lossless path is already about 2× faster for both tasks and both durations. Cache-DiT reuses work between denoising steps, while SubBlock sparse attention reduces the cost of the steps that still run. Together they form the fastest path in this matrix.
For a quality-first accelerated default, use Cache-DiT conservative or Cache-DiT stride without SubBlock. For a balanced speed/quality trade-off, use SubBlock 0.75 + Cache-DiT stride, delivering 4.90–5.64× speedup at 5 s and 5.44–5.93× at 10 s.
The charts below summarize the aggregate benchmark tables.
Detailed Results
Every configuration in this post is reproducible with the SGLang cookbook page for MiniMax-H3, which carries the exact launch flags for each mode.
We report generation-side inference time; server startup, warmup, HTTP polling, and MP4 download time are excluded. For each task and duration, latency and SSIM are evaluated across three distinct prompts. Speedup is measured against the matching Diffusers case. SSIM is computed over all frames in YUV420 against the matching SGLang lossless video.
T2VA
| Mode | 5 s median / speedup | 10 s median / speedup | 5 s mean SSIM | 10 s mean SSIM |
|---|---|---|---|---|
| Diffusers | 74.34 s / 1.00× | 207.71 s / 1.00× | — | — |
| SGLang lossless | 39.67 s / 1.87× | 112.44 s / 1.85× | 1.0000 | 1.0000 |
| Cache-DiT conservative | 28.02 s / 2.65× | 78.28 s / 2.65× | 0.8986 | 0.9179 |
| SubBlock 0.75 | 30.90 s / 2.41× | 77.12 s / 2.69× | 0.8006 | 0.8301 |
| SubBlock 0.75 + Cache-DiT conservative | 21.41 s / 3.47× | 57.48 s / 3.61× | 0.7936 | 0.8288 |
| Cache-DiT stride | 18.13 s / 4.10× | 52.07 s / 3.99× | 0.8037 | 0.8078 |
| SubBlock 0.75 + Cache-DiT stride | 15.16 s / 4.90× | 38.21 s / 5.44× | 0.7713 | 0.7834 |
| SubBlock 0.80 | 29.49 s / 2.52× | 72.85 s / 2.85× | 0.7858 | 0.8193 |
| SubBlock 0.80 + Cache-DiT stride | 14.68 s / 5.06× | 36.29 s / 5.72× | 0.7584 | 0.7765 |
FL2VA
| Mode | 5 s median / speedup | 10 s median / speedup | 5 s mean SSIM | 10 s mean SSIM |
|---|---|---|---|---|
| Diffusers | 80.44 s / 1.00× | 217.31 s / 1.00× | — | — |
| SGLang lossless | 41.31 s / 1.95× | 114.02 s / 1.91× | 1.0000 | 1.0000 |
| Cache-DiT conservative | 26.90 s / 2.99× | 78.24 s / 2.78× | 0.9389 | 0.9771 |
| SubBlock 0.75 | 31.27 s / 2.57× | 76.95 s / 2.82× | 0.8946 | 0.9385 |
| SubBlock 0.75 + Cache-DiT conservative | 20.64 s / 3.90× | 56.39 s / 3.85× | 0.8924 | 0.9414 |
| SubBlock 0.75 + SageAttention | 30.64 s / 2.63× | 74.42 s / 2.92× | 0.8827 | 0.9219 |
| Cache-DiT stride | 18.02 s / 4.46× | 51.31 s / 4.24× | 0.8903 | 0.9248 |
| SubBlock 0.75 + Cache-DiT stride | 14.27 s / 5.64× | 36.62 s / 5.93× | 0.8629 | 0.9202 |
| SubBlock 0.80 | 29.74 s / 2.71× | 72.44 s / 3.00× | 0.8837 | 0.9350 |
| SubBlock 0.80 + Cache-DiT stride | 13.73 s / 5.86× | 34.80 s / 6.24× | 0.8498 | 0.9144 |
Key takeaways
- SGLang's dense path is the first easy win. It delivers a 1.85–1.95× speedup over Diffusers across both tasks and durations, with the workload held constant.
- SubBlock 0.75 + Cache-DiT stride is the balanced profile. It maintains good output quality while delivering 4.90–5.64× speedup at 5 seconds and 5.44–5.93× at 10 seconds.
- Stride caching adds the largest throughput gain. It reaches 3.99–4.46× on its own, compared with 2.65–2.99× for the conservative profile.
- FL2VA benefits slightly more from cache + sparse combinations. The fastest FL2VA case reaches 5.86×/6.24×, versus 5.06×/5.72× for T2VA.
- The speed–quality trade-off is clear. Conservative Cache-DiT retains 0.8986–0.9771 SSIM; the aggressive 0.80 + stride profile gives up some of that margin for the lowest latency.
Where the Speedup Comes From
Three mechanisms drive the profile-level gains.
Fused kernels reduce the cost of each step that still runs. The H3 path fuses indexed AdaLN updates, gated residuals, SwiGLU activation, and QK RMSNorm with 3D RoPE, reducing intermediate tensors, memory traffic, and kernel launches. The next section reports these isolated kernel measurements; they are part of the per-step implementation, while Cache-DiT and SubBlock determine how much of that implementation is executed.
Cache-DiT attaches one DBCache context to MiniMax-H3's shared DiT block stack. After the warmup steps, it evaluates the configured boundary blocks and compares the normalized residual change with the previous cached state. If the change stays below the threshold and the consecutive-cache limit allows it, the middle blocks reuse their cached result; otherwise the stack is recomputed and the cache is refreshed. All cache modes use Fn=1, Bn=0, and four warmup steps:
- conservative: shared packed-stack RDT
0.04, maximum consecutive cached steps1; - stride: shared packed-stack RDT
0.08, maximum consecutive cached steps3.
MiniMax-H3 has one MiniMaxH3DiTModel whose block stack carries packed video and audio tokens. Cache-DiT therefore makes one shared decision for the whole packed stack; it does not maintain independent video and audio caches. The worker records one combined Cache-DiT step list, and the trace legend follows that execution model.
SubBlock sparse attention reduces the KV blocks read on computed steps. It uses n_k=n_q=4; the first ten denoising steps use dense attention, and SubBlock is enabled afterward. The minimum sequence length is 4096. The matrix tests sparsity 0.75 and 0.80; the latter is faster but has lower SSIM on several T2VA cases.
The aggregate profile results show how the profiles behave end to end; they do not isolate kernel time or provide a per-step cost breakdown. The trace below is a request-level execution trace, not an operator timing measurement.
One measured 49-step trace
The workload is configured with 50 inference steps. Because the sigma schedule includes both interval endpoints, the denoising loop performs 49 model evaluations (len(sigmas) - 1); “49-step trace” refers to these model evaluations.
To make the execution pattern concrete, one 5-second T2VA request was run for six profiles: lossless, Cache-DiT conservative, SubBlock 0.75, SubBlock 0.75 + conservative Cache-DiT, Cache-DiT stride, and SubBlock 0.80 + stride. The worker recorded the actual cached_steps list for each request. Because video and audio tokens share one packed H3 block stack, a cache hit reuses the combined output; there is no separate “video cached, audio computed” state in this path. Blue cells in the SubBlock rows mark computed steps that use sparse attention after the first ten denoising steps.
The trace-run timings are 37.78 s (lossless), 26.82 s (Cache-DiT conservative), 29.97 s (SubBlock 0.75), 22.18 s (SubBlock 0.75 + conservative Cache-DiT), 17.23 s (Cache-DiT stride), and 14.34 s (SubBlock 0.80 + stride). These numbers identify the trace run; they do not replace the three-prompt aggregate medians.
The Kernel Layer
Caching determines how many denoising steps run; kernels determine how fast each computed step is. MiniMax-H3 packs video and audio tokens into one sequence, so the non-GEMM path benefits from the same basic principle throughout: less memory traffic, fewer intermediate tensors, and fewer kernel launches. AdaLN modulation and gated residuals look up parameters by token index and update the activation in one pass. SwiGLU operates directly on the fused gate_up buffer. QK RMSNorm and 3D RoPE are fused into a single kernel instead of running as separate eager operations.
The table below uses the real per-rank shape for a 5-second T2VA request at 1344×768×124 frames: 4,722 rows after SP/Ulysses-8 padding, hidden size 5,376, 56 attention heads, head dimension 128, RoPE dimension 96, and BF16 inputs. Each number is the median per-call CUDA-event time across 10 rounds of 20 calls. The baseline is the corresponding eager composition.
| Operator | Eager composition | SGLang kernel | Speedup |
|---|---|---|---|
| AdaLN modulation (indexed scale-shift) | 136.7 μs | 38.2 μs | 3.58× |
| AdaLN gated residual (indexed) | 93.2 μs | 46.6 μs | 2.00× |
| SwiGLU activation (in place) | 364.5 μs | 105.2 μs | 3.46× |
| QK RMSNorm | 334.0 μs | 76.9 μs | 4.35× |
| QK RMSNorm + 3D RoPE, one kernel | 1335.6 μs | 109.8 μs | 12.16× |
These are microbenchmarks of the isolated sites, not additive end-to-end latency savings. The fused QK-Norm + RoPE result uses the exact-rounding path available on main (round_norm_before_rope=True).
How SubBlock Sparse Attention Works
SubBlock is a training-free router for block-sparse attention. It divides the sequence into 64-token query and key blocks, then splits each block into four 16-token sub-blocks on both sides (n_q=n_k=4). A lightweight pooling and log-sum-exp score estimates each key block's unnormalized softmax mass for each query block and head. The router keeps the highest-scoring key blocks and passes their indices to the block-sparse attention kernel; the full attention matrix is never materialized.
The sparsity value is the fraction of key blocks allowed to be dropped, not the fraction retained. Thus sparsity=0.75 keeps roughly 25% of key blocks per query block. The more aggressive 0.80 setting is faster but has a larger approximation error budget, which is consistent with the lower SSIM observed in the most aggressive rows.
The curves below show the score distributions; the vertical lines show the medians of the per-row routing cutoffs for the two displayed budgets. Here, sparsity=0.50 is included as a diagnostic reference; the benchmark profiles use 0.75 and 0.80. Because the router ranks key blocks independently for each query block and head, sparsity=0.50 and 0.75 retain roughly the top half and top quarter of that row's available key blocks, subject to 8-block budget rounding. Across these workloads, the 0.75 budget retains most of the score mass above the row-local median while concentrating selection on the high-score tail.
The sparse path is enabled only for the long, non-causal DiT attention calls that the kernel supports: BF16 inputs, head dimension 128, and sequences of at least 4096 tokens. The first ten denoising steps use dense attention; short segments, the token refiner, and unsupported calls use the dense fallback. On H200/SM90, the selected 64×64 routing plan is executed by SGLang's CuTe block-sparse FlashAttention kernel.
Demos
The demo set contains four modes for each selected prompt:
- Prompt 1 · T2VA · 5 s · three cats carrying brass instruments and playing beside a sleeping owner;
- Prompt 2 · T2VA · 10 s · a rainy cyberpunk city at night;
- Prompt 3 · FL2VA · 5 s · a clay fox continuation.
The four modes are SGLang lossless, Cache-DiT conservative, SubBlock 0.75 + Cache-DiT stride, and SubBlock 0.80 + Cache-DiT stride. Filenames encode the prompt, task, mode, and duration; the SVG figures are in the same folder.
Prompt 1 · T2VA · 5 s
SGLang lossless
Cache-DiT conservative
SubBlock 0.75 + Cache-DiT stride
SubBlock 0.80 + Cache-DiT stride
Prompt 2 · T2VA · 10 s
SGLang lossless
Cache-DiT conservative
SubBlock 0.75 + Cache-DiT stride
SubBlock 0.80 + Cache-DiT stride
Prompt 3 · FL2VA · 5 s
SGLang lossless
Cache-DiT conservative
SubBlock 0.75 + Cache-DiT stride
SubBlock 0.80 + Cache-DiT stride
Prompt 1 · full prompt
integrated_multimodal_description: [Shot 1] Live-action, whimsical cinematic, a medium-wide shot frames a dim bedroom at night where the owner sleeps under the covers. A bedroom door opens and three cats enter in single file, each carrying a tiny brass instrument. The camera tracks sideways with small amplitude at slow speed as the cats march beside the bed and play a short, lively diegetic brass tune in synchrony; the sleeping owner shifts slightly but does not wake. The cats finish with one crisp flourish, pivot together, and abruptly file back out through the doorway, with the last cat's tail disappearing from frame. No character speaks and no human voice is heard.
overall_soundscape: Quiet nighttime room tone, the owner's steady breathing, soft pawsteps on the floor, a faint door creak, and light bedding rustle as the procession passes.
non_diegetic_music: N/A
Prompt 2 · full prompt
integrated_multimodal_description: [Shot 1] Live-action, cinematic, a wide establishing shot frames a futuristic cyberpunk city at night as rain falls across dense towers, elevated transit lines, and a crowded street lined with vivid neon light. The camera pushes forward with small amplitude at slow speed above the wet pavement while pedestrians in reflective coats pass beneath transparent umbrellas, a compact hovering vehicle glides through the intersection, and saturated magenta, cyan, and amber reflections ripple across puddles. Steam drifts from a street vent and briefly catches the neon glow as the vehicle recedes between the towers. No dialogue or voiceover is heard.
overall_soundscape: Steady rainfall, distant traffic, the low hum of elevated transit, electrical buzzing from signs, soft footsteps through shallow water, and a brief rush of air as the hovering vehicle passes.
non_diegetic_music: A slow electronic pulse with deep analog bass, sparse metallic percussion, and sustained synthesizer tones that gradually increase in volume before fading.
Prompt 3 · full prompt
For the target video, at 0.00 seconds into the target video, <Picture 1> is fully referenced.
integrated_multimodal_description:
[Shot 1] A handcrafted stop-motion clay animation begins from <Picture 1>. A small orange clay fox with large expressive eyes trots along a mossy path through a warm, richly detailed miniature forest. The camera tracks the fox smoothly at eye level while layered clay trees and shrubs create gentle parallax. The fox looks curiously toward the camera, slows near the middle of the path, flicks its tail, then continues toward the small wooden cabin in the distance. Preserve the exact clay textures, warm amber lighting, forest layout, fox proportions, and family-friendly whimsical tone established by <Picture 1>. Motion remains coherent and physically plausible for stop-motion animation.
overall_soundscape:
Soft clay footsteps, rustling leaves, distant birds, and a light forest breeze accompany the fox's movement.
non_diegetic_music:
A gentle playful score with pizzicato strings, wooden percussion, and soft flute.
Acknowledgement
This benchmark is the result of work by several teams, and we are grateful to all of them.
- SGLang Diffusion Team — wrote the first version of this post, drives the SGLang kernel work these results build on, and provides the diffusion runtime, the fused kernels, and the parallelism measured here.
- Ji Huang (@IPostYellow), Ant Group — ran the H200 benchmark, brought SubBlock sparse attention into SGLang Diffusion, and revised this post.
- Cache-DiT Team — @DefTruth and the vipshop.com team, for Cache-DiT and for support integrating its cache profiles into SGLang Diffusion.
- MiniMax — for open-sourcing MiniMax-H3, the model every measurement here runs on.
- NVIDIA — for the underlying SubBlock sparse attention support, including the block-sparse attention forward these results depend on.
Measured 2026-08-18 on 8× NVIDIA H200. Reproduction details and the raw per-prompt numbers are in the benchmark repository.