今天，我们发布了针对 LFM2.5 系列中三款模型的 DSpark 草稿模型检查点：LFM2.5-1.2B-Instruct、LFM2.5-2.6B 和 LFM2.5-8B-A1B。这些模型增加了一条投机解码路径，以极小的内存开销换取大幅度的解码速度提升，且不改变输出质量：
- 更快的推理：在 GPU 上吞吐量最高提升 3.18 倍，在端侧设备上最高提升 2.87 倍。
- 迈向端侧智能体推理：LFM2.5-2.6B 的函数调用延迟平均降低 57%。
- 首日支持 llama.cpp 和 SGLang：兼容 LFM 的 DSpark 集成已在上游开源。
DSpark 是如何工作的
大语言模型推理中的解码阶段传统上受内存带宽限制。大部分延迟来自将权重从 DRAM 流式加载到 SRAM，而非密集计算。投机解码通过使用轻量级草稿模型生成候选 token，然后由目标模型在单次前向传播中一次性验证所有候选 token 来解决这一问题，从而将加载权重的成本分摊到我们验证的所有 token 上。
多年来，业界提出了多种投机方法，其中最著名的是 EAGLE-3、DFlash，以及最新的 DSpark，后者结合了三个组件：
- 基于 DFlash 风格的并行主干网络，以目标模型的上下文特征为条件，在单次前向传播中为所有草稿 token 生成隐藏状态。
- 一个轻量级的顺序头，以相邻 token 之间的马尔可夫链为模型，增加了 token 间的依赖关系，从而提高了后续位置的接受率。
- 一个基于置信度调度的验证器，用于预测每个 token 的存活概率，并在验证成本高于其节省成本时，剪除低置信度的后缀。
训练与架构
我们遵循 DSpark 的方法，但使用了更大、更多样化的数据混合，涵盖 SFT、聊天、代码和函数调用数据。根据我们的消融实验，草稿模型的初始版本是简化的仅注意力（attention-only）草稿模型，包含 5 层和 9 个块。对于每个草稿模型，我们在整个数据集上运行了 15 个 epoch，并选择了接受率最高而非损失最低的那个 epoch。
由此产生的草稿模型相对较小，每个模型约有 ~3 亿参数。
| 组件 | LFM2.5-1.2B-Instruct | LFM2.5-8B-A1B | LFM2.5-2.6B |
|---|---|---|---|
| 解码器堆栈（5 层） | 241.2M | 241.2M | 241.2M |
| 隐藏状态投影 | 21.0M | 21.0M | 21.0M |
| Markov 头 | 33.6M | 65.5M | 65.5M |
| 归一化层 + 置信度头 | 27.5k | 27.5k | 27.5k |
| 总计 | 295.7M | 327.7M | 327.7M |
质量对齐
在贪心解码下，草稿 token 只有在与目标模型的分布匹配时才会被接受。被拒绝时，目标模型自身的 token 会取而代之。因此，生成的序列在构造上与基线贪心解码完全一致，所以基准测试的准确率（pass@1 或精确匹配）保持不变。
CPU 和 GPU 上的推理加速
我们为 LFM2.5 推出的 DSpark 草稿模型在发布首日即支持 llama.cpp（实现基于官方代码库构建，我们使用实验性的 Metal 内核运行）和 **SGLang**（实现基于官方 SGLang 对 DSpark 的实现构建）。
我们使用 llama.cpp 和 Metal 在 M4 Max MacBook Pro 上测量端侧吞吐量，采用 FP16 GGUF 权重，最多生成 256 个输出 token。我们使用 SGLang 在单块 H100 80 GB 上以 BF16 测量 GPU 吞吐量。两种配置均使用 DSpark 块大小 9、批大小 1 和温度 0。我们在五个基准数据集上进行了评估。
三个草稿模型在大型加速器（H100）和边缘部署（M4 Max MacBook）上都带来了显著的吞吐量提升。
对于 LFM2.5-2.6B，MacBook 上的加速尤为明显，它将用户可享受的交互性水平推向了远超大多数专有云模型（约 ~140 tok/s，视数据集而定）的吞吐量。
| 数据集 | 接受率（满分 10） | H100 上的加速 | M4 Max 上的加速 |
|---|---|---|---|
| MATH500 | 5.42 | 3.06 倍 326 → 1000 tok/s | 2.25 倍 61 → 137 tok/s |
| HumanEval | 4.54 | 2.56 倍 326 → 835 tok/s | 2.63 倍 61 → 161 tok/s |
| MBPP | 4.71 | 2.64 倍 326 → 861 tok/s | 2.11 倍 62 → 132 tok/s |
| GSM8K | 4.32 | 2.22 倍 312 → 693 tok/s | 2.36 倍 60 → 143 tok/s |
| MT-Bench | 5.07 | 2.87 倍 325 → 933 tok/s | 1.99 倍 62 → 123 tok/s |
| 平均值 | 4.81 | 2.67 倍 323 → 864 tok/s | 2.27 倍 61 → 139 tok/s |
在各种多工具场景中，DSpark 为 LFM2.5-2.6B 平均降低了 57% 的延迟。
对于 LFM2.5-1.2B-Instruct，我们看到数据集接受率的波动要大得多，因此加速效果会因底层文本分布的不同而出现高达 52% 的差异。
| 数据集 | 接受率（满分 10） | H100 上的加速 | M4 Max 上的加速 |
|---|---|---|---|
| MATH500 | 6.02 | 2.56倍 668 → 1712 token/秒 | 2.62倍 140 → 366 token/秒 |
| HumanEval | 5.31 | 2.26倍 664 → 1499 token/秒 | 2.87倍 136 → 389 token/秒 |
| MBPP | 5.52 | 2.37倍 667 → 1578 token/秒 | 2.74倍 137 → 375 token/秒 |
| GSM8K | 4.34 | 1.67倍 624 → 1041 token/秒 | 2.73倍 140 → 381 token/秒 |
| MT-Bench | 3.90 | 1.66倍 657 → 1091 token/秒 | 1.72倍 137 → 237 token/秒 |
| 平均值 | 5.02 | 2.10倍 656 → 1384 token/秒 | 2.54倍 138 → 350 token/秒 |
对于 LFM2.5-8B-A1B，其接受率相比两个稠密模型有所提升，但在端侧设备上我们平均仅获得 18% 的改进。这一差距源于 llama.cpp 的 Metal 后端中当前 MoE 实现的限制，以及验证 k 个 token 会比单次解码步骤激活更多专家，从而产生更大的权重流量。
| 数据集 | 接受率（满分 10） | H100 上的加速比 | M4 Max 上的加速比 |
|---|---|---|---|
| MATH500 | 8.27 | 3.18倍 428 → 1362 token/秒 | 1.21倍 93 → 112 token/秒 |
| HumanEval | 7.02 | 2.58倍 426 → 1100 token/秒 | 1.12倍 91 → 101 token/秒 |
| MBPP | 6.93 | 2.64倍 426 → 1122 token/秒 | 1.09倍 89 → 97 token/秒 |
| GSM8K | 4.02 | 1.29倍 385 → 496 token/秒 | 1.44倍 90 → 129 token/秒 |
| MT-Bench | 8.52 | 3.02倍 426 → 1288 token/秒 | 1.04倍 87 → 90 token/秒 |
| 平均值 | 6.95 | 2.54倍 418 → 1074 token/秒 | 1.18倍 90 → 106 token/秒 |
如何使用 LFM2.5-DSpark
使用 SGLang 运行 DSpark 草稿模型，需要构建支持 LFM2 目标 DSpark 功能的 SGLang 版本（PR #31041）。启动目标模型并附加草稿模型：
python -m sglang.launch_server \
--model-path LiquidAI/LFM2.5-2.6B \
--speculative-algorithm DSPARK \
--speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \
--speculative-draft-attention-backend flashinfer \
--disable-radix-cache --mem-fraction-static 0.75 --port 30000
然后查询位于 http://localhost:30000/v1 的 OpenAI 兼容端点。块大小从草稿模型的 config.json 中读取；基线是去掉三个 --speculative-* 标志后的相同命令。
使用 llama.cpp 运行它们需要相应的 llama.cpp 构建版本（PR#27383）。
llama-server -m LFM2.5-2.6B-F16.gguf \
-md LFM2.5-2.6B-DSpark-F16.gguf \
--spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \
-fa on -ngl 99
块大小从 sidecar 元数据中读取（n-max 会被限制在该值内）。投机解码是精确的：目标模型会验证每一个提议的 token，因此贪婪解码的输出与单独运行目标模型完全一致；每次响应的计时报告会显示 draft_n / draft_n_accepted。
DSpark 草稿模型检查点已在 Hugging Face 上以 Safetensors 和 GGUF 格式提供：
- Safetensors：LFM2.5-2.6B-DSpark、LFM2.5-1.2B-Instruct-DSpark 和 LFM2.5-8B-A1B-DSpark
- GGUF：LFM2.5-2.6B-DSpark-GGUF、LFM2.5-1.2B-Instruct-DSpark-GGUF、LFM2.5-8B-A1B-DSpark-GGUF
我们迫不及待想看到你的成果。
引用
如需引用，请使用以下参考文献或 BibTeX：
Liquid AI 发布“LFM2.5-DSpark：从 H100 到 MacBook，推理速度最高提升 3.2 倍”，Liquid AI 博客，2026 年 8 月。
@article{liquidAI2026dspark,
author = {Liquid AI},
title = {LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/lfm2.5-dspark},
}
Today, we release DSpark draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality:
- Faster inference: up to 3.18 throughput improvement on a GPU and up to 2.87x on-device.
- Toward on-device agentic inference: cuts function-calling latency by 57% on average for LFM2.5-2.6B
- Day-one support for llama.cpp and SGLang: LFM-compatible DSpark integration is open-sourced upstream
How does DSpark work
The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify.
Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3, DFlash, and, most recently, DSpark, which combines three components:
- DFlash-style parallel backbone conditioned on the target model’s context features, producing hidden states for all draft tokens in a single forward pass.
- A lightweight sequential head, modeled as a Markov chain between neighboring tokens, that adds inter-token dependency, raising the acceptance rate at later positions.
- A confidence-scheduled verifier that predicts each token’s survival probability and prunes low-confidence suffixes when verification would cost more than it saves.
Training and Architecture
We follow the DSpark recipe with a larger and more diverse data mix covering SFT, chat, code, and function-calling data. Based on our ablations, the first versions of the draft models are simplified attention-only draft models, with 5 layers and a block of 9. For each draft model, we ran 15 epochs on the entire dataset and selected the epoch with the highest acceptance rate rather than the lowest loss.
The resulting draft models are relatively small, with each around ~300M parameters.
| Component | LFM2.5-1.2B-Instruct | LFM2.5-8B-A1B | LFM2.5-2.6B |
|---|---|---|---|
| Decoder stack (5 layers) | 241.2M | 241.2M | 241.2M |
| Hidden-state projection | 21.0M | 21.0M | 21.0M |
| Markov head | 33.6M | 65.5M | 65.5M |
| Norms + confidence head | 27.5k | 27.5k | 27.5k |
| Total | 295.7M | 327.7M | 327.7M |
Quality parity
Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place. The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is unchanged.
Inference Speed Up on CPU and GPU
Our DSpark draft models for LFM2.5 ship with day-one support for llama.cpp (implementation builds on top of the official codebase, which we run with experimental metal kernels) and **SGLang (**implementation builds on the official SGLang implementation of DSpark).
We measure on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Both configurations use a DSpark block size of 9, a batch size of 1, and a temperature of 0. We evaluate them on five benchmark datasets.
All three drafter models deliver noticeable throughput improvements on both the large-scale accelerator (H100) and the edge deployment (M4 Max MacBook).
For LFM2.5-2.6B, speedup on the MacBook is especially noticeable, as it pushes the interactivity level a user can enjoy far beyond the throughput offered by most proprietary cloud models (around ~140 tok/s, depending on the dataset).
| Dataset | Acceptance (of 10) | Speedup on H100 | Speedup on M4 Max |
|---|---|---|---|
| MATH500 | 5.42 | 3.06x 326 → 1000 tok/s | 2.25x 61 → 137 tok/s |
| HumanEval | 4.54 | 2.56x 326 → 835 tok/s | 2.63x 61 → 161 tok/s |
| MBPP | 4.71 | 2.64x 326 → 861 tok/s | 2.11x 62 → 132 tok/s |
| GSM8K | 4.32 | 2.22x 312 → 693 tok/s | 2.36x 60 → 143 tok/s |
| MT-Bench | 5.07 | 2.87x 325 → 933 tok/s | 1.99x 62 → 123 tok/s |
| Mean | 4.81 | 2.67x 323 → 864 tok/s | 2.27x 61 → 139 tok/s |
Across various multi-tool scenarios, DSpark reduces the latency by 57% on average for LFM2.5-2.6B.
For LFM2.5-1.2B-Instruct, we see much more variance in dataset acceptance rates, so speedup varies by as much as 52% depending on the underlying text distribution.
| Dataset | Acceptance (of 10) | Speedup on H100 | Speedup on M4 Max |
|---|---|---|---|
| MATH500 | 6.02 | 2.56x 668 → 1712 tok/s | 2.62x 140 → 366 tok/s |
| HumanEval | 5.31 | 2.26x 664 → 1499 tok/s | 2.87x 136 → 389 tok/s |
| MBPP | 5.52 | 2.37x 667 → 1578 tok/s | 2.74x 137 → 375 tok/s |
| GSM8K | 4.34 | 1.67x 624 → 1041 tok/s | 2.73x 140 → 381 tok/s |
| MT-Bench | 3.90 | 1.66x 657 → 1091 tok/s | 1.72x 137 → 237 tok/s |
| Mean | 5.02 | 2.10x 656 → 1384 tok/s | 2.54x 138 → 350 tok/s |
For LFM2.5-8B-A1B, the acceptance rate increases compared to two dense models, yet on-device we get only an 18% improvement on average. This gap is due to the current MoE implementation in llama.cpp's Metal backend, and to the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step.
| Dataset | Acceptance (of 10) | Speedup on H100 | Speedup on M4 Max |
|---|---|---|---|
| MATH500 | 8.27 | 3.18x 428 → 1362 tok/s | 1.21x 93 → 112 tok/s |
| HumanEval | 7.02 | 2.58x 426 → 1100 tok/s | 1.12x 91 → 101 tok/s |
| MBPP | 6.93 | 2.64x 426 → 1122 tok/s | 1.09x 89 → 97 tok/s |
| GSM8K | 4.02 | 1.29x 385 → 496 tok/s | 1.44x 90 → 129 tok/s |
| MT-Bench | 8.52 | 3.02x 426 → 1288 tok/s | 1.04x 87 → 90 tok/s |
| Mean | 6.95 | 2.54x 418 → 1074 tok/s | 1.18x 90 → 106 tok/s |
How to use LFM2.5-DSpark
Running the DSpark draft models with SGLang requires an SGLang build with DSpark support for LFM2 targets (PR #31041). Launch the target with the draft attached:
python -m sglang.launch_server \
--model-path LiquidAI/LFM2.5-2.6B \
--speculative-algorithm DSPARK \
--speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \
--speculative-draft-attention-backend flashinfer \
--disable-radix-cache --mem-fraction-static 0.75 --port 30000
Then query the OpenAI-compatible endpoint at http://localhost:30000/v1. The block size is read from the draft's config.json; the baseline is the same command without the three --speculative-* flags.
Running them with llama.cpp requires the respective llama.cpp build (PR#27383).
llama-server -m LFM2.5-2.6B-F16.gguf \
-md LFM2.5-2.6B-DSpark-F16.gguf \
--spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \
-fa on -ngl 99
The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.
The DSpark draft model checkpoints are available on Hugging Face as Safetensors and in GGUF format:
- Safetensors: LFM2.5-2.6B-DSpark, LFM2.5-1.2B-Instruct-DSpark, and LFM2.5-8B-A1B-DSpark
- GGUF: LFM2.5-2.6B-DSpark-GGUF, LFM2.5-1.2B-Instruct-DSpark-GGUF, LFM2.5-8B-A1B-DSpark-GGUF
We can’t wait to see what you build.
Citation
For citations, please use the following reference or BibTeX:
Liquid AI, "LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook", Liquid AI Blog, Aug 2026.
@article{liquidAI2026dspark,
author = {Liquid AI},
title = {LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/lfm2.5-dspark},
}