你的笔记本电脑现在可以运行一个能力几乎媲美云端任何模型的模型。我把 Qwen3.8-27B 换进了我的智能体，它运行得非常好。这只鸟的飞行方式与飞机截然不同。
这个小小的 Qwen 模型在 135 个模型中排名第一，在 Artificial Analysis 的智能指数上得分 52，比来自 Z.ai 的最先进开源模型 GLM-5.2 高出 1 分，后者参数量为 753b。一个笔记本电脑模型击败了一个规模约为其 28 倍的、公认的前沿级云端同类模型。
一只大黄蜂是如何实现与客机相同的飞行的？更大的模型可以存储更多知识，因此它们可以直接跳到答案，就像精通多个领域的专家一样。较小的模型记忆的内容没那么多，因此它们必须更多地推理，几乎是从第一性原理出发，来弥合这一差距。
我在将 DeepSeek V4 云端模型与两个本地模型进行基准测试时，亲眼看到了这一点。我在相同的工作上对它们进行了比较，即 25 项风险投资任务（研究初创公司、总结文章、转录播客），并由一个裁判模型进行评分。
Qwen3.8-27B 是稠密模型：它会在每个问题上用到书中的每一个章节。而“跳读型”的 DeepSeek 和 Qwen 3.6 35b（我加入测试的另一个本地模型）只会翻到与问题相关的章节。
| 模型 | 质量 /9 | token/秒 | 平均 token 数 | 平均延迟 |
|---|---|---|---|---|
| deepseek-v4-flash（飞机） | 8.0 | 137.3 | 159 | 1.1秒 |
| qwen3.8-27b（大黄蜂） | 8.0 | 51.9 | 369 | 7.2秒 |
| qwen3.6-35b-a3b（蜂鸟） | 7.9 | 113.4 | 1,143 | 10.0秒 |
这些模型给出的答案质量完全相同。但速度各不相同。本地 Qwen 35b 以最高速度运行，但需要比云端模型多想约 7.2 倍的 token，比 DeepSeek 晚 9 秒到达终点。最新的 Qwen 模型快了 3 秒，而云端模型又快 6 秒。
云端模型直接跳到正确答案；本地模型则以不同的速度和准确度在内部进行思考和辩论。
例如：在一个分诊任务中，35B 花费了 993 个 token 来生成六个词：“分类：日程安排 / 动作：回复。” 在回复之前进行 1000 个 token 的思考，是蜂鸟冲刺向金银花。而大黄蜂只需要 369 个思考 token，以一半的速度嗡嗡前行。
本地模型可以达到与云端模型相同的结果，但它们到达那里的飞行路径会有所不同。
-
Artificial Analysis 将当前的领先者 Qwen3.8-27B 排在智能指数 135 个模型中的第 1 位，得分 52，比 GLM-5.2 的 51 分高出 1 分，后者是 Z.ai 两个月前发布的 753b 参数前沿模型。同一页面将其在每任务输出 token 数上排在第 23 位（共 135 个模型），加权输出为 1.6 亿 token，而同类中位数为 4300 万。智能排名与冗长程度排名是独立变动的，而这正是整篇文章要讨论的权衡。↩︎
-
模仿差距的解释。较小的模型会产生流畅的思维链，但更有可能在逻辑上出现漂移和不一致，因为一旦被迫偏离通往答案的直接路径，它们可借鉴的邻近正确示例的图谱就更稀疏。参见《大语言模型中的思维链：引发的推理还是受约束的模仿？》我在《当模型学习时》一文中涵盖了这种权衡的一般形态，即用推理时计算换取能力。↩︂
-
方法。25 个风险投资任务（研究初创公司、总结文章、转录播客）取自于我自己的智能体队列。一个独立的评判模型 deepseek-v4-pro 对输出进行盲评，从完整性、准确性和简洁性三个维度打分，每项 3 分，共 9 分。每次运行的 max_tokens 均为 4096。两个本地模型都通过 Ollama 在相同的 MLX 运行时上提供服务，因此比较不受运行时差异的干扰。我通过对相同输出进行重新评分来确立评判模型的噪声底限，得到的平均绝对差值为 0.16。↩︎
-
Qwen3.6-35B-A3B 是一个 350 亿参数模型，每个 token 有 30 亿激活参数，采用稀疏混合专家架构，共有 256 个专家，每个 token 有 8 个被路由的专家和 1 个共享专家处于激活状态。根据 Qwen 在 Hugging Face 上的模型卡及 vLLM 的模型配方。↩︎
Your laptop can now run a model as capable as nearly anything in the cloud. I swapped Qwen3.8-27B into my agent & it works brilliantly. This bird flies differently than a plane.
This little Qwen model ranks #1 of 135 models, scoring 52 on Artificial Analysis’s Intelligence Index, a point above GLM-5.2, the state-of-the-art open-source model from Z.ai, at 753b parameters.1 A laptop model beats a recognizable, frontier-class cloud peer roughly 28 times its size.
How does a bumblebee achieve the same flight as an airliner? Bigger models can store more knowledge, so they can skip straight to an answer, like an expert in many different fields. Smaller models don’t have as much memorized, so they must reason more, almost from first principles, to close that gap.2
I saw this firsthand when benchmarking the DeepSeek V4 cloud model against two local models. I compared them on the same work, 25 venture-capital tasks (researching startups, summarizing articles, transcribing podcasts), scored by a judge model.3
Qwen3.8-27B is dense : it uses every chapter in the book on every question. Book skimmers DeepSeek & Qwen 3.6 35b (another local model I threw into the test), flips only to the relevant chapters for a question.4
| model | quality /9 | tok/s | avg tokens | avg latency |
|---|---|---|---|---|
| deepseek-v4-flash (plane) | 8.0 | 137.3 | 159 | 1.1s |
| qwen3.8-27b (bumblebee) | 8.0 | 51.9 | 369 | 7.2s |
| qwen3.6-35b-a3b (hummingbird) | 7.9 | 113.4 | 1,143 | 10.0s |
These models provide identically good answers. But the speed varies. The local Qwen 35b shreds at top speed, but needs to think about 7.2x more than the cloud model, crossing the line 9 seconds after DeepSeek. The newest Qwen model is three seconds faster, & the cloud is 6 seconds faster yet.
The cloud model jumps to the right answer ; the local models contemplate & debate internally at different rates of speed & accuracy.
For example : on one triage task, the 35B spent 993 tokens to produce six words, “Classification: Scheduling / Action: Respond.” 1000 tokens of deliberation before the response is a hummingbird’s sprint to a honeysuckle. The bumblebee needed 369 thinking tokens, buzzing along at half the speed.
Local models can achieve the same result as cloud models, but they’ll take a different flight path to get there.
-
Artificial Analysis ranks the incumbent here, Qwen3.8-27B, #1 of 135 models on the Intelligence Index, scoring 52, a point above GLM-5.2’s 51, a 753b-parameter frontier model Z.ai shipped two months earlier. The same page ranks it #23 of 135 on output tokens per task, 160M weighted tokens against a class median of 43M. Intelligence rank & verbosity rank move independently, & that’s the trade this whole post is about. ↩︎
-
The imitation-gap explanation. Smaller models produce fluent chain-of-thought that’s more likely to drift logically inconsistent, because they have a sparser map of nearby correct examples to draw on once forced off the direct path to an answer. See Chain of Thought in Large Language Models: Elicited Reasoning or Constrained Imitation? I covered the general shape of this tradeoff, trading inference-time compute for capability, in When Models Learn. ↩︎
-
Method. 25 venture-capital tasks (researching startups, summarizing articles, transcribing podcasts) drawn from my own agent queue. A separate judge model, deepseek-v4-pro, scored outputs blind on completeness, accuracy & conciseness, 3 points each for 9 total. max_tokens was 4096 for every run. Both local models were served through Ollama on the same MLX runtime, so the comparison isn’t confounded by runtime differences. I established the judge’s noise floor by re-scoring identical outputs, which returned a mean absolute difference of 0.16. ↩︎
-
Qwen3.6-35B-A3B is a 35b parameter model with 3b active parameters per token, a sparse mixture-of-experts architecture, 256 total experts with 8 routed & 1 shared active per token. Per Qwen’s model card on Hugging Face & vLLM’s model recipe. ↩︎