简而言之：约半数智能体任务可在本地 35B 模型上运行。真正的优势不在于成本或隐私——而在于延迟。速度快 2.1 倍意味着每次会话中可完成更多迭代周期。
随着 AI 推理需求爆发式增长，我将对我这台小电脑提出更多要求。
具体多多少？
过去五周，我一直在使用本地模型，测试在不依赖云端万亿参数模型的情况下，能完成多少日常工作。答案是半数。
| 类别 | 数量 | 占比 | 示例 |
|---|---|---|---|
| 其他 | 521 | 35.3% | 非结构化请求的统称 |
| 日程安排 | 254 | 17.2% | 检查空闲时间，提议会议时间 |
| 市场调研 | 192 | 13.0% | 竞品分析，融资数据 |
| 摘要 | 184 | 12.4% | 转录稿审阅，视频摘要 |
| 邮件与入站消息 | 170 | 11.5% | 草拟回复、跟进邮件、转发 |
| 工程 | 147 | 9.9% | 调试脚本、API 修复、CLI 任务 |
| 行政 | 10 | 0.7% | 差旅、费用、报销 |
如果将这 1400 个任务按类别划分，半数可在本地 35B 模型上成功完成。邮件与入站消息、日程安排、摘要及行政类任务合计 618 个（41.8%）。市场调研和工程类任务中，简单任务（数据查询、脚本修复）与复杂任务（多源信息综合、架构决策）大致各占一半。这样加起来就达到了 50%。
使用本地模型有很多理由：隐私、成本、资产折旧。
但实际上，唯一真正重要的因素是延迟。
今天早上我进行了一场头对头基准测试。八个智能体任务，相同的提示词，两个模型均已预热。Qwen 3.6 35B-A3B-4bit 在我的 MacBook Pro M5 上运行，对比通过 API 调用的 Claude Opus 4.5。
本地模型并不更聪明。Opus 4.5 在推理基准测试上得分高出约 20%。本地模型落后前沿模型 3-4 个月，对于大规模复杂任务，这一差距确实重要。但对于常规智能体任务，它很少产生影响。
Opus 在结构与格式上胜出：项目符号、标题、更整洁的代码。Qwen 在简洁性上胜出，输出 token 数通常只有一半。我逐行对比了每个输出，两个模型都正确完成了任务。对于输出会输入到另一个系统的智能体任务而言，简洁本身就是一种优势。
本地最大化，即将更多推理任务推向本地模型，是对 token 最大化的必然回应。随着本地模型不断改进并缩小与前沿模型的差距，更多用户将把工作负载转移到自己的硬件上。
如果有一半的工作在我的笔记本电脑上能以两倍速度运行，我每次都会选择这个方案。我的小电脑即将证明自己的价值。
-
无论你是否使用，MacBook Pro 都在贬值。在转售之前，通过运行本地推理，可以从这个正在贬值的资产中提取计算价值。↩︎
In short : About half of agent tasks can run on a local 35B model. The real advantage isn't cost or privacy — it's latency. 2.1x faster means more iteration cycles per session.
As demand for AI inference explodes, I’ll be asking a lot more of my little computer.
How much more?
Over the past five weeks, I’ve been using local models to see how much of my daily work I can accomplish without the trillion parameter models in the cloud. The answer is half.
| Category | Count | % of Total | Example |
|---|---|---|---|
| Other | 521 | 35.3% | Catch-all for unstructured requests |
| Scheduling | 254 | 17.2% | Check availability, propose meeting times |
| Market Research | 192 | 13.0% | Competitor analysis, fundraising data |
| Summarization | 184 | 12.4% | Transcript review, video summaries |
| Email & Inbound | 170 | 11.5% | Draft replies, follow-ups, forwards |
| Engineering | 147 | 9.9% | Debug scripts, API fixes, CLI tasks |
| Admin | 10 | 0.7% | Travel, expenses, reimbursements |
If you classify these 1.4k tasks by category, half can succeed on a local 35B model. Email & Inbound, Scheduling, Summarization, & Admin total 618 tasks (41.8%). Market Research & Engineering split roughly 50/50 between simple tasks (data lookups, script fixes) and complex ones (multi-source synthesis, architectural decisions). That gets us to 50%.
There are many reasons to use local models : privacy, cost, asset depreciation.1
But in reality, the only one that really matters is latency.
I ran a head-to-head benchmark this morning. Eight agentic tasks, same prompts, both models warmed. Qwen 3.6 35B-A3B-4bit on my MacBook Pro M5 vs Claude Opus 4.5 via API.
The local model isn’t smarter. Opus 4.5 scores ~20% higher on reasoning benchmarks. Local models lag frontier by 3-4 months, and for large-scale complex tasks, that gap matters. But for routine agent tasks, it rarely does.
Opus wins on structure & polish : bullet points, headers, cleaner code. Qwen wins on brevity, often half the tokens. I read every output side by side, and both completed the tasks correctly. For agent tasks where output feeds into another system, terseness is a feature.
Localmaxxing, pushing more inference to local models, is an inevitable response to tokenmaxxing. As local models improve & close the gap with frontier, more users will shift workloads to their own hardware.
If half the work runs 2x faster on my laptop, I’ll take that trade every time. My little computer is about to earn its keep.
-
A MacBook Pro depreciates whether you use it or not. Running local inference extracts compute value from a sinking asset before resale. ↩︎