# FreeToken 引擎让 8GB 游戏本跑 35B 模型

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-29 04:45
- AIHOT 分数：36
- AIHOT 链接：https://aihot.virxact.com/items/cmtdfbreu040trobx8jep24vs
- 原文链接：https://x.com/rohanpaul_ai/status/2093439825408082139

## AI 摘要

伯克利和得克萨斯大学推出 FreeToken，新引擎可在普通设备上运行大型开源模型。8GB 游戏本跑 35B 模型达 39.3 tokens/秒，快于生产环境中的 Codex。FreeToken 让 GPU 缓存跟随路由请求，将专家缺失率从 llama.cpp 的 62% 降至 16%，其余按 PCIe 和内存速度比例分配。

## 正文

An 8 GB gaming laptop can run a 35B model at 39.3 tokens a second, faster than Codex in production, if the software stops guessing which parts of the model to keep on the GPU.

That's FreeToken, New research from Berkeley and University of Texas, a new engine for running big open models on ordinary machines.

Every other local engine picks which experts sit on the GPU at load time and leaves them there, so when the model routes elsewhere, that expert gets copied over PCIe while the CPU sits idle.

FreeToken lets the GPU cache follow whatever the router just asked for, cutting missed experts from llama.cpp's 62% to 16%.

The rest it splits two ways: some copied to the GPU, the others run in place in system RAM, in a ratio computed from your machine's real PCIe and memory speeds, so neither side waits on the other.