UC Berkeley open-sourced FreeToken. Wild results:
A single RTX PRO 6000 runs the 753B GLM-5.2 at 14.9 tok/s!
An 8GB RTX 4060 laptop (~$1,000) runs Qwen3.6-35B at 39.3 tok/s!
FreeToken is 2–4x faster than Ollama across consumer GPUs. Local AI inference is getting very real. Great work by @Andy_ShuoYang and UC Berkeley Sky Lab!
Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 3...