DFlash 2 is very cool!
Speculative decoding is such a cool technology.
I’m waiting for the day we can run a 1T model locally at 100 toks/s on an iPhone.
DFlash 2 is here! Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro. ⚡ Up to 4.6× the speed of autoregressive decoding, with the same output. This is the next ge...