This is so interesting for local LLM.
Atomic Chat compressed the 295B-parameter Hy3 （to an extreme 1-bit ） model into a 92GB file and ran it 2.2x faster than its cloud API.
They installed the model locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API.
The big deal is that the extreme compression kept useful quality after this extreme compression.
Both versions built working Flappy Bird， Arkanoid， and Snake games from one-shot prompts， with no crashes or obvious visual gap.
Outputs： Hy3 1-bit local： 76.9K tokens， 15.5 min Hy3 cloud API： 75.1K tokens， 34.3 min
Hy3 is Tencent's open-sourced Apache 2.0 licensed model， i.e. commercially usable by enterprises. A massive 295B MoE （21B active parameters）.
1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compared one-shot outputs. 1-bit Hy3 295B GGUF ...