PrismML just released Bonsai 27B. It is a low-bit representation of Qwen3.6-27B, not a new pretrain. The architecture is unchanged.

Two variants ship under Apache 2.0. Ternary Bonsai 27B uses {−1, 0, +1} weights at a true 1.71 bits per weight. Its ideal size is 5.9GB. 1-bit Bonsai 27B uses binary {−1, +1} weights at 1.125 bits per weight, for 3.9GB.

Both are multimodal. The split is ~24.8B language weights, a 0.46B vision tower, and 2.5B in embeddings and the LM head. The vision tower is held separately at 4-bit (HQQ). Context is 262K tokens, kept practical because ~75% of Qwen3.6-27B attention is linear.

That architecture shapes the compression method below.

How the Compression Works

Each weight is a code, with one shared FP16 scale per group of 128. The effective weight is w_i = s_g · t_i.

A ternary value carries log2(3) ≈ 1.585 bits. One FP16 scale per 128 weights adds 16/128, giving ≈1.71 bits per weight. That is a ~9.4× reduction against FP16. Binary costs 1 + 16/128 = 1.125 bits, a ~14.2× reduction.

The representation runs end to end across the matrix-heavy components. Those are embeddings, attention projections, MLP projections, and the LM head. Only a negligible tail of normalization and scale parameters stays higher precision.

Measured as a true average, the Qwen3.6-27B “4-bit” build (Q4_K_XL) is 5.2 bits per weight. The “2-bit” build (IQ2_XXS) is 2.8. Bonsai also departs from BitNet, which avoids collapse only by pretraining from scratch.

The obvious question is what compression costs in accuracy.

Performance

PrismML evaluated 15 benchmarks in thinking mode, using EvalScope with vLLM on H100 GPUs. Ternary Bonsai 27B retains 94.6% of the FP16 baseline, and 1-bit Bonsai 27B retains 89.5%.

Variant	True bpw	Footprint	Thinking avg	Density (1/GB)
Qwen3.6-27B FP16	16.0	54GB	85.07	0.051
Qwen3.6-27B Q4_K_XL (“4-bit”)	5.2	17.6GB	84.99	0.155
Qwen3.6-27B IQ2_XXS (“2-bit”)	2.8	9.4GB	72.73	0.199
Ternary Bonsai 27B	1.71	5.9GB	80.49	0.400
1-bit Bonsai 27B	1.125	3.9GB	76.11	0.530

Category	FP16	Ternary	1-bit
Math	95.33	93.40	91.66
Coding	88.74	85.96	81.88
Knowledge and reasoning	83.15	76.96	73.39
Agentic and tool calling	80.00	74.01	66.03
Instruction following	78.47	71.77	65.74
Vision	72.61	65.19	59.57

Conventional sub-4-bit builds fail differently. IQ2_XXS falls to 57.5 on AIME26 and 56.4 on LiveCodeBench. It still scores 88.93 on MMLU-Redux, so short-form benchmarks mask the collapse. Gemma-4-31B Q2_K_XL repeats that pattern on a second base model.

Scores alone, however, do not explain the release. Memory does.

Memory is the Binding Constraint

Fitting a phone is stricter than storage numbers suggest. iOS limits a single app to roughly half of physical memory. A 12GB iPhone therefore exposes about 6GB.

The KV cache is the second budget. Only 16 of 64 layers carry a growing full-attention cache, so FP16 costs ≈64 KiB/token. A 262K window costs ≈17.2GB, and a 4-bit KV cache cuts that to ≈4.3GB.

Tolerance is measured. Against its own FP16-KV baseline, Ternary Bonsai shows 0.0011 nats of output forward-KL on MATH-500. Q4_K_XL shows 0.0146.

Peaks follow. At 100K tokens with an FP16 cache, 1-bit peaks at 11.6GB and ternary at 14.7GB. The derived Q4_K_XL row needs ≈25.6GB.

Once a model fits, throughput is the next question.

Throughput and DSpark Speculative Decoding

Platform	Variant	tg128	pp512
M5 Max	Binary	66.4	874
M5 Pro	Ternary	26.2	393
iPhone 17 Pro Max	Binary	11.0	111
H100 (CUDA)	Binary	104.8	2755

Generation is memory-bandwidth-bound, so fewer bytes per step means more tokens per second. Prefill is compute-bound and gains less.

PrismML also ships a DSpark drafter trained against the Bonsai 27B target. On an H100 at draft depth k=4, the binary target reaches accepted length τ=3.6. That is 143.8 tok/s, a 1.37× speedup. Verification is lossless, so output stays distribution-identical. On Apple Silicon the drafter is off by default at batch size 1.

Running It

Ternary 27B is the demo repo default. Start the server, or generate directly:

./scripts/start_llama_server.sh   # OpenAI-compatible API + chat/vision UI on :8080

./llama-cli -m ./Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-Q2_0.gguf \
  --mmproj ./Ternary-Bonsai-27B-gguf/mmproj.gguf -c 0 \
  -p "Explain KV cache growth."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
  --prompt "Explain KV cache growth."

Tool calling uses the standard OpenAI-style tools array:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'

The call returns in choices[0].message.tool_calls. Thinking mode is on by default; thinking_budget_tokens toggles it per request.

That maps onto four deployment patterns.

Use Cases

Laptop-local agents run the ternary build for full-repository code work over 262K tokens. Phone-local reasoning runs the 1-bit build; the whitepaper measures 672 tokens per 1% of iPhone battery. Privacy-sensitive and offline workflows keep prompts on-device by construction. Combined with the 4-bit KV cache, single-GPU serving fits 27B-class quality on a 24GB card.

Key Takeaways

• Bonsai 27B moves Qwen3.6-27B into binary or ternary weights, not a new pretrain.
• Ternary retains 94.6% of FP16 at 5.9GB; 1-bit retains 89.5% at 3.9GB.
• PrismML claims the 1-bit build is the first 27B-class model to fit a phone.
• Conventional sub-4-bit builds collapse selectively on AIME, LiveCodeBench, and agentic tasks.
• Everything ships under Apache 2.0, on llama.cpp (CUDA, Metal) and MLX.

---