A 27-billion-parameter model that normally needs 54 gigabytes of memory now fits in under 6 — and according to its own benchmark suite, still answers correctly 98% as often. That’s the kind of number that invites suspicion, so it’s worth checking what the compression actually costs.

What shipped

Ternary-Bonsai-2-27B, from a company called Prism ML, compresses a 27B-parameter model (built on Qwen3.8-27B) down to “1.72 bits per weight” — each weight takes one of three values (-1, 0, or +1) instead of the usual 16-bit floating point number — for “~9.3x smaller than FP16 (ideal)” (Hugging Face model card: Hugging Face). The packaged file lands at 5.95GB, down from roughly 54GB for the full-precision version.

What the compression actually costs

Per the model card’s own benchmark table across 14 tasks: “98.2% of FP16 intelligence retained: 84.78 average… far above the conventional IQ2_XXS build (72.59) at less than two-thirds of its footprint.” The model card is unusually specific about where the accuracy gap concentrates: “math within half a point of full precision (96.57), coding level with the baseline (89.42)” — but it doesn’t hide the trade-off entirely; the comparison table shows the gap widening on the hardest reasoning benchmarks. Worth noting directly: this is Prism ML’s own benchmark suite, run on their own hardware, not independently reproduced at time of writing.

One detail in the model card is worth quoting exactly, because it’s an unusually candid admission for a vendor to make about competitors: “Unlike conventional low-bit builds — whose advertised labels understate their true average bit-width (a widely-used ‘2-bit’ build of Qwen3.8-27B is really 2.8 bits/weight at 9.4 GB) — the Bonsai representation carries a bit-width that matches its name.” That’s a real, checkable claim about a common practice in the quantization community (rounding down advertised precision), not just marketing language about their own model.

A real counterpoint, from a different model in the same family

An earlier Prism ML release, Bonsai-8B (compressed to 1-bit, not the 2-bit format used in this newer 27B release), drew criticism on Reddit’s r/LocalLLaMA in April 2026 for underperforming a similarly-sized Gemma model on basic knowledge questions — “Bonsai-8B is MUCH dumber than Gemma-4-E2B,” per the original post (reddit.com/r/LocalLLaMA, discussion thread). Commenters on that thread themselves noted the comparison wasn’t entirely fair: Bonsai-8B was “built on Qwen3, not Qwen3.5,” an older base model, and one commenter pointed out that “it’s not really what these models are for” — arguing tiny quantized models are better suited to narrow tasks like sentiment analysis than general knowledge recall. That criticism targets a different model (1-bit, older base) than the one trending this week (2-bit, newer Qwen3.8 base) — it’s relevant context on how skeptically this category of claim gets received, not a direct rebuttal of Ternary-Bonsai-2-27B specifically.

Why this matters

If the 98.2%-retention claim holds up under independent testing (which doesn’t yet exist for this specific release), running a 27B-class reasoning model on a standard laptop — the model card reports “~47 tok/s on an Apple M5 Max” — changes what’s practical for privacy-sensitive or offline use without a GPU cluster. The honest caveat about competitors mislabeling bit-width is a good sign of technical seriousness; the lack of independent benchmarking, and the rockier reception of an earlier, more aggressively compressed release from the same team, are reasons to treat the 98.2% figure as a claim to watch confirmed, not a settled fact.