Inside Bonsai 27B: The 3.9 GB, 1-Bit LLM That Fits on an iPhone

Bonsai 27B is a 27.8-billion-parameter multimodal language model launched on July 14, 2026 by AI startup PrismML, built by compressing Qwen3.6 27B end-to-end to 1-bit or ternary weights. It ships in two Apache 2.0 variants — a 3.9 GB 1-bit build sized for an iPhone and a 5.9 GB ternary build for laptops — while retaining 90% and 95% of the full-precision baseline respectively across PrismML's own 15-benchmark suite. Both variants carry the full 262,144-token context window of the Qwen3.6 base and handle text, vision, tool calling, and multi-step agentic loops on-device.

Key Takeaways

• Bonsai 27B launched on July 14, 2026 under Apache 2.0, with weights on Hugging Face.
• Two variants ship: a 1-bit build at 3.9 GB (1.125 effective bits per weight) and a Ternary build at 5.9 GB (1.71 effective bits per weight), both derived from Qwen3.6 27B.
• PrismML reports Ternary Bonsai 27B retains 95% and 1-bit Bonsai 27B retains 90% of Qwen3.6 27B's full-precision average across a 15-benchmark thinking-mode suite.
• The model supports a 262K-token context, structured tool calling, and multimodal vision input via a 4-bit vision tower.
• PrismML reports up to 163 tok/s (1-bit) and 134 tok/s (Ternary) on an NVIDIA RTX 5090, and roughly 11 tok/s on an iPhone 17 Pro Max via MLX Swift.
• Community evaluations are mixed — strong math and coding retention, but noticeable regressions in instruction following, vision, and tool reliability compared to the FP16 base.

What Is Bonsai 27B?

Bonsai 27B is PrismML's flagship low-bit compression of Qwen3.6 27B, positioned as the first 27B-class multimodal model practical to run on a phone. The base model is a dense 27.8-billion-parameter network with a vision encoder and a native 262K context window; Bonsai 27B keeps that architecture and its capabilities — multi-step reasoning, structured tool calls, vision tasks, and long-horizon agentic loops — while pushing the language weights down to 1.125 or 1.71 effective bits per weight.

The release shipped with a blog post on the PrismML site, a whitepaper, a Hugging Face collection with GGUF, MLX, and AWQ builds, and a free developer-preview API. Coverage from 9to5Mac paired the launch with a CNBC report that PrismML CEO Babak Hassibi confirmed Apple is "evaluating" the company's technology, though he characterized the discussions as very early.

The model is fully released and downloads on Hugging Face crossed hundreds of thousands within two days of launch, with the GGUF build alone passing 559k downloads.

Bonsai 27B at a Glance

Field	Value
Developer	PrismML
Base model	Qwen3.6 27B
Type	Dense multimodal LLM (text + vision)
Variants	1-bit (binary) · Ternary ({−1, 0, +1})
Size on disk	3.9 GB (1-bit) · 5.9 GB ideal / ~7.2 GB deployed GGUF (Ternary)
Effective bits/weight	1.125 (1-bit) · 1.71 (Ternary)
Context window	262,144 tokens
Max output	262,144 tokens
Modalities	Text in/out, vision input
License	Apache 2.0
Runtimes	MLX (Apple), CUDA custom kernels, llama.cpp fork, WebGPU, atomic.chat (iOS/Android)
Release date	July 14, 2026
Pricing	Free weights; free-tier developer preview API from PrismML

How Bonsai 27B Works

Bonsai 27B's core mechanism is end-to-end low-bit representation across the entire language network — embeddings, attention, MLPs, and the LM head — with no higher-precision escape hatches. PrismML calls this out explicitly in its announcement: the low-bit encoding is not layered onto a higher-precision base but runs through every language block, with only the vision tower kept at a compact 4-bit form.

Two weight schemes are shipped. Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, giving 1.71 effective bits per weight. The 1-bit variant uses binary {−1, +1} weights with the same group-wise scaling, giving 1.125 effective bits per weight. Both preserve the Qwen3.6 architecture's linear-attention backbone and its 262K-token context, and both add speculative decoding via a drafter model PrismML calls DSpark.

The compression story matters because a conventional FP16 build of a 27.8B model occupies roughly 54 GB, and even a 4-bit build sits near 18 GB — too large for phones and most laptops. A phone never exposes its full memory to a single app; as 9to5Mac notes, a 12 GB iPhone offers about 6 GB for a model to use on-device, and the model has to share that budget with its KV cache and activations. At about 4 GB, 1-bit Bonsai 27B is the first 27B-class build to clear that gate with headroom.

PrismML frames the result with its own coined metric — Intelligence Density, expressed as benchmark score per GB — where 1-bit Bonsai 27B lands at 0.53 per GB, over 10× the FP16 baseline by their measure.

What Bonsai 27B Can Do

Bonsai 27B's target workloads are the ones enabled by keeping capability on-device. The Puter developer catalog lists long-document analysis, coding agents, and tool-calling workflows as the ternary variant's headline use cases. Structured JSON output is supported, and the model handles vision inputs — scoring 68.96 on MMMU-Pro in the ternary configuration.

The on-device story is now live in shipping software. The model runs interactively inside the atomic.chat app on both iPhone and Android, and PrismML measures about 11 tokens per second on an iPhone 17 Pro Max via MLX Swift.

Community benchmarks on tool calling look promising in early runs — one independent evaluation posted the full results on GitHub claiming the 2-bit ternary build performs comparably to NVIDIA's Qwen3.6-27B NVFP4 for agentic workflows in the 12–16 GB VRAM class.

Source: @MiaAI_lab

That said, on the r/LocalLLaMA thread tied to the release, several testers reported the model felt "benchmaxxed" — strong on the published suite but weaker in day-to-day coding tasks, with degraded context handling and unreliable tool use compared to the FP16 base. Ternary is generally preferred to 1-bit for real workloads, and PrismML's own model card acknowledges that long-horizon agentic coding is not yet a strong target of this release — a dedicated coding-tuned variant is next on the roadmap.

How Bonsai 27B Compares

Bonsai 27B's most direct reference point is its own base, Qwen3.6 27B, and the conventional low-bit builds of that base. The table below is drawn from PrismML's own benchmark suite as summarized in the release post.

Category	Qwen3.6 27B (FP16)	Ternary Bonsai 27B	1-bit Bonsai 27B
Math (GSM8K, MATH-500, AIME25/26)	95.3	93.4	91.7
Coding (HumanEval+, MBPP+, LiveCodeBench)	88.7	86.0	81.9
Agentic / Tool (BFCL v3, TauBench)	80.0	74.0	66.0
Instruction following (IFEval, IFBench)	78.4	71.8	65.8
Knowledge / STEM (MMLU-Redux, MuSR)	83.1	77.0	73.4
Vision (MMMU-Pro, OCRBench)	72.6	65.2	59.6
Overall (15 benchmarks)	85.0	80.5	76.1

Read column by column, the drop is largest on vision and instruction following and smallest on math. For a deeper look at the FP16 baseline these compression numbers ride on, see our Qwen 3.6 27B deep dive.

A common alternative for the same VRAM budget is Google's Gemma 4 12B in its 4-bit QAT form, which sits at roughly 7 GB. Hacker News commenters on the Bonsai 27B thread pegged Bonsai as clearly ahead on math and coding, comparable on tool calling, and noticeably behind on vision — a trade-off worth measuring against your actual workload rather than a headline average. Our Gemma 4 deep dive covers that family in detail.

Availability: How to Access Bonsai 27B

The weights are on Hugging Face in the prism-ml Bonsai 27B collection, published as GGUF, MLX (1-bit and 2-bit), AWQ 4-bit, and unpacked variants. The GGUF repo lists integrations for llama.cpp, LM Studio, Ollama, vLLM, Docker Model Runner, and Atomic Chat among others. PrismML also offers a free-tier developer-preview API through its own site, and the model is available on iPhone and Android inside the atomic.chat app today.

If you want to compare Bonsai 27B against fully hosted general-purpose models on the same prompts before committing to a local deployment, Claude Sonnet 5 is a reasonable reference point in the same task space via API.

For local inference, community reports indicate the model runs on an NVIDIA RTX 3060 12 GB at roughly 28 tok/s and on an RTX 4070 Ti Super at ~35 tok/s in the ternary Q2 form. Phone deployment currently works best on high-end iOS hardware; PrismML measures about 11 tok/s on an iPhone 17 Pro Max via MLX Swift, and the atomic.chat integration is the most polished way to try it on-device today.

One caveat for llama.cpp users: several commenters on r/LocalLLaMA noted that stock llama.cpp unpacks the low-bit weights to 8-bit for matmul unless the build ships the dedicated ternary path — meaning you can pay the unpack cost without the speed gain. PrismML's own llama.cpp fork carries the packed ternary kernel; using it is a prerequisite for the published throughput numbers. Backend-by-backend upstreaming is in progress: CPU (ARM NEON + scalar) and Metal ternary support are merged into mainline llama.cpp, with Vulkan approved and CUDA in review.

What We Don't Know Yet

Two load-bearing questions remain. The depth of Apple's interest, framed by Hassibi as "very early," is unconfirmed by Apple itself. And sustained on-device performance — throughput after thermal throttling, battery drain during long agent runs, and mid-range Android compatibility — is largely uncharted outside a handful of demos and the atomic.chat rollout.

Frequently Asked Questions

What is Bonsai 27B?

Bonsai 27B is a 27-billion-parameter multimodal language model released by PrismML on July 14, 2026, built by compressing Qwen3.6 27B to 1-bit or ternary weights. It ships in a 3.9 GB 1-bit variant and a 5.9 GB ternary variant, both under the Apache 2.0 license.

Who made Bonsai 27B?

Bonsai 27B was built by PrismML, an AI startup led by CEO Babak Hassibi. The company specializes in low-bit model compression and, according to a CNBC report cited by 9to5Mac, has been in early discussions with Apple about its technology.

Is Bonsai 27B open source?

Yes. Bonsai 27B is released under the Apache 2.0 license, with weights, GGUF builds, MLX packs, and AWQ variants published in the prism-ml Hugging Face collection. Both the 1-bit and ternary variants are downloadable and commercially usable.

How much memory does Bonsai 27B need?

The 1-bit Bonsai 27B variant occupies about 3.9 GB for its resident language weights, and the ternary variant is 5.9 GB in its ideal representation. A third-party review notes that the current ternary GGUF pack actually occupies around 7.2 GB, and both variants also need memory for the KV cache and activations.

Can Bonsai 27B really run on a phone?

Yes. The 1-bit variant is small enough to fit within the memory budget of a current high-end phone, and it is available now inside the atomic.chat app for both iPhone and Android. PrismML reports roughly 11 tokens per second on an iPhone 17 Pro Max via MLX Swift.

How does Bonsai 27B compare to Qwen3.6 27B?

Bonsai 27B is a compressed derivative of Qwen3.6 27B. PrismML reports that the ternary variant retains 95% of the full-precision average across its 15-benchmark suite and the 1-bit variant retains 90%, with math and coding holding up best and vision, instruction following, and multi-step tool use degrading more.

What is the context window of Bonsai 27B?

Bonsai 27B carries a 262,144-token context window, inherited from the Qwen3.6 27B base. That is roughly 500 pages of text and supports long-document analysis, coding agents, and agentic tool-calling workflows.

What to Watch Next

Three signals are worth tracking. First, PrismML's coding-tuned follow-up — flagged on the model card as next on the roadmap — which will test whether the low-bit format holds up on longer, tool-heavy code generation runs. Second, any concrete outcome of the Apple evaluation Hassibi described; that would meaningfully shift the on-device LLM landscape beyond a single startup's model. Third, independent third-party benchmark reproductions, particularly on agentic reliability and vision — the largest gap between PrismML's published numbers and community sentiment lives there.

Building similar on-device or long-context workflows? On kie.ai you can try Gemini 2.5 Flash, Claude Sonnet 5, and Gemini 3 Flash.