Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
Recorded: Sept. 17, 2026, 10:09 p.m.
| Original | Summarized |
PrismML — Introducing Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint AboutBlogNewsCareersContactDocsModelsBonsai 2 27BBonsai 27BBonsai 8BBonsai 4BBonsai 1.7BBonsai ImageAboutBlogNewsCareersDocsLAUNCH1001011100 11010 1 001Back to all postsIntroducing Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller FootprintSeptember 17, 2026•PrismMLTwo months ago, we released our first Bonsai 27B models and showed that a 27B-class multimodal model could be compressed enough to run efficiently on a local device. Today, we’re releasing Ternary Bonsai 2 27B, our most capable model yet.Based on Qwen3.8 27B, Ternary Bonsai 2 27B brings stronger reasoning, coding, vision, and agentic capability to the Bonsai series while preserving the deployment profile that defines it: a dramatically smaller memory footprint, high local throughput, and better energy efficiency.Ternary Bonsai 2 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, for 1.76 effective bits per weight and a total model footprint of 5.9GB. The low-bit representation is applied end to end across the language model. It supports a 262K-token context window, multimodal text-and-image input, and is released under the Apache 2.0 license.Against its full-precision counterpart, Ternary Bonsai 2 27B is more than 9x smaller while retaining 98.2% of aggregate benchmark performance. At this level of retention, compression becomes a deployment unlock: nearly the same capability, in a footprint that can run in far more places.What changed from the first Bonsai 27B releaseOur first Bonsai 27B release was an important milestone, offering a practical way to run 27B-class intelligence on local devices. Bonsai 2 27B focuses on the next step: improving the model quality and runtime performance needed for real-world local applications. Compared with the previous Bonsai 27B generation, Bonsai 2 27B brings:a stronger base model, Qwen3.8 27Bhigher aggregate capability retention of 98.2% against the full-precision modelimproved reasoning, coding, vision, and long-horizon agentic performanceHigher capability at the same deployment pointAcross a benchmark suite spanning reasoning, math, coding, instruction following, vision, and agentic tool use, Ternary Bonsai 2 27B scores 83.9, retaining 98.2% of Qwen3.8 27B’s aggregate performance. Capability Agentic & Tool Calling 77.57 Coding 81.58 Instruction Following 82.66 Knowledge & Reasoning 83.95 Math 96.57 Vision 78.59 Overall Figure I: Benchmark scores of Ternary Bonsai 2 27B (thinking mode) compared with the full-precision Qwen3.8 27B and Qwen3.6 27B baselines. Full per-benchmark results are in the whitepaper.The key result is not only the aggregate score, but where the capability is retained. Coding agents, tool-use systems, multimodal workflows, and long-horizon tasks are particularly sensitive to model degradation because small errors can compound over many steps. Bonsai 2 27B preserves much of the full-precision model’s performance in exactly these areas while operating at a fraction of the memory footprint.Compared with the full-precision model and other low-bit alternatives, Bonsai 2 27B stands out as an outlier on intelligence density. Many low-bit alternatives become deployable only by giving up meaningful capability in coding, vision, or agentic tool use. Bonsai 2 27B pushes the frontier toward both higher capability and lower memory usage.Figure II: Intelligence density (per GB) of Ternary Bonsai 2 27B compared to other models in the same parameter class.Demo I: Coding agents with Cline, powered by Ternary Bonsai 2 27B on NVIDIA GeForce RTX 5090.Demo II: Computer use powered by Ternary Bonsai 2 27B model on NVIDIA GeForce RTX 5090.‹›With Bonsai 2 27B, local models can start to take on real knowledge work: coding-agent loops, computer-use workflows, private document analysis, multimodal debugging, and hybrid orchestration where local models handle sensitive or high-frequency tasks while escalating selectively to the cloud.Throughput and energy efficiencyTernary Bonsai 2 27B reaches up to 143 tokens/second on NVIDIA GeForce RTX 5090 and 46.8 tokens/second on M5 Max. On an RTX 4090, Ternary Bonsai 2 27B consumes just 0.714 mWh/token, making it 40% more energy-efficient than an 8B model running in full-precision.For coding assistants, higher throughput means faster edit-debug loops. For multimodal agents, it means quicker iterations over screenshots, documents, and tool calls. For private local workflows, better energy efficiency means more useful inference on the same device, longer battery life, and a more realistic path to assistants that can stay available in the background without constantly calling the cloud.Why this release mattersCompared to Ternary Bonsai 27B, the new Ternary Bonsai 2 27B has closed the retention gap between the full precision model from 95% to over 98%. This is a significant improvement that makes the current release practically “lossless”. It further cements the notion that low-bit models can be the best way to deploy AI. That has implications well beyond local inference. Low-bit models can change the economics and architecture of AI systems across devices, workstations, and datacenters: fitting larger models into the same memory envelope, serving more users on the same hardware, reducing energy per inference, and enabling hybrid systems that dynamically decide what should run locally and what should run in the cloud.The question will increasingly be not just how capable a model is, but how much useful intelligence can be delivered within a given memory, compute, and power budget. If capability can continue to scale while those requirements fall dramatically, the deployment envelope for future models expands across the stack: from personal devices to large-scale datacenters.Platform CoverageBonsai 2 27B runs on NVIDIA GPUs via CUDA and on Apple devices (Mac, iPhone, iPad) via MLX, through custom low-bit kernels. Model weights are available today under the Apache 2.0 License.Full technical details of our compression, evaluation, and benchmarking processes are available in our whitepaper.Work with UsWe work with teams to tailor Bonsai models to their applications, from post-training on domain-specific data to optimizing inference for target hardware. If you’re building AI products with tight memory, latency, or power requirements, we’d love to explore how Bonsai can help. Reach out at contact@prismml.com.Join UsPrismML emerged from a team of Caltech researchers and was founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung. We've spent years tackling one of the field's hardest problems: compressing neural networks without sacrificing their reasoning ability.If you want to help build the next generation of state-of-the-art AI, we'd love to hear from you. Check out our careers page.Back to all postsAnnouncing Bonsai 27B: The First 27B-Class Model to Run on a PhoneJuly 14, 2026Today we're announcing Bonsai 27B, our multimodal flagship: ternary at 5.9GB for laptops, 1-bit at 3.9GB for an iPhone 17 Pro, with a 262K-token context.PrismML Launches Bonsai 2 27B, Its Most Capable Model YetSeptember 17, 2026New flagship model brings 27B-class reasoning, coding, vision, and agentic capability into a dramatically smaller, faster deployment footprintThanks, we’ll keep you posted!Something went wrong.ResourcesDemoWhitepaperDocsModelsHugging FaceGitHubFollowXDiscordLinkedInAboutGet updates on new releases and company news:Thanks, we’ll keep you posted!Something went wrong.© 2026 Prism ML, Inc. All rights reserved.ModelsBonsai 27BBonsai 8BBonsai 4BBonsai 1.7BBonsai ImageCompanyAboutBlogNewsContactCareersContactLegalTermsPrivacyFollowContact usHave a question, partnership idea, or a project that needs efficient intelligence? Reach out—we’d love to hear from you.Thanks, your message has been received!Oops! Something went wrong while submitting the form. |
PrismML introduced Ternary Bonsai 2 27B, a multimodal model that demonstrates near-lossless compression achieving a nine-fold reduction in footprint while maintaining strong performance, aiming to enable efficient deployment on local devices. This achievement is based on compressing the model using ternary weights, which are limited to the values of minus one, zero, and plus one, combined with FP16 group-wise scaling to represent each weight using approximately 1.76 effective bits. The resulting model has a total footprint of 5.9GB and supports a context window of 262K tokens along with multimodal text-and-image inputs, operating under the Apache 2.0 license. Ternary Bonsai 2 27B retains a high degree of capability against its full-precision counterpart, preserving 98.2% of aggregate benchmark performance across a wide range of tasks. The method successfully preserves performance in areas highly sensitive to model degradation, such as coding, vision, reasoning, instruction following, and agentic tool use, indicating that this compression unlocks practical deployment possibilities for complex intelligence tasks. This superior intelligence density positions Bonsai 2 27B as an outlier, suggesting that low-bit representations can provide meaningful capability without sacrificing critical performance attributes required for real-world applications. The release represents an evolution from previous Bonsai models, focusing on enhancing the quality and runtime performance necessary for local applications. The model is built upon the Qwen3.8 27B base, incorporating improvements in reasoning, coding, vision, and long-horizon agentic performance. In a comprehensive benchmark suite covering reasoning, mathematics, coding, instruction following, vision, and agentic tool use, Ternary Bonsai 2 27B achieved a strong aggregate score of 83.9, retaining the core performance of the larger precision models. Beyond sheer capability retention, the efficiency of Ternary Bonsai 2 27B is a critical feature. It demonstrates superior throughput and energy efficiency when deployed on various hardware. For instance, the model can achieve up to 143 tokens per second on an NVIDIA GeForce RTX 5090, and it exhibits significant energy savings, consuming only 0.714 mWh per token on an RTX 4090, which is 40% more energy-efficient than an 8B model running in full precision. This efficiency translates directly into faster iterative loops for coding assistants and multimodal agents, and enables more useful inference on local devices with extended battery life. This advancement has profound implications for the architecture and economics of AI systems. By enabling larger models to fit within smaller memory envelopes, lower-bit methods facilitate running complex intelligence across diverse platforms, from personal devices to large-scale data centers. This shifts the focus from absolute model size to the useful intelligence delivered per memory, compute, and power budget. The work suggests that future AI deployment will increasingly be governed by how much useful intelligence can be delivered within strict hardware constraints, pushing the deployment envelope for subsequent models across the entire technology stack. The model is accessible across platforms, running on NVIDIA GPUs via CUDA and Apple devices through MLX, utilizing custom low-bit kernels. |