PrismML Aims to Revolutionize AI Accessibility with Compact LLMs
Startup PrismML introduces Bonsai 2, a highly compressed model for easy smartphone integration.
The Full Story
PrismML, a burgeoning AI lab founded by researchers from Caltech, aims to transform how users interact with AI through its newly released Bonsai 2 model. Unlike traditional large language models (LLMs) that often require extensive computational resources, PrismML is pioneering a new wave of compact LLMs designed to function efficiently on personal devices like PCs and smartphones. Bonsai 2 compresses the popular open-source Qwen3.8 model from Alibaba down to just 5.9 GB, marking a staggering reduction in size of 9 to 10 times compared to its original version.
CEO Babak Hassibi, a Caltech professor noted for expertise in compression technologies, asserts that this approach allows significant performance for high-capacity models without compromising quality. Impressively, Bonsai 2 maintains 98% of the benchmark scores set by its larger counterpart, a notable increase from its predecessor which only achieved 95%. Already, the first version of the Bonsai model has found its way onto more than 11 million downloads, with subsequent models quickly gaining traction as well.
The concept of reducing the memory footprint while preserving functionality represents a significant shift in the AI landscape, indicating a move towards democratizing AI tools for broader accessibility. Rather than relying on hefty infrastructure, users can now harness the power of advanced AI directly on their personal devices. By simplifying the data storage of models—transitioning from traditional 16-bit weights to a ternary weighting system, which uses only +1, -1, or 0—PrismML significantly reduces the required storage space.
Though the journey toward achieving perfect benchmark equivalence may still carry some uncertainty, Hassibi emphasizes that the practical performance difference might not impact actual use cases significantly. “Natural language processing tasks often do not reflect directly on benchmarks, and a slight degradation in benchmark performance may not affect the model’s usability in practical scenarios,” he notes. Future plans for PrismML involve applying their compression techniques to even larger models, potentially in the hundreds of billions of parameters range. According to Hassibi, as model sizes increase, so does the capacity for effective compression without sacrificing intelligence.
Ion Stoica, a key adviser and co-founder of Databricks, highlighted the potential ramifications of this technology in providing advanced AI capabilities to end-users while ensuring privacy and data protection. “With intelligence available right at your fingertips, running on the devices you already own, the landscape of user interaction with AI will irreversibly change,” Stoica remarked. This innovative approach could pave the way for a new era where powerful AI tools are more accessible and integrated into daily life, further dismantling the barriers that have historically restricted such technology to only the largest corporate infrastructures. As PrismML continues to enhance its offerings, the impact on how we interact with AI tools remains a critical topic of exploration.
Why It Matters
PrismML's innovative model design makes advanced AI tools accessible on everyday devices, revolutionizing user interaction with AI and enhancing privacy and usability for individuals and businesses alike. Through effective compression, users can leverage AI directly on their devices without significant performance loss.
What's Next
PrismML plans to work on even more substantial models in the following months, aiming for performance improvements and facilitating high-capacity AI applications for personal devices by utilizing compression techniques effectively on larger frameworks.