DeepGrove AI's Maple-Preview Hits Triple-Digit Token Speeds on Mobile
The 20B MoE model uses a native ternary-weight architecture to replace complex matrix multiplications with simple additions for ultra-fast on-device AI.
DeepGrove AI has released Maple-Preview, an open-source reasoning large language model designed to bring high-performance inference to local devices. The release marks a significant shift toward ultra-low precision architectures that prioritize speed and efficiency over cloud-based monolithic scaling.
Maple-Preview is a 20B-A1B Mixture-of-Experts (MoE) model featuring 20.2 billion total parameters, with 1.49 billion active parameters per token. By utilizing a natively trained ternary-weight architecture, the model replaces traditional, computationally expensive matrix multiplications with simple additions. This reduction in arithmetic workload allows the model to operate at extreme speeds on consumer hardware: it achieves 127 tokens per second on an iPhone—roughly 13 times faster than the 1-bit Bonsai 27B model's 9.6 tokens per second—and 218 tokens per second on an M4 Mac mini with a checkpoint size of 5.31 GB. In a high-end benchmark, the model solved IMO 2024 Problem 1 at a rate of 281.5 tokens per second on a MacBook Pro (M5 Pro).
The Shift to Native Low Precision
The development of Maple-Preview arrives as the industry explores "ultra-low precision" models, such as 1-bit and ternary systems, to migrate AI workloads from the cloud to the edge. While many developers rely on post-training quantization—compressing a full-precision model after it is trained—DeepGrove AI argues for a different approach. According to the company, "the precision a model runs at should be the precision it learns at," suggesting that native training at low precision is essential to maximizing both performance and efficiency.
Implications for Edge AI
This leap in on-device performance pushes the Pareto frontier for memory-to-performance in edge AI. If 20B-class reasoning models can maintain triple-digit speeds on mobile hardware, it enables the creation of "always-active" personalized assistants. Such systems could potentially learn and adapt to user preferences locally, eliminating the latency associated with cloud round-trips and removing the privacy risks inherent in sending sensitive data to remote servers.
Technical Architecture and Outlook
Under the hood, Maple-Preview consists of 24 layers and 256 experts. To handle large-scale data, it employs a hybrid sliding-window and global attention mechanism, allowing it to manage a context window of 131,072 tokens. As the industry watches the viability of ternary weights, the primary remaining question is how these efficiency gains balance against raw reasoning accuracy compared to full-precision counterparts in complex, multi-step tasks.