AI Startups Pivot to Diffusion to Break the Autoregressive Bottleneck
Inception Labs and Google DeepMind are experimenting with parallel text generation to slash latency and compute costs.
A new wave of AI startups and research labs is attempting to dismantle the autoregressive paradigm that has defined large language models (LLMs) for nearly a decade. By shifting from token-by-token prediction to parallel generation, these developers aim to eliminate the primary speed and cost bottlenecks currently hindering the industry.
Inception Labs has entered this fray with the release of Mercury 2, a diffusion-based LLM. Unlike traditional models that predict the next word in a sequence, Mercury 2 utilizes a diffusion process to refine multiple tokens simultaneously. According to the company, this approach allows the model to be up to 10 times faster than traditional autoregressive models in its class. Independent tests have clocked the model at 1,196 tokens per second, while the company claims speeds around 1,000 tokens per second. Mercury 2 is currently available as a paid, closed-weight API.
The Shift from Transformers
Since the publication of the seminal "Attention Is All You Need" paper in 2017, the AI industry has relied almost exclusively on the transformer architecture. Most current LLMs are autoregressive, meaning they generate text sequentially. While effective, this method creates a linear dependency where each new token requires a full pass through the model, driving up latency and computational expense.
To solve this, researchers are adapting diffusion technology—the mathematical framework that powers image generators like Midjourney and video tools like Sora—and applying it to text. Stefano Ermon, cofounder and CEO of Inception, explains that while the system still utilizes a large transformer model, the key innovation is the ability to predict many tokens at the same time.
Industry Implications
If diffusion-based LLMs can maintain the intelligence of autoregressive models while delivering a tenfold increase in speed, the economics of artificial intelligence will shift fundamentally. The industry's primary metric would move toward maximizing intelligence per dollar spent on compute. This efficiency could enable high-reasoning AI to operate in real-time on wearables and in data centers where current latency and costs are prohibitive.
The Competitive Landscape
Inception Labs is not alone in this pursuit. Google DeepMind has also entered the space with DiffusionGemma, an open-weights diffusion-based text model built on the Gemma 4 26B Mixture-of-Experts (MoE) architecture. The emergence of both closed-API and open-weights diffusion models suggests a broader industry consensus that the autoregressive method may have reached a point of diminishing returns regarding efficiency.
What remains to be seen is whether these parallel generation techniques can scale to the most complex reasoning tasks without sacrificing the coherence and precision that autoregressive models provide. As more labs move beyond the standard transformer pipeline, the focus will shift to whether diffusion can handle long-form logic as effectively as it handles rapid-fire text generation.