Moonshot AI's Kimi Linear Cuts KV Cache 75%, Delivers 6x Throughput at 1M Context
The Kimi Team's hybrid attention architecture outperforms full attention while slashing memory costs across short and long contexts.
Moonshot AI's Kimi Team has unveiled Kimi Linear, a hybrid attention architecture that delivers up to 6x decoding throughput at 1 million token contexts while reducing KV cache memory usage by 75% compared to full Multi-Head Latent Attention (MLA).
The breakthrough centers on Kimi Delta Attention (KDA), which extends Gated DeltaNet with finer-grained channel-wise gating to more effectively leverage finite-state RNN memory. The Kimi K3 model deploys 69 KDA layers alongside 24 Gated MLA layers in its 93-layer text backbone—unlike previous large Kimi models that relied solely on MLA layers.
Performance Across Regimes
Kimi Linear outperforms full attention across short-context, long-context, and reinforcement learning scaling regimes—a significant claim given that linear attention methods have historically traded expressivity for efficiency. The pre-trained model features 3 billion activated parameters within a 48 billion parameter total, demonstrating that sparse activation can maintain quality while dramatically improving inference efficiency.
The architecture addresses a primary bottleneck in large language model scaling: KV cache growth. Standard Transformers exhibit quadratic complexity in sequence length, making million-token contexts prohibitively expensive on conventional hardware. Kimi Linear's linear complexity enables these extreme context windows on standard infrastructure, a critical enabler for agentic workflows and deep document analysis.
Open Release
The Kimi Team has open-sourced the KDA kernel in the FLA repository, vLLM implementations, and model checkpoints—including both Base and Instruct versions—on Hugging Face. The decision to release production-ready implementations alongside the research suggests confidence in the architecture's readiness for real-world deployment.
Context and Implications
Moonshot AI's Kimi series has previously distinguished itself through massive context window support, including 128k token models. Kimi Linear represents the next evolution: maintaining high expressivity while removing the memory bottleneck that has constrained practical long-context deployment.
If linear attention can operate as a drop-in replacement without sacrificing quality, the implications extend beyond Kimi itself. The approach could enable significantly faster inference and million-token contexts across the broader LLM ecosystem, potentially reshaping how developers architect applications that depend on extensive context retention.
The release positions Moonshot AI as a leader in efficient attention research, competing directly with other linear attention approaches while claiming superior performance across all evaluated regimes.