AMD MI355X Outpaces NVIDIA B300 in Cost-Efficiency for 2.8T Parameter Kimi K3
New deployment data shows AMD's high-VRAM hardware offers superior performance-per-dollar for massive frontier models.
Wafer has successfully deployed the Kimi K3 model on AMD MI355X GPUs, demonstrating that the hardware provides a higher performance-per-dollar ratio than NVIDIA's B300. While NVIDIA maintains a lead in raw speed, the results suggest that memory capacity is becoming the primary bottleneck for the industry's largest models.
According to data from Wafer, the MI355X achieved a peak aggregate throughput of 952 tok/s per node and 118 tok/s for a single stream. In terms of economic efficiency, the MI355X is estimated at 48 tok/s/$, significantly outperforming the B300's 33 tok/s/$ and the B200's 7 tok/s/$. This efficiency is driven largely by pricing; the MI355X is approximately 2.4x cheaper per GPU on average than the B300 and 1.7x cheaper than the B200.
The VRAM Bottleneck
The shift toward cost-efficiency is driven by the sheer scale of modern frontier models. Kimi K3 is a massive model with 2.8 trillion parameters, requiring more than 1.5TB of VRAM before any KV cache is even allocated. For hardware like NVIDIA's B200, fitting such a model often requires multi-node configurations, which introduces communication overhead via cross-node all-reduce operations.
In contrast, the MI355X features 288GB of VRAM per GPU. This high capacity allows massive models to fit on a single node (TP8), effectively offsetting some of the software and kernel advantages traditionally held by NVIDIA's CUDA ecosystem. To reach these peak performance levels, Wafer implemented specific software fixes for prefill optimizations and speculative decoding.
Industry Implications
These findings suggest that for the largest LLMs, high-bandwidth memory (HBM) capacity may become a more significant competitive moat than raw compute power or established software ecosystems. If AMD continues to close the software gap—as evidenced by Wafer's successful optimizations of sglang and AITER kernels—the cost-efficiency of AMD hardware could disrupt NVIDIA's dominance in the high-end inference market.
The Performance Gap
Despite the cost advantages, NVIDIA's hardware still leads in absolute power. B300 nodes outperform the MI355X in absolute aggregate throughput by approximately 1.65x, delivering 1,568 tok/s compared to the AMD node's 952 tok/s.
As the industry weighs the trade-off between maximum possible throughput and the cost of scaling, the MI355X presents a viable alternative for operators of trillion-parameter models. The primary question remaining for the market is whether the "CUDA moat" is truly eroding in the face of overwhelming hardware memory requirements.