Nvidia Unveils NVHBM to Boost Custom AI Accelerator Performance
The new custom high-bandwidth memory technology offers 30% more bandwidth and reduced power for NVLink Fusion partners.
Nvidia has introduced NVHBM, a custom high-bandwidth memory (HBM) base-die technology designed to optimize the next generation of AI accelerators. The technology is available to partners through the company's NVLink Fusion program, aiming to replace commodity HBM4e in custom silicon designs.
According to Nvidia and Tom's Hardware, NVHBM delivers up to 30% higher memory bandwidth per stack compared to standard HBM4e. The technology also improves energy efficiency, reducing HBM power usage by up to 15%. These gains are achieved by moving the memory controller directly into the base die of the HBM stack and utilizing a custom PHY. This architectural shift reduces the PHY and support area by as much as 67%, which in turn allows partners to allocate 25% to 30% more compute die area on the primary silicon die.
The Push for Custom Silicon
As hyperscalers increasingly develop their own custom AI accelerators, known as XPUs, they are hitting critical bottlenecks in memory bandwidth, package area, and power delivery. The NVLink Fusion program was created to provide the necessary IP and connectivity to integrate these third-party custom chips into Nvidia's rack-scale AI infrastructure. NVHBM serves as a specialized toolkit component, designed and validated alongside leading memory vendors to shorten the time-to-market for these semi-custom "AI factories."
Industry Implications
Memory bandwidth remains one of the primary constraints for the training and inference of large-scale AI models. By increasing this bandwidth while simultaneously lowering the power and physical footprint, Nvidia is enabling its partners to build more powerful custom silicon that remains within the strict thermal and power envelopes of modern data centers. This move strategically positions Nvidia's ecosystem as the central hub for AI infrastructure, maintaining its influence even as major cloud providers build their own proprietary hardware.
Future Adoption
Amazon's Annapurna Labs is the first announced partner to adopt NVHBM for future AWS infrastructure designs. Nafea Bshara, VP at Annapurna Labs, stated that the company looks forward to the collaboration to benefit future AWS designs. Industry observers will now watch for further adoption among other hyperscalers and the specific performance gains realized in the first wave of NVHBM-powered XPUs.