d-Matrix AI Accelerator Hits 100 TB/s Memory Bandwidth
New 3D architecture stacks a TSMC 4nm compute die directly on custom DRAM to shatter the memory wall.
d-Matrix has introduced a new AI accelerator designed to shatter the memory bottleneck currently limiting large-scale model performance. Unveiled at Hot Chips 2026, the hardware achieves a memory bandwidth of 100 TB/s per card.
The accelerator's performance is driven by a specialized 3D architecture. The system features a compute die manufactured using TSMC's 4nm process, which is bonded face-to-face directly onto a custom-designed DRAM die. This physical integration is made possible by a highly dense 36-micron pitch for the interconnects between the compute and memory layers, allowing for unprecedented data transfer speeds.
Overcoming the Memory Wall
This architectural shift addresses a persistent industry challenge known as the "memory wall." In traditional AI hardware, the speed at which data moves between the processor and memory often lags behind the processor's actual computing power, creating a performance ceiling. While High Bandwidth Memory (HBM) has been the standard solution for GPUs and NPUs, HBM is still constrained by the physical distance and the density of the interconnects between the memory stacks and the main processor.
Implications for AI Inference
By placing the DRAM directly beneath the compute die, d-Matrix significantly reduces latency and increases the volume of data that can be processed simultaneously. This approach is particularly critical for the execution of massive Large Language Models (LLMs). Because AI inference is primarily limited by memory bandwidth rather than raw compute cycles, this design could allow for the more efficient deployment of the world's largest models by eliminating the primary data movement bottleneck.
The Path Forward
As the industry seeks ways to scale LLMs without exponential increases in power and physical footprint, 3D stacking of custom memory represents a potential shift away from generic HBM implementations. While the technical specifications presented at Hot Chips 2026 demonstrate a massive leap in bandwidth, the industry will now be watching for real-world benchmarks and the scalability of the face-to-face bonding process in high-volume manufacturing. This transition toward tighter integration suggests a future where memory is no longer a peripheral component but a core part of the compute fabric itself.