NVIDIA-Owned Groq Moves Next-Gen LPUs From SRAM to 3D Stacked DRAM

NVIDIA-acquired Groq reportedly plans to use 3D stacked DRAM instead of SRAM in its next-generation LPU chips, according to a semiconductor industry insider. Current LPUs pack 500 MB of SRAM at 150 TB/s and 1.20 PFLOPs FP8, while the shift to DRAM trades bandwidth for capacity as AI models grow.

• Current LPU: 500 MB SRAM, 150 TB/s, 1.20 PFLOPs FP8
• A 256-chip rack delivers 128 GB SRAM and 640 TB/s
• 3D stacking uses TSVs similar to HBM DRAM designs
• Groq prioritizes memory capacity over raw speed