Cerebras details Weight-Streaming architecture for trillion-parameter models

Cerebras outlined how its Wafer-Scale Engine 3 sidesteps the memory wall for trillion-parameter models: weights live in an external DRAM pool scalable to petabytes and are streamed onto the wafer layer by layer. The approach avoids interconnect overhead in GPU clusters and simplifies the software stack.

• Model weights are stored in external DRAM scalable to petabytes
• WSE-3 packs roughly 800k–900k AI cores on a single wafer
• Gradients are streamed back to external memory for weight updates
• PyTorch and TensorFlow models run with minimal code changes