Cloudflare Reclaims 100 TB of RAM by Optimizing Consistent Hashing

Cloudflare cut the memory footprint of its internal load-balancing service by more than 100 TB by optimizing the Rust implementation of consistent hashing in pingora-ketama. The team reduced hashes per backend by about 90% while preserving load-balancing accuracy and shrank hash-ring entries from 8 to 6 bytes.

• Memory savings exceeded 100 TB through compacted hash-ring structures
• Hashes per server were cut by 90% without losing distribution accuracy
• The Point struct shrank from 8 to 6 bytes by packing the server index
• The new ring rolled out gradually to avoid origin traffic spikes