BesselAI Fluid Computing Cuts AI Inference Latency 60%, Targets 200K GPUs

South Korean neocloud BesselAI unveiled its Fluid Computing platform, which raised the KV-cache hit rate for Upstage's Solar Pro 4 from 46% to 94% and cut output token latency from 35 ms to 14 ms. The company plans to scale from 7,000 GPUs to 200,000 by 2030.

Tech Times

Open source

BesselAI Fluid Computing Cuts AI Inference Latency 60%, Targets 200K GPUs

KV-cache hit rate rose from 46% to 94%, token latency fell from 35 ms to 14 ms
Peak load sustains up to 155 billion tokens per day on Solar Pro 4 inference
GPU fleet: 7,000 now, 10,000 by end-2026, 50,000 by 2027, 200,000 by 2030
Predictive fault detection catches over 80% of GPU failures before disruption

#BesselAI#Upstage#Nvidia

TokenRouter: token-level LLM routing cuts inference cost 18% and latency 31%

Hardware

Techies Are Building Home AI Rigs With GPUs Instead of Renting Cloud Compute

Hardware

BesselAI Fluid Computing Cuts AI Inference Latency 60%, Targets 200K GPUs

TokenRouter: token-level LLM routing cuts inference cost 18% and latency 31%

Techies Are Building Home AI Rigs With GPUs Instead of Renting Cloud Compute

High-end GPUs may be slowing China's AI compute expansion

Iterate.ai launches Lifeboat inference engine with confidential computing