METASTONE launches TokenFleet to schedule LLM inference across GPU clusters

METASTONE has introduced TokenFleet, a distributed scheduling framework for large language model inference across GPU clusters and data centers. The company reports that predicted-latency routing on a tuned GLM5.3 deployment cut P95 time-to-first-token by 32.4% and P50 by 71.3%. The figures have not been independently verified.

• TokenPilot picks an instance weighing cached prefix reuse against queue wait
• FleetMind routes across sites based on deployed model version and data rules
• SLOMax separates interactive requests from long-context and batch jobs
• P95 time-to-first-token fell 32.4% and P50 fell 71.3% in tests