llm-d hits 1.0 as Kubernetes tackles idle GPUs for AI

CNCF sandbox project llm-d reached v1.0, bridging PyTorch developers with the production cloud-native stack. AWS and HPE also outlined automation for pod restarts and scheduler-autoscaler coordination to stop idle GPUs from stalling AI workloads.

• llm-d 1.0 launched at PyTorch Conference North America
• IBM test showed 11.5x lower inference cost with llm-d
• Cache reuse reached about 98% in IBM's demonstration
• AWS proposed APIs to link scheduler and autoscaler