AI agents turn one request into about 20, and Kubernetes is adapting

CNCF says Kubernetes is becoming the default base layer for AI agents: a single human request can trigger about 20 agent requests, and agents expect sub-200-millisecond responses while a new pod is only guaranteed to spin up in about a second. Emerging projects like llm-d for distributed LLM inference and Agent Substrate for large-scale agent deployments aim to close that gap.

• One human request can trigger about 20 agent requests
• Agents expect sub-200 ms responses; pods start in about a second
• llm-d handles distributed LLM inference in the cluster
• Agent Substrate is a runtime for large-scale agent deployments