vLLM 0.32 adds /v1/systemone endpoint for in-engine decision routing

vLLM 0.32 introduced the /v1/systemone endpoint (PR #59299) that evaluates choice, boolean and scoring queries inside the engine instead of generating JSON over HTTP. This keeps KV and prefix caches warm, avoiding repeated prefill and routing latency for agent pipelines.

DEV Community

Open source

vLLM 0.32 adds /v1/systemone endpoint for in-engine decision routing

The /v1/systemone endpoint evaluates choice, boolean and scoring queries in-engine
Decisions skip autoregressive decode and speculative token buffers
Prefix tokens stay pinned in PagedAttention with 100% prefix cache hits
The release also adds NVFP4 KV cache and built-in Hugging Face tool-call and reasoning parsers

#VLLM#HuggingFace

vLLM 0.32 adds /v1/systemone endpoint for in-engine decision routing

vLLM 0.32 stops letting any website call your model server

OpenAI and Microsoft launch typed decision APIs for AI agents

vLLM Semantic Router Releases Decision 3.0 Open Multimodal Decision Models

TaichuAI Open-Sources ZDTaichu5.0-9B Spatial Multimodal Model