vLLM 0.30.0 weight cache can serve another checkpoint's weights

In vLLM 0.30.0 the weight cache (load_format="ipc_cache") fingerprints checkpoints only by shard file names and safetensors headers, not tensor values. A base model and its fine-tune with identical layout get the same key, so an engine can silently run on another checkpoint's weights.

DEV Community

Open source

vLLM 0.30.0 weight cache can serve another checkpoint's weights

Fingerprint hashes tensor names, shapes, dtypes and offsets, but not values
On an RTX 2070 an engine on a Qwen3-0.6B copy with zeroed model.norm.weight returned the original model's output
When layouts differ, the cache falls back to disk loading with a WeightCacheKey mismatch warning
A fix is proposed in PR #59648: fingerprint sampled tensor bytes instead of headers only

#VLLM#Qwen

vLLM 0.30.0 weight cache can serve another checkpoint's weights

Repacked QAT Gemma 4 on one TPU v5e: 12B serves at 675 tokens/s

QAT Gemma 4 26B-A4B on one TPU v6e: 15.6x KV cache, 1.9x throughput vs FP8

Lightbits Inferra KV Cache Engine Claims 16x Session Density and 10M-Token Contexts

Six Billion Requests Later: A Full Year of LLM Serving