NVIDIA NVFP4 boosts prompt processing by 46% on Blackwell

Nvidia introduced NVFP4, a 4-bit floating-point format for Blackwell GPUs that compresses model weights and cuts VRAM usage. Prompt processing speeds up by an average of 46%, while reply generation gains only about 1% and overall throughput improves 9–12%. The format requires native Blackwell support and an optimized software stack.

• Prompt processing on Blackwell speeds up by an average of 46%
• Reply generation gains only ~1%, overall throughput 9–12%
• 4-bit weight compression cuts VRAM use and allows longer contexts
• Older GPUs without native NVFP4 support see no gains