L40S beats H100 by 3.6–6.8x on cost per BGE embedding

A benchmark rented RTX 5090, L40S, RTX A6000 and H100 PCIe GPUs and measured end-to-end throughput for BGE small, base and large at 128 and 512 tokens. The L40S at $0.54/hour was cheapest in all six cases: $0.032 per million 128-token passages with bge-small and $0.49 per million 512-token passages with bge-large. The H100 at $3.86/hour was fastest only for bge-base and bge-large, at 3.6–6.8x the L40S cost per passage.

• L40S: $0.032 per 1M bge-small passages (128 tokens), $0.49 for bge-large (512)
• H100 leads only on bge-base and bge-large, at 3.6–6.8x the cost per passage
• On bge-small the H100 trails the RTX 5090: 4,952 vs 6,415 passages/s
• Peak memory stayed at or below 2.3 GB even for bge-large at 512 tokens