StartLux open-sources Prism inference engine for decision models
Chinese team StartLux has released Prism on GitHub, an inference engine built specifically for decision models with bit-exact outputs. On a single H200 a short question finishes in 19 ms versus 53.1 ms before, with up to 5.3x speedup for three questions about the same material.
- Narrowing the output layer cuts computation to about 0.01% of the original
- On 231 JevBench questions Prism averages 41 ms versus 75.6 ms for SGLang
- Four H200 cards in data-parallel mode handle 121 requests per second
- The int8 build scored 92.94% accuracy versus 92.97% for BF16
Read next
AI