AWS and Booking.com shrink 70B model to 1.1B and speed it up 10–20x

AWS and Booking.com won the ACL 2025 Industry Track best-paper award for a framework that compresses a 70B-parameter Llama-3.1 model to a 1.1B TinyLlama student while retaining over 90% of output quality, then runs it 10–20x faster. The pipeline combines knowledge distillation with Medusa speculative decoding and is deployed across three live Booking.com products.

• Distillation yields ~6x speedup, Medusa 2–3.6x, combined 10–20x
• 1.1B-parameter TinyLlama student keeps over 90% of teacher quality
• Validated with A/B tests on real Booking.com user traffic
• Paper chosen best of 421 submissions at ACL 2025 Industry Track