Microsoft triples enterprise agent reliability with RL on benchmark graders

Microsoft Research and Hugging Face showed that the executable database-state checks used to grade the ThinkingBox benchmark can double as reinforcement learning rewards. Fine-tuning Qwen3.8-27B raised tasks passed on all 20 attempts from 38 to 107 of 507 and cut cost per dependable task by 67% to about $8.

• Pass@1 rose from 51.70% to 60.78% across 10,140 attempts
• Tasks passed on all 20 tries climbed from 38 to 107 of 507
• Cost per dependable task fell from $24.36 to about $8.00
• Training used 50 GRPO steps on the open Qwen3.8-27B model