IBM: learned guidelines lift GPT-4.1 to 67.5% Pass^4 on τ²-bench
IBM Research reported that ALTK-Evolve, which distills an agent's own training runs into short behavioral rules appended to its system prompt, raised a stock GPT-4.1 retail agent to 67.5% Pass^4 on τ²-bench, up from 57.5% at baseline. The method needs no weight updates or human annotation, though per-conversation cost rose 65% in retail.
- Retail: 67.5% Pass^4 vs 57.5% baseline, a 10-point gain
- Airline with gpt-5.5: 70% Pass^4 vs 55% baseline, +15 points
- Retail conversation cost rose from $0.084 to $0.139 (+65%)
- Assistant turns fell from 10.59 to 10.16 in retail
Read next
AI