AI Professions benchmark: 1,500 graded runs of ChatGPT, Claude and Gemini on work tasks

AI Professions published an open benchmark of 1,500 graded runs: 100 prompts across ten professions, 5 models, 3 runs per prompt and 5 pass/fail criteria hashed before testing. The main finding is that 71% of round-one failures came from models adding content absent from the input, not from factual errors.

• 78 of 110 round-one failures were added content; only 24 were factual errors
• GPT-6 Astra scored 99% vs 92% for Claude Opus 5.5, but judges were cross-assigned
• Claude Opus dropped from 92% on English/Spanish to 79% on French/Portuguese
• Excel tasks scored 83–93%, with VLOOKUP causing 20 failures against XLOOKUP requirements