Claude Code's eval grader can't see a no-break space
A Driftproof test found that AI graders in OpenEvals, Promptfoo, SkillEvaluator and Claude Code misjudge code containing a no-break space (U+00A0): 4 of 36 correct calls without a hint versus 34 of 36 when hidden characters are listed. Claude Code's grader scored 0 of 6 with Haiku 5.5 and 1 of 6 with Sonnet 5.5, but 6 of 6 once the character was written as \u00a0.
- Without a hint, graders made 4 of 36 correct calls; with a character list, 34 of 36
- Claude Code's grader scored 0 of 6 with Haiku 5.5 and 1 of 6 with Sonnet 5.5
- Writing U+00A0 as \u00a0 raised both graders to 6 of 6
- Fixes proposed in OpenEvals #293, Promptfoo #12005 and SkillEvaluator #242
Read next
Science