chiprook
← AI
AIOctober 30, 2026, 06:49

Audit: 'pick the longest answer' scores 52.7% on MMLU international law

An MV2/Munim audit found that a question-free 'pick the longest option' heuristic scores 28.05% on MMLU versus 25% chance. The edge is uneven: 23 of 57 subjects show a lift of 5+ points, reaching 52.7% on international law but falling below chance (17.7%) on public relations.

Audit: 'pick the longest answer' scores 52.7% on MMLU international law
#MMLU#OpenAI
Read next
AI

Audit: picking the longest answer scores 43% on OpenBookQA vs 25% chance

AI

x402 audit: 69 of 2,695 checked paid API answers were wrong

AI

Audit finds answer leakage in OpenBookQA, SciQ and QASC annotations

AI

VIDRAFT's Darwin-180B-RSI tops MMLU-Pro and GPQA Diamond