A 2015-era CNN beats GPT-6 Astra and Claude Fable 5.1 on skin cancer

Dermatologist Magnus Lynch tested an Inception v3 CNN against frontier LLMs on 200 biopsy-proven lesions from the MILK10k dataset. The CNN named the correct diagnosis 62% of the time versus 42% for both GPT-6 Astra and Claude Fable 5.1. At its default threshold, however, the CNN would have discharged 18 of 30 melanomas.

• CNN got 125 of 200 diagnoses right; the LLMs got 84 and 85
• Allowing a second choice lifted CNN to 173 of 200 versus 134 and 138
• CNN caught 13 of 30 melanomas first, GPT-6 caught 8, Claude 11
• The CNN was fine-tuned on 68,011 images in about 20 minutes on one H100