chiprook
← AI
AIOctober 24, 2026, 06:33

Developer reproduces softmax and cross-entropy numerical bugs in NumPy

A developer verified on NumPy 1.26.4 a paper claiming two numerical bugs in standard softmax and cross-entropy implementations. In fp32 the loss and target gradient round to zero at a logit margin near 16, and distillation gradients hit an error floor of about 4.6e−08.

Developer reproduces softmax and cross-entropy numerical bugs in NumPy
#NumPy
Read next
Software

GNOME 50.5 released with numerous bug and security fixes

Security

Epic pauses product development to fix MyChart security bugs

AI

TypeSafe AI raises $40M seed for model that outputs numerical responses with probability estimates

AI

MIT's AI agent society experiment reproduced on four RTX 3090 GPUs