Datalab launches OmniExtractBench to fix bias in extraction benchmarks

Datalab released OmniExtractBench, an open benchmark for structured document extraction: 620 PDFs from four suites, one deterministic scorer with six auditable verdicts. Datalab's accurate mode leads at 93.85% accuracy, with Reducto deep_extract v2 close behind at 93.47%.

• 620 documents pooled from LlamaIndex, micro1, Extend and Datalab suites
• Scorer assigns each value one of six auditable verdicts
• Datalab accurate leads at 93.85%, Reducto deep_extract v2 at 93.47%
• Code on GitHub under Apache 2.0, data on Hugging Face under CC BY 4.0