Jim Clifford, Jacob Polay, and Jessica Jack, University of Saskatchewan, and Mark Humphries and Lianne C. Leddy, Wilfrid Laurier University, have posted Reading the Archive by Machine: An OCR Benchmark for Historians, 1612–1921, in Working Papers in Critical Search:
A benchmark of six OCR systems (Tesseract, olmOCR 2, Chandra 2, Infinity Parser 2, GLM-OCR, and Gemini 3.5 Flash) on human-transcribed archival documents spanning 1612–1921: early-modern print, nineteenth-century newspapers, full multi-column pages, and handwriting. The free, open models a historian can run on their own hardware have caught up with the paid commercial service on printed sources — level on clean newspaper print, ahead on complex multi-column pages — leaving the commercial model a real lead only on difficult handwriting. Because accuracy has converged, the choice between tools is now practical rather than qualitative: it turns on layout fidelity and on speed and cost. We report per-content-type results, characterise each tool’s failure profile (olmOCR silently modernizes archaic spelling and collapses on multi-column pages; Gemini refuses some pages outright), and argue for a tiered workflow that transcribes a collection with a fast open tool and spends the paid model only on the pages that reward it. Every number is produced by the harness in the paper’s repository, and the expandable transcription panels are generated from the same result files, so the prose and the evidence cannot drift apart. This is Version 1.0 of a paper we intend to keep current as new models are released.--Dan Ernst
