When the first version of Nemotron OCR failed on non-Latin scripts, the team faced a classic data engineering problem: scaling high-quality, annotated training data across multiple languages was economically impossible. Their solution was architectural: bypass manual annotation entirely by building a synthetic data pipeline.

The core system is a modified version of SynthDoG, engineered to generate 12 million training images with perfect ground truth. The pipeline's design choices are instructive. It produces hierarchical annotations—word, line, paragraph—with a relation graph for reading order, directly addressing a common gap in public OCR datasets. To ensure generalization, the system applies randomized augmentations at three levels: glyph effects, image distortions, and full-page modifications. Source text is sampled from the mOSCAR web corpus to preserve natural character distributions, while an open-source font pool provides stylistic variety.

This data generation strategy enabled a significant model redesign. Nemotron OCR v2 uses a single, unified architecture for five languages, eliminating the need for pre-detection or language-specific models. The FOTS-inspired design shares a RegNetX backbone across the detector, recognizer, and relational model. This feature reuse is the key to its throughput, processing 34.7 pages per second on an A100. A critical trade-off was shifting from word-level to line-level recognition for the multilingual variant, a necessary adjustment for languages like Chinese and Japanese that lack clear word boundaries.

The results are clear. On synthetic benchmarks, error rates (NED) for languages like Korean and Russian dropped from above 0.9 to under 0.05. The pipeline is generic; adding a new language requires only fonts and source text, not architectural changes. The model and the 12-million-image dataset are publicly available for use and extension.

Source: Hugging Face Blog