Scene text recognition that ditches the standalone language model

What it does PARSeq is a scene text recognition system that reads text in images using a Vision Transformer. The repository serves as the reference implementation for the ECCV 2022 paper and bundles reproduced baseline models—ABINet, CRNN, TRBA, and ViTSTR—into what the authors call the Scene Text Recognition Model Hub. Pretrained weights are available through Torch Hub, and an interactive demo runs on Hugging Face.

The interesting bit The core trick is unification: one Transformer body acts as an ensemble of autoregressive decoders, a non-autoregressive decoder, and a bidirectional refiner simply by swapping its attention mask. Trained with Permutation Language Modeling, it predicts characters in arbitrary order from arbitrary context subsets, delivering language-model-like refinement without the dedicated overhead.

Key highlights

• Ranked #1 overall for scene text recognition on Papers With Code at the time of its 2022 release.
• Integrated into major OCR ecosystems including PaddleOCR and docTR.
• One Transformer handles context-free, context-aware, and bidirectional refinement inference without a separate language model.
• Pretrained weights available for PARSeq and reproduced baselines via Torch Hub.
• Recently updated for PyTorch 2.0 and Lightning 2.0 as of February 2024.

Verdict Researchers and practitioners building OCR pipelines should care, especially if they want strong accuracy without a separate language model. Casual users looking for a simple, drop-in text extraction tool might find the research orientation overkill.