Receipts at 559 img/s, no vision-language model required

What it does

TurboOCR is a C++ document parser that runs entirely on a local NVIDIA GPU. It converts images and PDFs into structured text, using PP-OCRv6 for detection and recognition, then optionally adds layout analysis, table extraction to HTML, and formula recognition to LaTeX, all emitted as reading-order Markdown. The whole pipeline is served over HTTP and gRPC behind a single multi-stream CUDA/TensorRT engine.

The interesting bit

Instead of routing documents through a heavy vision-language model, the project stitches together a stack of small, specialized PaddlePaddle models—detection, recognition, layout, tables, formulas—and runs them in-process with hand-written C++ decoders. The result is a fully offline parser that the README says reaches 559 images per second on receipts and roughly 20 pages per second on full structured documents, versus about one page per second for VLM-based alternatives.

Key highlights

• Three accuracy tiers (tiny, small, medium) using PP-OCRv6, covering Latin, Chinese, and Japanese; retained PP-OCRv5 recognizers handle Arabic, Cyrillic, Korean, Thai, and Greek.
• Layout analysis uses PP-DocLayoutV3 across 25 classes, with class-aware XY-cut for reading order.
• Table and formula recognition are strictly opt-in per request, so the default path pays no overhead for stages you do not need.
• Ships as a Docker image with TensorRT engines that auto-build on first startup and cache for subsequent runs; exposes Prometheus metrics natively.
• PDF pages render and OCR in parallel, with optional auto-rotation.

Caveats

• First startup builds TensorRT engines from ONNX, which the README says takes about 90 seconds on an RTX 5090 and up to an hour on older GPUs; during this window the server refuses connections.
• Full pipeline layout plus tables plus formulas needs roughly 8 GB of VRAM, and each extra pipeline replica adds another full set, so smaller cards must run lean.
• The 559 img/s figure is for receipts on an RTX 5090; dense documents drop to 200+ img/s, and enabling higher accuracy levers like larger models or higher detection resolution cuts throughput further.

Verdict

Worth a look if you are running high-volume document ingestion on Linux servers with modern NVIDIA hardware and want structured output without cloud APIs or VLM latency. Skip it if you are on Windows, CPU-only, or need the last few accuracy points that vision-language models provide.