A C++ AI model zoo that refuses to hide your inference engine

What it does

Lite.ai.toolkit is a C++ collection of ready-made inference implementations for popular vision and generative models—think YOLO variants, face detection and recognition, segmentation, matting, and Stable Diffusion. It packages these behind a consistent lite::cv::Type::Class syntax and ships with pre-converted weights for ONNX, MNN, TNN, and NCNN. By default the only hard dependencies are OpenCV and ONNXRuntime, though you can opt into MNN or TensorRT backends.

The interesting bit

The twist is that it deliberately refuses to be a framework. The README states the goal is explicitly not to abstract on top of MNN or ONNXRuntime, so you can mix its high-level model classes with raw Ort::Session or MNN::Interpreter calls in the same translation unit. It is essentially a curated model zoo with matching plumbing rather than a black-box wrapper.

Key highlights

• The README advertises 300+ C++ implementations and 500+ weights, covering detection, segmentation, face analysis, and generative tasks.
• Supports ONNXRuntime, MNN, TensorRT 10.x, TNN, and NCNN backends.
• Minimal default footprint: only OpenCV and ONNXRuntime are required out of the box.
• Prebuilt Linux libraries and a dedicated model zoo with downloadable weights are available.

Caveats

• Linux-only for recent releases; the build script is explicitly tested only on Ubuntu 20.04.6 LTS.
• The original author has shifted focus to LLM/VLM inference; day-to-day maintenance is now handled by another contributor.
• TensorRT support demands a specific stack: TensorRT 10.x and CUDA 12.x or later.

Verdict

Worth a look if you want ready-to-compile C++ inference for a wide range of vision models without adopting a heavy framework. Skip it if you need first-class Windows or macOS support, or if you are looking for a tightly integrated, opinionated inference engine.