Rust's ONNX Runtime wrapper that actually ships to users

What it does

ort is a Rust crate for running ONNX models—both inference and training—backed primarily by Microsoft’s ONNX Runtime C++ library. It wraps the foreign-function interface so you can load models exported from PyTorch, TensorFlow, Keras, or scikit-learn without dragging along a Python interpreter.

The interesting bit

The crate is deliberately not a from-scratch runtime; it’s a well-maintained wrapper that also plumbs into alternative pure-Rust backends. The README’s hardware-accelerator claim is vague on specifics, but the project tracks ONNX Runtime v1.26.0 and emphasizes on-device deployment—think trail cameras, mobile apps, and browser WASM—not just datacenter GPUs.

Key highlights

• Supports inference and training workflows, not just forward passes
• Claims compatibility with “almost any hardware accelerator” via ONNX Runtime execution providers
• Used in production by recognizable names: Hugging Face’s TEI, Google’s Magika, Ultralytics YOLO
• Cross-platform enough to appear in iOS, Android, Flutter, Unity, and WASM projects
• Active v2.0 rewrite with migration guide; not abandonware

Caveats

• The “other pure-Rust runtimes” link exists but no details are given in the README; it’s unclear how mature or complete that support is
• v2.0 is still at release-candidate stage (2.0.0-rc.12), so expect API churn

Verdict

Worth a look if you’re shipping Rust binaries that need to run ONNX models on edge hardware and want to avoid Python packaging hell. If you’re already comfortable with Python deployment or need a fully native Rust ML stack without C++ dependencies, this may not be your endpoint.