Baking NMS Into YOLOv8 TensorRT Engines

What it does

This repo is a deployment accelerator for YOLOv8 on NVIDIA hardware. It provides export scripts that fold bbox decoding and NMS into the ONNX graph, then build a TensorRT engine that ingests images and emits final detections without extra Python post-processing. It also ships C++ inference code and covers segmentation, pose, classification, and oriented bounding boxes alongside standard detection.

The interesting bit

Most deployment tutorials leave NMS as a post-processing homework assignment; this project bakes it into the engine via ONNX plugins, turning the model into a self-contained detection appliance. It also targets edge deployment with DeepStream and Jetson recipes, which is where TensorRT actually earns its keep.

Key highlights

• End-to-end ONNX export that embeds NMS and bbox decoding directly in the graph.
• Python and C++ inference paths, plus an experimental PyTorch-free mode using cuda-python or pycuda.
• Deployment recipes for DeepStream and Jetson (specifically tested on Jetson-NX 4GB).
• Covers multiple YOLOv8 tasks: detection, segmentation, pose, classification, and oriented bounding boxes.
• Includes an engine profiling script to measure whatever speedups you actually get.

Caveats

• The TensorRT API build path explicitly does not support YOLOv8-seg models at the moment.
• The PyTorch-free inference option is available but the README admits its performance is “much worse.”
• No benchmark numbers are published in the README, so you will need to run the included profiler to verify gains on your own hardware.

Verdict

Grab this if you are shipping YOLOv8 on NVIDIA GPUs or Jetson and want post-processing fused into the engine without writing the plugin boilerplate yourself. Look elsewhere if you are still training or tweaking architectures—this is strictly an export-and-run toolkit.