A Kalman filter and a webcam walk into a self-driving car
A readable, from-scratch pipeline that marries off-the-shelf object detection with classic state estimation to track vehicles through video.

What it does
This repo wires a pre-trained SSD MobileNet (from TensorFlow’s Object Detection API) to a Kalman filter tracker for multi-vehicle detection and tracking in dashcam-style video. The detector spots cars, buses, and trucks; the tracker predicts where each bounding box should be next, matches predictions to fresh detections via the Hungarian algorithm, and smooths out noisy or missing frames.
The interesting bit
The author treats the four corners of a bounding box as an 8-dimensional state vector—position plus velocity for each corner—then runs the full predict-update cycle in plain NumPy. It’s a deliberate throwback to readable, tweakable code at a time when most tracking is buried inside opaque frameworks. The Hungarian (Munkres) assignment for matching detections to tracks is the unsung hero here; without it, the Kalman filter would be guessing at identities.
Key highlights
- Uses a lightweight COCO-pretrained SSD MobileNet for fast inference, then filters for vehicle classes (car, bus, truck) with confidence and aspect-ratio thresholds.
- Kalman filter state includes corner velocities under a constant-velocity assumption; measurement noise tuning is exposed and visualized.
- Detection-to-track assignment uses IOU cost and scikit-learn’s
linear_assignment(Hungarian algorithm) with min-hits and max-age gating for track birth and death. - Code is explicitly structured for swapping in other detectors or trackers—
detector.py,tracker.py, andmain.pyare cleanly separated.
Caveats
- Built on the older TensorFlow 1.x Object Detection API and scikit-learn’s deprecated
linear_assignment; modern users will need to port or pin dependencies. - The README trails off mid-sentence in the pipeline section, suggesting the repo may be unmaintained.
Verdict
Worth a look if you’re teaching (or re-learning) multi-object tracking fundamentals, or need a hackable baseline where you can swap the detector without touching the filter. Skip it if you want a batteries-included, production-ready tracker with modern PyTorch models and online updates.