Lego + Raspberry Pi + TensorFlow = a self-driving toy car

What it does

A Lego Mindstorms NXT robot, wired to a Raspberry Pi 3, drives around a track using a CNN trained in TensorFlow. You collect camera images from the Pi, train on a real computer (the README explicitly warns: “don’t train the model in the Raspberry Pi!”), then deploy the checkpoint back to the Pi for live inference. Two modes run on the robot: regular driving, or debug mode, which saves every frame with prediction probabilities stamped on it.

The interesting bit

The project treats hyperparameter search as a first-class citizen. There are separate scripts to grid-search architectures, learning rates, and optimizers, each dumping results to a text file. It’s a surprisingly methodical pipeline for what is essentially a toy car — closer to a mini MLOps workflow than a weekend hack.

Key highlights

• Full stack: NXT-Python for motor control, TensorFlow 1.2.1+ for training, scp/tar for “data transfer”
• Data pipeline: raw images → pickle → numpy arrays → TFRecords, with augmentation and binarization options
• Simulation mode runs inference on saved track images before you risk the physical robot
• Python 2/3 split: master branch is Python 2; python3 branch exists but requires switching
• Travis CI badge and unit tests for both Pi connection and model correctness

Caveats

• README is frozen in TensorFlow 1.x era; no mention of TF 2.x or modern Keras
• The “don’t train on the Pi” warning suggests the hardware is exactly as limited as you’d expect
• No candidate images provided, so the track photo and GIF in the README are unavailable for preview

Verdict

Grab this if you want a complete, low-budget reference architecture for end-to-end tinyML robotics — or if you need to convince someone that a $35 computer and decade-old Lego can still teach CNN deployment. Skip it if you’re looking for ROS2, SLAM, or anything that handles real roads.