What it does

This is a do-it-yourself self-driving car built from a cheap toy chassis, Raspberry Pi, camera, and colored tape on the floor. You drive it manually with WASD keys over your homemade track to collect ~50,000 images, train an NVIDIA end-to-end CNN on a proper computer, then copy the .h5 model back to the Pi and let it steer itself.

The interesting bit

The project treats keyboard inputs as classification labels: filenames start with 0 for straight, 1 for left, etc., turning human reflexes into supervised training data without any explicit lane detection or path planning. It’s end-to-end in the original NVIDIA sense—raw pixels in, motor commands out.

Key highlights

• Uses NVIDIA’s 5-conv + 3-fc architecture, normalization layer included
• Full pipeline: zth_car_control.py → zth_collect_data.py → zth_process_img.py → zth_train.py → zth_drive.py
• ~30k–40k filtered images from ~60k collected; track is just colored tape twice the car’s width
• Hardware: Taobao toy car + Pi + battery pack + acrylic mounting, “具体方法，看手头的工具吧”
• Controls via Pygame over VNC; Keras model runs inference on the Pi

Caveats

• README warns about camera angle sensitivity and lighting issues, with no mitigation yet implemented
• Class imbalance is acknowledged as an open problem; transfer learning fine-tuning is listed as “正在进行” (in progress)
• Code appears to be mostly glue wiring Pi GPIO, Pygame, and Keras together

Verdict

Great weekend project if you want to feel the pain of real-world data collection and watch a toy car follow your living room floor. Skip it if you need reproducible benchmarks or anything resembling a safety system.