End-to-end AI pilot that actually flies (sometimes)

What it does

This repo contains Python scripts for autonomous drone flight on ArduCopter-based quadcopters. The headline feature is an end-to-end neural network that ingests sensor data and outputs joystick commands directly. A second, fully working system tracks and follows people using a MobileNet detector, a LiDAR rangefinder, and PID controllers. Both run on a Jetson Nano at the edge—no cloud required.

The interesting bit

The end-to-end approach skips the traditional robotics pipeline of perception → mapping → planning → control, instead letting a single model learn to map raw inputs to stick movements. The README shows a visualization comparing AI pilot commands (red dots) against human commands (blue dots) during real flight. Meanwhile, the person-following system is the pragmatic sibling: proven, tested, and already airborne.

Key highlights

• End-to-end neural pilot trained and evaluated against human pilots on real flights
• Person-following stack: MobileNet detection + TF Luna LiDAR + dual PID controllers for yaw and roll
• Full edge deployment on Jetson Nano with TensorRT-optimized models
• SITL simulator support for testing without risking hardware
• Debug tooling included: PID data plotter for tuning flights

Caveats

• The end-to-end training pipeline and tooling are no longer open source; only pre-trained models will be released
• The “Q to exit and auto-land” shortcut does not work in real flight—you must manually switch to Loiter and land yourself
• All code is explicitly labeled experimental; the authors disclaim responsibility for crashes

Verdict

Worth a look if you’re building autonomous drones on ArduPilot and want a working reference for person tracking, or if you’re curious about end-to-end learned pilots in constrained embedded environments. Skip it if you need reproducible training pipelines or production-grade safety guarantees.