What it does

EdgeML is a collection of training algorithms and inference runtimes from Microsoft Research India designed to squeeze machine learning onto severely resource-constrained devices. The goal is models that occupy mere kilobytes, run with low latency, and consume minimal energy — no cloud connection required. It ships with TensorFlow and PyTorch implementations, plus C and C++ inference code that can be further quantized to fixed-point arithmetic.

The interesting bit

The library doesn’t just optimize existing architectures; it grows its own. Bonsai is a shallow tree-based classifier, ProtoNN is a prototype k-NN method, and FastGRNN is literally named after what it does: Fast, Accurate, Stable and Tiny Gated RNN cells. There’s even SeeDot, a dedicated compiler that converts floating-point models to fixed-point for microcontrollers that lack an FPU.

Key highlights

• Bonsai and ProtoNN: custom classifiers designed from scratch for tiny footprints
• FastRNN / FastGRNN: RNN cells engineered for size and stability, not just accuracy
• EMI-RNN: extracts critical signatures from time-series so RNNs can stop early
• SeeDot: floating-point to fixed-point quantization tool with its own PLDI ‘19 paper
• GesturePod: a working gesture-recognition pipeline for actual microcontrollers
• Dual TensorFlow and PyTorch packages with matching C reference implementations

Caveats

• The README notes v0.4 and points to ELL for optimized ONNX binaries, suggesting this library stops at the “research code” stage for some deployment paths
• Several algorithms (Bonsai, ProtoNN) have C++ training code; others appear to be Python-only for training with C inference

Verdict

Worth a look if you’re building ML for actual sensors and need to prove it can run without a cloud umbilical cord. Skip if your “edge” is a Raspberry Pi with a GPU — this is for the truly silicon-starved.