AutoML that shrinks neural nets for mobile deployment

What it does PocketFlow compresses and accelerates deep learning models with minimal human effort. You specify a target compression or acceleration ratio, and it automatically searches for hyper-parameters that produce an efficient model ready for deployment. Tencent AI Lab built it to shrink industrial neural networks for resource-constrained devices.

The interesting bit Instead of manually tuning pruning ratios or quantization levels, PocketFlow treats compression as an optimization loop: a learner generates candidate compressed models, while a hyper-parameter optimizer—using Gaussian Processes, TPE, or DDPG reinforcement learning—uses accuracy and speed as feedback to pick the next settings. It is essentially AutoML for model compression, augmented with network distillation and multi-GPU training to limit accuracy loss.

Key highlights

• Supports channel pruning, weight sparsification, and both uniform and non-uniform quantization
• Uses reinforcement learning (DDPG) to find per-layer pruning ratios and sparsity levels, which the README shows can outperform uniform compression on MobileNet-v1
• Can quantize 32-bit MobileNet models to 8-bit with no accuracy loss—and sometimes a slight gain—while cutting inference time by roughly 2.2× to 2.5× on a Snapdragon 845 CPU
• Augments training with knowledge distillation from the original uncompressed model to soften the accuracy hit
• Enables multi-GPU distributed training across all learner algorithms to speed up the search

Caveats

• The README only demonstrates ImageNet classification results for ResNet and MobileNet, despite mentioning speech recognition and NLP in the introduction
• One quantization learner explicitly requires TensorFlow APIs, and the README does not clarify whether PyTorch or other frameworks are supported
• A duplicated image path (docs/docs/pics/) and a typo (“fast fast-tuning”) are visible in the documentation

Verdict Worth exploring if you are deploying vision models to mobile and want automated compression without hand-tuning layer-wise sparsity. Look elsewhere if you need PyTorch support or work primarily outside image classification, since the README leaves both questions unanswered.