What it does DeepDetect is a C++ server that exposes training and inference through a JSON REST API. It wraps multiple backends—Caffe, PyTorch, TensorFlow, XGBoost, TensorRT, NCNN, Dlib, T-SNE—so you can swap between them without rewriting client code. It handles images, text, CSV, time series, and sparse SVM data, with built-in connectors for on-the-fly augmentation and preprocessing.

The interesting bit The real value is the backend matrix: Caffe covers almost everything (training, inference, segmentation, autoencoders), while TensorRT and NCNN handle GPU and ARM embedded inference respectively. It’s an admission that no single framework wins—so the project orchestrates them like a grumpy but efficient union steward.

Key highlights

• Supports training on CPU/GPU for Caffe, Caffe2, PyTorch, XGBoost, and T-SNE; inference adds TensorRT, NCNN, TensorFlow, and Dlib
• Built-in similarity search via neural embeddings (Annoy, FAISS)
• No database dependency—everything lives on the filesystem
• Docker images, AWS AMIs, and Kubernetes Helm charts available
• Swagger-generated API docs and clients in Python, JavaScript, Java, plus scikit-style bindings

Caveats

• Caffe is still the workhorse; newer backends like PyTorch and TensorFlow have patchy coverage (no TensorFlow training, no PyTorch segmentation or object detection per the matrix)
• The README claims C++11; the repo description says C++14—minor drift
• Some features exist as “templates” rather than proven paths; check the functionality matrix before betting on a backend

Verdict Good fit if you’re maintaining legacy Caffe pipelines or need to deploy the same model to both server GPUs (TensorRT) and ARM edge devices (NCNN). Skip if you’re all-in on modern PyTorch or just need a quick Python microservice—this is heavier machinery.