ML containers for people who'd rather not write Dockerfiles

What it does

Cog wraps machine learning models in standard Docker containers using a YAML configuration and a small Python class. It generates images with sensible defaults for Nvidia drivers, Python versions, and dependency caching, then exposes a RESTful HTTP inference server built in Rust. The container runs on your own hardware or on Replicate.

The interesting bit

The tool was built by the creator of Docker Compose and a former Spotify ML infrastructure engineer, so it treats CUDA compatibility as a problem to be hidden rather than memorized. Cog maintains its own mapping of valid CUDA, cuDNN, PyTorch, TensorFlow, and Python combinations and wires them together automatically.

Key highlights

• Replaces Dockerfile maintenance with a cog.yaml config that declares GPU support, system packages, and Python versions.
• Automatically resolves compatible CUDA/cuDNN/PyTorch/TensorFlow/Python stacks instead of forcing you to debug them.
• Derives an OpenAPI schema and a Rust/Axum HTTP API directly from your Python type hints.
• Runs anywhere Docker runs, including self-hosted infrastructure or Replicate.
• Supports macOS and Linux natively; Windows 11 requires WSL 2 and Docker.

Caveats

• It still needs Docker under the hood, plus Buildx if you are on Docker Engine rather than Docker Desktop.
• Windows support is explicitly limited to Windows 11 with WSL 2.

Verdict

Researchers and small teams who want to turn a Python model into a deployable HTTP API without writing Dockerfiles or memorizing CUDA compatibility tables. If your organization already has a mature, custom MLOps stack, Cog will feel like a convenient shortcut rather than a necessity.