Swap PyTorch bloat for ONNX speed on Lambda

What it does FastEmbed is a Python library that turns text, images, and mixed documents into vectors using a growing roster of state-of-the-art models. It handles dense text embeddings, sparse SPLADE++ representations, late-interaction ColBERT matrices, CLIP-style image vectors, ColPali multimodal embeddings, and cross-encoder rerankers, all through a single API. The default model is BAAI/bge-small-en-v1.5, and it runs entirely on the ONNX Runtime.

The interesting bit The project’s core wager is that most embedding jobs don’t need a GPU or gigabytes of PyTorch dependencies. By standardizing on ONNX, it keeps the footprint small enough for AWS Lambda while still claiming to beat OpenAI Ada-002. It also exposes data parallelism for large batches and lets you register custom ONNX checkpoints if your preferred model isn’t bundled.

Key highlights

• Runs on CPU out of the box; a separate fastembed-gpu package adds CUDA 12.x support.
• Supports query and passage prefixes for retrieval-tuned models.
• Covers dense, sparse, late-interaction, image, and multimodal embeddings plus rerankers.
• Integrates directly with the Qdrant vector-database client.
• Allows adding custom models by supplying an ONNX file, pooling type, and dimension.

Caveats

• BM42 sparse embeddings are teased in the README but remain wrapped in a commented-out placeholder with a dead “ToDo” link, so that modality is not yet available.

Verdict Reach for FastEmbed if you need a wide-spectrum embedding toolkit that fits inside a serverless runtime. Stick with your current stack if you are already happy lugging PyTorch around.