Treating retrieval like a buffet—40 datasets, 24 rerankers, one import

What it does

Rankify is a Python toolkit that bundles retrieval, reranking, and RAG into a single modular framework. It ships with 40 pre-retrieved benchmark datasets and exposes a one-line pipeline() API that can switch between search, rerank, and full RAG modes without rewriting boilerplate. Under the hood it supports dense retrievers such as BGE and ColBERT, more than two dozen reranking models, and pluggable generators.

The interesting bit

The project does not invent new retrieval algorithms; instead, it acts as a unified chassis for the current zoo of models. The most useful twist is the RankifyAgent, which uses an LLM backend to recommend the best retriever-reranker pair for your hardware and task, and the RankifyServer that turns the same pipelines into a REST API without extra wrapping.

Key highlights

• One-line pipeline API (pipeline("rag"), pipeline("search"), pipeline("rerank")) with swappable retrievers, rerankers, and generators
• 40 pre-retrieved benchmark datasets and support for 24+ reranking models and 7+ retrieval techniques
• RankifyAgent for AI-powered model selection based on task constraints and GPU availability
• Built-in server mode and integrations for LangChain and LlamaIndex
• Modular packaging with separate extras for the retrieval, reranking, and RAG stacks

Caveats

• ColBERT support requires manual GCC and environment variable wrangling that breaks the otherwise smooth install
• The dependency matrix is wide: full functionality expects PyTorch 2.5.1 with CUDA 12.4/12.6 and optional extras for each subsystem
• The visible README is front-loaded with quickstart snippets; deeper architectural rationale is referenced but not shown

Verdict

Use this if you need to benchmark or prototype across many retrievers, rerankers, and RAG configurations without writing glue code. Look elsewhere if your production stack already relies on a single, heavily optimized retrieval backend that you do not plan to swap.