The 505KB library that wants you to stop rewriting chunkers

What it does

Chonkie is a Python toolkit for splitting and ingesting text into RAG pipelines. It ships nine chunkers—from basic token splitting to semantic, neural, and LLM-based “Slumber” chunking—plus post-processing refineries, vector database handshakes, and an optional self-hosted REST API. The base package is advertised as 505KB and uses optional dependencies to keep the install minimal.

The interesting bit

Instead of treating chunking as a standalone utility, Chonkie wraps it in a Pipeline object that lets you chain chunkers, overlap refineries, embedding generators, and database exporters with async support. There is even a SIMD-accelerated FastChunker that the README claims processes bytes at over 100 GB/s, which either solves your throughput problem or gives you a fun number to quote in standup.

Key highlights

• Nine chunking strategies including recursive, semantic, code-aware, and agentic/LLM-based splitters
• Native handshakes for eight vector databases including Chroma, Pinecone, Qdrant, Weaviate, Elasticsearch, MongoDB, PostgreSQL/pgvector, and Turbopuffer
• Optional REST API server with SQLite-backed pipeline storage and interactive documentation
• Async pipeline execution for high-throughput batch processing
• Out-of-the-box support for 56 languages and 32+ integrations across tokenizers and embedding providers

Caveats

• The chonkie[all] install is explicitly labeled “Not recommended for production environments” in the README
• Most vector DB handshakes, semantic chunking, and neural chunking require optional dependencies, so the base package is narrower than the full feature matrix implies

Verdict

Use this if you want a modular alternative to writing yet another custom text splitter for RAG. Look elsewhere if you need deep document parsing or heavy ETL; Chonkie stays narrowly focused on chunking and lightweight ingestion.