← all repositories

dleemiller/WordLlama

A lightweight NLP toolkit for similarity, ranking, deduplication, and clustering using LLM token embeddings.

1.5k stars Python RAG · SearchData Tooling
WordLlama
Velocity · 7d
+2.0
★ / day
Trend
steady
star history

WordLlama is a fast, lightweight NLP toolkit that operates on token embeddings from LLMs. It provides CPU-optimized functionality for fuzzy deduplication, semantic similarity computation, document ranking, clustering, and text splitting. The toolkit supports model2vec static embeddings and achieves competitive results on the MTEB benchmark.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.