Sentence embeddings with an angle on state-of-the-art

What it does AnglE is a training and inference library for sentence embeddings built around an ACL 2024 paper. It wraps both BERT-family encoders and decoder-only LLMs like LLaMA and Mistral into one pipeline for fine-tuning and deploying embedding models. The project also maintains pretrained checkpoints that have ranked first on the MTEB leaderboard, including models produced by the commercial team Mixedbread.

The interesting bit The framework centers on an angle-optimized loss that manipulates geometric relationships in vector space rather than relying on off-the-shelf contrastive objectives. It also supports bidirectional LLMs via BiLLM, letting causal decoder-only models act as full encoders for retrieval, and offers layer truncation for flexible Matryoshka-style compression.

Key highlights

• Four loss objectives: AnglE loss, contrastive, CoSENT, and Espresso (2DMSE from ICLR 2025).
• Backbone support spans BERT, RoBERTa, ModernBERT, LLaMA, Mistral, Qwen, and bidirectional LLM variants.
• Pretrained models have hit SOTA on MTEB (64.68 and 64.64 average scores) and the STS Benchmark.
• Can load and run third-party transformer embeddings, including sentence-transformers and BGE checkpoints.
• Training scales from single-GPU to multi-GPU via CLI or Python API.

Verdict Reach for this if you need to fine-tune domain-specific retrievers or chase leaderboard scores. If you only need generic off-the-shelf embeddings with zero training, it is likely more machinery than you need.