← all repositories
facebookresearch/InferSent

Sentence embeddings from a dead-simple idea: train on entailment

Facebook Research's 2017 encoder that proved natural language inference data beats unsupervised methods for sentence representations.

2.3k stars Jupyter Notebook RAG · SearchLanguage Models
InferSent
Velocity · 7d
+0.7
★ / day
Trend
steady
star history

What it does InferSent turns English sentences into 4096-dimensional vectors using a pre-trained BiLSTM encoder. You download a pickle, point it at GloVe or fastText vectors, and call encode(). The README claims ~1000 sentences per second on a single GPU with batch size 128.

The interesting bit The trick is the training objective: natural language inference (entailment, contradiction, neutral) rather than raw language modeling. The authors showed this supervised signal transfers better than SkipThought to classification, similarity, and entailment benchmarks — a result that looks obvious now but wasn’t then.

Key highlights

  • Two model versions: V1 (GloVe + PTB tokenization) and V2 (fastText + Moses tokenization, no zero-padding in max-pooling)
  • Includes a visualize() function to inspect per-word importance in sentence encodings
  • Evaluation table against SkipThought and fastText-BoV on 11 standard transfer tasks
  • Ships with a demo.ipynb for quick start

Caveats

  • Maintenance mode: the author removed train_nli.py and explicitly states no time to maintain beyond “simple scripts to get sentence embeddings”
  • Python 2/3 compatibility noted, but PyTorch version requirements are vague (“recent version”)
  • Tokenization mismatch between V1/V2 is a footgun — the README warns you must match training tokenization for fair evaluation

Verdict Grab this if you need a battle-tested baseline for English sentence embeddings without fighting transformers. Skip it if you want multilingual support, active maintenance, or state-of-the-art results — the field has moved to sentence-transformers and larger pre-trained models.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.