segment-any-text/wtpsplit
A Python toolkit implementing transformer-based models to robustly segment text into sentences across 85 languages.
Not currently ranked — collecting fresh signals.
star history
wtpsplit provides pretrained deep learning models for sentence boundary detection and semantic unit segmentation. It implements the SaT (Segment Any Text) model as the primary offering, achieving state-of-the-art results across 8 corpora in 85 languages. The models can run on GPU, CPU via ONNX, or TPU, and support batched inference for efficient processing of multiple texts.
Frequently asked
- What is segment-any-text/wtpsplit?
- A Python toolkit implementing transformer-based models to robustly segment text into sentences across 85 languages.
- Is wtpsplit open source?
- Yes — segment-any-text/wtpsplit is open source, released under the MIT license.
- What language is wtpsplit written in?
- segment-any-text/wtpsplit is primarily written in Python.
- How popular is wtpsplit?
- segment-any-text/wtpsplit has 1.3k stars on GitHub.
- Where can I find wtpsplit?
- segment-any-text/wtpsplit is on GitHub at https://github.com/segment-any-text/wtpsplit.