Label data by committee, then let statistics sort it out

What it does

skweak is a Python toolkit that automates data labeling for NLP by running documents through a battery of labelling functions—think regexes, dictionaries, and model stubs—and then reconciling their disagreements. It targets sequence labeling and text classification tasks where annotated data is scarce, such as low-resource languages or niche domains. The toolkit is built on SpaCy and uses a hidden Markov model to estimate how much trust to place in each labelling function.

The interesting bit

Instead of treating weak supervision as a bespoke research pipeline, skweak wraps the whole workflow—labelling, aggregation, and corpus generation—into a concise API that sits naturally inside existing SpaCy pipelines. The aggregation model learns the accuracy and confusion patterns of each labelling function by pitting them against one another, so you don’t need ground truth to calibrate trust.

Key highlights

• Integrates tightly with SpaCy Doc objects and existing NLP pipelines
• Supports diverse labelling functions: heuristics, gazetteers, machine learning models, and crowd-worker annotations
• Uses a generative HMM to aggregate conflicting labels and estimate per-function reliability
• Handles both sequence labeling and text classification
• Published and presented at ACL 2021

Caveats

• The project is no longer actively maintained; the authors are explicitly seeking a new maintainer
• Performance depends heavily on having enough unlabeled domain data and a sufficiently large set of labelling functions

Verdict

Worth exploring if you need to bootstrap an NLP model in a data-scarce domain and already live in the SpaCy ecosystem. If you require actively maintained libraries with commercial support, look elsewhere.