google-research/electra
ELECTRA is a self-supervised method for pre-training transformer text encoders using a discriminator that distinguishes real tokens from generated fakes.
Not currently ranked — collecting fresh signals.
star history
The repository provides code to pre-train ELECTRA models, including small models trainable on a single GPU, and supports fine-tuning on downstream NLP tasks including GLUE classification, SQuAD question answering, and sequence tagging. It also includes Electric, an energy-based version of ELECTRA that can produce pseudo-likelihood scores for re-ranking speech or translation outputs.
Frequently asked
- What is google-research/electra?
- ELECTRA is a self-supervised method for pre-training transformer text encoders using a discriminator that distinguishes real tokens from generated fakes.
- Is electra open source?
- Yes — google-research/electra is open source, released under the Apache-2.0 license.
- What language is electra written in?
- google-research/electra is primarily written in Python.
- How popular is electra?
- google-research/electra has 2.4k stars on GitHub.
- Where can I find electra?
- google-research/electra is on GitHub at https://github.com/google-research/electra.