microsoft/DeBERTa
Microsoft's implementation of DeBERTa, a decoding-enhanced BERT model with disentangled attention and ELECTRA-style pre-training.
Not currently ranked — collecting fresh signals.
star history
This repository provides the official implementation of DeBERTa and DeBERTa V3, transformer-based language models that improve on BERT through disentangled attention mechanisms. The implementation includes code for model pre-training, continuous training, and fine-tuning on downstream NLP tasks including SuperGLUE benchmarks. Models ranging from 22M to 1.5B parameters are available via the Hugging Face model hub.
Frequently asked
- What is microsoft/DeBERTa?
- Microsoft's implementation of DeBERTa, a decoding-enhanced BERT model with disentangled attention and ELECTRA-style pre-training.
- Is DeBERTa open source?
- Yes — microsoft/DeBERTa is open source, released under the MIT license.
- What language is DeBERTa written in?
- microsoft/DeBERTa is primarily written in Python.
- How popular is DeBERTa?
- microsoft/DeBERTa has 2.2k stars on GitHub.
- Where can I find DeBERTa?
- microsoft/DeBERTa is on GitHub at https://github.com/microsoft/DeBERTa.