AnswerDotAI/ModernBERT
Research repository for ModernBERT, a scaled and modernized BERT-style encoder model with modular FlexBERT architecture.
Not currently ranked — collecting fresh signals.
star history
This repository contains the code for training and evaluating ModernBERT, an encoder-only transformer model that updates the classic BERT architecture with modern improvements like Flash Attention 2. The project provides modular FlexBERT building blocks defined via YAML configuration files, and includes pre-training and GLUE benchmark evaluation code. Model checkpoints are available on HuggingFace.
Frequently asked
- What is AnswerDotAI/ModernBERT?
- Research repository for ModernBERT, a scaled and modernized BERT-style encoder model with modular FlexBERT architecture.
- Is ModernBERT open source?
- Yes — AnswerDotAI/ModernBERT is open source, released under the Apache-2.0 license.
- What language is ModernBERT written in?
- AnswerDotAI/ModernBERT is primarily written in Python.
- How popular is ModernBERT?
- AnswerDotAI/ModernBERT has 1.7k stars on GitHub.
- Where can I find ModernBERT?
- AnswerDotAI/ModernBERT is on GitHub at https://github.com/AnswerDotAI/ModernBERT.