This repository contains the code for training and evaluating ModernBERT, an encoder-only transformer model that updates the classic BERT architecture with modern improvements like Flash Attention 2. The project provides modular FlexBERT building blocks defined via YAML configuration files, and includes pre-training and GLUE benchmark evaluation code. Model checkpoints are available on HuggingFace.