This project provides a从头实现 implementation of the BERT model architecture and demonstrates its application on common NLP downstream tasks. It includes implementations for Chinese text classification, English entailment (MNLI), multiple choice (SWAG), question answering (SQuAD), and named entity recognition. The repository also contains a pre-training module that trains BERT from scratch using Next Sentence Prediction and Masked Language Model objectives.