← all repositories

MuQiuJun-AI/bert4pytorch

A lightweight PyTorch reimplementation of BERT with Chinese comments and support for BERT family models like ALBERT, T5, NEZHA, and ELECTRA.

417 stars Python Language Models
bert4pytorch
Velocity · 7d
+0.2
★ / day
Trend
steady
star history

This repository provides a clean, lightweight implementation of BERT in PyTorch with detailed Chinese comments for easy modification. It supports masked language modeling (MLM), sequence classification, and implements advanced training techniques such as adversarial training (FGM), focal loss, and label smoothing cross-entropy. The codebase follows the API structure of bert4keras and supports various BERT-family architectures.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.