This project extracts token-level embeddings from BERT’s pre-trained model weights. Instead of fine-tuning BERT end-to-end for downstream NLP tasks, users can extract and reuse the contextual token embeddings directly to build their own models. Implemented on Apache MXNet with the GluonNLP toolkit.