huawei-noah/Pretrained-Language-Model
Collection of pretrained Chinese language models (PanGu-α up to 200B parameters, NEZHA, GPT variants) and compression techniques from Huawei Noah's Ark Lab.
Not currently ranked — collecting fresh signals.
star history
This repository provides pretrained language models and related optimization techniques developed by Huawei Noah’s Ark Lab. Models include PanGu-α (large-scale autoregressive model up to 200B parameters), NEZHA (TensorFlow and PyTorch variants), and GPT-based generators for Chinese classical poetry. Optimization techniques include knowledge distillation and model compression via TinyBERT and DynaBERT, byte-level vocabulary tools (BBPE), and probabilistically masked language modeling (PMLM).
Frequently asked
- What is huawei-noah/Pretrained-Language-Model?
- Collection of pretrained Chinese language models (PanGu-α up to 200B parameters, NEZHA, GPT variants) and compression techniques from Huawei Noah's Ark Lab.
- Is Pretrained-Language-Model open source?
- Yes — huawei-noah/Pretrained-Language-Model is an open-source project tracked on heatdrop.
- What language is Pretrained-Language-Model written in?
- huawei-noah/Pretrained-Language-Model is primarily written in Python.
- How popular is Pretrained-Language-Model?
- huawei-noah/Pretrained-Language-Model has 3.2k stars on GitHub.
- Where can I find Pretrained-Language-Model?
- huawei-noah/Pretrained-Language-Model is on GitHub at https://github.com/huawei-noah/Pretrained-Language-Model.