Tencent/TencentPretrain
A PyTorch-based multimodal pre-training and fine-tuning framework implementing BERT, GPT, T5, and other transformer architectures with a model zoo.
Not currently ranked — collecting fresh signals.
star history
TencentPretrain is a modular pre-training and fine-tuning toolkit supporting text, vision, and audio modalities. It provides implementations of major transformer architectures including BERT, GPT-2, ELMo, T5, CLIP, and others with standardized interfaces for embedding, encoder, target, and decoder modules. The framework includes a model zoo with pre-trained weights and inherits from the UER-py open-source project.
Frequently asked
- What is Tencent/TencentPretrain?
- A PyTorch-based multimodal pre-training and fine-tuning framework implementing BERT, GPT, T5, and other transformer architectures with a model zoo.
- Is TencentPretrain open source?
- Yes — Tencent/TencentPretrain is an open-source project tracked on heatdrop.
- What language is TencentPretrain written in?
- Tencent/TencentPretrain is primarily written in Python.
- How popular is TencentPretrain?
- Tencent/TencentPretrain has 1.1k stars on GitHub.
- Where can I find TencentPretrain?
- Tencent/TencentPretrain is on GitHub at https://github.com/Tencent/TencentPretrain.