TencentPretrain is a modular pre-training and fine-tuning toolkit supporting text, vision, and audio modalities. It provides implementations of major transformer architectures including BERT, GPT-2, ELMo, T5, CLIP, and others with standardized interfaces for embedding, encoder, target, and decoder modules. The framework includes a model zoo with pre-trained weights and inherits from the UER-py open-source project.