This repository provides a PyTorch implementation of BERT-based seq2seq using the UniLM approach. It supports multiple pretrained models including BERT, RoBERTa, GPT2, and T5, and enables tasks such as automatic summarization, text classification, sentiment analysis, named entity recognition, and relation extraction. Users configure model type via model_name and task type via model_class, with example scripts provided for each task. A distributed training version supporting multi-GPU training is also available.