The Chinese GPT-2 repo that admits it was just practice

What it does This repository wraps HuggingFace Transformers to train and run GPT-2 on Chinese text. It handles tokenization through BERT’s tokenizer—or BPE with minor tweaks—and supports character-level, word-level, or subword training. Beyond the training scripts, it serves as a distribution point for community-trained models spanning prose, classical poetry, couplets, lyrics, and classical Chinese.

The interesting bit The author openly calls this a personal PyTorch learning exercise and has stopped active maintenance, yet the project became an early communal hub for Chinese generative models. Users continue to share checkpoints via Baidu Pan and Google Drive, and downstream projects like a regulated-verse poetry generator and a TensorFlow 2.0 sibling repo have spun out from it.

Key highlights

• Supports three tokenization modes: character, word, and BPE (the last two need minor script changes)
• Community-contributed models include prose, ~800k ancient poems, couplets, lyrics, and a 1.8GB classical Chinese corpus
• Includes gradient accumulation and optional fp16, though the author warns fp16 “may not converge, reason unclear”
• Shared models require prefixing prompts with [CLS] for proper generation
• train_single.py handles the edge case of training on one massive text, like an entire web novel

Caveats

• The author explicitly states this was a practice project and has stopped updating it
• fp16 support is experimental and reportedly unstable
• Word-level and BPE modes require manual tweaks to train.py

Verdict Grab it if you want a lightweight, proven starting point for Chinese GPT-2 fine-tuning or to tinker with community poetry and prose models. Otherwise, modern HuggingFace tooling has largely superseded it.