THUDM/SwissArmyTransformer
A PyTorch library providing shared backbone code for building and training Transformer variants including BERT, GPT, T5, GLM, CogView, and ViT.
Not currently ranked — collecting fresh signals.
star history
SwissArmyTransformer is a framework for developing custom Transformer models that share a unified architecture backbone. It supports various model architectures such as BERT, GPT, T5, GLM, CogView, and ViT through lightweight mixin components. The library integrates DeepSpeed ZeRO and model parallelism to enable efficient pretraining and fine-tuning of large-scale models ranging from 100M to 20B parameters.
Frequently asked
- What is THUDM/SwissArmyTransformer?
- A PyTorch library providing shared backbone code for building and training Transformer variants including BERT, GPT, T5, GLM, CogView, and ViT.
- Is SwissArmyTransformer open source?
- Yes — THUDM/SwissArmyTransformer is open source, released under the Apache-2.0 license.
- What language is SwissArmyTransformer written in?
- THUDM/SwissArmyTransformer is primarily written in Python.
- How popular is SwissArmyTransformer?
- THUDM/SwissArmyTransformer has 1.1k stars on GitHub.
- Where can I find SwissArmyTransformer?
- THUDM/SwissArmyTransformer is on GitHub at https://github.com/THUDM/SwissArmyTransformer.