NVIDIA/TransformerEngine
NVIDIA's library for accelerating Transformer model training and inference on GPUs using FP8/FP4 precision.
Not currently ranked — collecting fresh signals.
star history
TransformerEngine is a performance library that provides optimized transformer layers, attention backends, and fused kernels for NVIDIA GPUs including Hopper, Ada, and Blackwell architectures. It supports low-precision training and inference via FP8 and NVFP4 formats to reduce memory usage and increase throughput. It integrates with PyTorch, JAX, and major LLM frameworks including NeMo and Megatron.
Frequently asked
- What is NVIDIA/TransformerEngine?
- NVIDIA's library for accelerating Transformer model training and inference on GPUs using FP8/FP4 precision.
- Is TransformerEngine open source?
- Yes — NVIDIA/TransformerEngine is open source, released under the Apache-2.0 license.
- What language is TransformerEngine written in?
- NVIDIA/TransformerEngine is primarily written in Python.
- How popular is TransformerEngine?
- NVIDIA/TransformerEngine has 3.4k stars on GitHub.
- Where can I find TransformerEngine?
- NVIDIA/TransformerEngine is on GitHub at https://github.com/NVIDIA/TransformerEngine.