bytedance/ByteTransformer
ByteTransformer is a GPU-accelerated inference library optimized for BERT-like transformer models on NVIDIA GPUs.
Not currently ranked — collecting fresh signals.
star history
ByteTransformer is a high-performance inference library for BERT-like transformers that provides both Python and C++ APIs with PyTorch integration. It implements architectural-aware optimizations including padding-free algorithms, QKV encoding, softmax, feed-forward networks, and multi-head attention specifically for BERT inference. The library has been deployed in production at ByteDance to serve transformer models with superior performance compared to PyTorch, TensorFlow, FasterTransformer, and DeepSpeed on A100 GPUs.
Frequently asked
- What is bytedance/ByteTransformer?
- ByteTransformer is a GPU-accelerated inference library optimized for BERT-like transformer models on NVIDIA GPUs.
- Is ByteTransformer open source?
- Yes — bytedance/ByteTransformer is open source, released under the Apache-2.0 license.
- What language is ByteTransformer written in?
- bytedance/ByteTransformer is primarily written in C++.
- How popular is ByteTransformer?
- bytedance/ByteTransformer has 478 stars on GitHub.
- Where can I find ByteTransformer?
- bytedance/ByteTransformer is on GitHub at https://github.com/bytedance/ByteTransformer.