bytedance/lightseq
A CUDA library for high-speed training and inference of transformer-based sequence models with int8/fp16 mixed-precision support.
Not currently ranked — collecting fresh signals.
star history
LightSeq provides optimized CUDA kernels for transformer operations used in NLP sequence processing tasks including machine translation, BERT, and GPT. The library offers both training acceleration and high-throughput inference with support for int8 and fp16 mixed-precision computation. It integrates with Fairseq and Hugging Face frameworks for model deployment and supports beam search, diverse decoding, and sampling strategies.
Frequently asked
- What is bytedance/lightseq?
- A CUDA library for high-speed training and inference of transformer-based sequence models with int8/fp16 mixed-precision support.
- Is lightseq open source?
- Yes — bytedance/lightseq is an open-source project tracked on heatdrop.
- What language is lightseq written in?
- bytedance/lightseq is primarily written in C++.
- How popular is lightseq?
- bytedance/lightseq has 3.3k stars on GitHub.
- Where can I find lightseq?
- bytedance/lightseq is on GitHub at https://github.com/bytedance/lightseq.