lucidrains/rotary-embedding-torch
A standalone PyTorch library implementing rotary positional embeddings from the Roformer paper for transformer architectures.
Not currently ranked — collecting fresh signals.
star history
This library provides an efficient implementation of rotary embeddings (RoPE), a technique that rotates information into tensor axes to encode relative positional information in transformers. It integrates into attention mechanisms by rotating queries and keys prior to the dot-product attention operation. The approach improves upon traditional positional encodings by enabling seamless generalization to sequence lengths not seen during training.
Frequently asked
- What is lucidrains/rotary-embedding-torch?
- A standalone PyTorch library implementing rotary positional embeddings from the Roformer paper for transformer architectures.
- Is rotary-embedding-torch open source?
- Yes — lucidrains/rotary-embedding-torch is open source, released under the MIT license.
- What language is rotary-embedding-torch written in?
- lucidrains/rotary-embedding-torch is primarily written in Python.
- How popular is rotary-embedding-torch?
- lucidrains/rotary-embedding-torch has 819 stars on GitHub.
- Where can I find rotary-embedding-torch?
- lucidrains/rotary-embedding-torch is on GitHub at https://github.com/lucidrains/rotary-embedding-torch.