philipturner/metal-flash-attention
A Swift/Metal implementation of FlashAttention optimized for Apple Silicon GPUs.
Not currently ranked — collecting fresh signals.
star history
Ports the FlashAttention algorithm to Apple silicon using Metal. Implements single-headed attention with JIT compilation, featuring an alternative backward pass that achieves full parallelization efficiency across both dimensions of the attention matrix. Uses intentional register spilling and optimized block dimensions (16-32 along parallelization, 80-128 along traversal) to overcome register pressure bottlenecks at large head dimensions like 256.
Frequently asked
- What is philipturner/metal-flash-attention?
- A Swift/Metal implementation of FlashAttention optimized for Apple Silicon GPUs.
- Is metal-flash-attention open source?
- Yes — philipturner/metal-flash-attention is open source, released under the MIT license.
- What language is metal-flash-attention written in?
- philipturner/metal-flash-attention is primarily written in Swift.
- How popular is metal-flash-attention?
- philipturner/metal-flash-attention has 611 stars on GitHub.
- Where can I find metal-flash-attention?
- philipturner/metal-flash-attention is on GitHub at https://github.com/philipturner/metal-flash-attention.