brontoguana/krasis
A Rust/CUDA-based LLM runtime for efficiently running multi-hundred-billion-parameter MoE models on consumer NVIDIA GPUs.
Not currently ranked — collecting fresh signals.
star history
Krasis is an inference engine that executes large language models on consumer-grade hardware by managing expert residency between VRAM and CPU RAM. It moves performance-critical paths from Python to Rust and CUDA, providing GPU-accelerated prefill and decode, HQQ attention caching, and compact KV cache formats like k6v6 and k4v4. The project includes an OpenAI-compatible API, interactive launcher, benchmark tooling, and GitHub-release based installation.
Frequently asked
- What is brontoguana/krasis?
- A Rust/CUDA-based LLM runtime for efficiently running multi-hundred-billion-parameter MoE models on consumer NVIDIA GPUs.
- Is krasis open source?
- Yes — brontoguana/krasis is an open-source project tracked on heatdrop.
- What language is krasis written in?
- brontoguana/krasis is primarily written in C++.
- How popular is krasis?
- brontoguana/krasis has 477 stars on GitHub.
- Where can I find krasis?
- brontoguana/krasis is on GitHub at https://github.com/brontoguana/krasis.