withcatai/node-llama-cpp
Node.js bindings enabling local LLM inference via llama.cpp with Metal, CUDA, and Vulkan GPU support.
Not currently ranked — collecting fresh signals.
star history
This library wraps llama.cpp to provide a complete Node.js interface for running large language models locally. It supports GPU acceleration across multiple backends, pre-built binaries for easy installation, and enforces structured output formats like JSON schemas during generation. The library includes embedding generation, function calling capabilities, and a CLI for chatting with models without writing code.
Frequently asked
- What is withcatai/node-llama-cpp?
- Node.js bindings enabling local LLM inference via llama.cpp with Metal, CUDA, and Vulkan GPU support.
- Is node-llama-cpp open source?
- Yes — withcatai/node-llama-cpp is open source, released under the MIT license.
- What language is node-llama-cpp written in?
- withcatai/node-llama-cpp is primarily written in TypeScript.
- How popular is node-llama-cpp?
- withcatai/node-llama-cpp has 2.1k stars on GitHub.
- Where can I find node-llama-cpp?
- withcatai/node-llama-cpp is on GitHub at https://github.com/withcatai/node-llama-cpp.