abetlen/llama-cpp-python
Python library providing high-level bindings to run quantized LLM inference via llama.cpp.
Not currently ranked — collecting fresh signals.
star history
This package wraps llama.cpp in Python, offering both low-level ctypes access and a high-level API for text completion. It ships an OpenAI-compatible web server, supports function calling, vision models, and multiple concurrent model sessions. Integration with LangChain and LlamaIndex enables use in RAG and agent pipelines.
Frequently asked
- What is abetlen/llama-cpp-python?
- Python library providing high-level bindings to run quantized LLM inference via llama.cpp.
- Is llama-cpp-python open source?
- Yes — abetlen/llama-cpp-python is open source, released under the MIT license.
- What language is llama-cpp-python written in?
- abetlen/llama-cpp-python is primarily written in Python.
- How popular is llama-cpp-python?
- abetlen/llama-cpp-python has 10.5k stars on GitHub.
- Where can I find llama-cpp-python?
- abetlen/llama-cpp-python is on GitHub at https://github.com/abetlen/llama-cpp-python.