liltom-eth/llama2-webui
A Gradio-based web interface for running Llama 2 models locally on GPU or CPU with support for multiple quantization formats and inference backends.
Not currently ranked — collecting fresh signals.
star history
This project provides a user-friendly web UI (built with Gradio) to run Llama 2 models locally on any platform. It supports various model sizes (7B, 13B, 70B) and quantization formats including GPTQ, GGML, and GGUF. Multiple inference backends are supported such as transformers, bitsandbytes for 8-bit, AutoGPTQ for 4-bit, and llama.cpp. The project also exposes an OpenAI-compatible API for integration with other applications and agents.
Frequently asked
- What is liltom-eth/llama2-webui?
- A Gradio-based web interface for running Llama 2 models locally on GPU or CPU with support for multiple quantization formats and inference backends.
- Is llama2-webui open source?
- Yes — liltom-eth/llama2-webui is open source, released under the MIT license.
- What language is llama2-webui written in?
- liltom-eth/llama2-webui is primarily written in Jupyter Notebook.
- How popular is llama2-webui?
- liltom-eth/llama2-webui has 1.9k stars on GitHub.
- Where can I find llama2-webui?
- liltom-eth/llama2-webui is on GitHub at https://github.com/liltom-eth/llama2-webui.