hyperonym/basaran
OpenAI-compatible API server for running open-source text generation models from Hugging Face.

Velocity · 7d
+1.1
★ / day
Trend
→steady
star history
Basaran provides an open-source alternative to the OpenAI text completion API, enabling applications to switch from OpenAI to self-hosted open-source models without code changes. It wraps Hugging Face Transformers-based text generation models with a streaming API supporting various decoding strategies, quantization, and multi-GPU setups, and includes a web-based playground interface.