One Gradio UI to Wrangle Dozens of Quirky Open-Source LLMs

What it does

This repository is essentially an integration layer that turns a zoo of open-source instruction-following models—Alpaca, Vicuna, StableLM, and others—into a chat service. It combines a Gradio-based web UI and a Discord bot so you can interact with them without wrestling with each model’s unique prompt syntax. A companion library called Ping Pong handles the conversation state, keeping only the last few turns to stop context windows from ballooning and slowing generation.

The interesting bit

The author treats prompt formatting as a dialect problem: every model speaks differently, so the project abstracts conversation management away from the UI layer. That separation is what lets the same Gradio front-end and Discord bot switch between a 3B Flan model and a 65B Alpaca-LoRA checkpoint without rewriting chat logic.

Key highlights

• Ships with a Gradio UI (GradioChat) styled after HuggingChat and a mention-based Discord bot.
• Supports internet search via Serper API in both interfaces.
• Context window is deliberately capped to the last 2–3 turns to keep generation fast.
• Can run offline with locally cached weights or pull from Hugging Face Hub.
• Pre-integrated with cloud GPU providers Jarvislabs.ai and dstack.

Caveats

• The README contains visible typos and stale notes, including a warning that full-gpu mode “means half” and that some model repos “will be made private soon.”
• Discord interaction is limited to mentions; there are no slash commands.
• It is unclear how actively the supported model list is maintained, since the README still points to potentially private checkpoints.

Verdict

Good for developers who want to compare many open-source chat models side-by-side without writing custom loaders. Skip it if you are looking for a single-model production backend or a polished, actively maintained SaaS.