Your local LLM switchboard when 6 GB VRAM is all you have

What it does

闻达 is essentially polished integration glue—a self-hosted LLM gateway and web UI that bundles inference backends, knowledge retrieval, and multi-user chat into one interface. It targets Chinese individuals and small businesses who lack high-end GPUs but need private, domain-specific content generation. The platform wraps a growing list of local models—ChatGLM, RWKV, Baichuan, and others—and can fall back to online APIs such as OpenAI or ChatGLM-130B.

The interesting bit

The standout is the “Auto” system: you extend the platform by dropping JavaScript files into an autos directory, almost like browser userscripts for your backend. These scripts can chain LLM calls, query the vector knowledge base, trigger Stable Diffusion, or even turn lip movements into voice-input triggers—an oddly lightweight way to automate complex workflows without touching Python.

Key highlights

• Runs on modest hardware: the “lazy package” targets 6 GB VRAM out of the box, with CPU and quantized fallbacks for most supported models.
• Knowledge bases stay local: choose between offline semantic search (rtst), a local search engine (fess), or Bing, with retrieved context injected directly into prompts.
• Broad model support: offline backends include ChatGLM-6B/2-6B, ChatRWKV, InternLM, and Aquila-7B, plus online fallbacks via OpenAI or ChatGLM-130B API.
• Multi-user and intranet-ready: built for shared office or SMB deployment rather than single-developer laptops.
• JavaScript plugin architecture: Auto scripts handle everything from article generation to QQ bots, using an async API that talks to the backend over HTTP.

Caveats

• The README is honest about model limitations: Llama and MOSS are “not recommended” for Chinese, and Baichuan “needs LoRA or results are poor,” so model selection requires reading the fine print.
• Several browser-facing features, such as face-recognition voice input and clipboard access, are gated behind local-host or TLS restrictions that can trip up first-time deployers.
• Documentation and community support are heavily centered on Chinese-language QQ groups, which may slow down English-speaking contributors.

Verdict

Worth a look if you are a Chinese-speaking team that needs a private, low-cost LLM hub with plugin extensibility. Skip it if you are looking for a polished, English-first SaaS experience or expect turnkey performance from every supported model.