A Swiss Army knife for LLMs that lives in your terminal

What it does

AIChat is a Rust CLI that wraps more than 20 LLM providers—OpenAI, Claude, Gemini, Ollama, Groq, and many others—behind a single interface. It runs in two modes: fire-and-forget CMD mode for one-shot queries, and an interactive REPL with tab completion, history search, and multi-line input. You can pipe files, directories, remote URLs, or shell command output straight into either mode.

The interesting bit

The project doesn’t stop at chat. It bundles a lightweight HTTP server (--serve) that exposes chat-completion, embedding, and rerank endpoints, plus an LLM Playground and an LLM Arena for side-by-side model comparisons. That turns a terminal tool into a local API gateway and testing lab.

Key highlights

• Shell Assistant generates OS- and shell-aware commands from natural language descriptions
• Built-in RAG for grounding responses in external documents
• Configurable roles, persistent sessions, and macros to automate repetitive REPL workflows
• AI Agents combine instructions, tool calling, and RAG into reusable “CLI GPTs”
• Custom dark/light themes with syntax highlighting for code blocks

Caveats

• The README is heavy on feature lists and light on architecture or performance details; exact latency or memory footprint isn’t stated
• Function-calling tools live in a separate repository (llm-functions), so getting the most out of agents requires extra setup

Verdict

Worth a look if you juggle multiple LLM providers and want one tool for quick terminal queries, scripted automation, and local API proxying. Skip it if you need a managed cloud service or deep IDE integration.