A phone book for the self-hosted AI crowd

What it does This is a curated awesome-list that maps out the sprawling ecosystem for running large language models locally. It indexes inference platforms, engines, chat UIs, specific models, agent frameworks, RAG tools, training pipelines, hardware notes, and tutorials — essentially the whole supply chain for keeping your data off someone else’s API.

The interesting bit The list is opinionated enough to be useful: it separates inference platforms (the click-and-run apps like LM Studio and Jan) from bare-metal engines (llama.cpp, vLLM, exo for clustering old laptops), and even carves out niche territory like Apple Silicon optimizations (mlx-lm, omlx) and AMD NPU support (FastFlowLM). Someone is paying attention to which tools actually ship.

Key highlights

• Covers the full stack: models, UIs, agent frameworks, MCP servers, browser automation, memory management, sandboxing, and hardware
• Surfaces less-obvious options alongside the obvious ones — e.g., distributed-llama for home device clusters, krasis for VRAM-starved GPUs
• Includes benchmarking resources and community links, not just software
• Explicitly scoped to local and self-hosted; no cloud API wrappers pretending to count

Caveats

• Being a curated list, quality depends on maintainer attention; some entries are just GitHub links with one-line descriptions
• No stated criteria for inclusion or ranking — unclear why some forks make the cut and others don’t
• Hardware section exists but is thin compared to the software depth

Verdict Worth bookmarking if you’re building or maintaining a local AI stack and tired of rediscovering the same twelve projects. Skip it if you want deep reviews or comparative benchmarks; this is a map, not a travel guide.