Local LLaMA without the Makefile headaches

What it does

Dalai is a JavaScript orchestration layer around llama.cpp and alpaca.cpp. It automates model downloads and exposes both a hackable web UI and a Socket.io API for prompting LLaMA and Alpaca locally, while papering over the cross-platform build differences. Inference stays on your machine, with memory footprints starting around 4 GB for a 7B model.

The interesting bit

The value is not the chat interface but the build babysitting. Dalai hides the messy toolchain requirements—Visual Studio on Windows, cmake on macOS, Python version locks on Linux—so you can get the underlying C++ engines running without reading a Makefile.

Key highlights

• Runs on Linux, Mac, and Windows, with quantized models shrinking disk usage down to roughly 4–40 GB depending on size.
• Ships with a built-in web UI and a programmatic JavaScript/Socket.io API for local or remote requests.
• Supports both LLaMA and Alpaca models from 7B up to 65B, and lets you discard the full weights after installation to reclaim hundreds of gigabytes.
• Acts as an NPM package that manages the underlying engines, so you can embed it into existing Node.js applications.

Caveats

• Windows users must avoid PowerShell entirely; the README warns it causes silent failures due to strict permissions.
• Linux requires Python 3.10 or lower, not 3.11, because downstream dependencies are not yet compatible.
• The full LLaMA models are massive—up to 432 GB for the 65B version—so plan to delete the originals and keep only the quantized files.

Verdict

Worth a look if you want a local LLaMA or Alpaca sandbox without becoming a C++ build engineer. If you already have a working llama.cpp setup and just want a raw CLI, this adds little.