What it does

This is a set of Windows batch files, PowerShell scripts, and shell scripts that download and configure Ollama + AnythingLLM onto a USB drive. After a one-time setup, you can plug the drive into any Windows, Mac, or Linux machine and run a local LLM without leaving traces on the host PC. The README emphasizes privacy: no registry keys, no local files, no telemetry.

The interesting bit

The project is essentially a well-organized glue layer around existing tools, but it solves a genuinely annoying problem: making a portable Ollama installation that survives being unplugged and moved between computers. The Windows launcher auto-clears cached paths to prevent “JavaScript errors” when switching machines—a rough edge the author actually bothered to sand down.

Key highlights

• Six curated models including two “uncensored” options (NemoMix 12B, Dolphin 8B) and four standard ones (Mistral, Qwen, Llama 3.2, Phi-3.5)
• Custom GGUF support via direct HuggingFace links
• 4K token default context, user-adjustable via .env (though re-running the installer resets it)
• CPU-only inference; 10–30 second response times depending on hardware
• 16 GB USB minimum, 32–64 GB recommended for multiple models

Caveats

• Performance is modest: everything runs on CPU, and the 12B model needs 8 GB RAM minimum
• Linux setup is rougher: requires manual chmod, running a preflight script, and launching the AppImage yourself
• AnythingLLM install requires manual path selection during setup; miss this step and portability breaks
• Token limit changes are fragile: re-running install.bat or install.sh silently resets your custom context size back to 4096

Verdict

Worth a look if you need a genuinely portable, air-gapped AI demo or personal assistant that fits in your pocket. Skip it if you already have a working local Ollama setup and don’t need the USB mobility theater.