Colab as a Fitting Room for Local LLMs

What it does

This repository houses dozens of Jupyter notebooks that launch WebUIs for quantized local LLMs inside Google Colab. Each notebook wraps a specific model—mostly GGUF builds from TheBloke—so you can run inference on consumer hardware without installing anything locally. The author originally tried scoring 7B and 13B models against GPT-4, but gave up when the release cadence made static comparisons feel pointless.

The interesting bit

Instead of posing as a rigorous evaluation framework, the project acts like a speed-dating service for models. The notes column is endearingly honest, flagging which models are censored, which handle roleplay, and which have “weird logic flaws”—details you rarely see in official leaderboards.

Key highlights

• One-click Colab badges for models ranging from 2.7B parameters (phi-2) up to 34B (deepsex-34b) and mixture-of-experts builds like Mixtral-8x7B.
• Plain-English model notes drawn from Reddit suggestions and the author’s own testing, covering censorship levels and roleplay quality.
• Includes mainstream instruct models (Mistral-7B-Instruct, Orca-2, Starling-LM) alongside niche fine-tunes that rarely appear on leaderboards.
• Runs entirely in-browser via Colab, making it a zero-install smoke-test for local LLM candidates.

Caveats

• The README is marked WIP and the comparison scores were abandoned in November 2023; newer entries have no quantitative evaluation at all.
• The model notes are subjective and sparse—“roleplay okay” and “some logic flaws” are as deep as the commentary gets.
• It is fundamentally an index rather than a framework: the value is in curation and one-click access, not in novel code or evaluation methodology.

Verdict

Grab this if you are shopping for a local LLM and would rather spend an afternoon clicking through models than reading benchmark whitepapers. Skip it if you need reproducible evaluation metrics or automated regression testing.