Is LLM-ToolMaker open source?

Yes — ctlllll/LLM-ToolMaker is an open-source project tracked on heatdrop.

What language is LLM-ToolMaker written in?

ctlllll/LLM-ToolMaker is primarily written in Jupyter Notebook.

How popular is LLM-ToolMaker?

ctlllll/LLM-ToolMaker has 1.1k stars on GitHub.

Where can I find LLM-ToolMaker?

ctlllll/LLM-ToolMaker is on GitHub at https://github.com/ctlllll/LLM-ToolMaker.

← all repositories

ctlllll/LLM-ToolMaker

Let GPT-4 build the hammer, let GPT-3.5 swing it

A framework where a powerful LLM authors reusable Python tools so a cheaper model can deploy them, cutting inference costs without cutting accuracy.

★1.1k stars Jupyter Notebook Agents Language Models LLMOps · Eval

View on GitHub ↗

Not currently ranked — collecting fresh signals.

star history

What it does LATM (LLMs As Tool Makers) is a research framework that splits problem-solving between two language models. A capable but expensive model—think GPT-4—writes a generic Python utility function from a handful of task demonstrations. A lighter, cheaper model—think GPT-3.5—then translates natural-language questions into calls to that function. The README says this division keeps quality on par with using the expensive model end-to-end while “significantly” reducing inference cost. The code is delivered as Jupyter notebooks that walk through tool creation, verification, and dispatch against Big-Bench reasoning tasks.

The interesting bit The tool maker doesn’t just guess code; it proposes a function, runs unit tests on validation samples, and iteratively fixes both the implementation and the tests before wrapping everything into a reusable API. It is essentially an LLM-driven CI pipeline compressed into a few notebook cells.

Key highlights

A closed loop: the tool maker generates Python utilities, verifies them with unit tests, and wraps them for a separate tool user.
Cost asymmetry: the authors validate that pairing GPT-4 as maker with GPT-3.5 as user matches GPT-4-only performance on Big-Bench tasks at lower cost.
The tool user can be the same model as the maker or a different, lighter one.
All tooling is task-specific but generated from only a few (k = 3) demonstrations.
Ships with Big-Bench task data and chain-of-thought prompts in dedicated folders.

Caveats

The repository is a collection of Jupyter notebooks and JSON data rather than a packaged library, so expect research artifacts rather than a polished SDK.
The README claims cost is “significantly reduced” and quality is “on par,” but offers no specific dollar figures, latency numbers, or failure-rate statistics.

Verdict Worth a look if you are experimenting with model cascading and want a concrete paper implementation. Give it a pass if you are after a production SDK—this is strictly research code in notebook form.

Frequently asked

What is ctlllll/LLM-ToolMaker?: A framework where a powerful LLM authors reusable Python tools so a cheaper model can deploy them, cutting inference costs without cutting accuracy.
Is LLM-ToolMaker open source?: Yes — ctlllll/LLM-ToolMaker is an open-source project tracked on heatdrop.
What language is LLM-ToolMaker written in?: ctlllll/LLM-ToolMaker is primarily written in Jupyter Notebook.
How popular is LLM-ToolMaker?: ctlllll/LLM-ToolMaker has 1.1k stars on GitHub.
Where can I find LLM-ToolMaker?: ctlllll/LLM-ToolMaker is on GitHub at https://github.com/ctlllll/LLM-ToolMaker.