Composite LLMs without the training bill

What it does

mergekit is a toolkit for merging pretrained language models by operating directly on their weights. It takes multiple models—say, a Llama fine-tune for coding and another for reasoning—and bakes them into one model that runs at the speed and memory cost of a single model. The whole operation can run on CPU or with as little as 8 GB of VRAM, thanks to lazy tensor loading and an out-of-core design that keeps memory use low.

The interesting bit

The toolkit treats model surgery as a configuration problem: you write a YAML file specifying slices, merge methods, and parameter gradients, and mergekit handles the rest. It even lets you frankenstein layers from different architectures or perform tokenizer transplants, adjusting embedding matrices and vocabularies automatically when models do not share the same tokens.

Key highlights

• Supports Llama, Mistral, GPT-NeoX, StableLM, and others.
• Runs on CPU or GPU with minimal VRAM (as little as 8 GB).
• Offers many merge methods, including Mixture of Experts merging, LoRA extraction, and evolutionary approaches.
• Handles tokenizer mismatches via union vocabularies, forced token embeddings, and chat template overrides.
• Includes multi-stage merging and raw PyTorch model support for complex workflows.

Verdict

Anyone sitting on a pile of specialized LLM fine-tunes they wish were one generalist model should look here. If you are expecting a managed training platform rather than a weight-space scripting toolkit, this is not your stop.