mergekit provides tools for merging pre-trained large language models using an out-of-core approach that works on CPU or with as little as 8 GB of VRAM. It supports multiple merging algorithms including evolutionary methods, multi-stage merging, LoRA extraction, and mixture of experts merging. The toolkit enables combining the strengths of different models without the computational overhead of ensembling.