sunrainyg/RandOpt
A framework for finding diverse task experts through optimization around pretrained LLM weights, followed by distillation into a single model.

Velocity · 7d
+5.5
★ / day
Trend
→steady
star history
This repository implements the Neural Thickets approach for post-training LLMs, using black-box optimization and neuroevolution to discover diverse specialized models around pretrained weights. It leverages LoRA for efficient fine-tuning and provides tooling to post-train on custom or standard datasets. The top-performing specialized models are then distilled into a unified model.