← all repositories

sunrainyg/RandOpt

A framework for finding diverse task experts through optimization around pretrained LLM weights, followed by distillation into a single model.

598 stars Python Language ModelsML Frameworks
RandOpt
Velocity · 7d
+5.5
★ / day
Trend
steady
star history

This repository implements the Neural Thickets approach for post-training LLMs, using black-box optimization and neuroevolution to discover diverse specialized models around pretrained weights. It leverages LoRA for efficient fine-tuning and provides tooling to post-train on custom or standard datasets. The top-performing specialized models are then distilled into a unified model.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.