princeton-nlp/LESS
A data selection framework that identifies influential training examples to improve specific LLM capabilities through targeted instruction tuning.
Not currently ranked — collecting fresh signals.
star history
LESS provides a method to select the most impactful training data for LLM instruction tuning by building a gradient datastore and scoring examples based on their influence on target capabilities. The pipeline involves warmup training, gradient collection, and influence-based selection across datasets like Flan v2, COT, Dolly, and Open Assistant. The selected data is then used for fine-tuning to induce specific capabilities in models like Llama and Mistral.
Frequently asked
- What is princeton-nlp/LESS?
- A data selection framework that identifies influential training examples to improve specific LLM capabilities through targeted instruction tuning.
- Is LESS open source?
- Yes — princeton-nlp/LESS is open source, released under the MIT license.
- What language is LESS written in?
- princeton-nlp/LESS is primarily written in Jupyter Notebook.
- How popular is LESS?
- princeton-nlp/LESS has 532 stars on GitHub.
- Where can I find LESS?
- princeton-nlp/LESS is on GitHub at https://github.com/princeton-nlp/LESS.