A bibliography for the 'do what I mean' era of language models

What it does

This is a hand-curated reading list tracking the instruction-tuning literature from its 2021 roots—Natural-Instructions, FLAN, and T0—through the current arXiv firehose. Each entry is a paper link with authors and a date, organized chronologically. It is essentially a community-maintained bibliography for a field that went from niche conference papers to the dominant LLM recipe in about two years.

The interesting bit

The list unintentionally serves as a timeline of the paradigm shift from pure pretraining to instruction following. By collecting everything from early multitask prompted training to self-generated instructions and GPT-4 distillation, it maps how the community converged on the “finetune with instructions” formula now taken for granted.

Key highlights

• Spans the full arc from the 2021 Natural-Instructions/FLAN/T0 triad through 2023 work on Chinese models and GPT-4 tuning
• Explicitly defines the core goal: teaching models to follow natural-language constraints, prompts, and examples for better generalization
• Includes adjacent threads like retrieval-augmented tuning, multimodal instruction, and toxicity control via instructions
• Community-driven with PRs welcome, so it drifts toward whatever the current conference cycle thinks is hot

Caveats

• The README is a flat numbered list with no taxonomy, search, or tagging—finding a specific technique means scrolling
• Some entries are truncated (the README cuts off mid-title at “Chinese”) and there is no annotation or summary beyond the paper name itself

Verdict

Worth bookmarking if you are writing a literature review, designing a course, or trying to remember what that one ICLR 2022 paper was called. Skip it if you expect code, benchmarks, or anything more interactive than a well-organized reading list.