A field guide to the LLM zoo of science

What it does This repository is the companion bibliography to an EMNLP 2024 survey paper. It catalogs pre-trained language models tailored to scientific domains—mathematics, physics, chemistry, biology, medicine, geoscience, and more—cross-referenced by parameter count (from 38M up to 120B) and modality (text, graph, protein, molecule, climate time series). Each entry links directly to papers, code, and Hugging Face checkpoints.

The interesting bit The taxonomy reads like a map of how general NLP dissolves into domain dialects: chemistry needs graph+molecule encoders, biology splinters into protein, DNA, RNA, and multiomics branches, and geoscience picks up climate time series. The list makes that balkanization explicit.

Key highlights

• Seven scientific domains, from pure math to geoscience, plus a general cross-domain section
• Scale span is wide: 38M parameters (INDUS) up to 120B (Galactica)
• Multi-modal scope beyond text: vision, graph, table, protein, genome, molecule, and climate time series
• Chronological sorting within sections traces the shift from early masked LMs like SciBERT to instruction-tuned SciGLM and mixture-of-experts SciDFM
• Strict curation: only papers with named authors and live model links make the cut

Caveats

• Inclusion is gated by availability of author names and public model links, so closed-source work is excluded
• It is a reading list with links, not a framework or benchmark suite; expect no glue code

Verdict Researchers deciding which scientific LM to fine-tune or cite should treat this as a first stop. If you are looking for a single unified toolkit to download and run experiments, this is just the index—not the library.