Making LLMs cite their sources, sentence by sentence, at 128K

What it does LongCite releases two fine-tuned models—LongCite-glm4-9b and LongCite-llama3.1-8b—that answer questions over long documents and append exact sentence-level citations to every claim. The project also ships LongBench-Cite, an evaluation suite that scores both answer correctness and citation precision, and a CoF data-construction pipeline for building similar training sets from scratch.

The interesting bit The CoF (Coarse to Fine) pipeline automates a tedious labeling problem: it first generates chunk-level evidence, then narrows it down to specific sentences, producing the LongCite-45k dataset without manual annotation. The models themselves rely on trust_remote_code=True, which signals the implementation is not vanilla Transformers.

Key highlights

• Two 8B/9B parameter models supporting up to 128K context, built on GLM-4-9B and Llama-3.1-8B.
• CoF pipeline under CoF/ for automated creation of fine-grained citation data via four scripts.
• LongBench-Cite benchmark adapted from LongBench and LongBench-Chat, using GPT-4o as a judge for citation quality and correctness.
• Evaluation code and data provided; the README includes result charts comparing fine-tuned models against baselines like GPT-4o.
• Dataset LongCite-45k available on Hugging Face for mixing with general SFT data.

Caveats

• The README notes that the lightweight training path via LongAlign supports only up to 32K sequences, so reproducing the full 128K fine-tuning likely requires Megatron-LM.
• Model inference requires trust_remote_code=True, meaning custom modeling code outside standard Transformers.
• The CoF pipeline and benchmark both need an OpenAI API key configured in utils/llm_api.py.

Verdict Grab this if you are building RAG or document-QA systems and need verifiable, attributable outputs over very long inputs. Skip it if you are looking for a drop-in, standard-Hugging-Face-model solution with no extra API dependencies or custom code.