Docling Graph Replaces Document Embeddings with Typed Relationships

The Hype Moment: GraphRAG Needs Better Raw Material

The current cycle of AI retrieval is hitting a ceiling. Dense vector search powers most production RAG systems, yet it remains stubbornly approximate: it retrieves chunks that seem related but cannot guarantee that a specific chemical compound is linked to a particular reaction step, or that a legal clause references an exact instrument. Knowledge graphs promise an escape hatch. Neo4j cites research that GraphRAG can make AI agents substantially more truthful by grounding answers in explicit relationships rather than statistical nearness. The industry has responded with enthusiasm, but enthusiasm alone does not build graphs. Constructing a knowledge graph from unstructured text with large language models is computationally punishing and introduces latency that many enterprise pipelines cannot absorb. One recent enterprise study found that dependency-based extraction using traditional NLP libraries could achieve roughly ninety-four percent of the performance of LLM-driven construction at a fraction of the cost, highlighting the economic tension at the heart of modern GraphRAG.

Docling Graph arrives in this gap. Released under the Linux Foundation AI & Data umbrella and backed by IBM, it does not try to eliminate LLMs from the pipeline. Instead, it disciplines them. The tool ingests documents—PDFs, Office files, images, markdown—and forces their contents through a schema-defined extraction process that outputs validated Pydantic objects before anything touches a graph database. The bet is that high-stakes domains such as chemistry, finance, and legal require exact entity connections, not semantic similarity.

The Core Bet: Schema-First Extraction

Most document-to-graph tools treat extraction as a text-generation problem: prompt an LLM to emit triples, then dump them into Neo4j or Memgraph and hope the syntax is valid. Docling Graph treats extraction as a validation problem. Users define Pydantic models that serve as both the extraction schema and the graph topology. A Person model is not merely a data class; it is a node type. Relationships are declared explicitly within the model using an edge helper, so the same Python class that validates the LLM’s output also dictates how NetworkX will wire the resulting directed graph.

This conflation of schema and structure is the project’s central insight. It means the LLM is not free-associating entities and hoping a downstream parser catches the mess. The output is schema-enforced by default. Stable identifiers are generated from designated fields, so the same person mentioned across multiple pages becomes a single node rather than a cluster of near-duplicate embeddings waiting for a deduplication pass. The graph carries edge metadata, supports provenance tracking, and is exportable to Cypher or CSV for ingestion into graph databases.

The pipeline is declarative and supports two distinct backends. For visually complex documents—scanned forms, instrument diagrams, or tables embedded in images—a local vision-language model backend leverages Docling’s layout analysis before extraction. For text-heavy corpora, an LLM backend handles the parsing, with automatic chunking to stay within context limits and a programmatic merge step that reassembles fragments without additional inference. This dual-backend design acknowledges a reality often ignored by text-only extraction tools: many high-value documents are not clean paragraphs, but messy visual artifacts that must be read before they can be reasoned over.

The approach inherits Docling’s document intelligence capabilities—reading order correction, table structure recovery, figure classification, and mathematical formula extraction—so the text fed into the LLM is already semantically coherent. As one integration partner notes, Docling excels at high-fidelity structural parsing, which is exactly the raw material a knowledge graph builder needs before it can reason over content. Without that structural fidelity, relationship extraction is reduced to guessing whether a number belongs to the row above or the caption below.

Provenance Without the LLM Tax

A persistent critique of LLM-driven extraction is the black-box problem: if the model claims a contract clause imposes a liability cap, where exactly did it read that? Docling Graph attempts to solve this with a deterministic provenance ledger. Every extracted node carries a __provenance__ attribute by default that points back to the source chunk and page, complete with bounding-box geometry. The documentation emphasizes that this requires no extra LLM calls; the pipeline captures it deterministically during conversion.

This is not a minor convenience. In regulated industries, an extraction tool that cannot show its work is a liability. By binding graph nodes to physical coordinates in the original document, Docling Graph creates an audit trail that vector-based RAG systems simply do not possess. The provenance data is part of the graph metadata, not an afterthought. When a downstream agent retrieves a node about a rheology experiment, it can point to the exact region of the PDF that justified the extraction, a feature that becomes essential when human experts must verify machine output. Because the provenance is captured during the initial document conversion stage, it survives even when the extraction falls back to a different backend or processing mode. The ledger is deterministic, meaning identical inputs produce identical provenance records, which simplifies regression testing when templates evolve.

Template Generation and the Ontology Bridge

Hand-writing Pydantic templates for every document type would doom the tool to niche academic use. Docling Graph offers two escape routes. First, it can induce templates from a small set of example documents. The LLM proposes classes, fields, and relationships as structured data—not as Python code—and a deterministic renderer compiles the result into a module. Candidates are filtered by deterministic gates: every identity example must appear verbatim in the source before it is accepted. A trial-run mode then performs a real extraction on the first document and prints an advisory quality report, giving the user a sanity check before they commit to the schema.

Second, for organizations that already maintain ontologies, the toolkit can compile templates directly from OWL, RDFS, SKOS, LinkML, or JSON Schema without invoking an LLM at all. The resulting template is paired with an editable SPEC YAML file; renaming an edge or reclassifying an entity becomes a one-line edit in the SPEC file rather than a code patch. This respect for existing semantic infrastructure suggests the authors understand that enterprise knowledge engineering rarely starts from a blank page. It also means that a domain expert can refine a template without touching Python.

Where It Fits—and Where It Doesn’t

Docling Graph occupies a specific tier in the extraction stack. Below it sit traditional NLP pipelines—spaCy, dependency parsers, industrial-grade tokenizers—that are cheap and fast but brittle across domains. Above it sit pure LLM extraction services that are flexible but unstructured and expensive. Docling Graph uses LLMs and vision-language models, routed through LiteLLM to support local runtimes like vLLM and Ollama as well as commercial APIs, but it constrains them heavily. The trade-off is cost and latency. A recent study on scalable GraphRAG noted that LLM-based knowledge graph construction remains the primary bottleneck for enterprise adoption, and that lightweight, non-LLM extraction can recover most of the accuracy at dramatically lower resource cost. Docling Graph does not eliminate that cost; it pays it in exchange for schema fidelity and provenance.

The project is also, candidly, sophisticated glue. It orchestrates Docling for parsing, LiteLLM for inference, Pydantic for validation, and NetworkX for graph construction. The value lies in the opinions it layers between those components: the chunking strategies, the processing modes, the deterministic graph fusion that merges multiple extracted graphs without additional LLM calls. That is not a criticism. Most production software is glue; the question is whether the adhesive is strong enough to hold under load. Docling Graph’s CLI and Python API suggest it is built for pipeline engineers who want to integrate document conversion into a larger MLOps workflow rather than for casual end users seeking a single-button solution.

Still, users should not expect a magic wand. Some advanced features—multi-pass delta extraction, dense skeleton-then-flesh modes—are marked experimental or advanced in the documentation. A developer’s first-hand account of building with the toolkit notes that several anticipated features, including native graph database integration and an interactive template builder, remain on the roadmap rather than in the release.

Outlook: From Conversion to Integration

The immediate practical value of Docling Graph lies in its deterministic graph fusion. Organizations can extract knowledge from separate documents—contracts, research papers, compliance reports—and merge the resulting graphs with a fully audited, deterministic process that requires no further inference. This addresses one of the dullest but most urgent problems in enterprise knowledge management: graph sprawl. When each document spawns its own isolated subgraph, the value of the graph collapses; fusion reassembles the fragments into a coherent whole.

Looking forward, the project appears to be marching toward deeper ecosystem integration. The documentation teases upcoming support for direct export to Neo4j and ArangoDB, as well as ontology-based template matching that would automatically select the best Pydantic schema for a given document based on semantic similarity. An existing integration with the Semantica framework already demonstrates this trajectory, combining Docling’s structural parsing with semantic-layer construction to produce RDF triplets and property graphs for downstream GraphRAG systems. If these capabilities mature, Docling Graph will move from a conversion utility toward a continuous knowledge-layer platform.

For now, it is a precise, opinionated tool for teams that have already decided vector search is insufficient. It will not replace cheap embedding pipelines for casual question-answering. But for domains where a wrong relationship is worse than no relationship at all, the discipline of Pydantic-shaped knowledge is a compelling proposition.