Knowledge graph construction from documents (GraphRAG)

Plain vector RAG retrieves similar text chunks, but struggles with questions such as "how are these suppliers, projects and incidents connected?" or "what are the main themes across 10,000 reports?". GraphRAG lets an LLM read the documents once, extract entities and relationships into a knowledge graph, and then answers by walking the graph and its pre-computed summaries. It is powerful for connected, multi-document questions, but indexing is costly and it does not beat simple RAG on simple lookups, so it should be added deliberately on top of KNOW-002, not instead of it.

Problem & value

• Today: knowledge is scattered across thousands of documents. Chunk-based RAG finds passages that look similar to the question but cannot reliably join facts from several documents or summarise a whole corpus. Manually built ontologies and graphs are accurate but slow and expensive.
• AI value: LLM-based extraction builds and updates the graph automatically; the graph adds explicit structure (who/what/how related), traceable provenance and corpus-level summaries.
• Evidence: the original Microsoft GraphRAG paper reports substantial gains over conventional RAG in comprehensiveness and diversity of answers for global "sensemaking" questions over corpora of about 1 million tokens [1]. Independent benchmarking is more sober: GraphRAG-Bench notes that GraphRAG frequently underperforms vanilla RAG on many real-world tasks and studies when it actually helps [5]. Treat the graph as a targeted tool for relational and global questions.

Variants

• Corpus-wide sensemaking: community summaries for themes, trends, risks across large report sets.
• Multi-hop enterprise Q&A: connecting projects, people, products, suppliers, incidents (with KNOW-012 expert finding).
• Regulatory / contract graphs: clauses, obligations, standards and their cross-references (KNOW-009).
• Scientific / patent graphs: entities and claims across papers (KNOW-007, KNOW-008).
• Agent memory: temporal graphs of facts that change over time for assistants and agents (Graphiti, Cognee).
• Investigative link analysis: entity resolution over case material (KNOW-026; see its legal limits).
• Private use: a personal knowledge graph over notes and papers (KNOW-018).

How it works

1. documents
2. parsing/OCR (layout, tables)
3. chunking
4. LLM extraction of entities, relations, claims (guided by schema/ontology)
5. entity resolution & merging (dedupe "BMW AG" / "BMW")
6. graph store (+ embeddings of chunks/entities)
7. community detection
8. community summaries
9. query: classify question
10. local (entity neighbourhood) | global (community summaries) | hybrid graph+vector | plain vector
11. LLM answer with citations
12. human review of schema, sample extractions and high-stakes answers

• Extraction: an LLM prompted with a schema (or free-form) returns entity/relation triples with source references. Quality depends on schema, model and chunking; precision and recall must be audited on a sample.
• Graph + vector hybrid: retrieval combines graph traversal with embedding search and reranking.
• Query modes in Microsoft GraphRAG: Global Search (community summaries), Local Search (entity neighbourhood), DRIFT Search (local plus community context) and Basic Search (plain vector top-k) [2].
• Lazy / cheap indexing: LazyGraphRAG replaces LLM extraction at index time with NLP noun-phrase extraction and defers LLM work to query time. Microsoft reports indexing cost equal to vector RAG and 0.1% of full GraphRAG (own evaluation) [3].
• Human-in-the-loop: domain experts define the schema, review sampled extractions, resolve merge conflicts and approve graph changes for critical domains.

Data preparation

The graph is only as good as the documents behind it: contracts, reports, wiki pages, tickets, e-mail and project folders. In an SMB these sit in file shares, a DMS, SharePoint and mailboxes; in ERP/CMDB you find master data that can anchor entities. Typical gaps are scanned PDFs without text, duplicate and outdated versions, inconsistent names for the same supplier or product, missing metadata, and access rights that do not match what the graph would reveal.

Preparation steps: (1) inventory sources and pick one bounded corpus (PREP-001, PREP-002); (2) OCR scans and convert legacy formats (PREP-006, PREP-007, DOC-014); (3) remove duplicates and old versions and clean the folder structure (PREP-009); (4) define the entity and relation types and a synonym/abbreviation list before extraction (PREP-015, PREP-012); (5) carry over document permissions so answers respect access rights (PREP-023); (6) build a small question set with domain experts for evaluation (PREP-018).

Tools: Docling or Unstructured for parsing, Paperless-ngx or a DMS for OCR, OpenRefine or simple scripts for entity matching. Expect roughly 50-60 percent of the total effort in data preparation and schema design, the rest in extraction tuning and evaluation. Start with one department and a few hundred documents.

Implementation paths

Which path for whom

• Enterprise: Path B - graph extraction is LLM-heavy but runs well on EU-region cloud APIs, and the existing data and knowledge platform can host the graph database.
• Developer: Path C - open frameworks (LightRAG, Microsoft GraphRAG) with local models give full control over schema, cost and experiments.
• SMB: Path B - no mature SMB product fits, so a small build on cloud APIs with the own document sources beats buying a platform.
• Public sector: Path C - sensitive records and procurement rules favour self-hosting or a sovereign EU/German cloud.

Path A — Buy (off-the-shelf)

• Graph platforms with built-in GraphRAG: Neo4j (Aura, LLM Knowledge Graph Builder) [4], Stardog, Graphwise (Ontotext + Semantic Web Company, Bulgaria/Austria, EU), metaphacts (Walldorf, DE, EU), eccenca (Leipzig, DE, EU), Memgraph, FalkorDB. Check each vendor's current GraphRAG feature set.
• Enterprise search and RAG suites increasingly add graph features; see KNOW-001/KNOW-002.
• Choose A when: you already run a graph or semantic-data platform, or need enterprise governance and support.

Path B — Build on cloud APIs (recommended starting point)

• Frameworks: Microsoft GraphRAG (MIT, Python; the repository states it is a research effort now largely in maintenance mode) [6]; neo4j-graphrag-python with SimpleKGPipeline, supporting OpenAI, Azure OpenAI, Vertex AI, Gemini, Anthropic, Mistral, Cohere, Ollama and Bedrock [7]; LlamaIndex PropertyGraph and LangChain graph transformers.
• Models: a cheap, fast LLM for bulk extraction (e.g. Claude Haiku/Sonnet class, Gemini Flash, GPT mini class, Mistral Small/Medium via the EU provider Mistral AI), a stronger model for answers. Use EU regions or EU providers when documents contain personal or confidential data.
• Graph store: Neo4j Aura (EU region), Amazon Neptune (Frankfurt), Azure Cosmos DB (Gremlin), or Postgres with a graph layer.
• Cost warning: the GraphRAG repository itself warns that indexing "can be an expensive operation" and advises starting small [6]. Budget tokens for a pilot subset first.

Path C — Self-host (open source)

• Stack: Microsoft GraphRAG, LightRAG (MIT; graph + vector, five query modes, incremental updates and deletion, PostgreSQL recommended for production, Neo4j/Memgraph/Milvus/Qdrant/OpenSearch backends, Docling/MinerU parsing) [8], HippoRAG 2 (ICML 2025, Personalized PageRank retrieval) [9], Graphiti (Apache-2.0, temporal graphs on Neo4j/FalkorDB) [10], Cognee (Apache-2.0, agent memory graph, runs locally with small models) [11].
• Models: open-weight LLMs of ~30B class and up for extraction (Qwen3, Mistral Small/Medium open weights, Llama, gpt-oss) served via vLLM or Ollama; embedding models such as BGE-M3 or Qwen3-Embedding; a reranker (bge-reranker).
• Hardware: HW2 for small corpora (thousands of pages) with a quantised ~30B model; HW3 for larger corpora and higher extraction quality, since indexing is token-heavy and runs for hours/days.
• Ops effort: significant: graph DB operations, re-indexing on schema change, extraction QA.

Path D — Custom training

Worth it only for narrow, high-volume domains (biomedical, legal, patents) where a fine-tuned small model or a NER/relation-extraction encoder (BERT-class) can replace expensive LLM extraction. Needs a few hundred to thousands of annotated entity/relation examples; reviewer corrections of the extraction step are the natural source. Also consider training a domain embedding model.

State of the art (2026)

• Cost-aware GraphRAG: LazyGraphRAG and similar "light" variants avoid full LLM extraction at index time. Microsoft's own evaluation (5,590 news articles, 100 synthetic queries, LLM judge) shows it beating competing conditions on local queries at vector-RAG cost and needing a larger budget to match global search; self-reported results should be re-tested on your data [3].
• Memory-inspired retrieval: HippoRAG 2 claims to outperform standard RAG across factual, sense-making and associative tasks (about 7% gain on associative memory over a strong embedding baseline) after earlier graph methods regressed on simple factual questions [9].
• Temporal graphs for agents: Graphiti invalidates superseded facts instead of deleting them and keeps provenance to source episodes [10]; Cognee targets persistent agent memory [11].
• Ecosystem consolidation: LightRAG merged multimodal parsing (RAG-Anything) in 2026 and added OpenSearch support [8]; Neo4j ships an official GraphRAG package and a no-code graph builder [4][7].
• Honest evaluation: GraphRAG-Bench evaluates graph construction, retrieval and generation across fact retrieval, complex reasoning, summarisation and creative generation, and shows graphs do not always win [5].
• Limits and failure modes: extraction errors and hallucinated relations; entity-resolution mistakes merging different entities; high indexing cost and slow re-indexing; stale graphs; Neo4j's builder works best on long-form English text and is weaker on tables, images and slides [4]; loss of document access rights unless ACLs are carried into the graph.

Shared resources & combinations

• Shared stack: parsing (DOC-014), embeddings + vector DB + reranker and LLM serving are the same as KNOW-002 and KNOW-001; a hybrid index (vector + graph + BM25) serves all three. One HW2/HW3 vLLM server can serve extraction, answering and embeddings.
• Natural combinations: KNOW-002 (graph as an additional retriever for multi-hop and global questions), KNOW-009 (standards and regulation cross-references), KNOW-012 (expert finding from the people-project-skill graph), KNOW-014 (organisational memory and lessons learned), KNOW-026 (entity resolution and link analysis, with strict legal limits), AGENT-002 (agents using the graph as tool/memory).
• Graph database and ontology are reusable assets across all of these.

Effort, cost & KPIs

• Path A: weeks for a pilot; licence per graph platform plus LLM usage.
• Path B: 2–4 months to a production pipeline (schema design, extraction QA, ACLs, evaluation).
• Path C: 3–6 months incl. GPU server and graph DB operations.
• Cost drivers: LLM tokens for extraction and summarisation (dominant), re-indexing frequency, graph DB hosting, expert review time.
• KPIs: answer correctness on a multi-hop/global test set vs. plain RAG baseline, extraction precision/recall on a labelled sample, citation faithfulness, cost per 1,000 pages, latency, index freshness, share of questions routed to the graph.

People & change

Involve management early for scope and budget, and the department lead as process owner who defines the questions the graph must answer. The data owners of the document sources (DMS, file shares, wiki) must approve the corpus and its access rights; IT integrates parsing, graph database and identity; information security checks the model and prompt-injection risks (see GUIDE-013). End users join from the pilot onwards and judge whether answers and citations are useful.

Works council: depends. If the corpus only holds technical documents and no usage logs are evaluated, co-determination is unlikely; as soon as the system logs user queries or extracts facts about named employees, §87(1) no. 6 BetrVG applies, and §90(1) no. 3 requires timely information about AI-supported work processes. Public bodies involve the Personalrat. Documents with personal data (e-mail, HR files, contracts with named contacts) need the DPO, an AVV with each processor and often a DPIA (GUIDE-005); exclude HR and mailboxes in the first phase.

AI literacy: support staff with a short training on citations, hallucinated relations and verification (GUIDE-006). Communicate that the graph supports research and does not replace experts; keep source links visible and let humans confirm critical facts. Typical concern: "the graph exposes what I was not meant to see" - address it with permission-aware retrieval and a pilot group of 5-10 knowledge workers (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: typically minimal risk (internal knowledge tooling); AI literacy duty (Art. 4) applies. Transparency duties (Art. 50) apply if it is exposed as a chatbot to users. It becomes high-risk only through context, e.g. when graph outputs feed decisions on employees (Annex III employment) or law enforcement; Annex III obligations were postponed to 2 December 2027 by the AI Omnibus [12]. Graph-based profiling of individuals (social scoring) is prohibited under Art. 5.
• GDPR: graphs concentrate personal data (people, roles, relations) and make profiling easy; need a legal basis, data minimisation, deletion that propagates to nodes, edges and summaries, and DPIA for people-centric graphs. Choose EU hosting or on-prem for sensitive corpora.
• Security: preserve document ACLs in graph queries; community summaries can leak restricted content to users without access to the sources. Prompt injection in ingested documents can poison extraction.
• Quality risks: hallucinated edges. Mitigations: schema constraints, confidence scores, provenance links on every edge, sampled expert audits, citation-required answers.

Getting started

1. Collect 20–30 real questions that plain RAG answers badly (multi-hop, "across all documents") and a baseline from KNOW-002.
2. Pick a pilot corpus of a few hundred documents; define a small schema (5–10 entity and relation types).
3. Run a framework (Microsoft GraphRAG, LightRAG or Neo4j SimpleKGPipeline) with a cheap extraction model; audit 100 extracted triples.
4. Compare graph vs. vector vs. hybrid on your question set and measure cost per 1,000 pages.
5. Add ACL handling, incremental updates and a routing layer before rolling out.

Sources

1. Edge et al., "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" — https://arxiv.org/abs/2404.16130
2. Microsoft GraphRAG documentation — https://microsoft.github.io/graphrag/
3. Microsoft Research, "LazyGraphRAG: Setting a new standard for quality and cost" — https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/
4. Neo4j LLM Knowledge Graph Builder — https://neo4j.com/labs/genai-ecosystem/llm-graph-builder/
5. "When to use Graphs in RAG" (GraphRAG-Bench) — https://arxiv.org/abs/2506.05690
6. microsoft/graphrag repository — https://github.com/microsoft/graphrag
7. neo4j/neo4j-graphrag-python — https://github.com/neo4j/neo4j-graphrag-python
8. HKUDS/LightRAG — https://github.com/HKUDS/LightRAG
9. HippoRAG 2, "From RAG to Memory" — https://arxiv.org/abs/2502.14802
10. getzep/graphiti — https://github.com/getzep/graphiti
11. topoteretes/cognee — https://github.com/topoteretes/cognee
12. European Commission — AI Act regulatory framework and timeline — https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai