Summarisation of long documents & document sets

Knowledge workers lose hours reading 80-page reports, contract bundles, study packs and case files just to find out what matters. Long-context LLMs (1M-token windows are now standard at the frontier) can read an entire document or a stack of documents and produce a brief tailored to the reader: one-pager, key risks, action items, or a comparison table — each statement linked to the page it came from. The value lies in faster orientation and triage. The risk lies in plausible omissions and distortions, so good systems are built for verifiability, not just brevity.

Problem & value

• Today: managers, lawyers, analysts, case workers and researchers skim or read in full; summaries are written by hand (management briefs, Vermerke, Sachstandsberichte) or not at all. Document sets (data rooms, tender bundles, consultation submissions, regulatory packs) are often too large to read.
• AI value: first-read triage in minutes, consistent summary templates, cross-document synthesis that humans rarely have time for, multilingual input with summaries in the reader's language.
• Evidence: in the UK cross-government Microsoft 365 Copilot experiment (20,000 civil servants, Sept–Dec 2024), users reported saving an average of 26 minutes per day. Summarising documents, e-mails and meetings was among the most cited uses, but participants also flagged inaccurate outputs on nuanced, context-heavy material, and human oversight was required throughout [1].
• Quality reality check: on the Vectara leaderboard (short to medium documents up to ~24K words), hallucination rates of current models in Oct 2026 range from about 2% to over 20% depending on the model, so model choice matters [2]. For book-length input, FABLES found that summaries contain unfaithful claims about events and character states and over-weight the end of the text, and that LLM-based faithfulness checks agree poorly with human judgement [3].

Variants

• Single long document: annual reports, studies, expert opinions (Gutachten), standards, manuals, policy papers → executive summary, key-figure extraction, section-by-section digest.
• Document sets / multi-document synthesis: data rooms (see DOC-011), tender documents, file shares, literature packs (KNOW-008), consultation submissions (LANG-026) → themes, contradictions, comparison grids.
• Query-focused summarisation: "summarise everything about warranty clauses across these 40 contracts" — overlaps with RAG (KNOW-002) and GraphRAG (KNOW-013) for corpus-wide questions.
• Update / diff summaries: what changed between contract v3 and v4, between two versions of a regulation (feeds LANG-020 horizon scanning).
• Public sector: summaries of case files (e-Akte), council documents, Bundestag/Landtag printed matter and court decisions for internal briefing (Vorlagen, Sprechzettel).
• Domain variants: clinical record summaries (HLTH-018, own regulation), log/incident summaries (CODE-011), threat-intel reports (SEC-008), meeting transcripts (SPEECH-001).
• Format variants: audio overviews / podcast-style briefs (SPEECH-006 TTS), slide outlines, plain-language summaries (LANG-010), summaries in another language (LANG-009).
• Private use: understanding long terms and conditions, rental contracts, official letters (PERS-008), insurance policies, study materials — mostly via consumer chat apps or local models.

How it works

1. ingest (upload, DMS/SharePoint connector, e-mail, data room export)
2. parsing (PDF/scan
3. text + layout + tables via OCR/document parsing; keep page anchors)
4. routing by size: fits context (≤ few hundred k tokens)
5. single long-context call too large / many docs
6. map (per document/section)
7. reduce (merge, dedupe, rank) corpus-wide questions
8. retrieval or graph index (RAG/GraphRAG)
9. query-focused summary
10. summary generation against a brief (audience, length, mandatory fields, template)
11. grounding check (every claim cites page/section; optional NLI/hallucination scorer)
12. human review (owner of the decision reads summary + jumps to cited passages)
13. output (DMS note, Word/PDF brief, ticket, CRM/case record)

• Long-context first: frontier models accept 1M tokens (e.g. Claude Opus/Sonnet/Haiku 5.5 and Gemini models); for single documents this is simpler and usually better than chunking [4][5]. Put the instructions after the document and ask for quotes or page references [5].
• Map-reduce / hierarchical summarisation remains necessary for very large sets, cheaper models or self-hosted models with smaller windows; it also gives per-document summaries as a by-product.
• GraphRAG-style indexing (entity graph + pre-computed community summaries) beats plain RAG on "what are the main themes across the whole corpus" questions [6].
• Human-in-the-loop: the summary is a reading aid. Decisions, legal assessments and outgoing communications are made on the source, with citations making spot checks cheap.

Data preparation

The effort is moderate (roughly 20-30%) and sits mainly in getting documents readable, not in labelling. Needed: the documents themselves (PDF, Word, mail, minutes, reports, contracts, studies) and, for recurring use, templates for the summary format.

Where they live in an SMB: file shares, DMS, mailboxes, Teams or SharePoint, paper files and scans. Typical gaps: scanned PDFs without text layer, multi-column or table-heavy layouts that break parsing, duplicates and drafts next to final versions, no metadata, and confidential documents in folders that are open to everyone.

Preparation steps: 1. Choose a few document types to start with (for example contracts, minutes, tenders) and collect ten examples each. 2. OCR scans and convert formats (PREP-006, PREP-007; DOC-014 for robust extraction with Docling). 3. Remove duplicates and mark final versions (PREP-009, PREP-010). 4. Define the output formats with the readers (executive summary, risk list, action items) and keep two or three model summaries as gold standard (PREP-016, PREP-018). 5. Clean permissions before connecting a folder to an assistant (PREP-023), and set rules about which classes of documents may go into which tool (PREP-024). 6. For personal data in the documents, consider pseudonymisation (PREP-022).

Tools: Paperless-ngx for scans, Docling or MinerU for parsing, a spreadsheet for the test set.

Implementation paths

Which path for whom

• SMB — Path C: contracts, reports and customer documents are often confidential, Ollama or Open WebUI with a mid-size model runs on HW2 and the same platform also serves other use cases (Baukasten effect); Path A is fine for non-sensitive material when Copilot or Workspace is already licensed.
• Enterprise — Path B: a shared internal AI platform on EU-hosted APIs scales to many document types, with access rights and logging.
• Developer — Path C: vLLM or Ollama with Docling gives full control over chunking, long-context handling and costs.
• Private — Path C: local apps such as LM Studio or Ollama keep personal documents (health, tax, contracts) on the own device.
• Public — Path C: self-hosted or sovereign-cloud models keep files with personal and official data under own control.

Path A — Buy (off-the-shelf)

• Built into office/productivity suites: Microsoft 365 Copilot (Word, Outlook, Teams, SharePoint; EU Data Boundary), Google Gemini in Workspace and Google's notebook research tool (formerly NotebookLM) for source-grounded summaries and audio overviews, Adobe Acrobat AI Assistant for PDFs and multi-file PDF Spaces with page-level citations [7].
• EU/German AI workplaces: Langdock (Berlin; EU hosting, own-cloud or on-prem options) [8], Mistral Le Chat (France), and German public-sector/enterprise assistants built on sovereign clouds.
• Domain tools: legal (contract and case-file summaries in legal-tech suites — see DOC-010, KNOW-006), finance research platforms, DMS/ECM vendors adding summarisation (e.g. d.velop, DocuWare).
• Choose A when: documents live in M365/Workspace/a DMS that already offers it, and a DPA with EU hosting is in place. Check: no training on customer data, retention settings, citation display.

Path B — Build on cloud APIs

• Models: long-context frontier models via EU-routed endpoints — Claude via Amazon Bedrock (Frankfurt/EU regions) or Google Cloud, Gemini on Vertex AI EU regions, GPT models via Azure EU Data Zone, Mistral Large/Medium via la Plateforme (EU provider) [4][9].
• German-hosted open models as API: STACKIT AI Model Serving, IONOS AI Model Hub (e.g. gpt-oss-120b, Qwen3.5, Mistral Small) for data that must stay with a German provider [10][11].
• Pattern: document parser (Docling, Azure Document Intelligence, Mistral OCR) → prompt template per summary type with JSON schema (fields + citations) → prompt caching / context caching for repeated questions over the same document → batch API for bulk jobs (50% cheaper on Claude) [4][5].
• Orchestration: LangChain/LangGraph or LlamaIndex summarisation chains, n8n for DMS/e-mail triggers; evaluation harness with a hallucination scorer (e.g. HHEM-2.1-Open) on samples [2].

Path C — Self-host (open source)

• Models: Qwen3 family (e.g. Qwen3-30B-A3B-Instruct-2507: Apache-2.0, 262K native context, ~1M with extended config) [12]; Mistral Small 4 and Ministral 3 (Apache 2.0) [9]; gpt-oss-120b; larger Qwen3.5 MoE models for HW3/HW4.
• Stack: vLLM (prefix caching, OpenAI-compatible API) or SGLang/Ollama [13]; Open WebUI or AnythingLLM as front end; Docling/MinerU for parsing; Qdrant/pgvector if query-focused retrieval is needed; GraphRAG (Microsoft, open source) for corpus-level synthesis [6].
• Hardware: HW2 (24–32 GB GPU) runs a quantised 8–30B model with 32–128K context — enough for map-reduce over arbitrary sets. Very long contexts are memory-bound: the Qwen3-30B 1M-token setup needs about 240 GB of GPU memory (HW4) [12]; HW3 is the realistic sweet spot for 128–256K windows.
• Ops effort: moderate; main work is parsing quality, prompt templates and evaluation.

Path D — Custom training

Rarely worth it. Consider fine-tuning (LoRA on a 7–30B model) only for high-volume, strongly templated summaries (e.g. insurance claim files, standardised Vermerke in public administration) where house style and fixed fields matter, or to distil a frontier model's summaries into a cheaper self-hosted model. Needs a few hundred to a few thousand source/summary pairs approved by domain experts.

State of the art (2026)

• Context windows: 1M tokens is standard for frontier APIs (Claude 5.5 family, Gemini models); open models reach 256K natively and ~1M with extensions [4][5][12].
• Long ≠ understood: models still use information in the middle of long inputs worse than at the edges ("lost in the middle") [14], and multi-fact retrieval from long context is less reliable than single look-ups [5]. On LongBench v2 (contexts 8K–2M words), human experts scored 53.7% under time limits, the best direct-answer model 50.1% and a reasoning model 57.7% — reasoning helps [15].
• Faithfulness: hallucination rates differ by an order of magnitude across models; small models and some heavy reasoning models fare worse on summarisation faithfulness [2]. Omission (missing the crucial clause) is as dangerous as invention and harder to detect automatically [3].
• Techniques: citation-forced output, extract-then-abstract (first quote key passages, then summarise), map-reduce with dedupe, GraphRAG for global questions [6], and claim-level verification with NLI/hallucination scorers [2].
• Limits: tables and figures in scans, numbers and negations, legal nuance, mixed-language bundles, and confident summaries of documents that were only partially parsed.

Shared resources & combinations

• Same stack (llm-general + ocr-docparse + embedding/vector-db): KNOW-002 RAG assistant, KNOW-013 GraphRAG, KNOW-015 deep research, DOC-014 document parsing — one parsing pipeline and one LLM server (HW2/HW3, vLLM) serve all of them.
• Typical chains: SPEECH-001 transcript → LANG-008 summary; LANG-008 → LANG-009 translation or LANG-010 plain language; LANG-008 → SPEECH-006 audio brief; LANG-016 topic modelling + LANG-008 for per-theme summaries (also LANG-026 consultations).
• Domain consumers: DOC-011 due diligence, DOC-010 contracts, LANG-020 regulatory scanning, HLTH-018 patient charts, CODE-011 logs, SEC-008 threat intel, PERS-008 official letters.
• Guardrails (PII redaction, prompt-injection filtering for untrusted documents) shared with every LLM use case that ingests external content.

Effort, cost & KPIs

• Path A: days; per-user licence (office copilots, AI workplaces) — main effort is rollout, permissions hygiene (Copilot sees what the user can see) and training.
• Path B: 2–6 weeks for a pipeline with DMS integration and templates; cost driven by input tokens — long documents make caching and batch processing important.
• Path C: 1–3 months incl. hardware procurement; one GPU server can also host RAG and other text use cases.
• KPIs: reading/briefing time saved per document, share of summaries accepted without correction, error rate in sampled claim-level audits (unsupported claims, critical omissions), citation coverage, cost per summarised page, user adoption.

People & change

Involve management (which documents may go into AI tools), the department leads whose documents are summarised, and the end-users who read and rely on the summaries. IT and it-security operate the platform and set access rights; data-owners release the folders; the dpo checks personal data in documents (HR files, customer mails), the AVV with any provider, and whether a DPIA is needed.

Works council: depends. A summarisation tool alone does not monitor staff, but M365 Copilot or similar assistants log user activity and are objectively suited to monitoring (BAG), so §87(1) no. 6 BetrVG applies there; self-hosted tools with logging switched off or anonymised are less critical. Inform it in any case under §90(1) no. 3, and consider an expert under §80(3). Public sector: staff council.

Support AI literacy (Art. 4 AI Act): teach that summaries can omit or distort details, how to check against the source with citations, and never to rely on a summary for legal, medical or safety decisions (GUIDE-006). Humans stay in control: summaries are reading aids, the original is binding. Pilot with one team and one document type for a few weeks (GUIDE-009). Typical concerns: fear of surveillance (explain what is logged), trust in correctness (show citations and sample checks) and shadow AI (offer an approved tool, GUIDE-007).

Risks, regulation & compliance

• EU AI Act: summarisation as a reading aid is minimal risk (AI literacy duty, Art. 4). It becomes high-risk when the summary materially shapes decisions in Annex III areas — e.g. summarising applicant files in recruitment, benefit or asylum case files, or evidence for courts. Art. 6(3) can exempt purely preparatory tasks, but this must be assessed and documented. Art. 50(4) (from 2 Aug 2026): AI-generated text published to inform the public must be disclosed unless it underwent human review and someone holds editorial responsibility [16][17].
• GDPR: case files, HR documents and e-mails contain personal data → legal basis, DPA with provider, EU processing, no training on inputs, DPIA where special categories or large-scale public administration data are processed; German DSK guidance on AI and data protection applies [18]. Summaries are new personal data records — accuracy (Art. 5(1)(d)) and retention rules apply.
• Confidentiality: professional secrecy (§203 StGB for lawyers, doctors, tax advisers), trade secrets and classified material (VS-NfD) push towards Path C or sovereign hosting.
• Sector: GoBD if summaries become part of booking-relevant records; MDR if a clinical summary tool is intended for diagnosis/treatment decisions (see HLTH-018); DORA ICT third-party rules for financial entities using cloud LLMs.
• Risks & mitigations: omissions and distortions → mandatory citations, "what was not covered" section, sampled expert audits; prompt injection hidden in third-party documents → guardrails, no tool actions from summarisation pipelines; automation bias → train users to treat summaries as pointers, not substitutes.

Getting started

1. Pick 2–3 high-volume document types (e.g. tender bundles, supplier contracts, monthly reports) and define what a good summary must contain (fields, length, audience).
2. Build a test set of 20–50 real documents with expert-written reference summaries or checklists.
3. Compare an existing tool (Path A, e.g. your office copilot or an EU AI workplace) against a long-context API prompt with forced citations (Path B) on that test set.
4. Audit claim-level faithfulness and critical omissions on a sample; set review rules accordingly.
5. Roll out with templates and citations; collect corrections and re-audit quarterly or on model change.

Sources

1. UK Government — Microsoft 365 Copilot experiment: cross-government findings report — https://www.gov.uk/government/publications/microsoft-365-copilot-experiment-cross-government-findings-report/microsoft-365-copilot-experiment-cross-government-findings-report-html
2. Vectara — Hallucination Leaderboard (HHEM-2.3, updated Oct 2026) — https://github.com/vectara/hallucination-leaderboard
3. Kim et al. — FABLES: Evaluating faithfulness and content selection in book-length summarization — https://arxiv.org/abs/2404.01261
4. Anthropic — Claude models overview (context windows, batch pricing) — https://platform.claude.com/docs/en/about-claude/models/overview
5. Google — Gemini API long-context guide — https://ai.google.dev/gemini-api/docs/long-context
6. Edge et al. (Microsoft) — From Local to Global: A Graph RAG Approach to Query-Focused Summarization — https://arxiv.org/abs/2404.16130
7. Adobe — Acrobat AI Assistant for PDFs — https://www.adobe.com/acrobat/generative-ai-pdf.html
8. Langdock — EU-hosted AI workplace — https://langdock.com/
9. Mistral AI — Models overview — https://docs.mistral.ai/getting-started/models/models_overview/
10. STACKIT — AI Model Serving documentation — https://docs.stackit.cloud/products/data-and-ai/ai-model-serving/
11. IONOS — AI Model Hub — https://cloud.ionos.com/managed/ai-model-hub
12. Qwen — Qwen3-30B-A3B-Instruct-2507 model card — https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507
13. vLLM — inference and serving engine — https://github.com/vllm-project/vllm
14. Liu et al. — Lost in the Middle: How Language Models Use Long Contexts — https://arxiv.org/abs/2307.03172
15. Bai et al. — LongBench v2 — https://arxiv.org/abs/2412.15204
16. EU AI Act, Article 50 (transparency obligations) — https://artificialintelligenceact.eu/article/50/
17. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
18. Datenschutzkonferenz — Orientierungshilfe Künstliche Intelligenz und Datenschutz (May 2024) — https://www.datenschutzkonferenz-online.de/media/oh/20240506_DSK_Orientierungshilfe_KI_und_Datenschutz.pdf