Catalog / Healthcare & Clinical / HLTH-018

Patient chart summarisation & longitudinal record search for clinicians

Summarise one patient's whole longitudinal record and answer clinician questions over it, with every statement linked to its source entry.

Parts needed to run it yourselfDocument readingLanguage modelSearch & RAG

Hospital and practice records grow over years into hundreds of notes, lab series and external letters, and clinicians have minutes to find what matters. An AI layer over one patient's record produces a pre-visit summary and answers questions such as "when was anticoagulation last changed and why?", citing the exact source entry for every statement. Benefit: less chart-review time and fewer missed facts, provided that summaries stay verifiable and the system is validated for its intended purpose.

Problem & value

  • Today: clinicians scroll through the EHR or PVS, open PDFs from other providers and reconstruct the history by hand, often at admission, handover or before a consultation. Relevant facts hide in free text and in external letters; copy-forward notes add noise.
  • AI value: a verified overview in seconds, search by meaning rather than keyword, and a consistent hand-over format. Vendors report time savings, but figures come from vendor pilots; measure locally before relying on them [1][2].
  • Distinct from neighbours: HLTH-003 transcribes encounters, HLTH-011 writes outgoing letters, DOC-007 structures single documents and KNOW-010 answers general medical questions. This use case reads one patient's special-category record and must cite it.
  • EU context: the German ePA (electronic patient record) and the European Health Data Space (EHDS, Regulation (EU) 2025/327) make patient summaries and medication data portable, so a summarisation layer increasingly sees records from several providers [3].

Variants

  • Pre-visit brief (outpatient): one-page summary before an appointment from the PVS plus ePA.
  • Admission and handover (hospital): problem-oriented summary for ward rounds, shift change, ED arrival.
  • Question answering ("chat with the chart"): free-text queries with cited answers.
  • Longitudinal timeline: trends of labs, medication, weight or symptoms over years.
  • Patient-facing summary: plain-language view of own record; different risk profile, see HLTH-030.
  • Specialty views: oncology (treatment history), psychiatry (high sensitivity), long-term care.

How it works

  1. EHR / PVS / ePA (FHIR R4, HL7 v2, CDA, PDFs)
  2. access check (treatment relationship, role, purpose)
  3. scope = ONE patient
  4. normalise (FHIR resources; OCR for scanned letters; de-duplicate copy-forward text)
  5. index (hybrid: BM25 + embeddings, chunks keep resource ID, date, author)
  6. retrieval + rerank for the question / visit context (e.g. cardiology follow-up)
  7. LLM draft with mandatory citations to chunk IDs
  8. verification (citation exists, quote matches, dates/units/dosages cross-checked)
  9. clinician view: summary with click-through to source, "not found" explicit
  10. audit log (who asked what, which records were read)
  • Long context vs. retrieval: whole charts often exceed what a model uses reliably, even with large windows; production systems combine structured pre-aggregation (problem list, active medication, last values) with retrieval and map-reduce summarisation of notes.
  • Grounding: citation-first generation; sentences without a verifiable source are dropped or flagged.
  • Terminology: SNOMED CT, LOINC, ICD-10-GM and ATC mapping (see HLTH-028) makes queries like "any anticoagulant" work across brand names.
  • Human-in-the-loop: the clinician reads and remains responsible; summaries are never written back as facts without review.

Data preparation

Data lives in the hospital information system or EHR (notes, labs, medication, problem lists), a PACS/RIS for imaging reports, document archives with scanned external letters, and increasingly the ePA and EHDS patient summary. Typical gaps: free-text notes with abbreviations and local jargon, scanned letters and faxes, duplicate or fragmented patient identities, missing coding, contradictory medication entries, and fine-grained access rights that must follow the treatment relationship.

Preparation steps: get structured and unstructured data out through FHIR or HL7 interfaces (PREP-005); digitise and OCR external letters (PREP-006, DOC-014); build an abbreviation and synonym list per specialty (PREP-015); define the patient-identity and record-linking rules; map roles so that retrieval respects the treatment relationship (PREP-023); pseudonymise all data used for development and testing (PREP-022); create an evaluation set with clinicians (PREP-018). Tools: HAPI FHIR, Docling or Tesseract for scans, a vector index per patient context. The check in GUIDE-003 helps scope it. Rough effort: 50 to 60 percent of the project, since interfaces, identity and access rights dominate.

Implementation paths

Which path for whom

  • Enterprise (C): Whole patient records are special-category data and one platform can serve summarisation, coding and search, so a self-hosted open-weight stack inside the hospital network (shared internal AI platform) is the better base than per-seat products.
  • Developer (C): Open-weight models and FHIR tooling let you build and validate in a closed environment without sending real patient data to third parties.
  • Public (C): Public hospitals and authorities typically need sovereign, on-premises or German/EU government-cloud operation; buying is only an option where the EHR vendor offers a certified, EU-hosted module.

Path A — Buy (off-the-shelf)

  • EHR-native: Epic's generative-AI chart summaries (Epic is US-based, deployed also in Europe), Oracle Health Clinical AI, Microsoft Dragon Copilot (adds chart context to ambient notes), Meditech Expanse AI [1]. Penn Medicine's in-house "Chart Hero" shows the direction of travel for hospital-built tools [2].
  • EU/DACH vendors: CompuGroup Medical (CGM), Doctolib (FR; assistant products) and Dedalus (IT, hospital information systems) with AI add-ons; availability varies, so check current product pages.
  • Choose A when: your EHR vendor offers a validated module with EU hosting, and a data processing agreement plus MDR position is documented.

Path B — Build on cloud APIs

  • Models: Claude, Gemini, GPT and Mistral models via EU-region endpoints (Azure OpenAI EU Data Zone, Google Vertex AI EU, AWS Bedrock Frankfurt, Mistral La Plateforme); long-context models with structured output and citation features.
  • Data layer: FHIR servers (HAPI FHIR, Azure Health Data Services, Google Cloud Healthcare API, AWS HealthLake); hybrid search in OpenSearch or pgvector; Azure AI Search.
  • Choose B when: you have a development team, an existing FHIR facade and a DPA with SCC-free EU processing. Note that patient data to a US-headquartered provider still raises Chapter V questions.

Path C — Self-host (open source)

  • Models: open-weight LLMs such as Llama 3.x/4, Qwen3, Mistral Small/Large, gpt-oss, Gemma 3 and medical fine-tunes (MedGemma, OpenBioLLM); serving with vLLM or llama.cpp.
  • Retrieval: BGE-M3 or multilingual-e5 embeddings, bge-reranker, OpenSearch or pgvector; Docling for scanned letters; German clinical NER (e.g. medspaCy-like pipelines, GBERT-based models).
  • Hardware: HW3 (one 48-96 GB GPU) for a ~30-70B quantised model serving a hospital department; HW4 for house-wide concurrency. Data never leaves the hospital network.
  • Fits university hospitals and Länder-run clinics with strict data residency.

Path D — Custom training

Rarely first. Fine-tune embedding or reranker models on in-house query/document pairs, or train a small model for note-type classification and negation/temporality. Needs clinician-labelled examples and ethics/data-protection approval for secondary use of records.

State of the art (2026)

  • Evidence: studies on LLM clinical summarisation report summaries rated comparable to or better than clinician-written ones on completeness, but also omissions, wrong attributions and fabrications [4].
  • Long-context limits: performance degrades on facts buried in the middle of very long inputs; chunked retrieval plus structured pre-aggregation is the robust pattern.
  • Techniques: citation-constrained decoding, claim-level verification by a second model pass, FHIR-aware tool use (agent queries the FHIR API for "latest HbA1c" instead of reading text), temporality and negation handling ("no history of X").
  • Interoperability: FHIR R4 and the EHDS patient summary/medication priority categories [3].
  • Failure modes: omitted rare but critical facts (allergy, anticoagulation), mixed-up patients or timelines, outdated copy-forward text presented as current, over-trust (automation bias), prompt injection through text inside external documents.

Shared resources & combinations

  • Same stack (embedding-text, reranker, vector-db, llm-general): DOC-014 document parsing for RAG, HLTH-004 clinical decision support, HLTH-030 message reply drafting, HLTH-011 discharge letters.
  • Upstream/downstream: HLTH-003 supplies new encounter notes; HLTH-028 terminology mapping improves retrieval; HLTH-011 reuses the summary as the draft basis.
  • Infrastructure: one self-hosted HW3 model server plus a FHIR/search layer can serve all clinical assistants in a hospital.

Effort, cost & KPIs

  • Path A: weeks to quarters, driven by EHR vendor roadmap, contracts and clinical governance.
  • Path B: 3-6 months for a validated pilot on one department; cost is dominated by integration, evaluation and governance rather than tokens.
  • Path C: 6-12 months plus GPU operations team.
  • KPIs: chart-review time per patient, citation validity rate, omission rate against a clinician gold summary (sampled), clinician acceptance and edit rate, safety incidents, access-log anomalies.

People & change

Involve management and the medical director first for the decision and clinical responsibility, then the ward or department leads as process owners. IT and IT security are needed for interfaces, hosting and access control; the DPO must be in from the start, because this is special-category data (Art. 9 GDPR) at scale, so a DPIA is required and an AVV with every processor. Clinicians and nursing staff are the daily users and should co-design what a summary must contain and how sources are shown.

Works council: yes in practice. Summaries and search logs record who looked up which record and when, which makes the system objectively suited to monitor behaviour (§87(1) no. 6 BetrVG, BAG); §90(1) no. 3 also requires timely information, and engaging an AI expert counts as necessary under §80(3). In public hospitals the staff council (Personalrat) applies under BPersVG or state law. Agree on logging scope, retention and a ban on performance evaluation.

AI literacy (Art. 4 AI Act, support measures): train on automation bias, omissions and checking the cited source entry. Run a pilot on one ward and one use (handover or pre-visit summary), keep the clinician as the decision-maker and make every statement clickable to its source. Address time-pressure fears by measuring documentation time saved, not output. See GUIDE-004, GUIDE-005 and GUIDE-008.

Risks, regulation & compliance

  • EU AI Act: depends on intended purpose. A tool that merely summarises or retrieves may fall outside the medical-device definition, but one that supports diagnosis or treatment decisions is likely software as a medical device (MDR Rule 11, class IIa or higher) and thus high-risk under Art. 6(1) with Annex I [5]. Vendors should state the intended purpose explicitly; the AI literacy duty (Art. 4) applies to all deployers. Not an Art. 5 prohibited practice.
  • GDPR: health data is special-category (Art. 9); needs a legal basis for treatment (Art. 9(2)(h)), DPIA (Art. 35), strict role-based access limited to the treating team, processor contracts, and national rules (e.g. §630f BGB, state hospital laws, §203 StGB confidentiality) [6].
  • EHDS: primary-use rules on patient access, summaries and interoperability phase in over the coming years; plan for EEHRxF-compatible data [3].
  • Risks and mitigations: hallucination and omission (citations, verification, sampling audits); access creep (per-patient scope, logging); bias across languages and dialects; automation bias (show sources, not just conclusions); prompt injection (treat record text as untrusted data, no tool actions from it).

Getting started

  1. Pick one workflow (e.g. admission summary for internal medicine) and record current chart-review time.
  2. Check data access: FHIR API or export, ePA connection, and a DPIA before any model sees real data.
  3. Build a gold set of 50-100 records with clinician-written key facts, including traps (allergies, anticoagulants, negations).
  4. Prototype with citation-mandatory retrieval; measure omissions and citation validity, not fluency.
  5. Clarify intended purpose and MDR status with the manufacturer or your regulatory team, then pilot with clinician-in-the-loop and an incident reporting path.

Sources

  1. Epic Systems — AI and generative-AI features overview — https://www.epic.com/
  2. Penn Medicine / Penn Medicine News — AI tools for clinicians (Chart Hero) — https://www.pennmedicine.org/news
  3. Regulation (EU) 2025/327 — European Health Data Space — https://eur-lex.europa.eu/eli/reg/2025/327/oj
  4. Van Veen et al. — Adapted large language models can outperform medical experts in clinical text summarization, Nature Medicine 2024 — https://www.nature.com/articles/s41591-024-02855-5
  5. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  6. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
  7. HL7 FHIR R4 specification — https://hl7.org/fhir/R4/
  8. MDCG 2019-11 — Guidance on qualification and classification of software under MDR — https://health.ec.europa.eu/system/files/2020-09/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf