Practices and hospitals spend large amounts of staff time on phone calls, paper forms and re-typing the same history, while patients do not know whether to wait, book or go to the emergency room. A conversational assistant collects symptoms and history in the patient's language, flags red flags, recommends a care level and hands the clinician a structured summary. The decisive question is regulatory: anything that recommends urgency or care level is normally medical-device software and must be CE-marked, not a generic chatbot.
Problem & value
- Today: intake by phone, clipboard or portal forms; staff triage by protocol; patients self-triage with search engines. Phone lines overload and urgent cases wait with routine ones.
- AI value: structured, multilingual pre-visit data collection; earlier detection of red flags; steering patients to the right level (self-care, GP, urgent care, emergency); less re-typing for staff; better scheduling and capacity use.
- Evidence is mixed and must be read carefully. Peer-reviewed tests of general-purpose LLMs show both over-triage and dangerous under-triage. A 2026 Nature Medicine study of a consumer health chatbot reported under-triage in 52 % of gold-standard emergencies, and recommendations shifted toward less urgent care when relatives downplayed symptoms [1]. Other studies on emergency-department conversations show high sensitivity but low specificity, i.e. frequent over-triage [2]. Purpose-built, certified triage engines with fixed clinical logic are evaluated differently. Validate any vendor figure on your own population.
Variants
- Pre-visit intake (no urgency advice): forms as conversation plus clinician summary; often not a medical device if it only collects and relays data.
- Symptom triage / care navigation: urgency recommendation. Medical-device software.
- Hospital front door: kiosk or tablet intake supporting nurse triage (Manchester, ESI).
- Payer / health portal: members routed to care, next to the 116117 service in Germany.
- Telephone voice intake: voicebot with human escalation (CONV-002).
- Private use: consumer symptom-checker apps, see Path A.
How it works
- channel (web chat, app, WhatsApp-style messenger, voice, kiosk)
- identity/consent step + transparency notice (AI, not a clinician)
- red-flag screen (deterministic rules, always first; chest pain, stroke signs, bleeding, suicidality)
- on hit: immediate escalation message + human/emergency handover
- adaptive interview (medical knowledge base or constrained LLM; asks next best question)
- triage engine (certified model/rules)
- urgency level + care recommendation
- summary generation (LLM drafts, clinician reviews)
- routing: booking, nurse call-back, EHR/FHIR draft entry
- logging, outcome follow-up, quality monitoring
- Two architecture families. (1) Certified probabilistic/rule-based engines (Bayesian or knowledge-graph based) drive the questions and the urgency decision; the LLM only handles natural language in and out. (2) LLM-first assistants reason freely; these are harder to validate and should be limited to non-clinical tasks such as form filling and summaries.
- Deterministic safety net: red-flag rules and escalation never depend on an LLM answer.
- Human-in-the-loop: a clinician or triage nurse sees the summary and can override; borderline or low-confidence cases, children, pregnancy and mental-health crises go to people. Patients can always reach a human or the emergency number.
Data preparation
Data needed: the clinic's intake questionnaires, triage protocols and guidelines, appointment and specialty catalogues, and anonymised past intake records for testing. In a practice or hospital these live in the practice or hospital information system, the DMS, paper forms at reception, scanned referral letters and PDFs of internal SOPs. Typical gaps: handwritten or scanned forms, free-text anamnesis without structure, inconsistent specialty names, outdated protocols, no labelled outcomes and strict access rights on patient records.
Preparation steps: (1) inventory sources and owners with the practice manager and medical lead (GUIDE-003); (2) digitise paper forms and move to digital capture at source (PREP-006, PREP-008); (3) agree one authoritative protocol set with clinicians (PREP-013, PREP-016); (4) build an evaluation set of cases with expert-labelled urgency levels (PREP-018, PREP-019); (5) pseudonymise any records used for testing (PREP-022) and check the legal basis for secondary use (PREP-025). Tools: Paperless-ngx or a DMS with OCR, Presidio, FHIR/HL7 export from the practice system. Expect about 30 % of the effort in data work, mostly protocol curation and test sets.
Implementation paths
Which path for whom
- SMB: Path A, because practices need a CE-marked, EU-hosted product with clinical validation; building your own triage logic creates medical-device liability.
- Enterprise: Path C, because hospital groups handle health data at scale and can run an open-source stack on a shared internal platform.
- Developer: Path B, because EU-region APIs allow a fast prototype of intake flows, to be certified before any clinical use.
- Public: Path B on a sovereign EU or German cloud, because procurement rules and data sovereignty favour controlled hosting; use Path A only for certified products.
- Private: Path A, because patients should use a regulated symptom checker rather than a general chatbot.
Path A — Buy (recommended)
- Certified triage/intake platforms: Infermedica (PL/EU; states its Medical Guidance Platform with Triage, Intake and Conversational Triage API is a Class IIb medical device under EU MDR) [3], Ada Health (DE; Ada Assess certified as Class IIa by TÜV SÜD) [4], Isabel Health, Symptoma (AT), Klinik-/Praxis-integrated tools such as Doctolib and Dedalus patient portals. Always check the current certificate and intended purpose yourself.
- Intake and forms only: EHR/PVS modules and patient portals.
- Choose A when: you need urgency advice, have no regulatory team and must be live in a few months. Verify EU hosting, a data processing agreement, MDR certificate scope, languages, EHR interfaces (HL7 FHIR, KIM, Practice systems).
Path B — Build on cloud APIs
- Use a certified triage API for the urgency decision (e.g. Infermedica Conversational Triage API, a stateful API where the server holds the conversation [3]) and an LLM from a provider with EU data residency (Mistral, Azure OpenAI EU regions, Google Vertex EU, AWS Bedrock Frankfurt) for language, summaries and form filling.
- Integration: FHIR R4 writes, calendar and queue APIs, n8n or Power Automate for routing.
- Caution: if your own LLM issues urgency recommendations, you become the manufacturer of medical-device software with all MDR duties.
Path C — Self-host (open source)
- Models: open-weight LLMs such as Mistral, Qwen, Llama or Gemma families, served with vLLM or Ollama; medically tuned open models (e.g. MedGemma) for summarisation and intake extraction; Whisper/faster-whisper for ASR; FHIR servers such as HAPI FHIR.
- Guardrails: NeMo Guardrails or Llama Guard-style classifiers plus deterministic red-flag rules.
- Hardware: HW3 for a 30-70B quantised model serving a hospital; HW2 for intake-only. Triage decisions still need a validated, regulated component.
Path D — Custom training
Justified for large hospital groups or health insurers with historic triage outcomes: fine-tune classifiers on triage-level labels and outcomes (admission, return visits). Needs thousands of labelled, de-identified cases, ethics approval, a clinical validation study and, for urgency output, full MDR conformity assessment including post-market surveillance.
State of the art (2026)
- Certified symptom assessment has matured: several EU vendors hold MDR Class IIa/IIb certification for triage; Infermedica also states its products are classed as high-risk under the AI Act [3][4][8].
- LLM front-ends on top of validated engines are the dominant pattern: natural conversation, structured medical logic underneath. Generic consumer health chatbots were launched in 2026 but independent testing showed serious under-triage in emergencies [1].
- Retrieval-augmented triage with guideline context reduced over-triage in a retrospective evaluation (28.8 % to 12.7 % in one study) while keeping under-triage low, but needs prospective validation [5].
- Limits and failure modes: under-triage when symptoms are vague or minimised; over-triage triggered by keywords such as bleeding or pain [2]; weak on children, multimorbidity, atypical presentation and dialects; hallucinated clinical statements; automation bias in staff.
Shared resources & combinations
- Same stack: CONV-003 booking, CONV-001 service bot, CONV-002 voicebot, HLTH-003 (ASR and documentation), KNOW-010 (guideline RAG), DOC-007 (record extraction).
- Downstream: HLTH-004 clinical decision support, HLTH-008 / PRED-011 (capacity and volume planning using intake data).
- Neighbouring: CONV-021 emergency call triage, CONV-009 wellbeing companions.
Effort, cost & KPIs
- Path A: 2-4 months incl. data-protection impact assessment, EHR connection and clinical sign-off; licence per patient interaction or per site.
- Path B: 4-9 months; certified API licence plus LLM usage; integration is the main cost.
- Path C/D: 9-18+ months if urgency output is in scope; intake-only far shorter.
- KPIs: completion rate of intake, % of summaries accepted without edits, under-triage and over-triage rate against clinician reference (primary safety KPI), time of staff per patient, phone-line wait time, no-show rate, share of patients escalated to humans, patient satisfaction, accessibility/language coverage, adverse-event reports.
People & change
Start with the medical lead and practice or hospital management, then the nursing and reception staff who will use the output daily; add IT for integration with the practice system. Bring the data protection officer in early: health data is special-category data, so a DPIA (Art. 35 GDPR) and an AVV (Art. 28 GDPR) with every processor are required. Legal/compliance clarifies the MDR status and AI Act high-risk duties, which apply to Annex III systems from 2 December 2027 and to AI in Annex I products from 2 August 2028. Works council: depends. If the tool only serves patients it may not trigger co-determination, but as soon as staff interactions or override rates are logged, it is suited to monitor performance and §87(1) no. 6 BetrVG applies; §90(1) no. 3 requires information when work processes change, and §80(3) allows an AI expert. Hospitals under church or public law use their staff representation.
AI literacy: under Art. 4 AI Act, take measures to support staff literacy; train triage staff on limits, automation bias and when to ignore the output. Pilot in one location or specialty, communicate that the assistant prepares but never decides, and address fears of misdirected urgent cases with mandatory red-flag escalation to a human. See GUIDE-005, GUIDE-004 and GUIDE-009.
Risks, regulation & compliance
- EU AI Act: high risk. Software intended for medical purposes under MDR/IVDR that needs a notified-body assessment is covered by Annex I via Art. 6(1), and patient triage systems for emergency healthcare are named in Annex III point 5(d). Date check: the Digital Omnibus (Regulation (EU) 2026/1744 per secondary reports) defers high-risk obligations, with Annex III reported at 2 Dec 2027 and Annex I product-embedded systems at 2 Aug 2028; verify against the Official Journal [6]. Transparency duty (Art. 50: tell people they talk to AI) and AI literacy (Art. 4) already apply.
- Art. 5: no manipulative techniques, no emotion inference where prohibited.
- Medical devices: MDR (EU) 2017/745 Rule 11 classifies software providing information for diagnostic or therapeutic decisions in class IIa or higher; qualification guidance MDCG 2019-11 applies [7]. Using an LLM for urgency advice makes you a manufacturer; use a certified product within its intended purpose and do not change it by prompt.
- GDPR: health data is special-category data (Art. 9); needs explicit consent or another Art. 9(2) basis, a DPIA (Art. 35), minimisation, EU processing, no training on patient chats without a legal basis, and strict retention. Art. 22: do not let automated decisions alone deny care access. In Germany note §203 StGB (professional secrecy; bind vendors as helpers) and the AVV.
- Ethics: no diagnosis claims, equal access (languages, digital divide), bias testing, prompt-injection protection.
- Mitigations: deterministic red-flag layer, certified engine, clinical safety case, continuous monitoring of under-triage, incident process, human override.
Getting started
- Decide the scope: administrative intake and summary only, or urgency advice. The second means MDR; involve data protection and regulatory advisors first.
- Map current intake flows and baseline KPIs (calls per day, intake minutes, no-show rate).
- Shortlist 2-3 certified products; request MDR certificates, intended purpose, clinical evaluation, hosting and sub-processors; run a DPIA.
- Pilot in one clinic with clinician review of every case and shadow-mode triage compared with the nurse decision.
- Review safety KPIs (especially under-triage) monthly before widening scope or channels.
Sources
- Nature Medicine (2026) — ChatGPT Health performance in a structured test of triage recommendations — https://www.nature.com/articles/s41591-026-04297-7
- BMC Emergency Medicine (2025) — Performance of ChatGPT, Gemini and DeepSeek for non-critical triage support using real-world conversations in the ED — https://link.springer.com/article/10.1186/s12873-025-01337-2
- Infermedica — Regulatory compliance and developer documentation (Conversational Triage API) — https://infermedica.com/regulatory-compliance and https://developer.infermedica.com/documentation/conversational-triage-api/faq/
- Ada Health — EU-MDR Class IIa certification press release — https://about.ada.com/press/221215-ada-health-receives-eu-mdr-certification/
- JMIR/PMC — Multi-evidence clinical reasoning with RAG for emergency triage, retrospective evaluation — https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12887567/
- DLA Piper — Digital AI Omnibus: deferral of high-risk obligations under the AI Act — https://knowledge.dlapiper.com/dlapiperknowledge/globalemploymentlatestdevelopments/2026/The-Digital-AI-Omnibus-Proposed-deferral-of-high-risk-AI-obligations-under-the-AI-Act
- European Commission — MDCG 2019-11 guidance on qualification and classification of software — https://health.ec.europa.eu/medical-devices-sector/new-regulations/guidance-mdcg-endorsed-documents-and-other-guidance_en
- Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj