Ambient clinical documentation (AI scribe)

Clinicians spend a large share of their day on documentation. An AI scribe listens to the consultation, separates speakers, transcribes and drafts a note in the practice's template, which the clinician edits and signs. The aim is less after-hours charting and more eye contact with the patient. In the EU the hard parts are not the models but consent, data residency, medical-device status and clinician accountability for the final note.

Problem & value

• Today: doctors type during the visit, dictate afterwards (see SPEECH-005) or write notes in the evening. Documentation is a recognised driver of burnout and lost patient time.
• AI value: the draft note exists when the patient leaves; structure follows the template; patient letters and referrals can be generated from the same transcript.
• Evidence: early deployments and vendor studies report reduced documentation time and lower burnout scores; independent controlled evidence is still thin, and results vary by specialty and by how much editing the drafts need. Treat vendor-reported percentages as marketing until reproduced locally with your own pilot [1][6].

Variants

• GP / outpatient practice (Praxis): one-clinician, one-room, tight integration with the practice management system (PVS), German/European languages.
• Hospital ward and emergency: noisy, multi-speaker, handover notes; integration with the hospital information system.
• Specialty templates: psychiatry/psychotherapy (especially sensitive, often excluded), radiology reporting is a different workflow (see HLTH-001).
• Telemedicine: record the video call audio directly, no room microphone.
• Nursing and care: voice-based documentation without a dialogue partner is HLTH-025.
• Private use: patients recording their own visit for personal summaries is possible but needs the clinician's consent in the EU; no medical advice use.

How it works

1. consent capture (patient informed, recording opt-in)
2. audio capture (room mic / phone / telehealth stream), on-device or encrypted upload
3. ASR + speaker diarization (clinician vs patient vs relatives)
4. de-identification / context pull from EHR (optional)
5. LLM note generation into template (SOAP, specialty sections) with source-linked statements
6. safety checks (missing sections, unsupported claims, medication/dose consistency)
7. clinician review & edit (mandatory)
8. sign-off
9. write-back to EHR/PVS; audio and transcript deleted per retention policy

• ASR: medical vocabulary, accents and code-switching are the weak spot; domain lexicons and speaker diarization matter more than the base model.
• Note generation: an LLM turns transcript plus template into prose; the best systems link each sentence back to the transcript passage so the clinician can verify quickly.
• Human-in-the-loop: the clinician reads and signs every note; AI never writes into the legal record without sign-off. Patient consent is checked before recording.

Data preparation

Ambient scribing needs little historical data; the main inputs are the live consultation audio, the patient context from the PVS/KIS (diagnoses, medication, last notes) and the practice's documentation templates. These live in the practice system, in Word templates and often only in doctors' heads. Typical gaps: no consent routine for recording, poor microphones and noisy rooms, dialect and medical terminology, inconsistent templates between doctors, abbreviations, and locked-down practice systems without an interface.

Preparation steps: (1) check the audio setup, meaning microphone, room, and a tablet or phone policy; (2) collect two or three gold-standard notes per specialty and the preferred note structure (PREP-016); (3) compile a list of practice terminology, drug names and abbreviations (PREP-015); (4) clarify the interface to the PVS/KIS (PREP-005) or accept copy-and-paste in the pilot; (5) define how patient consent is obtained, shown and documented; (6) decide on audio retention, ideally immediate deletion after transcription. For a self-built variant, collect a small test set of de-identified consultations with a doctor (PREP-018, PREP-022).

Effort: low. Most work is workflow, consent and template alignment, not data cleaning, which is why this case suits a quick pilot. See GUIDE-003.

Implementation paths

Which path for whom

• SMB (medical practice, small clinic): Path A, because mature scribe products with EU hosting, AVV and PVS/KIS connectors exist at per-user prices and the process is standard.
• Enterprise (hospital, MVZ group): Path B, because deep integration into the clinical information system and own templates matter, and volume makes seat costs high.
• Developer: Path C, because Whisper-class speech recognition and an open LLM keep consultation audio in-house and allow experiments.

Path A — Buy (recommended for most practices)

• European vendors: Tandem Health (Sweden; markets its scribe as a CE-marked MDR class IIa device, to be verified against the certificate) [2], Nabla (France), Doctolib integrations in France/Germany, and various German PVS vendors adding scribe modules; check your PVS vendor first.
• International: Heidi Health, Abridge, Microsoft Dragon Copilot (successor of Nuance DAX), Suki. Verify EU hosting, subprocessor list (which LLM, where) and training-on-customer-data policy.
• Check list: EU data residency, DPA and subprocessors, no secondary use of audio, MDR status and intended purpose, PVS/EHR integration, German/other language quality, deletion of audio, audit log.
• Choose A when: you are a practice or small hospital without ML staff.

Path B — Build on cloud APIs

• ASR: EU-region speech services (Azure AI Speech, Google Cloud Speech-to-Text, AWS Transcribe Medical/HealthScribe where available in the region) or Mistral's Voxtral API (EU provider) [4].
• LLM: Claude, GPT, Gemini or Mistral models via EU-region endpoints under a zero-retention DPA; structured outputs for template fields.
• Orchestration: FHIR (DocumentReference, Composition) for EHR write-back; consent service; audit log.
• Caveat: building a product that drafts clinical documentation for third parties may make you a medical-device manufacturer (see risks); an in-house tool for your own staff is a lighter case but still needs a risk assessment.

Path C — Self-host (open source)

• ASR: Whisper large-v3 / large-v3-turbo [3] via faster-whisper or whisper.cpp; Voxtral (open weights) [4]; pyannote for diarization; fine-tune with medical term lists.
• LLM: open-weight 30-120B models (Qwen, Mistral, Llama, gpt-oss families) served with vLLM [5]; a smaller model for template filling on HW2.
• Hardware: HW3 for a hospital-scale service with 70B-class quality; HW2 can serve a single practice with a mid-size model and a batch ASR job.
• Why: audio never leaves the building; strongest answer for psychiatry, hospitals with strict data-protection officers and public providers.

Path D — Custom training

Fine-tune ASR on in-domain audio (dialects, specialty terminology) and note-style adaptation on a few hundred clinician-edited notes. Only worthwhile at scale; the corrected notes from daily use are the natural data source, subject to a lawful basis for secondary use.

State of the art (2026)

• Ambient scribes are the fastest-adopted clinical LLM application; large US health systems have rolled them out widely, and European providers are following, with Tandem, Nabla and Heidi expanding in EU markets [2].
• Trend: scribes move from notes to orders, coding hints (HLTH-005), referral letters and patient summaries; EHR vendors bundle them.
• Trend: source-linked notes and clinician-editable evidence panels reduce review time and catch hallucinations.
• Limits: omissions (the draft misses a relevant statement), hallucinated findings or negation errors ("no chest pain" becomes "chest pain"), wrong attribution between speakers, weak performance on dialects and noisy rooms, and automation bias: tired clinicians sign without reading.
• Independent evaluation is rare; vendor comparisons are mostly self-published, so run your own blinded comparison [6].

Shared resources & combinations

• Same stack (asr + diarization + llm-general): SPEECH-001 meeting transcription, SPEECH-005 professional dictation, HLTH-025 nursing documentation.
• Downstream: HLTH-011 discharge letters, HLTH-005 coding and billing, HLTH-030 patient messages, HLTH-018 chart summarisation as the context input.
• Privacy tooling: SEC-018 for records of processing and DPIA support; PERS-005 for local deployment patterns.
• One GPU server (HW3) can host ASR and LLM for all the speech-to-text use cases above.

Effort, cost & KPIs

• Path A: weeks; per-clinician monthly subscription; pilot with 5-10 clinicians.
• Path B: 3-6 months incl. EHR integration, consent flows and DPIA.
• Path C: 4-9 months; GPU server, MLOps, language-specific evaluation.
• Cost drivers: per-user licences, integration with the PVS/EHR, change management and training.
• KPIs: documentation time per visit, after-hours charting time, note edit distance (share of text changed), omission/hallucination rate found in audit samples, clinician burnout and patient-experience scores, patient opt-out rate.

People & change

Start with the practice owner or medical director for the decision and budget, then the doctors and medical assistants who use the tool every day, because their templates and habits define quality. IT or the PVS vendor handles integration and devices. The DPO is mandatory in practice: audio of patient conversations is special-category data, so legal basis, AVV with the supplier, a DPIA and the information duty towards patients must be settled before the pilot (GUIDE-005). Patients are stakeholders too: tell them openly, ask for consent, and accept a refusal without disadvantage.

Works council: depends. In a practice or hospital with a works council, co-determination under §87(1) no. 6 BetrVG is likely, as recordings and usage logs can monitor staff behaviour (objective suitability suffices), and §90(1) no. 3 requires early information. Small practices without a works council need no formal step but should still involve staff (GUIDE-004).

AI literacy: support doctors with short training on checking every generated note, typical errors such as invented or missed details and automation bias (GUIDE-006). Choose two or three volunteers from different specialties as pilot group for four to eight weeks (GUIDE-009). Concerns are privacy, liability and fear of being recorded: the doctor stays responsible and signs every note, recordings can be paused, and no audio is used to evaluate staff (GUIDE-008).

Risks, regulation & compliance

• GDPR: health data is special-category data (Art. 9); the voice recording of the patient and any third parties is personal data. Lawful basis typically Art. 9(2)(h) treatment plus explicit information; some regulators and professional bodies expect patient consent for recording. DPIA is required; processor contract (Art. 28); avoid third-country transfers; minimise audio retention (delete after sign-off).
• Professional secrecy: German §203 StGB and equivalents bind clinicians and require care when engaging service providers.
• EU AI Act: classification depends on the intended purpose. Pure transcription and note drafting with clinician review is often seen as low risk, but if the product qualifies as a medical device under MDR (class IIa or higher via Rule 11 for software that informs clinical decisions), it is high-risk under AI Act Art. 6(1)/Annex I with obligations phased in until 2028 [7][8]. Features that suggest diagnoses or treatments push the product towards medical-device status. Transparency toward patients (an AI system is listening) is good practice and often a legal expectation.
• MDR: the manufacturer decides the intended purpose; documentation-only tools may stay outside MDR, but scribes with clinical suggestions may not. Ask vendors for the declaration of conformity and notified-body certificate [8].
• Risks and mitigations: hallucinated content in the legal record (mandatory review, source links, audit sampling), consent gaps (visible recording indicator, opt-out path), data leakage to model training (contractual ban), bias across accents and languages (test per language).

Getting started

1. Ask your PVS/EHR vendor and the data-protection officer what is already approved; start the DPIA early.
2. Define consent wording and a visible recording indicator; decide what happens when a patient declines.
3. Pilot 2-3 products with a handful of clinicians for 4-8 weeks, measuring time saved and auditing 50 notes for omissions and errors.
4. Fix the review rule: no note enters the record unsigned; train staff on typical error types.
5. Decide on retention (delete audio after sign-off) and plan rollout by specialty.

Sources

1. Tandem Health, product and evidence pages — https://tandemhealth.ai/ai-info (vendor material, accessed 2026-10)
2. Tandem Health, news and press (CE/MDR claims, vendor-authored) — https://tandemhealth.ai/resources/news
3. OpenAI Whisper — https://github.com/openai/whisper
4. Mistral AI, Voxtral speech models — https://mistral.ai/news/voxtral
5. vLLM project — https://github.com/vllm-project/vllm
6. Best AI scribes in Europe 2026 (vendor-affiliated comparison, treat with caution) — https://www.drinfo.ai/clinical-workflows/best-ai-medical-scribes-europe
7. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
8. Regulation (EU) 2017/745 (MDR) — https://eur-lex.europa.eu/eli/reg/2017/745/oj
9. Regulation (EU) 2016/679 (GDPR), Art. 9 — https://eur-lex.europa.eu/eli/reg/2016/679/oj