Ultrasound is cheap, radiation-free and portable, but image quality depends heavily on the operator, and trained sonographers are scarce. AI closes the loop during scanning: it tells a nurse, midwife, GP or paramedic how to move the probe, recognises when a diagnostic view is reached, records it and computes measurements such as ejection fraction, lung B-lines or fetal biometry. The result widens who can scan, shortens time to a first answer and standardises quality; the clinician remains responsible for the diagnosis.
Problem & value
- Today: point-of-care ultrasound (POCUS) is used in emergency, intensive care, anaesthesia, cardiology, obstetrics and primary care, but acquiring echo or lung views takes months of training and the findings vary between operators. Where no sonographer is available, patients are referred, delayed or scanned with poor-quality images. Reading is done by the same bedside clinician or later by a cardiologist/radiologist.
- AI value: (1) acquisition guidance using real-time view recognition and probe-movement prompts, (2) quality gating and auto-capture, (3) automated measurement (e.g. LVEF, strain, B-line counts, biometry, bladder volume) and (4) draft interpretation for triage.
- Evidence: a prospective study of nurses without ultrasound experience using FDA-cleared AI guidance (Caption) found most obtained diagnostic-quality echo for left-ventricular size/function, right-ventricle size and pericardial effusion after a short training [1]. Automated echo measurement platforms report agreement with expert readers in validation studies and shorten reporting time [2]. Treat vendor-reported figures as indicative and validate locally.
- Why distinct: unlike expert reading of stored CT/MRI/X-ray (HLTH-001), the core is closed-loop feedback on operator-dependent, real-time video.
Variants
- Cardiac (echo): guided parasternal/apical views, auto-LVEF, strain, valve screening; heart failure clinics, emergency departments, primary care.
- Lung: B-line counting for pulmonary congestion, pleural effusion, pneumothorax screening.
- Obstetrics: guided fetal biometry, presentation and placenta position; midwife-led and low-resource settings (blind-sweep protocols with AI).
- Abdominal / vascular / MSK: bladder volume, aorta, DVT compression, nerve-block guidance, gallbladder and FAST exam support.
- Training and veterinary: simulators with automated feedback; see HLTH-016 for animals. a medical-device matter.
How it works
- probe
- live video (+ optional IMU/ECG)
- view/plane classification (which standard view is on screen? quality score)
- guidance policy (next movement: slide, rotate, tilt, press; target-view prediction)
- auto-capture of best clip (ECG-gated for echo)
- segmentation / landmark detection / measurement (EF, B-lines, biometry)
- plausibility checks + uncertainty (reject low-quality clips)
- draft report + DICOM SR
- clinician review/sign-off
- PACS/EHR
- Guidance: deep networks trained on large libraries of expert scans paired with probe motion recommend the next move; some systems use learned image-quality regressors (e.g. echo view quality scores) rather than explicit pose data.
- Measurement: segmentation of heart chambers or lung artefacts, time-series modelling across the cardiac cycle, regression for EF/strain; foundation-model style encoders are increasingly used.
- Edge: tens-of-milliseconds latency means on-device inference (CoreML, ONNX Runtime, TensorRT).
- Human-in-the-loop: the operator can always override; the clinician reviews clips and measurements and signs the report; low-confidence results are flagged "not diagnostic".
Data preparation
The core function runs on the device and needs no preparation. Data matters for image storage, quality assurance and training. Ultrasound clips typically live on the device, in a vendor cloud, on USB sticks or in the PACS, and are often not linked to the patient record or the final report.
Typical gaps: clips stored on handheld devices without backup, patient identifiers typed freehand, inconsistent exam types, no record of who scanned, and unclear consent for use in teaching. Measurements by AI and by the clinician are rarely stored side by side.
Preparation steps: (1) decide where images are archived (PACS or vendor cloud with EU region) and who may access them; (2) standardise exam labels and a worklist link to avoid wrongly assigned patients; (3) enable structured storage of AI measurements together with the clinician's confirmed value; (4) define a quality-assurance sample, for example 5 percent of scans reviewed by an expert, see PREP-018 and PREP-021; (5) set access rights for devices, see PREP-023; (6) pseudonymise teaching material, see PREP-022.
Rough share of effort: about 15 to 20 percent of the project, mainly device and PACS integration and the QA process. See GUIDE-003.
Implementation paths
Which path for whom
- Enterprise (clinic groups, emergency services, outpatient practices, device distributors): Path A, because probe guidance and automated measurements ship with CE-marked handheld or cart devices and need no integration work.
- Developer: Path C, because open ultrasound datasets and models allow prototypes and research but cannot be used for diagnosis without own MDR responsibility.
- Public (municipal hospitals, rescue services, universities): Path A with EU-hosted image storage, because certified devices simplify procurement and training programmes.
Path A — Buy (recommended)
Regulated products are the realistic route because the clinical claim is a medical-device claim. - Guidance and echo: Caption AI / Caption Guidance (GE HealthCare, built into Vscan Air and other devices), Us2.ai (Singapore; CE/FDA, automated echo measurement and reporting), Philips Lumify and ultrasound AI packages, Butterfly Network (Butterfly iQ3 / Compass AI, Copilot guidance), Exo Iris, EchoNous Kosmos (AI guidance for cardiac and lung), Siemens Healthineers and Samsung Medison/Canon platforms with built-in AI, Sonio (France, EU; fetal ultrasound quality control and reporting; part of Samsung Medison), Ultromics EchoGo (UK, heart-failure and amyloid detection), DiA Imaging Analysis (Israel; LVEF, RV, strain). - EU-specific checks: CE marking under MDR (not just FDA clearance), intended-use statement (who may operate: physician only or also nurse/midwife?), DICOM/PACS and HL7 FHIR integration, EU hosting of cloud features, cybersecurity documentation. - Choose A when: you want a legally supported clinical tool; this is almost always the case for hospitals, practices and EMS.
Path B — Build on cloud APIs
- General multimodal models (Claude, Gemini, GPT) are not suitable for real-time probe guidance and are not cleared for diagnostic ultrasound reading. Useful niches: drafting report text from measurements, structuring findings, patient letters (see HLTH-011), training-feedback explanations, and retrieval over protocols.
- Cloud fits storage (Azure Health Data Services DICOM, Google Cloud Healthcare API, AWS HealthImaging; EU regions) and QA analytics.
Path C — Self-host (open source)
Research and training, not clinical use without MDR compliance. - Datasets and models: EchoNet-Dynamic and EchoNet-Pediatric (Stanford, echo video with EF labels), CAMUS (cardiac segmentation), MIMIC-IV-ECHO, FetalPlanes and other ultrasound plane datasets, MedSAM / MedSAM2 and SAM-based medical segmentation, USFM-style ultrasound foundation models, MONAI for training and deployment, NVIDIA Clara/Holoscan for real-time pipelines. - Runtime: ONNX Runtime, TensorRT, CoreML or OpenVINO on a Jetson Orin / laptop (HW1); training on one workstation GPU (HW2). - Patient use as a diagnostic aid makes you an MDR manufacturer.
Path D — Custom training
Only for device manufacturers and research hospitals. Needs thousands to hundreds of thousands of expert-labelled cine loops across scanners, body habitus, pathologies and operators, plus paired probe-motion data for guidance. Expect clinical validation studies, external test sets and post-market surveillance; domain shift between vendors/probes is the main technical risk.
State of the art (2026)
- Guidance has moved from research to cleared products: FDA first cleared AI cardiac acquisition guidance (Caption, 2020), and guidance now exists in several handheld and cart systems for cardiac, lung and obstetric exams [1][3].
- Automated echo measurement (EF, strain, valve metrics) is commercial and being embedded in reporting workflows; ultrasound foundation models and vision-language models for echo interpretation (e.g. EchoPrime-style video-text models) are research-grade and show promise for multi-view report generation but are not yet routine devices.
- Limits and failure modes: performance drops on obese patients, poor acoustic windows, rare pathology, paediatric cases and non-training scanners; automated EF can be wrong with arrhythmia or foreshortened views; B-line counts are sensitive to gain/depth; guidance can create false confidence in novices who cannot recognise a non-diagnostic scan; automation bias in readers; most validations are single-centre or vendor-sponsored.
Shared resources & combinations
- Same stack: HLTH-001, HLTH-014, HLTH-022 (real-time video AI), HLTH-023 (AI at scan time).
- Signals/downstream: HLTH-013 ECG, HLTH-007 remote monitoring, HLTH-004 decision support, HLTH-011 reports, HLTH-018 chart summaries; shared DICOM/PACS, de-identification and medical-device MLOps.
Effort, cost & KPIs
- Path A: weeks to months for procurement, IT/PACS integration, DPIA and training; handheld devices plus per-user or per-study subscriptions are the cost drivers.
- Path B (support tasks only): 1-3 months, small API cost.
- Path C/D: 12+ months and a regulatory team for any clinical use.
- KPIs: share of exams with diagnostic-quality views, time to first diagnostic image, agreement with expert (bias, limits of agreement for EF), repeat/referral rate, time to decision, rate of incidental findings missed, clinician override rate, training time to competence.
People & change
Involve management for the device and licence budget, and the department lead (emergency medicine, cardiology, obstetrics) as owner. Clinicians, nurses and paramedics who are not ultrasound experts are the target users; bring in an experienced sonographer as trainer. IT covers device management and PACS; the data protection officer (DPO) approves storage; HR helps with training and competence records.
Works council: depends. Devices that log operator identity, scan counts and the quality score of each scan can serve to monitor individual performance, which triggers co-determination under §87(1) no. 6 BetrVG (objective suitability, BAG). §90(1) no. 3 requires timely information about AI-supported processes. In public clinics and rescue services the Personalrat applies. Clarify that quality scores are for learning, not for evaluation.
Data protection: patient images and measurements are health data; check legal basis and run a DPIA when images are stored centrally or in a vendor cloud, with an AVV (Art. 28 GDPR).
AI literacy: support staff literacy under Art. 4 AI Act with a structured credentialing programme (supervised scans, then independent use). Warn against over-trusting guidance: the clinician interprets, and a negative AI result never replaces clinical judgement. Start with one ward or ambulance as pilot group and collect feedback after 3 months.
Guides: GUIDE-004, GUIDE-006, GUIDE-008.
Risks, regulation & compliance
- EU AI Act: high risk. AI systems that are safety components of products, or products themselves, covered by MDR/IVDR and requiring third-party conformity assessment are high-risk (Art. 6(1), Annex I). Most diagnostic ultrasound software is MDR class IIa or higher, so AI Act requirements (risk management, data governance, logging, human oversight, accuracy and robustness) apply to providers, integrated into MDR assessment; deployer duties (use per instructions, oversight by trained staff, monitoring) apply to hospitals [4]. Note the high-risk application date for Annex I products: 2 Aug 2028 under the Digital Omnibus on AI (Reg. (EU) 2026/1744). Art. 50 transparency is not central here. No Art. 5 prohibition applies, provided the tool is not used for emotion inference or prohibited biometric purposes.
- MDR 2017/745: software is class IIa-IIb under Rule 11; the intended-use statement determines who may operate it. Clinical evaluation, post-market surveillance and cybersecurity apply [5].
- GDPR: images and measurements are health data (Art. 9); need legal basis (treatment), processor agreements, DPIA, EU hosting, minimisation of identifiers in DICOM headers and retention aligned with national medical-records law. Fetal images are linked to the pregnant patient's record.
- Clinical risks: missed pathology from false reassurance, over-reliance by novices, training bias (skin tone, body habitus, scanner brand). Mitigations: local validation, escalation rules, audit of a sample of scans, quality-assurance review, clear "not diagnostic" messaging.
Getting started
- Pick one narrow clinical question (e.g. LVEF screening in the ED, or B-lines in heart failure).
- Shortlist 2-3 MDR-certified products; check intended use for non-expert operators and scanner compatibility.
- Run a prospective pilot with expert over-read on 100-200 exams; measure quality, agreement and time.
- Complete DPIA, PACS/EHR integration and operator training/credentialing.
- Set up monitoring: drift, override rate and periodic expert audit.
Sources
- Narang A et al. Utility of a deep-learning algorithm to guide novices to acquire echocardiograms, JAMA Cardiology 2021 — https://jamanetwork.com/journals/jamacardiology/fullarticle/2775252
- Us2.ai — automated echocardiography measurement platform — https://www.us2.ai
- U.S. FDA — Artificial Intelligence-Enabled Medical Devices list — https://www.fda.gov/medical-devices/artificial-intelligence-enabled-medical-devices/list-artificial-intelligence-enabled-medical-devices
- Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- Regulation (EU) 2017/745 (MDR) — https://eur-lex.europa.eu/eli/reg/2017/745/oj
- Butterfly Network — AI-guided handheld ultrasound — https://www.butterflynetwork.com
- MONAI — https://monai.io
- EchoNet-Dynamic dataset (Stanford AIMI) — https://echonet.github.io/dynamic/
- Sonio — fetal ultrasound AI — https://www.sonio.ai