Radiology image analysis (X-ray, CT, MRI)

Imaging volumes grow faster than the number of radiologists. AI tools read DICOM studies in the background, flag urgent findings (bleed, pneumothorax, pulmonary embolism), mark suspicious lesions, take measurements and increasingly draft the report. Used as triage, second reader or pre-reader, they shorten turnaround for critical cases and, in screening, can cut reading workload. Every product that influences diagnosis is a regulated medical device; the radiologist stays responsible.

Problem & value

• Today: radiologists read studies in a PACS worklist, mostly first-in-first-out, compare with priors by hand, measure lesions manually and dictate reports. Shortages of radiologists, growing CT/MRI volumes and night/weekend coverage lead to backlogs and delayed reads of urgent findings; double reading in screening (e.g. mammography in many EU programmes) binds scarce capacity.
• AI value: (1) triage - urgent cases move up the worklist; (2) detection aid / second reader - fewer missed findings; (3) quantification - reproducible measurements and volumes; (4) workload reduction - e.g. in screening, AI-supported single reading; (5) report drafting with generative models.
• Evidence: the Swedish MASAI randomised trial of AI-supported mammography screening reported a screen-reading workload reduction of about 44% with non-inferior cancer detection in its interim safety analysis [1]. Evidence for other modalities is mostly retrospective or from single-centre prospective studies; benefits depend on local prevalence, case mix and workflow, so validate locally.

Variants

• Chest X-ray triage and detection (pneumothorax, effusion, nodules, TB screening).
• CT: intracranial haemorrhage, large-vessel occlusion, pulmonary embolism, lung nodules and lung-cancer screening, coronary calcium, fracture detection, incidental findings (opportunistic screening).
• MRI: brain volumetry (MS, dementia), prostate lesion detection (PI-RADS support), musculoskeletal.
• Mammography / breast screening (population programmes; double-reading replacement) - the best-evidenced variant.
• Musculoskeletal X-ray fracture detection in emergency departments and urgent care.
• Teleradiology / outsourced reading with AI pre-triage across sites.
• Report-generation variant: vision-language models draft findings from images (still largely pre-regulatory or investigational in the EU).
• Adjacent imaging: pathology HLTH-002, skin/eye HLTH-006, endoscopy HLTH-022, ultrasound HLTH-024, dental HLTH-014, radiotherapy contouring HLTH-021, reconstruction HLTH-023.

How it works

modality (CT/MR/XR) -> PACS/VNA -> DICOM routing rule (modality, body part, indication)
  -> AI orchestrator / platform (de-identify if cloud) -> inference per algorithm
  -> results: DICOM SR / GSPS / secondary capture, priority flag, measurements
  -> PACS viewer + RIS worklist re-prioritisation
  -> radiologist reads, accepts / rejects findings -> signed report
  -> feedback + monitoring (drift, discordance, turnaround time)

• Models: CNN/ViT classifiers (study-level flags), detectors (nodules, fractures), segmentation networks (nnU-Net-style for organs, tumours, volumetry), and increasingly radiology foundation models and vision-language models for retrieval, pre-reporting and report-image consistency checks.
• Integration: DICOM / DICOMweb, HL7 v2 / FHIR, IHE profiles (AI Results, AIW-I); an AI orchestrator routes studies to several algorithms and returns results to one viewer.
• Human-in-the-loop: AI output is advisory. The radiologist views images, confirms or rejects every finding and signs the report; critical-finding results follow the local escalation process. In screening, recall decisions stay with radiologists.

Data preparation

Radiology AI needs DICOM studies from PACS/VNA, prior studies, the clinical indication from RIS/KIS and the signed reports. In a hospital these live in the PACS, the RIS and sometimes on local modality archives. Typical gaps: inconsistent DICOM tags (series descriptions, body-part fields), missing or free-text indications, different scanner protocols, duplicated or mis-assigned studies, reports that differ in structure between radiologists, and patient identifiers inside image headers or burned-in annotations.

Preparation steps: (1) inventory modalities, PACS routing and DICOM tag quality with the PACS vendor; (2) define which studies are in scope and how priors are matched; (3) for buying, run a local validation on a representative sample of your own scanners, because performance can drop on unfamiliar protocols; (4) for training or research, de-identify headers and pixel data (PREP-022) and build a reference set with radiologists (PREP-018, PREP-019); (5) harvest labels from signed reports with care, as report text is a noisy label (PREP-020); (6) capture radiologist corrections as feedback (PREP-021). Use the DICOM interfaces (DICOMweb, HL7) instead of exports where possible (PREP-005).

Effort: high. Roughly half of a build or validation project goes into data access, de-identification, labelling and local validation; a certified product shifts most of it to integration and acceptance testing. See also GUIDE-003 and DATA-003.

Implementation paths

Which path for whom

• Enterprise (hospital, radiology group): Path A, because MDR-certified products with PACS integration exist and self-certifying a medical device is rarely worth it.
• Developer: Path C, because open-source frameworks such as MONAI and open datasets allow research and prototypes without sending patient images to a third party.
• Public (university hospital, municipal clinic): Path A with certified products, because procurement rules and the medical-device regime favour certified, auditable suppliers.

Path A - Buy (recommended)

• CE-marked (MDR) algorithm vendors: Aidoc (aiOS platform, CT/X-ray triage), Annalise.ai (chest X-ray/CT brain), Lunit (INSIGHT CXR/MMG), Gleamer (BoneView, fracture; EU, France), Qure.ai, Vara (EU, Germany, breast screening), ScreenPoint Transpara (EU, Netherlands, mammography), Therapixel (EU, France), Oxipit ChestLink (EU, Lithuania, autonomous normal chest X-ray reporting), Incepto (EU, France), icometrix (EU, Belgium, brain MRI), Brainomix (stroke), Mediaire (EU, Germany, neuro volumetry), Quantib/RadNet (prostate/brain), deepc (EU, Germany, AI marketplace/orchestrator), Blackford (Bayer).
• Platform/viewer incumbents: Siemens Healthineers AI-Rad Companion, GE HealthCare Edison, Philips HealthSuite Imaging, Sectra and Agfa PACS with AI app stores.
• Choose A when: a clear clinical need exists, the vendor has MDR certification for your intended use and patient population, and published validation. Run a local silent-mode pilot before going live.

Path B - Build on cloud APIs

• Orchestration and storage: Azure Health Data Services (DICOM service), Google Cloud Healthcare API (DICOMweb), AWS HealthImaging - all with EU regions. Combine with an EU-hosted de-identification step.
• Models for research/prototypes: Google MedGemma (open-weight multimodal, incl. chest X-ray and report tasks) and MONAI-based models served from cloud GPUs. Frontier general-purpose multimodal LLMs are not certified medical devices and must not be used for clinical interpretation outside studies.
• Choose B when: you build an internal research or quality-assurance tool, not a diagnostic product. Putting your own algorithm into clinical use makes you a manufacturer (or in-house device operator under MDR Art. 5(5)) with all duties that follow.

Path C - Self-host (open source)

• Frameworks: MONAI (PyTorch, Project MONAI Label/Deploy), nnU-Net, TotalSegmentator (CT/MR organ segmentation), Orthanc or dcm4chee as DICOM server, OHIF viewer.
• Open-weight models: MedGemma, Microsoft RAD-DINO and BiomedParse-class models, CheXagent-type chest X-ray VLMs, MONAI model zoo; check licence terms (non-commercial restrictions are common).
• Hardware: HW2 for CT/X-ray inference at moderate volume; HW3 for large 3D models, MRI and VLMs. Strictly on-premises inside the hospital network.
• Reality check: open models are research-grade; clinical use requires validation plus MDR conformity, or use limited to research under ethics approval.

Path D - Custom training

Worth it for local populations, new indications or scanner-specific shifts. Needs thousands of expert-annotated studies (more for rare findings), multi-site data, ethics approval, a legal basis (research exemption, consent) and often federated learning or secure processing environments. Pre-training on public sets (CheXpert, MIMIC-CXR, NIH ChestX-ray14, LIDC-IDRI, TCIA collections) then fine-tuning on local data is standard. Expect a medical-device quality system (ISO 13485, IEC 62304, ISO 14971) if the product goes to market.

State of the art (2026)

• Triage and detection for stroke/haemorrhage CT, chest X-ray, fractures and mammography are mature, widely CE-marked (and FDA-cleared), with prospective data in screening.
• Screening: AI-supported single reading and AI triage of normal/high-risk cases are moving from trials into national programmes; evidence from MASAI [1] and other European trials drives adoption.
• Foundation and vision-language models (radiology-pretrained encoders, MedGemma-class models) give strong results on benchmark tasks and enable report drafting, but suffer from hallucinated findings, weak localisation, and sensitivity to prompts; benchmark scores do not transfer cleanly to clinical practice.
• Autonomous reporting of normal chest X-rays is CE-marked in at least one product (Oxipit ChestLink), but remains an exception with strict conditions.
• Limits and failure modes: dataset shift between scanners, protocols and populations; automation bias (radiologists follow wrong AI hints); false positives that increase workload; subgroup bias; shortcut learning (e.g. tubes, scanner markers); performance drift after software or protocol updates.

Shared resources & combinations

• Same stack (image-classification, segmentation, DICOM infrastructure): HLTH-002 pathology, HLTH-006 skin/eye, HLTH-013 ECG, HLTH-021 contouring, HLTH-022 endoscopy, HLTH-023 reconstruction, HLTH-024 ultrasound - one DICOM/orchestration layer and one GPU pool (HW3) can serve all imaging use cases.
• Report side: HLTH-011 report generation, SPEECH-005 dictation, HLTH-018 chart summarisation, DOC-007 extraction.
• Privacy tooling: DOC-018 and VIS-030 for anonymisation of exports and burned-in annotations.
• Decision support: HLTH-004 for downstream clinical decision pathways.

Effort, cost & KPIs

• Path A: weeks for contracting and integration per algorithm, 2-4 months with silent-mode validation; pricing is typically per study, per modality or per site licence; orchestrators reduce integration work.
• Path B/C: 6-12+ months for a validated research tool; GPU, MLOps and clinical annotation dominate costs.
• Path D: 12+ months and a regulatory budget if aiming for market entry.
• KPIs: sensitivity/specificity and AUC on local data; turnaround time for critical findings; report time per study; recall and cancer-detection rates (screening); acceptance/override rate of AI findings; discordance with final diagnosis; drift metrics; worklist time-to-open for flagged cases.

People & change

Involve management and the radiology head from the start, as the case changes a core clinical process and needs a budget and a named clinical owner. IT and IT security join early for PACS/RIS integration, network zones and updates; the DPO assesses the legal basis for patient data, the processor contract (AVV) and a DPIA, which is likely because health data is processed at scale (GUIDE-005). Add data owners of the PACS archive and, where reuse for training is planned, the ethics committee.

Works council: depends. A company or hospital with a works council must still inform it in good time (§90(1) no. 3 BetrVG), and co-determination under §87(1) no. 6 BetrVG is likely if the system logs which radiologist accepted or overruled findings, since objective suitability for monitoring is enough. Public hospitals involve the staff council (Personalrat). See GUIDE-004.

AI literacy: support radiologists and technologists with training on strengths, false-positive and false-negative patterns, and automation bias (GUIDE-006, Art. 4 AI Act). Start with a pilot group of two to four willing radiologists on one modality, run in shadow mode, then agree go/no-go criteria (GUIDE-009). Typical concerns are deskilling, liability and speed pressure: address them by keeping the radiologist as the decision-maker, making AI output overridable, and not using AI metrics for individual performance rating (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: AI that is a medical device (or its safety component) requiring notified-body conformity assessment under MDR (Class IIa and above for most diagnostic software, Rule 11) is high-risk under Art. 6(1) / Annex I. High-risk obligations for such products apply from 2 August 2028 (deferred from August 2027 by the Digital Omnibus on AI, Reg. (EU) 2026/1744; verify in the Official Journal). Deployers must ensure human oversight, trained staff, logging and monitoring (Art. 26) and AI literacy (Art. 4). No Art. 5 prohibition applies to imaging analysis as such; avoid inferring emotions or sensitive characteristics from images.
• MDR (EU) 2017/745: software for diagnosis is classified by Rule 11 (generally IIa-III); needs CE marking, clinical evaluation, post-market surveillance. Hospital in-house development is only allowed within Art. 5(5) conditions (no equivalent CE product on the market, justified need). General-purpose chatbots are not CE-marked for diagnosis.
• GDPR: imaging data is special-category health data (Art. 9). Needs a legal basis for treatment (Art. 9(2)(h)), DPA with processors, DPIA, EU/EEA processing, de-identification for secondary use (DICOM headers, burned-in text, facial reconstruction from head CT/MRI). Research use needs ethics approval and often consent. Germany: state hospital laws and the Patientendaten-Schutz-Gesetze; professional secrecy (§203 StGB) for cloud providers.
• European Health Data Space (EHDS, Reg. (EU) 2025/327): governs secondary use of health data over the coming years; plan for permit-based access to imaging data.
• Professional rules: radiation-protection law and requirements for qualified radiologists remain; AI does not replace the medical report responsibility.
• Mitigations: local validation and silent-mode pilot, subgroup performance monitoring, drift dashboards, user training against automation bias, documented intended use, fallback to manual reading.

Getting started

1. Pick one high-value, well-evidenced indication (e.g. head CT haemorrhage triage or mammography second read).
2. Check that the candidate product's CE certificate covers your intended use, modality and population; ask for peer-reviewed validation and the vendor's post-market surveillance plan.
3. Set up DICOM routing and an orchestrator or vendor gateway in a test environment; clear the DPIA and DPA.
4. Run a 2-3 month silent-mode pilot against final radiology reports; define go-live thresholds.
5. Go live with training, an escalation process for AI-flagged critical findings, and monthly monitoring.

Sources

1. Lang K. et al. - AI-supported mammography screening, MASAI randomised trial (Lancet Oncology, 2023) - https://www.thelancet.com/journals/lanonc/article/PIIS1470-2045(23)00298-X/fulltext
2. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
3. Regulation (EU) 2017/745 (MDR) - https://eur-lex.europa.eu/eli/reg/2017/745/oj
4. European Commission - MDCG 2019-11 guidance on qualification and classification of software - https://health.ec.europa.eu/medical-devices-sector/new-regulations/guidance-mdcg-endorsed-documents-and-other-guidance_en
5. Project MONAI - https://monai.io/
6. Google Health AI Developer Foundations - MedGemma - https://developers.google.com/health-ai-developer-foundations/medgemma
7. TotalSegmentator - https://github.com/wasserth/TotalSegmentator
8. nnU-Net - https://github.com/MIC-DKFZ/nnUNet
9. Regulation (EU) 2025/327 (European Health Data Space) - https://eur-lex.europa.eu/eli/reg/2025/327/oj
10. European Society of Radiology - AI in radiology resources - https://www.myesr.org/ai-in-radiology/