Dermatology & ophthalmology screening

Skin cancer and diabetic eye disease are both common, treatable when found early, and bottlenecked by specialist capacity. Image-classification models can triage dermoscopic images and grade retinal photographs, so that low-risk cases are cleared quickly and suspicious ones reach a specialist sooner. In the EU such software is a regulated medical device and a high-risk AI system, so the technology is the easy part and clinical validation, certification and governance are the real work.

Problem & value

• Today: primary-care doctors refer suspicious lesions to dermatologists, often with long waits; diabetic retinopathy (DR) screening relies on trained human graders reading large volumes of fundus images, and glaucoma/AMD are found late. Specialists spend much of their time on normal findings.
• AI value: prioritise referrals, reduce unnecessary referrals of benign lesions, increase screening coverage in primary care or pharmacies, shrink grader workload, and give a second read.
• Evidence: the first autonomous AI diagnostic, IDx-DR (now LumineticsCore), was authorised by the FDA in 2018 on the basis of a prospective pivotal trial [1]. In dermatology, the UK NHS has evaluated AI-assisted teledermatology pathways (for example Skin Analytics DERM) in real clinical settings [2]. Published performance varies strongly with population, camera and device; treat vendor figures as claims until verified in your own setting.

Variants

• Dermatology: dermoscopy-based lesion classification (melanoma, BCC, SCC vs benign), total-body photography follow-up, smartphone triage for primary care, teledermatology prioritisation.
• Ophthalmology: DR screening from fundus photos (diabetes programmes, optometrists, primary care), glaucoma and AMD detection from fundus/OCT, retinopathy of prematurity, OCT-based progression tracking.
• Setting: hospital departments, national screening programmes, optician chains, telemedicine, pharmacy-based screening.
• Consumer apps: skin-check apps for private users are a distinct, contested segment (many are regulated devices or make unsupported claims); wound photos are covered in HLTH-027.

How it works

image acquisition (dermoscope / fundus camera / OCT, quality check)
  -> image-quality gate (focus, illumination, field coverage; reject and retake)
  -> classification / segmentation model (lesion class, DR grade, optic-disc and macula analysis)
  -> calibrated risk score + uncertainty / out-of-distribution flag
  -> clinician review (always for positive or uncertain; sampling for negatives)
  -> report + referral routing (PACS/EHR, screening-programme software)
  -> post-market monitoring (drift, subgroup performance, outcome follow-up)

• Models: convolutional or vision-transformer classifiers trained on labelled image sets, often with a pretrained retinal or dermatological foundation model as backbone. Segmentation (lesion borders, retinal lesions such as haemorrhages and exudates) supports explainability.
• Modes of use: assistive (clinician sees the suggestion), triage (AI orders the worklist), or autonomous (AI issues a negative/positive result without a physician). Autonomous use is rare in the EU and requires the highest evidence and an explicit intended-purpose claim.
• Human-in-the-loop: any suspicious or low-confidence result goes to a dermatologist or ophthalmologist; the clinician owns the diagnosis. Image-quality and uncertainty gates are essential.

Data preparation

Images (dermoscopy, photos, fundus images, OCT) come from imaging devices, the PACS or a vendor cloud, with the clinical context in the EHR. For buying a certified product the preparation is mostly operational; for development or validation it is heavy.

Typical gaps: inconsistent image quality (lighting, focus, scale markers), missing or wrong metadata, images without confirmed diagnosis (histology or specialist reading), skewed skin-type and age distributions, duplicates per patient, and consent for secondary use.

Preparation steps: 1. Check the image pipeline: standard capture protocol, DICOM or defined export, quality checks at capture (PREP-005, PREP-008). 2. Define the finding categories before any labelling (PREP-019). 3. Have specialists label a local evaluation set, ideally with histology as reference (PREP-018). 4. Harvest existing confirmed diagnoses as labels where legally allowed (PREP-020, PREP-025). 5. Pseudonymise or anonymise images and metadata (PREP-022). 6. Check subgroup coverage (skin types, devices) before trusting accuracy figures.

Effort share: low for a certified product (capture standardisation and PACS link), high for any validation or training work, where labelling by specialists dominates. See also GUIDE-003.

Implementation paths

Which path for whom

• enterprise (clinic groups, screening providers): Path A, because diagnostic use requires a CE-marked medical device with clinical evidence; a self-built system would make you the manufacturer.
• public (university and municipal hospitals, screening programmes): Path A with certified products, because regulatory responsibility and procurement rules favour approved devices; a sovereign hosting option should be demanded in the tender.
• developer: Path C, because research and prototype work on open-weight vision models with local data keeps patient images in-house; clinical use needs certification.

Path A — Buy (CE-marked medical device)

• Retina / DR: LumineticsCore (Digital Diagnostics, US; autonomous DR), Eyenuk EyeArt (US), RetinAI (Switzerland, EU-adjacent; OCT/fundus analytics platform), Optomed (Finland, handheld cameras with AI), Thirona RetCAD (Netherlands), Retmarker (Portugal), Altris / Topcon and Zeiss platforms with integrated AI. Verify the current CE certificate and intended purpose per product.
• Dermatology: Skin Analytics DERM (UK; autonomous and assistive pathways), MoleMap and FotoFinder (Germany) with AI-supported total-body photography, Heuron/Dermicus-style and other CE-marked dermoscopy aids. Check class (typically IIa/IIb under MDR) and the exact claim.
• Choose A when: you are a provider or screening programme without an ML team. Only a CE-marked product with a matching intended purpose may be used clinically; ask for clinical evaluation report summaries, performance by skin type / camera, and a data processing agreement with EU hosting.

Path B — Build on cloud APIs

• Generic multimodal LLM APIs (Claude, Gemini, GPT, Mistral) are not certified for diagnosis and must not be used for clinical decisions. Legitimate uses: report drafting from validated findings, patient-facing explanations (PERS-014), and research prototyping on de-identified data.
• Medical-imaging platforms (Google Health AI developer foundations, Microsoft Azure health services, AWS HealthImaging) offer foundation models and hosting for building your own regulated product in EU regions; the resulting product still needs its own conformity assessment.

Path C — Self-host (open source)

• Datasets and tooling: ISIC Archive and HAM10000 for dermoscopy, EyePACS / APTOS / Messidor-2 and OCT sets for retina (research only; check licences and bias), MONAI for medical-imaging pipelines, timm for backbones, PyTorch.
• Foundation models: open retinal and dermatology foundation models (for example RETFound for retinal images) can be fine-tuned on local data for research.
• Hardware: HW2 is enough for training small classifiers and for inference at screening-site scale; HW1 edge devices run quantised models on handheld cameras.
• Status: research tooling, not a medical device; clinical use requires full MDR conformity.

Path D — Custom training

Worth it for device manufacturers and hospitals with large labelled archives, or for adapting a model to a local camera and population. Needs thousands to hundreds of thousands of expert-graded images with multi-reader consensus labels, representation across skin phototypes and ethnicities, and external validation sites. The organisation becomes the legal manufacturer (MDR, QMS per ISO 13485, IEC 62304 software lifecycle, ISO 14971 risk management).

State of the art (2026)

• Retina: DR grading models reach sensitivity and specificity comparable to human graders in prospective studies; foundation models pretrained on large unlabelled retinal image sets improve data efficiency and generalise to systemic-disease prediction research (e.g. cardiovascular risk from fundus photos, still investigational).
• Dermatology: classifiers match or exceed dermatologists on curated test sets, but real-world performance drops with different cameras, lesion locations and darker skin tones; training sets are skewed towards light skin. Multimodal and LLM-based assistants are emerging but are not validated diagnostics.
• Limits and failure modes: distribution shift between devices and populations, poor image quality, rare lesion types, automation bias (clinicians over-trusting a negative), dataset shortcuts (rulers, surgical marks, skin markers) and unequal performance across skin types. Explainability heatmaps are not proof of correct reasoning.

Shared resources & combinations

• Same stack (image-classification + segmentation, HW2): HLTH-001 radiology, HLTH-002 digital pathology, HLTH-027 wound photos, HLTH-014 dental imaging and HLTH-013 ECG share MONAI-based pipelines, DICOM/PACS integration, annotation tooling and MLOps for monitoring.
• Combinations: HLTH-004 clinical decision support (referral decisions), HLTH-018 chart summarisation, HLTH-011 report generation, PERS-014 patient-friendly explanations.
• Hardware: one GPU server per site serves several imaging specialities; edge devices for point-of-care cameras.

Effort, cost & KPIs

• Path A: weeks to a few months for procurement, IT and clinical-safety sign-off; per-scan or subscription pricing. Cost drivers: camera hardware, integration with EHR/PACS, training.
• Path B/C prototype: months for a research pipeline; not a deployable device.
• Path D: 1–3+ years including clinical studies and MDR certification; expensive.
• KPIs: sensitivity and specificity versus reference standard (per subgroup), ungradable-image rate, referral appropriateness, time to specialist, screening coverage, grader workload, missed cases found at follow-up, and clinician override rate.

People & change

Involve management and the medical lead (responsibility for clinical use), dermatologists or ophthalmologists and screening staff as users, IT for PACS and device integration, and the data protection officer and legal/compliance early: health images are special-category data, a DPIA is likely, the vendor needs an AVV, and the product must be CE-marked as a medical device. Patients are affected by the output and should be informed that AI supports the reading.

Works council: depends. If the system only supports diagnosis and does not log individual physicians' performance, §87(1) no. 6 BetrVG may not apply; usage logs and per-physician agreement statistics make it applicable. Inform the council under §90(1) no. 3 in good time. Public hospitals involve the staff council.

AI literacy (Art. 4 AI Act, supporting measures): train clinicians on sensitivity and specificity, known failure modes (skin types, image quality) and automation bias. Start with a pilot group of willing specialists in one clinic, run the tool in shadow mode first, and compare with their own readings. Typical concerns are liability and deskilling; clarify that the physician decides and that every finding is documented as a physician's decision.

See GUIDE-005, GUIDE-008 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: high-risk. AI that is a medical device (or its safety component) under MDR/IVDR and needs third-party conformity assessment falls under Annex I and Art. 6(1), which applies on the later transition timeline for product-embedded systems; check the current timeline, as EU simplification proposals may shift dates. Obligations include risk management, data governance, logging, transparency, human oversight, accuracy and robustness [3]. AI literacy (Art. 4) applies.
• MDR: diagnostic software is generally class IIa or higher under Rule 11; this requires a notified body, clinical evaluation and post-market surveillance [4]. Do not use unlicensed tools for clinical decisions.
• GDPR: images are health data (Art. 9); facial or periocular images can be identifying; need a legal basis, DPIA, EU hosting, strict access control and retention rules. Secondary use for training needs a separate basis; the European Health Data Space regulation adds rules for secondary use [5].
• Fairness: document performance across skin phototypes, age and devices; bias is a patient-safety issue.
• Mitigations: intended-purpose discipline, local validation before go-live, image-quality gates, clinician sign-off, continuous performance monitoring and incident reporting.

Getting started

1. Pick one pathway (for example DR screening in diabetology or dermoscopy triage in primary care) and define the clinical question, reference standard and acceptable sensitivity.
2. Shortlist CE-marked products whose intended purpose matches your use; request certificates and evidence by subgroup.
3. Run a silent-mode pilot on local images (AI output hidden) and compare with specialist reads.
4. Complete a DPIA, clinical-safety review and workflow design (who acts on positive, negative and ungradable results).
5. Go live in assistive mode, monitor KPIs monthly, and review subgroup performance quarterly.

Sources

1. U.S. FDA — De novo authorization of IDx-DR (autonomous diabetic retinopathy detection), 2018 — https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/denovo.cfm
2. NHS England / NICE — early value assessment of AI for skin cancer detection in teledermatology — https://www.nice.org.uk/guidance/hte15
3. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
4. Regulation (EU) 2017/745 (MDR) — https://eur-lex.europa.eu/eli/reg/2017/745/oj
5. Regulation (EU) 2025/327 (European Health Data Space) — https://eur-lex.europa.eu/eli/reg/2025/327/oj
6. MONAI — Medical Open Network for AI — https://github.com/Project-MONAI/MONAI
7. ISIC Archive — https://www.isic-archive.com/
8. RETFound retinal foundation model — https://github.com/rmaphoh/RETFound_MAE