Microscopes produce far more images than people can annotate or measure by hand. Modern segmentation and foundation models (Cellpose, StarDist, Segment Anything derivatives) now find cells, nuclei, organelles, particles or grains across imaging modalities with little or no retraining, and turn pixels into quantitative tables. Typical benefit: hours of manual outlining reduced to minutes, more reproducible measurements, and screening at scales that were not feasible by eye.
Problem & value
- Today: researchers threshold images in Fiji/ImageJ, hand-trace objects, or tune bespoke classical pipelines (CellProfiler) per experiment. Results vary between people and instruments; high-content screens and 3D/time-lapse data (gigabytes to terabytes per run) are a bottleneck.
- AI value: generalist segmentation works across stains and cell types out of the box, interactive tools make correction fast, and fine-tuning on a few dozen annotated images adapts a model to a new specimen. Deep-learning restoration and denoising also allow lower light dose and faster imaging.
- Beyond biology: the same methods count particles, measure grain size and porosity in SEM/optical micrographs of metals, ceramics, polymers and batteries, and detect defects in quality labs.
- Published evidence on speed-ups is task-specific; measure your own baseline (see KPIs) rather than relying on vendor claims.
Variants
- Cell biology / high-content screening: nuclei and cell segmentation, phenotypic profiling (Cell Painting), drug-response screens.
- Live-cell tracking: time-lapse segmentation plus tracking and lineage (TrackMate, Ultrack-style pipelines).
- 3D / volume EM / light-sheet: organelle or neuron segmentation on large volumes with chunked processing.
- Cryo-EM / cryo-ET: particle picking and tomogram segmentation (links to SCI-001 for structure work).
- Materials & industrial labs: microstructure, grain and defect analysis (SCI-003).
- Teaching / citizen science: classroom microscopes, biodiversity and plankton imaging, with browser-based tools.
- Clinical histopathology is a regulated medical-device domain: see HLTH-002.
How it works
- acquisition (microscope, OME-TIFF/Zarr, metadata)
- preprocessing (illumination correction, denoising/restoration, tiling of large images)
- model inference (instance segmentation | detection | classification | tracking)
- human review & correction (interactive label editing in napari/QuPath)
- measurement & feature extraction (morphology, intensity, counts, tracks)
- statistics, QC and reporting (tables, plots, provenance log)
- optional: corrections fed back to fine-tune the model
- Segmentation models: Cellpose (generalist, "Cellpose-SAM"/Cellpose 4 builds on a Segment-Anything encoder), StarDist (star-convex nuclei), nnU-Net-style self-configuring U-Nets, Mesmer/DeepCell for tissue, and SAM-based interactive tools such as micro-SAM (napari plugin).
- Vision-language / agents: VLMs and LLM agents help choose tools, write Fiji/napari/Python scripts and caption images; they are assistants for workflow, not the measuring instrument.
- Human-in-the-loop: spot-check masks per plate/batch, correct failures, and sign off the final dataset. Quantities that enter publications or regulatory files must come from validated, versioned pipelines.
Data preparation
Data needed: microscopy images (2D/3D/time-lapse) with acquisition metadata and, for fine-tuning, a few manual annotations. In practice they sit on instrument PCs, local disks, USB drives, NAS or file shares, in vendor formats (CZI, ND2, LIF), with metadata in Excel or notebooks.
Typical gaps: scattered storage, inconsistent file names, missing metadata (magnification, stain, condition), proprietary formats, duplicates, few annotations, and for tissue or patient-derived samples unclear access rights and consent.
Preparation steps: 1. Inventory where images live and who owns them (PREP-001, PREP-004). 2. Convert to OME-TIFF/OME-Zarr via Bio-Formats and centralise in OMERO (PREP-007, PREP-009). 3. Add metadata and a naming convention (PREP-010). 4. Create a small expert-annotated test set (PREP-018). 5. Pseudonymise sample IDs for human material (PREP-022); see DOC-014 for paper lab records.
Tools: Bio-Formats, OMERO, Fiji, napari.
Share of effort: about 20-30 percent.
Implementation paths
Which path for whom
- Enterprise: Path C, because image data and assays are sensitive R&D IP and Cellpose, StarDist and napari run well on a local HW2 GPU.
- Developer: Path C, because open models can be fine-tuned and scripted into custom pipelines.
- Public: Path C, because universities and institutes need open, reproducible tools and OMERO-based on-premises or sovereign hosting, with patient-derived data staying local.
Path A — Buy (off-the-shelf)
- Vendor platforms: ZEISS arivis and ZEISS ZEN Intellesis (Germany, EU), Leica Aivia / LAS X (Germany, EU), Oxford Instruments Imaris and Andor, Thermo Fisher Amira/Avizo, Nikon NIS.ai, Evident cellSens, Molecular Devices MetaXpress/IN Carta, Revvity Harmony/Signals Image Artist, Dassault Systemes BIOVIA.
- Cloud/AI-first: Aiforia, Ilastik-based commercial services, Biodock, Cytena (Germany, EU) and Sartorius (Germany, EU) instrument-bundled analysis.
- Choose A when: the lab buys a microscope or screening system anyway and needs validated, supported software with instrument integration and GxP features.
Path B — Build on cloud APIs
- Managed GPU compute in EU regions (AWS, Azure, Google Cloud, EU providers such as OVHcloud, STACKIT, Scaleway) running open models as containers; BioImage Model Zoo (bioimage.io) models and the BiaPy/DeepImageJ ecosystems give portable, versioned models.
- Frontier multimodal LLMs (Claude, Gemini, GPT, Mistral) via EU-region endpoints for figure description, protocol QA and generating analysis scripts; do not use them for primary quantification.
- Orchestration: Nextflow/Snakemake, OMERO for image management, OME-Zarr for cloud-friendly storage.
Path C — Self-host (open source; recommended default for research labs)
- Tools: Fiji/ImageJ, napari, QuPath, CellProfiler, ilastik, Cellpose / Cellpose-SAM, StarDist, micro-SAM, nnU-Net, DeepCell, TrackMate, BiaPy, OMERO (Open Microscopy Environment).
- Hardware: HW2 (one 16-32 GB GPU) handles most 2D and moderate 3D workloads; HW3 for large volumes or cohort-scale batch inference. Many tools also run slowly on CPU for pilots.
- Ops effort: low to moderate; main costs are storage, data management and training researchers.
Path D — Custom training
Worthwhile when the specimen, stain or modality is unusual (EM of a specific tissue, rare phenotypes) or throughput justifies it. Fine-tune Cellpose/StarDist/nnU-Net with typically tens to a few hundred annotated images or crops; human-in-the-loop correction loops in Cellpose or micro-SAM are the cheapest way to produce labels. Hold out whole experiments (not random tiles) for evaluation to avoid leakage.
State of the art (2026)
- Generalist models (Cellpose, and Cellpose-SAM which reports strong cross-dataset generalisation) and SAM-derived tools mean many projects need no training at all. Independent comparisons of foundation cell-segmentation models on live microscopy and spatial omics show results still vary by modality [1][2].
- Standardised formats and model sharing (OME-Zarr, BioImage Model Zoo) improve reproducibility and let models run in many front ends [5].
- Agentic assistants that drive Fiji/napari or write analysis code are emerging; treat their output as untrusted until checked.
- Limits and failure modes: dense or overlapping cells, out-of-focus or low-signal images, unusual morphologies, 3D anisotropy and domain shift between instruments; models can silently merge or split objects. Foundation models trained on public data may carry annotation bias. Always report segmentation quality (F1 at IoU thresholds) on your own data, and validate against manual ground truth.
Shared resources & combinations
- Vision stack (segmentation, object-detection, image-classification) is shared with HLTH-001/HLTH-002 and industrial inspection use cases; one HW2/HW3 GPU server can serve all.
- Combine with: SCI-007 (self-driving labs: closed-loop imaging and decisions), SCI-017 (phenotypes as input for target and biomarker discovery), SCI-001 (cryo-EM to structure), SCI-003 (microstructure to property models), SCI-011 (agents generating hypotheses from image-derived features), DOC-014 for literature/protocol retrieval.
Effort, cost & KPIs
- Path A: weeks, licence and service contracts dominate cost.
- Path B/C: 2-8 weeks for a working pipeline for one assay; ongoing curation of storage and models.
- Path D: 1-3 months incl. annotation.
- KPIs: segmentation F1/AP at fixed IoU against expert labels, fraction of images needing manual correction, analyst time per experiment, inter-operator variance, reproducibility across sites, throughput (images/hour), and storage cost per TB.
People & change
- Involve early: the lab or department lead, researchers and analysts as end users, IT for storage and GPU workstations, the DPO when human tissue or patient-derived data are used, data owners of the image archives, and management for storage budget.
- Works council: depends. Analysing images is not employee monitoring, but shared platforms such as OMERO log user activity, which may be suited to monitor behaviour (§87(1) no. 6 BetrVG; BAG: objective suitability). Inform under §90(1) no. 3 and see GUIDE-004. Universities: Personalrat.
- Data protection: DPIA (Art. 35 GDPR) likely where special-category data (health, genomic) are linked to images; AVV (Art. 28 GDPR) for any cloud use; ethics approval (GUIDE-005).
- AI literacy: under AI Act Art. 4 (since 27 Jul 2026) employer and provider must take measures to support staff AI literacy; plan a short training (see GUIDE-006).
- Pilot and communication: start with one assay and a few interested researchers; show time saved against hand outlining and publish validation results internally (GUIDE-009, GUIDE-008).
- Typical concerns: trust in automatic segmentation and research-integrity rules; address with expert-annotated test sets and mandatory spot checks.
- Human control: researchers review masks, QC flags and report model versions; no undisclosed AI enhancement in publications.
Risks, regulation & compliance
- EU AI Act: minimal risk for research and industrial image analysis (AI literacy, Art. 4). If used for diagnosis or as part of a medical device or IVD (clinical pathology), it becomes high-risk and falls under MDR/IVDR (see HLTH-002). No Art. 5 prohibition is relevant.
- GDPR: research images are usually non-personal, but human tissue, patient-derived cells, biopsy slides and linked metadata or genomic data can be personal or special-category data (Art. 9); requires legal basis, ethics approval, pseudonymisation, and DPAs for cloud use. Prefer EU hosting or on-premises.
- Research integrity: image-manipulation rules for publication (no undisclosed AI enhancement, report model versions and parameters); AI-restored images must not be used as quantitative evidence without validation.
- GxP / data integrity: regulated pharma needs validated software (ALCOA+, audit trail, 21 CFR Part 11 and EU Annex 11).
- Risks: silent failures on new data, overfitting to one instrument, licence terms (some models or datasets are non-commercial), supply-chain security of Python packages. Mitigate with versioned containers, held-out evaluation and periodic QC images.
Getting started
- Pick one assay and manually annotate 20-50 representative images including failure cases; store them in OME-TIFF/Zarr with metadata.
- Run Cellpose-SAM or StarDist in napari or Fiji without training and score F1 against your annotations.
- If scores are short, correct masks interactively and fine-tune (Path D lite) with held-out experiments for testing.
- Wrap the pipeline in a container with Nextflow/Snakemake, log versions, and add automatic QC flags.
- Define sign-off rules: who reviews which fraction of images, and when the model must be re-validated.
Sources
- Pachitariu et al. — Cellpose-SAM: superhuman generalization for cellular segmentation (bioRxiv) — https://www.biorxiv.org/content/10.1101/2025.04.28.651001
- Cellpose documentation — https://cellpose.readthedocs.io/
- Stringer et al. — Cellpose: a generalist algorithm for cellular segmentation (Nature Methods) — https://pubmed.ncbi.nlm.nih.gov/33318659/
- Israel et al. — A Foundation Model for Cell Segmentation (CellSAM) — https://arxiv.org/abs/2311.11004
- BioImage Model Zoo — https://bioimage.io/
- napari — https://napari.org/
- StarDist — https://github.com/stardist/stardist
- Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj