Plant disease & crop health detection

Crop losses from fungal, bacterial and viral diseases, pests and nutrient stress are often noticed late, and fields are scouted on foot for only a fraction of their area. Vision models on phones, machines, drones and satellites recognise symptoms early, localise them in the field and drive targeted, documented treatment, which can reduce unnecessary plant-protection product use.

Problem & value

• Today: farmers, growers and advisers scout by eye, rely on calendar spraying or disease-warning models and send samples to labs. Symptoms are confused easily (fungal vs. nutrient vs. herbicide damage), and scouting capacity is the bottleneck on large or fragmented holdings.
• AI value: earlier detection, per-zone instead of whole-field decisions, consistent documentation, advice available on a phone for smallholders. The EU Farm to Fork strategy and the proposed Sustainable Use Regulation target reduced pesticide use, which makes targeted application a policy driver [1][2].
• Evidence caveat: published laboratory accuracy (often >95% on curated sets such as PlantVillage) does not transfer to field conditions; field accuracy is markedly lower and depends on crop, stage and lighting [3]. Treat vendor savings claims as unverified until tested on your own fields.

Variants

• Smartphone diagnosis for farmers, gardeners and advisers (leaf photo → candidate diseases).
• Greenhouse / vertical farm fixed cameras with controlled lighting (highest accuracy).
• Tractor / sprayer-mounted camera detection of weeds and disease patches for on-the-go spot spraying.
• Drone scouting with RGB or multispectral cameras; vineyards, orchards, potatoes, cereals.
• Satellite monitoring (Sentinel-2) for field-level stress indices at coarse resolution.
• Seed / breeding phenotyping in research plots (see SCI-008).
• Home gardeners / private: plant-identification and diagnosis apps.

How it works

1. image capture (phone / machine / drone / satellite, with GPS + timestamp)
2. pre-processing (tiling, radiometric correction, orthomosaic for drones)
3. model inference (classification | detection | segmentation | VLM)
4. georeferencing + aggregation to zones, growth-stage and weather context
5. decision logic (thresholds, disease-risk models, economic damage threshold)
6. recommendation / spray map / alert
7. agronomist review
8. application and record keeping
9. feedback labels

• Classification: CNN / ViT models (EfficientNet, ConvNeXt, DINOv2 backbones) assign a disease class to a leaf photo.
• Detection / segmentation: YOLO-class detectors and SAM-type segmenters locate lesions, weeds and fruit on cluttered field images; needed for spot spraying.
• VLMs give explanatory answers and handle rare classes zero- or few-shot, but are less reliable than trained specialists on fine-grained symptoms.
• Multispectral indices (NDVI, NDRE) show stress before it is visible, but do not tell the cause.
• Human-in-the-loop: low-confidence cases go to an adviser; any plant-protection product decision stays with the certified applicant; lab confirmation for notifiable quarantine pests.

Data preparation

Data: labelled leaf, fruit and stem photos, drone and satellite imagery (Sentinel-2), field boundaries, crop type, growth stage and weather data. In a typical farm these live on phones, drone SD cards, a farm-management system (FMS), spreadsheets and paper spray logs.

Typical gaps: photos without labels or location, different cameras and lighting, field boundaries that do not match the plot register, handwritten plant-protection records, too few images of rare diseases, and drifting seasons (a model trained in a dry year fails in a wet one).

Preparation steps: 1. Inventory sources and agree on one field and plot register (PREP-001, PREP-005). 2. Export FMS and weather data via API or CSV and tidy it into one table (PREP-011). 3. Define a small class scheme (disease, pest, deficiency, healthy) before labelling (PREP-019). 4. Collect and label images with an agronomist, using tools such as Label Studio or CVAT, and keep a separate test set (PREP-018). 5. Digitise paper spray records at source (PREP-008).

Effort: with Path A, roughly 10-20% of the project (field boundaries, onboarding); with B/C or D, 40-60%, dominated by labelling. See GUIDE-003 for a readiness check.

Implementation paths

Which path for whom

• SMB (farms, growers): A, because mature crop-scouting apps and field-monitoring platforms with EU hosting exist at farm-level prices and the process is standard.
• Enterprise (large agri groups, food retailers' supply bases): B, because scouting should be integrated into own farm-management data and many fields make seat-based pricing expensive.
• Developer: C, because open datasets and open-source vision models run on HW1 and give full control over classes and edge deployment.
• Private (gardeners): A, because free or cheap plant-identification apps cover the need without any setup.

Path A: Buy (recommended for most farms)

• Farm management and scouting platforms: xarvio (BASF Digital Farming, Germany/EU), Climate FieldView, 365FarmNet (EU), Farmonaut, Taranis, CropX, Agremo; drone service providers with analytics.
• Smartphone apps: Plantix (PEAT, Germany/EU), Pl@ntNet (INRIA/CIRAD, France/EU, identification), FAO-linked PlantVillage Nuru (offline cassava, maize, potato diagnosis).
• Machine vision: Bosch/BASF smart spraying, John Deere See & Spray, Bonirob-style robots, Carbon Robotics and Ecorobotix (Switzerland) for weeds.
• Choose A when: common crops, existing machinery brands, no ML team.

Path B: Build on cloud APIs

• Sentinel-2 / Copernicus data via the Copernicus Data Space Ecosystem and Sentinel Hub (Sinergise, EU) plus openEO for time series [4].
• Cloud vision: Azure AI Custom Vision / Document-style custom models, Google Vertex AI (EU regions), AWS SageMaker (Frankfurt); frontier multimodal APIs (Claude, Gemini, GPT, Mistral) for explanatory diagnosis with retrieval over extension-service texts.
• Orchestration: n8n / custom service with PostGIS or GeoServer, ISOBUS/ISOXML export for machine terminals.
• Choose B when: you have a specific crop and own images, and need integration into a farm system.

Path C: Self-host (open source)

• Models: Ultralytics YOLO (AGPL, check licence), RT-DETR, Segment Anything, DINOv2, Prithvi-EO and other Earth-observation foundation models (IBM/NASA), PlantNet-300K-trained models; open VLMs (Qwen-VL) for explanation.
• Stack: PyTorch, ONNX Runtime / TensorRT, CVAT or Label Studio for annotation, QGIS / rasterio / TorchGeo.
• Edge: Jetson Orin Nano/NX on sprayers or drones (HW1); training on one 16-24 GB GPU (HW2).

Path D: Custom training

Often worth it: public datasets are curated lab-style images and generalise poorly. Collect field images across seasons, varieties, cameras and lighting; typically hundreds to thousands of expert-labelled images per class. Use active learning, domain adaptation and held-out test fields (not random splits of the same field) [3].

State of the art (2026)

• Foundation models (DINOv2/DINOv3, SAM, Earth-observation FMs) make few-shot adaptation realistic, cutting labelling needs.
• Real-time weed/crop detection for spot spraying is commercial; disease detection on moving machines is less mature.
• Hyperspectral and multispectral sensing detects stress before visible symptoms in research and early products.
• VLM assistants answer farmer questions with photos, but hallucination and over-confident diagnoses are a known risk.
• Limits and failure modes: domain shift (new variety, lighting, camera, region), background clutter, overlapping or co-occurring stresses, early asymptomatic infection, class imbalance, small training fields, and symptoms that look alike (nutrient vs. disease). Accuracy figures from PlantVillage-style data overstate field performance [3].

Shared resources & combinations

• Same stack (object-detection + segmentation + remote-sensing): VIS-021 satellite/aerial analysis, VIS-003 drone inspection, VIS-026 counting and measurement, VIS-032 species identification, VIS-006 produce grading.
• Downstream: OPT-014 precision irrigation and fertilisation, PRED-013 yield prediction, ROBO-007 weeding and harvesting robots.
• Shared hardware: Jetson-class edge devices (HW1) also serve VIS-001, VIS-014; GPU servers for training (HW2).

Effort, cost & KPIs

• Path A: days to weeks to onboard; subscription per hectare or per app licence.
• Path B: 2-4 months for a pilot on one crop with field data collection.
• Path C/D: 4-9 months; at least one full season of data is needed to validate. Drone and sensor hardware, labelling by agronomists and seasonality (data only available once a year) drive cost.
• KPIs: per-class precision/recall in field tests, early-detection lead time vs. scouting, scouted area per hour, plant-protection product use per hectare, yield loss avoided, false-alarm rate, adviser override rate.

People & change

Involve early: the owner or management (budget, risk appetite), the farm or crop manager as process owner, the field staff who take the photos, IT or the machinery dealer for integration, and the advisers or agronomists who validate results. The data owners of the FMS and drone data should agree on export and sharing rules.

Works council: usually not relevant, as small farms rarely have one and the system observes plants, not people. If a works council exists and tractor cameras or phone apps log employee positions or work times, §87(1) no. 6 BetrVG applies because objective suitability for monitoring is enough; then involve it under §90(1) no. 3 as well (see GUIDE-004).

Data protection: mostly non-personal data. Drone flights over neighbouring land, people in images and GPS tracks of workers can be personal data; involve the DPO, sign an AVV (Art. 28 GDPR) with the platform and check whether a DPIA is needed (GUIDE-005).

AI literacy: support staff with a short hands-on training on what the model can and cannot see, confidence values and false alarms (GUIDE-006).

Communication and pilot: start with one crop and a few fields run by willing field staff, compare AI findings against manual scouting for one season (GUIDE-009).

Typical concerns: "the app replaces my experience" (it advises, the grower decides on every treatment) and "wrong spraying advice" (agronomist sign-off, plant-protection law remains the farmer's responsibility). Keep humans in control: no automatic sprayer actuation without confirmation.

Risks, regulation & compliance

• EU AI Act: typically minimal risk (Art. 4 AI literacy applies). Not Annex III. If integrated as a safety component of an autonomous machine, machinery rules (Regulation (EU) 2023/1230) apply [5]. Worker monitoring cameras would be a separate question.
• GDPR: mostly non-personal data, but farm data and field boundaries can be linked to identifiable farmers; drones may capture people or neighbours. Apply the EU Code of Conduct on agricultural data sharing and the EU Data Act (connected-machine data access rights) [6].
• Plant protection law: Directive 2009/128/EC (sustainable use) and Regulation (EC) 1107/2009; aerial spraying is generally banned with narrow derogations. AI recommendations do not replace the licensed applicant's responsibility. Quarantine pests (Regulation (EU) 2016/2031) require official reporting and lab confirmation.
• Drones: EU drone rules (Regulation (EU) 2019/947) on operator registration and categories.
• Risks and mitigations: wrong diagnosis leading to over- or under-treatment, so keep an adviser in the loop, show confidence, log images and recommendations, and validate per region and variety.

Getting started

1. Define the crop, the 3-5 diseases or stresses that matter economically and the decision they trigger (spray, scout, sample).
2. Try a Path A app or platform on a few fields for one season, comparing it with agronomist scouting.
3. Collect and label your own images (varied lighting, stages, devices); store GPS and growth stage.
4. Test on held-out fields; report per-class precision/recall rather than overall accuracy.
5. Integrate with farm records and spray-map export; define the adviser review step.

Sources

1. European Commission, Farm to Fork Strategy: https://food.ec.europa.eu/horizontal-topics/farm-fork-strategy_en
2. European Commission, Sustainable use of pesticides: https://food.ec.europa.eu/plants/pesticides/sustainable-use-pesticides_en
3. Mohanty, Hughes, Salathe, Using Deep Learning for Image-Based Plant Disease Detection (Frontiers in Plant Science, 2016): https://www.frontiersin.org/articles/10.3389/fpls.2016.01419/full
4. Copernicus Data Space Ecosystem: https://dataspace.copernicus.eu/
5. Regulation (EU) 2023/1230 (Machinery Regulation): https://eur-lex.europa.eu/eli/reg/2023/1230/oj
6. EU Code of Conduct on agricultural data sharing by contractual agreement: https://copa-cogeca.eu/publications
7. Regulation (EU) 2024/1689 (AI Act): https://eur-lex.europa.eu/eli/reg/2024/1689/oj
8. Pl@ntNet: https://plantnet.org/en/