Earth-observation compliance checks for public authorities (CAP area monitoring, undeclared construction)

Public authorities must check that what citizens and farms declare matches reality on the ground. Visiting every parcel is slow and expensive. Machine learning on free Sentinel satellite time series (and on aerial photos for buildings) screens whole territories, marks the parcels that clearly agree with the declaration and routes only doubtful ones to follow-up. The model proposes; a civil servant decides and the citizen can contest.

Problem & value

• Today: paying agencies check a small sample of CAP applications by on-the-spot inspection and desk review of aerial photos. Tax and building authorities find undeclared pools, extensions and sheds through tip-offs, manual photo interpretation or random inspections.
• AI value: continuous coverage of every parcel during the season instead of a sample, earlier feedback to applicants (fix the claim before payment), fewer field visits, and a prioritised list for the checks that remain.
• Legal driver (CAP): the CAP Integrated Administration and Control System (IACS) allows and, from 2024 on, requires an Area Monitoring System (AMS) that continuously observes parcels with Sentinel data; "checks by monitoring" are a recognised alternative to classical inspections [1][2].
• Sourced evidence: the European Commission's JRC documents checks by monitoring and the open Sen4CAP toolbox; the French tax administration (DGFiP) uses aerial-image AI in its "Foncier innovant" programme to detect undeclared pools and buildings [2][3][4]. Savings and detection rates differ per country and year; use the agency's own pilot figures, not generic ones.

Variants

• CAP area monitoring (core): crop-type verification, mowing and grassland maintenance, ploughing and bare-soil events, catch crops, fallow, eco-schemes and conditionality markers per parcel.
• Photo-assisted follow-up: inconclusive parcels trigger a geotagged-photo request through a farmer app, closing the case without a visit.
• Undeclared construction and pools: orthophotos compared with cadastre and building registers for property-tax and planning enforcement (FR "Foncier innovant", similar projects in other states).

How it works

1. declarations (GSAA/LPIS parcels, cadastre, tax filings) + Sentinel-1/2 time series (cloud-masked, parcel-level aggregation) | aerial orthophotos
2. feature extraction (NDVI/backscatter curves, markers; building footprints & pools)
3. classification / segmentation (crop type, mowing, ploughing; building & pool detection)
4. rule engine: compare result with declaration, tolerance, confidence
5. traffic light per parcel / candidate list + explanation (time-series plot, image chips)
6. human review: desk check, photo request, on-the-spot check
7. administrative decision with notification and appeal

• Key AI components: time-series classifiers (random forest/gradient boosting on markers, or temporal deep models such as transformers/TempCNN on Sentinel-2/Sentinel-1 stacks), semantic segmentation of buildings and pools on very-high-resolution imagery, and geospatial foundation models that reduce labelling effort.
• Inconclusive is a valid output: cloud cover, small or irregular parcels and mixed crops often give no safe answer; these go to photo or field follow-up, never to automatic penalties.
• Human-in-the-loop: every sanction, reduction of payment or tax reassessment is decided by an official who sees the evidence; models only prioritise and pre-sort.

Data preparation

• Data needed: farmers' geospatial aid applications and the land-parcel identification system (paying agency databases), cadastre and building registers (surveying offices), tax and permit records, past inspection results and farmer photos (case files, photo apps), plus Sentinel time series and orthophotos (Copernicus, state surveying agencies).
• Typical gaps: parcel geometries and registers with different coordinate systems and update cycles, outdated building footprints, inspection results kept as scanned PDFs or free-text notes, no unified crop-code scheme across years, farmer photos without reliable geotags, and strict access rights between agencies.
• Preparation steps: (1) inventory sources and owners (PREP-001, PREP-004); (2) export registers and claims via APIs or database dumps (PREP-005); (3) harmonise coordinate systems and crop codes, validate geometries in PostGIS/QGIS; (4) digitise old inspection files (PREP-006) and harvest results as labels (PREP-020); (5) build stratified test sets per region and year with inspectors (PREP-018); (6) clarify purpose limitation and legal basis for reusing data (PREP-025).
• Tools: GDAL, PostGIS, QGIS, openEO, Sen4CAP preprocessing, STAC catalogues.
• Share of effort: roughly 50-60 percent; data harmonisation, labelling and cloud masking dominate over model building, and must be repeated each season.

Implementation paths

Which path for whom

• Public authorities: C - sovereignty and procurement rules favour open Copernicus data and the open Sen4CAP/QGIS/PostGIS stack on sovereign EU or in-house infrastructure; buy a managed service (A) only if no GIS data-science team exists and the vendor is accredited.
• Enterprise (service providers, large landowners, insurers): B - EU-hosted openEO and cloud processing with fine-tuned EO foundation models gets to production fastest without building a GIS platform.
• Developer: C - Sen4CAP, TorchGeo and TerraTorch with open Sentinel data give full control and reproducibility.

Path A — Buy (off-the-shelf)

• EU / national platform vendors: Sinergise/Planet (Slovenia, EU) with EO Browser and Sentinel Hub, Planet and Airbus Defence and Space for aerial and VHR data, EOX (Austria), VITO Remote Sensing (Belgium), GAF (Germany), CS Group and Capgemini as integrators, and national service providers contracted by paying agencies. IGN (France) and national mapping agencies supply orthophotos.
• Choose A when the paying agency wants a managed service with vendor support and accreditation and has no in-house data-science team.

Path B — Build on cloud APIs

• EU data access: Copernicus Data Space Ecosystem (openEO, Sentinel Hub APIs, STAC catalogue), national data hubs; pre-processed products such as Copernicus HR-Layers.
• Processing: openEO or Sen4CAP workflows on EU cloud (IONOS, OVHcloud, T-Systems, STACKIT); gradient boosting and temporal deep nets for crop/marker models; Earth-observation foundation models for building and pool detection (e.g. Prithvi, Clay, TerraMind) fine-tuned on local labels.

Path C — Self-host (open source; recommended for public authorities)

• Stack: Sen4CAP (JRC open toolbox), Orfeo ToolBox, GDAL, rasterio/xarray, STAC, PyTorch with TorchGeo, scikit-learn/LightGBM, PostGIS, QGIS for analysts.
• Models: segmentation with SAM-family or U-Net/SegFormer variants for buildings; TerraTorch with Prithvi, Clay or TerraMind for fine-tuning.
• Hardware: HW3 (one 48 GB class GPU plus large storage) handles a country-scale season; HW2 is enough for regional pilots. Storage and I/O of imagery dominate cost more than compute.

Path D — Custom training

Usually necessary for production: crop and marker models must be trained on the agency's own reference data (past inspection results, farmer photos) per region and year. Needs thousands of labelled parcels and annual re-validation. Building and pool detectors need local orthophoto labels.

State of the art (2026)

• Sentinel-1/2 time series at 10 m resolution with cloud-robust SAR fusion give reliable crop-type and event markers for large, homogeneous parcels; accuracy drops for small parcels, orchards, heterogeneous grassland and mixed cropping.
• Markers instead of end-to-end verdicts: mowing, ploughing and cover are detected as events and compared with rules from the scheme, which is more explainable than a single crop label [2].
• Aerial AI for buildings and pools is operational in tax administrations; main errors come from shadows, tarps, solar covers, seasonal pools and registry timing.
• Limits: cloud cover, revisit gaps, parcel boundary errors, scheme rules changing yearly, and distribution shift between years and regions. Reported accuracy must be measured per scheme and region.

Shared resources & combinations

• Same stack as VIS-061 (satellite verification for EUDR) and VIS-058 (satellite and aerial change detection): shared Copernicus data access, time-series and segmentation components reusable across agencies and use cases.
• Downstream: case management and anomaly screening on declarations (ANOM-009); drones for follow-up (VIS-003, VIS-004).

Effort, cost & KPIs

• Path A: 6-12 months incl. procurement; annual service fees per country/region.
• Path B: 6-12 months for a pilot-to-production AMS on a few schemes.
• Path C/D: 12+ months, a team of GIS engineers and data scientists; labelling campaign each season.
• KPIs: share of parcels with a conclusive result, agreement with field inspection (precision and recall per marker), reduction of on-the-spot checks, time to farmer feedback, share of overturned decisions on appeal, false-flag rate (building detections confirmed on review).

People & change

• Involve early: management and the head of the paying agency or tax/planning office, the inspectors who run desk and field checks, GIS/IT, the data protection officer and legal counsel (administrative law, appeals), and the staff council. A citizen-facing process also needs a communication plan for farmers and building owners. See GUIDE-001, GUIDE-009 and GUIDE-014.
• Works council: depends. Authorities have a staff council (Personalrat, BPersVG or state law), not a works council. Parcel analysis does not monitor staff, but case-processing logs, inspector productivity statistics and changed work processes trigger co-determination and information duties (analogous to §87(1) no. 6 and §90 BetrVG); involve it early (GUIDE-004).
• Data protection: parcel and building data link to identifiable persons; a DPIA is likely (systematic evaluation, new technology at scale), legal basis Art. 6(1)(e) GDPR, AVV with every contractor (GUIDE-005).
• AI literacy: train inspectors on what a traffic light means, the "inconclusive" class and model limits (Art. 4 AI Act; GUIDE-006).
• Pilot and concerns: pilot in one region and one scheme with experienced inspectors. Address fears of false accusations and loss of discretion: the AI only prioritises, every sanction is decided by an official, farmers can contest, and accuracy is reported per region and farm size (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: classification depends on use. Pure screening that a human verifies is often minimal risk, but systems that evaluate eligibility for or reduce public benefits can fall under Annex III (point 5, access to essential public benefits and services), making them high-risk with duties on data governance, logging, transparency, human oversight and accuracy. Check Annex III case by case [5]. No Art. 5 prohibition applies, but do not extend the system to profiling individuals or social scoring.
• CAP rules: Regulation (EU) 2021/2116 and Implementing Regulation (EU) 2022/1173 define IACS, AMS and the duty to let farmers correct claims after warnings [1].
• GDPR: parcel and building data link to identifiable owners and farmers; legal basis is a task in the public interest (Art. 6(1)(e)); Art. 22 limits solely automated decisions, so keep a human review and a right to contest; DPIA recommended; EU hosting and minimisation of VHR imagery of homes.
• National law: tax-procedure and planning law set what imagery evidence may be used; some states needed specific legislation for tax image analysis.
• Risks and mitigations: false accusations (inconclusive class, human review, appeal), bias against small farms or parcels (stratified accuracy reporting), model drift (annual validation), vendor lock-in (open formats, Sen4CAP-compatible).

Getting started

1. Pick one scheme or marker (for example mowing on grassland) and agree legal consequences with the legal unit.
2. Collect reference data: past inspection results and geotagged photos, split by region and year.
3. Run Sen4CAP or an openEO baseline on Copernicus Data Space for one region and measure per-parcel agreement.
4. Design the traffic-light workflow, evidence view and farmer notification before tuning models.
5. For buildings, test an aerial segmentation model on a pilot municipality and review every hit manually.

Sources

1. Regulation (EU) 2021/2116 (CAP horizontal regulation, IACS) — https://eur-lex.europa.eu/eli/reg/2021/2116/oj
2. JRC — Checks by Monitoring and Sen4CAP — https://marswiki.jrc.ec.europa.eu/wikicap/index.php/Checks_by_Monitoring
3. Sen4CAP project — https://www.esa-sen4cap.org/
4. DGFiP — Foncier innovant (detection of undeclared buildings and pools) — https://www.economie.gouv.fr/dgfip/foncier-innovant
5. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
6. Copernicus Data Space Ecosystem — https://dataspace.copernicus.eu/
7. TorchGeo — https://github.com/microsoft/torchgeo
8. IBM/NASA Prithvi geospatial foundation model — https://huggingface.co/ibm-nasa-geospatial