Sewer & pipe CCTV inspection with automatic damage coding (EN 13508-2)

Municipal and industrial network operators must inspect their drains and sewers on a regular cycle, and each inspection produces hours of crawler video that a trained coder reviews and codes by hand. Computer vision pre-detects defects, proposes the standardised EN 13508-2 code with position, and pre-fills the report. The coder confirms or corrects instead of coding from scratch, which shortens turnaround and makes condition data more consistent across contractors.

Problem & value

• Today: a crawler operator drives through a section, watches the live feed and dictates or enters observations; the coding is then checked or redone by a certified inspector. Quality varies between operators and contractors, backlog builds up and reports arrive months after the drive.
• Obligation (DACH): German states require operators to inspect and document their networks (self-monitoring, e.g. the Hessian "Eigenkontrollverordnung", NRW "SüwVO Abw" and equivalent state rules). Coding follows the European standard EN 13508-2 and, in Germany, the DWA-M 149 series (DWA-M 149-2 for condition classification and evaluation) [1][2].
• AI value: automatic pre-detection and pre-coding of defects, search and review of the video by defect type, consistent classification, faster reports and cleaner input to rehabilitation planning (PRED-033). Peer-reviewed work on automated sewer defect classification reports strong results on benchmark data such as the Sewer-ML dataset, but real-world transfer remains the open issue [3].

Variants

• Municipal self-monitoring: city or association network inspected on a 10–15-year cycle.
• Inspection contractors: service companies using AI to raise throughput and consistency per coder.
• Industrial and private drains: plants, property drainage, house connections with push cameras.
• Acceptance inspection: new-build or relined pipes after rehabilitation.
• Live assist: on-device detection during the drive so the operator is prompted at each finding.

How it works

1. crawler / push camera video + chainage + pipe master data
2. pre-processing (de-interlace, de-blur, lighting normalisation, frame sampling)
3. frame-level detection & segmentation (crack, root, deposit, infiltration, joint offset, ...)
4. tracking along the pipe
5. merge frames into one defect event with chainage & clock position
6. quantification (extent in %, crack width class, deformation %) from geometry or laser/sonar
7. code mapping to EN 13508-2 (main code, characterisations, quantification)
8. section evaluation (condition class per DWA-M 149-2 / national scheme)
9. human review (coder confirms, corrects, adds missing observations)
10. report + exchange file
11. GIS / asset-management / rehabilitation planning

• Detection: CNN or transformer classifiers for multi-label frame classification (the Sewer-ML style task), plus object detectors (YOLO family, RT-DETR) and segmentation for extents.
• Position and size: chainage from the cable counter; pipe-wall geometry or calibrated lasers give crack width and deformation; without calibration, sizes are estimates only.
• Rule layer: deterministic mapping from detected class and attributes to the code catalogue and the condition class; this part is rules, not a neural net, and must be auditable.
• VLM role: vision-language models help with free-text remarks, report drafts and natural-language search ("show all root intrusions at joints in DN300 clay pipes"), not as sole coder.
• Human-in-the-loop: every automatic code is a proposal; certified coders release the report and the operator remains responsible for the condition assessment.

Data preparation

Needed are inspection videos, the operator's coded observations with chainage and clock positions, and pipe master data (ID, material, diameter, year laid). In practice videos sit on crawler hard disks, USB sticks and file shares, observation data in the inspection software, and master data in a GIS or sewer register. Older inspections exist only as paper protocols or DVDs.

Typical gaps: inconsistent or missing chainage, videos without a link to the pipe ID, mixed codes (older DWA-M 149-2 versus EN 13508-2), duplicate or truncated recordings, and highly varying image quality (fog, water, lighting). Defect codes are often applied inconsistently between operators, so labels are noisy.

Preparation steps: (1) link every video to a pipe ID via the register, see PREP-005; (2) digitise paper protocols and convert legacy video formats, see PREP-006 and PREP-007; (3) agree on a category scheme and code usage, see PREP-019; (4) harvest labels from existing coded reports, see PREP-020; (5) build a test set re-coded by two experienced inspectors, see PREP-018. Tools: ffmpeg for conversion, CVAT or Label Studio for labelling, QGIS for master data. Expect about half of the project effort in data work, mostly relabelling and quality control. Use GUIDE-003 for the readiness check.

Implementation paths

Which path for whom

• Public: Path A, because municipal network operators need certified inspection software that exports the standard coding exchange formats and works with existing inspection contractors.
• Enterprise: Path A, because mature sewer-inspection suites exist and the process is standardised by EN 13508-2.
• SMB: Path A, because inspection service firms get a ready product with damage detection at affordable per-project prices, whereas building own models needs far more labelled video than a small firm has.
• Developer: Path C, because open detection models fine-tuned on own labelled frames run on modest GPUs (HW2) and can be embedded into the tools of several customers.

Path A — Buy (recommended)

• Inspection and coding software with AI assistance (EU/DACH marked): Pallon (CH, Zurich; ETH spin-off, formerly Hades; AI defect detection and EN 13508-2 coding of uploaded inspection video), IBAK (DE) and IPEK (DE) crawler hardware with their own software, RedZone Robotics (US). Check each vendor's current AI module and code support.
• Choose A when: you are an operator or contractor who needs EN 13508-2 export, ISYBAU/XML exchange, GIS link and vendor support. Verify supported code version, exchange formats, EU hosting and whether the AI runs in a cloud or on a workstation.

Path B — Build on cloud APIs

• Upload sampled frames to a multimodal model for triage and report-drafting; keep the code mapping in your own rules engine. Use EU-region offerings (Azure AI Foundry, Google Vertex AI, Mistral).
• Managed vision training (Google Vertex AI AutoML, AWS Rekognition Custom Labels) can host a classifier trained on your own footage.
• Choose B when: you already hold labelled video and want a cheap pilot; do not stream raw third-party footage from public roads without a data-protection check.

Path C — Self-host (open source)

• Models: YOLO-family detectors, RT-DETR, SAM 2 for segmentation assistance, DINOv2 or similar backbones for classification, OpenCLIP for search; fine-tune on the Sewer-ML dataset plus own data.
• Stack: PyTorch, ONNX Runtime or TensorRT, CVAT or Label Studio for annotation, FFmpeg, a video store and PostGIS for the network.
• Hardware: HW2 (one 24 GB GPU) is enough for offline analysis of several hours of video per day; HW1 (Jetson Orin class) for on-robot live assist.
• Effort is mostly data preparation and the code-mapping logic, not the model.

Path D — Custom training

Usually required in some form, because defect appearance varies by material (concrete, clay, PVC, liner), camera and lighting. You need hundreds to thousands of coder-labelled sections per defect type; historic inspections with verified codes are the natural source. Class imbalance is severe (rare defects such as collapse), so combine augmentation, sampling and active learning on reviewer corrections. Keep a held-out set per contractor and per material to measure generalisation.

State of the art (2026)

• Frame-level multi-label classification on Sewer-ML-type data is well studied; transformer and CNN baselines reach useful but not perfect recall, and benchmark scores overstate field accuracy [3].
• Production tools combine detection with chainage and clock position and export standardised codes, but quantification (crack width, extent) and subtle observations (water infiltration, joint condition, root density) remain the weak points.
• Open-vocabulary detectors and VLMs make quick pilots and video search practical, but are not yet reliable for certified coding.
• Typical failure modes: reflections and water level hiding the invert, fog or steam, lens dirt, camera tilt giving wrong clock positions, shadows read as cracks, rare defects missed, and domain shift between camera models.

Shared resources & combinations

• Same vision stack: VIS-003 (drone inspection of infrastructure), VIS-004 (road damage), VIS-026 (measurement and counting); annotation and detector training pipelines are reusable.
• Downstream planning: PRED-033 (buried network failure prediction and renewal planning) consumes condition classes; ANOM-013 (leak detection) and OPT-024 (wastewater optimisation) use the same network master data.

Effort, cost & KPIs

• Path A: weeks to onboard; licence per user or per metre; cost driver is coder time saved.
• Path B/C: 3–9 months to a validated pilot including labelling; cost drivers are annotation, GPU for training and the review tool.
• KPIs: recall of structural defects (priority over precision), code agreement with certified coder, chainage error, coder minutes per 100 m, report turnaround time, share of sections needing re-inspection, and agreement of condition classes with later field findings.

People & change

Involve the department lead (network operation) and the inspectors who code defects from the start; they own the quality criteria and will review every AI suggestion. IT integrates the tool with the inspection software and GIS; data owners release archives of old inspections; management decides on budget and on the use of results for rehabilitation planning. Inspection contractors and suppliers must agree on exchange formats and on who may use their recordings for training.

Works council: depends. Where the operator has an elected works council, it is likely involved, because the software records which inspector coded or corrected what, and objective suitability to monitor performance is enough under §87(1) no. 6 BetrVG (BAG). Inform it in good time under §90(1) no. 3; in municipalities the staff council (Personalrat) applies. Personal data are limited (occasional persons or vehicle plates in shaft videos), so involve the DPO, check whether a DPIA is needed and conclude an AVV (Art. 28 GDPR) with processors.

AI literacy: support inspectors with training on the limits of detection and on correcting suggestions (Art. 4 AI Act, GUIDE-006). Start with one pilot crew and a few hundred metres of pipe, see GUIDE-009. Typical concern: the tool replaces the inspector or judges their work. State clearly that the inspector confirms every code and that statistics are not used for individual assessment. Communication tips in GUIDE-008; works council steps in GUIDE-004.

Risks, regulation & compliance

• EU AI Act: typically minimal risk. A tool that only analyses pipe video is not listed in Annex III. Wastewater and drainage networks are not generally the "safety component of critical digital infrastructure or water supply" in Annex III point 2, but a system that is a safety component of the management and operation of critical infrastructure should be checked case by case, especially for drinking-water networks [4]. AI literacy (Art. 4) applies.
• GDPR: video is mostly non-personal, but footage from private property, house connections or open manholes can capture people, number plates or in-house details; apply data minimisation, retention limits and contractual terms for contractors [5].
• Technical standards: EN 13508-2 (visual inspection coding), EN 13508-1 (general requirements), DWA-M 149 series in Germany; national or state rules define inspection cycles and who may code [1][2].
• Liability and procurement: the operator stays responsible for the assessment; contracts should state accepted code version, QA sampling and required recall.
• Mitigations: human release of every report, sampling audits, versioned models, retained original video, and transparency on tool limits to the contracting authority.

Getting started

1. Collect 50–100 already coded sections per material type and check code consistency.
2. Define target codes and the exchange format your GIS/asset system needs.
3. Run 2–3 vendor trials from Path A on the same blind set of sections.
4. Measure recall by defect class and coder minutes saved, not just overall accuracy.
5. Set up a correction loop so reviewer changes flow into retraining.

Sources

1. CEN — EN 13508-2 Investigation and assessment of drain and sewer systems outside buildings, visual inspection coding system — https://standards.cencenelec.eu/
2. DWA — Merkblatt DWA-M 149 series (condition recording and assessment of drain and sewer systems) — https://de.dwa.de/
3. Haurum & Moeslund — Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark (CVPR 2021) — https://vap.aau.dk/sewer-ml/
4. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
5. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
6. Ultralytics YOLO documentation — https://docs.ultralytics.com/