Buried network failure prediction & renewal planning (water mains, gas pipes, cables)

Most of a utility's assets are buried, carry no sensors and are replaced on the basis of age rules of thumb or the last visible break. Survival models and gradient-boosted trees learn from the asset register, repair history and GIS context which segments are most likely to fail next, combine this with the consequence of failure and turn it into a ranked, budget-aware renewal programme. The value is better targeting of scarce capex, not real-time alarms.

Problem & value

• Today: renewal is planned from age and material rules (e.g. "replace all cast iron before year X"), from the last damage events, or from engineer judgement. Renewal rates are constrained by budget, road-works coordination and fee approvals, so which pipe to replace first is the decisive question.
• AI value: replace one-size-fits-all age rules with segment-level failure forecasts. Typical use: choose the renewal list for the next 1-5 years, decide where to inspect or place loggers first, and justify capex to regulators, owners and city councils with transparent risk scores.
• Evidence (honest scope): published studies show ML beating simple heuristics. A gradient-boosted model forecast water main breaks three years ahead for Syracuse (NY) and beat baselines, and a Random Survival Forest was used for long-term prediction in Australian cities [5][6]. Vendor material gives an illustrative (not audited) example: with budget for 20 km, selecting pipes by predicted break rate avoids roughly 1,200 breaks over 10 years versus about 680 when selecting by recorded breaks [3]. Treat such numbers as vendor simulations and validate with a back-test on your own network.
• Why it matters in the EU: the recast Drinking Water Directive (EU) 2020/2184 requires risk-based management of supply systems and leakage assessment, and the DVGW names preserving infrastructure as a central challenge for the sector [7][12].

Variants

• Water mains and service lines: the best-documented case (break prediction, lead service line inventory).
• Gas distribution: cast iron / steel / PE pipes; the focus is consequence and compliance, with strict safety regulation (DVGW rules); models support prioritisation, never replace statutory inspections.
• Medium/low-voltage cables and joints: failure histories by cable type, joint, load and soil temperature; combine with partial-discharge or load data where available (see PRED-006).
• District heating and sewer pipes: same method; for sewers condition data comes from CCTV coding (VIS-043), which is the strongest predictor input.
• Municipal Stadtwerke (SMB scale): small networks have few breaks, so pooling data across utilities (cooperatives, associations, vendor models) or using expert-informed priors is essential.
• Private use: none.

How it works

1. asset register + work orders + GIS layers + climate + (optional) CCTV/pressure data
2. data cleaning (segment IDs, install-year gaps, break-to-segment matching, censoring)
3. feature engineering (material, age, diameter, soil, traffic, previous breaks, pressure, neighbours)
4. failure model (survival regression | gradient boosting | random survival forest | NHPP)
5. consequence model (customers, critical sites, repair cost, traffic impact)
6. risk = likelihood x consequence
7. ranking on GIS map
8. optimisation under budget and road-works constraints (renewal programme, scenarios)
9. engineer review
10. capex plan, inspection and logger placement
11. new work orders feed back

• Models: the core are time-to-event methods that handle right-censoring (most pipes have not failed yet) and left-truncation (history starts late): Cox / Weibull models (lifelines, scikit-survival), random survival forests, and gradient-boosted trees on yearly segment-year tables [8][9]. Deep models rarely beat well-tuned trees on such tabular data.
• Spatial effects: soil and ground-movement layers from remote sensing and geology maps, plus neighbour features (a break on an adjacent segment raises risk).
• Optimisation: renewal selection is a knapsack-type problem with road-works clustering; solvers (OR-Tools, MIP) turn risk scores into a programme.
• LLM layer (optional): natural-language questions on the risk model and drafting of capex justification texts; numbers always come from the model, not from the LLM.
• Human-in-the-loop: asset engineers validate rankings, override segments with known local context, and decide on renewal. Statutory inspections and safety decisions stay human.

Data preparation

The model needs the asset register per segment (install year, material, diameter, joints), break and repair history, GIS layers (soil, groundwater, traffic) and consequence data. In a typical utility these live in the GIS, the work-order or maintenance system (SAP PM or similar), paper repair cards and Excel lists of individual engineers. This is the dominant effort, roughly 50-60 % of the project.

Typical gaps: missing or guessed installation years, inconsistent material codes, breaks not linked to the correct segment, repair reports only on paper or as scans, duplicated segments after GIS migrations, and knowledge about "problem streets" held only by long-serving fitters.

Concrete steps: inventory sources and name data owners (PREP-001, PREP-004); export from ERP, GIS and SCADA (PREP-005); digitise repair cards with OCR (PREP-006); tidy Excel lists into tables (PREP-011); harmonise material and cause codes with a category scheme (PREP-019); and harvest break labels from work orders (PREP-020). Tools: QGIS or PostGIS for cleaning and linking, Python/pandas, an LLM to extract structured fields from free-text repair reports. See also GUIDE-003.

Implementation paths

Which path for whom

• smb (small Stadtwerke, municipal utilities): A - mature asset-management and renewal-planning products exist at affordable prices, and the process (rank segments, plan budgets) is standard.
• enterprise (large utilities): B - renewal planning is core and must use own asset register, GIS and work-order data, so a custom model on cloud APIs and platforms fits better than a closed product.
• public (municipal operators, water associations): B - build on a sovereign EU/German cloud to meet procurement and sovereignty rules while integrating existing GIS.
• developer: C - open-source survival analysis and gradient-boosting libraries run on HW1 and give full control over the model.

Path A — Buy (recommended for most utilities)

• Dedicated predictive-risk platforms for water: VODA.ai (break prediction, capital planning, condition assessment, logger placement; starts from GIS, pipe attributes, break records and work orders) [1], Fracta (likelihood of failure, cost of failure, total risk; as-built digitisation) [2], Baseform (Portugal-based, EU vendor; predictive renewal planning plus leakage and water-balance analytics) [3].
• Network asset-management suites: GIS and hydraulic-modelling vendors (Esri ArcGIS Utility Network, Bentley, Autodesk/Innovyze) and German asset-management consultancies/software houses (e.g. 3S Consult) often include rule-based or statistical renewal modules; verify the model type.
• Check: can the vendor run in EU regions or on-prem, import your GIS/work-order export, show the back-test (predicted vs actual breaks on held-out years), explain drivers, and export the ranked list?
• Choose A when: more than a few hundred km of network with at least ~10 years of repair history and no data-science team. Small Stadtwerke should ask about pooled/benchmark models.

Path B — Build on cloud APIs

• Managed ML on EU regions (Azure ML, Google Vertex AI, AWS SageMaker Frankfurt, STACKIT/IONOS as EU sovereign options) for training and scheduling; BigQuery/PostGIS or Snowflake for the segment-year table.
• Satellite and geodata via Copernicus services (ground motion EGMS, land cover) and national geology portals as features. LLMs (Claude, Gemini, GPT, Mistral) only for the assistant layer and report drafting.
• Choose B when: you want custom features and integration with in-house GIS/ERP, but are fine with cloud processing of non-personal asset data.

Path C — Self-host (open source)

• Stack: PostgreSQL/PostGIS or QGIS for GIS, Python with lifelines, scikit-survival, XGBoost/LightGBM, statsmodels; optional Hugging Face tabular or time-series models; OR-Tools for programme optimisation; Metabase/Superset or a QGIS plugin for maps; an open-weight LLM only if an assistant is wanted.
• Hardware: HW1 (a CPU server) is enough: tens of thousands of segments train in minutes. No GPU needed.
• Effort: data cleaning dominates (segment-ID matching, missing install years), not computing.
• Choose C when: critical-infrastructure data must stay on-prem and in-house data science exists.

Path D — Custom training

This is the normal way to get the best results: train on your own break history, since soil, materials and operating practice are local. A few thousand pipe segments and a few hundred recorded failures are a practical minimum; with fewer, pool data across utilities or use Bayesian/expert priors. Add imbalance handling, spatial cross-validation (hold out regions, not random rows) and temporal back-testing.

State of the art (2026)

• Methods: survival analysis and gradient-boosted trees remain the workhorses. Random survival forests and uncertainty quantification are used for long-term forecasts [5]; the Data61/Western Water work derives failure drivers for prioritising renewals [4]. Foundation-model approaches for tabular data (e.g. TabPFN) are being tried on small datasets; no published evidence of clear superiority in this domain was verified for this file.
• Data side: AI-assisted digitisation of paper as-builts (Fracta) and CCTV auto-coding (VIS-043) fill the register gaps that most limit accuracy [2].
• Integration: risk models are increasingly combined with leakage and water-balance analytics, pressure management and meter planning in one platform [3]; LLM assistants query these data in plain language.
• Limits and failure modes: (1) unreliable install years and unrecorded repairs bias results; (2) breaks are rare events, so small networks give noisy models; (3) past repairs change the future (replaced pipes disappear from the data; "survivor bias"); (4) models predict probability, not the failure date; (5) a segment without breaks is not necessarily safe; (6) results do not transfer across utilities without recalibration; (7) causal claims about drivers need care (correlated age/material). Use risk scores for prioritisation and sample-based inspection to verify.

Shared resources & combinations

• Same stack (tabular-ml + time-series): PRED-006 predictive maintenance, PRED-025 field-failure early warning, PRED-007 load forecasting, PRED-026 battery RUL: gradient boosting and survival tooling, one CPU/HW1 server and a shared feature store.
• Data inputs from: VIS-043 sewer/pipe CCTV coding (condition grades), ANOM-013 leak detection (acoustic/pressure evidence), VIS-038 and remote-sensing pipelines (ground movement, land cover), PRED-017 weather (frost and drought triggers), PRED-027 flood forecasting.
• Downstream/sideways: PRED-046 structural health monitoring (same risk-based capex logic for bridges and buildings), ANOM-008 (non-technical loss/theft), capex and budget planning (controlling).
• Shared building blocks: GIS (PostGIS, QGIS, Esri), asset register / CMMS export, work-order text mining with an LLM (turn free-text repair notes into failure causes), OR-Tools optimisation.

Effort, cost & KPIs

• Path A: weeks to onboard data; platform pricing typically subscription by network length (ask the vendor); vendor timelines claim first risk outputs within 3-6 months [1].
• Path B/C: 3-6 months for a first validated model, most of it data cleaning and GIS work; 0.5-1 FTE data engineer/scientist plus asset engineer time.
• Path D: ongoing; annual retraining and recalibration after each renewal campaign.
• KPIs: back-test lift (share of future breaks captured in the top 10 % riskiest segments vs age-based ranking), breaks per 100 km per year, renewal cost per avoided break, share of renewals executed on model-ranked segments, inspection hit rate, supply interruptions (minutes per customer), real losses/ILI, data completeness (% segments with install year and material).

People & change

Involve management early for the budget and the risk appetite behind renewal decisions, the department lead (network planning) as process owner, IT and the data owners of GIS, SAP and SCADA from the start, and IT security if SCADA data is used. Fitters and network engineers are the key end users: their field knowledge validates the ranking and corrects wrong records.

Works council: normally no co-determination under §87(1) no. 6 BetrVG, because the system evaluates pipes, not employees; inform it anyway under §90(1) no. 3 if planning processes change, and check that work-order data is not used to assess individual fitters. Personal data is essentially absent, so a DPIA is not expected, but the DPO confirms this (customer addresses).

AI literacy: support staff with a short training on probabilities and model limits (GUIDE-006). Pilot on one network section or material class. Typical concern: "the model replaces experience" - address it by showing the ranking as a proposal; engineers decide on every renewal and can override. Keep records of overrides to improve the model. See GUIDE-008 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: depends. Annex III point 2 lists AI systems intended as safety components in the management and operation of critical digital infrastructure, road traffic, or the supply of water, gas, heating and electricity. A planning model that ranks pipes for multi-year capex does not run the network in real time and is typically not a safety component, so minimal/non-listed; but a model that directly controls or gates safety-critical operation (e.g. automated shut-off decisions) could fall under high risk. Document the intended purpose and classification (Art. 6) [10]. AI literacy (Art. 4) applies.
• Sector law: recast Drinking Water Directive (EU) 2020/2184 (risk-based approach, leakage) [7]; the CER Directive (EU) 2022/2557 and NIS2 apply to operators of critical entities and their IT/OT security, so asset data and risk maps of critical infrastructure are sensitive [11]; in Germany DVGW technical rules for water and gas, and regulator rules on network operation/investment recognition.
• GDPR: asset and GIS data are non-personal; service-line, customer-connection and complaint data can be personal - keep customer attributes out of the model or aggregate them.
• Security: network maps and risk rankings reveal vulnerabilities; restrict access, host in EU, log access.
• Risks and mitigations: over-trust in scores (keep engineers in the loop, inspect samples); bias towards poorly documented areas (report uncertainty); legal liability for gas safety stays with the operator and statutory inspection intervals cannot be extended on the basis of a model.

Getting started

1. Export the asset register and 10+ years of repair/work-order data; check segment-ID matching and the share of segments with install year and material.
2. Build a baseline: failure rate by material and age band in a notebook (lifelines Cox/Weibull), then a gradient-boosted segment-year model.
3. Back-test: train up to year N-3, predict the last three years, compare with the current age-based ranking on lift in the top 10 % segments.
4. Add consequence data (customers, critical sites, repair cost) and run a budget scenario for the next renewal programme; review the top 100 segments with field engineers.
5. In parallel, request demos with 2 vendors from Path A using the same data export for a head-to-head test.

Sources

1. VODA.ai - product overview, data inputs and deployments - https://www.voda.ai/
2. Fracta - AI risk analysis for water utilities (likelihood/cost of failure, as-built digitisation) - https://fracta.ai/
3. Baseform - predictive renewal planning and water analytics (illustrative renewal example) - http://baseform.com/np4/home
4. Weeraddana et al. - Utilizing machine learning to prevent water main breaks by understanding pipeline failure drivers - https://arxiv.org/abs/2006.03385
5. Weeraddana et al. - Long-Term Pipeline Failure Prediction Using Nonparametric Survival Analysis - https://arxiv.org/abs/2011.08671
6. Kumar et al. - Using Machine Learning to Assess the Risk of and Prevent Water Main Breaks (Syracuse) - https://arxiv.org/abs/1805.03597
7. Directive (EU) 2020/2184 on the quality of water intended for human consumption - https://eur-lex.europa.eu/eli/dir/2020/2184/oj
8. lifelines - survival analysis in Python (documentation) - https://lifelines.readthedocs.io/en/latest/
9. scikit-survival - documentation - https://scikit-survival.readthedocs.io/en/stable/
10. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
11. Directive (EU) 2022/2557 (Critical Entities Resilience) - https://eur-lex.europa.eu/eli/dir/2022/2557/oj
12. DVGW - Wasser (sector overview, network topics, research) - https://www.dvgw.de/themen/wasser