Energy theft & grid anomaly detection

Utilities lose revenue and grid stability to meter tampering, illegal taps, faulty meters and billing errors (so-called non-technical losses, NTL), and operators need early warning of abnormal behaviour in the grid itself. Machine learning on smart-meter and feeder data ranks meters and network sections by suspicion so that scarce field crews inspect the right sites. In Europe the value lies mostly in focused revenue assurance and metering-quality control, with strict limits on how customer consumption data may be used.

Problem & value

• Today: utilities rely on rule-based checks (zero consumption, tamper-event flags, sudden drops), periodic meter reading plausibility checks, random or tip-based inspections, and manual energy-balance comparisons between a transformer and the meters behind it.
• Scale of the problem: a survey of the NTL literature notes that in some countries NTL can reach up to 40% of the electricity distributed [1]. Wikipedia summarises a widely quoted Northeast Group estimate of about USD 96 billion lost per year globally, but the figure is an industry estimate and the article itself does not reconcile it with an earlier 89.3 billion figure [2]. Losses are concentrated in emerging markets (e.g. India, Brazil); in Germany and most of the EU, theft is a much smaller share and is a criminal offence (§ 248c StGB, up to five years' imprisonment, prosecuted on complaint) [3].
• AI value: better inspection hit rates than random or rule-based selection, earlier detection of faulty meters (which are far more common than theft in mature markets), and detection of grid-side anomalies (phase imbalance, voltage excursions, unexpected back-feed from PV) that rules miss. No reliable public hit-rate figure for EU deployments was found; vendors publish case studies, but treat them as marketing until measured on your own inspection data.
• Honest framing: much of the "theft" signal in smart-meter data is actually a defective meter, a vacant property, a new PV system or a changed household. Models must not be presented to customers as proof of wrongdoing.

Variants

• Revenue protection (NTL detection): customer-level scoring of meters for tampering, bypass, illegal connections; typical in emerging markets and for large industrial customers in the EU.
• Metering quality control: detecting stuck, drifting, mis-wired or mis-configured smart meters (CT ratio errors, reversed phases) — the dominant EU use case.
• Energy-balance / feeder loss analysis: compare transformer-level input with the sum of customer meters to localise loss to a network section before inspecting individual customers.
• Grid operations anomaly detection: detecting abnormal events in SCADA/PMU/low-voltage sensor streams (voltage, frequency, flows) for network operators; overlaps with ANOM-005 and OPT-022.
• Cyber/physical tamper signals: meter communication anomalies, firmware or time-sync oddities, flagged to the security team.
• Water/gas networks: analogous problems for leakage and illegal tapping are in ANOM-013.
• Private use: not applicable as theft detection; households get consumption anomaly hints via PRED-031.

How it works

1. smart meters / head-end system (MDM) + SCADA + billing + inspection history
2. data cleaning, gap filling, meter-event decoding
3. feature engineering (load profile shape, seasonality, peer-group comparison, feeder balance)
4. detectors: supervised classifier (labelled inspections) + unsupervised anomaly scores + feeder-balance rules
5. ranking and reason codes, calibrated to inspection capacity
6. human review (analyst)
7. field inspection / meter exchange / customer contact
8. inspection result fed back as label (closed loop)

• Supervised models: gradient boosting on load-profile features trained on past inspection outcomes. The main weakness is label bias, because past inspections were not random and so the model learns where inspectors already looked [1].
• Unsupervised / semi-supervised: isolation-forest-type, autoencoder and clustering methods for peer-group deviation when labels are scarce. The TSB-AD benchmark (1,070 time series, 40 datasets) found that simple statistical methods often match or beat deep networks, so start simple [5].
• Forecast-residual detection: a time-series foundation model such as Chronos-2 (Apache-2.0, zero-shot, covariate-aware) can forecast expected consumption and flag large residuals [6]; this needs validation against classic baselines.
• Graph methods: the grid topology (feeder → transformer → meter) provides natural structure for balance checks and graph-based localisation.
• LLM role (supporting only): summarising technician notes, drafting inspection reports and explaining reason codes to analysts; LLMs are not the detector.
• Human-in-the-loop: an analyst validates every high-ranked case; inspection and any contact with the customer are human decisions. Automated billing corrections or disconnections based solely on a model score should not happen.

Data preparation

Smart-meter interval data sits in the MDM/AMI head-end, grid measurements in SCADA, customer and billing history in the billing system (e.g. SAP IS-U), and inspection outcomes in field-service tools, spreadsheets or paper reports. Technician notes are free text.

Typical gaps: missing or late meter intervals, meter replacements that break time series, unreliable tamper flags, feeder-to-meter mapping errors (the basis of energy balances), few confirmed theft labels, inspection results stored only on paper, and strict access rights because consumption data is personal data.

Preparation steps: - Validate and fill meter data (VEE logic), and link each meter to its transformer and feeder. - Join customer, tariff and billing history through pseudonymised keys (PREP-022). - Harvest labels from inspection reports, including negative findings (PREP-020), and digitise paper reports (PREP-006, PREP-008). - Structure technician notes with a simple category scheme (PREP-019). - Define who may see consumption data and for which purpose (PREP-024, PREP-025). - Build an evaluation set with revenue-protection staff (PREP-018).

Tools: SQL, pandas/Polars, a time-series database, n8n for exports. Plan around half of the project effort for preparation; GUIDE-003 provides a readiness check and PREP-005 covers extraction from billing and SCADA systems.

Implementation paths

Which path for whom

• Enterprise: Path C, because large utilities hold sensitive consumption data of millions of customers, run their own data platforms and benefit from reusing the stack for other grid analytics.
• Developer: Path C, because open-source time-series, tabular and graph methods can be run and tested on meter data on own HW1 hardware.
• Public: Path C, because municipal utilities and public operators need sovereign hosting and transparent, auditable models.
• SMB: Path A, because small Stadtwerke already run AMI/MDM platforms whose revenue-protection modules are supported products with inspection work-order integration.

Path A — Buy (off-the-shelf)

• Categories: revenue-protection and analytics modules of meter data management (MDM) and AMI platforms, utility analytics suites, and grid-monitoring software. Representative vendors with such offerings include Itron, Landis+Gyr (Swiss), Siemens (grid software), Oracle Utilities, SAP (IS-U/S4 Utilities), and specialist analytics providers; European Stadtwerke typically also use German metering and billing providers. Check each vendor's current module names and EU hosting before shortlisting.
• Choose A when: you already run an AMI/MDM platform and want a supported module with inspection work-order integration. Most small and mid-sized Stadtwerke fit here.
• Check: training on your own inspection outcomes, reason codes, GDPR/MsbG role (processor under Art. 28), on-prem or EU hosting, and an exit path for the models.

Path B — Build on cloud APIs

• Stack: managed data platform in an EU region (Azure, AWS Frankfurt, Google Cloud EU, or an EU provider such as STACKIT or IONOS), streaming ingest, a feature store, and managed ML training. Use cloud LLM APIs (e.g. Claude, Mistral, Gemini in EU regions) only for text tasks such as report drafting, never for raw customer-level consumption data without a legal basis.
• Choose B when: you have a data team and need custom logic across several data sources.
• Constraint: § 49 MsbG limits who may process metering personal data and for what purposes (authorised entities, with processors under GDPR Art. 28) [4]; align the architecture with your data protection officer.

Path C — Self-host (open source)

• Libraries: scikit-learn, XGBoost/LightGBM, and PyOD (BSD-2, 60+ detectors incl. isolation-forest and autoencoder families) for detectors [7]; Chronos-2 or similar time-series foundation models for forecasting [6]; TSB-AD for benchmarking detector choices [5].
• Platform: Kafka or MQTT, TimescaleDB or InfluxDB, Airflow/Dagster, MLflow, a case-management tool, and n8n for work-order workflows.
• Hardware: HW1 is enough — CPU servers handle millions of meters for tabular and time-series models; a GPU is optional for foundation-model inference or an on-prem LLM for note summarisation.
• Choose C when: data must stay on-prem (common for grid operators) and you have data-engineering capacity.

Path D — Custom training

• Usually the core of this use case: train a utility-specific classifier on your own inspection outcomes plus unsupervised models per customer segment. Needs thousands of labelled inspections with confirmed outcomes (theft / faulty meter / no finding) and a deliberately random inspection sample to de-bias labels. Active learning (inspecting the most informative cases) improves labels over time [1].

State of the art (2026)

• Research direction: AI/ML is the main research direction for NTL detection [1][2]. The survey literature highlights label bias, class imbalance, concept drift (new PV, heat pumps, EV charging change load shapes) and weak evaluation protocols as the central problems [1].
• Benchmarks: time-series anomaly detection now has more rigorous evaluation (TSB-AD with VUS-PR as preferred measure); simple statistical baselines frequently remain competitive, so claims of large gains from deep models need scrutiny [5]. Public NTL datasets are small or anonymised relative to real utility data, so published accuracy rarely transfers.
• Foundation models: pre-trained time-series models (Chronos-2, October 2025) bring zero-shot forecasting with covariates [6]; their use for anomaly detection is promising but not yet a proven replacement for tuned per-utility models.
• Agentic tooling: PyOD 3 adds an orchestration layer with LLM-agent workflows for exploring anomaly-detection tasks, but the detector itself remains classical or neural [7].
• Limits and failure modes: new prosumers (PV, battery, heat pump) look like anomalies; vacancies and seasonal homes trigger false positives; legitimate load changes after tariff switches; manipulated meters that produce plausible profiles; and demographic bias if neighbourhood features proxy for protected groups.

Shared resources & combinations

• Same stack (anomaly-detection + time-series + tabular-ml + streaming): ANOM-005 (industrial sensor anomalies), ANOM-013 (leak detection in water/gas networks), ANOM-030 (business KPI anomalies), ANOM-001 (fraud scoring uses the same ranking-and-calibration pattern). One feature store and streaming platform on HW1 can serve all of them.
• Data sharing with forecasting: PRED-007 (load and generation forecasting) provides expected-load baselines; PRED-031 (smart-meter disaggregation) offers appliance-level signatures; PRED-033 (buried network failure prediction) shares grid asset data.
• Downstream actions: OPT-022 (distribution grid congestion management) consumes grid anomaly alerts; ANOM-006 covers billing and ledger anomalies after a correction is booked.
• Shared hardware: CPU-based time-series and tabular cluster (HW1); optional single GPU (HW2) if an on-prem LLM is also used for case-note summarisation.

Effort, cost & KPIs

• Path A: weeks to a few months for a module on an existing MDM; licence cost typically scales with meter count.
• Path B: 3–6 months for a first production pipeline including data access, feature engineering and work-order integration.
• Path C/D: 6–12 months with a data science team; the largest cost driver is data quality, labelling and field-inspection capacity, not compute.
• KPIs: inspection hit rate (confirmed findings per inspection) versus random baseline, share of faulty-meter findings, recovered energy (kWh) and revenue, time-to-detection, false-positive rate per analyst hour, feeder loss percentage, precision at the inspection capacity (top-k).

People & change

Involve management for the mandate, the head of revenue protection or grid operations as process owner, IT and the owners of MDM, SCADA and billing data, plus the DPO and legal/compliance from the start. Field technicians and revenue-protection staff are the daily users; customers are affected by inspections, so the communication and legal rules for visits must be agreed in advance.

Works council: depends. The model scores meters and customers, not staff, but inspection assignments, technician performance statistics and system logs can be suitable for monitoring under §87(1) no. 6 BetrVG, and §90(1) no. 3 requires information on AI-planned work processes. Municipal utilities in public ownership involve the Personalrat (GUIDE-004). Household consumption profiles are personal data and reveal behaviour, so a DPIA under Art. 35 GDPR is likely; an AVV is needed for every processor (GUIDE-005).

Support AI literacy of inspectors and analysts (Art. 4 AI Act, GUIDE-006). Choose one grid area as pilot, compare ranked cases with experienced inspectors, and report hit rates openly. Typical concerns are wrongly accused customers and extra workload from false alarms; address them by treating a score only as a reason to inspect, never as proof, with a human deciding on every case and every follow-up measure (GUIDE-008, GUIDE-009).

Risks, regulation & compliance

• EU AI Act: classification depends on use. Annex III point 2 lists AI systems intended as safety components in the management and operation of critical digital infrastructure, road traffic or the supply of water, gas, heating or electricity as high-risk [8]. A revenue-protection model that only ranks meters for inspection is not a safety component and is typically minimal risk; a system used as a safety component for operating the grid (e.g. automated protection or switching decisions) can fall under high-risk obligations. Document the intended purpose carefully. AI literacy (Art. 4) applies in all cases.
• GDPR and metering law: 15-minute smart-meter data is personal data revealing household behaviour. In Germany § 49 MsbG restricts processing to authorised entities and purposes [4]; purpose limitation, data minimisation (aggregate where possible), DPIA, and transparency to customers are needed. Automated decisions with legal effect on customers (billing corrections, disconnection) trigger GDPR Art. 22 safeguards; keep a human decision.
• Criminal and civil law: theft of electricity is an offence under § 248c StGB [3]; the model score is a lead, not evidence. Evidence must come from physical inspection.
• Fairness: avoid features that proxy for neighbourhood, ethnicity or income; monitor flag rates by area.
• Security: NIS2 and sector rules apply to grid operators; treat the analytics platform and meter data as critical assets.
• Mitigations: random control inspections to measure true hit rate, drift monitoring, reason codes, audit logs and a customer complaint route.

Getting started

1. Inventory data: meter interval data, tamper/event logs, feeder balance data, and every past inspection with its recorded outcome.
2. Build a baseline of simple rules and a feeder energy-balance report; measure the current inspection hit rate.
3. Train a gradient-boosting model and an unsupervised detector (PyOD) offline; compare to the baseline using precision at the available inspection capacity.
4. Agree the legal basis, DPIA and § 49 MsbG role assignment with the data protection officer before using customer-level data.
5. Pilot with one region: inspect the top-ranked cases plus a random sample, feed outcomes back, and review monthly.

Sources

1. Glauner et al. — The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey — https://arxiv.org/abs/1606.00626
2. Wikipedia — Electricity theft (summary of Northeast Group estimates and country figures) — https://en.wikipedia.org/wiki/Electricity_theft
3. § 248c StGB, Entziehung elektrischer Energie — https://www.gesetze-im-internet.de/stgb/__248c.html
4. § 49 MsbG, Verarbeitung personenbezogener Daten — https://www.gesetze-im-internet.de/messbg/__49.html
5. TSB-AD benchmark (NeurIPS 2024, "The Elephant in the Room") — https://github.com/TheDatumOrg/TSB-AD
6. Amazon Science — Chronos-2 / Chronos-Bolt time-series forecasting models — https://github.com/amazon-science/chronos-forecasting
7. PyOD — Python library for anomaly detection — https://github.com/yzhao062/pyod
8. EU AI Act, Annex III (critical infrastructure) — https://artificialintelligenceact.eu/annex/3/