Smart-meter energy disaggregation (NILM) & personalised energy-saving insights

Households and sites only see one number: total consumption. Non-intrusive load monitoring (NILM) estimates from that single meter signal how much energy heating, a heat pump, an EV, the fridge or standby loads used. Utilities and energy apps use the result to explain bills, coach customers towards savings and dynamic tariffs, and spot failing appliances, increasingly with a genAI assistant on top that turns the numbers into plain-language advice.

Problem & value

• Today: customers receive an annual or monthly total and do not know what drives it. Utilities field "why is my bill high?" calls; efficiency programmes are mass-mailed rather than targeted. Sub-metering every appliance is expensive and intrusive.
• AI value: a software-only "virtual sub-meter" from data already collected. It allows targeted offers (heat-pump or HVAC replacement, EV charging tariffs), fewer high-bill calls, behaviour change, and appliance fault hints.
• Evidence (vendor-reported, no independent methodology given): Bidgely states on its website that deployments at utilities reduced high-bill call volume by more than 50% and doubled EV managed-charging enrolment [3]. The EU H2020 project Eco-Bot (2017-2021, coordinated by RISA Sicherheitsanalysen, Berlin) combined low-resolution smart-meter data, NILM and a virtual energy assistant in B2B, B2B2C and B2C pilots across Germany, Spain, the UK, Greece, Poland and Turkey [1].
• Market driver: smart-meter rollout under the German MsbG [5] and dynamic-tariff offers such as Tibber's 15-minute-price tariff, which requires a smart meter [6], create both the data and the need for guidance.

Variants

• Utility / Stadtwerk programme (B2B2C): disaggregation across the whole customer base for bill explanation, marketing targeting and demand-response enrolment.
• Energy app for households (B2C): app showing appliance breakdown from a meter reader (e.g. an optical reader on the meter interface) or the metering operator's portal, plus saving tips.
• Dynamic-tariff and EV/heat-pump advice: detect flexible loads and recommend shifting to cheap hours; verify the shift happened.
• Industrial virtual sub-metering (merged candidate): estimate machine- or line-level energy from one feeder meter in factories, as a cheap complement to ISO 50001 energy monitoring; links to PRED-006 and OPT-007.
• SMB / building: shops, offices, small hotels: HVAC, refrigeration and lighting split.
• Appliance health: drift of a fridge or heat-pump cycle signature as an early-fault signal.

How it works

1. smart meter / meter reader / MSB portal (15 min ... 1 s)
2. cleaning, gap filling, resampling, weather join
3. disaggregation model (per end-use or per appliance power estimate)
4. calibration with survey / known appliance inventory (optional)
5. insight engine (rules + ranking: biggest saving, tariff fit, anomalies)
6. LLM layer: explanation and chat in the customer's language
7. app / email / call-centre agent screen; customer feedback loop

• Disaggregation: the classic task is energy estimation per appliance from aggregate power. Approaches range from hidden Markov models and sparse coding to neural sequence models (seq2point, denoising autoencoders, TCN, transformers). The NILMTK ecosystem implements all of these [2][7]. Accuracy depends strongly on sampling rate: at 1-s to 1-min resolution, high-load switching appliances can be identified; at the 15-min resolution of most European smart meters, only coarse end-uses (heating/cooling, always-on base load, EV charging) are realistic.
• LLM layer: the LLM does not disaggregate; it receives the structured results and writes explanations, grounded on the numbers (RAG/tool calls). Numbers are computed, not generated.
• Human-in-the-loop: customers can correct appliance labels ("this is my heat pump"); utilities review campaign targeting and wording.

Data preparation

Needed data: smart-meter load curves (ideally 1 Hz to 15-minute resolution), tariff and contract data, building or site master data, weather and occupancy context, and for training some labelled appliance-level measurements. They typically live in the meter data management system, the billing system or ERP, a gateway or energy-management portal, and in an SMB often only in the supplier's web portal or monthly PDF invoices. Typical gaps: coarse 15-minute resolution (disaggregation is weak there), missing intervals, unit and time-zone errors, no ground-truth labels, unclear customer consent and scattered site master data.

Steps: (1) inventory meters and data sources (PREP-001, PREP-002); (2) get data out of the portals via API, CSV or the smart-meter gateway (PREP-005); (3) convert paper or PDF invoices if needed (PREP-006); (4) clean and resample time series, fill gaps, align timestamps (pandas, InfluxDB or TimescaleDB); (5) create labels through short sub-metering campaigns (PREP-018); (6) pseudonymise customer IDs and check consent (PREP-022, PREP-025). Roughly 30 to 40 percent of effort is data work; a bought product shifts most of it to the vendor. See GUIDE-003.

Implementation paths

Which path for whom

• SMB: Path A, because energy-monitoring and sub-metering products exist at SMB prices and the process is standard.
• Enterprise (utilities, energy-service providers): Path C, because disaggregation is part of the core customer offer, volumes make per-meter fees high and household load curves are sensitive.
• Developer: Path C, because open-source NILM toolkits and time-series stacks allow full control and research reproducibility.
• Private: Path A, because supplier apps and home-energy monitors work out of the box without own infrastructure.
• Public (municipal utilities, housing bodies): Path C, because sovereignty, procurement rules and tenant data protection favour self-hosted or sovereign-cloud operation.

Path A — Buy (recommended for SMBs and households)

• Utility analytics platforms: Bidgely (UtilityAI, patented disaggregation, SaaS or in the utility's cloud; US utilities plus e.g. Sorgenia in Italy) [3]; Oracle Utilities Opower and Itron analytics for home energy reports and engagement (check current feature scope with vendors).
• Consumer energy apps / tariffs: Tibber (DE; live consumption with the Pulse reader, dynamic tariff) [6]; Octopus Energy (Agile, Intelligent Octopus Go) [4]. These show consumption and automate charging; appliance-level breakdown varies by product and market.
• Choose A when: you are a utility with tens of thousands of meters and no ML team; verify EU hosting, GDPR role (processor vs. joint controller), and whether disaggregation works at your meter resolution.

Path B — Build on cloud APIs

• Run feature engineering and models in an EU cloud region (managed ML such as Azure ML, Vertex AI, SageMaker, or EU providers such as STACKIT, IONOS, OVHcloud); time-series store (TimescaleDB, InfluxDB) and streaming ingestion.
• Add a frontier or EU-hosted LLM (Claude, Mistral, Gemini, GPT with EU data residency) for explanations and chat, with tool calls into the insight engine.
• Choose B when: you are a software vendor, Stadtwerk with data team, or app start-up that wants its own UX but not its own GPU fleet.

Path C — Self-host (open source)

• Toolkit and models: NILMTK for data conversion, metrics and classical methods; nilmtk-contrib (Apache-2.0) for PyTorch models incl. DAE, seq2point, WindowGRU, BERT-style, TCN, PatchTST and NILMFormer [2][7]. Public datasets such as REDD, UK-DALE and REFIT are used for pre-training and evaluation; they must be fetched from their custodians.
• Stack: PostgreSQL/TimescaleDB, Python, ONNX Runtime for inference; a small open-weight LLM (7-14B class, served by vLLM or Ollama) for explanations.
• Hardware: the disaggregation models are small; CPU or a single modest GPU (HW1) handles hundreds of thousands of customers in batch. Only the chat layer benefits from HW2.
• Edge variant: on-device inference in a home gateway or Raspberry-Pi-class box keeps high-resolution data in the home (strong GDPR argument).

Path D — Custom training

Often necessary: models trained on public (mostly UK/US) households transfer poorly to German heat pumps, EVs and 15-minute data. Fine-tune on a labelled subset: customers with sub-metered appliances, heat-pump manufacturers' data, or survey-based appliance inventories. Transfer learning plus a few hundred labelled homes per archetype is a realistic starting point. Ground-truth acquisition (consented sub-metering pilots) is the main cost.

State of the art (2026)

• Research moved from per-appliance seq2point networks to transformers, mixture-of-experts and foundation-style time-series models; the open toolchain now includes such architectures (NILMFormer, NILMMoE, PatchTST, TimesNet) in nilmtk-contrib [7].
• Benchmarking is being standardised via a separate benchmark project around NILMTK [2], because published results on REDD/UK-DALE/REFIT are hard to compare (different splits, houses, metrics).
• Commercial practice relies on low-frequency data and hybrid methods: ML disaggregation plus physics models (e.g. weather-normalised heating) and appliance-ownership priors [1][3].
• GenAI assistants (as in the Eco-Bot concept of a virtual energy assistant [1]) convert insights into dialogue; the open question is grounding and avoiding wrong savings claims.
• Limits and failure modes: 15-min data hides short cycles; simultaneous similar loads and multi-state appliances are confused; models generalise poorly across countries and building types; vendor accuracy claims are rarely independently audited; rebound effects reduce real savings; per-appliance estimates must be presented as estimates with uncertainty.

Shared resources & combinations

• Shared building blocks: time-series store, weather data join, forecasting and anomaly libraries, LLM explanation layer, meter-data ingestion (MaKo/EDIFACT, MQTT, vendor APIs).
• PRED-007 energy load and renewable forecasting: same data; disaggregation sharpens flexibility forecasts (heat pump, EV) per customer.
• ANOM-008 energy theft and grid anomaly detection: shares meter data and anomaly models; keep separate purposes and consents (insight vs. enforcement).
• OPT-007 building energy management and HVAC optimisation: acts on the insights for commercial sites.
• PRED-006 predictive maintenance: appliance/heat-pump signature drift as a fault signal.
• PERS-005 private local assistant on a home server: self-hosted dashboard for household data.
• Hardware: HW1 for models, HW2 only if the same server also runs the chat LLM.

Effort, cost & KPIs

• Path A: weeks to a few months for integration with meter-data management and CRM; SaaS priced per meter/customer.
• Path B: 3-6 months for a production pipeline incl. app and LLM layer.
• Path C: 4-9 months; ML engineering plus MLOps; low hardware cost.
• Path D: adds ground-truth data collection (the main cost driver, 3-12 months).
• KPIs: per-end-use estimation error versus sub-metered ground truth (e.g. normalised disaggregation error, F1 for on/off), share of customers engaging with insights, high-bill calls, measured kWh savings against a control group, enrolment in tariffs/programmes, customer satisfaction, opt-out rate.

People & change

Involve management and the facility or energy manager (process owner) for goals and budget, IT for gateways and network access, and the data owner (utility portal, billing). The data protection officer is needed because household load curves reveal behaviour such as presence and routines; a DPIA (Art. 35 GDPR) is likely for large-scale customer analysis, with AVV (Art. 28) for any vendor. Customers need clear consent and a simple opt-out.

Works council: depends. If the analysis covers a company site, appliance-level data can reveal when individual machines or workplaces are used and hence employee behaviour, so §87(1) no. 6 BetrVG may apply; agree purpose limits (energy only, no staff evaluation) in a short works agreement and inform the council early (§90(1) no. 3). For private households the question does not arise.

AI literacy: the AI Act (Art. 4) requires measures to support staff literacy; brief the energy manager and customer service on accuracy limits of disaggregation (estimates, not measurements). Pilot with one site or a volunteer customer group, communicate savings tips as suggestions, and keep a human in the loop for fault hints and tariff advice. Typical concern: "you are watching us"; address with transparency and data minimisation. See GUIDE-004, GUIDE-005, GUIDE-009.

Risks, regulation & compliance

• GDPR: appliance-level profiles reveal habits (presence, sleep, working from home), so load data is personal data and sensitive in practice. Legal basis normally consent or contract; a DPIA is advisable; purpose limitation (no repurposing for marketing or enforcement without basis); data minimisation (use the lowest resolution that works); on-device processing where possible.
• German MsbG: the act regulates processing, transmission and deletion of metered values and the customer's right to information (see §§ 49 ff., 61) [5]; third-party use of data needs the customer's explicit involvement. Check requirements with your Messstellenbetreiber.
• EU AI Act: typically minimal risk: no Annex III category for energy insight and advice. Note that AI in the management and operation of critical digital infrastructure and electricity supply can be high-risk under Annex III point 2 if it is a safety component; consumer advice is not. Art. 50 transparency applies to chat assistants (disclose that the user talks to AI) [8]. Do not infer emotions or sensitive attributes from load data.
• Consumer protection: savings claims in advice must be substantiated; avoid guaranteed-saving language.
• Mitigations: opt-in, transparent explanation of estimates, deletion rights, aggregation for analytics, control groups for measuring claimed savings.

Getting started

1. Check which meter data you actually get (resolution, latency, legal access path) and decide target end-uses accordingly (15 min: heating/EV/base load only).
2. Install NILMTK and nilmtk-contrib, convert a public dataset (UK-DALE or REFIT), and reproduce a baseline (seq2point, DAE) to understand achievable accuracy [2][7].
3. Run a consented pilot of 50-200 homes with sub-meters or smart plugs for ground truth.
4. Build the insight layer first with simple rules (standby share, heating vs. outdoor temperature) and add the LLM explainer only on top of verified numbers.
5. Measure savings with a control group before scaling; complete the DPIA before launch.

Sources

1. CORDIS — H2020 project Eco-Bot (ID 767625), personalised virtual energy assistant with NILM — https://cordis.europa.eu/project/id/767625
2. NILMTK — Non-Intrusive Load Monitoring Toolkit (Apache-2.0) — https://github.com/nilmtk/nilmtk
3. Bidgely — disaggregation / UtilityAI (vendor claims) — https://www.bidgely.com/
4. Octopus Energy — Agile Octopus and Intelligent Octopus Go smart tariffs — https://octopus.energy/smart/
5. Gesetz über den Messstellenbetrieb (MsbG), table of contents — https://www.gesetze-im-internet.de/messbg/
6. Tibber Deutschland — dynamic tariff, Pulse reader, app — https://tibber.com/de
7. nilmtk-contrib — PyTorch NILM models (Apache-2.0) — https://github.com/nilmtk/nilmtk-contrib
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
9. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj