Energy load & renewable generation forecasting

Every party in the power system needs to know, at 15-minute resolution, how much electricity will be consumed and how much wind and solar will be fed in — for the next hour, the next day and the next week. Errors cost balancing energy, redispatch and missed trading value. AI forecasting combines weather model output with plant and meter history in gradient-boosted or deep time-series models, and since 2025 increasingly with zero-shot time-series foundation models and AI weather models. It delivers probabilistic forecasts that feed trading, storage dispatch and grid operation.

Problem & value

• Who has the problem: TSOs and DSOs (grid load, vertical load, renewable feed-in for redispatch), utilities and balancing-responsible parties (portfolio load for procurement), direct marketers and VPP operators (wind/PV for day-ahead/intraday trading), industrial sites (peak shaving, procurement, on-site PV), building operators and municipal utilities (Stadtwerke), and prosumers with PV, battery, heat pump and EV.
• Today: standard load profiles (SLP), simple regressions on temperature, "same day last week" baselines, or a forecast bought as part of the trading contract. These struggle with fast-growing behind-the-meter PV, heat pumps, EV charging and weather-driven demand changes.
• Why AI helps: ML models learn site-specific, non-linear relationships between weather, calendar and load or generation. They can be retrained automatically and give quantiles, not just a single value.
• Measured value: Google DeepMind reported that ML forecasts of wind output 36 hours ahead raised the value of its 700 MW US wind fleet by roughly 20% compared with no time-based commitments [1]. German forecast vendors say they cover very large fleets: emsys/energy & meteo systems reports forecasts for ~400 GW of wind and ~400 GW of solar and more than 100 grid-operator customers. enercast reports 240 GW of capacity in 30 countries [2][3]. The open Quartz solar forecast from Open Climate Fix is used daily by Great Britain's system operator NESO [4].
• Regulatory pull (DE): Redispatch 2.0 (§§ 13, 13a, 14 EnWG, in force since 1 Oct 2021) brought renewable and CHP plants from 100 kW into redispatch. The regime "relies more on planning data and forecasts", so grid operators and plant operators need forecasts at plant level [5].

Variants

• Grid load forecasting (TSO/DSO): control-area, substation or transformer load, vertical load, losses. Feeds congestion management, which is covered in OPT-022.
• Renewable generation forecasting: wind farm and PV park feed-in, day-ahead and intraday. Includes nowcasting (0–6 h) from satellite cloud motion and live measurements, and ramp-event alerts.
• Portfolio / supplier load forecasting: aggregated RLM and SLP customers for procurement and balancing-group management. Trading on the forecast is covered in OPT-039, prices in PRED-008.
• Industrial & commercial sites: site load plus on-site PV for peak-load management, procurement and ISO 50001 energy management. Building-level control is covered in OPT-007.
• Heat and gas: district-heating load and gas demand for utilities (temperature-driven, similar stack).
• EV charging demand: depot and charging-hub load. Charging control is covered in OPT-021.
• Private / prosumer: PV yield forecast for a home system (e.g. Solcast hobbyist API, open-source Quartz, Open-Meteo-based tools in Home Assistant). These feed home energy management (OPT-032).
• Public sector: municipal heat planning and public-building load monitoring.

How it works

1. data intake (SCADA / smart meter / RLM / inverter APIs, 15-min)
2. cleaning (gaps, curtailment & outage flags, DST, capacity changes)
3. weather features (NWP from DWD ICON, ECMWF IFS/AIFS; irradiance on plane of array via pvlib; hub-height wind; optional satellite nowcast)
4. model: gradient boosting / deep TS model / TS foundation model (+ physical PV/wind power curve)
5. probabilistic output (quantiles, ensembles) + portfolio aggregation / reconciliation
6. delivery: schedule files, API, EMS/trading/grid systems
7. monitoring: error by horizon (MAE/nMAE, RMSE, pinball loss), drift, automatic retraining

• Key AI components: (1) weather input. Forecast quality is limited mainly by the NWP, and using several NWPs is standard. (2) A statistical/ML layer that maps weather to power or load per site. (3) Physical models such as PV clear-sky and transposition (pvlib [6]) and wind power curves, used as features or fallback for new plants without history.
• Human-in-the-loop: forecasters/traders review ramp alerts and adjust for known events (outages, curtailment, industrial shutdowns, holidays). Grid operators keep forecasts as decision support. A human or a certified control system makes the switching decisions.

Data preparation

Needed are metered load and generation data (smart meters, RLM 15-minute profiles, inverter and SCADA data), plant master data and weather forecasts (DWD ICON, ECMWF). In an SMB the load data sits with the grid operator portal or supplier, in the energy management system, in inverter cloud portals and in Excel exports; master data is often only in planning documents or on paper. Typical gaps: missing or implausible values, daylight-saving time errors, curtailment or outages not flagged, changes of plant configuration without records, differing time resolutions and several portals with separate logins.

Preparation steps: (1) inventory meters and data sources and request 2-3 years of 15-minute data, (2) automate export via APIs or scheduled downloads, (3) clean gaps and outliers and flag outages and curtailments, (4) build a plant master data table (kWp, tilt, azimuth, location), (5) add calendar features and weather forecasts as archived forecasts (not only observations), (6) use pandas and a time-series database for storage. Useful PREP cases: PREP-005, PREP-011, PREP-001. Also DATA-008.

Rough share of effort: 30-40 percent. Smart meter data of households is personal data, aggregate or pseudonymise it (PREP-022). See GUIDE-003.

Implementation paths

Which path for whom

• SMB — Path A: Forecasting services from energy suppliers, direct marketers and energy management systems are mature and include weather feeds.
• Enterprise — Path B: Utilities and large consumers need integration with own SCADA, metering and trading systems on cloud APIs and weather services.
• Developer — Path C: Open-source models (gradient boosting, open weather and time-series foundation models) run on HW1 with DWD open data.
• Private — Path A: Home energy management systems and inverter apps give PV and load forecasts without own set-up.
• Public — Path C: Municipal utilities and grid operators keep metering and plant data on sovereign, self-operated infrastructure.

Path A — Buy (off-the-shelf) — recommended for most SMBs and utilities

• Specialised forecast providers: energy & meteo systems / emsys renewables (Oldenburg, DE — wind, solar, grid load, VPP, Redispatch 2.0 platform) [2], enercast (Kassel, DE; German data centres) [3], meteocontrol (Augsburg, DE — PV monitoring and forecasts), Meteomatics (CH), Volue (NO), Solcast (now part of DNV) for irradiance and PV power APIs [7].
• Embedded in other platforms: trading/VPP platforms and direct marketers (Next Kraftwerke, Statkraft, Entelios and similar) usually include forecasts. Energy-management systems for industry and buildings offer load forecasts. Home energy managers (e.g. SMA, Fronius, evcc) include PV forecasts.
• Choose A when: forecasting is not your core competence, you need a contractually guaranteed SLA feed for trading or Redispatch 2.0, or you have fewer than a few hundred sites. Check: EU hosting, nMAE reporting by horizon, quantile output, API and schedule formats, and handling of curtailment.

Path B — Build on cloud APIs

• Weather/irradiance APIs: Open-Meteo (ICON, ECMWF IFS and AIFS, GTI/DNI/DHI at 15-min resolution in Central Europe; commercial licence available) [8], Solcast, Meteomatics, DWD Open Data (free).
• Managed forecasting services: Google BigQuery ML with TimesFM [9], Amazon SageMaker/AutoGluon with Chronos-2 [10], Azure ML AutoML forecasting, Databricks. All run in EU regions (Frankfurt, etc.).
• Integration: ingest meter data (e.g. via EDIFACT MSCONS from the meter operator), store it in a time-series database, run forecasts on a schedule, and push them to the EMS or trading system. LLMs play only a supporting role here, e.g. for writing explanations of forecast deviations.

Path C — Self-host (open source)

• Models: gradient boosting (LightGBM, XGBoost, CatBoost) remains a strong baseline. Deep models via Nixtla NeuralForecast (NHITS, TFT, PatchTST) [11], Darts or skforecast. Zero-shot/fine-tunable foundation models: Chronos-2 (120M, Apache-2.0, supports covariates such as weather) [10], TimesFM 2.5 (Apache-2.0 weights; note: TimesFM 3.0 weights are non-commercial) [9], Moirai.
• Solar-specific: pvlib (BSD-3) [6], Open Climate Fix Quartz solar forecast (MIT, site-level 0–48 h from GFS/ICON/ECMWF) [4], PVNet (satellite + NWP fusion) [12].
• Weather: self-hosted Open-Meteo, DWD ICON open data, ECMWF open data incl. AIFS (CC BY 4.0) [13].
• Stack: TimescaleDB/InfluxDB, Kafka or MQTT for live data, Airflow/Dagster, MLflow, Grafana.
• Hardware: HW1. GBMs and the small foundation models (Chronos-2 small or base) run on CPU servers. A GPU (HW2) is useful only for training deep models on thousands of series or for satellite nowcasting.

Path D — Custom training

Worth it for TSOs/DSOs, large portfolio operators and forecast vendors. Typical work: global deep models across thousands of sites, satellite-based nowcasting, hierarchical reconciliation from site up to control area, or fine-tuning a foundation model on your own fleet. Needs at least 1–2 years of 15-min history per site, clean curtailment and availability flags, and archived historical NWP runs (not reanalysis) so that training matches what the model will see at inference.

State of the art (2026)

• Tabular GBM + multiple NWPs is still the workhorse in production. Most accuracy gains come from better weather inputs, NWP blending and clean data, not from model architecture.
• Time-series foundation models: Chronos-2 (Amazon, Oct 2025) reports the best results among pretrained models on fev-bench, GIFT-Eval and Chronos Benchmark II. Its largest gains are on tasks with covariates, and the paper includes an energy case study [10][14]. TimesFM 3.0 (Aug 2026) adds native covariates, but its weights are non-commercial outside Google Cloud [9]. In practice: a strong zero-shot baseline for new sites and load series, still validated against a tuned GBM.
• AI weather models: ECMWF has run AIFS operationally since Feb 2025 (ensemble since Jul 2025, ENS v2 since May 2026) under an open licence [13]. ML NWP is now an input option for renewable forecasts.
• Nowcasting: satellite + NWP fusion models (e.g. PVNet) for 0–8 h PV [12].
• Limits & failure modes: snow on panels, fog, Saharan dust, icing on turbines, unreported curtailment and outages, sudden load changes (COVID-type shocks, plant shutdowns), and fast growth of behind-the-meter PV that hides true load. Probabilistic calibration in extreme ramps is still weak.

Shared resources & combinations

• Weather pipeline (scientific-fm + NWP ingestion) shared with PRED-017 (weather impact), PRED-027 (flood forecasting) and PRED-013 (yield prediction). One NWP store can serve all of them.
• Same time-series/tabular stack (time-series + tabular-ml, HW1 CPU) as PRED-001 demand forecasting, PRED-008 price forecasting, PRED-011 staffing and PRED-016 cash flow.
• Downstream optimisation: OPT-039 power trading, OPT-011 battery dispatch, OPT-022 congestion management/redispatch, OPT-032 home energy management, OPT-007 HVAC, OPT-021 EV smart charging.
• Same smart-meter data as PRED-031 (NILM), ANOM-008 (energy theft & grid anomalies), PRED-048 (building energy performance) and PRED-026 (battery health).
• Streaming infrastructure (MQTT/Kafka, time-series DB) shared with PRED-006 predictive maintenance.

Effort, cost & KPIs

• Path A: days to weeks. Priced per site, per MW or per forecast feed.
• Path B/C: 1–3 months for a production pipeline for a portfolio. Main cost drivers are data cleaning, NWP data licences and operations, not compute.
• Path D: quarters, with a dedicated data-science/meteorology team.
• KPIs: nMAE/nRMSE by horizon (as % of installed capacity for generation, % of peak for load), pinball loss / CRPS for quantiles, balancing-energy cost (€/MWh), imbalance volume, peak-charge savings, ramp-event hit rate, and forecast availability (SLA).

People & change

Involve management (investment and risk appetite) and the energy or facility manager as process owner. End users are plant operators, energy managers and dispatchers who act on the forecast. IT and IT security are needed for access to SCADA and metering systems, which count as critical infrastructure for utilities (GUIDE-013). Metering data owners and suppliers (grid operator, direct marketer, weather service) must agree on data access and formats.

Works council: no in the typical case, because the forecast concerns plants and grid load and does not monitor employees. It becomes relevant only if the data is combined with operator or shift performance; §90(1) no. 3 BetrVG still calls for informing an existing works council early when work processes change.

Data protection: plant-level data is not personal, but household smart meter data is. Then involve the DPO, check the legal basis and a DPIA, and pseudonymise (GUIDE-005). Support AI literacy (Art. 4 AI Act) for dispatchers on forecast uncertainty and typical errors, such as fog or storm events (GUIDE-006). Pilot on one plant or one site and compare with the current method over several weeks. Typical concern: trust in a black box, addressed with confidence bands and a manual override. People keep the final dispatch decision. See GUIDE-009.

Risks, regulation & compliance

• EU AI Act: usually minimal risk. Annex III point 2 covers AI used "as safety components in the management and operation of … the supply of water, gas, heating or electricity" [15]. A forecast used for trading, procurement or planning is not a safety component. Re-check classification if forecasts directly drive automated protective actions in grid control, which could be high-risk. Art. 4 AI literacy applies.
• GDPR: household smart-meter load curves are personal data and can reveal habits. Aggregate before forecasting where possible and follow the German MsbG rules on purpose limitation for metering data. Commercial and grid-level data are not personal data.
• Sector law: EnWG/Redispatch 2.0 data exchange, REMIT (inside information when trading on forecasts), NIS2 / KRITIS duties for grid operators' IT systems, and the EU Network Code on Cybersecurity for electricity. Forecast APIs must not open a path into OT networks.
• Risks & mitigations: NWP outage → keep a fallback (persistence or a second provider); model drift when portfolios change → automatic retraining and capacity tracking; overconfident quantiles → calibration checks.

Getting started

1. Collect 1–2 years of 15-min load or feed-in history with plant master data and curtailment flags.
2. Establish baselines (persistence, last-week, SLP or a vendor feed) and measure nMAE by horizon.
3. Add NWP features (Open-Meteo or DWD ICON) and train a LightGBM model. Compare it zero-shot with Chronos-2 and the Quartz solar forecast for PV.
4. Ask 2–3 vendors from Path A for a blind trial on the same sites and period.
5. Put the best forecast into your EMS, trading or storage dispatch and track balancing cost monthly.

Sources

1. Google DeepMind — Machine learning can boost the value of wind energy — https://deepmind.google/discover/blog/machine-learning-can-boost-the-value-of-wind-energy/
2. emsys renewables / energy & meteo systems — company and forecasting offering — https://www.emsys-renewables.com/
3. enercast — wind and solar power forecasts — https://www.enercast.de/
4. Open Climate Fix — Quartz solar forecast (open source) — https://github.com/openclimatefix/open-source-quartz-solar-forecast and https://www.openclimatefix.org/
5. BDEW — Redispatch 2.0 — https://www.bdew.de/energie/redispatch-20/
6. pvlib-python — https://github.com/pvlib/pvlib-python
7. Solcast API documentation — https://docs.solcast.com.au/
8. Open-Meteo weather API documentation — https://open-meteo.com/en/docs
9. Google Research — TimesFM — https://github.com/google-research/timesfm
10. Amazon Science — Chronos forecasting (Chronos-2) — https://github.com/amazon-science/chronos-forecasting
11. Nixtla NeuralForecast — https://nixtlaverse.nixtla.io/neuralforecast/docs/getting-started/introduction.html
12. Open Climate Fix — PVNet — https://github.com/openclimatefix/PVNet
13. ECMWF — AIFS machine learning data — https://www.ecmwf.int/en/forecasts/dataset/aifs-machine-learning-data
14. Amazon — Chronos-2: From Univariate to Universal Forecasting (arXiv:2510.15821) — https://arxiv.org/abs/2510.15821
15. EU AI Act Annex III — https://artificialintelligenceact.eu/annex/3/
16. Bundesnetzagentur — SMARD electricity market data (generation, consumption, forecasts) — https://www.smard.de/en