Add US power forecasting open ecosystem guide

2026-07-19 06:23:48 -04:00 · 2026-07-19 06:23:48 -04:00 · bd3ed1e101
commit bd3ed1e101
parent d4382f7b6c
2 changed files with 380 additions and 0 deletions
--- a/README.md
+++ b/README.md
@ -53,6 +53,7 @@ Market-neutral educational material — no venture-specific analysis.
 |---|---|---|---|
 | Field guide 实务指南 | [field_guide_en.md](us_market/field_guide_en.md) | EN only | Bidding, trading, and the impact of AI — consolidated working discussions, July 2026. 竞价、交易与 AI 影响综合笔记。 |
 | Economists survey 经济学家综述 | [economists_survey_en.md](us_market/economists_survey_en.md) | [economists_survey_zh.md](us_market/economists_survey_zh.md) | How economists shaped US electricity markets — theory, practice, milestones. 经济学家如何塑造美国电力市场——理论、实践与里程碑。 |
 | Forecasting open ecosystem 预测开放生态 | [forecasting_open_ecosystem_en.md](us_market/forecasting_open_ecosystem_en.md) | EN only | Open US wholesale-market datasets, models, benchmarks, structural tools, and evaluation design for price, load, renewable, and capacity forecasting. 美国批发电力市场预测的开放数据、模型、基准、结构化工具与评估设计。 |
 ### `pjm/` — US market 美国市场
--- a/us_market/forecasting_open_ecosystem_en.md
+++ b/us_market/forecasting_open_ecosystem_en.md
@ -0,0 +1,379 @@
 # Open Models, Datasets, and Benchmarks for Forecasting in US Wholesale Power Markets
 **Scope:** Day-ahead (DA) and real-time (RT) prices, load/demand, renewable generation, and capacity in the seven US ISO/RTO markets (CAISO, ERCOT, PJM, MISO, SPP, NYISO, ISO-NE).
 **Status:** Compiled and verified July 2026. Everything listed is free/open unless flagged otherwise. Web-verified items are marked ✅; items from general domain knowledge that you should re-confirm before relying on are marked ◻︎.
 ---
 ## 1. How to read this report
 Forecasting in US wholesale markets breaks into four related but distinct problems, each with its own data and benchmark ecosystem:
 | Target | Typical granularity | Open-benchmark maturity |
 |---|---|---|
 | DA prices (LMPs) | Hourly, nodal/zonal/hub | Moderate (one canonical academic benchmark, aging) |
 | RT prices / DA–RT spread | 5-min–hourly, nodal | Weak (no standard benchmark) |
 | Load | 5-min–hourly, BA/zone | Strong (GEFCom + government data) |
 | Wind/solar (price drivers) | 5-min–hourly, site/zone | Strong (NREL/ARPA-E datasets) |
 | Capacity (installed & pipeline) | Monthly/annual, unit/project | Strong data, no "forecasting benchmark" per se |
 A key structural fact: **there is no single canonical benchmark for US price forecasting the way ImageNet exists for vision.** The academic standard (Section 4) uses one aging PJM zone; practitioners assemble bespoke datasets from ISO portals using the tools in Section 2. Treat Section 2 as the foundation and Sections 4–10 as what's built on top of it.
 ---
 ## 2. The data-access layer (start here)
 ### 2.1 gridstatus ✅
 **Repo:** https://github.com/gridstatus/gridstatus • **Docs:** https://opensource.gridstatus.io
 The de facto standard open-source Python library for US grid data. Covers CAISO, ERCOT, PJM, MISO, SPP, NYISO, ISO-NE (plus Canada's IESO and AESO, and EIA), with a standardized API across sources. Available datasets (varying by ISO): load, load forecasts, fuel mix, DA and RT LMPs, ancillary service prices, storage, solar/wind forecasts, generation outages, and interconnection queues.
 Why it matters for forecasting specifically:
 - It exposes the ISOs' **own published forecasts** (load, wind, solar) — these are both model features and the operational baselines any new model must beat.
 - It normalizes ISO quirks (e.g., it flips NYISO's congestion sign convention to match other ISOs; handles ERCOT's API migration).
 - Caveats: history depth varies by ISO and dataset; some ISO endpoints require free registration/API keys (PJM Data Miner, ERCOT public API, ISO-NE web services); nodal LMP pulls are large (PJM has >10,000 pricing nodes — a single day of hourly RT nodal data is ~300k rows).
 The same team runs a hosted API (gridstatus.io) with a free tier and a data catalog — convenient, but the open-source library against ISO portals is fully free.
 ### 2.2 Raw ISO portals (free, authoritative) ◻︎
 All seven ISOs publish DA/RT prices, load, and forecasts publicly. These are the ground-truth sources gridstatus wraps:
 | ISO | Portal | Notes |
 |---|---|---|
 | CAISO | OASIS (oasis.caiso.com) | DA hourly + RT 15-min/5-min LMPs at all nodes; no registration needed ✅ |
 | ERCOT | MIS / ERCOT Public API (data.ercot.com) | SPPs by settlement point; 60-day SCED/DAM disclosure files are uniquely rich (bid curves, telemetry, released with 60-day lag) |
 | PJM | Data Miner 2 (dataminer2.pjm.com) | Free API key; deep LMP history, load, forecasts, outages |
 | MISO | Data Exchange / Market Reports (miso.com) | API-based access rolled out mid-2020s; check current endpoints |
 | SPP | Marketplace portal / SPP file server | LMPs, MTLF (load forecast), wind forecasts |
 | NYISO | OASIS + open CSV archives (mis.nyiso.com) | Among the easiest bulk downloads; zonal + nodal, decades of history |
 | ISO-NE | Web Services API / ISO Express | Free registration; hourly DA/RT LMPs, load, forecasts |
 Two ISO-specific gems worth knowing: ERCOT's **60-day disclosure** data (actual resource-level offers and dispatch, the only place in the US you can see near-complete bid stacks, lagged 60 days) and PJM/MISO **generation outage** reports, which are first-order features for RT price models.
 ### 2.3 EIA (federal) ✅
 - **EIA API v2** (api.eia.gov, free key): hourly demand, DA demand forecast, net generation by fuel, and interchange for every US balancing authority via **Form EIA-930** (the "Hourly Electric Grid Monitor"), from July 2015 onward. Also fuel prices, monthly generation, and much more.
 - **Cleaned EIA-930 demand** (Ruggles, Farnham et al.): screened + MICE-imputed hourly demand for the 56 demand-reporting BAs in CONUS. Methods paper: *Scientific Data* 7, 155 (2020), doi:10.1038/s41597-020-0483-x. Data versioned on Zenodo (doi:10.5281/zenodo.3517196) and GitHub (`truggles/EIA_Cleaned_Hourly_Electricity_Demand_Data`). The January 2025 release covers 2020-01-01 → 2025-01-01 including BA subregions. This is the standard cleaned load dataset in academic work — raw EIA-930 contains missing/anomalous values you must otherwise handle yourself.
 - Note EIA-930's known-issues guide: BA-reported data has quirks (timezone conventions, revisions); PUDL (below) retains only UTC timestamps for this reason.
 ### 2.4 PUDL (Catalyst Cooperative) ✅
 **Docs:** https://catalystcoop-pudl.readthedocs.io • **Repo:** catalyst-cooperative/pudl
 Open data pipeline that cleans and links **EIA-860/860M/861/923, EIA-930, FERC Form 1, and EPA CEMS** into analysis-ready tables (Parquet/SQLite, also on Datasette and Kaggle). This is the go-to source for:
 - **Capacity:** unit-level generator characteristics, capacities, fuel, planned additions and retirements (EIA-860/860M).
 - **Generation & costs:** monthly unit generation and fuel costs (EIA-923), utility financials (FERC 1).
 - **Hourly unit-level operation:** EPA CEMS hourly gross load and emissions for fossil units — effectively an open proxy for historical unit commitment, invaluable for structural price models.
 - EIA-930 demand/forecast/interchange tables (all but hourly CO₂ available through PUDL).
 ### 2.5 Weather (the other half of every model) ◻︎
 - **NOAA HRRR** (3-km, hourly, CONUS; free on AWS/Google/Azure via NODD) — the workhorse for operational US load/renewables features; `herbie` (Python) is the standard access library.
 - **GFS/ECMWF open data** for longer horizons; **ERA5** (Copernicus) for reanalysis/backtesting.
 - **NOAA ISD/ASOS** station observations for actuals.
 - **Open-Meteo** — free API conveniently exposing HRRR/GFS/ECMWF forecasts and historical archives.
 - Caution for backtesting: use **archived forecasts** (as-of data), not reanalysis, for features that would have been available at prediction time. HRRR archives on AWS go back to ~2014.
 ---
 ## 3. What "the benchmark to beat" means operationally
 Before any ML: every ISO publishes its **own DA load forecast**, and several publish wind/solar forecasts (all retrievable via gridstatus). For prices, the DA LMP itself is the market's consensus forecast of RT conditions, and simple benchmarks (persistence, naive-with-seasonality, DA price as RT predictor) are strong. Any serious study should report skill relative to these, not just absolute error.
 ---
 ## 4. Price forecasting (DA and RT)
 ### 4.1 The canonical academic benchmark: Lago et al. / epftoolbox ✅
 **Paper:** Lago, Marcjasz, De Schutter, Weron, "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," *Applied Energy* 293:116983 (2021).
 **Code/data:** https://github.com/jeslago/epftoolbox (AGPL-3.0) • docs: epftoolbox.readthedocs.io
 What it provides:
 - **Five open datasets, six years each** of hourly DA prices + two exogenous forecast series per market: EPEX-BE, EPEX-FR, EPEX-DE, Nord Pool, and **PJM (ComEd zone)** — the one US market in the set (data ends 2018).
 - **Two open reference models**: LEAR (LASSO-estimated autoregressive) and a hyperparameter-optimized DNN, implementable on any DA market without expert tuning.
 - **Evaluation infrastructure**: MAE/sMAPE/MASE-style metrics plus **Diebold-Mariano and Giacomini-White** statistical tests, and pre-computed state-of-the-art forecasts so you can compare without retraining anything.
 - A widely cited **best-practices checklist** (long rolling test windows, open data, statistical testing, strong simple baselines) — arguably the paper's most durable contribution.
 Limitations for US use: only one US zone, data ends in 2018 (pre-dating today's storage/renewables-driven price regimes and events like Winter Storm Uri), zonal not nodal, and its exogenous features are minimal. It remains the field's shared yardstick; hundreds of follow-up papers (single- vs multi-output NNs, distributional networks, XAI studies, transformer comparisons) evaluate on it.
 ### 4.2 Open probabilistic-EPF methods (built on the same benchmark) ◻︎
 The Weron group (Wrocław) and collaborators maintain open implementations that are the standard probabilistic baselines: **Quantile Regression Averaging (QRA)** and variants, **distributional deep networks** (e.g., DDNN with Normal/JSU output distributions — Marcjasz et al. 2023), **Isotonic Distributional Regression**, **conformal-prediction** approaches, and the **Schaake shuffle** for multivariate (24-hour joint) probabilistic forecasts. Most have code on GitHub keyed to the epftoolbox datasets. Metrics: pinball loss, CRPS, coverage; tests: DM/GW.
 ### 4.3 US-specific and RT-price work: no standard benchmark ✅ (the gap is verified)
 - **RT prices and the DA–RT ("DART") spread** are studied mostly in the virtual-bidding/trading literature (e.g., LSTM/seq2seq nodal DART forecasting, spike-classification papers such as "Forecasting electricity DART spikes"). Papers overwhelmingly use self-assembled ISO data; code is sometimes released, standardized datasets essentially never.
 - Recent US DA studies (e.g., ISO-NE forecasting with public ISO data + published demand forecasts and generation-capacity data, arXiv:2405.14893) illustrate the pattern: public data, bespoke splits.
 - Scattered open repos exist (e.g., CAISO LMP ML forecasts with paired load-weather datasets on GitHub) but are individual projects, not maintained benchmarks.
 **Practical implication:** for US nodal/RT work you will build your own benchmark from Section 2 sources. Standard recipe: DA + RT LMPs (target), ISO load/wind/solar forecasts, generation and transmission outages, net-load features, gas prices (Section 11 caveat), calendar features; rolling-origin evaluation over ≥1 year including at least one summer and one winter.
 ### 4.4 Open models for price forecasting ◻︎ (tiered)
 Only the first tier below is actually price-specific; the rest are general tools applied to prices.
 **Tier 1 — Purpose-built open EPF reference models.** The epftoolbox suite (Section 4.1) is the only widely accepted open model set built specifically for DA price forecasting: **LEAR** (LASSO-estimated autoregressive) and a hyperparameter-optimized **DNN**, both designed to run on any DA market without expert tuning. These are the baselines every new EPF paper is expected to beat, and the toolbox ships pre-computed forecasts so comparisons need no retraining. Practical note: LEAR is famously hard to beat cost-adjusted — a well-tuned LASSO on hourly price lags plus load/renewables forecasts captures most of the achievable skill.
 **Tier 2 — Open probabilistic EPF methods** (research code, largely from the Weron/Wrocław group): **Quantile Regression Averaging (QRA)** and variants, **distributional deep networks** (Normal / Johnson's SU outputs — Marcjasz et al. 2023), **isotonic distributional regression**, **conformal prediction** wrappers, and the **Schaake shuffle** for jointly coherent 24-hour probabilistic paths. Published with code, typically evaluated on epftoolbox data with pinball loss/CRPS; research artifacts rather than maintained packages, so expect assembly.
 **Tier 3 — General open frameworks applied to prices** (what most practitioners actually use):
 - **Gradient-boosted trees** (XGBoost/LightGBM) — the workhorse for tabular price features (load/renewable forecast lags, outages, gas proxies, calendar); competitive with or better than deep nets on spiky US nodal data in many settings.
 - **Nixtla neuralforecast** (N-BEATS, N-HiTS, TFT, PatchTST, TimesNet, DeepAR-style) and **statsforecast** (ARIMA/ETS/Theta baselines); **GluonTS** and **Darts** as alternative frameworks.
 **Tier 4 — Open-weight time-series foundation models:** Amazon **Chronos / Chronos-Bolt / Chronos-2**, Google **TimesFM (2.x)**, Salesforce **Moirai / Moirai-2**, **TiRex**, Datadog **Toto**, **Lag-Llama**. The honest read of the 2025–2026 literature: respectable zero-shot on load, but prices — spikes, negative values, regime shifts — remain a weak spot, and vanilla zero-shot use generally does not beat a tuned LEAR/GBM with proper exogenous features. Leakage is a live concern on public benchmarks (Section 8). They get interesting with fine-tuning or covariate-aware use (Chronos-2 accepts covariates natively).
 **Tier 5 — Structural / simulation-based models.** Open production-cost and OPF stacks — **PyPSA / PyPSA-USA**, NREL **Sienna** (PowerSimulations.jl), **Egret**, **UnitCommitment.jl** — run against open synthetic grids (RTS-GMLC, Texas 7k, Breakthrough Energy US test system) simulate LMPs from fuel prices, capacities, and topology. This is how you forecast *structural* price changes (entry, retirements, transmission) that purely statistical models cannot see.
 **What does not exist in the open:** a maintained, pretrained, off-the-shelf model for US nodal LMPs. Commercial vendors sell that; the open ecosystem composes it — typically GBM or a distributional NN per node/hub, epftoolbox-style evaluation, ISO forecasts as features.
 ---
 ## 5. Load forecasting
 ### 5.1 GEFCom (the classic benchmarks) ◻︎
 The **Global Energy Forecasting Competitions** (Tao Hong et al.) remain the standard academic load benchmarks:
 - **GEFCom2012** — hierarchical zonal load (US utility) + temperature; point forecasting.
 - **GEFCom2014** — established **probabilistic** load (and price/wind/solar) forecasting with pinball loss; the reference dataset for quantile load models.
 - **GEFCom2017** — hierarchical probabilistic load on ISO-NE zones with rich weather.
 Data circulates via the competition papers (*IJF* 2014/2016/2019) and Dr. Hong's site (drhongtao.com). Age is the main limitation; they predate electrification/data-center load growth.
 ### 5.2 System-level US load ✅
 - **EIA-930** (raw via EIA API, cleaned via Ruggles/Zenodo, tabulated via PUDL) — hourly demand + the BA's own DA forecast for every US BA since mid-2015. Forecasting the demand of specific BAs with the BA's published forecast as the baseline is a well-posed, fully open task.
 - **ISO zonal loads + forecasts** via gridstatus for finer geography (e.g., PJM zones, NYISO zones).
 - **ARPA-E PERFORM** load actuals + probabilistic forecasts (Section 6) for risk-aware work.
 ### 5.3 Building/feeder level (if you need it) ◻︎
 **BuildingsBench** (NREL): ~900K simulated buildings (Buildings-900K) plus curated real building datasets for short-term load forecasting, with pretrain/eval splits designed for foundation-model work — including pretrained transformer weights for building-level forecasting. Overkill for wholesale-market questions but the standard at that scale.
 ### 5.4 Open models for load forecasting ◻︎ (tiered)
 Load is the most mature target: models are simpler, and the open baselines are genuinely strong.
 **Tier 1 — Classic regression benchmarks.** The canonical open model is Tao Hong's **"vanilla benchmark"** from the GEFCom lineage: multiple linear regression on calendar effects (hour, weekday, month) crossed with polynomial temperature terms (T, T², T³ and their interactions with hour/month). Trivially reproducible, published in full, and still the standard baseline — beating it by a meaningful margin on zonal load is harder than expected, because system load is dominated by weather and seasonality the regression already captures. Documented variants (lagged/exponentially smoothed temperature, weather-station selection, recency effects) are collected in Hong & Fan's open-access probabilistic load forecasting tutorial review (*IJF* 2016).
 **Tier 2 — GAMs, the operational workhorse.** Generalized additive models (smooth splines on temperature, time-of-year, lagged load) are what several system operators actually run. Open tooling: **mgcv** (R) is the standard, with well-published recipes from the French school (Goude, Obst et al.), including online/adaptive versions (Kalman-filtered GAMs, aggregation of experts) that handled the COVID demand shift well. Python-native substitute: **quantile LightGBM** on the same features, with comparable performance.
 **Tier 3 — Probabilistic methods from the GEFCom lineage:** quantile regression on vanilla features, quantile GBM, **temperature-scenario simulation** (feed many historical weather years through a point model to obtain a predictive distribution — probably the highest value-per-line-of-code trick in open load forecasting), and sister-forecast combination. All published in reproducible detail with pinball-loss evaluation.
 **Tier 4 — Deep learning.** Same frameworks as prices (neuralforecast: N-HiTS/TFT/PatchTST; GluonTS: DeepAR; Darts). Deep models help most at finer granularity (nodal/feeder, many related series learned jointly) and matter less at BA/system level where weather regression dominates.
 **Tier 5 — Foundation models: load is their best subject.** System load is smooth, strongly seasonal, and heavily represented in pretraining corpora, so Chronos, TimesFM, Moirai, TiRex et al. do genuinely well zero-shot; several 2024–2026 studies find TimesFM competitive with or better than task-trained models, especially with long contexts. Two caveats: **leakage** (public load datasets sit in the pretraining corpora, so public-benchmark wins overstate real-world skill) and **covariates** (vanilla zero-shot use ignores weather forecasts, where much of the operational skill lives — covariate-capable variants like Chronos-2, or fine-tuning, close that gap).
 **Operational bar:** ISO/BA published DA load forecasts (via gridstatus, or EIA-930 which carries every BA's DA demand forecast alongside actuals) typically run ~1–2% MAPE day-ahead at system level. A model that doesn't beat the published forecast on a proper out-of-sample window isn't adding value — a test that a surprising share of published deep-learning results quietly fails.
 ---
 ## 6. Wind, solar, and the "net load" drivers
 ### 6.1 ARPA-E PERFORM datasets (NREL) ✅ — closest thing to a modern US market forecasting benchmark
 **Portal:** https://data.openei.org/submissions/5772 • **Docs:** https://github.com/PERFORM-Forecasts/documentation • DOI: 10.25984/1891136
 Time-coincident **load, wind, and solar actuals plus deterministic and probabilistic forecasts** at 5-minute resolution for **ERCOT, MISO, NYISO, and SPP**, at site, zone, and BA level. Coverage: ERCOT actuals 2017–2018 with forecasts for 2018; MISO/NYISO/SPP actuals 2018–2019 with forecasts for 2019. Wind/solar profiles include both existing sites and planned sites from interconnection queues (as of 2021). Distributed as HDF5 on AWS S3 (public), timestamps in UTC; NREL technical reports document methods and error metrics.
 This is now a common substrate in the literature for evaluating probabilistic forecasting, conformal calibration, and foundation models on realistic US grid data. Limitations: fixed vintage (2017–2019), synthetic-but-validated actuals for renewables, no prices — you can pair it with historical LMPs from Section 2 at your own risk of misalignment.
 ### 6.2 Resource datasets ◻︎
 - **NREL NSRDB** (solar irradiance, 4-km half-hourly, 1998–present) and **WIND Toolkit** / **WTK-LED** (wind resource) — free via NREL APIs/OEDI.
 - **ISO-published renewable forecasts** (ERCOT STWPF/STPPF, CAISO, SPP, MISO) via gridstatus — operational baselines.
 - **Solar/wind competition datasets**: GEFCom2014 tracks; more recent Kaggle/CityLearn-style sets exist but none is US-wholesale-specific.
 ### 6.3 Open models for renewable output forecasting ◻︎ (tiered)
 Renewables forecasting is the most physics-flavored target. The canonical chain is **weather forecast → power conversion → statistical correction**, with open components at every stage; the right model family depends primarily on horizon.
 **Tier 1 — Physical model chains (NWP → power), the day-ahead workhorse.** Take an NWP forecast (HRRR for CONUS; ECMWF/GFS for longer horizons) and convert to power:
 - **pvlib-python** — the canonical open PV modeling library (NREL/Sandia lineage): irradiance transposition, temperature and inverter models; the backbone of most open solar stacks.
 - **PySAM / SAM** (NREL) — full-plant simulation including tracking, losses, storage.
 - **windpowerlib** and **PyWake** (DTU) — hub-height wind to turbine/farm power with wake modeling, using open turbine/power-curve databases.
 Pure physical chains carry bias (plant metadata is never quite right), so operational systems wrap them in a statistical correction:
 **Tier 2 — Statistical/ML correction and direct ML.** The dominant practical pattern: gradient-boosted trees (LightGBM/XGBoost) mapping NWP features (multi-member, multi-model wind/irradiance at several grid points and heights, plus solar geometry from pvlib) directly to plant or fleet output — the approach that won most GEFCom2014 wind/solar tracks and remains state of the art at hours-to-days horizons. Quantile versions give calibrated probabilistic output. The **analog ensemble** (find historically similar forecast conditions; use the distribution of realized outputs) is a well-published, easily reproduced probabilistic alternative.
 **Tier 3 — Nowcasting (minutes to ~4 hours), a genuinely distinct problem.** NWP is stale at these horizons; skill comes from recent observations and satellite imagery. **Smart persistence** (persist the clear-sky index for solar, not raw power) is the standard baseline and hard to beat inside ~1 hour. **Open Climate Fix** is the notable fully open effort: PVNet forecasts solar generation from satellite imagery + NWP with open code and open weights on Hugging Face, running operationally for Great Britain's system operator — the architecture transfers to US data though pretrained weights are GB-centric. Deep learning on satellite/sky-camera sequences (ConvLSTM/transformer variants) has abundant published code but no dominant packaged model.
 **Tier 4 — AI weather models (the big recent shift).** Open-weights ML weather models — **GraphCast/GenCast** (DeepMind), **Pangu-Weather** (Huawei), **FourCastNet** (NVIDIA), **Aurora** (Microsoft), plus ECMWF's **AIFS** — rival or beat physical NWP on many variables at a fraction of the compute. For renewables they matter mainly as a cheap way to generate large ensembles of the *inputs* (wind fields, radiation), upgrading the probabilistic layer downstream. Radiation and hub-height winds remain weaker than temperature/pressure — validate before swapping them in for HRRR.
 **Practical guidance.** (1) The leakage rule applies doubly: evaluate against archived *forecasts* as issued, never reanalysis-derived weather, or backtests will flatter you. (2) Fleet-level forecasting (what moves prices) is much easier than single-plant — errors decorrelate across sites — so match aggregation level to use case; a mediocre per-plant model can be excellent for ERCOT-wide net load. (3) For wholesale-market purposes the quantity you usually want is **net load** (load − wind − solar) and its ramps, the driver of RT prices: forecast the three components separately with the tools above, combine, and benchmark against the ISO's implied net-load forecast. Baselines to beat: each ISO's published wind/solar forecasts (ERCOT STWPF/STPPF, CAISO, SPP — via gridstatus) and smart persistence at short horizons.
 ---
 ## 7. Capacity: installed base and the pipeline
 "Capacity forecasting" in practice means tracking the installed fleet and projecting additions/retirements. The open data here is excellent even though there's no ML benchmark:
 - **EIA-860 / 860M** ✅ (via EIA or, cleaned, via PUDL): unit-level installed capacity, technology, and **planned additions and retirements** with expected online dates; 860M updates monthly. This is the baseline capacity forecast used across the industry.
 - **LBNL "Queued Up" + interconnection queue dataset** ✅: Berkeley Lab (with interconnection.fyi) compiles project-level interconnection queue data from >50 grid operators (7 ISO/RTOs + ~50 non-ISO BAs, ≈98% of US installed capacity). Latest data file covers requests **through end of 2025** (over 2,060 GW of active generation + storage capacity; gas up 86% YoY to 253 GW, solar 773 GW, storage 749 GW, wind 220 GW; 549 GW already with draft/executed interconnection agreements). Includes a codebook and dozens of summary tables. The accompanying PDF analysis ("Queued Up: 2025 Edition") covers data through 2024. Key modeling fact from the report: historically **only ~13% of queued capacity (2000–2019 requests) reached commercial operation**, and median IR→COD duration now exceeds 4–5 years — so queues need survival-rate modeling, not face-value use. Download: https://emp.lbl.gov/queues
 - **gridstatus interconnection-queue scrapes** ✅ for the live per-ISO queues between annual LBNL releases.
 - **EPA CEMS** (via PUDL) for realized unit-level operation; **EIA-411/NERC ES&D** ◻︎ for reserve-margin/resource-adequacy projections; ISO planning reports (ERCOT CDR, PJM load forecasts, CAISO IRP) ◻︎ for the ISOs' own capacity outlooks.
 ---
 ## 8. Generic time-series benchmarks and foundation models (context you'll bump into)
 - **GIFT-Eval** ✅ (Salesforce; arXiv:2410.10393): the current standard general forecasting benchmark — 97 tasks from ~23–55 datasets, 144K series, 7 domains including Energy, with an explicit pretraining corpus and leaderboard. Recent foundation models (Chronos-2, TimesFM-2.5, TiRex, Toto, Moirai-2) report on it. Its energy data is building/consumption-flavored (UCI Electricity, BDG-2, ETT…), **not** US wholesale prices — so treat GIFT-Eval rank as weak evidence for LMP-forecasting skill.
 - **Leakage warning** ✅: GIFT-Eval's authors note partial pretraining leakage for TimesFM/Chronos/Moirai on common public sets, and load-forecasting benchmark papers disqualify closed-data models (e.g., TimeGPT-1) for unverifiable training data. If you benchmark foundation models on public ISO/EIA data, assume contamination unless the model documents its corpus.
 - Other useful open evaluation stacks: Nixtla's **statsforecast/neuralforecast/utilsforecast**, **GluonTS**, **Darts**, **sktime**, and Monash Forecasting Archive ◻︎.
 ---
 ## 9. Structural and market-simulation models
 Pure time-series models extrapolate from history and therefore struggle with regime changes: new transmission, generator retirements, fuel shocks, major outages, storage and renewable additions, or revised scarcity rules. Open structural tools provide a complementary view:
 - **Prescient** — Python production-cost framework for unit commitment, economic dispatch, and sequential simulation, with RTS-GMLC examples.
 - **Sienna / PowerSimulations.jl** — Julia framework for sequential operations, unit commitment, dispatch, and scenario studies.
 - **PyPSA** — economic dispatch, unit commitment, linear optimal power flow, transmission modeling, and capacity expansion.
 - **QuESt PCM** — Sandia's Python production-cost model with detailed storage representation.
 - **PyPSA-USA** — configurable US transmission, production-cost, capacity-expansion, and regional-scenario model.
 - **PowerGenome** — builds harmonized US generator fleets, load profiles, regional aggregations, and technology assumptions from public data.
 A useful hybrid architecture is:
 1. A structural model estimates system lambda, dispatch, reserves, and congestion state.
 2. A statistical or ML model corrects systematic structural-model error.
 3. A probabilistic layer produces calibrated intervals or scenarios.
 This is most valuable when the forecast period differs materially from the training period. For controlled experiments, use an open synthetic network such as RTS-GMLC; do not imply that synthetic-grid results establish live nodal-market performance.
 ---
 ## 10. Recommended benchmark design
 A credible open US benchmark should separate tasks rather than collapse unlike targets into one score:
 | Task | Horizon | Target |
 |---|---:|---|
 | DA hub or zonal price | Next 24 hours | Hourly LMP, SPP, or LBMP |
 | RT price | 1–24 hours | 5-minute or 15-minute price |
 | DART spread | Next operating day | RT minus DA |
 | Nodal congestion | Next operating day | Congestion component or binding probability |
 | Load | 5 minutes–7 days | System and zonal MW |
 | Available capacity | 1–7 days | Available and forced-outage MW |
 | Renewable availability | 5 minutes–2 days | Wind or solar potential output |
 | Scarcity event | 1–48 hours | Reserve-shortage or price-spike probability |
 ### 10.1 Evaluation protocol
 - Use rolling-origin backtests, never random train/test splits.
 - Enforce strict as-of availability and preserve every forecast vintage, publication time, revision time, and delivery time.
 - Handle market calendars, daylight-saving transitions, and interval-ending conventions explicitly.
 - Report results by season, hour, node liquidity, congestion regime, and normal versus scarcity periods.
 - Include extreme-weather windows rather than allowing average periods to dominate the score.
 - Compare against the ISO's published forecast where one exists, plus naive seasonal and problem-specific baselines.
 ### 10.2 Metrics
 For point forecasts, report MAE, RMSE, MASE, and target-specific peak or ramp error. Avoid MAPE for power prices because prices can be zero, near zero, or negative.
 For probabilistic forecasts, report pinball loss or CRPS, prediction-interval coverage, and calibration. Use energy score when evaluating multivariate scenarios. For scarcity, congestion, and ramp events, add precision, recall, PR-AUC, and a lead-time- or cost-weighted event score.
 Use Diebold-Mariano or Giacomini-White tests, block bootstrap, and regime-specific comparisons where appropriate; a small metric improvement without statistical or economic significance is not enough.
 ### 10.3 Economic evaluation
 Price forecasts should also be tested in a fully specified decision simulation. Include bid timing, position and volume limits, transaction costs, slippage, fees, price caps and floors, settlement rules, risk-adjusted P&L, drawdown, and tail performance. Keep the forecasting model comparison separate from the trading-policy comparison so improvements are attributable.
 ### 10.4 Modeling invariants
 1. **Preserve forecast vintages.** Never train with realized or revised weather when the operational model would have seen an earlier forecast.
 2. **Separate ordinary and extreme regimes.** Report both; do not let common hours hide failure on high-value events.
 3. **Forecast components where useful.** Decompose LMP into energy, congestion, and loss components; renewable output into potential, availability, and curtailment; net load into load, wind, and solar.
 4. **Use global models carefully.** Cross-node or cross-market training can help sparse series but requires identifiers, geography, market structure, and per-series reporting.
 5. **Evaluate operational usefulness.** Statistical accuracy, calibration, latency, stability, and economic value are distinct requirements.
 ---
 ## 11. Known gaps and paywalls (so you don't discover them the hard way)
 1. **No maintained modern US nodal DA/RT price benchmark** with aligned, as-of-time features. Everyone rebuilds; results across papers are rarely comparable. (Verified gap; also called out in the EPF literature itself.)
 2. **Natural gas hub prices** — the single most important fuel driver — are the binding data constraint. Henry Hub (EIA, free, daily) is insufficient for ERCOT/PJM/New England, which price off **Waha, TETCO M3, Algonquin Citygates**, etc.; those daily indices are commercial (Platts, NGI, ICE). Partial free workarounds: EIA weekly/monthly regional prices, futures settlements, and lagged proxies — all materially degrade winter price models in gas-constrained regions. ◻︎
 3. **As-of-time weather forecasts**: reanalysis leakage is the most common silent bug in published load/price models; use archived HRRR/GFS.
 4. **Network models/topology** are confidential (CEII); nodal congestion forecasting must rely on learned patterns or synthetic grids (Section 9: RTS-GMLC, Texas A&M synthetic cases like the Texas 7k, Breakthrough Energy's US test system, PGLib-OPF, with open tooling in PyPSA-USA, NREL Sienna, Egret) ◻︎.
 5. **ISO data licensing**: data is free to access but some ISOs restrict redistribution — one more reason public benchmarks are scarce. Check each ISO's terms before republishing datasets. ◻︎
 ---
 ## 12. Recommended starting stacks (by problem)
 **DA zonal/hub price forecasting (research-grade, comparable):**
 epftoolbox (models + tests + PJM/ComEd set) for methodology → rebuild a modern dataset via gridstatus (DA LMPs, ISO load + renewable forecasts) for 2021→present → report rMAE vs naive + LEAR + DNN, DM/GW tests, pinball/CRPS if probabilistic.
 **RT nodal / DART spread:**
 gridstatus RT+DA LMPs at chosen nodes; features: ISO forecasts, outage reports, net-load ramps, gas proxy, DA prices; two-stage (spike classifier + magnitude regressor) is the common open pattern; evaluate economically (trading P&L of virtual bids) as well as statistically.
 **Load:**
 EIA-930 (cleaned) or ISO zonal loads; baseline = the BA/ISO published forecast; GEFCom2014-style quantile evaluation; HRRR weather features via herbie/Open-Meteo.
 **Wind/solar & risk-aware operations:**
 ARPA-E PERFORM as the benchmark substrate; NSRDB/WIND Toolkit for resource; compare against ISO operational forecasts.
 **Capacity outlooks:**
 EIA-860M (planned units) + LBNL queue data with historical completion rates by region/technology/queue-vintage as survival priors; PUDL for the installed-fleet backbone.
 ---
 ## 13. Link index
 | Resource | URL |
 |---|---|
 | gridstatus (library) | https://github.com/gridstatus/gridstatus |
 | epftoolbox (EPF benchmark) | https://github.com/jeslago/epftoolbox |
 | Lago et al. 2021 (open access) | https://doi.org/10.1016/j.apenergy.2021.116983 |
 | EIA API / Hourly Grid Monitor | https://www.eia.gov/opendata/ |
 | Cleaned EIA-930 demand | https://github.com/truggles/EIA_Cleaned_Hourly_Electricity_Demand_Data |
 | PUDL | https://catalystcoop-pudl.readthedocs.io |
 | ARPA-E PERFORM datasets | https://data.openei.org/submissions/5772 |
 | PERFORM documentation | https://github.com/PERFORM-Forecasts/documentation |
 | LBNL Queued Up / queue data | https://emp.lbl.gov/queues |
 | NREL NSRDB | https://nsrdb.nrel.gov |
 | NREL WIND Toolkit | https://www.nrel.gov/grid/wind-toolkit |
 | GEFCom archives | http://www.drhongtao.com/gefcom |
 | GIFT-Eval | https://arxiv.org/abs/2410.10393 |
 | Chronos | https://github.com/amazon-science/chronos-forecasting |
 | TimesFM | https://github.com/google-research/timesfm |
 | Nixtla forecasting libraries | https://github.com/Nixtla |
 | CAISO OASIS | http://oasis.caiso.com |
 | PJM Data Miner 2 | https://dataminer2.pjm.com |
 | ERCOT public API | https://data.ercot.com |
 | NYISO archives | https://mis.nyiso.com/public/ |
 | RTS-GMLC (synthetic grid) | https://github.com/GridMod/RTS-GMLC |
 | Texas A&M synthetic grids | https://electricgrids.engr.tamu.edu |
 | PyPSA-USA | https://github.com/PyPSA/pypsa-usa |
 | Prescient | https://github.com/grid-parity-exchange/Prescient |
 | Sienna | https://sienna-platform.github.io/Sienna/ |
 | PyPSA | https://pypsa.org/ |
 | QuESt PCM | https://www.sandia.gov/ess/tools-resources/quest/quest-pcm |
 | PowerGenome | https://github.com/PowerGenome/PowerGenome |
 | Herbie (HRRR access) | https://github.com/blaylockbk/Herbie |
 | Open-Meteo | https://open-meteo.com |
 | pvlib-python | https://github.com/pvlib/pvlib-python |
 | PySAM (NREL) | https://github.com/NREL/pysam |
 | windpowerlib | https://github.com/wind-python/windpowerlib |
 | PyWake (DTU) | https://github.com/DTUWindEnergy/PyWake |
 | Open Climate Fix (PVNet et al.) | https://github.com/openclimatefix |
 | GluonTS | https://github.com/awslabs/gluonts |
 | Darts | https://github.com/unit8co/darts |
 | mgcv (GAMs, R) | https://cran.r-project.org/package=mgcv |
 | Hong & Fan, probabilistic load forecasting review | https://doi.org/10.1016/j.ijforecast.2015.11.011 |
 ---
 ## 14. Verification notes
 Items marked ✅ were confirmed against primary sources in July 2026 (GitHub repos, OEDI/OSTI records, LBNL publication pages, journal/arXiv papers). Items marked ◻︎ reflect well-established domain knowledge but were not individually re-verified for this report. The tiered model sections (4.4, 5.4, 6.3) are ◻︎ overall: the existence and character of each listed model/library is standard knowledge, and the epftoolbox specifics within them are ✅, but claims about *relative performance* (e.g., LEAR/GBM vs. foundation models on spiky prices, TimesFM competitiveness on load) summarize the 2024–2026 literature's general findings and will shift as models evolve — re-check against current papers before citing — in particular, exact ISO portal URLs/registration requirements and commercial gas-index licensing change periodically and should be checked at time of use. Dataset vintages stated (PERFORM 2017–2019; epftoolbox ending 2018; cleaned EIA-930 through Jan 2025; LBNL queues through end-2025) are as published at compile time.