power-market-trading-docs/primers/grid/05_trading_under_uncertainty_en.md

Trading Renewables Under Uncertainty: Newsvendor Logic, Quantile Bidding, and the Economics of Forecast Errors

The mathematical foundation beneath declaration optimization — where it comes from, the canonical results, the literature that established them, and how China's market design changes the classical answers. Companion primer to the venture documents' Pool 1 Economics; self-contained for readers with basic probability.

---

1. The problem, stated once

A renewable producer must commit, before delivery, to a quantity q for each future interval — a day-ahead offer, a declaration, a schedule. Actual production G is random at commitment time. The market pays the committed quantity at one price and settles the deviation (G − q) at another. The producer's question: what q maximizes expected revenue (or expected utility), given a probabilistic forecast of G and beliefs about the prices?

This is one of the oldest problems in operations research wearing an electricity costume, and recognizing the costume is half the education.

2. Two-settlement revenue and imbalance pricing schemes

Per interval, with day-ahead price P_DA and imbalance settlement prices, revenue is:

R(q) = q \cdot P_{DA} + \psi^{+}\,(G-q)^{+} - \psi^{-}\,(q-G)^{+}

where (x)⁺ = max(x, 0), ψ⁺ is the price received for surplus (over-delivery), and ψ⁻ the price paid to cover shortfall (under-delivery). Market designs differ in how ψ± relate to P_DA and to the real-time/balancing price P_RT:

• Dual pricing (two-price imbalance) — the classic European balancing design: deviations that aggravate system imbalance settle at the (unfavorable) balancing price, deviations that help settle at P_DA. Consequence: ψ⁺ ≤ P_DA ≤ ψ⁻ always; deviating is never profitable in expectation, only costly. This is the setting of the canonical results.
• Single pricing (one-price imbalance) — both deviation directions settle at the same real-time price: ψ⁺ = ψ⁻ = P_RT. Deviations can be profitable (over-deliver into a high P_RT and you gain). US real-time markets and China's provincial DA/RT settlement are of this type: the deviation term is (G − q)(P_RT), i.e., R = q·P_DA + (G−q)·P_RT — exactly the identity used throughout the venture documents.
• Hybrids — single pricing plus regulatory penalty envelopes (bands, assessments) that reintroduce dual-pricing-like asymmetry administratively. China's 两个细则 regime makes the Chinese design effectively a hybrid: financial single pricing inside a penalty corridor.

Keep this taxonomy in hand — the optimal-offer answer changes qualitatively across it.

3. The newsvendor problem and its classical solution

The 19th-century original: a news vendor stocks Q papers before knowing demand D. Each unsold paper loses the overage cost c_o; each unmet customer loses the underage cost c_u. Expected cost is minimized not at E[D] but at the famous critical fractile:

Q^{*} = F_D^{-1}\!\left(\frac{c_u}{c_u + c_o}\right)

where F_D is demand's CDF. The intuition is a marginal argument: at the optimum, the probability-weighted cost of stocking one more unit (c_o with probability F(Q)) equals the probability-weighted cost of stocking one less (c_u with probability 1 − F(Q)). Asymmetric costs ⇒ deliberately biased decision. The decision-relevant object is the distribution F, not the point forecast — a 150-year-old argument for probabilistic forecasting.

4. The mapping to renewable offering (dual pricing)

Under dual pricing, define the producer's two unit opportunity costs: offering too little forgoes c_u = P_DA − ψ⁺ per surplus MWh (sold at ψ⁺ instead of P_DA); offering too much costs c_o = ψ⁻ − P_DA per shortfall MWh (bought back above P_DA). Maximize E[R(q)]:

\frac{d\,\mathbb{E}[R]}{dq} = P_{DA} - \psi^{+}\big(1 - F_G(q)\big) - \psi^{-} F_G(q) = 0

\boxed{\;q^{*} = F_G^{-1}\!\left(\frac{P_{DA} - \psi^{+}}{\psi^{-} - \psi^{+}}\right) = F_G^{-1}\!\left(\frac{c_u}{c_u + c_o}\right)\;}

The optimal offer is a quantile of the production distribution at the newsvendor critical ratio. If buying back shortfalls is twice as painful as discounting surpluses (c_o = 2c_u), the ratio is 1/3: offer the 33rd percentile of your production forecast — deliberately underbid, because the market punishes shortfalls harder. This is the foundational result of renewable trading: "offer a quantile, not the mean." In expectation-maximizing form it needs only the marginal prices' expectations (replace ψ± by E[ψ±] when prices are independent of G — an assumption we are about to break).

Worked micro-example: P_DA = ¥400/MWh, E[ψ⁺] = ¥320, E[ψ⁻] = ¥520. c_u = 80, c_o = 120, ratio = 80/200 = 0.40. If the plant's hourly production forecast has quantiles Q30 = 41 MWh, Q40 = 44, Q50 = 47 — declare 44, three MWh below the median, and that shading is optimal, not conservative.

5. What single pricing does — and why correlation becomes the whole story

Set ψ⁺ = ψ⁻ = P_RT and expected revenue becomes E[R] = E[G·P_RT] + q·E[P_DA − P_RT]. If prices were independent of the producer's output, q's coefficient is a constant: the risk-neutral optimum is a corner (declare the maximum if E[S] > 0, the minimum if E[S] < 0) and the "forecast" is irrelevant to the offer — an old and initially shocking observation about one-price imbalance settlement. Reality restores an interior optimum through three channels, and they are precisely the venture documents' structure:

1. Error–price correlation. The producer's error shares weather with the fleet, so ε and S are dependent; expected deviation cost gains the term Cov(ε, S) — the "structural bleed." For solar the covariance is adverse on both tails (surplus sells into crashed P_RT; shortfall buys back at spikes), so the effective cost function is again asymmetric and forecast-dependent even under single pricing. This is the literature's "price-taker in volume, price-maker in error" regime, studied for wind under system-imbalance-correlated balancing prices.
2. Penalty envelopes. Regulatory assessments on |G − q| (China's 两个细则, tolerance bands) act as an administrative dual-pricing layer, restoring newsvendor curvature with rule-defined c_u, c_o that vary by band.
3. Risk aversion. Even absent both, variance matters to a leveraged plant; CVaR-constrained bidding (a standard extension) pulls the offer off the corner toward distribution-aware interior solutions.

The general problem — maximize E[q·S] − Cov-adjusted deviation cost − penalty(q) over the joint distribution of (G, P_DA, P_RT) — no longer has a closed-form fractile, but the newsvendor logic survives as its skeleton: the optimum still balances marginal over- against under-commitment costs, now computed from a joint, regime-dependent distribution. That is exactly what a modern declaration optimizer does numerically.

6. The pinball connection: why forecast evaluation and trading are the same mathematics

The pinball (quantile) loss for a quantile forecast \hat{q}_\alpha is L_α(G, \hat q) = α(G − \hat q)⁺ + (1−α)(\hat q − G)⁺. Compare with the newsvendor cost c_u(G−q)⁺ + c_o(q−G)⁺: they are the same function with α = c_u/(c_u+c_o). Minimizing pinball loss at level α is solving a newsvendor with that critical ratio — which yields two practically important corollaries. First, a forecaster trained with pinball loss at the market's critical ratio is directly producing optimal offers; forecasting and bidding are one optimization, not a pipeline. Second, the standard probabilistic-forecast skill score (CRPS = pinball loss integrated over all α) measures exactly the economic quantity a trader cares about, averaged over all possible cost asymmetries — the rare case where the statistician's loss function and the trader's P&L are the same object.

7. A literature map

• Foundations of probabilistic wind power forecasting: Bremnes (2004) on local quantile regression for wind power; the subsequent decade of ensemble-based predictive densities (Pinson and co-authors; Gneiting's proper-scoring framework giving CRPS/pinball their theoretical footing).
• The canonical trading results: Pinson, Chevallier & Kariniotakis (2007), Trading wind generation from short-term probabilistic forecasts of wind power — the quantile-offer result in operational form; earlier strands on imbalance-cost minimization (e.g., Bathurst, Weatherill & Strbac 2002; Skytte's 1999 regression analysis of imbalance prices).
• The textbook: Morales, Conejo, Madsen, Pinson & Zugno, Integrating Renewables in Electricity Markets (Springer, 2014) — the standard graduate treatment: newsvendor offering, stochastic programming formulations, dual vs. single pricing, risk measures.
• Correlated/endogenous prices: Zugno, Morales, Pinson & Madsen on price-maker bidding (bilevel/MPEC formulations); the strand analyzing balancing prices as functions of system imbalance, which formalizes Cov(ε, S).
• Extensions: CVaR/risk-averse offering (Conejo et al.'s stochastic programming school); multi-market sequencing (day-ahead → intraday → balancing) and the value of trading closer to delivery; storage-renewable co-optimization as multistage stochastic programs; reinforcement-learning bidding (recent, promising, validation-hungry).
• Practice: European renewable balancing-responsible parties (Statkraft, Axpo, Danske Commodities and peers) have run quantile-based imbalance optimization commercially since roughly the early 2010s — the standing existence proof that the theory monetizes.

8. The China mapping — what transfers, what mutates, what is new

Classical element	Chinese provincial reality	Consequence
Dual-price imbalance	Single-price DA/RT settlement + 两个细则 penalty envelope	Hybrid: newsvendor curvature comes from penalties and from Cov(ε,S), not from ψ± spreads
Price-taking offer, exogenous prices	Fleet-correlated errors in solar-saturated provinces	Cov(ε, S) is first-order; joint (ε, S) modeling is the core asset
Quantity-only offer	报量报价 where provided: the pair (q, p)	The price bid adds a floor-regime self-curtailment option absent from the classical problem
Continuous liquid intraday for position repair	Limited/no re-declaration	The D−1 offer carries more weight; nowcasting informs operations, not re-trading (yet)
CfD/PPA hedges on own output	Mechanism price vs. a fleet-average reference	Relative-performance economics: beating the fleet average is retained alpha on covered volume — no classical analog
Stationary market microstructure	Frozen offers, non-bidding demand	Spreads unusually physically determined; regime models outperform behavioral ones (a transient, valuable state)

The honest summary for practitioners: the skeleton (newsvendor logic, quantile thinking, pinball-trained forecasting, stochastic-program offering) transfers intact and is fifteen-plus years validated; the flesh (which distributions, which penalties, which options exist) is province-specific, months old, and partially unwritten — the part where operating in China means contributing to the literature rather than consuming it.

9. Five takeaways

1. The optimal offer is a quantile at a critical ratio — asymmetric costs make deliberate bias optimal; "accuracy" is not the objective.
2. Under single pricing, the game is the joint distribution: Cov(ε, S) plus penalties replace the ψ± spread as the source of curvature.
3. Forecast evaluation and trading share one loss function (pinball ↔ newsvendor) — build the forecaster for the decision, not for RMSE.
4. Everything above is established science with a commercial track record; the defensible assets are calibrated local distributions, rule encodings, and data — never the algebra.
5. Where Chinese rules diverge (price bids, fleet-average CfDs, frozen-offer microstructure), the classical literature goes silent — those are the open problems, and the opportunity.

End of primer.