power-market-trading-docs/primers/technical_scuc_sced_en.md

# A Primer on SCUC and SCED: The Optimization Engines of Wholesale Electricity Markets

## 1. What These Acronyms Actually Are

Security-Constrained Unit Commitment (SCUC) and Security-Constrained Economic Dispatch (SCED) are the two optimization problems that clear organized wholesale electricity markets in the United States and, in closely related forms, around the world. Every locational price you see published by PJM, MISO, CAISO, ERCOT, SPP, NYISO, or ISO-NE is a byproduct — literally a dual variable or its mixed-integer analogue — of one of these two problems. Understanding them means understanding how roughly two-thirds of US electricity is scheduled and priced.

The two problems answer different questions on different timescales. **SCUC answers "which units should be on?"** — a commitment decision involving startup costs, minimum run times, and hours of lead time, solved primarily in the day-ahead market. **SCED answers "given the units that are on, how much should each produce right now?"** — a dispatch decision solved every five minutes in real time. The "security-constrained" prefix, common to both, means the solution must respect the transmission network not only under normal conditions but also under a defined list of contingencies: the system must survive the sudden loss of any single major element. This is the engineering criterion known as N-1 security, and embedding it in the market optimization is what distinguishes modern LMP markets from simple single-node auctions.

## 2. The Physical and Institutional Setting

Three physical facts shape everything. First, supply and demand must balance continuously — frequency deviates within seconds of an imbalance, and large deviations cascade into blackouts. Second, power flows distribute themselves across the network according to Kirchhoff's laws, not according to contracts; you cannot route power, so congestion on one line is determined by injections everywhere. Third, generators are lumpy and inflexible: a large thermal unit may need 8–24 hours notice to start, cost tens of thousands of dollars per startup, be unable to run below 40–50% of capacity, and be limited in how fast it can ramp.

Institutionally, an Independent System Operator (ISO) or Regional Transmission Organization (RTO) runs a two-settlement market. The **day-ahead market** closes in the morning of the day before the operating day: participants submit offers (generators) and bids (loads and financial traders), and the ISO runs SCUC to produce hourly commitments, schedules, and day-ahead LMPs at which those schedules are financially settled. In the **real-time market**, SCED runs every five minutes against actual conditions, producing real-time LMPs; deviations between real-time delivery and day-ahead schedules settle at real-time prices. Between the two, a **reliability unit commitment (RUC)** pass re-runs a SCUC-like model against the ISO's own load forecast to commit any additional capacity needed for reliability that the financial day-ahead market did not procure.

## 3. The SCED Formulation

SCED is the simpler problem and the right place to start. In its DC-linearized form it is a linear program:

```
minimize    Σ_g C_g(p_g)                        (offer-based cost)
subject to  Σ_g p_g = Σ_n d_n                   (system power balance)  → dual λ
            P_g^min ≤ p_g ≤ P_g^max             (dispatch limits of committed units)
            |p_g − p_g^prev| ≤ R_g · Δt          (ramp limits over the interval)
            −F_k ≤ Σ_n PTDF_{k,n} · inj_n ≤ F_k  (base-case line flow limits) → duals μ_k
            (same flow constraints under each contingency c in the monitored set)
            reserve constraints (co-optimized regulation, spin, non-spin)
```

The cost functions C_g are piecewise-linear, built from each generator's offer curve (up to ten price-quantity segments in most ISOs). The network is represented through Power Transfer Distribution Factors (PTDFs) — linear sensitivities mapping nodal injections to line flows under the DC approximation, which neglects losses and reactive power (losses are typically added back through marginal loss factors). Contingency security enters by adding flow constraints computed with post-contingency sensitivities (LODFs, line outage distribution factors): for each monitored line k and each contingency c, the post-outage flow must remain within the line's emergency rating. In practice only a screened "active set" of binding or near-binding constraint pairs is included, because the full set (thousands of lines × thousands of contingencies) is enormous and almost entirely slack.

Modern SCED co-optimizes energy with **operating reserves and regulation**: a unit's headroom can be sold as reserve instead of energy, and the LP chooses the split that minimizes total cost against reserve demand curves. It typically also runs in a short **look-ahead** mode (several 5–15 minute intervals) so that ramp constraints are managed proactively rather than myopically.

## 4. Locational Marginal Prices

The elegance of SCED is that its dual solution *is* the market price. The LMP at node n decomposes as:

```
LMP_n = λ  +  loss component  −  Σ_k μ_k · PTDF_{k,n}
```

that is, the system marginal energy price, plus the marginal cost of losses at that location, plus a congestion component built from the shadow prices of binding transmission constraints. When nothing is congested, all nodes see (nearly) the same price. When a constraint binds, prices separate: nodes whose injections would relieve the constraint are paid more, nodes that would aggravate it are paid less — the price system internalizes Kirchhoff's laws. Because SCED is a convex LP, these prices support the dispatch: every generator dispatched between its limits is exactly indifferent at the margin, and no committed generator loses money on its incremental energy at the LMP. Congestion rents collected by the ISO (load pays more than generation receives, in aggregate, when lines bind) fund Financial Transmission Rights, the hedging instruments for congestion risk.

## 5. The SCUC Formulation

SCUC wraps the dispatch problem in binary commitment decisions over a 24–48 hour horizon:

```
minimize    Σ_t Σ_g [ SU_g · v_{g,t} + NL_g · u_{g,t} + C_g(p_{g,t}) ]
subject to  u_{g,t} ∈ {0,1}                         (on/off status)
            v_{g,t} ≥ u_{g,t} − u_{g,t−1}            (startup logic)
            minimum up and down times                (once on, stay on ≥ UT_g hours, etc.)
            P_g^min · u_{g,t} ≤ p_{g,t} ≤ P_g^max · u_{g,t}
            ramp limits, startup/shutdown ramp rates
            hourly power balance, reserve requirements
            base-case and contingency transmission limits
```

The objective now includes **startup costs** (SU, often dependent on how long the unit has been off — hot, warm, cold starts) and **no-load costs** (NL, the cost of running at zero net output), alongside incremental energy. The binary variables and minimum up/down time logic make SCUC a large mixed-integer linear program: a large RTO's day-ahead SCUC spans roughly 1,000+ generators, 36–48 hourly periods, tens of thousands of network constraints after screening, and on the order of 10⁵ binary variables. ISOs solve it within strict market timelines (roughly 1–4 hours of wall-clock budget) to an optimality gap tolerance, typically around 0.1%.

Several modeling refinements matter enormously in practice. **Combined-cycle units** are modeled with configuration-based transitions (1x1, 2x1, etc.), each configuration its own pseudo-unit with transition logic. **Tight MIP formulations** of the up/down-time and ramping polytopes (e.g., the Rajan–Takriti facets and their successors) dramatically reduce solve times — formulation strength has been as important as solver progress in making large SCUC tractable. And **demand bids, virtual bids (INCs/DECs), and price-sensitive load** enter the day-ahead problem symmetrically with generation, which is what makes the day-ahead market a genuinely financial, two-sided auction rather than a pure engineering schedule.

## 6. The Pricing Problem SCUC Creates: Non-Convexity and Uplift

SCUC breaks the clean duality story of SCED. With binary variables, the problem is non-convex, and no uniform set of prices generally supports the solution: a unit committed for reliability may, at the resulting LMPs, fail to recover its startup and no-load costs; conversely a unit might see prices at which it would prefer a different schedule. Markets patch this with **make-whole payments (uplift)**: side payments guaranteeing that any resource following ISO instructions recovers its as-offered costs over the day. Uplift is discriminatory (different resources receive different effective prices) and non-transparent (it is not hedgeable the way LMPs are), so market designers work to minimize it.

Two pricing responses are in active use. **Fast-start pricing** relaxes the binary constraints of online fast-start units so their startup and no-load costs can set the LMP when they are marginal — several ISOs adopted variants after FERC pushed the issue in the late 2010s. More ambitiously, **Extended LMP / convex-hull pricing** seeks prices that minimize total uplift by pricing off the convex hull of the non-convex cost functions; MISO runs an approximation of this. The theoretically clean object — the convex-hull price — is the Lagrangian dual optimum of the commitment problem, but computing it exactly at scale remains hard, and the debate over how far prices should deviate from marginal-cost logic to reflect commitment costs is one of the liveliest in market design.

## 7. How the Pieces Fit Together Across the Operating Timeline

A useful mental model is a cascade of the same core problem re-solved with shrinking horizons and hardening decisions. Day-ahead SCUC (financial, bid-in demand) sets hourly commitments and day-ahead settlements. RUC (reliability pass, forecast demand) tops up physical commitment. Intraday or hour-ahead SCUC processes re-commit fast and intermediate units as forecasts update (e.g., ERCOT's HRUC, CAISO's short-term unit commitment). Real-time SCED, every five minutes, dispatches the committed fleet and sets real-time prices, with a look-ahead of an hour or so; fast-start units can still be started by a real-time commitment layer. Each stage inherits the previous stage's decisions as fixed and re-optimizes what remains flexible — the cascade itself is an implicit, rolling form of recourse against uncertainty, even though each individual solve is deterministic.

This is also where the connection to stochastic optimization sits: production SCUC/SCED are deterministic in their treatment of load and renewable uncertainty (a single forecast, exogenous reserve requirements), while handling *equipment* uncertainty robustly through the N-1 contingency set. Stochastic SCUC — scenarios for net load, commitment as here-and-now, dispatch as recourse — is the research extension of exactly this problem, used today in advisory look-ahead tools and in setting reserve and ramp-product requirements rather than in the clearing engine itself, chiefly because of the pricing, cost-recovery, and transparency questions that scenario-based clearing raises.

## 8. Computation

Day-ahead SCUC at RTO scale is among the largest MIPs solved on a daily industrial cadence. The practical toolkit: state-of-the-art commercial MIP solvers (branch-and-cut with presolve, cutting planes, heuristics); tight formulations as noted above; **lazy-constraint handling of transmission security** — solve with a small active set of network constraints, check the full N-1 set against the solution, add violated constraints, and iterate, which typically converges in a handful of passes because so few constraints ever bind; decomposition legacies (Lagrangian relaxation was the pre-MIP workhorse and still informs pricing theory; Benders variants appear in extensions); and warm-starting from the previous day's solution. SCED, being an LP (or nearly so) with an active-set network model, solves in seconds — its challenge is data pipeline latency and robustness at a five-minute cadence, not optimization difficulty. A notable research thread applies machine learning to accelerate SCUC by predicting active constraint sets, good commitment patterns, or screening contingencies.

## 9. Variations Outside the US

The SCUC/SCED architecture is characteristic of US-style centralized pool markets with locational pricing. European markets differ structurally: day-ahead clearing (EUPHEMIA algorithm) is a zonal, portfolio-based auction in which unit commitment is left to the participants and non-convexities enter through block orders; congestion within zones is managed afterward by TSO redispatch. Australia's NEM runs a five-minute regional dispatch resembling SCED without a day-ahead commitment market. The trade-offs between nodal/centralized-commitment and zonal/self-commitment designs — pricing accuracy and congestion management versus simplicity and participant autonomy — are a standing debate, with Europe gradually adding flow-based coupling and discussions of finer granularity, and the US designs serving as the reference point for locational pricing.

## 10. Current Pressure Points

Several forces are reshaping both problems. **Storage** requires state-of-charge dynamics inside SCUC/SCED and raises subtle questions about opportunity-cost-based offers and whether the ISO or the owner should manage the state of charge. **Renewables and net-load volatility** stress the deterministic-forecast assumption, driving ramping products, more frequent intraday commitment, and interest in stochastic and robust look-ahead tools. **Distributed energy resources and demand flexibility** (FERC Order 2222 in the US) push aggregations of small resources into the same optimization, multiplying model size. **Co-optimization scope** keeps expanding — energy, multiple reserve classes, ramping products, and in some designs day-ahead ancillary demand curves — enlarging the already-large MIP. And **pricing reform** (convex-hull approximations, fast-start pricing, scarcity pricing via operating reserve demand curves) continues to evolve the mapping from these optimizations to the prices that drive investment.

## 11. Suggested Reading

For the market-design foundations: Schweppe, Caramanis, Tabors, and Bohn, *Spot Pricing of Electricity* (1988), the origin of LMP theory; and Hogan's papers on contract networks and FTRs. For formulations: Knueven, Ostrowski, and Watson, "On Mixed-Integer Programming Formulations for the Unit Commitment Problem" (2020) is the modern survey; O'Neill et al., "Efficient market-clearing prices in markets with nonconvexities" (2005) and Gribik, Hogan, and Pope's 2007 paper on ELMP frame the pricing problem; Rajan and Takriti's 2005 report gives the classic tight up/down-time polytope. For practice, the ISOs' own business practice manuals (e.g., PJM Manual 11, MISO's energy and operating reserve market documentation) are unusually informative primary sources on how these models are actually configured and solved.