Add us_market/: field guide and economists survey (EN/ZH)

Field guide consolidates July 2026 working discussions on bidding,
trading, and AI impact; dropped a stale reference to an archive/
directory that was never part of this repo. Economists survey ZH
renamed from its Chinese filename to economists_survey_zh.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

us_market/economists_survey_en.md

@@ -0,0 +1,272 @@
+# How Economists Have Shaped the US Electricity Markets: Theory, Practice, and Key Milestones
+
+**A Survey**
+
+---
+
+## Abstract
+
+Few industries have been as thoroughly re-engineered by academic economics as the US electric power sector. Over roughly four decades, economists supplied the intellectual critique that dismantled cost-of-service regulation of generation, developed the pricing theory that underpins every organized wholesale market in the country, diagnosed the failures of first-generation market designs, and continue to shape the rules governing capacity, demand response, storage, and the integration of renewable energy. This survey traces that influence across four dimensions: (i) the intellectual origins of restructuring in the economics of regulation; (ii) the theoretical foundations of modern market design, most notably peak-load pricing, spot pricing, and locational marginal pricing; (iii) the institutional milestones—from PURPA (1978) through FERC Orders 888, 2000, 745, 841, and 2222—where economic ideas were translated into regulation; and (iv) the empirical and applied literature through which economists diagnosed crises (California 2000–01, Texas 2021), built market-monitoring institutions, and evaluated whether restructuring delivered its promised efficiencies. We argue that US electricity restructuring is best understood not as "deregulation" but as a sustained exercise in applied mechanism design, in which economists moved from critics of regulation to architects of institutions, and we conclude with the open design questions—resource adequacy under deep decarbonization, non-convex pricing, hybrid markets—where economic theory is once again being pressed into service.
+
+---
+
+## 1. Introduction
+
+When the Federal Energy Regulatory Commission (FERC) issued Order No. 888 in April 1996, requiring open, non-discriminatory access to the interstate transmission grid, it set in motion the most consequential reorganization of the American electric power industry since the 1930s. Within a decade, two-thirds of US electricity demand was served through organized wholesale markets operated by Independent System Operators (ISOs) and Regional Transmission Organizations (RTOs), in which the price of energy is computed every five minutes at thousands of distinct network locations by solving a large-scale optimization problem whose objective function—maximization of social surplus subject to the physics of power flow—comes straight out of welfare economics.
+
+That last fact is the central theme of this survey. In most deregulated industries—airlines, trucking, telecommunications, natural gas—economists supplied the *case* for competition, and markets then evolved through decentralized entrepreneurial activity. Electricity was different. Because power flows obey Kirchhoff's laws rather than contracts, because electricity was (until recently) not economically storable, and because supply and demand must balance second by second to avoid cascading blackouts, a workable electricity "market" could not simply emerge; it had to be *designed*. The result is that economists did not merely advocate for electricity competition—they wrote its constitution. The pricing algorithm at the heart of every US organized market (locational marginal pricing), the financial instruments that hedge congestion (financial transmission rights), the auctions that procure future capacity, the scarcity-pricing mechanisms that replace them in Texas, and the market-power mitigation machinery that polices all of the above are, to an unusual degree, the direct product of academic economic research.
+
+This survey synthesizes that history for a general economics audience. Section 2 traces the intellectual origins of restructuring in the postwar economics of regulation and natural monopoly. Section 3 develops the theoretical core: peak-load pricing, spot pricing, and Hogan's locational marginal pricing framework. Section 4 walks through the legislative and regulatory milestones through which these ideas became law. Section 5 examines the California crisis of 2000–01, the episode in which empirical economists demonstrated their value as market diagnosticians and which permanently institutionalized economic market monitoring. Section 6 covers the resource-adequacy problem—"missing money," capacity markets, and energy-only alternatives. Section 7 surveys mechanism-design contributions: auctions, non-convex pricing, and virtual bidding. Section 8 turns to the demand side: dynamic retail pricing and the demand-response wars. Section 9 addresses the modern frontier—renewables, storage, distributed resources, and the Texas crisis of 2021. Section 10 reviews the empirical literature evaluating restructuring's effects. Section 11 discusses critiques and the emerging debate over "hybrid" markets, and Section 12 concludes.
+
+Three caveats. First, we focus on the United States; the UK privatization of 1990, though enormously influential on American thinking (and itself heavily shaped by economists such as Stephen Littlechild), enters only as background. Second, "economists" is interpreted broadly to include the engineering-economics tradition at MIT and elsewhere—Fred Schweppe was an electrical engineer, but *Spot Pricing of Electricity* is a work of economics. Third, no survey of this length can be exhaustive; our aim is an accurate map of the main intellectual lineages and institutional turning points.
+
+---
+
+## 2. Intellectual Origins: The Economics of Regulation and Its Discontents
+
+### 2.1 The old regime and its theoretical rationale
+
+From the 1920s until the 1980s, the US electricity industry was organized around vertically integrated, investor-owned utilities holding exclusive retail franchises, with prices set by state commissions on a cost-of-service basis and wholesale/interstate matters governed by the Federal Power Act of 1935. The intellectual rationale was the theory of natural monopoly: with enormous fixed costs and (it was believed) pervasive economies of scale in generation, transmission, and distribution alike, competition was thought to be wasteful or unsustainable, and regulated monopoly the least-bad institutional response.
+
+Economists were present at the creation of this regime—the "fair rate of return on fair value" apparatus descends from institutionalist economics and rate-base litigation of the early twentieth century—but the profession's most important contribution came later, in the form of critique.
+
+### 2.2 The postwar critique
+
+Four strands of postwar economics converged to undermine confidence in the regulated-monopoly model.
+
+**Incentive distortions.** Averch and Johnson (1962) showed formally that rate-of-return regulation gives utilities an incentive to over-invest in capital—the "gold-plating" or A–J effect. Whatever the empirical magnitude of the effect (still debated), the paper reframed regulation as an incentive problem rather than an accounting exercise, and seeded the later literature on incentive regulation (Laffont and Tirole 1993) that would inform performance-based ratemaking.
+
+**Regulatory capture.** Stigler (1971) and Peltzman (1976) argued that regulation is supplied to politically effective interest groups rather than in the public interest, while Demsetz (1968) asked why natural monopoly required regulation at all when the *right to serve* could be auctioned—franchise bidding as competition *for* the market. These Chicago contributions eroded the presumption that regulation reliably corrected market failure.
+
+**Evidence on scale economies.** Empirical work—most influentially Christensen and Greene (1976)—found that by the 1970s most large US generating firms had exhausted economies of scale at the plant and firm level. If generation was no longer a natural monopoly, the case for regulating it collapsed, even if wires remained naturally monopolistic. This "unbundling" insight—competition where feasible (generation, retail), regulation where necessary (transmission, distribution)—became the master blueprint of restructuring.
+
+**The deregulation movement in practice.** Alfred Kahn's two-volume *The Economics of Regulation* (1970–71) supplied the synthesis, and Kahn himself, as chairman of the Civil Aeronautics Board under President Carter, demonstrated that an academic economist could dismantle a regulatory regime from within. Airline (1978), trucking (1980), railroad (1980), and natural gas (1978–92) deregulation created both the political template and the professional confidence for tackling electricity—by common consent the hardest case.
+
+### 2.3 The proximate crisis and the pivotal book
+
+The 1970s destroyed the postwar equilibrium of the utility industry: oil shocks, the collapse of nuclear construction economics (with celebrated cost overruns and the WPPSS default), stagnating demand growth, and double-digit rate increases turned regulators and consumers alike against the incumbent model. The Public Utility Regulatory Policies Act of 1978 (PURPA), discussed in Section 4, inadvertently ran a natural experiment demonstrating that non-utility generators could build and operate power plants.
+
+Into this environment came Paul Joskow and Richard Schmalensee's *Markets for Power: An Analysis of Electric Utility Deregulation* (1983), the single most influential economic analysis of electricity restructuring ever written. Joskow and Schmalensee did two things. First, they systematically dismantled the claim that the status quo was efficient, documenting the incentive problems of cost-of-service regulation. Second—and this is often forgotten—they were deeply *cautious* about wholesale competition, cataloguing the technical obstacles (loop flow, reliability externalities, market power in transmission-constrained regions) that naïve deregulation would encounter. The book's lasting contribution was to define the research agenda: every obstacle they identified became, over the following two decades, a design problem that economists set out to solve. Joskow's subsequent forty-year output—on transmission investment, contract structures, capacity markets, and lately decarbonization—constitutes the connective tissue of the entire field, and his role as advisor, board member, and expert in countless proceedings made him arguably the most influential single economist in the sector's history.
+
+---
+
+## 3. Theoretical Foundations: From Peak-Load Pricing to Locational Marginal Prices
+
+### 3.1 Marginal-cost pricing and the peak-load problem
+
+The deep theory begins in postwar France. Marcel Boiteux, an economist-engineer at Électricité de France (and student of Maurice Allais), worked out in the late 1940s and 1950s how to price electricity at marginal cost when demand varies over time and capacity is costly: off-peak users should pay energy cost only, while peak users pay energy cost plus the marginal cost of capacity (Boiteux 1949, 1956; translated and introduced to Anglophone readers largely via Nelson 1964). Steiner (1957) independently developed the peak-load pricing model in the American literature, and Williamson (1966) refined the welfare analysis. Boiteux's companion work on second-best pricing under a budget constraint (Ramsey–Boiteux pricing) became the standard framework for recovering fixed costs with minimal distortion. EDF's *tarif vert*, implemented in 1956, proved these were not blackboard curiosities.
+
+Two further ingredients completed the classical toolkit. Turvey (1968) and others connected peak-load pricing to investment planning. And the theory of pricing under uncertainty with rationing—what is the efficient price when capacity is short?—pointed toward the concept that would later dominate resource-adequacy debates: the value of lost load (VoLL), the willingness of consumers to pay to avoid interruption. Efficient scarcity prices should rise toward VoLL as the system approaches involuntary curtailment; this single idea underlies everything from ERCOT's price caps to capacity-market demand curves.
+
+### 3.2 Spot pricing: Schweppe and the MIT school
+
+The decisive step from tariff theory to market design was taken at MIT in the late 1970s and 1980s by Fred Schweppe—an electrical engineer—working with economists and engineers Michael Caramanis, Richard Tabors, and Roger Bohn. Their program, culminating in *Spot Pricing of Electricity* (Schweppe, Caramanis, Tabors, and Bohn 1988), posed the question: what is the *true* marginal cost of electricity, recognizing that it varies not just by time of day but continuously, stochastically, and—crucially—by location on the network?
+
+Their answer: the efficient spot price at each moment and each bus of the network equals the system lambda (the marginal cost of energy at the reference bus) adjusted for marginal transmission losses and the shadow prices of binding transmission constraints. Prices differ across locations whenever the network is congested, because Kirchhoff's laws prevent cheap power from reaching expensive locations. Schweppe et al. envisioned these prices as the basis for a decentralized "homeostatic" control of the power system: generators and (in principle) consumers responding to real-time prices would replicate the least-cost dispatch that central engineers had always computed—but through markets. The book was written before any institution existed to implement it; it became the technical bible of every subsequent market design.
+
+### 3.3 Hogan: locational marginal pricing and financial transmission rights
+
+The person who transformed spot-pricing theory into implementable market architecture was William Hogan of Harvard's Kennedy School. Beginning with "Contract Networks for Electric Power Transmission" (Hogan 1992), Hogan solved the problem that had stymied decentralized approaches: how to define tradable transmission rights on an AC network where power flows cannot be directed along contract paths.
+
+Hogan's insight had two parts. First, embrace rather than fight central dispatch: let an independent system operator run a *bid-based, security-constrained economic dispatch*—the same optimization utilities had always solved, but with offer curves submitted by competing generators—and let the dual variables (shadow prices) of that optimization define the market-clearing **locational marginal prices (LMPs)** at every node. The LMP at a node decomposes exactly as Schweppe prescribed: energy plus losses plus congestion. Second, define transmission rights *financially* rather than physically: a **financial transmission right (FTR)** between nodes A and B entitles its holder to the congestion-rent difference LMP(B) − LMP(A) on a specified quantity. FTRs are simultaneously feasible whenever the underlying injections are (the "revenue adequacy" theorem), they hedge congestion risk perfectly for matching physical schedules, and they can be auctioned or allocated to historical grid users—thereby solving the political problem of transition as well as the economic problem of hedging.
+
+This "pool-plus-FTR" or "integrated" architecture—centralized dispatch, nodal prices, financial rights, two settlements (day-ahead and real-time), co-optimized energy and ancillary services—was fought over ferociously in the 1990s against a rival "decentralized/bilateral" vision (associated with, among others, some California designers and the original UK Pool's critics), which favored self-scheduling, zonal prices, and physical rights. The verdict of experience was unambiguous. PJM abandoned zonal for nodal pricing in 1998 after gaming of zonal prices; New York launched with LMP (and its FTR variant, Transmission Congestion Contracts) in 1999; New England converted in 2003; the Midwest ISO launched nodal in 2005; California, after its crisis, reorganized around full nodal pricing in 2009; Texas converted from zones to nodes in 2010; and the Southwest Power Pool followed in 2014. Every organized US market now runs on Hogan's architecture. It is difficult to name another instance in which a single economist's design is executed, every five minutes, across most of a continental economy.
+
+Hogan's subsequent work extended the framework to virtually every contested design margin: multi-settlement systems, FTR options and obligations, scarcity pricing via operating reserve demand curves (Section 6.4), uplift and non-convex pricing (with Gribik and Pope), transmission investment incentives, and—continuing today—pricing for decarbonized grids. Harvey and Hogan's running debate with the Berkeley empiricists over California (Section 5) also set the methodological standards for market-power measurement.
+
+### 3.4 The supporting cast of theory
+
+Three further theoretical literatures fed the design canon:
+
+- **Auction and mechanism design.** Vickrey's (1961) foundations; the Wilson–Milgrom research program on auctions with common values and multi-unit demand; and Robert Wilson's *Architecture of Power Markets* (Econometrica, 2002), which framed electricity design as a problem in "market architecture"—the decomposition of an economic problem into linked auction sub-markets. Wilson himself advised on power-market designs internationally and in California, and Paul Milgrom's consulting practice extended auction design into electricity procurement. The 2020 Nobel to Milgrom and Wilson cited work whose applications explicitly include electricity.
+- **Supply function equilibrium.** Klemperer and Meyer (1989) modeled competition in supply schedules under uncertainty—precisely what generators submit to ISOs—and Green and Newbery (1992) applied SFE to the British Pool, providing the workhorse oligopoly model for electricity markets and an early, influential warning that a duopolistic pool would price far above marginal cost. This transatlantic result shaped American thinking about market structure and divestiture requirements.
+- **Contracts and vertical arrangements.** Joskow's earlier empirical work on long-term coal contracts and asset specificity (in the Williamsonian tradition), and the Allaz–Vila (1993) result that forward contracting can mitigate spot market power, informed both the design of transition contracts and the enduring policy preference for high contract cover—one of the clearest lessons economists drew from comparing California (uncontracted, catastrophic) with other markets.
+
+---
+
+## 4. Institutional Milestones: Ideas Become Law
+
+### 4.1 PURPA (1978): the accidental experiment
+
+The Public Utility Regulatory Policies Act of 1978, passed as energy-crisis legislation, required utilities to purchase power from "qualifying facilities" (cogenerators and small renewables) at the utility's *avoided cost*. Avoided cost is, of course, an economist's concept—the marginal cost the utility escapes—and its implementation launched a thousand rate proceedings in which economists testified about how to measure it. PURPA's deeper significance was unintended: the wave of non-utility generation it induced (especially in California, New York, and Texas) demonstrated that independent power producers could finance, build, and operate generation, destroying the empirical premise that generation required vertically integrated monopoly. Economists, including Joskow, documented both the demonstration effect and the pathologies of administratively determined avoided-cost contracts—which, when set too high (as in California's Standard Offer 4 contracts), saddled ratepayers with above-market costs and taught a lasting lesson about administered prices versus auctions.
+
+### 4.2 The Energy Policy Act of 1992 and FERC Orders 888/889 (1996)
+
+The Energy Policy Act of 1992 created "exempt wholesale generators," freeing independent generation from the Public Utility Holding Company Act, and empowered FERC to order transmission access case by case. FERC generalized this in Order No. 888 (1996), which required all transmission-owning utilities to file open-access tariffs offering service to third parties on the same terms they provided themselves, and Order No. 889, which established the OASIS information system and functional unbundling. The economic logic—that the bottleneck monopoly (wires) must not be permitted to leverage its position into the potentially competitive segment (generation)—was the essential-facilities/vertical-foreclosure reasoning of industrial organization, argued to FERC by a small army of economists on all sides. Order 888 also blessed the recovery of "stranded costs," itself the subject of a substantial economic literature (with William Baumol, Alfred Kahn, and others debating whether recovery was efficient transition compensation or an unwarranted bailout).
+
+### 4.3 State restructuring: California AB 1890 and the Northeast
+
+Restructuring of *retail* markets was a state affair. California moved first and most ambitiously: AB 1890 (1996), implementing CPUC decisions shaped by years of hearings in which academic economists were central witnesses, created the Power Exchange (PX) and the California ISO (launched March 31, 1998), required the utilities to divest most fossil generation, froze retail rates at 10% below prior levels, and—fatefully—discouraged long-term contracting, pushing procurement into the day-ahead and real-time markets. The design reflected a compromise between the bilateral/decentralized school and the pool school that satisfied neither; several prominent economists (including members of what became the ISO's Market Surveillance Committee) warned before launch about the combination of tight supply, uncontracted demand, capped retail prices, and market-power exposure. Massachusetts, Rhode Island, Pennsylvania, New York, New Jersey, Maryland, Illinois, Ohio, Michigan, and Texas followed with their own restructuring statutes between 1996 and 1999, typically pairing retail choice with generation divestiture and transition-cost charges. By 2000, roughly half the states had enacted or opened restructuring proceedings.
+
+### 4.4 Order 2000 (1999) and the Standard Market Design episode (2002–05)
+
+FERC's Order No. 2000 encouraged (but did not compel) formation of Regional Transmission Organizations—independent grid operators with defined characteristics and functions. The intellectual case for the ISO/RTO institution—an independent, non-profit operator running open, bid-based markets—had been developed by Hogan, Joskow, and others as the resolution of the transmission-independence problem. In 2002, FERC proposed to go further: the Standard Market Design (SMD) NOPR would have mandated the full nodal architecture—LMP, FTRs, day-ahead/real-time settlements, market monitoring, and resource adequacy—everywhere in the country. SMD was the high-water mark of economists' direct influence on federal policy: its text reads like a summary of the design literature. It also provoked a fierce political backlash from the South and West (regions with cheap regulated power and no appetite for markets), and FERC formally withdrew the proposal in 2005. The result is today's map: organized nodal markets covering roughly two-thirds of US load (PJM, MISO, CAISO, ERCOT, NYISO, ISO-NE, SPP), with the Southeast and much of the West remaining vertically integrated—an enduring natural experiment that empiricists would later exploit (Section 10).
+
+### 4.5 The later federal orders
+
+Subsequent FERC rulemakings, each examined in later sections, continued to translate economic analysis into rules: Order 719 (2008) on demand response and ISO governance; Order 745 (2011) on demand-response compensation; Order 755 (2011) on performance-based regulation payments; Order 1000 (2011) on transmission planning and cost allocation—a subject on which Joskow and Jean Tirole had written foundational theory; Order 841 (2018) on electric storage participation; and Order 2222 (2020) on distributed energy resource aggregation. The pattern is consistent: a design question surfaces; economists write papers and file affidavits; FERC codifies a (contested) resolution; litigation and further papers follow.
+
+---
+
+## 5. Trial by Fire: The California Crisis and the Rise of Market Monitoring
+
+### 5.1 The crisis
+
+Between May 2000 and June 2001, average wholesale prices in California rose by a factor of five to ten relative to prior years; the state suffered rolling blackouts; Pacific Gas & Electric entered bankruptcy; Southern California Edison approached it; the Power Exchange ceased operations; and the state spent tens of billions of dollars buying power. The proximate causes included drought (reduced Northwest hydro), rising natural gas and NOx permit prices, demand growth, and a genuinely tight market. But the crisis became an economics event because of what happened *on top of* scarcity.
+
+### 5.2 The diagnosis: economists as forensic analysts
+
+A group of empirical industrial-organization economists—principally Severin Borenstein, James Bushnell, and Frank Wolak, working through the University of California Energy Institute and the CAISO Market Surveillance Committee (which Wolak chaired)—developed and applied a method for *measuring* market power directly: simulate the competitive counterfactual price by stacking actual marginal costs against actual demand (accounting for hydro opportunity costs and imports), and attribute the gap between actual and competitive prices to the exercise of market power. Their central publication, Borenstein, Bushnell, and Wolak, "Measuring Market Inefficiencies in California's Restructured Wholesale Electricity Market" (*American Economic Review*, 2002), found that market power accounted for a large share of the price increases during summer 2000—unilateral withholding by a handful of suppliers facing inelastic, uncontracted demand, not necessarily illegal conspiracy. Joskow and Edward Kahn (2002) reached similar conclusions with complementary methods, documenting substantial output withholding. Scott Harvey and William Hogan contested the magnitudes and the attribution in a series of papers, arguing that cost mis-measurement and operational constraints could explain much of the gap—a debate that, whatever one's verdict, hardened the methodological standards of the entire field. Subsequent enforcement actions and the Enron tapes ("Death Star," "Fat Boy," "Ricochet"—strategies that exploited precisely the seams between California's zonal market and its neighbors that the pool school had warned about) settled the public argument.
+
+The economists' account of *why* California failed became canonical: (i) retail price freezes severed demand from wholesale prices, making demand almost perfectly inelastic; (ii) the utilities were prohibited/discouraged from forward contracting, leaving them exposed to spot prices and giving suppliers maximal incentive to raise them (the Allaz–Vila logic in reverse); (iii) the separation of the PX energy market from the ISO's congestion and reliability functions, and zonal rather than nodal pricing, created arbitrage seams; and (iv) soft price caps invited strategic exports and re-imports. Each element had been flagged in advance by parts of the profession; the crisis converted those warnings into design orthodoxy: contract cover matters, demand response matters, nodal beats zonal, and market power mitigation must be built in, not bolted on.
+
+### 5.3 Institutionalizing economics: the market monitors
+
+The most durable institutional legacy of California is the **market monitor**. Every ISO/RTO now has an independent internal and/or external market monitoring unit—Potomac Economics (led by economist David Patton) serves as independent monitor for several markets (including ERCOT, NYISO, and MISO), Monitoring Analytics (led by economist Joseph Bowring) monitors PJM, and CAISO maintains a Department of Market Monitoring and a Market Surveillance Committee that has been chaired by academic economists (Wolak, Bushnell, and colleagues). These units run conduct-and-impact mitigation screens (automatic offer-capping when suppliers with local market power bid far above reference costs), publish annual State of the Market reports dense with concentration and price-cost-margin analysis, and refer cases to FERC enforcement. Market monitoring is applied industrial organization practiced as a permanent regulatory function—a professional niche that simply did not exist before economists created it.
+
+---
+
+## 6. Resource Adequacy: Missing Money, Capacity Markets, and Scarcity Pricing
+
+### 6.1 The "missing money" problem
+
+In the theoretical ideal, energy-only markets finance capacity through scarcity rents: prices spike toward the value of lost load in shortage hours, and the expectation of such rents induces efficient investment in exactly enough capacity that the marginal plant recovers its fixed costs. In practice, US markets systematically suppress those rents: offer caps (long $1,000/MWh in the East), out-of-market reliability actions by operators who dispatch reserves rather than let prices rise, and mitigation rules together truncate the right tail of the price distribution. The resulting shortfall between what generators need to cover fixed costs and what suppressed energy prices deliver was dubbed the **"missing money" problem**—a term popularized through the work of Peter Cramton and Steven Stoft and central to Joskow's influential treatments (e.g., Joskow 2008). The diagnosis reframed reliability from an engineering mandate into a pricing failure, and the profession split over the remedy: fix the energy prices (scarcity pricing) or bolt on a separate market for capacity.
+
+### 6.2 Capacity markets and the demand-curve innovation
+
+The Eastern ISOs chose capacity markets: mandatory forward auctions in which load-serving entities procure enough accredited capacity to meet a reserve-margin target. Early "vertical demand" designs (a fixed requirement, price collapsing to zero when supply barely exceeded it) proved pathologically volatile and manipulable. The fix—first implemented in NYISO in 2003 and now universal—was the **sloped administrative demand curve**: a downward-sloping schedule anchored at the estimated "cost of new entry" (CONE) that makes the capacity price a smooth function of the surplus margin. The demand curve is pure economic engineering—an administratively constructed marginal-value schedule for reliability—and its parameters (CONE, the shape and anchor points) are set through recurring proceedings that are essentially econometric litigation. Cramton and Stoft's design work (much of it commissioned by the ISOs), together with Joskow's analyses, shaped PJM's Reliability Pricing Model (implemented 2007) and ISO-NE's Forward Capacity Market (first auction 2008), including features such as three-year-forward procurement, locational capacity zones, and—later—"pay for performance" penalty structures (ISO-NE, and PJM's Capacity Performance reforms after the January 2014 polar vortex exposed the weakness of capacity that fails to show up). The subsequent generation of disputes—minimum offer price rules (MOPR) aimed at state-subsidized entrants, buyer-side market power, and the accreditation of intermittent and duration-limited resources—has kept capacity-market economics among the most litigated areas of the field.
+
+### 6.3 The energy-only alternative: ERCOT and the operating reserve demand curve
+
+Texas took the other road. ERCOT—intrastate, outside FERC jurisdiction, restructured under Senate Bill 7 (1999)—runs an energy-only market with no capacity mechanism and, historically, a high offer cap (reaching $9,000/MWh). Its signature mechanism, adopted in 2014, is the **Operating Reserve Demand Curve (ORDC)**, based directly on William Hogan's proposal ("Electricity Scarcity Pricing Through Operating Reserves," 2013): a real-time price adder equal to the loss-of-load probability (as a function of the current reserve level) multiplied by the value of lost load, so that prices rise smoothly and *automatically* toward VoLL as reserves shrink—administratively replicating the scarcity rents that an ideal market with responsive demand would generate. The ORDC is perhaps the purest example in any industry of a formula from an economics working paper being written into the settlement software of a major market. The 2021 crisis (Section 9.4) subjected the design to its most severe test.
+
+### 6.4 The unresolved debate
+
+The capacity-versus-energy-only debate remains live and has been sharpened by decarbonization. Capacity-market advocates emphasize investment-risk reduction and political robustness (regulators will never tolerate true VoLL pricing for long); energy-only advocates (Hogan, and in a different register Wolak, who is skeptical of both constructs and emphasizes contracting obligations) counter that capacity markets pay for "iron in the ground" rather than performance, invite endless administrative gaming, and mute the scarcity prices needed to reward flexibility and demand response. The rise of storage and renewables—whose capacity value (measured by effective load-carrying capability, ELCC, another economics-adjacent construct) depends on penetration—has turned resource-adequacy design into one of the most active current research areas (Section 9).
+
+---
+
+## 7. Mechanism Design in the Engine Room: Auctions, Non-Convexities, and Virtual Bidding
+
+### 7.1 The market as an optimization problem
+
+An ISO's day-ahead market is a sealed-bid, multi-unit, multi-product auction cleared by security-constrained unit commitment and economic dispatch—a mixed-integer program whose scale (tens of thousands of constraints, thousands of resources) makes it among the largest auctions run anywhere. The economic questions are classic mechanism design: What prices support the efficient allocation? What are the incentives to bid truthfully? How should linked products (energy, regulation, spinning reserve) be co-optimized? Economists—often working with operations researchers—have supplied the answers now embedded in the software.
+
+### 7.2 Non-convexities and uplift: the pricing frontier
+
+Power plants have non-convex costs—start-up costs, minimum run levels, minimum up/down times—so a Walrasian equilibrium supporting the efficient commitment may not exist: at the LMP, some committed units lose money and some uncommitted units would profit. US markets patch this with "uplift" or make-whole side payments, which are discriminatory, opaque, and blunt price signals. The theoretical response has been one of the liveliest applied-theory literatures of the past two decades: O'Neill, Sotkiewicz, Hobbs, Rothkopf, and Stewart (2005) showed how to construct market-clearing prices with non-convexities using integer-activity prices; Gribik, Hogan, and Pope (2007) proposed **convex hull pricing** ("extended LMP"), which minimizes total uplift by pricing off the convex envelope of the cost function. Variants of these ideas have been implemented: MISO adopted approximations of convex-hull pricing, and FERC's fast-start pricing orders (2016–2020) pushed ISOs to let inflexible fast-start units set prices. This is an area where economic theory, operations research, and settlement software co-evolve in real time.
+
+### 7.3 Virtual bidding and the two-settlement system
+
+Hogan's multi-settlement architecture separates a financially binding day-ahead market from real-time balancing. To keep the two aligned, markets admit **virtual (convergence) bidding**: purely financial positions that arbitrage expected day-ahead/real-time spreads. The theory says arbitrageurs should converge the two prices, improving unit commitment; the empirical literature broadly confirms it while documenting nuances—Jha and Wolak's work on California finds convergence bidding improved price convergence and productive efficiency, while episodes like the JP Morgan make-whole manipulation cases (settled with FERC in 2013) and studies of loss-related arbitrage in MISO illustrate how financial products interact with market seams. The design of FTR auctions, their chronic revenue underperformance relative to auction prices, and the question of who should bear FTR-portfolio default risk (brought to a head by the 2018 GreenHat default in PJM) have similarly been analyzed principally by economists.
+
+### 7.4 Procurement auctions and default service
+
+Beyond ISO markets, economists designed the auctions through which restructured states procure default retail service—most prominently New Jersey's Basic Generation Service **descending-clock auction**, designed with the involvement of auction economists (including Ausubel and Cramton's circle) and running annually since 2002, and Illinois' procurement events. Milgrom's and Wilson's broader auction-design consulting practices, celebrated in the 2020 Nobel citation, include electricity procurement among their applications. Contract design for renewable procurement (indexed PPAs, contracts-for-differences) is the current frontier of this tradition.
+
+---
+
+## 8. The Demand Side: Dynamic Pricing and the Demand-Response Wars
+
+### 8.1 The economists' oldest complaint
+
+From Boiteux onward, economists have insisted that the demand side is half the market: if retail consumers face time-invariant prices, the price elasticity that should discipline wholesale markets is absent, scarcity cannot clear the market efficiently, and (as California proved) the system is fragile. Borenstein's work in the 2000s (e.g., "The Long-Run Efficiency of Real-Time Electricity Pricing," 2005; Borenstein and Holland 2005 on the distortions when only some customers face real-time prices) made the modern welfare case for **dynamic retail pricing**; Wolak designed and evaluated field experiments (including in Anaheim and internationally) measuring household response to critical-peak pricing; and a large subsequent experimental literature (including Jessoe and Rapson's work on information feedback) quantified how technology (smart meters, automation) raises effective elasticity. Actual adoption remains limited—default flat tariffs persist almost everywhere—which economists attribute to political economy and behavioral frictions rather than to the economics; the gap between the theory's clarity and retail practice is a standing embarrassment the profession continues to probe (including through work on the distributional effects of dynamic pricing).
+
+### 8.2 Order 745 and *EPSA v. FERC*: economists on both sides
+
+Wholesale **demand response**—paying consumers to curtail—produced the sector's most famous economics dispute. FERC Order 745 (2011) required ISOs to pay demand response the full LMP. Alfred Kahn, in one of his final interventions, filed in support of full-LMP compensation; William Hogan argued forcefully that the correct payment is LMP minus the retail rate the customer avoids ("LMP−G"), since paying full LMP to someone who also saves the retail price double-compensates curtailment and subsidizes inefficient demand reduction. The dispute—at bottom about the definition of the counterfactual and the treatment of the retail-rate distortion—went to the Supreme Court as *FERC v. Electric Power Supply Association* (2016), where the Court upheld FERC's jurisdiction and Order 745. Economists' briefs and papers populated both sides; the episode is now a teaching case in how second-best reasoning cuts in opposite directions depending on which distortion one takes as fixed.
+
+---
+
+## 9. The Modern Frontier: Renewables, Storage, Distributed Resources, and Texas 2021
+
+### 9.1 Zero-marginal-cost entry and price formation
+
+Subsidized and mandated wind and solar—driven by production tax credits, investment tax credits, and state renewable portfolio standards—entered wholesale markets in volumes that transformed price formation: negative prices in windy regions (a predictable consequence of the PTC paying generators to produce, analyzed early by economists), the California "duck curve," depressed mid-day energy prices, and merit-order displacement of thermal units. The economics literature responded on several fronts: measuring the market value decline of variable renewables as penetration rises; quantifying the emissions and price effects of renewable entry; analyzing how out-of-market subsidies interact with capacity markets (the MOPR wars); and pressing the first-best alternative—carbon pricing—in ISO markets, including formal proposals for carbon-adder designs in NYISO and PJM analyzed by economists at Resources for the Future and elsewhere. The broad professional consensus that technology-neutral carbon pricing dominates technology-specific mandates has had, it must be said, limited legislative success; economists have therefore increasingly turned to analyzing second-best instrument design (clean-energy standards, ELCC-based accreditation, hybrid procurement).
+
+### 9.2 Storage and distributed resources: Orders 841 and 2222
+
+FERC Order 841 (2018) required ISOs to create participation models letting storage set prices as both buyer and seller; Order 2222 (2020) extended participation to aggregations of distributed energy resources. Both orders resolve questions economists had framed: storage arbitrage and its welfare effects, the double-charging of storage for transmission, and the treatment of behind-the-meter resources. A growing empirical literature (on ERCOT and CAISO battery fleets) now measures how quickly storage competes away its own arbitrage margins—a clean test of entry economics playing out in real time.
+
+### 9.3 Transmission, again
+
+Transmission planning and cost allocation—the subject of Joskow and Tirole's theoretical work on merchant transmission and of Hogan's beneficiary-pays principle—returned to the center with Order 1000 (2011) and Order 1920 (2024), which require forward-looking regional planning with benefit-based cost allocation. The "beneficiary pays" standard that courts (notably the Seventh Circuit in the *Illinois Commerce Commission* cases, in opinions by Judge Posner—an economist-judge) have enforced is straight cost-benefit economics.
+
+### 9.4 Winter Storm Uri (February 2021): stress-testing the energy-only design
+
+The Texas blackout of February 2021—days of rotating outages, prices administratively held at the $9,000/MWh cap for over four days, roughly 200+ deaths, and tens of billions of dollars of financial redistribution—triggered the most intense economic post-mortem since California. The economics discussion separated several issues: the failure was primarily one of *supply availability* (unweatherized gas supply and generation) rather than market design per se; scarcity pricing performed as designed but the duration of cap-level prices exposed the absence of retail hedging (the Griddy customers on wholesale pass-through rates) and raised the question whether VoLL-level prices sustained for days are politically or contractually tolerable; and the PUCT's decision to hold prices at the cap after load shed ended became a celebrated dispute about out-of-market intervention. Peter Cramton, then an ERCOT board member (who, with other out-of-state directors, resigned after the storm), Hogan, Wolak, Bushnell, and many others produced dueling analyses; the aftermath brought a lower price cap, a modified ORDC, weatherization mandates, and a new Performance Credit Mechanism proposal—each debated in explicitly economic terms. Uri also revived academic interest in the interaction between gas and electricity market design, and in mandatory retail/load-serving hedging obligations of the kind Wolak has long advocated.
+
+### 9.5 Resource adequacy for a decarbonized grid
+
+The current research frontier asks whether the LMP-plus-scarcity architecture, designed for a fuel-burning fleet with meaningful marginal costs, remains adequate for a system dominated by zero-marginal-cost, weather-driven, and duration-limited resources. Questions under active study by economists include: price formation when the marginal resource is storage or curtailed renewables (opportunity-cost pricing); capacity accreditation via marginal ELCC; long-duration adequacy and tail risk (climate-correlated outages); the financing of capital-intensive clean firm resources under volatile revenue streams (motivating contract-for-differences and reliability options, the latter a Cramton–Stoft design adopted in Colombia and ISO-NE's FCM in modified form); and whether organized markets should evolve toward centralized long-term procurement—"hybrid markets"—a debate joined by Joskow (whose recent papers argue the pure energy-market model is under increasing strain), Schmalensee, Wolak, Hogan, and a younger generation. The wheel, in other words, is turning again, and economists are once more on both sides of it.
+
+---
+
+## 10. Did It Work? The Empirical Assessment Literature
+
+Restructuring created a natural experiment—organized-market regions versus traditionally regulated regions; divested plants versus utility-retained plants—that empirical economists have mined for two decades. The headline findings:
+
+- **Operating efficiency.** Fabrizio, Rose, and Wolfram (*AER*, 2007) found that investor-owned plants in restructuring states reduced labor and nonfuel operating expenses several percent relative to those in non-restructuring states—evidence that competition (or its anticipation) sharpened cost discipline. Davis and Wolfram (*AEJ: Applied*, 2012) found that divestiture of nuclear plants to independent merchant operators raised capacity factors by roughly 10 percentage points, a large efficiency gain concentrated in reduced outage durations. Cicala (*AER*, 2015) showed deregulated plants procured coal more cheaply once cost pass-through incentives were removed.
+- **Dispatch efficiency.** Cicala (*AER*, 2022) used the staggered expansion of ISO markets to estimate that market-based dispatch reduced production costs on the order of billions of dollars per year by reallocating output toward lower-cost plants and increasing gains from trade across utility boundaries—the cleanest evidence that the LMP machinery does what its designers promised.
+- **Investment and technology mix.** Merchant investment responded to price signals with technology choices (fast, modular gas; later renewables and batteries) very different from the regulated era's, and the literature documents both the responsiveness and the boom-bust cycles that pure merchant exposure produces.
+- **Retail competition.** The retail literature is more mixed: evidence from Texas and elsewhere documents meaningful search frictions, price dispersion, and confusion pricing in residential retail choice (with Hortaçsu, Madanizadeh, and Puller's 2017 study of Texas a benchmark), tempering the more expansive claims for retail liberalization even where wholesale gains are accepted.
+- **Prices.** Simple regulated-versus-restructured retail price comparisons are confounded (restructured states were high-cost to begin with; gas price cycles dominate), and the profession has largely converged on the view that the welfare gains of restructuring show up in costs, dispatch, and plant performance rather than in unambiguous retail price declines—an honest, if politically unsatisfying, verdict.
+
+Alongside evaluation, electricity became empirical IO's favorite laboratory: because engineering marginal costs are measurable, researchers can compute markups directly rather than infer them, and the industry has consequently generated foundational studies of oligopoly bidding (Wolfram on the England–Wales pool; Hortaçsu and Puller on ERCOT bidders' deviations from optimal bidding), forward contracting, auction behavior, and environmental policy interactions. The methodological traffic runs both ways: electricity data disciplined IO theory, and IO tools staffed the market monitors.
+
+---
+
+## 11. Critiques, Countercurrents, and the Limits of Design
+
+An honest survey must record that economists' stewardship of electricity restructuring has critics, including within the profession.
+
+First, **the promised consumer savings were oversold** in the 1990s political campaigns, and the profession's more careful voices (Joskow prominent among them) spent years distinguishing what the evidence supports from what advocates claimed. Second, **complexity itself is a cost**: the layered edifice of energy, ancillary, capacity, FTR, and virtual products—each patching the previous layer's incentive problems—has been criticized (from the left as designed-for-traders opacity, from parts of the engineering community as fragile, and by public-power advocates as an expensive detour). Third, **the governance critique**: ISO stakeholder processes and FERC litigation allocate rents through processes in which well-funded incumbents are structurally advantaged—a Stiglerian observation that sits uncomfortably with the design tradition's technocratic self-image. Fourth, **the decarbonization critique**: state clean-energy policies now drive most investment, and the resulting collision between subsidized entry and market price formation (MOPR, hybrid-market proposals) suggests to some scholars that the 1990s architecture is being quietly superseded by a return to planned procurement with markets relegated to short-run balancing—"markets for dispatch, planning for investment." Whether that constitutes the failure of the design program or its adaptive success is perhaps the central interpretive question of the field's next decade, and—fittingly—it is being argued out by the same cast of economists, their students, and their students' students.
+
+---
+
+## 12. Conclusion
+
+The arc surveyed here runs from Boiteux's peak-load tariffs at EDF, through the Chicago and MIT critiques of regulation, Schweppe's spot prices, and Hogan's contract networks, to the five-minute nodal prices that today clear most American wholesale electricity. Along the way economists built institutions (ISOs, market monitors, capacity auctions), fought and refereed crises (California, Uri), created a permanent professional apparatus of market design and surveillance, and produced one of empirical economics' richest bodies of evidence on what competition and regulation each do well. Three lessons generalize beyond electricity. First, where physics or technology preclude decentralized exchange, markets are engineered artifacts, and the quality of the engineering—the mechanism design—determines whether competition delivers its textbook benefits. Second, transitions are governed by second-best problems (stranded costs, retail-price distortions, subsidized entry) at least as much as by first-best blueprints, and the profession's most valuable interventions have often been diagnostic rather than architectural. Third, market design is never finished: each solved problem (transmission access, congestion, missing money) exposes the next (non-convexities, accreditation, deep decarbonization). Forty years in, the US electricity markets remain what they were at the start—the most ambitious ongoing experiment in applied economics anywhere in the world.
+
+---
+
+## References (selected)
+
+- Allaz, B., and J.-L. Vila (1993). "Cournot Competition, Forward Markets and Efficiency." *Journal of Economic Theory* 59(1): 1–16.
+- Averch, H., and L. Johnson (1962). "Behavior of the Firm under Regulatory Constraint." *American Economic Review* 52(5): 1052–1069.
+- Boiteux, M. (1949). "La tarification des demandes en pointe." *Revue Générale de l'Électricité*; English translation, "Peak-Load Pricing," *Journal of Business* 33 (1960): 157–179.
+- Boiteux, M. (1956). "Sur la gestion des monopoles publics astreints à l'équilibre budgétaire." *Econometrica* 24(1): 22–40.
+- Borenstein, S. (2005). "The Long-Run Efficiency of Real-Time Electricity Pricing." *The Energy Journal* 26(3): 93–116.
+- Borenstein, S., J. Bushnell, and F. Wolak (2002). "Measuring Market Inefficiencies in California's Restructured Wholesale Electricity Market." *American Economic Review* 92(5): 1376–1405.
+- Borenstein, S., and S. Holland (2005). "On the Efficiency of Competitive Electricity Markets with Time-Invariant Retail Prices." *RAND Journal of Economics* 36(3): 469–493.
+- Christensen, L., and W. Greene (1976). "Economies of Scale in U.S. Electric Power Generation." *Journal of Political Economy* 84(4): 655–676.
+- Cicala, S. (2015). "When Does Regulation Distort Costs? Lessons from Fuel Procurement in US Electricity Generation." *American Economic Review* 105(1): 411–444.
+- Cicala, S. (2022). "Imperfect Markets versus Imperfect Regulation in US Electricity Generation." *American Economic Review* 112(2): 409–441.
+- Cramton, P., and S. Stoft (2005). "A Capacity Market that Makes Sense." *The Electricity Journal* 18(7): 43–54.
+- Davis, L., and C. Wolfram (2012). "Deregulation, Consolidation, and Efficiency: Evidence from US Nuclear Power." *American Economic Journal: Applied Economics* 4(4): 194–225.
+- Demsetz, H. (1968). "Why Regulate Utilities?" *Journal of Law and Economics* 11(1): 55–65.
+- Fabrizio, K., N. Rose, and C. Wolfram (2007). "Do Markets Reduce Costs? Assessing the Impact of Regulatory Restructuring on US Electric Generation Efficiency." *American Economic Review* 97(4): 1250–1277.
+- Green, R., and D. Newbery (1992). "Competition in the British Electricity Spot Market." *Journal of Political Economy* 100(5): 929–953.
+- Gribik, P., W. Hogan, and S. Pope (2007). "Market-Clearing Electricity Prices and Energy Uplift." Harvard Electricity Policy Group working paper.
+- Hogan, W. (1992). "Contract Networks for Electric Power Transmission." *Journal of Regulatory Economics* 4(3): 211–242.
+- Hogan, W. (2013). "Electricity Scarcity Pricing Through Operating Reserves." *Economics of Energy & Environmental Policy* 2(2): 65–86.
+- Hortaçsu, A., and S. Puller (2008). "Understanding Strategic Bidding in Multi-Unit Auctions: A Case Study of the Texas Electricity Spot Market." *RAND Journal of Economics* 39(1): 86–114.
+- Hortaçsu, A., S. Madanizadeh, and S. Puller (2017). "Power to Choose? An Analysis of Consumer Inertia in the Residential Electricity Market." *American Economic Journal: Economic Policy* 9(4): 192–226.
+- Joskow, P. (2008). "Capacity Payments in Imperfect Electricity Markets: Need and Design." *Utilities Policy* 16(3): 159–170.
+- Joskow, P., and E. Kahn (2002). "A Quantitative Analysis of Pricing Behavior in California's Wholesale Electricity Market During Summer 2000." *The Energy Journal* 23(4): 1–35.
+- Joskow, P., and R. Schmalensee (1983). *Markets for Power: An Analysis of Electric Utility Deregulation.* MIT Press.
+- Joskow, P., and J. Tirole (2005). "Merchant Transmission Investment." *Journal of Industrial Economics* 53(2): 233–264.
+- Kahn, A. (1970–71). *The Economics of Regulation: Principles and Institutions.* Wiley (2 vols.).
+- Klemperer, P., and M. Meyer (1989). "Supply Function Equilibria in Oligopoly under Uncertainty." *Econometrica* 57(6): 1243–1277.
+- Laffont, J.-J., and J. Tirole (1993). *A Theory of Incentives in Procurement and Regulation.* MIT Press.
+- O'Neill, R., P. Sotkiewicz, B. Hobbs, M. Rothkopf, and W. Stewart (2005). "Efficient Market-Clearing Prices in Markets with Nonconvexities." *European Journal of Operational Research* 164(1): 269–285.
+- Peltzman, S. (1976). "Toward a More General Theory of Regulation." *Journal of Law and Economics* 19(2): 211–240.
+- Schweppe, F., M. Caramanis, R. Tabors, and R. Bohn (1988). *Spot Pricing of Electricity.* Kluwer Academic Publishers.
+- Steiner, P. (1957). "Peak Loads and Efficient Pricing." *Quarterly Journal of Economics* 71(4): 585–610.
+- Stigler, G. (1971). "The Theory of Economic Regulation." *Bell Journal of Economics and Management Science* 2(1): 3–21.
+- Turvey, R. (1968). *Optimal Pricing and Investment in Electricity Supply.* MIT Press.
+- Vickrey, W. (1961). "Counterspeculation, Auctions, and Competitive Sealed Tenders." *Journal of Finance* 16(1): 8–37.
+- Williamson, O. (1966). "Peak-Load Pricing and Optimal Capacity under Indivisibility Constraints." *American Economic Review* 56(4): 810–827.
+- Wilson, R. (2002). "Architecture of Power Markets." *Econometrica* 70(4): 1299–1340.
+- Wolak, F. (2003). "Measuring Unilateral Market Power in Wholesale Electricity Markets: The California Market, 1998–2000." *American Economic Review* 93(2): 425–430.
+- Wolfram, C. (1999). "Measuring Duopoly Power in the British Electricity Spot Market." *American Economic Review* 89(4): 805–826.

us_market/economists_survey_zh.md

@@ -0,0 +1,272 @@
+# 经济学家如何塑造美国电力市场：理论、实践与关键里程碑
+
+**一篇综述论文**
+
+---
+
+## 摘要
+
+很少有哪个行业像美国电力部门那样，被学院派经济学如此彻底地重新设计。在大约四十年的时间里，经济学家提出了瓦解发电环节"成本加成"监管体制的思想批判，发展出支撑美国所有集中式电力批发市场的定价理论，诊断了第一代市场设计的失败，并持续塑造着有关容量、需求响应、储能以及可再生能源并网的市场规则。本综述从四个维度追溯这一影响：（一）电力市场化改革在监管经济学中的思想渊源；（二）现代市场设计的理论基础，尤其是峰荷定价、现货定价与节点边际电价（LMP）；（三）从《公用事业监管政策法》（PURPA，1978）到联邦能源监管委员会（FERC）第888、2000、745、841和2222号法令等一系列将经济学思想转化为监管规则的制度里程碑；（四）经济学家借以诊断危机（2000–01年加州危机、2021年得州危机）、建立市场监测机构、并评估市场化改革是否兑现其效率承诺的实证与应用文献。我们认为，美国电力市场化改革最恰当的理解不是"放松管制"（deregulation），而是一场持续的应用机制设计实践——经济学家在其中从监管的批评者转变为制度的设计师。本文最后讨论了经济理论再次被推向前台的若干悬而未决的设计问题：深度脱碳背景下的资源充裕性、非凸定价问题以及"混合市场"之争。
+
+---
+
+## 1. 引言
+
+1996年4月，美国联邦能源监管委员会（FERC）颁布第888号法令，要求对州际输电网实行开放、非歧视性接入，由此启动了自20世纪30年代以来美国电力工业最深刻的一次重组。不到十年，美国三分之二的电力需求已通过独立系统运营商（ISO）和区域输电组织（RTO）运营的集中式批发市场来满足。在这些市场中，电能价格每五分钟在数千个不同的网络节点上计算一次，其求解的是一个大规模优化问题，而该问题的目标函数——在电力潮流物理规律约束下最大化社会剩余——正是直接来自福利经济学。
+
+这最后一点正是本综述的核心主题。在大多数经历放松管制的行业——航空、货运、电信、天然气——经济学家提供的是支持竞争的*论证*，市场随后通过分散的企业家活动自行演化。电力则不同。因为电力潮流服从的是基尔霍夫定律而非合同条款，因为电能（直到最近）在经济上无法储存，因为供需必须每秒每刻保持平衡以避免连锁停电，一个可运转的电力"市场"不可能自发涌现，它必须被*设计*出来。其结果是，经济学家不仅仅是电力竞争的倡导者——他们撰写了电力市场的宪法。美国所有集中式市场核心的定价算法（节点边际电价）、对冲阻塞风险的金融工具（金融输电权）、采购未来容量的拍卖机制、在得克萨斯州替代容量市场的稀缺定价机制，以及监督上述一切的市场力缓解机制，在异乎寻常的程度上都是学院派经济学研究的直接产物。
+
+本综述为一般经济学读者梳理这段历史。第2节追溯市场化改革在战后监管经济学与自然垄断理论中的思想渊源。第3节展开理论核心：峰荷定价、现货定价以及Hogan的节点边际电价框架。第4节回顾这些思想成为法律的立法与监管里程碑。第5节考察2000–01年加州危机——正是在这一事件中，实证经济学家证明了自己作为市场"诊断医师"的价值，并使经济学市场监测永久性地制度化。第6节讨论资源充裕性问题——"缺失的货币"（missing money）、容量市场与纯电能量市场的替代方案。第7节综述机制设计的贡献：拍卖、非凸定价与虚拟报价。第8节转向需求侧：动态零售定价与围绕需求响应的论战。第9节讨论当代前沿——可再生能源、储能、分布式资源与2021年得州危机。第10节回顾评估市场化改革效果的实证文献。第11节讨论各种批评意见以及正在兴起的"混合市场"之争，第12节为结论。
+
+三点说明。第一，本文聚焦于美国；1990年英国电力私有化尽管对美国的思考影响巨大（其本身也深受Stephen Littlechild等经济学家的塑造），在此仅作为背景出现。第二，"经济学家"作宽泛理解，包括麻省理工学院（MIT）等机构的工程经济学传统——Fred Schweppe是电气工程师，但《电力现货定价》（*Spot Pricing of Electricity*）是一部经济学著作。第三，如此篇幅的综述不可能面面俱到；我们的目标是准确勾勒出主要的思想脉络与制度转折点。
+
+---
+
+## 2. 思想渊源：监管经济学及其不满
+
+### 2.1 旧体制及其理论依据
+
+从20世纪20年代到80年代，美国电力工业的组织形式是：纵向一体化的投资者所有制公用事业公司持有排他性的零售特许经营权，价格由州公用事业委员会按"服务成本"（cost-of-service）原则核定，批发与州际事务则由1935年《联邦电力法》管辖。其理论依据是自然垄断理论：由于固定成本巨大，且（当时人们相信）发电、输电、配电各环节都存在普遍的规模经济，竞争被认为是浪费的或不可持续的，受监管的垄断则是"最不坏"的制度安排。
+
+经济学家参与了这一体制的创建——"公允价值基础上的公允回报率"这套方法源自制度学派经济学以及20世纪初的费率基数诉讼——但经济学界最重要的贡献出现在其后，其形式是批判。
+
+### 2.2 战后的批判
+
+战后经济学的四条思想脉络汇聚在一起，动摇了人们对受监管垄断模式的信心。
+
+**激励扭曲。** Averch与Johnson（1962）形式化地证明，回报率监管会诱使公用事业公司过度投资于资本——即所谓"镀金"效应或A–J效应。无论该效应的实证规模如何（至今仍有争议），这篇论文将监管重新定义为一个激励问题而非会计问题，并为后来的激励性监管文献（Laffont与Tirole 1993）播下了种子，这一文献后来又影响了绩效导向的费率制定。
+
+**监管俘获。** Stigler（1971）与Peltzman（1976）指出，监管是提供给政治上有影响力的利益集团的，而非服务于公共利益；Demsetz（1968）则追问：既然"服务权"本身可以通过拍卖出售——即以特许经营权竞标实现"为市场而竞争"——自然垄断为何还需要监管？这些芝加哥学派的贡献侵蚀了"监管可靠地矫正市场失灵"这一预设。
+
+**关于规模经济的证据。** 实证研究——最有影响的是Christensen与Greene（1976）——发现到20世纪70年代，美国大多数大型发电企业在电厂与企业层面的规模经济已经耗尽。如果发电不再是自然垄断，那么对其实施监管的理由便不复存在，即便电网环节仍然是自然垄断。这一"拆分"（unbundling）洞见——在可行之处引入竞争（发电、零售），在必要之处保留监管（输电、配电）——成为整个市场化改革的总蓝图。
+
+**放松管制运动的实践。** Alfred Kahn的两卷本《监管经济学》（1970–71）提供了理论综合，而Kahn本人作为卡特总统任内的民用航空委员会主席，证明了一位学院派经济学家可以从体制内部拆解一个监管体制。航空（1978）、货运（1980）、铁路（1980）与天然气（1978–92）的放松管制，为攻克电力这一公认最难啃的硬骨头，同时提供了政治模板与职业自信。
+
+### 2.3 直接诱因与那本关键著作
+
+20世纪70年代摧毁了公用事业行业的战后均衡：石油危机、核电建设经济性的崩溃（伴随著名的成本超支事件与华盛顿公共电力供应系统WPPSS违约）、需求增长停滞，以及两位数的电价上涨，使监管者与消费者一同背弃了原有模式。1978年的《公用事业监管政策法》（PURPA，详见第4节）无意间进行了一场自然实验，证明非公用事业发电商同样能够建设并运营电厂。
+
+正是在这一背景下，Paul Joskow与Richard Schmalensee的《电力市场：电力公用事业放松管制分析》（*Markets for Power*，1983）问世，成为有史以来对电力市场化改革最具影响力的经济学分析。Joskow与Schmalensee做了两件事。第一，他们系统性地驳斥了"现状即有效率"的论断，翔实记录了服务成本监管的激励问题。第二——这一点常被遗忘——他们对批发竞争持深度*审慎*态度，逐一列举了天真的放松管制将遭遇的技术障碍（环流、可靠性外部性、输电受限地区的市场力）。这本书的持久贡献在于界定了研究议程：他们指出的每一个障碍，在随后二十年里都变成了经济学家着手求解的设计问题。Joskow此后四十年的研究产出——涉及输电投资、合同结构、容量市场，直至最近的脱碳问题——构成了整个领域的连接组织；而他作为顾问、董事会成员和无数监管程序中的专家证人的角色，使他堪称电力行业历史上最具影响力的一位经济学家。
+
+---
+
+## 3. 理论基础：从峰荷定价到节点边际电价
+
+### 3.1 边际成本定价与峰荷问题
+
+深层理论始于战后法国。法国电力公司（EDF）的经济学家兼工程师Marcel Boiteux（Maurice Allais的学生）在20世纪40年代末至50年代研究出：当需求随时间波动且容量成本高昂时，如何按边际成本为电力定价——非高峰时段用户只需支付电能成本，高峰用户则需支付电能成本加上容量的边际成本（Boiteux 1949、1956；其英译本主要经由Nelson 1964介绍给英语世界）。Steiner（1957）在美国文献中独立发展了峰荷定价模型，Williamson（1966）完善了其福利分析。Boiteux关于预算约束下次优定价的姊妹篇研究（拉姆齐–布瓦特定价）成为以最小扭曲回收固定成本的标准框架。EDF于1956年实施的"绿色电价"（tarif vert）证明这些绝非黑板上的空谈。
+
+另外两项要素补齐了古典工具箱。Turvey（1968）等人将峰荷定价与投资规划联系起来。而不确定性与配给条件下的定价理论——当容量短缺时，有效率的价格是什么？——则指向了后来主导资源充裕性论争的核心概念：**失负荷价值**（Value of Lost Load, VoLL），即消费者为避免停电而愿意支付的金额。当系统逼近非自愿限电时，有效率的稀缺价格应向VoLL攀升；从ERCOT的价格上限到容量市场的需求曲线，一切设计都建立在这一个思想之上。
+
+### 3.2 现货定价：Schweppe与MIT学派
+
+从电价理论迈向市场设计的决定性一步，是在20世纪70年代末至80年代由MIT的Fred Schweppe——一位电气工程师——与经济学家及工程师Michael Caramanis、Richard Tabors、Roger Bohn合作完成的。他们的研究计划以《电力现货定价》（Schweppe, Caramanis, Tabors, and Bohn 1988）一书达到顶峰，提出的问题是：如果承认电力的边际成本不仅随时段变化，而且是连续的、随机的、并且——关键地——随电网位置而变化，那么电力的*真实*边际成本究竟是什么？
+
+他们的答案是：每一时刻、电网每一母线上的有效率现货价格，等于系统lambda（参考节点的电能边际成本）经边际输电损耗和有约束力的输电约束的影子价格调整后的值。一旦电网发生阻塞，各地价格便会不同，因为基尔霍夫定律使廉价电力无法送达高价地区。Schweppe等人设想以这些价格为基础，实现电力系统的去中心化"自稳态"（homeostatic）控制：发电商和（原则上）消费者对实时价格作出反应，就能复现中央调度工程师历来计算的最小成本调度——但这一次是通过市场实现。这本书写成之时，尚不存在任何能够实施它的机构；它后来成为此后所有市场设计的技术圣经。
+
+### 3.3 Hogan：节点边际电价与金融输电权
+
+将现货定价理论转化为可实施市场架构的人，是哈佛大学肯尼迪学院的William Hogan。从《电力输送的合同网络》（Hogan 1992）开始，Hogan解决了长期困扰去中心化方案的难题：在交流电网中，电力潮流无法按合同路径流动，如何定义可交易的输电权？
+
+Hogan的洞见分为两部分。第一，拥抱而非对抗集中调度：让独立系统运营商执行*基于报价的安全约束经济调度*——这与公用事业公司历来求解的优化问题相同，只是报价曲线改由相互竞争的发电商提交——并让该优化问题的对偶变量（影子价格）定义每个节点的市场出清价格，即**节点边际电价（LMP）**。节点的LMP恰好按Schweppe的公式分解为：电能成本＋网损＋阻塞。第二，以*金融*而非物理方式定义输电权：一份从节点A到节点B的**金融输电权（FTR）**，赋予其持有者按指定电量获得阻塞租金差额LMP(B) − LMP(A)的权利。只要基础注入量在物理上可行，FTR就同时可行（"收入充足性"定理）；对于匹配的物理计划，FTR能够完美对冲阻塞风险；并且FTR可以拍卖或分配给历史电网用户——从而在解决对冲这一经济问题的同时，也解决了转型的政治问题。
+
+这一"电力库＋FTR"或称"一体化"架构——集中调度、节点电价、金融权利、双结算（日前与实时）、电能与辅助服务联合优化——在20世纪90年代与另一种"去中心化/双边"愿景（与部分加州设计者及英国电力库的批评者相关联）展开了激烈较量，后者主张自计划、分区电价与物理输电权。经验给出的裁决毫不含糊：PJM在分区电价遭到投机套利后于1998年放弃分区改用节点电价；纽约于1999年以LMP（及其FTR变体——输电阻塞合同TCC）启动市场；新英格兰于2003年转轨；中西部ISO于2005年以节点制启动；加州在危机之后于2009年围绕完全节点电价重组市场；得克萨斯于2010年从分区转向节点；西南电力库（SPP）于2014年跟进。如今美国所有集中式市场都运行在Hogan的架构之上。很难在其他任何行业找到这样的例子：一位经济学家的设计方案，每五分钟在一个大陆经济体的大部分区域内被执行一次。
+
+Hogan后续的工作将该框架延伸到几乎每一个有争议的设计维度：多重结算体系、FTR期权与义务、经由运行备用需求曲线实现的稀缺定价（见第6.4节）、附加费用（uplift）与非凸定价（与Gribik和Pope合作）、输电投资激励，以及——延续至今的——脱碳电网的定价问题。Harvey与Hogan同伯克利实证学派围绕加州问题的持续论战（见第5节），也为市场力测度确立了方法论标准。
+
+### 3.4 理论的配角群像
+
+另有三支理论文献汇入了设计典籍：
+
+- **拍卖与机制设计。** Vickrey（1961）的奠基性工作；Wilson–Milgrom关于共同价值与多单位需求拍卖的研究计划；以及Robert Wilson的《电力市场的架构》（*Architecture of Power Markets*，Econometrica，2002）——该文将电力设计表述为一个"市场架构"问题，即把一个经济问题分解为一系列相互衔接的拍卖子市场。Wilson本人曾为国际及加州的电力市场设计提供咨询，Paul Milgrom的咨询实践也将拍卖设计延伸至电力采购领域。2020年授予Milgrom与Wilson的诺贝尔经济学奖，其表彰的工作明确将电力列为应用领域之一。
+- **供给函数均衡。** Klemperer与Meyer（1989）建模了不确定性下以供给曲线展开的竞争——这正是发电商提交给ISO的东西；Green与Newbery（1992）将供给函数均衡应用于英国电力库，为电力市场提供了标准的寡头模型，并较早发出了有影响力的警告：双寡头电力库的定价将远高于边际成本。这一跨大西洋的结论深刻影响了美国对市场结构与资产剥离要求的思考。
+- **合同与纵向安排。** Joskow早年关于长期煤炭合同与资产专用性的实证研究（属威廉姆森传统），以及Allaz–Vila（1993）关于远期合约可以缓解现货市场力的结论，共同影响了过渡性合同的设计，以及一项经久不衰的政策偏好——保持高合同覆盖率。这是经济学家从加州（无合同覆盖，酿成灾难）与其他市场的对比中提炼出的最清晰教训之一。
+
+---
+
+## 4. 制度里程碑：思想成为法律
+
+### 4.1 PURPA（1978）：一场意外的实验
+
+1978年《公用事业监管政策法》（PURPA）作为能源危机立法获得通过，它要求公用事业公司按其*可避免成本*（avoided cost）购买"合格设施"（热电联产及小型可再生能源）的电力。可避免成本当然是一个经济学概念——即公用事业公司因此免于发生的边际成本——其落地引发了上千场费率听证，经济学家在其中就如何测算该成本出庭作证。PURPA更深层的意义是无心插柳：它引致的非公用事业发电浪潮（尤其在加州、纽约与得克萨斯）证明独立发电商有能力融资、建设并运营电厂，从而摧毁了"发电必须依托纵向一体化垄断"的经验前提。包括Joskow在内的经济学家们，既记录了这一示范效应，也记录了行政核定可避免成本合同的种种弊端——当定价过高时（如加州的"标准报价4号"合同），电力用户被迫背负高于市场水平的成本，这留下了一个关于"行政定价对拍卖定价"的长久教训。
+
+### 4.2 1992年《能源政策法》与FERC第888/889号法令（1996）
+
+1992年《能源政策法》创设了"豁免批发发电商"，将独立发电从《公用事业控股公司法》中解放出来，并授权FERC逐案下令开放输电。FERC在1996年第888号法令中将其普遍化：所有拥有输电资产的公用事业公司必须提交开放接入电价表，以其向自身提供服务的同等条件向第三方提供输电服务；第889号法令则建立了OASIS信息系统与职能拆分要求。其经济逻辑——瓶颈垄断环节（电网）不得利用其地位向潜在竞争性环节（发电）延伸市场势力——正是产业组织学中"关键设施/纵向封锁"的推理，由各方阵营中的一大批经济学家向FERC陈述。第888号法令还认可了"搁浅成本"的回收，这本身就是一个可观的经济学文献主题（William Baumol、Alfred Kahn等人曾就回收究竟是有效率的转型补偿还是不当纾困展开论战）。
+
+### 4.3 州层面的重组：加州AB 1890法案与东北部各州
+
+*零售*市场的重组属于各州事务。加州行动最早、最为激进：AB 1890法案（1996）落实了加州公用事业委员会经多年听证形成的决定（学院派经济学家是听证的核心证人），设立了电力交易所（PX）与加州独立系统运营商（CAISO，于1998年3月31日启动），要求公用事业公司剥离大部分化石能源发电资产，将零售电价冻结在此前水平的九折，并且——祸根就此埋下——不鼓励长期购电合同，把采购推向日前与实时市场。这一设计是双边/去中心化学派与电力库学派之间一个双方都不满意的折中；数位知名经济学家（包括后来组成CAISO市场监察委员会的成员）在市场启动前就已警告：供给紧张、需求无合同覆盖、零售价格封顶与市场力暴露的组合十分危险。马萨诸塞、罗德岛、宾夕法尼亚、纽约、新泽西、马里兰、伊利诺伊、俄亥俄、密歇根与得克萨斯在1996至1999年间相继通过各自的重组法案，通常将零售选择权与发电资产剥离及转型成本附加费配套推行。到2000年，约半数州已颁布重组法案或启动相关程序。
+
+### 4.4 第2000号法令（1999）与"标准市场设计"插曲（2002–05）
+
+FERC第2000号法令鼓励（但未强制）组建区域输电组织（RTO）——具备特定特征与职能的独立电网运营机构。ISO/RTO这一制度安排的理论依据——由独立、非营利的运营商运行开放的、基于报价的市场——是Hogan、Joskow等人为解决输电独立性问题而发展出来的。2002年，FERC试图更进一步：《标准市场设计》（SMD）规则提案拟在全国范围内强制推行完整的节点架构——LMP、FTR、日前/实时结算、市场监测与资源充裕性机制。SMD是经济学家对联邦政策直接影响力的最高峰：其文本读起来就像市场设计文献的摘要。它也激起了南部与西部各州（电价低廉、由监管体制供电、对市场毫无兴趣的地区）的激烈政治反弹，FERC于2005年正式撤回提案。其结果便是今日的版图：集中式节点市场覆盖美国约三分之二的用电负荷（PJM、MISO、CAISO、ERCOT、NYISO、ISO-NE、SPP），东南部与西部大部分地区仍维持纵向一体化——这一格局构成了一场持久的自然实验，实证研究者日后将充分加以利用（见第10节）。
+
+### 4.5 后续联邦法令
+
+此后的FERC规则制定（各项将在后文相应章节讨论）继续把经济分析转化为规则：第719号法令（2008）涉及需求响应与ISO治理；第745号法令（2011）涉及需求响应补偿；第755号法令（2011）涉及按绩效付费的调频补偿；第1000号法令（2011）涉及输电规划与成本分摊——Joskow与Jean Tirole曾就该主题撰写奠基性理论；第841号法令（2018）涉及储能参与市场；第2222号法令（2020）涉及分布式能源资源聚合。模式始终如一：一个设计问题浮出水面；经济学家撰写论文、提交专家意见；FERC以规则形式确立一个（有争议的）解决方案；随之而来的是诉讼与更多论文。
+
+---
+
+## 5. 火的考验：加州危机与市场监测的兴起
+
+### 5.1 危机
+
+2000年5月至2001年6月间，加州批发电价均值升至此前数年水平的五到十倍；全州遭遇轮流停电；太平洋燃气电力公司（PG&E）进入破产程序；南加州爱迪生公司濒临破产；电力交易所停止运营；州政府为购电耗资数百亿美元。危机的直接原因包括干旱（西北水电出力下降）、天然气与氮氧化物排放许可价格上涨、需求增长，以及市场供需的确紧张。但这场危机之所以成为一个经济学事件，在于稀缺之*上*又发生了什么。
+
+### 5.2 诊断：经济学家充当法医
+
+一批实证产业组织经济学家——以Severin Borenstein、James Bushnell与Frank Wolak为代表，依托加州大学能源研究所以及由Wolak担任主席的CAISO市场监察委员会——发展并应用了一种直接*测度*市场力的方法：以实际边际成本叠加实际需求（并考虑水电机会成本与外购电力），模拟出完全竞争情形下的反事实价格，再将实际价格与竞争性价格之间的差距归因于市场力的行使。他们的核心论文——Borenstein、Bushnell与Wolak，《测度加州重组后批发电力市场的市场无效率》（*American Economic Review*，2002）——发现市场力解释了2000年夏季价格上涨的很大一部分：在需求缺乏弹性且无合同覆盖的情况下，少数供应商单边持留产能所致，未必是非法共谋。Joskow与Edward Kahn（2002）用互补的方法得出类似结论，记录了大量的产出持留行为。Scott Harvey与William Hogan则在一系列论文中对上述测算的量级与归因提出质疑，认为成本测度误差与运行约束足以解释差距的大部分——无论人们对这场论战的裁决如何，它都锻造并抬高了整个领域的方法论标准。随后的执法行动与安然（Enron）录音带（"死星""胖小子""回旋镖"——这些策略所利用的恰恰是电力库学派早已警告过的加州分区市场与邻近市场之间的制度缝隙）为这场公共论争画上了句号。
+
+经济学家对加州*为何*失败的解释后来成为经典叙事：（一）零售价格冻结切断了需求与批发价格的联系，使需求几乎完全无弹性；（二）公用事业公司被禁止或不被鼓励签订远期合同，使其完全暴露于现货价格，从而给予供应商最大的抬价激励（Allaz–Vila逻辑的反面）；（三）PX电能市场与ISO阻塞管理及可靠性职能的分离，加之分区而非节点定价，制造了套利缝隙；（四）软性价格上限诱发了策略性外送再回购。上述每一要素事前都已有部分经济学家指出；危机将这些警告转化为设计正统：合同覆盖至关重要，需求响应至关重要，节点优于分区，市场力缓解必须内建于设计而非事后补丁。
+
+### 5.3 经济学的制度化：市场监测机构
+
+加州危机最持久的制度遗产是**市场监测机构**（market monitor）。如今每个ISO/RTO都设有独立的内部和/或外部市场监测单位——Potomac Economics公司（由经济学家David Patton领导）担任多个市场（包括ERCOT、NYISO与MISO）的独立监测方；Monitoring Analytics公司（由经济学家Joseph Bowring领导）监测PJM；CAISO则设有市场监测部及市场监察委员会，后者历来由学院派经济学家（Wolak、Bushnell及其同事）担任主席。这些单位执行"行为与影响"缓解筛查（当拥有局部市场力的供应商报价远超参考成本时自动实施报价封顶），发布充满集中度与价格–成本毛利分析的年度《市场状况报告》，并向FERC执法部门移交案件。市场监测是作为一项常设监管职能来实践的应用产业组织学——这一职业生态位在经济学家将其创造出来之前根本不存在。
+
+---
+
+## 6. 资源充裕性："缺失的货币"、容量市场与稀缺定价
+
+### 6.1 "缺失的货币"问题
+
+在理论理想中，纯电能量市场（energy-only market）通过稀缺租金为容量融资：短缺时段价格飙升至失负荷价值（VoLL）水平，对此类租金的预期恰好诱导出有效率的容量投资，使边际机组能够回收其固定成本。而在实践中，美国各市场系统性地压制了这些租金：报价上限（东部市场长期为1,000美元/兆瓦时）、系统运营商宁可调用备用也不让价格上涨的场外可靠性操作，以及市场力缓解规则，三者共同截断了价格分布的右尾。发电商回收固定成本所需的收入与被压制的电能价格实际提供的收入之间的缺口，被称为**"缺失的货币"问题**——这一术语经由Peter Cramton与Steven Stoft的工作得以流行，并成为Joskow若干重要论述（如Joskow 2008）的核心。这一诊断把可靠性从一项工程指令重新表述为一种定价失灵，而经济学界在药方上产生了分裂：是修复电能价格（稀缺定价），还是外挂一个独立的容量市场。
+
+### 6.2 容量市场与需求曲线创新
+
+东部各ISO选择了容量市场：强制性的远期拍卖，售电主体必须采购足以满足备用裕度目标的经认证容量。早期的"垂直需求"设计（固定需求量，一旦供给略微超过需求价格便崩至零）被证明具有病态的波动性且易被操纵。补救方案——2003年首先在NYISO实施、如今已被普遍采用——是**倾斜的行政需求曲线**：一条以估算的"新建机组成本"（CONE）为锚点的向下倾斜曲线，使容量价格成为容量盈余的平滑函数。这条需求曲线是纯粹的经济工程——一条为可靠性行政构造的边际价值曲线——其参数（CONE数值、曲线形状与锚点）通过周而复始的监管程序确定，而这些程序本质上是计量经济学的诉讼战。Cramton与Stoft的设计工作（其中许多受各ISO委托完成），连同Joskow的分析，塑造了PJM的"可靠性定价模型"（RPM，2007年实施）与ISO-NE的"远期容量市场"（FCM，2008年首次拍卖），包括提前三年采购、分区容量定价，以及后来的"按绩效付费"惩罚结构（ISO-NE首创；2014年1月极地涡旋暴露了"有容量却顶不上"的弱点后，PJM推出容量绩效改革）。此后一代的争端——针对受州补贴新进入者的最低报价规则（MOPR）、买方市场力、以及间歇性与时长受限资源的容量认证——使容量市场经济学始终位居该领域诉讼最密集的战场之列。
+
+### 6.3 纯电能量市场的替代方案：ERCOT与运行备用需求曲线
+
+得克萨斯走了另一条路。ERCOT——州内电网，不受FERC管辖，依据1999年参议院第7号法案完成重组——运行一个没有容量机制的纯电能量市场，其报价上限历史上很高（一度达到9,000美元/兆瓦时）。其标志性机制是2014年采纳的**运行备用需求曲线（ORDC）**，直接基于William Hogan的方案（《通过运行备用实现电力稀缺定价》，2013）：一个实时价格加价项，等于失负荷概率（作为当前备用水平的函数）乘以失负荷价值，使价格随备用收缩而平滑地、*自动地*向VoLL攀升——以行政方式复现一个需求侧可响应的理想市场本应产生的稀缺租金。ORDC或许是所有行业中最纯粹的例证：一篇经济学工作论文中的公式被直接写进了一个大型市场的结算软件。2021年的危机（见第9.4节）使这一设计经受了最严酷的检验。
+
+### 6.4 悬而未决的论战
+
+容量市场对纯电能量市场之争至今仍在继续，并因脱碳而愈发尖锐。容量市场的支持者强调投资风险的降低与政治上的稳健性（监管者绝不会长期容忍真正的VoLL定价）；纯电能量市场的支持者（Hogan，以及立场不同但同样持批评态度的Wolak——后者对两种机制都心存疑虑，而强调合同义务）则反驳说：容量市场付费买的是"立在地上的铁疙瘩"而非实际绩效，招致无休止的行政博弈，并削弱了奖励灵活性与需求响应所必需的稀缺价格信号。储能与可再生能源的兴起——其容量价值（以"有效带载能力"ELCC测度，又一个与经济学相邻的构造）随渗透率变化——使资源充裕性设计成为当下最活跃的研究领域之一（见第9节）。
+
+---
+
+## 7. 机器舱里的机制设计：拍卖、非凸性与虚拟报价
+
+### 7.1 作为优化问题的市场
+
+ISO的日前市场是一个密封报价、多单位、多产品的拍卖，通过安全约束机组组合与经济调度出清——这是一个混合整数规划，其规模（数万条约束、数千个资源）使其跻身世界上运行规模最大的拍卖之列。其中的经济学问题是经典的机制设计问题：什么价格能够支撑有效率的配置？如实报价的激励何在？相互关联的产品（电能、调频、旋转备用）应如何联合优化？经济学家——常与运筹学家合作——给出了如今已嵌入软件的答案。
+
+### 7.2 非凸性与附加费用：定价的前沿
+
+发电机组的成本是非凸的——启动成本、最小出力水平、最小开停机时间——因此支撑有效率机组组合的瓦尔拉斯均衡可能不存在：在LMP价格下，一些已开机机组亏损，而一些未开机机组本可盈利。美国各市场用"附加费用"（uplift）即"补齐"侧支付来打补丁，但这种支付具有歧视性、不透明，并且钝化了价格信号。理论界的回应构成了过去二十年最活跃的应用理论文献之一：O'Neill、Sotkiewicz、Hobbs、Rothkopf与Stewart（2005）展示了如何利用整数活动定价在存在非凸性时构造市场出清价格；Gribik、Hogan与Pope（2007）提出了**凸包定价**（convex hull pricing，又称"扩展LMP"），即基于成本函数凸包络定价以最小化总附加费用。这些思想的各种变体已付诸实施：MISO采纳了凸包定价的近似算法，FERC的快速启动机组定价系列法令（2016–2020）则推动各ISO允许缺乏灵活性的快启机组参与定价。在这一领域，经济理论、运筹学与结算软件正在实时协同演化。
+
+### 7.3 虚拟报价与双结算体系
+
+Hogan的多重结算架构把具有财务约束力的日前市场与实时平衡市场分离开来。为使二者保持一致，市场允许**虚拟（收敛）报价**：纯金融头寸，套利日前与实时价格的预期价差。理论预言套利者将促使两个价格收敛，改善机组组合；实证文献大体证实了这一点，同时也记录了诸多微妙之处——Jha与Wolak关于加州的研究发现收敛报价改善了价格收敛与生产效率，而摩根大通"补齐支付"操纵案（2013年与FERC和解）以及关于MISO网损套利的研究，则展示了金融产品如何与市场缝隙相互作用。FTR拍卖的设计、FTR收益长期低于拍卖价格的痼疾，以及FTR组合违约风险应由谁承担的问题（2018年PJM市场GreenHat违约事件将其推向风口浪尖），同样主要由经济学家进行分析。
+
+### 7.4 采购拍卖与保底供电服务
+
+在ISO市场之外，经济学家还设计了重组各州采购保底零售供电服务的拍卖机制——最著名的是新泽西州"基本发电服务"（BGS）**降价时钟拍卖**，其设计有拍卖经济学家（包括Ausubel与Cramton圈子）的参与，自2002年起每年举行；伊利诺伊州的采购活动亦然。Milgrom与Wilson更广泛的拍卖设计咨询实践（2020年诺贝尔奖颁奖词对此予以表彰）将电力采购列为其应用领域之一。可再生能源采购的合同设计（指数化购电协议、差价合约）是这一传统当前的前沿。
+
+---
+
+## 8. 需求侧：动态定价与需求响应之战
+
+### 8.1 经济学家最古老的抱怨
+
+从Boiteux开始，经济学家就坚持认为需求侧是市场的另一半：如果零售用户面对的是不随时间变化的价格，那么本应约束批发市场的需求价格弹性便不复存在，稀缺无法有效率地出清市场，而且（如加州所证明的）整个系统将变得脆弱。Borenstein在21世纪头十年的研究（如《实时电价的长期效率》，2005；Borenstein与Holland 2005关于仅部分用户面对实时电价时产生扭曲的分析）为**动态零售定价**建立了现代福利经济学论证；Wolak设计并评估了测度家庭对尖峰电价响应的现场实验（包括在阿纳海姆及海外进行的实验）；其后大量的实验文献（包括Jessoe与Rapson关于信息反馈的研究）量化了技术手段（智能电表、自动化）如何提高有效弹性。实际采用仍然有限——默认的固定费率几乎无处不在——经济学家将其归因于政治经济学与行为摩擦，而非经济学本身有误；理论之清晰与零售实践之落差，是经济学界一个持续存在的尴尬，学界至今仍在探究（包括关于动态定价分配效应的研究）。
+
+### 8.2 第745号法令与"EPSA诉FERC案"：对垒双方都是经济学家
+
+批发侧的**需求响应**——付费让用户削减用电——引发了电力行业最著名的一场经济学争论。FERC第745号法令（2011）要求各ISO按完整的LMP向需求响应付费。Alfred Kahn在其晚年最后的公共介入之一中提交意见支持全额LMP补偿；William Hogan则强力主张，正确的支付额应为LMP减去用户所规避的零售电价（"LMP−G"），因为向一个同时省下零售电费的人支付全额LMP，是对削减用电的双重补偿，会补贴无效率的需求削减。这场争论——其实质在于反事实基准的定义以及对零售电价扭曲的处理方式——最终以"FERC诉电力供应协会案"（*FERC v. EPSA*，2016）的形式打到最高法院，法院维持了FERC的管辖权与第745号法令。双方阵营中都布满了经济学家的法庭之友意见书与学术论文；这一事件如今已成为教学案例，示范"次优"推理如何因将哪一种扭曲视为给定而得出截然相反的结论。
+
+---
+
+## 9. 当代前沿：可再生能源、储能、分布式资源与2021年得州危机
+
+### 9.1 零边际成本的进入与价格形成
+
+受补贴与政策强制驱动的风电与光伏——依托生产税抵免（PTC）、投资税抵免（ITC）及各州可再生能源配额制——以改变价格形成规律的体量进入批发市场：多风地区出现负电价（PTC付钱让机组发电的可预见后果，经济学家很早就作了分析）、加州的"鸭子曲线"、被压低的午间电价，以及火电机组在报价排序中被挤出。经济学文献从多条战线作出回应：测度可变可再生能源的市场价值随渗透率上升而衰减；量化可再生能源进入对排放与价格的影响；分析场外补贴与容量市场的相互作用（MOPR之战）；并力推首优方案——在ISO市场中实行碳定价，包括NYISO与PJM的碳加价正式设计方案，"未来资源研究所"（RFF）等机构的经济学家对此进行了分析。必须承认，"技术中性的碳定价优于特定技术强制令"这一广泛的职业共识，在立法层面收效有限；经济学家因此日益转向次优工具设计的分析（清洁能源标准、基于ELCC的容量认证、混合采购机制）。
+
+### 9.2 储能与分布式资源：第841号与第2222号法令
+
+FERC第841号法令（2018）要求各ISO建立参与模型，允许储能同时作为买方和卖方参与定价；第2222号法令（2020）将参与资格扩展到分布式能源资源的聚合体。两项法令回应的都是经济学家提出的问题框架：储能套利及其福利效应、储能被双重收取输电费的问题，以及表后资源的处理。日益增长的实证文献（关于ERCOT与CAISO电池机群的研究）如今正在测度储能以多快的速度竞争掉自身的套利利润——这是进入经济学在现实中上演的一场干净利落的检验。
+
+### 9.3 输电，再一次
+
+输电规划与成本分摊——Joskow与Tirole关于商业化输电的理论研究以及Hogan"受益者付费"原则的主题——随着第1000号法令（2011）与第1920号法令（2024）重回舞台中央，后者要求开展前瞻性的区域规划并按受益分摊成本。法院（特别是第七巡回上诉法院在"伊利诺伊商务委员会"系列案件中，判决书出自经济学家出身的Posner法官之手）所执行的"受益者付费"标准，就是不折不扣的成本收益经济学。
+
+### 9.4 冬季风暴"乌里"（2021年2月）：对纯电能量设计的压力测试
+
+2021年2月的得克萨斯大停电——连续数日的轮流停电、电价被行政性地按在9,000美元/兆瓦时的上限超过四天、约二百余人死亡、数百亿美元规模的财务再分配——引发了自加州危机以来最激烈的经济学复盘。经济学讨论区分了几个层面的问题：这场失败首先是*供给可用性*的失败（未做防寒改造的天然气供应与发电设施），而非市场设计本身的失败；稀缺定价按设计运转了，但上限价格持续的时长暴露了零售侧对冲的缺失（使用批发价格直通费率的Griddy公司用户），并提出了一个问题——持续数日的VoLL水平价格在政治上或合同上是否可以承受；而得州公用事业委员会（PUCT）在限电结束后仍将价格维持在上限的决定，则成为一场关于场外干预的著名争议。时任ERCOT董事会成员的Peter Cramton（与其他州外董事一道在风暴后辞职）、Hogan、Wolak、Bushnell等众多学者发表了针锋相对的分析；危机之后，得州降低了价格上限、修改了ORDC、强制实施防寒改造，并提出了新的"绩效信用机制"方案——每一项都以明确的经济学语言进行辩论。"乌里"还重新激发了学界对天然气与电力市场设计相互作用的兴趣，以及对Wolak长期倡导的零售/售电主体强制对冲义务的兴趣。
+
+### 9.5 脱碳电网的资源充裕性
+
+当前的研究前沿在追问：为拥有可观边际成本的燃料发电机群设计的"LMP＋稀缺定价"架构，对于一个由零边际成本、气象驱动、时长受限资源主导的系统而言是否仍然适用。经济学家正在积极研究的问题包括：当边际资源是储能或被弃的可再生能源时的价格形成（机会成本定价）；基于边际ELCC的容量认证；长时充裕性与尾部风险（与气候相关的相关性停机）；资本密集型清洁可控资源在收入高度波动下的融资问题（由此催生差价合约与"可靠性期权"——后者是Cramton–Stoft的设计，已在哥伦比亚采用，ISO-NE的FCM也以修改形式采纳）；以及集中式市场是否应演化为集中化的长期采购——即"混合市场"。Joskow（其近年论文认为纯电能市场模式正承受日益加大的压力）、Schmalensee、Wolak、Hogan以及更年轻的一代学者已加入这场论战。换言之，车轮再次转动，而经济学家又一次站在了车轮的两侧。
+
+---
+
+## 10. 改革成功了吗？实证评估文献
+
+市场化改革创造了一场自然实验——集中式市场地区对传统监管地区；被剥离的电厂对公用事业保留的电厂——实证经济学家对此挖掘了二十年。主要发现如下：
+
+- **运营效率。** Fabrizio、Rose与Wolfram（*AER*，2007）发现，重组各州的投资者所有制电厂相对于未重组各州的电厂，其人工与非燃料运营支出降低了数个百分点——竞争（或对竞争的预期）强化了成本纪律的证据。Davis与Wolfram（*AEJ: Applied*，2012）发现，将核电厂剥离给独立的商业化运营商使容量因子提高了约10个百分点，这一巨大的效率增益集中体现在停机时长的缩短上。Cicala（*AER*，2015）表明，一旦成本转嫁激励被移除，放松管制的电厂采购煤炭的价格更为低廉。
+- **调度效率。** Cicala（*AER*，2022）利用ISO市场分批扩张的准实验设计估计，基于市场的调度通过将出力重新配置给低成本电厂并扩大跨公用事业边界的贸易利得，每年节约生产成本达数十亿美元量级——这是LMP机制兑现其设计者承诺的最干净的证据。
+- **投资与技术结构。** 商业化投资以与监管时代迥异的技术选择（快速、模块化的燃气机组，其后是可再生能源与电池）响应价格信号；文献既记录了这种响应性，也记录了纯商业化风险暴露所产生的繁荣–萧条周期。
+- **零售竞争。** 零售侧的文献结论更为复杂：来自得克萨斯等地的证据记录了居民零售选择中显著的搜寻摩擦、价格离散与"迷惑定价"（Hortaçsu、Madanizadeh与Puller 2017年关于得州的研究是这方面的标杆），这为零售自由化的宏大主张降了温——即便批发侧的收益已被接受。
+- **价格。** 简单地比较监管州与重组州的零售价格存在严重的混淆因素（重组各州本来就是高成本州；天然气价格周期主导一切），经济学界已大体形成共识：市场化改革的福利收益体现在成本、调度与电厂绩效上，而非明确无误的零售降价上——这是一个诚实的、尽管在政治上不能令人满意的裁决。
+
+在评估之外，电力还成了实证产业组织学最钟爱的实验室：由于工程口径的边际成本可以直接测量，研究者能够直接计算加成率而无需推断。这个行业因此产出了关于寡头报价行为（Wolfram关于英格兰–威尔士电力库的研究；Hortaçsu与Puller关于ERCOT报价者偏离最优报价的研究）、远期合约、拍卖行为以及环境政策交互作用的一系列奠基性研究。方法论的交流是双向的：电力数据锤炼了产业组织理论，而产业组织的工具则为市场监测机构输送了人才。
+
+---
+
+## 11. 批评、逆流与设计的限度
+
+一篇诚实的综述必须记录：经济学家对电力市场化改革的主导地位不乏批评者，其中也包括经济学界内部的声音。
+
+第一，20世纪90年代的政治动员中**对消费者节省的承诺被夸大了**，学界更为审慎的声音（Joskow是其中的代表）花了多年时间区分证据支持什么与倡导者宣称什么。第二，**复杂性本身就是一种成本**：电能、辅助服务、容量、FTR与虚拟产品层层叠加的大厦——每一层都在给上一层的激励问题打补丁——招致了多方批评（左翼视之为为交易商设计的不透明体系，部分工程界人士视之为脆弱，公共电力倡导者则视之为昂贵的弯路）。第三，**治理批判**：ISO的利益相关方程序与FERC诉讼是通过资金雄厚的在位者拥有结构性优势的程序来分配租金的——这一斯蒂格勒式的观察，与设计传统的技术官僚自我形象格格不入。第四，**脱碳批判**：各州清洁能源政策如今驱动着绝大部分投资，受补贴的进入与市场价格形成之间由此产生的碰撞（MOPR、混合市场方案）使一些学者认为，20世纪90年代的架构正在被悄然取代，回归"计划采购、市场只管短期平衡"——即"调度靠市场、投资靠规划"。这究竟意味着设计纲领的失败，还是其适应性的成功，或许是这个领域未来十年的核心解释学问题；而颇为应景的是，参与这场论争的仍是同一批经济学家、他们的学生，以及学生的学生。
+
+---
+
+## 12. 结论
+
+本文所综述的历史弧线，从Boiteux在法国电力公司的峰荷电价，经芝加哥学派与MIT对监管的批判、Schweppe的现货价格、Hogan的合同网络，一直延伸到今天为美国大部分批发电力出清的五分钟节点电价。沿途，经济学家建立了制度（ISO、市场监测机构、容量拍卖），参与并裁判了危机（加州、"乌里"），创造了一套市场设计与市场监督的常设职业体系，并产出了实证经济学中关于竞争与监管各自擅长什么的最丰富证据之一。有三条教训可以推广到电力之外。第一，当物理规律或技术条件排除了去中心化交易时，市场是被设计出来的人工制品，而设计的质量——机制设计——决定了竞争能否兑现其教科书上的收益。第二，转型受制于次优问题（搁浅成本、零售价格扭曲、受补贴的进入），其程度至少不亚于受制于首优蓝图；经济学界最有价值的介入往往是诊断性的，而非建筑师式的。第三，市场设计永无完工之日：每解决一个问题（输电开放、阻塞、缺失的货币），下一个问题（非凸性、容量认证、深度脱碳）便随之显现。四十年过去，美国电力市场依然是它开始时的样子——全世界规模最宏大的一场正在进行中的应用经济学实验。
+
+---
+
+## 参考文献（精选，保留英文原文）
+
+- Allaz, B., and J.-L. Vila (1993). "Cournot Competition, Forward Markets and Efficiency." *Journal of Economic Theory* 59(1): 1–16.
+- Averch, H., and L. Johnson (1962). "Behavior of the Firm under Regulatory Constraint." *American Economic Review* 52(5): 1052–1069.
+- Boiteux, M. (1949). "La tarification des demandes en pointe." *Revue Générale de l'Électricité*; English translation, "Peak-Load Pricing," *Journal of Business* 33 (1960): 157–179.
+- Boiteux, M. (1956). "Sur la gestion des monopoles publics astreints à l'équilibre budgétaire." *Econometrica* 24(1): 22–40.
+- Borenstein, S. (2005). "The Long-Run Efficiency of Real-Time Electricity Pricing." *The Energy Journal* 26(3): 93–116.
+- Borenstein, S., J. Bushnell, and F. Wolak (2002). "Measuring Market Inefficiencies in California's Restructured Wholesale Electricity Market." *American Economic Review* 92(5): 1376–1405.
+- Borenstein, S., and S. Holland (2005). "On the Efficiency of Competitive Electricity Markets with Time-Invariant Retail Prices." *RAND Journal of Economics* 36(3): 469–493.
+- Christensen, L., and W. Greene (1976). "Economies of Scale in U.S. Electric Power Generation." *Journal of Political Economy* 84(4): 655–676.
+- Cicala, S. (2015). "When Does Regulation Distort Costs? Lessons from Fuel Procurement in US Electricity Generation." *American Economic Review* 105(1): 411–444.
+- Cicala, S. (2022). "Imperfect Markets versus Imperfect Regulation in US Electricity Generation." *American Economic Review* 112(2): 409–441.
+- Cramton, P., and S. Stoft (2005). "A Capacity Market that Makes Sense." *The Electricity Journal* 18(7): 43–54.
+- Davis, L., and C. Wolfram (2012). "Deregulation, Consolidation, and Efficiency: Evidence from US Nuclear Power." *American Economic Journal: Applied Economics* 4(4): 194–225.
+- Demsetz, H. (1968). "Why Regulate Utilities?" *Journal of Law and Economics* 11(1): 55–65.
+- Fabrizio, K., N. Rose, and C. Wolfram (2007). "Do Markets Reduce Costs? Assessing the Impact of Regulatory Restructuring on US Electric Generation Efficiency." *American Economic Review* 97(4): 1250–1277.
+- Green, R., and D. Newbery (1992). "Competition in the British Electricity Spot Market." *Journal of Political Economy* 100(5): 929–953.
+- Gribik, P., W. Hogan, and S. Pope (2007). "Market-Clearing Electricity Prices and Energy Uplift." Harvard Electricity Policy Group working paper.
+- Hogan, W. (1992). "Contract Networks for Electric Power Transmission." *Journal of Regulatory Economics* 4(3): 211–242.
+- Hogan, W. (2013). "Electricity Scarcity Pricing Through Operating Reserves." *Economics of Energy & Environmental Policy* 2(2): 65–86.
+- Hortaçsu, A., and S. Puller (2008). "Understanding Strategic Bidding in Multi-Unit Auctions: A Case Study of the Texas Electricity Spot Market." *RAND Journal of Economics* 39(1): 86–114.
+- Hortaçsu, A., S. Madanizadeh, and S. Puller (2017). "Power to Choose? An Analysis of Consumer Inertia in the Residential Electricity Market." *American Economic Journal: Economic Policy* 9(4): 192–226.
+- Joskow, P. (2008). "Capacity Payments in Imperfect Electricity Markets: Need and Design." *Utilities Policy* 16(3): 159–170.
+- Joskow, P., and E. Kahn (2002). "A Quantitative Analysis of Pricing Behavior in California's Wholesale Electricity Market During Summer 2000." *The Energy Journal* 23(4): 1–35.
+- Joskow, P., and R. Schmalensee (1983). *Markets for Power: An Analysis of Electric Utility Deregulation.* MIT Press.
+- Joskow, P., and J. Tirole (2005). "Merchant Transmission Investment." *Journal of Industrial Economics* 53(2): 233–264.
+- Kahn, A. (1970–71). *The Economics of Regulation: Principles and Institutions.* Wiley (2 vols.).
+- Klemperer, P., and M. Meyer (1989). "Supply Function Equilibria in Oligopoly under Uncertainty." *Econometrica* 57(6): 1243–1277.
+- Laffont, J.-J., and J. Tirole (1993). *A Theory of Incentives in Procurement and Regulation.* MIT Press.
+- O'Neill, R., P. Sotkiewicz, B. Hobbs, M. Rothkopf, and W. Stewart (2005). "Efficient Market-Clearing Prices in Markets with Nonconvexities." *European Journal of Operational Research* 164(1): 269–285.
+- Peltzman, S. (1976). "Toward a More General Theory of Regulation." *Journal of Law and Economics* 19(2): 211–240.
+- Schweppe, F., M. Caramanis, R. Tabors, and R. Bohn (1988). *Spot Pricing of Electricity.* Kluwer Academic Publishers.
+- Steiner, P. (1957). "Peak Loads and Efficient Pricing." *Quarterly Journal of Economics* 71(4): 585–610.
+- Stigler, G. (1971). "The Theory of Economic Regulation." *Bell Journal of Economics and Management Science* 2(1): 3–21.
+- Turvey, R. (1968). *Optimal Pricing and Investment in Electricity Supply.* MIT Press.
+- Vickrey, W. (1961). "Counterspeculation, Auctions, and Competitive Sealed Tenders." *Journal of Finance* 16(1): 8–37.
+- Williamson, O. (1966). "Peak-Load Pricing and Optimal Capacity under Indivisibility Constraints." *American Economic Review* 56(4): 810–827.
+- Wilson, R. (2002). "Architecture of Power Markets." *Econometrica* 70(4): 1299–1340.
+- Wolak, F. (2003). "Measuring Unilateral Market Power in Wholesale Electricity Markets: The California Market, 1998–2000." *American Economic Review* 93(2): 425–430.
+- Wolfram, C. (1999). "Measuring Duopoly Power in the British Electricity Spot Market." *American Economic Review* 89(4): 805–826.

us_market/field_guide_en.md

@@ -0,0 +1,370 @@
+# US Power Markets: A Field Guide
+
+*Bidding, trading, and the impact of AI — consolidated from a series of working discussions, July 2026.*
+
+---
+
+## Contents
+
+1. [How the market clears](#1-how-the-market-clears)
+2. [How participants compete](#2-how-participants-compete)
+3. [Where edge actually lives](#3-where-edge-actually-lives)
+4. [How AI changes the picture](#4-how-ai-changes-the-picture)
+5. [Building a bidding system](#5-building-a-bidding-system)
+6. [China: market structure and deployment](#6-china-market-structure-and-deployment)
+7. [Open questions](#7-open-questions)
+8. [Sources and further reading](#8-sources-and-further-reading)
+
+---
+
+## 1. How the market clears
+
+The landscape splits into two worlds: **market operators** (the seven US ISOs/RTOs — PJM, CAISO, ERCOT, MISO, NYISO, SPP, ISO-NE) who *clear* the market, and **market participants** (generators, batteries, traders, hedge funds) who *bid into* it. Everything in this guide sits on one side or the other of that line.
+
+### 1.1 The clearing algorithms (ISO/RTO side)
+
+The core is large-scale **mixed-integer programming**, not ML:
+
+- **SCUC (Security-Constrained Unit Commitment)** clears the day-ahead market — a massive MILP solved with commercial solvers (Gurobi / CPLEX / Xpress), deciding unit commitments subject to network and reliability constraints.
+- **SCED (Security-Constrained Economic Dispatch)** runs every 5 minutes in real time (LP/QP), producing **Locational Marginal Prices (LMPs)** decomposed into energy + congestion + losses.
+- Energy and ancillary services are co-optimized in the same clearing.
+
+**Active frontiers on the clearing side:** convex hull pricing / extended LMP for non-convexities; stochastic and robust unit commitment for renewable uncertainty; and ML-accelerated optimization — learned warm starts, active-constraint prediction, transmission constraint screening (pushed by ARPA-E grid optimization competitions).
+
+### 1.2 The two-settlement structure: day-ahead vs. real-time
+
+All seven ISOs run a two-settlement (day-ahead + real-time) design. **Short answer: day-ahead is where the volume and money is; real-time is where the risk is.**
+
+**Day-Ahead Market (DAM)**
+- Primary market in most designs; ~90–95%+ of physical energy settles financially at the DA price in US ISOs
+- Clears once daily via auction with full network modeling
+- Reference price for the forward curve; most hedging keys off it
+- The classic academic electricity-price-forecasting (EPF) problem
+
+**Real-Time (Balancing) Market**
+- Settles only deviations from day-ahead positions — small volume, huge volatility
+- Prices can move from $30 to $3,000+/MWh (or deeply negative) in minutes
+- Critical for: risk management, batteries, fast-ramping peakers, demand response, DA–RT spread traders
+- Battery operators often earn most energy-arbitrage revenue from RT volatility
+
+Expected RT prices anchor DA prices (virtual bidding arbitrages them on average), but realized RT prices are far noisier. The practical forecasting hierarchy for most participants: (1) day-ahead hourly prices, (2) the DA–RT spread, (3) intraday/real-time — *reversed* if you operate storage or fast-response assets.
+
+Distinctively US features on the participant side: **virtual bidding** (INCs/DECs), **FTR/CRR auctions**, and the fast-growing **battery-storage bidding** specialty.
+
+---
+
+## 2. How participants compete
+
+### 2.1 Price forecasting: the foundation
+
+Electricity is not storable at scale, so prices reflect instantaneous supply-demand balance and can spike or go negative. Forecasting approaches fall into three families, usually blended:
+
+**Fundamental (structural) models** simulate the market itself: load forecast vs. the supply stack (merit order). Key inputs: fuel prices (especially gas, which often sets the margin), renewable output forecasts, planned/unplanned outages, transmission constraints, imports/exports. Tools: PLEXOS, Aurora, in-house production-cost / dispatch simulations. Used heavily by utilities, large generators, and ISOs.
+
+**Statistical / econometric models** — ARIMA/ARIMAX, GARCH for volatility, regressions on temperature, gas, load. They exploit strong seasonality (hour-of-day, day-of-week, season) and are best for "normal" conditions and short horizons.
+
+**Machine learning models** — gradient-boosted trees, neural nets (LSTMs, transformer-based), trained on weather, load, renewables, prices, fuel, and outage data. Sophisticated shops forecast the **full distribution**, not just the mean — the tails are where money is made or lost.
+
+**In practice: hybrid.** Fundamental model for structural shape + ML/statistical corrections + human trader judgment, especially for events models handle poorly (cold snaps, plant trips, unusual grid conditions).
+
+**Universal key inputs:** weather forecasts (the single biggest driver), natural gas prices, renewable generation forecasts, outage schedules, transmission/congestion conditions.
+
+**State of the art.** The benchmark progression runs **LEAR (regularized linear/LASSO) → deep neural nets → temporal architectures (LSTM, Transformer)**. Transformer models now forecast DA–RT price spreads in volatile markets (e.g., ERCOT), using load / solar / wind forecasts and temporal features, with walk-forward retraining. Well-tuned linear models remain surprisingly competitive; hybrid linear+nonlinear architectures with online learning are a current frontier. Practical SOTA is increasingly **probabilistic** — quantile regression, distributional deep nets, conformal prediction — because decisions need the full distribution, not a point forecast.
+
+### 2.2 Strategic bidding optimization
+
+Two state-of-the-art frameworks for price-makers: **bi-level optimization (MPEC-style)** and **reinforcement learning**. Deep RL (DDPG + prioritized experience replay, PPO, SAC) handles continuous state/action spaces and non-convex operating characteristics. But **stochastic programming and robust optimization remain the workhorses** for co-optimizing energy + ancillary service offers under uncertainty, with CVaR the standard risk overlay.
+
+### 2.3 Battery storage arbitrage (the hottest area)
+
+- Classical: stochastic dynamic programming, MPC over price scenarios, degradation costs in the objective.
+- Modern: deep RL for charge/discharge policies; **decision-focused learning** (forecasts trained on profit, not accuracy — see §4.4).
+- Research frontier: hierarchical / multi-agent RL coordinating DA + RT bidding under the two-settlement structure, with risk-adjusted rewards (fixed policies struggle under regime shifts).
+- Industry: optimizer firms (Habitat Energy, Gridmatic, Tyba, Fluence Mosaic) run ML forecasting + optimization stacks commercially.
+
+### 2.4 Virtual bidding and financial trading (INCs, DECs, FTRs)
+
+Essentially quant trading: gradient-boosted trees (XGBoost/LightGBM) and deep nets predicting DA–RT spreads at nodal granularity, with features drawn from weather, load/renewable forecast errors, outages, and congestion patterns. Portfolio construction runs under risk limits; FTR auctions add congestion-rent scenario optimization.
+
+### 2.5 Emerging directions
+
+- **Online / no-regret learning** — bidding policies updated directly from market outcomes, with sub-linear regret guarantees.
+- **Multi-agent RL** — Nash equilibrium approximation for market simulation and market-power analysis.
+- **LLM-enhanced trading frameworks** — very early, appearing in the literature.
+- **Open-source benchmarks** — Grid2Op, RL2Grid, new two-settlement bidding environments.
+
+### 2.6 Industry reality check
+
+Most real money is still made with **strong probabilistic forecasting + classical optimization (stochastic MIP/MPC) + human trader judgment**. RL is in production mainly for storage dispatch at a handful of sophisticated shops. Deep learning has clearly won the *forecast* layer; the *decision* layer remains dominated by optimization with ML inputs.
+
+> **One-sentence summary of the frontier:** DL/transformers have won the forecasting layer; RL is contesting the optimization layer but hasn't displaced stochastic optimization yet.
+
+Two nuances: the two moves are at **different maturity stages** — DL in forecasting is a completed takeover (and increasingly probabilistic), while RL in decisions is still a challenger vs. deployed stochastic optimization / MPC. And the **boundary between the layers is dissolving**: decision-focused learning trains forecasts directly on trading profit, collapsing forecast-then-optimize into one learned pipeline — arguably the most interesting current research direction (§4.4).
+
+### 2.7 Quick reference table
+
+| Layer | Deployed standard | Research frontier |
+|---|---|---|
+| Market clearing (ISO) | MILP SCUC + LP SCED, LMP | Convex hull pricing, stochastic UC, ML-accelerated optimization |
+| Price forecasting | Probabilistic deep nets, GBTs, tuned linear models | Transformers, hybrid+online learning, conformal prediction |
+| Bidding/dispatch decisions | Stochastic programming, MPC, bi-level MPEC | Deep RL (DDPG/PPO/SAC), hierarchical & multi-agent RL, no-regret online learning |
+| Forecast↔decision interface | Two-stage predict-then-optimize | Decision-focused / end-to-end learning |
+| Storage arbitrage | MPC + probabilistic forecasts | DFL, risk-aware hierarchical RL |
+| Virtual bidding / FTRs | GBT/DNN spread models + portfolio risk limits | Transformer spread forecasting, LLM-enhanced frameworks |
+
+---
+
+## 3. Where edge actually lives
+
+### 3.1 The hierarchy of durable edge
+
+Any bidding system decomposes into three functional layers: a **data and forecasting layer** that converts weather, grid, and market data into calibrated probability distributions; an **optimization layer** that converts distributions into offer curves and positions under risk constraints; and an **execution and risk layer** that submits bids, monitors positions, settles, and enforces discipline.
+
+The competitive value of these layers is not equal:
+
+1. **Forecast quality and calibration** dominate. Two participants running identical optimizers on different forecasts diverge widely in P&L, while two participants running different (competent) optimizers on identical forecasts land close together.
+2. **Execution and risk discipline** ranks second. In crowded strategies, the shop that re-optimizes every interval, never misses a submission window, and cuts losers per its own rules outperforms the one that doesn't, even with no analytical edge.
+3. **Optimizer formulation** ranks third. The underlying mathematics (quantile offering, two-stage stochastic programming, CVaR constraints, storage MILPs) is published and commoditized; what differs between shops is craft, not theory.
+
+Build implication: spend the best engineering on forecasting and backtesting, use boring proven math in the middle, and treat operational reliability as a feature.
+
+### 3.2 The insider-information myth (PJM / SCUC-SCED)
+
+A natural worry: wouldn't a vendor with inside knowledge of PJM's clearing algorithm — or of the bids flowing into it — crush all competitors? The concern dissolves under inspection.
+
+**What is NOT really secret.** The *algorithm*: SCUC/SCED are standard MIP/LP formulations, exhaustively documented (PJM Manual 11/12, the OATT, business practice manuals), solved with commercial CPLEX/Gurobi-class solvers. Competent quant teams build "shadow SCED" models from public documentation.
+
+**What IS secret — and legally fenced:**
+
+| Item | What it is | Why it's decisive |
+|---|---|---|
+| **Other participants' bids/offers** | Released only masked, ~4-month lag | DA prices are an emergent function of them |
+| **Network state** | Live state-estimator view: flows, voltages, topology, constraint headroom | Knowing which constraint binds next = seeing nodal price divergence before it happens |
+| **Derates** | Real-time reductions in unit capability (e.g., a 900 MW unit down to 600 MW) | Supply tighter than the market believes → position ahead of the spike; classic MNPI in FERC cases |
+| **Dispatch instructions** | 5-min SCED setpoints + out-of-market operator actions (reliability commitments, manual dispatch) | Reveals decisions invisible to outside models; reliability commitments distort local prices |
+
+Using these is a **federal crime**, not a vendor edge: FERC anti-manipulation rules (post-EPAct 2005) cover trading on material non-public information; PJM staff and systems vendors are bound by confidentiality and trading prohibitions; the Independent Market Monitor (Monitoring Analytics — external to PJM) and FERC's Office of Enforcement screen for anomalous profitability. There is precedent — enforcement cases against individuals trading on non-public grid information, and the Powhatan/UTC saga showed FERC pursues even aggressive rule exploitation.
+
+**Why perfect algorithm knowledge wouldn't get you far anyway.** SCED is deterministic *given its inputs*, but the inputs are unknowable in advance — even to PJM. DA prices depend on bids not yet submitted; RT prices depend on forced outages, weather errors, interchange, and binding constraints. Price is an emergent output of thousands of private decisions plus physical randomness.
+
+**Where real differentiation lives instead:** better weather ensembles and load/renewable models; **predicting which transmission constraints will bind** (the genuinely hard, high-value problem in nodal markets); modeling bidding behavior from lagged public data; reconstructing network topology (CEII model access, FTR results, historical shadow prices).
+
+**The honest residual:** EMS/market-software vendors and ex-ISO employees carry legal "soft" insider knowledge — operator behavior, solver quirks, out-of-market actions. It's valuable, common, and legal. The real moat: *people who know how the control room actually behaves at 6 PM on a July scarcity day.*
+
+### 3.3 Is accurate price prediction even the most valuable thing?
+
+**No — for most participants, point forecasts are an input, rarely the differentiator.**
+
+**Generators: optionality and risk, not prediction.** The right tail of the distribution matters far more than the mean — a peaker earns its year in a handful of scarcity hours; the question is 3 vs. 30 scarcity hours, not ±$2/MWh on the average. **Volumetric-price correlation risk** dominates hedging: being forced out exactly when prices spike (Winter Storm Uri: forward-sold generators with frozen plants buying back at $9,000/MWh). And the economics run on **spreads, not prices**: spark spread (gas), dark spread (coal), top-bottom spread (batteries).
+
+**LSEs / large buyers: load forecasting beats price forecasting.** Exposure is squared — a hot day means high load AND high price. Most valuable: your own load forecast, hedge-ratio policy, and shape-risk management. The ERCOT retailer failures (Griddy) were unhedged structural exposure, not bad price forecasts.
+
+**Financial traders: relative value, tails, speed.** The money is in **spreads and congestion** (DA–RT virtuals, UTCs, FTRs, hub-to-node basis) — the skill is predicting which constraints bind. **Calibration beats accuracy**: well-calibrated tails plus good sizing out-earn a sharper point forecast, and survive the blowups. **Speed** matters: reacting to unit trips, weather model runs, and interchange changes faster than the market reprices.
+
+**What arguably matters more than any forecast:**
+
+1. **Risk management and capital discipline** — the graveyard is full of risk-control failures, not forecast failures (the GreenHat FTR default, $180M+ socialized; the Uri casualties). *Approximately right and definitely solvent* wins.
+2. **Structural optionality** — batteries need a good response *policy*, not a forecast; market makers earn the spread; structured desks earn risk-transfer margin.
+3. **Understanding the plumbing** — settlement rules, uplift/make-whole, credit mechanics, FTR auction quirks, emergency operator discretion. Rule knowledge is durable; forecast edges decay.
+4. **Weather as the upstream input** — many firms' single most valuable proprietary asset is their weather-ensemble processing; price forecasts are largely derivative of weather + fuel + outages.
+
+> **Synthesis: price forecasting is necessary but low-moat; risk structuring and constraint/network insight are the actual differentiators.** The scarce capabilities are (a) calibrated tail distributions, (b) node-level congestion insight, (c) position-sizing discipline to survive being wrong, and (d) rule/plumbing expertise — hence firms pay up for ex-control-room operators, network engineers, and risk managers as eagerly as for forecasters. **The forecast is the ante; the other things are the game.**
+
+---
+
+## 4. How AI changes the picture
+
+The three-layer decomposition of §3.1 is the right lens for evaluating the current AI wave, because recent advances land on the layers very unevenly — and, conveniently, they land hardest on the layer that matters most.
+
+| Layer | Edge rank | AI impact | Maturity | Adoption posture |
+|---|---|---|---|---|
+| Forecasting | 1st | Step change (AI weather ensembles) | Operational now | Adopt aggressively; edge is decaying |
+| Execution / risk ops | 2nd | Large but unglamorous (LLM ops) | Production-ready with human review | Adopt quietly; compounding advantage |
+| Optimizer | 3rd | Incremental (decision-focused learning) | Research → early practice | Experiment; keep auditable structure |
+
+### 4.1 Layer 1 — Forecasting: the genuine step change
+
+**The AI weather model transition.** The most consequential AI development for power trading is not reinforcement learning or language models; it is the replacement (or augmentation) of physics-based numerical weather prediction (NWP) with learned atmospheric models. The milestone sequence is short but steep: Huawei's **Pangu-Weather** (Nature, July 2023) demonstrated forecasts roughly 10,000× faster than conventional ensemble systems in peer-reviewed tests; Google DeepMind's **GraphCast** (Science, December 2023) outperformed ECMWF's flagship HRES deterministic model on roughly 90% of 1,380 verification targets; ECMWF moved its AI-based **AIFS** model to operational status in 2024 — the first major meteorological agency to run a learned model operationally; Microsoft's **Aurora** foundation model (Nature, May 2025), pretrained on over a million hours of geophysical data, extended the approach across weather, air quality, and wave prediction at a fraction of traditional compute cost; and DeepMind's **GenCast** pushed the architecture family into probabilistic ensemble forecasting.
+
+**How learned atmospheric models work.** Neural networks are trained — mostly on the ERA5 reanalysis, ~45 years of hourly global atmospheric state — to learn the mapping "atmosphere at t → atmosphere at t+6h" directly, replacing numerical integration of physics equations with a single forward pass. Architectures span 3-D vision transformers (Pangu), icosahedral-mesh GNNs (GraphCast), graph/transformer hybrids (AIFS), diffusion models sampling plausible futures (GenCast — natively probabilistic), and pretrained geophysical foundation models (Aurora). Forecasts roll out autoregressively, which compounds errors and — under MSE training — produces progressive smoothing that underestimates extremes. Critical dependency: they still require physics-pipeline initial conditions; data assimilation is not replaced. Global resolution (~25–31 km) means site-level downscaling/calibration matters more, not less.
+
+**Why the economics matter more than the leaderboard.** Traditional NWP solves differential equations across 3-D grid cells on supercomputers, refreshing two to four times daily at thousands of dollars per cycle. Learned models run on GPUs in seconds to minutes, at accuracy parity in the 12–48h day-ahead window. Three consequences follow:
+
+- **Speed → probability.** When a forecast run costs seconds of GPU time, generating dozens or hundreds of ensemble members becomes trivial. The entire downstream bidding pipeline consumes probability distributions — production quantiles for the day-ahead offer, spread distributions for imbalance risk — so ensemble economics translate directly into better-calibrated inputs and therefore better offers. This is the cleanest causal path from "recent AI" to bidding P&L.
+- **Speed → freshness.** Commercial AI-weather providers now advertise up to 24 runs per day versus the 2–4 of traditional NWP. In a two-settlement market, fresher forecasts matter most between the day-ahead close and real-time delivery: real-time re-offering, intraday position adjustment, and imbalance management for wind and solar portfolios.
+- **Speed → iteration (and entry).** Cheap reforecasting powers backtests, and open model weights plus public initial-condition data put a near-state-of-the-art global forecast within reach of any shop with GPUs — infrastructure that previously required a national weather service or an expensive vendor contract.
+
+**Caveats and failure modes.** Three caveats keep this honest:
+
+1. **Extremes are the weak spot.** Models trained with mean-squared-error objectives produce smoothed fields that systematically underestimate sharp gradients — peak wind speeds in severe storms, for example. Power markets make and lose money precisely in the tails (scarcity events, ramps, icing), so a model that wins on average RMSE can still be the wrong tool for the hours that dominate annual P&L. Ensemble and generative approaches (GenCast-style) mitigate but do not eliminate this. Any adoption plan should include tail-specific verification against your own asset history, not just headline skill scores.
+2. **Vendor claims require independent verification.** The commercial AI-weather space is young and marketing-heavy. The practical test is a paid pilot scored against your incumbent provider on *your* nodes, *your* variables (hub-height wind, plane-of-array irradiance, temperature-driven load), and *your* loss function — ideally the downstream trading metric, not meteorological RMSE.
+3. **The edge decays.** These models are open or cheaply accessible, so the advantage from merely using them erodes as adoption spreads. Public AI raises the floor for everyone. What it cannot commoditize is what you combine it with: proprietary asset telemetry for local calibration, downscaling to your specific sites, and — above all — the translation from weather to *nodal price*, which runs through congestion.
+
+**Downstream of weather: prices, load, congestion.** Price forecasting has moved from classical time-series methods to gradient-boosted trees and increasingly to transformers and time-series foundation models; the practical gains are largest in probabilistic (quantile) forecasting of the DA/RT spread, the input that prices a renewable's imbalance risk. Load forecasting benefits from the same architectures plus improved temperature inputs. **Congestion and nodal-basis prediction remains the hardest and least commoditized problem** — learning the mapping from system conditions to binding transmission constraints — and, precisely because almost nobody does it well, it is where forecast-layer investment retains the longest-lived edge. Graph neural networks that encode grid topology are the active research direction; no vendor sells a turnkey solution worth having.
+
+*Layer 1 synthesis: adopt AI weather ensembles early and aggressively, verify tails independently, and reinvest the freed budget into the two things public models cannot provide — local calibration against your own telemetry, and nodal congestion modeling.*
+
+### 4.2 Layer 2 — Execution, operations, and risk: the quiet LLM win
+
+This is the layer vendor marketing ignores, which is exactly why it is underrated. It is unglamorous back-office work — but it attacks the second-ranked source of edge, carries low model risk because a human reviews the output, and its benefits compound.
+
+**Market-rule intelligence.** US ISO participation is governed by thousands of pages of tariffs, business practice manuals, and a continuous stream of market notices, protocol revisions, and FERC filings. Engineers building bidding systems consistently identify this — not the mathematics — as the dominant cost. It is a nearly ideal LLM workload: a retrieval corpus over the tariff and manual set for each ISO you trade; automated triage of daily market notices, flagging anything that touches your bid parameters, settlement formulas, or ancillary product definitions; and change-diffing of protocol revisions against the assumptions encoded in your optimizer. The failure mode to engineer against is hallucinated rule citations — mitigations are standard (retrieval-grounded answers, mandatory citation to source paragraphs, human sign-off on anything that changes system behavior).
+
+**Compliance documentation and the hedge/spec boundary.** For any shop running both physical assets and financial positions, FERC manipulation risk makes documentation a first-order concern: every physical bid should be independently defensible as profit-maximizing for the asset, on the record. LLMs are well suited to generating that record — drafting daily bid-rationale documentation from the optimizer's own inputs and outputs (forecast quantiles used, constraints binding, deviation from the neutral baseline and why) in consistent, auditable language. This converts a compliance burden that trading shops chronically under-resource into a largely automated byproduct of the bidding run, and materially strengthens the architectural separation between hedge and speculative books (§5.2).
+
+**Settlement, monitoring, and incident response.** Shadow settlement — independently recomputing what the ISO owes you and disputing discrepancies — is high-value, detail-heavy work mixing structured data with unstructured rule text: again a natural LLM-plus-tools workload (parsing settlement statements, reconciling against internal calculations, drafting dispute filings with rule citations). Overnight operations monitoring (telemetry anomalies, missed dispatch instructions, forecast-feed failures ahead of submission deadlines) similarly benefits from an agentic layer that triages, summarizes, and escalates rather than paging a human for everything.
+
+**Engineering velocity.** A second-order but real effect: agentic coding tools compress the build timeline of the entire system — ISO API integrations, backtesting harnesses, data pipelines — which disproportionately benefits small teams competing against incumbents with large engineering staffs. The state-of-the-art bidding shop of 2026 is not necessarily the one with the most exotic model; it is often the one whose five engineers ship like twenty.
+
+*Layer 2 synthesis: deploy LLMs as retrieval-grounded analysts and drafters across market rules, compliance, settlement, and monitoring, always with human review at the point of action. The advantage is quiet, defensible, and — because competitors under-invest in exactly these functions — durable.*
+
+### 4.3 Layer 3 — The optimizer: incremental by design, and that's fine
+
+The optimization mathematics was never the bottleneck. Newsvendor-style quantile offering for renewables, two-stage stochastic programs with CVaR, and MILP/dynamic-programming formulations for storage are published, taught, and tractable with commercial or open solvers. They are also **auditable**: every offer can be traced to a forecast input and a constraint, which matters enormously for both internal risk governance and regulatory defense. Any AI proposal for this layer must beat well-tuned classical methods *and* preserve explainability *and* bound its failure modes. Two candidate technologies are worth tracking, at very different maturity levels.
+
+### 4.4 Decision-focused learning: the credible upgrade
+
+The conventional pipeline trains forecasts to minimize statistical error (MSE, pinball loss) and then optimizes against them independently — without considering how forecast errors propagate into decision quality. **Decision-focused learning** (DFL, also called value-oriented forecasting or "smart predict-then-optimize") integrates the downstream optimization into the training loop, so the forecaster is trained against a regret-style loss measuring the sub-optimality of the *decisions* its forecasts induce. Founding results: Donti, Amos, and Kolter's task-based end-to-end learning (NeurIPS 2017) and Elmachtoub and Grigas's "Smart Predict-then-Optimize" (Management Science 2022); the energy literature has applied the framework to day-ahead scheduling against energy and reserve markets, storage arbitrage and predict-then-bid frameworks, and robust microgrid operation.
+
+The core insight is directly relevant to bidding: **the most accurate forecast is not necessarily the most valuable one.** A price forecast can improve its average error while getting worse in exactly the high-priced hours where offer decisions have consequences; the literature has repeatedly shown that a specific quantile choice, not the most accurate point forecast, maximizes trading value. DFL formalizes and automates that intuition. Critically — unlike end-to-end RL — it preserves the auditable optimizer: you change what the forecast is trained *for*, not who makes the decision.
+
+**Production-readiness: not broadly — late-stage research / early adoption.** What's mature: established frameworks (smart predict-then-optimize, learning-by-experience, black-box differentiable optimizers), and implicit differentiation through convex optimization layers demonstrated for economic dispatch and storage arbitrage, consistently beating accuracy-trained forecasts on realized profit for simple convex problems. Why it hasn't crossed into broad production:
+
+1. **Fragile differentiation through the optimizer** — no closed form for the backward pass; unrolling has accuracy/efficiency issues; analytical methods impose rigid problem-form requirements; mixed-integer/nonconvex bidding needs surrogates, subgradients, or perturbation estimators (brittle in practice).
+2. **Partial uncertainty coverage** — most DFL handles uncertainty in the objective only, assuming constraint parameters are known; real trading has uncertainty everywhere.
+3. **Task-specificity** — the model is welded to one decision problem; any change (asset size, market, bid format) forces retraining. Two-stage pipelines are modular: one forecast feeds many decisions.
+4. **Interpretability / risk control** — profit-trained forecasts are deliberately biased and hard to explain to risk committees; two-stage lets you audit forecast and decision separately.
+5. **Data scarcity and regime-shift fragility** — policies trained on historical profit can degrade badly exactly when markets shift.
+
+Recent evaluation work adds honest counter-evidence: DFL's benefits are application-dependent and do not always translate into higher economic value, while typically demanding substantially more computation. Adoption barriers are structural too — third-party forecast vendors cannot train against every client's private loss function, and cost-oriented forecast targets lack the intuitive interpretation (mean, median, quantile) that human reviewers rely on.
+
+**Rule of thumb:**
+
+- **Small convex inner problem + simple asset (one battery, one market):** DFL is production-viable today for a strong technical team.
+- **Integer commitments, multi-market co-optimization, ISO bid curves:** two-stage probabilistic forecasting + optimization remains the deployed standard, likely for several more years.
+- **Pragmatic middle ground** (what many shops actually do): "decision-aware" training — keep the two-stage architecture but weight forecast loss by economic consequence. Much of the benefit, little of the fragility.
+
+DFL is worth piloting *because you own your full pipeline* (an advantage of building over buying), evaluated against a strong quantile-forecast baseline on realized trading P&L, and expected to pay off most where the decision problem is asymmetric — storage bidding and imbalance-exposed renewable offers — rather than uniformly.
+
+### 4.5 Reinforcement learning: shadow mode, not production
+
+End-to-end RL — a learned policy directly emitting offer curves — remains overwhelmingly academic despite a large literature (comprehensive reviews cover 150+ papers). The structural obstacles have not moved: live trial-and-error is impossible at real-market cost, so policies train in simulators whose fidelity gap to the real market has never been closed; day-ahead auctions offer roughly 365 independent samples per year, a brutal sample-efficiency regime; and an unexplainable policy that misbehaves during a scarcity event is both a financial and a regulatory catastrophe.
+
+Where RL-adjacent methods *are* in production is instructive: **approximate/stochastic dynamic programming** — value-function methods over sequential decisions, mathematically RL's sibling — is standard inside commercial storage optimizers, and learned policies are credibly deployed in **European continuous intraday markets**, where thousands of order-placement decisions per day give RL the interaction density it needs.
+
+The sensible roadmap for a US system mirrors that pattern: classical stochastic optimization for the day-ahead auction; RL experiments confined to real-time re-offering and intraday adjustment, where decisions are frequent; mandatory shadow-mode evaluation against the production optimizer before any capital exposure; and, if RL earns its way in, deployment as a bounded residual correction on top of the optimizer's output rather than a replacement for it.
+
+### 4.6 Vendor "AI bidding" claims: substance vs. hype
+
+**Substance:** automated 24/7 bidding demonstrably works and is now *independently measurable* (Modo Energy leaderboards for storage optimizers); ML forecasting genuinely outperforms older methods; operational consistency alone beats human desks.
+
+**Hype patterns:** "AI" frequently means a standard GBM-forecast + LP-optimizer dressed up; uplift claims lack stated counterfactuals; and the edge from mere automation compresses as adoption spreads — ERCOT ancillary-services saturation being the clear example.
+
+**Due-diligence questions for any vendor:** Where do you rank on an independent benchmark? What exactly is the model architecture, and what counterfactual sits behind your uplift numbers? How did the system behave during Winter Storm Uri / the last scarcity event? What is the fee structure (revenue-share vs. fixed)?
+
+### 4.7 Cross-cutting dynamics
+
+**The floor rises; the ceiling is private.** Nearly every AI advance discussed here is public or purchasable: open weather-model weights, foundation time-series models, commodity LLMs, published DFL methods. Their diffusion compresses the edge available from any single technology — visible already in ERCOT battery markets, where algorithmic saturation of ancillary services shifted revenue toward energy arbitrage and shrank the gap between top and median operators. Durable differentiation migrates to what cannot be bought: proprietary telemetry and its use in local calibration, nodal congestion modeling, execution reliability, and the institutional discipline of a well-run risk process.
+
+**Auditability is a feature, not a constraint.** The regulatory environment (FERC manipulation doctrine, ISO market-monitor scrutiny) and internal risk governance both reward architectures whose decisions can be explained after the fact. This is a genuine, often-overlooked argument for forecast-then-optimize over end-to-end learned alternatives — and it is why the highest-value applications of the newest AI (weather ensembles, LLM operations, DFL) are precisely the ones that *strengthen* the classical architecture rather than replacing it.
+
+**Independent benchmarking is changing vendor dynamics.** Third-party leaderboards for storage optimizers have replaced self-defined vendor benchmarks with a shared reference point, and the same discipline should be applied internally: every AI adoption in the stack deserves a counterfactual (what would the incumbent method have earned?) and a tail-event stress test (how would this component have behaved during Winter Storm Uri or the most recent scarcity event?).
+
+### 4.8 Adoption roadmap
+
+Sequenced by expected risk-adjusted return on effort:
+
+1. **Now — AI weather ensembles (Layer 1).** Pilot one or more AI-weather feeds against the incumbent, scored on your assets and your trading loss function with explicit tail verification. Low integration risk, direct P&L path, and a decaying edge that rewards early movers.
+2. **Now — LLM operations layer (Layer 2).** Retrieval-grounded market-rule assistant, automated bid-rationale documentation, shadow-settlement triage, and agentic engineering tooling. Human review at every point of action.
+3. **Next two quarters — local calibration and congestion modeling (Layer 1).** Reinvest weather-layer savings into asset-specific downscaling and nodal congestion prediction; this is where forecast edge survives commoditization.
+4. **Next two quarters — DFL pilot (Layer 3).** Retrain the production-forecast loss against realized bidding regret for one asset class (storage or a single wind portfolio), evaluated against the quantile baseline on trading P&L, not forecast error.
+5. **Opportunistic — RL in shadow mode (Layer 3).** Real-time re-offering only; promotion to production contingent on sustained shadow-mode outperformance and bounded-action deployment.
+
+The through-line: recent AI does not overturn the architecture of a well-built bidding system. It makes the boring architecture better — sharper distributions in, cheaper discipline around, and a modestly smarter objective inside — which, given where the secrets actually live, is exactly the outcome a builder should want.
+
+---
+
+## 5. Building a bidding system
+
+### 5.1 Architecture for a renewables + wholesaler bidding system
+
+Five-stage pipeline: **data ingestion → probabilistic forecasting → bid optimization → submission → settlement/monitoring**, with a feedback loop retraining models on settlement outcomes.
+
+Key design decisions:
+
+- **Forecasting:** quantile forecasts (P10–P90) for production and load, built on NWP ensemble inputs blended with asset telemetry; DA LMP, RT LMP, and crucially the DA–RT *spread* distribution.
+- **Optimization core:** for a price-taking renewable, the optimal DA offer is a newsvendor quantile of the production distribution set by the expected DA/RT price ratio, wrapped in a two-stage stochastic program with CVaR. Tractable LP per asset per day (Gurobi/CPLEX/HiGHS). The wholesaler side is the mirror image: DA-vs-RT load procurement plus a hedging overlay (forwards, FTRs).
+- **Operations:** automated ISO submission with rule validation, shadow settlement, and a serious backtesting harness — the most commonly under-built component.
+- **Gotchas that dominate real P&L:** negative prices and PTC/REC-driven offer floors; nodal basis and congestion (often the highest-ROI forecasting work); PPA/hedge structures reshaping incentives; per-ISO rule differences.
+
+Note for hybrid assets: storage co-location changes the optimizer from an LP to a sequential problem (see §2.3).
+
+### 5.2 Separating hedging from speculation
+
+Separation is necessary for three reasons:
+
+1. **Performance attribution** — the two books have opposite definitions of success.
+2. **Risk governance** — bounded physical exposure vs. leveraged financial loss distributions (GreenHat as the cautionary tale).
+3. **FERC manipulation risk** — physical bids must be independently defensible; joint optimization of physical bids and financial positions is a manipulation-allegation generator.
+
+But complete separation is impossible: every DA offer quantile *is* a spread view. Resolution: define a **neutral baseline bid** (e.g., P50 offer / 100% DA load), log every deviation as "embedded alpha," and decompose P&L into baseline + embedded view + pure spec book. Architecturally: share data, forecasting, and backtesting; split optimizers, mandates, limits, and P&L; keep coupling one-directional — the physical optimizer never reads the financial book's positions.
+
+The two books consume the same forecast stack differently: the hedge book needs calibrated mid-distribution quantiles; the spec book needs tails and spread skew.
+
+Strategic note: virtuals/FTRs are crowded quant markets; embedded-view alpha inside a well-run asset book is often the better initial return on modeling effort.
+
+---
+
+## 6. China: market structure and deployment
+
+### 6.1 Market structure: different design, weaker safeguards
+
+**The market operator is NOT independent — the fundamental difference from the US.** Provincial spot markets are run by dispatch centers and trading centers *inside the grid companies* (State Grid, China Southern Grid). Trading centers are nominally independent via shareholding reform, but dispatch remains within grid organizations. Grid companies historically had commercial interests in outcomes — the "insider" is structural, not hypothetical. Dispatch/trading separation has been a reform demand since Document No. 9 (2015).
+
+**Transparency: improving but well below PJM.** A "1+6" national rule framework has been built since 2023: Basic Rules for Market Operation + spot market (2023), information disclosure (2024), medium/long-term trading, registration, ancillary services (2025), metering/settlement (2025). Provinces (Shanxi, Guangdong, Shandong) publish clearing methodology; several use centralized SCUC/SCED-style clearing, some nodal/zonal. Gaps vs. PJM: no bid-level data (even masked/lagged), little constraint-level detail, clearing-model internals undisclosed — shadow-SCED reconstruction is far harder. Rules are revised frequently by administrative notice and differ substantially by province; price caps/floors are tight, and administrative interventions are more common and less documented.
+
+**Laws and enforcement: embryonic.** No independent market monitor analogous to Monitoring Analytics — surveillance is done by trading centers/dispatch (the grid companies) plus thinly-staffed NEA bureaus. No developed body of power-market manipulation case law; general laws are untested on spot-power conduct. SOE-dominated participation and administrative dispute resolution mean little deterrence-by-precedent.
+
+**Net assessment.** All three PJM safeguards (§3.2) are weaker in China: partial algorithm disclosure, structural data asymmetry favoring grid-affiliated entities, nascent enforcement. The realistic information edge comes from **proximity to grid/dispatch institutions**, not rogue vendors. Mitigants: heavy medium/long-term contract coverage (~47% of consumption in 2024) plus price caps mean less money at stake in spot per unit of information advantage, and the reform direction clearly points toward disclosure and oversight. **Watch:** whether dispatch is ever truly separated from the grid companies. National unified market targeted ~2029; seven provinces in full spot operation as of 2025 (Shanxi, Guangdong, Shandong, Gansu, W. Inner Mongolia, Hubei, Zhejiang).
+
+### 6.2 Weather data sourcing for a China deployment
+
+Baseline reviewed stack — ECMWF HRES+EPS backbone, GFS/GEFS secondary, CMA regional models + observations, Himawari/Fengyun satellite irradiance, customer met masts, buy-first on WRF downscaling — is conventionally sound. Recommended changes:
+
+1. **Add AI weather models — the glaring omission.** AIFS open data plus open-weight models (GraphCast/Pangu) densify the ensemble at near-zero marginal cost; China's domestic AI-weather ecosystem (Pangu, Fuxi, FengWu, CMA-integrated) is both cheap and a compliance asset; GPU-cheap models unlock the intraday refresh cadence the licensed stack can't provide.
+2. **Add ERA5/reanalysis on day one** — the training substrate for calibration models and backtesting.
+3. **Make the blending/calibration layer explicit** with its own budget line — multi-model ensembles calibrated per-site against ground truth is where the money is; source diversity is calibration fuel.
+4. **Specify a fallback hierarchy** (EPS → AIFS open → GFS → persistence) as an engineering requirement.
+5. **Two-way compliance review early:** ECMWF redistribution terms × Chinese regulations on foreign-data commercial use and domestic-observation export — this constrains where the blending layer can physically run.
+6. **Add inverter/SCADA data as ground truth** (free, dense, bakes in real plant behavior), plus a QC pipeline for customer sensors.
+7. **Reframe "self-run WRF" as "self-run downscaling," method open** — learned downscaling will likely beat WRF on cost by decision time.
+8. **Design the satellite-nowcast → NWP blending handover** in the 0–4h window explicitly, to avoid a discontinuity in the intraday trading horizon.
+
+---
+
+## 7. Open questions
+
+Carried forward from the discussions:
+
+- **Which market(s)?** US ISOs vs. Chinese provincial spot pilots — gate-closure times and settlement rules change the forecast-refresh requirements.
+- **Asset mix?** Solar/wind/hybrid-with-storage — storage co-location changes the optimizer from an LP to a sequential problem.
+- **Risk mandate?** Appetite for a standalone financial (virtuals/FTR) book vs. embedded-alpha-only inside the asset book.
+
+---
+
+## 8. Sources and further reading
+
+- GraphCast: *Learning skillful medium-range global weather forecasting*, Science (Dec 2023). https://www.science.org/doi/10.1126/science.adi2336
+- Pangu-Weather: *Accurate medium-range global weather forecasting with 3D neural networks*, Nature (Jul 2023). https://www.nature.com/articles/s41586-023-06185-3
+- Aurora: *A foundation model for the Earth system*, Nature (May 2025). https://www.nature.com/articles/s41586-025-09005-y
+- ECMWF AIFS operational status (2024). https://www.ecmwf.int/en/about/media-centre/news/2024/ecmwfs-ai-forecasts-become-operational
+- Donti, Amos, Kolter: *Task-based end-to-end model learning in stochastic optimization*, NeurIPS 2017. https://arxiv.org/abs/1703.04529
+- Elmachtoub & Grigas: *Smart "Predict, then Optimize"*, Management Science (2022). https://arxiv.org/abs/1710.08005
+- Decision-focused learning for energy/reserve market participation. https://www.researchgate.net/publication/383107857
+- Decision-focused predict-then-bid for strategic storage. https://arxiv.org/abs/2505.01551
+- Statistical vs. economic evaluation of electricity-market forecasts (DFL counter-evidence). https://arxiv.org/abs/2511.13616
+- RL in deregulated energy markets: comprehensive review (150+ papers). https://www.sciencedirect.com/science/article/abs/pii/S0306261922014696
+- On-line RL for real-life energy trading (simulator-gap discussion). https://arxiv.org/abs/2303.16266
+- Modo Energy: independent benchmarking of storage optimizers. https://modoenergy.com
+
+*Prepared July 2026. Vendor performance claims cited in the underlying research should be independently verified against your own assets before procurement decisions.*