vpp-ai-platform/docs/14-shadow-run-runbook.md
Thomas Bayes 381b6d3521
Some checks failed
ci / typescript (push) Has been cancelled
ci / python (push) Has been cancelled
ci / evals (push) Has been cancelled
M5: shadow run, kill-switch hierarchy L0–L4, KPI dashboard, shadow replay CLI
Shadow run (ROADMAP M5, phase-1 acceptance form): SHADOW runtime mode records
bids without submitting them, simulates the market answer from the actual
day-ahead clearing price, dispatches to the simulation gateway, runs the D+1
review, and scores every day shadow-vs-human-vs-hindsight (ShadowDayRecord)
with a lineage-completeness audit. KpiReport regenerated after every day per
docs/12 §4 definitions (C1/C2 placeholders as named config).

Kill switches (docs/13 §8): BreakerService with L0 permit revocation, L1
envelope suspension, L2 loss breaker (mark-to-market, reduce-only bids),
L3 channel breaker (bids fall back to the file channel, dispatch BLOCKED),
L4 AI-off (templates run, no Proposal created); abnormal-day protocol on
EXTREME situations; per-level authority (B8 placeholder); auditable drill.

Runtime: shadow-close workflow, shadow schedule entries, live-data ingestion
through the quality gate, human-bid ingestion, breaker/KPI/shadow endpoints
and insight cards, replay CLI (npm run shadow). Ledger, time series and
streak counters are file-backed so a multi-week run survives restarts.

Domain: HumanBidRecord, ShadowDayRecord, KpiReport, BreakerRecord, SHADOW
receipt channel; contracts, fixtures and pydantic models regenerated.
Services: L2 metrics moved from evals so the shadow run and the harness
share one implementation.

Fixes: envelope/permit validity compared ISO timestamps as strings
('…00Z' vs '…00.000Z'); L2 baseline was stale since M4 (skill_versions only,
metrics unchanged) — rewritten from the live service.

Docs: docs/14 shadow-run runbook (timeline, breaker trigger/authority/
recovery, KPI definitions as implemented); README and CLAUDE.md status.

Tests: 21 consecutive shadow days with complete lineage, KPI report, WIDEN
recommendation produced but not acted on; restart durability; drill; L2/L3/L4
and abnormal-day paths; API.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wrZgPL9LoKaD69BpEQU4v
2026-09-02 22:54:14 -04:00

6.0 KiB
Raw Permalink Blame History

14 · 影子运行手册(M5 一期验收形态)

08 篇 §4 与 ROADMAP M5 的运行手册:影子运行怎么跑、熔断层级的「三要素」、 KPI 口径在代码中的落点。本篇是运行侧文档,架构决策仍以 03/12/13 篇为准。

1. 影子运行是什么

全链路接实时数据,一切外部效果仿真:申报只生成不提交(SHADOW 通道回执), 出清结果由当日实际日前出清价对影子申报仿真撮合(统一出清价规则),调度方案投递仿真网关, D+1 复盘照常。每个市场日产出一条 ShadowDayRecord(影子 vs 人工 vs 后见之明三线对比 + 血缘完整性审计),并自动重算 KpiReport。运行 20+ 个连续完整血缘日即满足 M5 验收条件。

时点(Asia/Shanghai) 触发项 流程模板
D-1 06:00 situation-0600 day-ahead-situation
D-1 08:00 bid-0800 day-ahead-bid → proposal-lifecycle(影子网关)
D-1 16:00 shadow-clearing-1600 影子撮合 → award-decomposition → proposal-lifecycle(仿真网关)
D+1 02:00 shadow-close-0200 shadow-close:仿真执行 → 计量 → review → 三线对比 → KPI

数据前提:D 日实际负荷/光伏/出清价须在 D-1 16:00 前经 POST /market-data 接入 (走质量门禁,不合格曲线隔离并留痕);人工实际申报经 POST /human-bids 接入 (无人工申报时对比记录 human = null,收益提升 KPI 记 NO_DATA)。 触发失败(如出清价未到)记 ScheduledTriggerFailed 事件并在下一轮巡检重试。

  • 实时模式:VPP_MODE=SHADOW npm run start -w @vpp/runtime
  • 历史重放:npm run shadow -w @vpp/runtime -- --days 21(人工基线可用 --human-baseline naive 生成占位,标记 SYNTHETIC_NAIVE,验收时必须替换为真实人工申报)

影子运行的两条建模假设(不是业务参数):仿真执行的履约率取单元内资源可靠性评分均值 (或固定比例,shadow.fulfillment);影子 P&L = 实际价撮合收益 − 边际成本 × 成交电量。

2. 熔断层级三要素(13 篇 §8 的落地)

级别 触发条件 授权岗位(breaker.authority,B8 待定) 效果(代码落点) 恢复条件
L0 单笔 人工判定某许可需撤 senior-trader / ops-lead / risk-officer authority.revoke → 网关拒收该 (Proposal, Permit) 新许可须重走现势复核;L0 记录由人工复位
L1 单包络 自动:连续 N 次偏差超阈(B4);人工 ops-lead / risk-officer envelopes.suspend → 该类动作回归人工 仅经 envelope-review 人工再批准(REACTIVATE)
L2 资金 自动:单日预期损失 > breaker.daily_loss_budget(B5) risk-officer / ops-lead 包络门全部改人工;规则校核只放行减仓类申报(breaker-l2-reduce-only) 人工复位并填写依据(复盘结论编号 + 量化条件)
L3 通道 人工:申报/控制通道故障 ops-lead / platform-admin 调度方案 AUTHORIZED 但 BLOCKED(边缘进入断连策略);申报转人工文件通道 通道恢复后人工复位
L4 全平台 人工:AI 建议停用 platform-admin / ops-lead 周期模板照跑出数据,不生成 Proposal(ProposalSuppressed) 人工复位
  • 自动触发只允许 L1/L2(breaker.automatic),执行者记为 runtime/system;复位永远是人。
  • 智能体身份(*-agent)对任何级别无操作权(I1/I2)。
  • 每次触发/复位写 BreakerTripped / BreakerReset 事件;GET /breakers 给出五级状态。
  • 演练:POST /breakers/drill(或 runBreakerDrill)对五级逐一「触发→在链路上验证效果→复位」, 全部使用演练对象,不触碰在途 Proposal;事件 BreakerDrillStep / BreakerDrillCompleted 留痕。 ROADMAP M5 要求演练一次,13 篇要求每季度至少一级。

异常日协议(13 篇 §1):态势报告 EXTREME(B7 触发条件待定,当前以价格区间比占位)→ 该市场日自动登记为异常日,所有 Proposal 走人工(abnormal-day protocol 原因进收件箱), 同时开专题工单;人工可 POST /abnormal-days/:date 登记或 /clear 解除。

3. KPI 口径(12 篇 §4 → computeKpiReport)

KPI 实现口径 目标(占位,C1/C2 待冻结)
FORECAST_LOAD_MAPE 聚合负荷 P50 vs 计量 96 点 MAPE,窗口均值 ≤ 0.08
FORECAST_PV_NRMSE 光伏 P50 RMSE / 装机(登记的 PV 资源额定功率之和,缺省 shadow.pvCapacityMw) ≤ 0.10(占位)
POTENTIAL_ACCURACY 已调度日中 |计划 − 履约| / 计划 ≤ 容差 的占比 ≥ 0.90,容差 0.10(占位)
DECISION_LATENCY_P95_MS 工单开启 → Proposal 到达 AUTO_APPROVED/PENDING_HUMAN 的 P95(不含人工等待) ≤ 180000
DISPATCH_SUCCESS_RATE 持有效许可的调度方案中,回执确认且偏差 ≤ 带宽 的占比 ≥ 0.98,带宽 10%(占位)
REVENUE_UPLIFT_VS_HUMAN 同日、同实际价撮合下 (Σ影子收益 − Σ人工收益) / Σ人工收益 ≥ 0.15(基线定义须冻结)
CROSS_REGION_MATCH 二期联邦工件,NOT_APPLICABLE ≥ 0.85

窗口 = 最近 kpi.windowDays(默认 30)个影子日;每条 KPI 附口径文字,验收时以报告中的文字为准签认。 另附三线合计(影子/人工/后见之明/朴素)、capture_ratio、连续完整血缘天数、 包络放宽建议数(只记录不执行——放宽须经 envelope-review 人工批准,属二期治理动作)。

4. 相关配置键

mode、breaker.dailyLossBudgetYuan(B5)、breaker.authority(B8)、breaker.automatic、 kpi.*(C1/C2)、shadow.fulfillment、shadow.pvCapacityMw(C2)、extremeDayPriceRatio(B7)、 widenAfterCompliantDays(B4/B9)。全部在 packages/runtime/src/runtime.ts 的 DEFAULT_RUNTIME_CONFIG 以占位值出现,并带 OPEN-QUESTION 注释。