vpp-ai-platform/packages/evals/reports/baselines/l2-synthetic-hubei-v0.json
Thomas Bayes 8796faca63 M2: skill contracts, Python skill service, L2 eval harness with baseline
- packages/domain: ForecastRequest, BidOptimizationRequest/Result,
  ReportRequest, SkillReport (+ golden and invalid fixtures, exported to
  contracts/ and regenerated as pydantic models).
- skills-py/vpp_skills: FastAPI service with versioned registry; load/PV/
  price forecasts (same-day-type EWM point forecast, conformal residual
  quantiles — coverage test as acceptance gate); bid-optimization MILP on
  HiGHS (binary block participation, hard ledger energy bounds, exact
  Decimal fit of the rounded curve inside the bounds, revenue distribution
  over quantile paths); report generator whose every figure is a
  {tool_call_id, path} reference, with a verifier. 48 tests incl. hypothesis
  property test that bids respect ledger constraints.
- packages/services: LedgerService.dayAheadBounds (the P7 cascade band
  handed to the optimizer); Decimal resolved once for CJS/ESM interop.
- packages/evals: L2 metrics (MAPE, nRMSE, coverage, direction accuracy,
  naive/hindsight revenue baselines), HTTP skill client, rolling-origin
  harness that pushes each bid through the real ledger, CLI with
  --check/--write-baseline; committed baseline on the SYNTHETIC dataset
  (no historical Hubei data yet — baselines measure the harness, not KPI).
- CI: evals job boots the skill service and fails on baseline digest drift.
- docs/open-questions: A6 (flexibility marginal cost = offer floor); A4/B6
  wired as placeholders. README/CLAUDE.md status → M2 done, M3 next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoYoGYzHkFyv3ALenkRPhA
2026-09-02 06:29:08 -04:00

63 lines
1.5 KiB
JSON

{
"layer": "L2",
"dataset": {
"name": "synthetic-hubei-v0-2026-01-01-120d",
"sha256": "3aa36eb46906233b1b311ff03a33e7d663f30ae921fc9a1f3f813ab5205c270a",
"synthetic": true
},
"skill_versions": {
"load-forecast": "1.0.0",
"pv-forecast": "1.0.0",
"price-forecast": "1.0.0",
"bid-optimization-milp": "1.0.0",
"report-generator": "1.0.0"
},
"config": {
"window": 28,
"holdoutFrom": 60,
"holdoutDays": 0,
"risk": {
"risk_aversion": "0.3",
"commitment_buffer_k": "0.9",
"min_block_mwh": "0.5",
"marginal_cost_yuan_per_mwh": "0"
},
"contractShareOfSellable": "0.6",
"daMonthlyDeviationBand": "0.05"
},
"metrics": {
"load": {
"days": 60,
"mape": 0.041,
"nrmse": null,
"coverage_p10_p90": 0.821007,
"direction_accuracy": null
},
"pv": {
"days": 60,
"mape": null,
"nrmse": 0.040938,
"coverage_p10_p90": 0.772109,
"direction_accuracy": null
},
"price": {
"days": 60,
"mape": 0.044013,
"nrmse": null,
"coverage_p10_p90": 0.81059,
"direction_accuracy": 0.962142
},
"bid": {
"days": 60,
"optimal_days": 60,
"ledger_accepted_days": 60,
"revenue_skill_yuan": 7146184.79,
"revenue_naive_yuan": 6012053.56,
"revenue_hindsight_yuan": 7154899.25,
"capture_ratio": 0.998782,
"uplift_vs_naive": 1.188643
}
},
"digest": "4018e32ab695422c613e211aba1392164f3bb9c30076986fdfda220fc9bba047"
}