vpp-ai-platform/skills-py/vpp_contracts/kpi_report.py

94 lines
2.7 KiB
Python
Raw Normal View History

M5: shadow run, kill-switch hierarchy L0–L4, KPI dashboard, shadow replay CLI Shadow run (ROADMAP M5, phase-1 acceptance form): SHADOW runtime mode records bids without submitting them, simulates the market answer from the actual day-ahead clearing price, dispatches to the simulation gateway, runs the D+1 review, and scores every day shadow-vs-human-vs-hindsight (ShadowDayRecord) with a lineage-completeness audit. KpiReport regenerated after every day per docs/12 §4 definitions (C1/C2 placeholders as named config). Kill switches (docs/13 §8): BreakerService with L0 permit revocation, L1 envelope suspension, L2 loss breaker (mark-to-market, reduce-only bids), L3 channel breaker (bids fall back to the file channel, dispatch BLOCKED), L4 AI-off (templates run, no Proposal created); abnormal-day protocol on EXTREME situations; per-level authority (B8 placeholder); auditable drill. Runtime: shadow-close workflow, shadow schedule entries, live-data ingestion through the quality gate, human-bid ingestion, breaker/KPI/shadow endpoints and insight cards, replay CLI (npm run shadow). Ledger, time series and streak counters are file-backed so a multi-week run survives restarts. Domain: HumanBidRecord, ShadowDayRecord, KpiReport, BreakerRecord, SHADOW receipt channel; contracts, fixtures and pydantic models regenerated. Services: L2 metrics moved from evals so the shadow run and the harness share one implementation. Fixes: envelope/permit validity compared ISO timestamps as strings ('…00Z' vs '…00.000Z'); L2 baseline was stale since M4 (skill_versions only, metrics unchanged) — rewritten from the live service. Docs: docs/14 shadow-run runbook (timeline, breaker trigger/authority/ recovery, KPI definitions as implemented); README and CLAUDE.md status. Tests: 21 consecutive shadow days with complete lineage, KPI report, WIDEN recommendation produced but not acted on; restart durability; drill; L2/L3/L4 and abnormal-day paths; API. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017wrZgPL9LoKaD69BpEQU4v
2026-09-02 22:54:14 -04:00
# generated by datamodel-codegen:
# filename: kpi_report.json
from __future__ import annotations
from enum import StrEnum
from pydantic import AwareDatetime, BaseModel, ConfigDict, Field, conint, constr
class Window(BaseModel):
model_config = ConfigDict(
extra='forbid',
)
from_: constr(pattern=r'^\d{4}-\d{2}-\d{2}$') | None = Field(..., alias='from')
to: constr(pattern=r'^\d{4}-\d{2}-\d{2}$') | None
days: conint(ge=0, le=9007199254740991)
class Id(StrEnum):
FORECAST_LOAD_MAPE = 'FORECAST_LOAD_MAPE'
FORECAST_PV_NRMSE = 'FORECAST_PV_NRMSE'
POTENTIAL_ACCURACY = 'POTENTIAL_ACCURACY'
DECISION_LATENCY_P95_MS = 'DECISION_LATENCY_P95_MS'
DISPATCH_SUCCESS_RATE = 'DISPATCH_SUCCESS_RATE'
REVENUE_UPLIFT_VS_HUMAN = 'REVENUE_UPLIFT_VS_HUMAN'
CROSS_REGION_MATCH = 'CROSS_REGION_MATCH'
class Comparator(StrEnum):
LTE = 'LTE'
GTE = 'GTE'
class Status(StrEnum):
MEET = 'MEET'
MISS = 'MISS'
NO_DATA = 'NO_DATA'
NOT_APPLICABLE = 'NOT_APPLICABLE'
class Kpi(BaseModel):
model_config = ConfigDict(
extra='forbid',
)
id: Id
value: constr(pattern=r'^-?\d+(\.\d+)?$') | None
unit: constr(min_length=1)
target: constr(pattern=r'^-?\d+(\.\d+)?$') | None
comparator: Comparator
samples: conint(ge=0, le=9007199254740991)
status: Status
definition: constr(min_length=1)
class Comparison(BaseModel):
model_config = ConfigDict(
extra='forbid',
)
shadow_yuan: constr(pattern=r'^-?\d+(\.\d+)?$')
human_yuan: constr(pattern=r'^-?\d+(\.\d+)?$') | None
hindsight_yuan: constr(pattern=r'^-?\d+(\.\d+)?$')
naive_yuan: constr(pattern=r'^-?\d+(\.\d+)?$')
capture_ratio: constr(pattern=r'^-?\d+(\.\d+)?$') | None
uplift_vs_naive: constr(pattern=r'^-?\d+(\.\d+)?$') | None
days_with_human_baseline: conint(ge=0, le=9007199254740991)
class Shadow(BaseModel):
model_config = ConfigDict(
extra='forbid',
)
days: conint(ge=0, le=9007199254740991)
complete_days: conint(ge=0, le=9007199254740991)
consecutive_complete_days: conint(ge=0, le=9007199254740991)
first_date: constr(pattern=r'^\d{4}-\d{2}-\d{2}$') | None
last_date: constr(pattern=r'^\d{4}-\d{2}-\d{2}$') | None
released_days: conint(ge=0, le=9007199254740991)
pending_days: conint(ge=0, le=9007199254740991)
widen_recommendations: conint(ge=0, le=9007199254740991)
breaker_trips: conint(ge=0, le=9007199254740991)
class KpiReport(BaseModel):
model_config = ConfigDict(
extra='forbid',
)
id: constr(min_length=1)
window: Window
kpis: list[Kpi]
comparison: Comparison
shadow: Shadow
generated_at: AwareDatetime