vpp-ai-platform/packages/evals/package.json

25 lines
455 B
JSON
Raw Normal View History

M2: skill contracts, Python skill service, L2 eval harness with baseline - packages/domain: ForecastRequest, BidOptimizationRequest/Result, ReportRequest, SkillReport (+ golden and invalid fixtures, exported to contracts/ and regenerated as pydantic models). - skills-py/vpp_skills: FastAPI service with versioned registry; load/PV/ price forecasts (same-day-type EWM point forecast, conformal residual quantiles — coverage test as acceptance gate); bid-optimization MILP on HiGHS (binary block participation, hard ledger energy bounds, exact Decimal fit of the rounded curve inside the bounds, revenue distribution over quantile paths); report generator whose every figure is a {tool_call_id, path} reference, with a verifier. 48 tests incl. hypothesis property test that bids respect ledger constraints. - packages/services: LedgerService.dayAheadBounds (the P7 cascade band handed to the optimizer); Decimal resolved once for CJS/ESM interop. - packages/evals: L2 metrics (MAPE, nRMSE, coverage, direction accuracy, naive/hindsight revenue baselines), HTTP skill client, rolling-origin harness that pushes each bid through the real ledger, CLI with --check/--write-baseline; committed baseline on the SYNTHETIC dataset (no historical Hubei data yet — baselines measure the harness, not KPI). - CI: evals job boots the skill service and fails on baseline digest drift. - docs/open-questions: A6 (flexibility marginal cost = offer floor); A4/B6 wired as placeholders. README/CLAUDE.md status → M2 done, M3 next. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UoYoGYzHkFyv3ALenkRPhA
2026-09-02 06:29:08 -04:00
{
"name": "@vpp/evals",
"version": "0.1.0",
"private": true,
"type": "module",
"exports": {
".": "./src/index.ts"
},
"scripts": {
"typecheck": "tsc -p tsconfig.json",
"test": "vitest run",
"eval": "tsx src/cli.ts"
},
"dependencies": {
"@vpp/domain": "*",
"@vpp/services": "*"
},
"devDependencies": {
"@types/node": "^26.4.1",
"tsx": "^4.19.0",
"typescript": "^5.6.0",
"vitest": "^3.0.0"
}
}