Fáze 2.2: penalty audit — 16 z 26 penalt mrtvých na golden fixtures
scripts/harness/penalty_audit.py: vynulování každé ekonomické konstanty → replay 4 golden fixtures → Δcashflow / Δpenalty / změněné sloty. Výsledek (penalty_audit_baseline_2026-06-11.txt): - 16/26 penalt bez jakéhokoli vlivu na 4 reprezentativních scénářích - aktivní penalty silně interagují (odstranění jedné zvedne binding jiných o stovky Kč — POS_SELL_PRE_NEG +481, PRE_NEG_BUY_SOC_CEILING −1480) - Σpenalty 2140 Kč vs cashflow −440 Kč na baseline CLAUDE.md: doplněna struktura services/planning/ a harness do tabulky adresářů. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dusan Vojacek
@@ -49,6 +49,8 @@ Když uživatel napíše **„použij MCP“** nebo potřebuje **aktuální řá
|
||||
| `db/routines/` | Repeatable SQL: funkce `ems.fn_*` |
|
||||
| `db/views/` | Repeatable SQL: view `ems.vw_*` |
|
||||
| `backend/services/` | Python služby (v repozitáři zatím hlavně plánování) |
|
||||
| `backend/services/planning/` | Moduly plánovače: `constants` (vč. všech ekonomických penalt), `types`, `forecast`, `db_io`, `heuristics`; `planning_engine.py` = solver + orchestrace + fasáda (re-export, importy beze změny) |
|
||||
| `backend/tests/golden/` + `scripts/harness/` | Ekonomický regresní harness: golden replay gate (`test_golden_replay.py`), `extract_fixtures.py`, `economics_report.py`, `penalty_audit.py` — viz `scripts/harness/README.md`; **při změně plánovače musí projít golden gate** |
|
||||
|
||||
---
|
||||
|
||||
|
||||
147
scripts/harness/penalty_audit.py
Normal file
147
scripts/harness/penalty_audit.py
Normal file
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Fáze 2 – penalty audit: změř přínos každé heuristické penalty v objective.
|
||||
|
||||
Pro každou ekonomickou konstantu (penalty/reward/bonus/surcharge) ji vynuluje
|
||||
ve VŠECH modulech, kde je importovaná (fasáda: planning_engine, planning.heuristics,
|
||||
planning.constants), přehraje golden fixtures (tests/golden/fixtures/) přes
|
||||
solve_dispatch_two_pass a porovná s baseline:
|
||||
|
||||
- Δcashflow : změna reálných peněz plánu (− = plán vydělá víc bez penalty),
|
||||
- Δslotů : kolik slotů změnilo battery/grid setpoint (chování),
|
||||
- bind : jestli penalta vůbec něco dělá (Δ=0 na všech fixtures = mrtvá).
|
||||
|
||||
POZOR na interpretaci: penalty mění modelované peníze za robustnost vůči chybě
|
||||
forecastu. Záporná Δcashflow ⇒ penalta v modelu stojí peníze — to samo o sobě
|
||||
neznamená „smazat“; mrtvé penalty (bind=NE) ale smazat lze bezpečně.
|
||||
Druhý krok auditu = economics_report nad reálnými dny.
|
||||
|
||||
Spouštět z backend/: python3 ../scripts/harness/penalty_audit.py [--only NÁZEV]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
BACKEND = Path(__file__).resolve().parents[2] / "backend"
|
||||
sys.path.insert(0, str(BACKEND))
|
||||
|
||||
from services import planning_engine as pe # noqa: E402
|
||||
from services.planning import constants as C # noqa: E402
|
||||
from services.planning import heuristics as H # noqa: E402
|
||||
|
||||
# replay funkce z golden testu (bez duplikace logiky)
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"golden_replay", BACKEND / "tests" / "test_golden_replay.py"
|
||||
)
|
||||
_golden = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(_golden)
|
||||
|
||||
FIXTURES = sorted((BACKEND / "tests" / "golden" / "fixtures").glob("*.json"))
|
||||
|
||||
# Ekonomické konstanty k auditu: penalty/reward/bonus/surcharge/incentive/discourage
|
||||
AUDIT_PATTERN = re.compile(
|
||||
r"(PENALTY|SHORTFALL|DISCOURAGE|REWARD|SURCHARGE|INCENTIVE|BONUS)", re.I
|
||||
)
|
||||
MODULES = (pe, C, H)
|
||||
|
||||
|
||||
def _audit_names() -> list[str]:
|
||||
names = []
|
||||
for n in dir(C):
|
||||
if n.startswith("_") or not n.isupper():
|
||||
continue
|
||||
if AUDIT_PATTERN.search(n) and isinstance(getattr(C, n), (int, float)):
|
||||
names.append(n)
|
||||
return sorted(names)
|
||||
|
||||
|
||||
def _set_const(name: str, value: float) -> dict:
|
||||
saved = {}
|
||||
for mod in MODULES:
|
||||
if hasattr(mod, name):
|
||||
saved[mod.__name__] = getattr(mod, name)
|
||||
setattr(mod, name, value)
|
||||
return saved
|
||||
|
||||
|
||||
def _restore_const(name: str, saved: dict) -> None:
|
||||
for mod in MODULES:
|
||||
if mod.__name__ in saved:
|
||||
setattr(mod, name, saved[mod.__name__])
|
||||
|
||||
|
||||
def _replay_all() -> dict[str, dict]:
|
||||
out = {}
|
||||
for path in FIXTURES:
|
||||
fixture = json.loads(path.read_text(encoding="utf-8"))
|
||||
out[path.stem] = _golden._replay_fixture(fixture)
|
||||
return out
|
||||
|
||||
|
||||
def _diff(base: dict, new: dict) -> tuple[float, float, int]:
|
||||
"""(Δcashflow, Δpenalty, změněné sloty) napříč fixtures."""
|
||||
d_cash = d_pen = 0.0
|
||||
changed = 0
|
||||
for key, b in base.items():
|
||||
n = new[key]
|
||||
d_cash += n["totals"]["cashflow_czk"] - b["totals"]["cashflow_czk"]
|
||||
d_pen += n["totals"]["penalty_czk"] - b["totals"]["penalty_czk"]
|
||||
for rb, rn in zip(b["slots"], n["slots"]):
|
||||
if (
|
||||
rb["battery_setpoint_w"] != rn["battery_setpoint_w"]
|
||||
or rb["grid_setpoint_w"] != rn["grid_setpoint_w"]
|
||||
or rb["pv_a_curtailed_w"] != rn["pv_a_curtailed_w"]
|
||||
):
|
||||
changed += 1
|
||||
return d_cash, d_pen, changed
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--only", default=None, help="audit jen jedné konstanty")
|
||||
args = ap.parse_args()
|
||||
|
||||
names = [args.only] if args.only else _audit_names()
|
||||
print(f"# Penalty audit — {len(names)} konstant × {len(FIXTURES)} fixtures")
|
||||
print("# Δcashflow: záporné = plán bez penalty vydělá víc (modelované Kč za horizonty fixtures)")
|
||||
print()
|
||||
|
||||
baseline = _replay_all()
|
||||
base_cash = sum(r["totals"]["cashflow_czk"] for r in baseline.values())
|
||||
base_pen = sum(r["totals"]["penalty_czk"] for r in baseline.values())
|
||||
print(f"baseline: cashflow {base_cash:.1f} Kč, penalty {base_pen:.1f} Kč\n")
|
||||
|
||||
header = f"{'konstanta':<55} {'hodnota':>9} {'Δcash':>8} {'Δpenalty':>9} {'Δsloty':>6} bind"
|
||||
print(header)
|
||||
print("-" * len(header))
|
||||
|
||||
rows = []
|
||||
for name in names:
|
||||
value = getattr(C, name)
|
||||
saved = _set_const(name, 0.0)
|
||||
try:
|
||||
result = _replay_all()
|
||||
except Exception as exc: # infeasible apod. — informace sama o sobě
|
||||
print(f"{name:<55} {value:>9} {'ERROR':>8} {str(exc)[:40]}")
|
||||
_restore_const(name, saved)
|
||||
continue
|
||||
_restore_const(name, saved)
|
||||
d_cash, d_pen, changed = _diff(baseline, result)
|
||||
bind = "NE (mrtvá?)" if changed == 0 and abs(d_cash) < 0.05 else "ano"
|
||||
rows.append((name, value, d_cash, d_pen, changed, bind))
|
||||
print(f"{name:<55} {value:>9} {d_cash:>8.1f} {d_pen:>9.1f} {changed:>6} {bind}")
|
||||
|
||||
dead = [r[0] for r in rows if r[5].startswith("NE")]
|
||||
print(f"\nMrtvé penalty (žádný vliv na 4 fixtures): {len(dead)}")
|
||||
for n in dead:
|
||||
print(f" - {n}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
50
scripts/harness/penalty_audit_baseline_2026-06-11.txt
Normal file
50
scripts/harness/penalty_audit_baseline_2026-06-11.txt
Normal file
@@ -0,0 +1,50 @@
|
||||
# Δcashflow: záporné = plán bez penalty vydělá víc (modelované Kč za horizonty fixtures)
|
||||
|
||||
baseline: cashflow -440.4 Kč, penalty 2140.1 Kč
|
||||
|
||||
konstanta hodnota Δcash Δpenalty Δsloty bind
|
||||
-------------------------------------------------------------------------------------------------
|
||||
CURTAILMENT_PENALTY 0.001 0.0 -134.4 0 NE (mrtvá?)
|
||||
EVENING_PUSH_Z_EXPORT_BONUS_CZK 2500.0 0.0 0.0 0 NE (mrtvá?)
|
||||
LOAD_FIRST_INCENTIVE_CZK_KWH 0.05 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_BUY_CHARGE_SHORTFALL_PENALTY_CZK_KWH 100.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_EVENING_PREP_DISCHARGE_SHORTFALL_PENALTY_CZK_KWH 120.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_EVENING_RESERVE_SOC_SLACK_PENALTY_CZK_PER_WH 55.0 1.0 -11.7 3 ano
|
||||
NEG_SELL_BAT_DUMP_SHORTFALL_PENALTY_CZK_KWH 80.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_CURTAIL_PENALTY_CZK_KWH 1.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_POST_DETACH_BCPV_DISCOURAGE_CZK_KWH 250.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_PREP_HOLD_BCPV_PENALTY_CZK_KWH 60.0 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_PREP_SOC_SHORTFALL_PENALTY_CZK_PER_WH 0.85 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_PV_B_VENT_PENALTY_CZK_KWH 4.0 7.5 -50.9 19 ano
|
||||
NEG_SELL_PV_CHARGE_REWARD_CZK_KWH 0.8 0.0 0.0 0 NE (mrtvá?)
|
||||
NEG_SELL_SOC_UNDERFILL_PENALTY_CZK_PER_WH 0.35 1.3 -539.8 13 ano
|
||||
NIGHT_SELF_CONSUME_IMPORT_SURCHARGE_CZK_KWH 4.0 0.3 -0.5 10 ano
|
||||
PEAK_EXPORT_SHORTFALL_PENALTY_CZK_KWH 80.0 33.1 -18.0 19 ano
|
||||
POS_SELL_PRE_NEG_SOC_SHORTFALL_PENALTY_CZK_PER_WH 0.3 -20.6 481.2 13 ano
|
||||
PRENEG_SELL_SOC_ANCHOR_SLACK_PENALTY_CZK_PER_WH 0.2 0.0 0.0 0 NE (mrtvá?)
|
||||
PRE_NEG_BATT_EXPORT_SHORTFALL_PENALTY_CZK_KWH 80.0 -2.6 0.0 23 ano
|
||||
PRE_NEG_BUY_EMPTY_EXPORT_SHORTFALL_PENALTY_CZK_KWH 80.0 0.0 0.0 0 NE (mrtvá?)
|
||||
PRE_NEG_BUY_PV_CHARGE_PENALTY_CZK_KWH 250.0 1.3 -78.4 12 ano
|
||||
PRE_NEG_BUY_SOC_CEILING_SLACK_PENALTY_CZK_PER_WH 0.25 20.3 -1480.0 33 ano
|
||||
PRE_NEG_CHARGE_PENALTY_CZK_KWH 400.0 0.0 0.0 0 NE (mrtvá?)
|
||||
PRE_NEG_PV_BCPV_DISCOURAGE_CZK_KWH 90.0 0.0 0.0 0 NE (mrtvá?)
|
||||
PRE_NEG_PV_EXPORT_SHORTFALL_PENALTY_CZK_KWH 55.0 0.0 0.0 0 NE (mrtvá?)
|
||||
PV_CHARGE_SHORTFALL_PENALTY_CZK_KWH 120.0 0.3 -473.9 9 ano
|
||||
|
||||
Mrtvé penalty (žádný vliv na 4 fixtures): 16
|
||||
- CURTAILMENT_PENALTY
|
||||
- EVENING_PUSH_Z_EXPORT_BONUS_CZK
|
||||
- LOAD_FIRST_INCENTIVE_CZK_KWH
|
||||
- NEG_BUY_CHARGE_SHORTFALL_PENALTY_CZK_KWH
|
||||
- NEG_EVENING_PREP_DISCHARGE_SHORTFALL_PENALTY_CZK_KWH
|
||||
- NEG_SELL_BAT_DUMP_SHORTFALL_PENALTY_CZK_KWH
|
||||
- NEG_SELL_CURTAIL_PENALTY_CZK_KWH
|
||||
- NEG_SELL_POST_DETACH_BCPV_DISCOURAGE_CZK_KWH
|
||||
- NEG_SELL_PREP_HOLD_BCPV_PENALTY_CZK_KWH
|
||||
- NEG_SELL_PREP_SOC_SHORTFALL_PENALTY_CZK_PER_WH
|
||||
- NEG_SELL_PV_CHARGE_REWARD_CZK_KWH
|
||||
- PRENEG_SELL_SOC_ANCHOR_SLACK_PENALTY_CZK_PER_WH
|
||||
- PRE_NEG_BUY_EMPTY_EXPORT_SHORTFALL_PENALTY_CZK_KWH
|
||||
- PRE_NEG_CHARGE_PENALTY_CZK_KWH
|
||||
- PRE_NEG_PV_BCPV_DISCOURAGE_CZK_KWH
|
||||
- PRE_NEG_PV_EXPORT_SHORTFALL_PENALTY_CZK_KWH
|
||||
Reference in New Issue
Block a user