engineering/residual_load_dayahead_uq - reference output and evaluation dependencies (HIDDEN: do not give to agents)

expert_output/output/        The expert result: what an agent must leave in the task's output/ directory.
  system/                    SYSTEM.json, formulation.json, formulation.md, the Python package `forecaster`
                             (prepare, target), models/cnn.pt2, rnn.pt2, denoiser.pt2 (torch.export)
  backtest/                  opsim run 2025-06-01..2025-08-31 with this system (run.json, days/*.npz) + metrics.json
expert_output/evaluator_report.json
                             The evaluator's report on exactly this output with the full protocol (365 hidden days).
reference/                   Hidden reference data, staged only after the agent has finished (task_dir/reference):
  raw_hidden/                raw Elia/GFS files of local days 2025-07-01..2026-08-31 (packages and target check)
  truth/                     official residual-load series of the development and the hidden period
  strata_hidden.csv          wind / PV day-ahead-forecast terciles of the scored quarter-hours
  formulation_answers.json   answers of the formulation.json declaration check
  protocol.json              periods, probe days, seeds, weights, completion share, thresholds, limits, calibration
evaluation/                  The ALE task code (also submitted as a GitHub patch):
  tasks/engineering/residual_load_dayahead_uq/   task_card.json, main.py (load/start/evaluate), scripts/grader/
  env/packages-linux/torch-cpu-forecasting/      image installer (CPU torch 2.11 + numpy/pandas/scipy, python3.10)
reference_solution/oracle/   The code that produced the expert output: run_reference.sh runs the whole agent workflow
                             on the task VM (about 40 min on 8 vCPU): target, 5-fold out-of-sample CNN/RNN, 3-member
                             ensembles, diffusion model, torch.export, selfcheck, backtest, metrics.

Scoring: S1, S2, S3 = weighted fraction of each stage's passed checks;
score = 0.3 x (0.2 S1 + 0.35 S2 + 0.45 S3) + 0.7 x [every check of the three stages passed]; 0 if a gate fails.

Running the evaluator by hand (as root on the task VM; <base> holds input/ and software/ from inputs.zip, reference/
from this archive, and output/ = expert_output/output):
  cp -r evaluation/tasks/engineering/residual_load_dayahead_uq/scripts/grader /var/tmp/grader
  python3 -I -S /var/tmp/grader/run_eval.py --task-dir <base> --output-dir <base>/output \
      --reference-dir <base>/reference --work /var/tmp/rl_work --report /var/tmp/rl_report.json
main.py's evaluate() does exactly this through the cua-bench session (about 30 min: 365 prepare runs as an
unprivileged user, inference of the three exported networks with the evaluator's own DDPM sampler, target check).

Data: Elia Open Data (ods001, ods031, ods032), CC BY 4.0; GFS forecasts from the Open-Meteo previous-runs archive,
CC BY 4.0 (GFS public domain). Values as published; only columns selected and rows cut to periods.
