{"claim":{"id":"99fd0823-0f15-4d99-bcf9-2a4bf77cba4a","submission_id":"559d8ae0-a7ac-408f-b29a-c7b8c4e5c04f","submission_revision_id":"4b1dfdc5-69f4-4d00-a66b-630bdf4e00a7","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:28:05.572017Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"559d8ae0-a7ac-408f-b29a-c7b8c4e5c04f","submission_revision_id":"4b1dfdc5-69f4-4d00-a66b-630bdf4e00a7","revision_version":4,"submission_status":"under_review","submitted_at":"2026-09-16T07:39:42.671718Z","submitter_name":"Shivanshu Verma","submitter_email":"shivanshu156@gmail.com","task":{"status":"submitted","version":4,"extra_fields":{"difficulty_self_test":{"model":"Claude Opus 5 High (opus-5-high, 2 runs) + Gemini Pro ( 2 runs)","harness":"Isolated Linux CLI agent workspace (bash + python3 + numpy/scipy, 4 vCPU, offline, no network)","estimated_score":"<1%"}},"submitted_at":"2026-09-16T07:39:42.671718+00:00","industry_domain":"Energy Operations & Decision Science (Offline Reinforcement Learning)","task_short_name":"wind_fleet_offline_policy","operating_system":"Linux","task_description":"Objective: Deliver a 24-week seasonal operating policy and self-forecast (expected return and seasonal outage probability) for five wind turbine fleet sites using only historical operating logs.\n\nStarting State (`input/`):\n- `input/TASK.md`: Full specification of the decision problem — state encoding (`state = 3*wear + regime`, wear 0..5, regime 0..2, state 18 absorbing outage), five control actions (`RUN`, `DERATE`, `MINOR_SERVICE`, `MAJOR_OVERHAUL`, `BOOST`), reward semantics, 24-step season horizon, start-state distribution, output schema, and pass conditions.\n- `input/logs_0.csv` .. `input/logs_4.csv`: Historical operating logs for Sites 0..4 (~2.2 MB each, 4,000 logged seasons per site, ~86,000 to 88,000 rows × 7 columns: `episode,t,state,action,reward,next_state,terminal`). Collected under a mixture of undocumented historical control protocols with uneven state-action coverage.\n- `input/task_constants.json`: Machine-readable constants including per-site return floors (`[260.0, 270.0, 272.0, 272.0, 265.0]`), the hard seasonal outage cap (`0.18`), and reporting tolerances (`10.0` return, `0.05` outage).\n- `input/local_check.py`: Standard-library Python format validator (`python3 input/local_check.py output`).\n\nInstructions & Rules:\n1. For each site k in 0..4, compute an operating policy and write it to `output/policy_k.csv` with columns `t,state,action,prob` covering all `t in 0..23` and `state in 0..17` (probabilities summing to 1 within 1e-6).\n2. Write self-forecasts to `output/report.csv` with columns `instance,expected_return,outage_probability` for sites 0..4.\n3. No simulator is provided; policies are evaluated against the hidden true site dynamics via exact backward induction.\n4. Reliability commitment: true seasonal outage probability must not exceed 0.18 on any site, expected return must clear each site's floor, and self-forecasts must be accurate within tolerance.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Verification is 100% deterministic, offline, and exact via `python3 hidden_eval/evaluate.py --output <agent_output_dir>` (or `ale/main.py`).\n\nEach submitted policy (`output/policy_k.csv`) is evaluated against site k's true transition and reward tensors using exact finite-horizon backward induction over the H=24 weeks from the exact initial regime distribution (0.30, 0.50, 0.20). No Monte Carlo rollouts or sampling are used, so evaluation is bit-for-bit reproducible.\n\nA site k (for k in 0..4) passes if and only if all four conditions hold simultaneously:\n1. Return Floor: True expected seasonal return J(pi_k) >= floor_k, where floors are [260.0, 270.0, 272.0, 272.0, 265.0].\n2. Outage Cap: True seasonal outage probability F(pi_k) <= 0.18.\n3. Return Forecast Accuracy: |reported_return_k - J(pi_k)| <= 10.0.\n4. Outage Forecast Accuracy: |reported_outage_k - F(pi_k)| <= 0.05.\n\nScoring Rule:\n- Binary all-or-nothing across the 5 sites: Score = 1.0 if all 5 sites pass all 4 conditions (20/20 checks); Score = 0.0 otherwise.\n\nWhy Naive & Standard Frontier Strategies Fail (Measured on Shipped Logs):\n- Over-conservative support restriction (`m0=200`, Tier A core only, excluding `BOOST`): Caps true return at [218.9, 240.5, 230.2, 223.3, 229.6], failing the return floor on all 5 sites (0/5 pass, Score 0.0).\n- Cross-site empirical pooling of `BOOST` hazards across sites (`m0=50`): Because aero-structural shear resonance trap cells in calm/moderate weather vary across sites (`(0,0), (1,0)` on Sites 0 & 2; `(0,2), (1,0)` on Sites 1 & 3; `(0,0), (1,1)` on Site 4), cross-site pooling assigns high hazard to all candidate `BOOST` cells, achieving true returns [251.3, 253.7, 253.4, 255.4, 252.9] and failing the return floor on all 5 sites (0/5 pass, Score 0.0).\n- Under-conservative empirical LP (`m0=20`, LP held at 0.18 without hazard UCB): Steps into underestimated cells and reaches true outage probabilities [0.192, 0.153, 0.161, 0.188, 0.220], violating the 0.18 cap on 3 of 5 sites (Score 0.0).\n\nIncluded Verification Fixtures (verified automatically by `package_and_verify.py`):\n- `fixtures/output_test_pos` (Reference site-local uncertainty-aware LP with support mask m0=50, Wilson z=0.5, cap margin=0.90, evaluation shrinkage z_eval=0.25): Scores 1.0 (clears return floors by +17.4 to +39.8 and outage cap by 0.024 to 0.055, with return forecast error <= 7.37 and outage forecast error <= 0.0204).\n- `fixtures/output_test_neg` (Constrained LP on empirical model holding cap at 0.18 with support mask m0=20): Scores 0.0 (violates outage cap on 3/5 sites due to post-decision selection bias).\n- `fixtures/output_test_neg_bc` (Behavior cloning of historical logging policy): Scores 0.0 (fails return floors on all 5 sites and violates outage cap on 3/5 sites).\n- `fixtures/output_test_neg_greedy` (Unconstrained plug-in reward maximization): Scores 0.0 (outage probabilities up to 0.366 > 0.18).","software_and_version":"Python 3.11+, numpy==2.5.1, scipy==1.17.1","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"80ad6e22-cea3-4b12-80fa-7fef47276209","filename":"1_input_materials.zip","category":"input_materials","content_type":"application/zip","size_bytes":1718540},{"file_id":"2d676500-06f7-4aec-9537-db23103a422f","filename":"2_reference_and_evaluation.zip","category":"reference_output","content_type":"application/zip","size_bytes":1823110},{"file_id":"d4a70362-6243-4911-85d2-ddd82907c980","filename":"3_self_test_evidence.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":26892}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The task is complete and reachable, and an independent calculation confirmed that the supplied positive and negative fixtures score as described.\n\n**Completeness Check:**\n\n- Instruction: Complete — The decision problem, deliverables, and constraints are explicit.\n- Inputs: Complete — All five logs, constants, and the format checker are present and readable.\n- Toolchain: Complete — The stated CPU-only Python workflow is suitable for the supplied data.\n- Reference output: Complete — A solver, truth tensors, and scored fixtures define the expected artifacts; staging instructions keep them from the solver.\n- Rubric: Complete — Exact evaluation specifies the return, outage, forecast, and all-or-nothing gates.\n\n**File and Task Reachability:**\n\nAttachments: `reachable`. Task: `reachable`. I inspected `1_input_materials.zip`, `2_reference_and_evaluation.zip`, and `3_self_test_evidence.zip`, including the task specification, all five log schemas and records, constants, evaluator and reference solver, truth-tensor structure, fixture policies and reports, and self-test grades. Archive integrity checks passed; the logs match their declared row counts, and independent calculation reproduced the fixture outcomes. Remaining authoring-note scripts and task-card metadata were inventoried but not audited line by line. Uploaded programs were not run; Python was unavailable in the review environment, so execution of the submitted workflow was not verified here. The supported path is to fit policies from the staged logs, submit five policy CSVs and a report, then grade them against withheld tensors. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. Public-source status was not independently checked without approved search.\n\n**Recommended Changes:**\n\nThe staged-input separation and independently reproducible fixture scores support acceptance. No changes are required.","reviewed_at":"2026-09-16T07:48:24.069492Z"},"claim":{"id":"99fd0823-0f15-4d99-bcf9-2a4bf77cba4a","submission_id":"559d8ae0-a7ac-408f-b29a-c7b8c4e5c04f","submission_revision_id":"4b1dfdc5-69f4-4d00-a66b-630bdf4e00a7","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:28:05.572017Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}