{"claim":{"id":"62bfc096-6d62-49bb-bff5-55e9e33b7d93","submission_id":"7bf96d12-32d0-46b9-b011-c438ddd71a94","submission_revision_id":"b513f757-a0a0-4a04-8b24-477a970d6df4","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:33:32.584803Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"7bf96d12-32d0-46b9-b011-c438ddd71a94","submission_revision_id":"b513f757-a0a0-4a04-8b24-477a970d6df4","revision_version":1,"submission_status":"under_review","submitted_at":"2026-09-29T19:57:16.514788Z","submitter_name":"HAIXIANG TANG","submitter_email":"hansont2@illinois.edu","task":{"status":"submitted","version":1,"extra_fields":{},"submitted_at":"2026-09-29T19:57:16.514788+00:00","industry_domain":"Energy and Environmental Data Engineering / Electricity Emissions Accounting","task_short_name":"egrid2023_revision_reconciliation_instance_1","operating_system":"Windows","task_description":"Audit two actual EPA eGRID2023 releases for an electricity-emissions reporting team. Reconstruct regional emissions factors from source observations, explain the arithmetic effect of revisions, and produce a source-preserving exception ledger.\n\nThe input ZIP contains six frozen CSV extracts: each release has 12,612 plant records (44 fields), 26,601 generator records (16 fields), and 26,186 unit records (32 fields). It also includes the official EPA Technical Guide, source dictionary, file/column/hash inventory, complete v1.0.0 output contract, task prompt, provenance and pinned dependencies. Source records are authentic published EPA data. Published rate columns, aggregate sheets and reference answers are evaluator-only.\n\nPreserve all source text and full entity keys in DuckDB. Reconcile the two visible field-name aliases; keep literal NA region codes distinct from blank cells. Calculate seven pollutant/window metrics across states, balancing authorities, subregions, NERC regions and the entire supplied national universe, including Puerto Rico. Sum mass and generation separately before division, use correct pollutant units and annual/ozone denominators, preserve signed negative generation, and distinguish missing observations from zero sums. A zero denominator produces a null audit rate under the visible rule.\n\nCreate the complete changed-plant-cell ledger, symmetric regional mass/generation rate-change decomposition, and all eight specified reconciliation/observation checks. Do not silently remove negative generation, collapse legitimate multi-prime-mover units, infer that every discrepancy is a source error, or average plant rates.\n\nDeliver six artifacts: analysis.duckdb with seven exact-schema tables; regional_results.csv, revision_changes.csv and exceptions.csv as sorted database mirrors; report.pdf with Scope, Methods, Revision impacts, Reconciliation exceptions and Limitations plus nine labeled numeric summaries; and manifest.json with output hashes and table counts. Every field, identifier, formula, null rule, exception predicate, report label and grading rule is solver-visible in output_contract.json.\n\nWork offline on one machine. The supplied reference and scorer were executed on Windows with the stated Python stack. A 4-vCPU/16-GB machine and two-hour solver budget are design targets, not claimed independent-model measurements. This is an independently prepared audit of an authentic completed EPA data-production workflow, with EPA attribution and no endorsement claim.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Use the included output-only evaluator: python evaluate.py --output OUTPUT_DIR --gold gold --contract output_contract.json --result grade.json. The evaluator ZIP contains all six complete reference deliverables, gold tables, the typed reference DuckDB, unchanged official workbooks for provenance, reference-building scripts, explicit external rounding checks and executed mutation-test evidence. Keep it separate from solver inputs.\n\nFor each table, key_F1=2*matched_keys/(actual_keys+gold_keys), with both empty=1. Non-key cell accuracy counts correct cells on matched rows divided by (gold_rows+extra_actual_rows)*number_of_nonkey_fields. A table score is 0.3*key_F1+0.7*cell_accuracy. Floating values pass if abs(actual-gold)<=max(1e-6,1e-9*abs(gold)); keys, text, counts, categories and nulls are exact. Gold is recomputed from the frozen source CSVs; published rounded rates are not substituted as the answer.\n\nOverall score: 25% mean of the three source-table scores; 35% regional-results score; 25% mean of revision_changes, regional_impacts and exceptions; 15% artifact-integrity score. Artifact integrity is 0.5 for mandatory structural/readability checks +0.2*required-section coverage +0.3*required-summary-label coverage. Present PDF summary values must agree with the database. Pass requires overall score>=0.80 and no hard failure.\n\nHard failure means score=0, passed=false: any missing/unreadable required file; missing/empty required table; missing column/wrong SQL type; duplicate key or JSON object key; invalid null/nonfinite value; incorrect CSV header/sort order; CSV/database disagreement; manifest task/version/hash/count mismatch; or a present PDF summary contradicting the database. Every PDF page must render, not merely have a PDF extension. Missing report sections/labels lose the defined content credit. An evaluator gold/setup failure is an infrastructure error, never a solver penalty.\n\nThe supplied positive fixture scores 1.0; isolated negative and partial-credit fixtures are documented with the scorer/contract/reference hashes. No independent-model difficulty result or human-expert sign-off is claimed.","software_and_version":"CPython 3.12.14; DuckDB 1.4.3 (engine and Python binding); openpyxl 3.1.5; ReportLab 4.4.9; pypdf 6.10.0; pypdfium2 5.13.0; Pillow 12.3.0. Pinned FOSS stack, offline after installation.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"ae63b832-d76a-41b9-91c9-932d06592f62","filename":"egrid2023_input_materials.zip","category":"input_materials","content_type":"application/x-zip-compressed","size_bytes":6155257},{"file_id":"c75bbdb9-51d6-47d5-9496-7c3c9071cdb0","filename":"egrid2023_reference_evaluation.zip","category":"reference_output","content_type":"application/x-zip-compressed","size_bytes":52876947}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — This is a complete, professionally grounded, objectively gradable audit task with no material submission defect identified.\n\n**Completeness Check:**\n\n- Instruction: Complete — The scope, six deliverables, seven table schemas, formulas, identifiers, null rules, and eight exception predicates are explicit.\n- Inputs: Complete — All six UTF-8 source CSVs match their declared dimensions, hashes, and full keys. Their selected observations agree with the supplied workbook caches.\n- Toolchain: Complete — The pinned FOSS stack supports a credible offline, single-machine workflow. Windows execution is documented; resource and solver-duration targets are appropriately distinguished from measured results.\n- Reference output: Complete — All six reference deliverables are present. Independent recalculation matches 2,226 regional rows, 1,134 changed-cell rows, 1,113 impact rows, and 1,197 exceptions within the stated tolerance.\n- Rubric: Complete — Weights, partial-credit formulas, numeric tolerances, hard failures, and reporting checks are solver-visible and consistent with the evaluator. Hash-linked evidence records 37 passing tests, including the positive reference scoring 1.0.\n\n**File and Task Reachability:**\n\nAttachment classification: `partially_reachable`, solely because independent DuckDB content inspection was unavailable. Task classification: `reachable`.\n\nBoth `egrid2023_input_materials.zip` and `egrid2023_reference_evaluation.zip` were downloaded and inspected. Their byte counts match the attachment manifest; archive integrity and packaged member hashes pass, with no unsafe paths or encryption barriers found. Inspection covered the six source CSVs, dictionaries, provenance, inventories, dependency locks, output contracts, reference and gold CSVs, manifests, both provenance workbooks, evaluator and rebuilding code as text, and validation evidence.\n\nThe Technical Guide received metadata, representative-text, and sample-render checks rather than page-by-page review. Every reference report page rendered readably, and its required summaries agree with the independently checked tables. The DuckDB files have recognizable format signatures and matching packaged hashes, but their internal schemas and CSV/database parity remain supported by submitted validation evidence rather than independent database inspection. Uploaded programs were not executed, and the evaluator was not rerun.\n\nThe completion path is coherent: install compatible dependencies before going offline, process the frozen extracts under the visible contract, generate the six artifacts, and evaluate them against the separate gold package. No solver-visible reference-answer leakage was found.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. The supplied workbook evidence supports authentic observations and revisions. Deliberate mutations are evaluator tests, not the task’s core defects. Public EPA provenance alone does not establish a public benchmark; external publication of this custom task or reference was not verified.\n\n**Recommended Changes:**\n\nThe source-preserving design, explicit arithmetic rules, reference separation, and documented validation are substantive strengths. No contributor changes are required.","reviewed_at":"2026-10-14T19:51:49.485743Z"},"claim":{"id":"62bfc096-6d62-49bb-bff5-55e9e33b7d93","submission_id":"7bf96d12-32d0-46b9-b011-c438ddd71a94","submission_revision_id":"b513f757-a0a0-4a04-8b24-477a970d6df4","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:33:32.584803Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}