{"submission_id":"f904a6b0-3c62-4b1a-83e4-2ddf209dc7ab","submission_revision_id":"63a4236d-0e3c-41a2-8285-c7acea023a45","revision_version":1,"submission_status":"under_review","submitted_at":"2026-09-29T21:28:31.404043Z","submitter_name":"HAIXIANG TANG","submitter_email":"hansont2@illinois.edu","task":{"status":"submitted","version":1,"extra_fields":{},"submitted_at":"2026-09-29T21:28:31.404043+00:00","industry_domain":"Aviation Operations; Transportation Analytics; Operational Reporting and Data Quality","task_short_name":"bts_airline_operations_reconstruction_instance_1","operating_system":"Linux","task_description":"Reconstruct an auditable May 2024 airline-operations handoff from genuine US Bureau of Transportation Statistics records. Separate United's marketing network from United and SkyWest as reporting operators, reconcile public monthly report cells, and identify long ground intervals without counting the same exact-key operation twice across source products. The frozen corpus contains 113,997 UA marketing rows and 126,263 UA/OO reporting rows, drawn without changing measured values from two official PREZIP archives.\n\nRead candidate/inputs/protocol.md and output_contract.json; these declare every identifier, field, null rule, formula, comparison tolerance and grading invariant. Source manifests, official field dictionaries and a transparent factual ATCR table transcription are supplied. The original full monthly ZIPs are preserved evaluator-side with their hashes. No data or outcomes are synthetic.\n\nProduce five outputs: flight_audit.parquet, carrier_metrics.csv, tarmac_events.csv, report_reconciliation.csv and results.json. Retain all source rows in the audit. Link the two populations using the declared operating-carrier/date/flight/airport/scheduled-time composite key; preserve unmatched records, original identifiers and cancellation/diversion flags. Do not conflate network and operator denominators or force-match superficially similar flights.\n\nApply cancellation precedence, preserve inapplicable delays as null, recognize air-return cancellations, and distinguish ordinary ArrDelay from DivArrDelay for diversions that reach their intended destination. The ATCR on-time numerator excludes cancellations/diversions and requires arrival less than 15 minutes late; its denominator is all scheduled operations in the relevant scope. Check departure-clock congruence without subtracting different airports' local clocks. Report full-precision rates and a separate observable-arrival-delay mean.\n\nFor the ground-interval ledger, use the exact-key union with marketing preferred where both sources represent a key. Emit each TaxiOut, TaxiIn, LongestAddGTime or Div1–5LongestGTime strictly exceeding 180 minutes as a separate event. Do not sum disconnected intervals or discard cancelled flights with ground delays. Reconcile 12 visible ATCR counts/rounded percentages against computed metrics and preserve a release-vintage uncertainty note. Return consistent counts and scope limitations in results.json.\n\nThese are operational facts, not proof of unlawful delay. The exact-key union is a contract-defined reconciliation, not independently certified distinct aircraft movements. The selected carrier scopes are not the entire domestic flight system, and ATCR Table 8 includes a broader Part 234/244 population. Public ATCR cells are comparators, not substitutes for reconstructing row-level outputs.\n\nThis newly authored, AI-assisted task adapts authentic professional airline reporting. It does not claim contributor participation in the original reporting program, government endorsement, independent human review, or a measured expert/agent solve time. Target: offline Linux, 4 vCPU, 16 GB, two hours; actual local timings and platform are separately recorded. No credentials, proprietary data or paid service is needed. Provenance distinguishes the official catalog's government-works rights evidence from the separate NTL supporting dataset's CC0 label; the latter is not asserted for the PREZIP files.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Extract both ZIPs together, verify frozen hashes, and run python candidate/evaluator/score.py --output path/to/agent_outputs --gold candidate/evaluator/reference_outputs --report grade.json. Do not execute solver code. Score weights: 25% flight audit, 30% carrier metrics, 30% deduplicated interval events, 15% report reconciliation/summary. Tables are scored by union of submitted and reference keys: 20% key intersection/union plus 80% correct non-key cells over all union-key cells. Missing/extra rows count as wrong; row order is ignored.\n\nThe audit component is half its whole-table score and half the equal average for six gold-defined subsets: cancelled, diverted, air-return-cancelled, gate-return, over-180, and unpaired. The final component is half the comparison-table score and half exact recursively typed results.json leaves. All rules are visible in output_contract.json. Integers, identifiers, categories, booleans and nulls match exactly; finite floating cells allow absolute error <= 1e-6 with no relative tolerance.\n\nPass requires total >= 0.80, rare-audit >= 0.80, interval-ledger >= 0.80, and no structural hard failure or semantic integrity error. Missing/unreadable files, invalid schemas/types, duplicate keys, nonfinite numbers, forbidden NUL CSV data or disallowed nulls are hard failures. Metrics must agree with the audit; peers must be reciprocal; events must use canonical records and agree with qualifying flights; comparisons and summary counts must agree with their underlying tables. Semantic errors retain partial scores but prevent a pass.\n\nRebuild the reference with reference.py and rerun the independent verification and isolated mutation suite as documented in BTS_EVALUATOR_README.md. The independent implementation computes the entire contract from inputs before opening reference values. Actual executed test counts, times, code hashes and platform are in candidate/evaluator/VALIDATION.json; do not infer a human expert or autonomous model trial from these deterministic tests. Optional difficulty self-test fields are intentionally blank because no such trial is claimed.","software_and_version":"CPython 3.12.14; PyArrow 19.0.1; pandas 3.0.1; NumPy 2.3.5. Independent checks use CSV/Decimal and SQLite. Linux target; actual local validation on Windows. Offline after dependency provision.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"73e40194-26f0-4d3a-a5f6-9658e833ece2","filename":"solver_inputs.zip","category":"input_materials","content_type":"application/x-zip-compressed","size_bytes":15347228},{"file_id":"abe928c1-418d-4952-afe0-20deda61e7c7","filename":"evaluator_bundle.zip","category":"reference_output","content_type":"application/x-zip-compressed","size_bytes":68555828}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — This is a complete, professionally grounded reconstruction task with coherent contracts, usable evidence, and objective evaluation.\n\n**Completeness Check:**\n\n- Instruction: Complete — Explicit identity, scope, cancellation, diversion, null, clock, interval, and reconciliation rules define all five deliverables.\n- Inputs: Complete — Both frozen subsets were verified against the preserved monthly archives; all 240,260 selected records retain their original cell values and source ordinals.\n- Toolchain: Complete — Standard open-source dependencies support an offline, single-machine workflow. Submitted Windows timings support feasibility; Linux execution is not claimed.\n- Reference output: Complete — All five artifacts are present. Independently recalculated carrier metrics, 22 interval events, 12 comparisons, and summary counts match the references.\n- Rubric: Complete — Visible scoring weights, tolerances, rare-record coverage, structural failures, and cross-artifact checks align with the inspected scorer.\n\n**File and Task Reachability:**\n\nAttachments: reachable. Task: reachable.\n\nInspected `solver_inputs.zip` and `evaluator_bundle.zip`, including both embedded PREZIP archives, `marketing_flights.csv`, `reporting_flights.csv`, `protocol.md`, `output_contract.json`, manifests, field dictionaries, report transcriptions, all five reference outputs, evaluator source, and validation evidence. Archive integrity, declared sizes, and frozen member hashes passed. The solver archive contains no protected reference outputs or implementation.\n\nParquet inspection covered its 240,260-row schema, metadata, and representative decoded pages, not every cell. Uploaded programs were not executed; submitted validation reports were inspected rather than rerun. The original ATCR PDF is absent, as disclosed, and external provenance was not independently rechecked.\n\nThe completion path is to provision dependencies, reconstruct the five artifacts from solver-visible inputs, and evaluate them against the protected reference using the documented scorer.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. Evidence supports unchanged source observations and newly authored specifications, not materially AI-fabricated inputs, a canonical public benchmark, or deliberately injected core defects. Public comparator cells are explicitly distinguished from protected reconstructed answers.\n\n**Recommended Changes:**\n\nThe submission handles population differences, unmatched records, release uncertainty, and legal limitations carefully. No changes are required.","reviewed_at":"2026-10-14T21:12:51.684998Z"},"claim":null}