{"claim":{"id":"d850cd4e-63b6-4fef-b0f0-3569d2845c4d","submission_id":"371b5316-1d4a-41f9-a556-a0dc01f4c2c4","submission_revision_id":"3440c449-d697-451f-80b3-67f84d4c2890","builder":"builder-01","status":"active","claimed_at":"2026-10-15T14:57:10.708736Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"371b5316-1d4a-41f9-a556-a0dc01f4c2c4","submission_revision_id":"3440c449-d697-451f-80b3-67f84d4c2890","revision_version":1,"submission_status":"under_review","submitted_at":"2026-08-12T08:15:35.019196Z","submitter_name":"Suo yiran","submitter_email":"suo1065962367@gmail.com","task":{"status":"submitted","version":1,"extra_fields":{},"submitted_at":"2026-08-12T08:15:35.019196+00:00","industry_domain":"Productionize AirLife: an offline, stateful flight ETL pipeline with deterministic publication","task_short_name":"airlife_stateful_etl_instance_1","operating_system":"Linux","task_description":"Objective:\nProductionize the unfinished Python repository in `airlife/` into an offline, stateful ETL application over the authentic nycflights13 v1.0.2 materials.\n\nStarting State:\nThe input package contains:\n1. `airlife/`, an unfinished MIT-licensed flight-data ETL repository.\n2. `reference/nycflights13/`, containing the source `.rda` objects and package documentation.\n3. `reference/source_archives/`, containing immutable source archives for provenance.\n4. A pinned Python 3.12 Linux environment.\n\nInstructions & Rules:\nPreserve everything under `reference/` byte-for-byte and place all implementation, generated state, output, tests, documentation, and evidence under `airlife/`. The workflow must run without network access.\n\nImplement stable CLI commands for ingestion, validation, and a local operational dashboard. Ingestion must support all or selected calendar months, remain idempotent when months are repeated, accept out-of-order months, count late records, maintain a monotonic event-time watermark, and converge to the same annual dataset regardless of ingestion order. Failed or schema-incompatible runs must persist diagnostics without overwriting the last successful publication.\n\nCreate `output/flights.parquet` by enriching every flight with airline, airport, aircraft, and matching weather data. Preserve cancellations and unmatched nullable values. Publish exactly 336,776 rows and the specified 34 columns using correct nullable Arrow types, UTC timestamps, deterministic ordering, valid SHA-256 flight keys, Zstandard compression, and no volatile metadata.\n\nMaintain durable run, schema-event, and quality history in `state/pipeline.sqlite`. Include executable quality rules, validation JSON, complete table- and field-level lineage, a self-contained dashboard with health and metrics APIs, automated tests, and reproducibility evidence. Repeated clean builds must produce byte-identical Parquet output.\n\nThe attached `task_instruction.txt` contains the complete authoritative column schema, commands, evidence requirements, and acceptance criteria.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Use automated checks only. The final score is the sum of the weighted sections below. A submission succeeds only if it scores at least 90/100 and passes every mandatory gate.\n\nMandatory gates:\n1. Every supplied file under `reference/` remains byte-identical.\n2. Ingestion and validation work without network access.\n3. The required ingest, validate, and dashboard commands execute successfully.\n4. Both `output/flights.parquet` and `state/pipeline.sqlite` exist and are readable.\n5. A failed or incompatible ingestion returns nonzero, records a diagnostic, and does not modify the last successfully published dataset.\n\nCanonical dataset — 30 points:\n- Exactly 336,776 rows and the exact 34 columns in the specified order.\n- Correct Arrow types and nullability, including UTC `time_hour` and date-typed `flight_date`.\n- All Parquet column chunks use Zstandard compression; no dataframe index or volatile schema metadata.\n- Every `flight_key` matches the specified SHA-256 formula and is unique.\n- Rows follow the specified canonical sort order with nulls last.\n- All flights are preserved through airline, airport, aircraft, and weather joins; missing values remain native nulls.\n- Table values must be semantically identical to the gold-standard dataset. The gold reference SHA-256 is `fe4912aa2ccbfacade5d64a6f405779aeb601201bd0cae4c558c2d6bb24e7e9b`.\n\nStateful behavior — 20 points:\n- Repeating a month does not increase the fact count.\n- Different eventually complete month orderings converge to the same canonical dataset.\n- Older records are retained and counted as late, while the watermark remains monotonic.\n- Key conflicts use update rather than append-conflict behavior.\n- Transactional and incompatible-schema failure tests preserve the prior publication.\n\nSQLite audit state — 12 points:\n- `pipeline_runs`, `schema_events`, and `quality_results` exist.\n- Run history contains all fields required by the task.\n- The latest successful run’s row count, watermark, and dataset hash match the actual Parquet file.\n- Required-field removals or incompatible type changes are persisted and block publication.\n\nQuality and validation — 12 points:\n- `quality/rules.yaml` covers every contract family listed in the task.\n- Validation JSON contains all required top-level and per-check fields, with checks sorted by ID.\n- All required checks pass and the identical results are persisted in SQLite.\n\nLineage — 8 points:\n- Source provenance contains URL, publisher, version/commit, size, SHA-256, and license basis.\n- All 34 target fields have source fields, transformation, join/derivation semantics, and null policy.\n\nDashboard and APIs — 8 points:\n- The root page is self-contained and displays the required persisted metrics.\n- `/api/health` and `/api/metrics` return the required structures and values from SQLite.\n\nReproducibility and delivery — 10 points:\n- Two clean annual builds produce byte-identical Parquet files.\n- All required tests pass after deleting generated output, state, and evidence.\n- Every required evidence file exists and matches `evidence/checksums.sha256`.\n- All required Make targets execute successfully.\n\nDo not reject an otherwise semantically identical solution solely because its Parquet binary hash differs from the gold reference; instead require that two clean builds of that candidate are byte-identical.","software_and_version":"Python 3.12 data-engineering stack — pandas 2.2.3, PyArrow 18.1.0, SQLite 3.46.1, FastAPI 0.115.6, Playwright 1.49.1","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"8a880193-1f84-435c-ab87-d74596ec5aad","filename":"evaluation_reference.json","category":"input_materials","content_type":"application/json","size_bytes":1675},{"file_id":"e021593f-f8eb-435a-a220-426602d33953","filename":"environment.Dockerfile","category":"input_materials","content_type":"application/octet-stream","size_bytes":702},{"file_id":"06b45996-5e48-4a7a-9f6f-655092d05c71","filename":"input_materials.zip","category":"input_materials","content_type":"application/zip","size_bytes":10600524},{"file_id":"626c42fb-c79c-4e4f-8cb1-07a841a36a21","filename":"reference_output_full.zip","category":"reference_output","content_type":"application/zip","size_bytes":92449057}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — The submission is complete, professionally scoped, objectively evaluable, and demonstrably achievable offline with the supplied materials.\n\n**Completeness Check:**\n- Instruction: Complete — The authoritative instruction defines commands, schemas, state behavior, failure handling, lineage, validation, dashboard, tests, and evidence requirements.\n- Inputs: Complete — The starter repository, authentic nycflights13 v1.0.2 objects and documentation, and immutable provenance archives are present and readable.\n- Toolchain: Complete — A pinned Python 3.12 container specifies all required ETL, validation, dashboard, browser, R, and SQLite dependencies.\n- Reference output: Complete — It includes the canonical Parquet dataset, readable SQLite state, implementation, tests, lineage, quality contracts, and reproducibility evidence.\n- Rubric: Complete — Mandatory gates and weighted criteria provide objective checks for canonical data, stateful behavior, audit state, validation, lineage, APIs, and reproducibility.\n\n**File and Task Reachability:**\nThe attachment set is `reachable`, and the end-to-end task is `reachable`. I inspected `evaluation_reference.json`, `environment.Dockerfile`, `input_materials.zip`, and `reference_output_full.zip`, including complete archive inventories, integrity tests, nested provenance containers, representative source records, documentation, implementation, tests, evidence, Parquet signatures and checksum, and SQLite schema and persisted metrics. Both ZIP archives are readable, unencrypted, and free of traversal entries; all declared checksum entries verify. One provenance file named as gzip is actually an uncompressed POSIX tar archive, but it remains safely readable and does not affect execution. The reviewer host lacked R and Arrow parsers, so RDA interiors and Parquet metadata were not independently decoded there; their formats, companion schemas, successful pinned-environment execution, exact gold hash, validation report, and matching SQLite facts provide consistent evidence. No attachment was otherwise left unreviewed. The completion path is clear: build the pinned environment, ingest packaged RDA objects offline, publish Parquet and SQLite state, validate contracts, serve persisted dashboard metrics, and evaluate behavior through automated tests, schemas, hashes, and failure-preservation checks. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n`public_benchmark` — nycflights13 v1.0.2 is affirmatively identified as a public data package. Risk is limited because the ETL/statefulness contract is bespoke, the solver workflow is offline, and the full reference implementation and gold artifacts are separated as reference output.\n\n**Recommended Changes:**\nThe submission already provides coherent inputs, a proven reference execution, durable audit evidence, and strong automated evaluation coverage. No changes are required.","reviewed_at":"2026-08-15T00:38:51.223462Z"},"claim":{"id":"d850cd4e-63b6-4fef-b0f0-3569d2845c4d","submission_id":"371b5316-1d4a-41f9-a556-a0dc01f4c2c4","submission_revision_id":"3440c449-d697-451f-80b3-67f84d4c2890","builder":"builder-01","status":"active","claimed_at":"2026-10-15T14:57:10.708736Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}