{"claim":{"id":"cda0fdd5-7a98-45f0-b28f-d28b86de2e6d","submission_id":"b3d5b4ca-366e-48bf-b330-9b5e90b81a7e","submission_revision_id":"cfe22ca3-d3e0-4497-8cc3-7949a0760a40","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:13:07.102812Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"b3d5b4ca-366e-48bf-b330-9b5e90b81a7e","submission_revision_id":"cfe22ca3-d3e0-4497-8cc3-7949a0760a40","revision_version":1,"submission_status":"under_review","submitted_at":"2026-08-27T20:07:20.945059Z","submitter_name":"Thorben Jansen","submitter_email":"tjansen@ipn.uni-kiel.de","task":{"status":"submitted","version":1,"extra_fields":{"difficulty_self_test":{"model":"GPT-5.5","harness":"Codex","estimated_score":"40–60%"}},"submitted_at":"2026-08-27T20:07:20.945059+00:00","industry_domain":"Educational Assessment, Psychometrics, and Learning Analytics","task_short_name":"pisa_variable_phase_transition_nearterm_20_v1","operating_system":"Linux","task_description":"### Objective\n\nWriting consists of planning, composing, and revising. In these authentic PISA 2025 English-writing sessions, approximately 15-year-old students explicitly triggered markers when they moved from planning into composing (Phase 3) and, if reached, from composing into revising (Phase 4). Reconstructing these transitions makes it possible to study where support is needed and to design phase-specific interventions—for example, to strengthen planning, composing, or revision.\n\nReconstruct the exact locations of these authentic writing-phase transitions from the raw PISA process streams after the explicit transition-marker rows have been removed. A session may contain one or both transitions. For each of 20 evaluation sessions, return the exact retained-row boundary for Phase 3 and Phase 4, or `null` when that phase was not reached.\n\n### Starting State\n\nThe input package contains 20 anonymized raw process streams from authentic PISA 2025 English-writing sessions completed by approximately 15-year-old students, an evaluation manifest, raw-format documentation, a submission template, and ten person-disjoint worked examples. The examples include both sessions with two transitions and sessions with only the Phase 3 transition.\n\nThe relevant process payloads remain in their original base64-encoded gzip JSON representation. The original transition rows and their event-name dictionary entries are removed, while all other rows preserve their semantic event, arguments, order, and decoded relative timing.\n\n### Instructions and Rules\n\n1. Decode and analyze each raw process stream.\n2. Infer whether Phase 3 and Phase 4 occurred from the process behavior and worked examples.\n3. For an observed transition, output the exact number of retained raw rows preceding the removed original marker.\n4. Output `null` only when the original session never reached that phase.\n5. Return exactly one JSONL row per evaluation case using the supplied case IDs and fields.\n6. If both boundaries are integers, Phase 3 must precede Phase 4.\n7. Do not reconstruct marker positions by generating new ground truth. Evaluator gold comes exclusively from the location or verified absence of the original PISA marker rows.\n\nExample output row:\n\n```json\n{\"case_id\":\"PHASE_01\",\"phase3_after_retained_event_count\":1234,\"phase4_after_retained_event_count\":null}\n```\n\nThere are 40 independently scored phase decisions. A decision earns one point only for the exact original retained-row boundary or the correct `null`. There is no partial credit.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Run the evaluator command:\n\n```bash\npython3 score_submission.py /path/to/submission.jsonl --output score.json\n```\n\nThe submission must contain exactly 20 rows matching the evaluation manifest, with both required phase fields present in every row. Each field must contain either an integer within the case's retained-row range or JSON `null`; when both are integers, Phase 3 must precede Phase 4.\n\nFor every case and phase, compare the submitted value to evaluator-only gold copied from the original process record. Award one point only when the value is identical: the exact retained-row boundary for a present original marker or `null` for a verified absent marker. The primary score is `exact_decisions / 40`. Time-distance and row-distance values are diagnostics only and never earn partial credit.\n\nBefore scoring, run `python3 validate_package.py`. The validator must report `PASS`, 20 evaluation cases, 10 training cases, 18 two-transition and two single-transition evaluation cases, zero participant overlap, no direct marker leakage, and no source-ID leakage. Scoring controls must produce perfect = 40/40, empty = 0/40, uniform thirds = 0/40, and all-null = 2/40.","software_and_version":"Python 3.11 using standard-library JSON, base64, and gzip modules","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"ebe23821-dfa1-4145-b194-692868d69018","filename":"pisa_variable_phase_transition_nearterm_20_v1_participant.zip","category":"input_materials","content_type":"application/zip","size_bytes":1575591},{"file_id":"264ec160-0c40-43f6-aeb5-163370a90d51","filename":"pisa_variable_phase_transition_nearterm_20_v1_reference_and_provenance.zip","category":"reference_output","content_type":"application/zip","size_bytes":1634308},{"file_id":"371b24d4-ffd6-4af1-936c-8f5f99aa35dc","filename":"pisa_variable_phase_transition_nearterm_20_v1_selftest_evidence.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":97372}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — The submission is complete, internally consistent, professionally constructed, and end-to-end reachable with objective exact scoring and effective evaluator-data separation.\n\n**Completeness Check:**\n- Instruction: Complete — Clearly defines the transition semantics, retained-row boundaries, `null` behavior, JSONL schema, ordering rule, and exact scoring.\n- Inputs: Complete — Provides 20 valid evaluation streams, ten person-disjoint worked examples, manifests, format documentation, and a submission template.\n- Toolchain: Complete — Requires only Python 3.11 standard-library functionality on Linux and is feasible on an ordinary single machine.\n- Reference output: Complete — Contains all 20 gold rows, original-marker provenance fields, selection records, construction logic, validation evidence, and scorer diagnostics.\n- Rubric: Complete — Defines 40 independently scored exact decisions, rejects partial credit, validates value ranges and ordering, and includes coherent perfect, empty, uniform-thirds, and all-null controls.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. Task classification: `reachable`.\n\nAll three files were inspected: `pisa_variable_phase_transition_nearterm_20_v1_participant.zip`, `pisa_variable_phase_transition_nearterm_20_v1_reference_and_provenance.zip`, and `pisa_variable_phase_transition_nearterm_20_v1_selftest_evidence.zip`. Their byte counts match the manifest, all ZIP members pass integrity checks, and no encryption, traversal paths, links, device entries, or excessive expansion were found.\n\nThe participant archive contains 47 regular files. All JSON and JSONL parsed successfully, all 30 raw-file hashes matched their manifests, and every base64-gzip message was decoded and schema-checked. Retained-row counts match for all streams; removed marker names and source identifiers were absent even after decoding. The reference archive contains 72 readable files, including gold, provenance, scorer source, controls, rankings, and a byte-identical participant copy. The self-test archive contains three structurally valid 20-row submissions and one valid PNG; the reported 16/40, 22/40, and 20/40 results were independently reproduced from the supplied submissions and gold.\n\nThe completion path is clear: decode the streams, infer boundaries using the worked examples, produce the required 20-row JSONL, and evaluate it against the sealed gold. Uploaded programs were inspected as source rather than executed; published validation and scoring results were independently checked from the underlying data. No relevant attachment content remained unreviewed.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. No affirmative evidence supports an AI-synthetic, public-benchmark, or synthetic-defect-fixture label for these protected session streams and task-specific gold.\n\n**Recommended Changes:**\nThe package has strong input/reference separation, reproducible boundary derivation, consistent manifests, and credible blind difficulty evidence. No changes are required.","reviewed_at":"2026-08-27T20:19:40.759101Z"},"claim":{"id":"cda0fdd5-7a98-45f0-b28f-d28b86de2e6d","submission_id":"b3d5b4ca-366e-48bf-b330-9b5e90b81a7e","submission_revision_id":"cfe22ca3-d3e0-4497-8cc3-7949a0760a40","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:13:07.102812Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}