{"submission_id":"0a835096-e468-4ecb-87f4-ef2f5ee0abd1","submission_revision_id":"00f42b56-d803-4189-ba48-6b23b6caaedd","revision_version":2,"submission_status":"under_review","submitted_at":"2026-10-05T21:29:43.284305Z","submitter_name":"Yufan Zhang","submitter_email":"yufanz@umd.edu","task":{"status":"submitted","version":2,"extra_fields":{"difficulty_self_test":{"estimated_score":"10–20%"}},"submitted_at":"2026-10-05T21:29:43.284305+00:00","industry_domain":"Transportation Engineering / Traffic Operations","task_short_name":"sumo_urban_incident_recovery_1","operating_system":"Linux","task_description":"Objective: Develop a traffic-management response for an urban lane-closure incident using Eclipse SUMO. Minimize network-wide delay and queueing by jointly optimizing vehicle rerouting proportions and coordinated traffic-signal timings.\nStarting State: You are given a deterministic SUMO simulation of a 3×3 signalized urban road network with 11,340 vehicles. During the visible incident, one of the two eastbound lanes between intersections J11 and J21 is closed from 08:00 to 08:30. A valid no-control starter solution, network, demand, detector, incident, and configuration files are provided.\nInstructions & Rules: Modify only public/solution.json. Choose the fractions of eligible west-to-east vehicles assigned to the direct, northern, and southern routes; the fractions must sum to 1. Configure cycle length, east–west green time, north–south green time, and offset for the five permitted traffic signals while satisfying all stated timing and safety constraints. Do not modify the network, demand, incident, detectors, evaluator, vehicle population, or simulation duration.\nEvaluate the solution using the provided visible scenario. Final evaluation also includes unseen demand and incident-timing perturbations, so the solution should be robust rather than overfitted to the visible case. Performance is based on mean time loss, queue lengths, throughput, and teleports.\nDeliverable: Submit the completed public/solution.json file.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"# How should we verify success?\n\nThe agent output must be evaluated by performance, not by exact file matching.\n\n## 1. Validate the submission\n\nThe submitted `public/solution.json` must parse successfully and match the supplied schema\nexactly. Missing or additional keys are invalid.\n\n- Route fractions must be finite values in `[0, 1]` and sum to `1.0` within an absolute\n  tolerance of `1e-6`.\n- All five required signal plans must satisfy the published cycle, green-time, offset, and\n  phase-clearance constraints.\n- The agent may modify only `public/solution.json`.\n- Verify the SHA-256 hashes of all immutable network, demand, detector, incident, and\n  configuration files before simulation.\n\n## 2. Run the deterministic evaluation\n\nCompile the submitted solution and run it using Eclipse SUMO 1.26.0 with the fixed seed and\ndeterministic vehicle assignment. Evaluate the visible scenario with weight `0.40` and three\nprivate scenarios with weight `0.20` each.\n\nFor each scenario, calculate:\n\n```text\nscenario_cost =\n    mean_time_loss\n  + 0.50 * queue_p95_m\n  + 0.25 * max_queue_m\n  + 1000 * (1 - throughput)\n  + 500 * teleports\n```\n\nThe final weighted cost is:\n\n```text\nweighted_cost =\n    0.40 * visible_cost\n  + 0.20 * hidden_cost_1\n  + 0.20 * hidden_cost_2\n  + 0.20 * hidden_cost_3\n```\n\nLower weighted cost is better.\n\n## 3. Calculate the continuous score\n\nConvert weighted cost to a continuous score using the frozen no-control and expert-reference\nanchors:\n\n```text\nscore = 100 * (172.0662 - weighted_cost)\n              / (172.0662 - 164.0022)\n```\n\nRound to three decimal places and clip the result to `[0, 100]`.\n\n- The no-control solution receives `0`.\n- The expert reference receives `100`.\n- Intermediate performance receives a proportional intermediate score.\n- A result better than the expert reference remains capped at `100`.\n- A result worse than no control remains floored at `0`.\n\nAny invalid JSON, violated constraint, modified immutable input, failed SUMO execution,\nnon-finite metric, or missing output receives a score of `0`.\n\n## 4. Reference acceptance criteria\n\nThe frozen expert reference has a weighted cost of `164.0022`, completes 100% of vehicles in\nevery scenario, and produces zero teleports. A valid agent solution is successful to the\ndegree that its deterministic weighted cost approaches or improves upon this reference.\n\nThe evaluator must retain both the continuous score and raw weighted cost. If multiple\nsubmissions receive the capped score of `100`, the lower raw weighted cost is the tie-breaker.\n","software_and_version":"Eclipse SUMO 1.26.0, Python 3.11.16, TraCI 1.26.0, sumolib 1.26.0","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"183f6926-6615-4c9a-a347-2c01fdfc81cc","filename":"SUMO_task_1_public.zip","category":"input_materials","content_type":"application/zip","size_bytes":104170},{"file_id":"55914673-2b01-4e15-876f-7e42e897f2ed","filename":"sumo_urban_incident_recovery_1_reference.zip","category":"reference_output","content_type":"application/zip","size_bytes":417564}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — Revision 2 supplies a coherent, feasible traffic-optimization task with complete inputs, valid reference solutions, and an objectively checkable performance rubric.\n\n**Completeness Check:**\n\n- Instruction: Complete — Permitted routes, five controllable signals, timing constraints, immutable inputs, and the single JSON deliverable are clearly defined.\n- Inputs: Complete — Evidence supports nine signalized intersections, 48 ordinary directed edges, 11,340 visible vehicles, 72 detectors, and the specified lane closure; supplied immutable-file hashes match.\n- Toolchain: Complete — Linux dependency specifications and a lockfile identify matching SUMO, TraCI, and sumolib versions, with a viable single-machine workflow.\n- Reference output: Complete — Expert JSON satisfies the published constraints. Scenario reports support weighted cost 164.0022, full throughput, and zero teleports; their arithmetic checks out.\n- Rubric: Complete — Validation, deterministic assignment, four scenario weights, metric calculation, continuous scoring, and raw-cost tie-breaking are specified consistently.\n\n**File and Task Reachability:**\n\nAttachments: `reachable`. Task: `reachable`.\n\nBoth `SUMO_task_1_public.zip` and `sumo_urban_incident_recovery_1_reference.zip` were inspected. Their byte counts match the manifest, archive integrity checks pass, and no encryption, traversal paths, unsafe links, or excessive expansion were found. The archives contain 20 and 87 entries, respectively, comprising readable UTF-8 text and incidental Macintosh metadata.\n\nInspection covered `TASK.md`, solution schemas and JSON files, network, demand, detector, incident, and configuration files, hidden scenarios, evaluator source, dependency specifications, generators, tests, and reference reports. Public task files match their reference-archive copies. No relevant attachment was left unreviewed; incidental operating-system metadata was only format-checked. XML was inspected as text rather than fully schema-validated, and simulation results were not independently rerun.\n\nThe completion path is to modify only the solution JSON, compile deterministic route assignments and temporary signal controls, evaluate the visible scenario, then score all four scenarios. Reference answers and hidden scenarios are separated from public inputs and must remain evaluator-only.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. Programmatic scenario generation does not establish AI fabrication. Public benchmark provenance could not be independently checked because approved public search was unavailable.\n\n**Recommended Changes:**\n\nThe supplied constraints, integrity manifests, reference reports, and ablations provide a sound evaluation foundation. No contributor changes are required.","reviewed_at":"2026-10-05T21:37:23.644250Z"},"claim":null}