{"submission_id":"1f61cd54-1aef-4343-b1e8-2c4d7aa26572","submission_revision_id":"a7d84919-5244-4d57-98da-92a2dc059287","revision_version":2,"submission_status":"under_review","submitted_at":"2026-09-15T04:38:08.463905Z","submitter_name":"Ziqi Liu","submitter_email":"ziqi.liu1030@gmail.com","task":{"status":"submitted","version":2,"extra_fields":{},"submitted_at":"2026-09-15T04:38:08.463905+00:00","industry_domain":"Game Development / XR Educational Simulation Engineering","task_short_name":"ziqi_liu_unity_classroom_001","operating_system":"MacOS","task_description":"# Reconstruct and validate a three-student classroom training simulation in Unity 6\n\nContributor: Ziqi Liu\nTask identifier: ziqi_liu_unity_classroom_001\nDomain: Game development / XR educational simulation engineering\n\n## Professional objective\n\nDeliver an editable, runnable Unity classroom simulation that a novice teacher can use to rehearse a scripted teaching sequence, respond to a classroom interruption, and replay the resulting session for technical verification. The work combines scene assembly, articulated character animation, object interaction, an asynchronous teaching workflow, event routing, and reproducible validation.\n\nThis is an engineering reconstruction task inspired by an existing Unity VR classroom research prototype. It does not evaluate clinical effectiveness or model any real child's diagnosis. The three fictional students have specified scripted responses. The agent being evaluated is the developer operating Unity, not a teacher being graded and not an LLM controlling the students.\n\n## Starting state and supplied inputs\n\nBegin with a blank project in Unity 6000.0.41f1. The input package provides this assignment, CONTRACT.md, scenarios.json, acceptance_cases.json, example replay JSON files, and ENVIRONMENT.md. There is no preassembled classroom or solved controller in the inputs. Construct the classroom, teaching aids, and three articulated characters with Unity primitives and original materials. No external model, proprietary classroom asset, paid asset, cloud account, audio recording, or credential is required. Legible subtitles supply the specified utterances; audio is optional and receives no extra credit.\n\nThe target is a self-contained desktop evaluation build on macOS plus an editable Unity project. Preserve an input abstraction suitable for XR ray selection and gripping; headset-dependent execution is not required for the core grade. This makes the workflow evaluable without physical hardware while retaining the interaction structure of the originating VR project.\n\n## Work to perform\n\n1. Assemble and save a classroom scene with three individually identifiable seated students, three desks, a teacher viewpoint, and four distinct labelled teaching aids: ruler, eraser, notebook, and pencil. All three students and the target aid must be inspectable using the desktop camera. Use an articulated transform hierarchy with separate arm, hand, head, torso, and leg segments. Primitive art is acceptable; an invisible simulation or status-only dashboard is insufficient.\n\n2. Implement teacher selection and physical presentation of an aid using a pointer, a selection highlight, and a hold/release action. The ruler is the required teaching target. Picking up a different object must not advance the teaching trial. A held object must follow its hold anchor; releasing it restores normal collision and placement behaviour.\n\n3. Implement all three exact event-driven scenarios in scenarios.json: DirectCorrect (4 steps), HalfPromptThenCorrect (17 steps), and FullPromptAfterHalfPromptError (21 steps). Bind them respectively to S01/Coco, S02/Leo, and S03/Anna. Student answers, correction pauses, partial and full prompts, two distractor actions, and final reinforcement must follow the contract. The lesson is not a fixed movie: later state must depend on the teacher's actual actions.\n\n4. Present visible responses with subtitles and animated articulated characters. Correct and incorrect responses have different text; clap and touch-nose instructions cause different hand motions. Gaze must turn toward the active target without rotating the whole classroom. During the interruption case, Coco leaves the seat while Anna's teaching flow continues, and returns on a targeted ReturnToSeat command. The interruption must not change the active learner or consume a teaching step.\n\n5. Route desktop controls and deterministic replay through the same application event dispatcher and state machine. Implement the replay schema in CONTRACT.md. An optional loopback UDP adapter may accept the same events, but internet connectivity and speech recognition are not requirements. Handle duplicates, wrong-target events, unsupported input, reset, and events arriving during delayed responses as specified. Reject malformed input without crashing.\n\n6. Add a visible monitor and append-only JSONL session export. The monitor must report active student, scenario, completed step count, current expected event, selected/held aid, pending response, and completion state. Log raw input disposition, accepted transitions, actual response starts, animation events, interruption events, and resets. Use monotonic session time and the prescribed fields; do not silently convert a timer recovery into successful teacher performance.\n\n7. Run every supplied acceptance case through the actual Unity application, export logs, capture the requested screenshots/video evidence, and build a macOS standalone player. Include a concise operating guide and a report that links every claimed pass to a run ID and artifact. Verify that the saved project reopens with resolved references and that the standalone player operates without the Unity Editor or a network service.\n\n## Deliverables\n\n- UnityClassroom/: Assets, Packages, and ProjectSettings; omit caches and generated Library/Temp directories.\n- Build/: runnable macOS standalone application plus launch instructions.\n- runs/<case_id>/events.jsonl: actual application output for every acceptance case, with a fresh session per case.\n- runs/<case_id>/summary.json: outcome, rejected-input counts, and final application state.\n- evidence/: screenshots and a short actual application recording showing selection/holding, a complete prompted trial, both distractor animations, and an interruption/resumption.\n- VALIDATION.md: environment, controls, steps to repeat each case, observed results, failures, and artifact links.\n\n## Constraints and success\n\nDo not edit the supplied scenario definitions or bypass input handling during replay. Do not fabricate run logs or replace required animation with pre-rendered images or video. The grader may vary student/object selection order, valid inter-event delay, and event IDs while retaining the declared semantics. The application must respond to those actions, not identify a case name and print expected output. Visual geometry and animation implementation may differ if the objective conditions are met.\n\nThe rubric covers executable delivery, exact scenario semantics, integrated physical and visible behaviour, interruption/recovery, and reproducible evidence. See CONTRACT.md and acceptance_cases.json for the agent-visible acceptance criteria. Passing the rubric is distinct from ALE's difficulty classification; no frontier-model score is asserted for this candidate.\n\n## Revision 2 reference and provenance\n\nContributor: Ziqi Liu. This revision responds to the Major Revision request for an executable reference and authentic execution evidence.\n\nThe evaluator-only reference archive contains an editable Unity 6000.0.41f1 / URP 17.0.4 project with a saved classroom scene, original primitive geometry, a universal macOS standalone player, and actual application evidence. The tested player version is 2.0.2. Build and fresh-editor reopen evidence are included.\n\nThe retained C01-C18 evidence comprises 50 actual sessions: C01-C16, ten normal repetitions for each of the three students, one complete 17-step desktop keyboard trial, and the three normal scenarios in a fresh network-denied standalone process. Raw application events, summaries, independent transform observations, timestamped original PNGs, eight recordings, source hashes, and a run index are provided. See VALIDATION.md and actual_run_checks.json for measured results and the check scope. The checker pass flag is not an official benchmark grade.\n\nRevision 2 clarifies departure onset versus achieved displacement, the 1,200 ms distractor collection period versus each 3,000 ms animation, subtitle persistence, and collider-backed target selection. The semantic 4/17/21-step scenario arrays are unchanged. The final reference adds visible number-key target selection so an entire trial can be operated through the keyboard as well as the existing pointer controls.\n\nThe original research project inspired the teaching workflows; its third-party avatars, scenes, audio, account configuration, and credentials are not redistributed. Only the input_materials_r2.zip contents are supplied to the solving agent. The reference_evaluation_r2.zip archive remains evaluator-only. Process logs clearly mark redactions of private machine/network/licensing identifiers; application events and captured frames are unmodified.\n\nActual execution was on macOS 26.5.2, Apple M4 Pro / arm64, Metal. Intel execution, headset performance, clinical effectiveness, human completion time, and frontier-model pass rate are not asserted. Official acceptance and Last Exam difficulty classification remain the organizer's decisions.","software_licensing":"Commercial","evaluation_criteria":null,"verification_method":"Score 100 points: executable editable project/player and shared dispatcher (15); exact 4/17/21-step workflows (30); visible scene, physical aid interaction and articulated animation (20); input ordering, buffering and cancellation (15); distractor concurrency and independent classroom interruption (10); invalid-input recovery, repeatability and evidence (10). Require at least 90/100, all three exact normal scenarios, a runnable player and editable project, and no fabricated evidence or replay bypass. See evaluation/rubric.json for the award rules.\n\nExtract input_materials_r2.zip into input/. Extract reference_evaluation_r2.zip beside it; that archive contains reference/ and evaluation/. Only input/ is agent-visible. Read REVIEW_RESPONSE.md, evaluation/REFERENCE_STATUS.md, reference/VALIDATION.md, and reference/RUN_INDEX.json first. The revision now includes an actual executed reference, not merely a specification or synthetic checker fixture.\n\n1. Provision Unity 6000.0.41f1 with valid activation/macOS build support and URP 17.0.4. Missing licensed infrastructure is an environment failure, not an agent failure. The included universal player was executed on arm64 only.\n2. Open the saved reference scene or launch/rebuild the player using reference/REFERENCE_README.md. Independently open/build the candidate project and inspect serialized references. Missing executable deliverables fail the mandatory delivery condition.\n3. Execute C01-C18 from input/acceptance_cases.json in the actual application. Preserve evaluator-generated logs and captures. Re-run ten normal trials per scenario with fresh IDs and varied valid idle delays, plus one complete prompted trial through visible desktop controls. Do not grade submitted JSONL alone as proof of execution.\n4. Check ordered normal traces with:\npython3 evaluation/verify_trace.py --spec input/scenarios.json --trace reference/runs/C02/events.jsonl --scenario HalfPromptThenCorrect --report C02_check.json\nThis validates one normal trace, not the entire task.\n5. Reproduce the actual-reference audit with:\npython3 evaluation/assess_reference.py --runs reference/runs --spec input/scenarios.json --output reference_recheck.json\nIts scope includes trace ordering, disposition accounting, adversarial cases, scene-transform observations, and evidence presence. Optional reference observations may be adapted for candidate outputs; do not impose undisclosed internal-schema requirements.\n6. Inspect real motion and interaction in the player and reference/videos/: ruler selection/hold/release, distinct clap/touch-nose actions, correct/incorrect subtitles, gaze, and Coco's departure/return while Anna remains active. Cross-check source, transforms, original frames, and event timing. See reference/VISUAL_REVIEW.md. The recordings are timestamped evidence captures, not a frame-rate benchmark.\n7. Award each rubric condition only when independently verified, linking it to the corresponding run/artifact. Vary only disclosed IDs, independent trial order, allowed distractors, and valid delays. Report score and failed mandatory conditions separately from any partial checker's passed flag.\n\nThis revision supplies executable feasibility and acceptance evidence. No frontier-model difficulty score or measured human completion time is claimed; Last Exam tier requires separate calibration.","software_and_version":"Unity 6000.0.41f1; Universal Render Pipeline 17.0.4; C# (Unity toolchain); Python 3.9+ standard library for evaluation","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"419fc464-57a7-44fb-9e93-f750a285a527","filename":"input_materials_r2.zip","category":"input_materials","content_type":"application/zip","size_bytes":14066},{"file_id":"341d2d42-ac7f-4553-a2bc-6c80e351ae65","filename":"reference_evaluation_r2.zip","category":"reference_output","content_type":"application/zip","size_bytes":370240352}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — The revision is complete, professionally scoped, objectively evaluable, and supported by coherent executable, source, trace, transform, and visual evidence without material risk.\n\n**Completeness Check:**\n- Instruction: Complete — Defines the professional objective, scene behavior, event semantics, timing tolerances, recovery rules, deliverables, and success conditions precisely.\n- Inputs: Complete — The solver archive contains the contract, environment specification, task, 18 acceptance cases, three exact 4/17/21-step scenarios, and six valid replay fixtures without evaluator-only leakage.\n- Toolchain: Complete — Unity, URP, C#, macOS build support, licensing expectations, package versions, offline runtime behavior, and the standard-library evaluation environment are specified.\n- Reference output: Complete — Includes an editable saved project, universal macOS player, 50 indexed sessions, raw JSONL events, independent transform observations, rendered evidence, build/reopen records, and operating and validation guides.\n- Rubric: Complete — Allocates 100 points across six objective groups, defines partial-credit boundaries, evidence requirements, a 90-point threshold, and mandatory executable/scenario conditions.\n\n**File and Task Reachability:**\nAttachment classification: reachable. Task classification: reachable. Both `input_materials_r2.zip` and `reference_evaluation_r2.zip` were inspected. The input archive was inspected in full. The reference archive’s complete 1,703-entry directory and safety metadata were inventoried, followed by inspection of `ARCHIVE_MANIFEST.json`, `REVIEW_RESPONSE.md`, reference status/readme/validation/visual-review documents, run and check indexes, rubric and checker source, Unity manifests and runtime source, the serialized scene, build metadata and logs, 20 representative event traces, three transform logs, nine PNGs, and three MP4 containers. The archives are valid, unencrypted ZIPs with no unsafe paths or special entries; selected members passed CRC and submitted-manifest hash checks. Remaining Unity runtime libraries, five video payloads, most repeated-run logs, and the bulk captured-frame set were inventory and manifest reviewed but not individually decoded. The reviewer environment could not launch the macOS player or Unity and lacked a video decoder, so executable behavior was cross-checked through bundle structure, static source, raw traces, independent transforms, build/reopen evidence, and rendered frames. An ordinary evaluator with the declared macOS Unity environment has a complete build, execution, and scoring path. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. Public provenance was not externally searched because it falls outside the permitted evidence boundary, and no affirmative evidence supports a screening label.\n\n**Recommended Changes:**\nThe revision resolves the prior executable-reference and authentic-evidence gaps comprehensively. No material changes are required.","reviewed_at":"2026-09-15T04:56:59.779095Z"},"claim":null}