{"submission_id":"31c64b19-9876-48c2-8192-1cda3fef7b34","submission_revision_id":"5295117c-c423-4719-b8fb-0d219b6d45d8","revision_version":1,"submission_status":"under_review","submitted_at":"2026-08-08T17:41:06.942995Z","submitter_name":"Jiageng Mao","submitter_email":"maojiageng@gmail.com","task":{"status":"submitted","version":1,"extra_fields":{"difficulty_self_test":{"model":"Opus 4.8","harness":"Claude Code","estimated_score":"<1%"}},"submitted_at":"2026-08-08T17:41:06.942995+00:00","industry_domain":"Robotics & Autonomous Systems (Engineering)","task_short_name":"tidybot","operating_system":"Linux","task_description":"**Objective:** Tidy a table into a bin. Produce ONE continuous whole-body\ntrajectory that makes a Unitree G1 humanoid (Dex3 3-finger hands) walk to each of\n**three** objects left out on a table — a thermos, a vitamin box and a clock —\npick it up, carry it to a bin standing on the floor, and put it inside. All three\nin a single uninterrupted episode, without falling at any point.\n\n**Starting State:** A fully self-contained MuJoCo simulation: `assets/scene.mjb`\n(robot standing away from the table, meshes embedded), a frozen initial physics\nstate (`init_state.npy`), the balance policy's internal state\n(`amo_init_state.npz`), and robot parameters (`robot_params.json`: joint order,\nPD gains, torque limits, the three target objects' body labels, the bin's\naxis-aligned bounds, and every scoring threshold). A bundled RL balance policy\n(`policy/`) converts the agent's locomotion command stream into leg motion.\n\n**Instructions & Rules:** This is a whole-body task — one trajectory drives both\nlocomotion (walking, turning) and the upper body (waist 3, arms 14, Dex3 hands\n14). It is ONE episode: nothing is teleported or reset partway, and the robot\nmust stay on its feet for the whole of it, not just at the end. How the robot accomplishes the task is entirely the agent's to work out.\nRecord the run as `output/trajectory.npz` containing `action [T,43]`,\n`amo_policy_command [T,9]`, `amo_policy_target_yaw [T,1]` and\n`amo_policy_turning_flag [T,1]` (contract in `TRAJECTORY_SCHEMA.md`). Do not\nwrite anything outside `output/`.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Verification is **functional, objective and deterministic** (a replay, not a\nsubjective comparison). The grader is run as:\n\n```\npython3 run_grader.py --output <submission dir> \\\n    --assets input/assets --policy input/policy --results result.json\n```\n\nStrict rules:\n1. `output/trajectory.npz` must exist and contain `action [T,43]`,\n   `amo_policy_command [T,9]`, `amo_policy_target_yaw [T,1]`,\n   `amo_policy_turning_flag [T,1]`; each 2-D with exactly those column counts,\n   numeric dtype, `T >= 1`, all finite, all sharing the same `T`, and\n   `T - 1 <= max_replay_steps` (200,000; the reference is 42,976). Any\n   violation → 0.\n2. The grader restores the frozen MuJoCo integration state and balance-policy\n   state and replays the submitted trajectory under the exact contract in\n   `TRAJECTORY_SCHEMA.md`.\n3. Score is **1.0 iff** at the final step **all three** objects lie within the\n   bin's XY bounds (`bin_aabb_lo/hi`, 0.02 m pad) **AND** below the rim\n   (z < 0.28 m) **AND** the robot is upright (pelvis > 0.60 m) **AND** the\n   pelvis never dropped to 0.45 m at any step of the replay; else **0.0**.\n   There is no partial credit — two objects in the bin scores the same as none.\n4. A malformed npz, a missing file, ragged lengths, a non-numeric dtype, an\n   over-length trajectory, a crashed replay, a dropped object or a fallen robot\n   all score 0.0.\n\nVerified reference result: `success: true`, `score: 1.0`, 3/3 in bin —\n103 thermos (1.4423, 0.3908, 0.1039), 105 vitamin_box (1.4470, 0.3160, 0.2680),\n106 clock (1.4945, 0.3887, 0.2254); `min_pelvis_z` 0.7613, final pelvis 0.7777,\n42,975 replay steps. Reproduced identically on three independent replays,\nincluding one from a clean extraction of the two uploaded tarballs.\n\nNegative controls, all → 0.0: missing file (`missing_trajectory`); wrong column\ncount and ragged array lengths (`schema_gate`); string dtype and pickled object\narray (`schema_gate`, no traceback); over-length trajectory\n(`trajectory_too_long`); all-zero trajectory (0/3 in bin, `never_fell` false);\na single-object trajectory that puts only the thermos in the bin (1/3 → 0.0,\nconfirming partial success is rejected).","software_and_version":"MuJoCo 3.3.6, PyTorch 2.7.0 (CUDA 12.x), NumPy, transforms3d 0.4.2, Python 3.10.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"05bb0bdd-e3cf-4c09-b4b7-111c6e5e2a31","filename":"tidybot_ref.tgz","category":"reference_output","content_type":"application/x-compressed","size_bytes":3179736},{"file_id":"474d62aa-f502-4975-9fd0-bf969c17d5a2","filename":"tidybot_data.tgz","category":"input_materials","content_type":"application/x-compressed","size_bytes":106527114}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — The submission is complete, professionally scoped, internally coherent, and provides a reachable single-machine workflow with a valid reference trajectory and objective evaluator.\n\n**Completeness Check:**\n- Instruction: Complete — The objective, continuous-episode constraint, whole-body controls, prohibited resets, required artifact, and success state are explicit.\n- Inputs: Complete — The compiled scene, frozen physics and policy states, robot parameters, policy implementation, model weights, normalization data, and supporting documentation are supplied.\n- Toolchain: Complete — Linux, MuJoCo, PyTorch/CUDA, NumPy, transforms3d, and applicable versions and GPU requirements are identified through legally obtainable software.\n- Reference output: Complete — The reference trajectory contains exactly the four required finite numeric arrays with 42,976 aligned rows and substantive locomotion and manipulation commands.\n- Rubric: Complete — Schema failures, replay limits, all-or-nothing object placement, final posture, continuous fall detection, and numerical thresholds are objectively defined and implemented.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. Task classification: `reachable`. Both `tidybot_ref.tgz` and `tidybot_data.tgz` were inspected and matched their manifest byte counts; both are readable gzip/tar containers without traversal paths, links, device entries, encryption, or excessive expansion. Inspected contents included `trajectory.npz`, `run_grader.py`, `task_brief.md`, `SETUP.md`, `TRAJECTORY_SCHEMA.md`, `SUCCESS.md`, `scene.mjb`, `init_state.npy`, `amo_init_state.npz`, `robot_params.json`, `amo_policy.py`, `amo_jit.pt`, `adapter_jit.pt`, and `adapter_norm_stats.pt`. Text files have valid ASCII/UTF-8 encoding and syntactically valid Python where applicable. NumPy artifacts were parsed without pickle support and have coherent schemas, finite values, and expected dimensions. The Torch artifacts are parseable ZIP-based containers with generated TorchScript code and tensor records; the normalization pickle references expected NumPy constructors. The MuJoCo binary contains every robot, object, actuator, and sensor identifier required by the parameters and grader. Embedded mesh geometry and all model-weight values were inventoried but not exhaustively interpreted. Independent replay was not performed because a matching MuJoCo 3.3.6/CUDA runtime was unavailable during review; this is a reviewer-tooling limitation, not a submission defect. The end-to-end path is clear: restore the supplied states, use the bundled balance policy while recording one whole-body trajectory, save the four-array NPZ, then schema-check and replay it to evaluate all three objects and robot stability. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. Public identifiability could not be independently checked because external search was unavailable; no affirmative evidence supports a special-screening label.\n\n**Recommended Changes:**\nThe submission already provides unusually thorough contracts, safety gates, negative controls, threshold margins, and reproducibility evidence. No changes are required.","reviewed_at":"2026-08-11T07:37:06.006406Z"},"claim":null}