{"claim":{"id":"65a72851-9207-4e45-838a-83f955c2b880","submission_id":"4d18e940-268b-4889-9d0b-67e32f65af5c","submission_revision_id":"6a77f319-2340-4269-9cb8-897227ca1fa7","builder":"builder-01","status":"active","claimed_at":"2026-10-15T14:40:09.539033Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"4d18e940-268b-4889-9d0b-67e32f65af5c","submission_revision_id":"6a77f319-2340-4269-9cb8-897227ca1fa7","revision_version":4,"submission_status":"under_review","submitted_at":"2026-06-29T00:44:58.004763Z","submitter_name":"Sidharth Mohla","submitter_email":"sidharthmohla@gmail.com","task":{"status":"submitted","version":4,"extra_fields":{"difficulty_self_test":{"model":"Claude Opus 4.8","harness":"Claude Code","estimated_score":"<5%"}},"submitted_at":"2026-06-29T00:44:58.004763+00:00","industry_domain":"3D Animation / Character Rigging (Media & Entertainment)","task_short_name":"skeleton_retargeting_map_instance_1","operating_system":"Linux","task_description":"Objective\nSet up the bone correspondence required to retarget an animation between two character rigs. Given a SOURCE skeleton (which carries the animation) and a TARGET skeleton (which should receive it), output the mapping source_bone -> target_bone (or null) for EVERY source bone. The mapping must pair bones by their role in the body using skeleton geometry and hierarchy -- not names -- and must get left vs right correct.\n\nStarting State\n/input/skeletons.json contains two objects, \"source\" and \"target\". Each is a list of \"bones\"; each bone has a \"name\", a \"parent\" (or null for the root), and rest-pose \"head\" and \"tail\" 3D coordinates [x, y, z]. The two rigs deliberately do not line up:\n- different naming conventions (in hard variants the target names are opaque ids);\n- different bone counts (different spine subdivisions and finger counts; the target may drop bones the source has -> those map to null, and may add non-deforming twist/helper bones that correspond to nothing);\n- the target is expressed in a different world frame (an arbitrary rotation, a uniform scale, and a translation relative to the source).\nThis instance is one of an unlimited seeded family; any integer seed yields a different, realistic rig pair.\n\nInstructions & Rules\n1. Parse both skeletons into bone trees with their rest-pose geometry.\n2. Establish each rig's intrinsic body frame (up, forward, lateral) so the target's rotation, uniform scale and translation are factored out.\n3. Match bones by body role using geometry + hierarchy, not names (names may be meaningless).\n4. Resolve left vs right correctly on the symmetric limbs.\n5. Map source bones with no target counterpart (dropped bones) to null; leave target-only twist/helper bones unused.\n6. Write /output/mapping.json: a single JSON object with one entry per source bone, e.g. {\"Hips\":\"pelvis\",\"LeftHand\":\"l_hand\",\"LeftToeBase\":null, ...}.\n\nOutput Specification\nWrite exactly one file: /output/mapping.json. It must be a FLAT JSON object (no nesting): each key is a source bone name and each value is the matching target bone name (a string) or null. Include one entry for EVERY source bone. Example:\n{\"Hips\": \"pelvis\", \"LeftHand\": \"l_hand\", \"LeftToeBase\": null}\n\nAllowed tools: Python 3 with NumPy (open-source), or a professional 3D pipeline (Autodesk MotionBuilder characterization, Maya HumanIK, Blender/Rokoko retargeting) which requires exactly this bone-correspondence step.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Grading is deterministic, exact, and needs no human/LLM judge: it is a bit-exact comparison of two JSON objects (the source->target bone map) with no floating-point tolerance and no EPS, so it is fully reproducible across machines. There is no golden/answer file to leak.\n\nContamination-proof and isolated: the TARGET rig is procedurally derived from the seed, so the grader regenerates the ground-truth correspondence from the seed and compares. The grader AND the seeded instance generator are the same bundled file, _shared_eval.py (included), so ground truth is regenerated from the seed using bundled code, not an external/referenced script; the per-instance seed is supplied by the task harness (main.py) at evaluation time and is deliberately not pre-shipped. At runtime the agent receives ONLY /input/skeletons.json -- the grader, reference solution, seed, and example answer are evaluator-only and are never staged to the agent (so even the shipped worked example is unreachable by an agent under test). Requires only Python 3 + NumPy. A passing example is included (examples/seed1/grade_report.json shows 1.0).\n\nGate-and-score procedure:\n1. Gates (score 0 on any failure): output present and parseable; it is a JSON object; keys are strings; values are a target bone name or null.\n2. Score: the fraction of source bones mapped to the CORRECT target bone. Mapping to null is correct only when that source bone genuinely has no counterpart in the target.\n3. Full credit (1.0) only if EVERY source bone is exactly right (a retarget with even one flipped or wrong bone is broken, so partial correctness is not a pass).\n4. Pass/Fail cutoff: Pass = 1.0; any score below 1.0 is a Fail. The grader returns an explicit \"pass\" boolean; the continuous fraction-correct score is recorded for diagnostics only.\n\nDifficulty (calibrated and verified; see docs/DIFFICULTY_PROFILE.md, reproducible via selfcheck.py and evaluations/agent_baselines.py): matching by names scores ~0.42 in the standard variant and ~0.16 when names are opaque; matching by raw coordinates scores ~0.07-0.33; an expert geometric method that does not disambiguate left/right reaches ~0.6-0.9 -- all with 0/6 full passes. Only the full method (intrinsic body frame + geometry + left/right + handling dropped/added bones) reaches 1.0 on all six public seeds in both difficulty modes.\n\nVariants: a few public seeds are shipped as examples; the live graded instance uses a different, private seed from the same generator (regenerated by the grader), so an agent cannot reuse a shipped example.","software_and_version":"Python 3.x with NumPy (the only third-party package; used by the bundled generator/grader). No other dependencies; runs headless. requirements.txt is included in the package.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"4f707854-249c-45d0-b6ff-aa2a153cc069","filename":"skeletons.json","category":"input_materials","content_type":"application/json","size_bytes":17286},{"file_id":"835c7692-2538-479a-a9cf-0de8e5da77b2","filename":"skeleton_retargeting_map.zip","category":"reference_output","content_type":"application/x-zip-compressed","size_bytes":42144}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The task is complete, objectively gradable, and reachable, with minor integration and validator discrepancies that should be tightened before deployment.\n\n**Completeness Check:**\n- Instruction: Complete — The source-to-target mapping objective, geometric and hierarchical constraints, null behavior, handedness requirement, and flat JSON deliverable are clear.\n- Inputs: Complete — `skeletons.json` is valid UTF-8 JSON containing 27 uniquely named source bones and 26 uniquely named target bones, each with valid hierarchy references and numeric 3D geometry.\n- Toolchain: Complete — Python 3 and NumPy provide a legal, headless, single-machine workflow; the benchmark adapter dependencies are evaluator infrastructure rather than solver requirements.\n- Reference output: Complete — The reference package includes a 27-entry passing mapping, grade report, static reference solver, seeded generator/grader, self-check, baseline analysis, metadata, and integration documentation.\n- Rubric: Complete — Exact per-source correspondence scoring and the 1.0 pass threshold are deterministic. Minor implementation variance exists: the grader accepts extra keys and treats an empty string as null, and its structural gate does not reject arbitrary string values before scoring.\n\n**File and Task Reachability:**\nThe attachment set is `reachable`: both manifest files matched their declared byte counts and actual formats. `skeletons.json` and every member of `skeleton_retargeting_map.zip` were inspected, including `main.py`, `requirements.txt`, `selfcheck.py`, `README.md`, `reference_solution.py`, `_shared_eval.py`, `task_card.json`, `SUBMISSION_FORM.md`, `INTEGRATION.md`, all three example JSON files, all documentation files, and `evaluations/agent_baselines.py`. The ZIP was readable, unencrypted, CRC-valid, path-safe, and contained only regular text files and directories; no content was left unreviewed.\n\nThe task is `reachable`: the solver reads the two skeleton trees, produces the flat mapping, and the evaluator regenerates seeded ground truth and scores every source bone. The supplied input is byte-identical to the packaged seed-1 standard-mode example, and its reference mapping covers every source bone, maps two dropped bones to null, and leaves one target helper unused. Evaluator-only separation prevents the worked mapping and reference solver from becoming solver-visible shortcuts. One integration caveat remains: the adapter currently registers public training seeds with standard mode, while the documentation describes a held-out private instance and arbitrary rotation in hard mode.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone\n\n**Recommended Changes:**\nThe submission provides a coherent task, valid evidence, and a strong deterministic evaluation path. No material contributor-side changes are required for reachability. Before deployment, configure an actual held-out private seed and the intended difficulty mode, remove unnecessary seed metadata from solver-visible input, and make the grader enforce exact source-key coverage, target-name membership, and JSON null without accepting empty-string aliases.","reviewed_at":"2026-08-15T23:20:38.380390Z"},"claim":{"id":"65a72851-9207-4e45-838a-83f955c2b880","submission_id":"4d18e940-268b-4889-9d0b-67e32f65af5c","submission_revision_id":"6a77f319-2340-4269-9cb8-897227ca1fa7","builder":"builder-01","status":"active","claimed_at":"2026-10-15T14:40:09.539033Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}