{"claim":{"id":"395bcf6b-0e4f-44d1-bb46-3dce0788406e","submission_id":"00a0789e-22b8-47b5-bc38-27584bf54d29","submission_revision_id":"50393603-bb3d-423f-908e-1b42b1d8be45","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:05:19.083099Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"00a0789e-22b8-47b5-bc38-27584bf54d29","submission_revision_id":"50393603-bb3d-423f-908e-1b42b1d8be45","revision_version":2,"submission_status":"under_review","submitted_at":"2026-08-16T13:39:33.388240Z","submitter_name":"Aditya Prakash","submitter_email":"prakashadityaap@gmail.com","task":{"status":"submitted","version":2,"extra_fields":{"ai_draft":{"source":"submissions_workbench","status":"ready","preview":{"domain":"Biomedical Engineering / Surgical Navigation","task_goal":"Derive and validate the rigid transform between an ultrasound image plane and an electromagnetic tracking sensor from tracked phantom sweeps.\n\nStarting state: Expert receives tracked ultrasound sweep recordings of a calibration phantom plus EM tracker pose logs and probe/sensor CAD offsets, but no existing image-to-sensor transform.\n\nInstructions and rules: Using the tracked freehand sweeps of a known calibration phantom (e.g., N-wire or cross-wire), determine the spatial transform ImageToProbe (or ImageToSensor) that maps ultrasound pixel coordinates into the EM sensor's tracked coordinate frame. The expert must segment fiducials in each frame, pair them with synchronized tracker poses, solve the calibration, and produce a machine-readable transform plus per-frame residual error data. Do not hand-tune the transform against a single held-out validation sweep; the calibration must generalize across all provided sweep sequences.","short_name":"us_em_probe_calibration","input_files":"- tracked_sweep_01.igs.mha (MHA (sequence), ~350 MB): Synchronized ultrasound B-mode frames with embedded EM tracker pose per frame from a phantom sweep; primary calibration data source\n- tracked_sweep_02.igs.mha (MHA (sequence), ~350 MB): Second independent tracked sweep of the same phantom used as held-out generalization check\n- phantom_geometry.xml (XML, <10 KB): Known 3D coordinates of calibration phantom fiducials (e.g., N-wire geometry) used to constrain the fiducial-to-tracker mapping","output_files":"- ImageToSensor.h5 (HDF5 (transform)) [graded]: 4x4 rigid transform matrix mapping ultrasound image pixel coordinates into EM sensor coordinate frame; primary calibration result","operating_system":"Windows","software_licensing":"Free/Open Source","verification_method":"ImageToSensor.h5: parse/schema pass; required fields = 100%; records > 0.\nImageToSensor.h5: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nCross-file check: outputs stay traceable to inputs (tracked_sweep_01.igs.mha, tracked_sweep_02.igs.mha); mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (HDF5 (transform); declared constraints).","evaluation_criterion":"Method: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_and_version":"PLUS Toolkit 2.9, 3D Slicer 5.6 (SlicerIGT), Python 3.11","tools_software_details":"Software: PLUS Toolkit 2.9, 3D Slicer 5.6 (SlicerIGT), Python 3.11\nSegment phantom fiducials in each tracked US frame and pair with EM poses (Tools: PLUS Toolkit, 3D Slicer)\nSolve rigid calibration transform and validate residuals across sweeps (Tools: SlicerIGT, Python)"},"warnings":["estimated_expert_hours \"\" does not meet the ≥5 hour requirement","expected_output has no structured data file (TSV/CSV/VCF/JSON/BAM etc.)"],"createdAt":1785380518609,"formPrefill":{"extra_fields":{},"industry_domain":"Biomedical Engineering / Surgical Navigation","task_short_name":"us_em_probe_calibration","operating_system":"Windows","task_description":"# Task Goal\n\nDerive and validate the rigid transform between an ultrasound image plane and an electromagnetic tracking sensor from tracked phantom sweeps.\n\nStarting state: Expert receives tracked ultrasound sweep recordings of a calibration phantom plus EM tracker pose logs and probe/sensor CAD offsets, but no existing image-to-sensor transform.\n\nInstructions and rules: Using the tracked freehand sweeps of a known calibration phantom (e.g., N-wire or cross-wire), determine the spatial transform ImageToProbe (or ImageToSensor) that maps ultrasound pixel coordinates into the EM sensor's tracked coordinate frame. The expert must segment fiducials in each frame, pair them with synchronized tracker poses, solve the calibration, and produce a machine-readable transform plus per-frame residual error data. Do not hand-tune the transform against a single held-out validation sweep; the calibration must generalize across all provided sweep sequences.\n\n# Input Files\n\n- tracked_sweep_01.igs.mha (MHA (sequence), ~350 MB): Synchronized ultrasound B-mode frames with embedded EM tracker pose per frame from a phantom sweep; primary calibration data source\n- tracked_sweep_02.igs.mha (MHA (sequence), ~350 MB): Second independent tracked sweep of the same phantom used as held-out generalization check\n- phantom_geometry.xml (XML, <10 KB): Known 3D coordinates of calibration phantom fiducials (e.g., N-wire geometry) used to constrain the fiducial-to-tracker mapping\n\n# Output Files\n\n- ImageToSensor.h5 (HDF5 (transform)) [graded]: 4x4 rigid transform matrix mapping ultrasound image pixel coordinates into EM sensor coordinate frame; primary calibration result\n\n# Tools / Software Details\n\nSoftware: PLUS Toolkit 2.9, 3D Slicer 5.6 (SlicerIGT), Python 3.11\nSegment phantom fiducials in each tracked US frame and pair with EM poses (Tools: PLUS Toolkit, 3D Slicer)\nSolve rigid calibration transform and validate residuals across sweeps (Tools: SlicerIGT, Python)\n\n# Evaluation Criterion\n\nMethod: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_licensing":"Free/Open Source","verification_method":"ImageToSensor.h5: parse/schema pass; required fields = 100%; records > 0.\nImageToSensor.h5: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nCross-file check: outputs stay traceable to inputs (tracked_sweep_01.igs.mha, tracked_sweep_02.igs.mha); mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (HDF5 (transform); declared constraints).","software_and_version":"PLUS Toolkit 2.9, 3D Slicer 5.6 (SlicerIGT), Python 3.11"},"draftSession":"949c9b45-13d4-4aea-8dda-2d2d925dbe08"}},"submitted_at":"2026-08-16T13:39:33.388240+00:00","industry_domain":"Robotics / Computer Vision (rigid-body calibration)","task_short_name":"hand_eye_calibration","operating_system":"Linux","task_description":"Summary: given N pairs of rigid motions (A_i, B_i) satisfying A_i X = X B_i for one\nunknown constant rigid transform X, recover X (closed-form hand-eye calibration).\nData is noise-free so X is exact and unique. Output X and the max residual to\n`result.json`. Formulas and schema are in `SPEC.md`. See below for reference as well:\n\n## Goal\n`scene.json` contains `N` pairs of rigid motions `(A_i, B_i)`, each a 4×4 homogeneous\nmatrix. They satisfy, for one **unknown constant** rigid transform `X`:\n\n```\nA_i · X = X · B_i     for every i\n```\n\nRecover `X` and write it to `result.json`. The data is **noise-free**, so `X` is exact\nand unique.\n\n## Conventions\n- 4×4 matrices are **row-major** rigid transforms: upper-left 3×3 is an orthonormal\n  rotation (det = +1), last column is translation (mm), bottom row is `[0,0,0,1]`.\n- Your `X` must be a valid rigid transform in the same form.\n\n## Method (any correct approach is accepted)\nA standard closed form (Park & Martin, or Tsai–Lenz):\n1. For each pair, take `α_i = log(R_A_i)`, `β_i = log(R_B_i)` (rotation-vector via the\n   SO(3) logarithm). Build `M = Σ β_i α_iᵀ`; then `R_X = (Mᵀ M)^(-1/2) Mᵀ`.\n2. Solve the stacked linear system `(R_A_i − I) t_X = R_X t_B_i − t_A_i` for `t_X` in a\n   least-squares sense.\n\nYou may use any method (dual quaternions, etc.) as long as the result satisfies the\nconstraint on all pairs.\n\n## Input: `scene.json`\n```\n{ \"pairs\": [ { \"A\": [[4x4]], \"B\": [[4x4]] }, ... ] }\n```\n\n## Output: `result.json`\n```\n{\n  \"X\": [[4x4]],\n  \"max_residual\": <float>   // max over i of the Frobenius norm ||A_i X - X B_i||\n}\n```\nReport `X` as a row-major 4×4 and `max_residual` as the largest residual your `X`\nachieves across all pairs. Python 3.11 + NumPy only. No network access.\n","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Binary pass/fail via `python grader.py scene.json result.json <agent_output_file>`.\n\n1. Submission has keys `X` (4x4 row-major) and `max_residual`.\n\n2. X is a valid rigid transform: rotation orthonormal and det = +1 (tol 1e-9), bottom row [0,0,0,1].\n\n3. X satisfies the data: max over all pairs of Frobenius norm ||A_i X - X B_i|| <= 1e-6.\n\n4. Self-reported max_residual matches the actual value (tol 1e-5).\n\n5. X matches the reference: geodesic rotation error <= 1e-4 rad and translation error  <= 1e-4 mm.","software_and_version":"Python 3.11 with NumPy 1.24+ (standard library only otherwise)","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"59e1f390-e33d-47cf-aff6-c777a460c7a4","filename":"SPEC.md","category":"input_materials","content_type":"text/markdown","size_bytes":1436},{"file_id":"f82f52cb-a2e9-4a3b-8d82-cb7e2189a687","filename":"scene.json","category":"input_materials","content_type":"application/json","size_bytes":9138},{"file_id":"a2cb9888-6b02-473d-99cd-24d9d5b86776","filename":"result.json","category":"reference_output","content_type":"application/json","size_bytes":494},{"file_id":"513eb150-5b42-4dc5-916e-5ab7875bd1c4","filename":"grader.py","category":"reference_output","content_type":"text/x-python","size_bytes":4172}],"deep_review":{"overall_verdict":"Strong Accept","review_text":"**Overall Verdict:** Strong Accept — The submission is complete, mathematically consistent, independently reproducible, objectively graded, and fully reachable with the declared toolchain.\n\n**Completeness Check:**\n- Instruction: Complete — The equation, matrix conventions, units, accepted methods, input schema, output schema, and required residual are clearly specified.\n- Inputs: Complete — `scene.json` parses successfully and contains eight exact-schema motion pairs; all matrices are finite 4×4 rigid transforms with valid rotations and homogeneous bottom rows.\n- Toolchain: Complete — Python 3.11 and NumPy 1.24+ provide a free, standard, single-machine implementation path without network access or specialized hardware.\n- Reference output: Complete — `result.json` contains a valid rigid transform and an accurately reported residual near \\(3.8\\times10^{-13}\\). Independent closed-form reconstruction matched its rotation and translation to numerical precision.\n- Rubric: Complete — `grader.py` statically verifies schema, finite values, rigid-transform validity, equation residual, reported-residual consistency, and agreement with the reference using explicit tolerances.\n\n**File and Task Reachability:**\nThe attachment set is `reachable`, and the end-to-end task is `reachable`. `SPEC.md`, `scene.json`, `result.json`, and `grader.py` were all inspected; their actual formats and byte counts match the manifest, the text is readable UTF-8, both JSON files parse successfully, and there are no containers, encrypted members, or unreviewed contents. The uploaded grader was inspected statically and was not executed, as required; independent safe numerical analysis confirmed full-rank rotational and translational systems, a unique solution, valid reference geometry, and residual consistency. The completion path is direct: read the eight motion pairs, solve for the rigid transform with NumPy, serialize `X` and `max_residual`, and evaluate the output against the supplied numerical checks. Solver-visible inputs and evaluator/reference assets are explicitly separated by manifest category. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. There is no affirmative evidence of AI-fabricated core evidence, a public benchmark, or a synthetic defect fixture. Public-source identity was not independently searched because external lookup is outside the permitted evidence boundary.\n\n**Recommended Changes:**\nThe submission already provides coherent instructions, valid data, a verified reference, and a hardened objective evaluator. No changes are required.","reviewed_at":"2026-08-16T13:47:06.526325Z"},"claim":{"id":"395bcf6b-0e4f-44d1-bb46-3dce0788406e","submission_id":"00a0789e-22b8-47b5-bc38-27584bf54d29","submission_revision_id":"50393603-bb3d-423f-908e-1b42b1d8be45","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:05:19.083099Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}