{"claim":{"id":"77479688-05dd-450f-8bba-34537732421e","submission_id":"2a9b2666-84dd-4606-98cb-cc84d19b7fb7","submission_revision_id":"4dd1dd96-c75d-43f7-b19a-47640048a3f0","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:24:57.034314Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"2a9b2666-84dd-4606-98cb-cc84d19b7fb7","submission_revision_id":"4dd1dd96-c75d-43f7-b19a-47640048a3f0","revision_version":2,"submission_status":"under_review","submitted_at":"2026-09-09T03:06:24.868067Z","submitter_name":"Yunze XIao","submitter_email":"yunzex@andrew.cmu.edu","task":{"status":"submitted","version":2,"extra_fields":{"difficulty_self_test":{"model":"GPT 5.6 Sol","harness":"Codex","estimated_score":"10–20%"}},"submitted_at":"2026-09-09T03:06:24.868067+00:00","industry_domain":"Mechanical Engineering / CAD design","task_short_name":"cad_reconstruction_from_views_instance_1","operating_system":"Linux","task_description":"Objective: Rebuild 12 standard mechanical parts (helical gear, round flange, ball knob, turnbuckle, T-slot rail, bellows, dovetail slide, star blank, mesh panel, torus link, bucket, chair) as editable 3D solids from catalogue-style orthographic views.\nStarting State: `input/parts/<stem>/` for each of the 12 parts: `view_0.png` .. `view_3.png` (orthographic renders), `composite.png` (all views side by side), `meta.json` (stem, part family, governing standard such as ISO 53 or DIN 319 when there is one, difficulty tag); `input/README.md` (the task text plus export tips).\nInstructions & Rules: Study the views and the standard, model each part as one closed solid in an open-source CAD tool, and export `output/<stem>.step` for every part (exactly one solid per file; units and placement are free; all 24 axis-aligned orientations are accepted). Proportions and internal features (holes, teeth, wall thickness, fillets) must match the views. A file with several solids is fused before grading but the deliverable is one solid. No searching for the original dataset or CAD programs online.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Deterministic geometry comparison. For each part: load `output/<stem>.step`; a missing, unreadable, no-solid, invalid (BRepCheck) or zero-volume file scores 0 for that part; several solids are first fused with a boolean union. Translate the agent shape and the reference solid to their bounding-box centre and scale each to unit bounding-box diagonal. Enumerate all 24 proper axis-aligned rotations of the agent shape (signed permutation matrices with determinant +1, fixed order); for each compute volume IoU = V(A∩B) / (V(A) + V(B) − V(A∩B)) with an exact OpenCASCADE boolean intersection and keep the maximum. No candidate pruning, no early exit. Part score = clip((IoU − 0.5) / 0.5, 0, 1); task score = mean over the 12 parts. Each part is graded in its own worker process (4 in parallel); a 900 s per-part budget is a safety valve for degenerate inputs (mesh-converted STEP with tens of thousands of faces) and keeps the best IoU found so far, flagged in the log. Calibration on the reference data with this grader: reference solids 1.000; copy rotated 90° about X and Z and translated 1.000 (per-part IoU at least 0.9998); bounding box of the right proportions 0.170. Implementation: `grade.py::score_parts`; one full evaluation takes about 3 minutes for the 12 parts with 4 worker processes on a 10-core machine (41 s for crude box outputs, 151-168 s when the outputs are the reference solids themselves, whose coincident faces make the slowest booleans).","software_and_version":"CadQuery 2.x (Python 3.12) recommended; build123d, FreeCAD 0.21, OpenSCAD 2021 also acceptable. Output must be STEP AP214.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"c5ce61cb-59aa-4ff4-8a02-2870b66b7de5","filename":"evidence.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":32367},{"file_id":"fe08be6e-d36d-4795-9605-f1a94f81a716","filename":"input.zip","category":"input_materials","content_type":"application/zip","size_bytes":474985},{"file_id":"e4cf7967-38ca-46d8-9dd4-7621306fd864","filename":"reference.zip","category":"reference_output","content_type":"application/zip","size_bytes":324166}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — complete, coherent, reachable, and objectively evaluable with no material submission gaps.\n\n**Completeness Check:**\n- Instruction: Complete — clearly defines reconstruction, output naming, STEP AP214, solid requirements, and evaluation.\n- Inputs: Complete — 12 part directories with README, metadata, four views, and composites; schemas and image headers are consistent.\n- Toolchain: Complete — an open-source CadQuery workflow is specified for Linux, with compatible alternatives.\n- Reference output: Complete — 12 matching STEP references, generator text files, and a manifest; all references report one valid solid.\n- Rubric: Complete — normalization, 24 rotations, exact volume IoU, scoring, invalid-file handling, calibration, and runtime limits are specified.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. Task classification: `reachable`. I inspected `evidence.zip` (scores, prompts, summaries, and valid JSONL trajectories), `input.zip` (README, all metadata files, all PNG members, and representative composites), and `reference.zip` (manifest, all 12 STEP files, and all generator text files). Manifest byte counts matched the downloaded files; ZIP integrity checks passed, with no encryption, traversal paths, or unsafe special entries. Only macOS AppleDouble sidecars were not decoded; they are non-required metadata. Uploaded generator programs were read as text and not executed. A local CAD/OCP runtime was unavailable for full BRep import, so STEP AP214 headers, one `MANIFOLD_SOLID_BREP` declaration per file, and the reference manifest’s `ok`/`solids: 1` records were used instead; this is a reviewer-tooling limitation. The end-to-end path is clear: obtain the inputs, model each part with an open-source CAD tool, export twelve STEP files, and evaluate them using the stated deterministic IoU grader. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. No affirmative evidence supports an `ai_synthetic`, `public_benchmark`, or `synthetic_defect_fixture` label. Public provenance was not independently checked, so no public-benchmark claim is asserted.\n\n**Recommended Changes:**\nThe submission provides a complete input package, usable toolchain path, protected reference set, and rigorous evaluator. No material changes are required.","reviewed_at":"2026-09-09T03:16:03.397158Z"},"claim":{"id":"77479688-05dd-450f-8bba-34537732421e","submission_id":"2a9b2666-84dd-4606-98cb-cc84d19b7fb7","submission_revision_id":"4dd1dd96-c75d-43f7-b19a-47640048a3f0","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:24:57.034314Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}