{"submission_id":"237770b3-9fdd-4ce1-8b45-b574360f521a","submission_revision_id":"b698a040-f9d0-49d7-bd90-3fdebe55a80e","revision_version":7,"submission_status":"submitted","submitted_at":"2026-07-02T02:05:35.315568Z","submitter_name":"Yukun Huang","submitter_email":"kevinhuangyk16@gmail.com","task":{"status":"submitted","version":7,"extra_fields":{},"submitted_at":"2026-07-02T02:05:35.315568+00:00","industry_domain":"Architecture","task_short_name":"cad_design_reproduce","operating_system":"MacOS","task_description":"## Task Goal\n\nGiven a CAD layout image in PNG format, reproduce the same architectural layout in AutoCAD and generate an editable DWG file. The model should use the input PNG as the visible target drawing and reconstruct the same floor plan as an editable CAD file.\n\nThe goal is to faithfully reproduce the main structural layout, dimensions, labels, and key architectural elements shown in the input image, so that the generated DWG represents the same design as the target floor plan.\n\n## Input Files\n\nThe input is a PNG image showing a 2D architectural floor-plan drawing. The specific input layout contains four labeled spaces: LIVING ROOM, KITCHEN, BEDROOM, and BATH.\n\nThe plan is approximately rectangular, with an overall horizontal dimension of 8000 and an overall vertical dimension of 5000. The top horizontal dimension is divided into 2700, 2600, and 2700, while the bottom horizontal dimension is divided into 5300 and 2700. The vertical dimensions include a left-side split of 2800 and 2200, and a right-side split of 1500 and 3500.\n\nThe drawing includes grid/axis markers labeled 1, 2, 3, 4 horizontally and A, B, C, D vertically. It also contains exterior walls, interior partitions, room labels, dimension lines, door openings with swing arcs, and window symbols. Windows appear along the top exterior wall for the living room, kitchen, and bedroom, along the bottom exterior wall for the living room and central area, and on the right exterior wall near the bath. Door openings connect the central area to the bedroom and bath, and another door opening appears along the bottom side of the plan.\n\nThe submitted DWG should reconstruct these visible architectural and drafting elements, including the room layout, wall topology, dimensions, grid labels, openings, window placements, room labels, and overall spatial arrangement.\n\n## Output Files\n\nThe required output is a reconstructed, editable DWG file that matches the layout shown in the input PNG. The DWG should contain editable CAD geometry rather than a simple embedded raster image.\n\nOptionally, the DWG may also be exported to DXF or PDF for inspection. For evaluation, the submitted DWG must be convertible into a PNG image using the provided dwg_to_png.py rendering script.\n\n## Tools / Software Details\n\nThe task should be completed using AutoCAD or compatible CAD software capable of creating and editing DWG files.\n\nThe evaluation environment should have Python 3 installed. Required Python packages should be installed with:\n\nbash python3 -m pip install -r requirements.txt \n\n\n## Reference and Evaluation Clarification\n\nThe input PNG is the target image provided to the model. The model must use this PNG to create the submitted DWG.\n\nFor evaluation, there is also a hidden reference DWG corresponding to the same layout. The reference DWG is not part of the model input. It is used only as the ground-truth CAD file during evaluation.\n\nEvaluation compares the submitted DWG against the hidden reference DWG after both files are rendered into PNG images. The comparison is therefore between:\n\nbash reference.dwg  ->  reference.png submission.dwg ->  submission.png \n\nrather than directly between the original input PNG and the submitted rendering.\n\n## Rendering Step\n\nBoth the hidden reference DWG and the submitted DWG should be converted to PNG using the provided dwg_to_png.py script:\n\nbash python3 dwg_to_png.py reference.dwg reference.png python3 dwg_to_png.py submission.dwg submission.png \n\nThe submitted DWG fails automatically if the rendering command exits with an error, crashes, or produces a blank, empty, or invalid PNG image.\n\n## Evaluation Criterion\n\nEvaluation will be performed automatically. After rendering both DWG files into PNG images, GPT-5.4 will compare reference.png and submission.png as a visual judge. The comparison should focus on CAD layout fidelity rather than exact pixel-level matching.\n\nThe visual judge should evaluate the following criteria:\n\n1. File validity and renderability: The submitted DWG must successfully render into a non-empty PNG using dwg_to_png.py without errors.\n\n2. Overall boundary and dimensions: The exterior boundary, orientation, proportions, and major visible dimension values should match the reference rendering. Important dimensions such as the overall width, overall height, and major segment lengths should be preserved.\n\n3. Grid lines, axes, and annotations: Grid markers, axis labels, dimension lines, room names, and other important drafting annotations should be present, readable, and placed in the correct regions.\n\n4. Walls, rooms, and spatial topology: Exterior walls, interior partitions, room boundaries, and major spaces should be reconstructed accurately. The number of rooms, relative room sizes, room adjacency, and circulation/connectivity should match the reference layout.\n\n5. Openings and architectural elements: Doors, windows, entrances, and other visible architectural symbols should appear in the correct locations and on the correct walls. The exact drafting style may differ, but the functional placement should be consistent with the reference.\n\n6. Overall visual and semantic fidelity: The rendered submission should clearly depict the same CAD design as the rendered reference. Minor differences in line thickness, font style, color, or small decorative details are acceptable, but major omissions, incorrect geometry, missing key annotations, altered dimensions, or changed layout topology should be treated as errors.\n\nA submission is successful if it passes the renderability check and GPT-5.4 determines that submission.png represents the same CAD layout as reference.png under the criteria above.","software_licensing":"Commercial","evaluation_criteria":null,"verification_method":"## Evaluation\n\nEvaluation will be performed automatically by rendering both the reference DWG and the submitted DWG into PNG images, then using GPT-5.4 (released by OpenAI after March 5, 2026; stronger models than that like gpt-5.5 is also acceptable) as the visual judge to compare the two rendered images.\n\n### Environment Setup\n\nThe evaluation environment should have Python 3 installed. Install the required Python packages with:\n\nbash python3 -m pip install -r requirements.txt \n\nThe requirements.txt file should include the Python dependencies needed by dwg_to_png.py. \n\n### Rendering Step\n\nBoth the reference DWG and the submitted DWG should be converted to PNG using the provided dwg_to_png.py script:\n\nbash python3 dwg_to_png.py reference.dwg reference.png python3 dwg_to_png.py submission.dwg submission.png \n\nThe submitted DWG fails automatically if the rendering command exits with an error, crashes, or produces an empty/invalid PNG image.\n\n### GPT-5.4 Visual Judgment\n\nAfter rendering, GPT-5.4 will compare reference.png and submission.png. The judgment should not require pixel-perfect matching. Minor differences in line thickness, font style, color, or rendering artifacts are acceptable. The focus should be whether the submitted DWG represents the same CAD layout as the reference DWG.\n\nThe visual judge should evaluate the following criteria:\n\n1. Overall boundary and dimensions: The exterior boundary, orientation, proportions, and visible major dimension values should match the reference.\n\n2. Grid lines, axes, and annotations: Important grid markers, axis labels, dimension lines, room labels, and drafting annotations should be present, readable, and placed in the correct regions.\n\n3. Walls, rooms, and spatial topology: Exterior walls, interior partitions, room boundaries, corridors, and major spaces should be reproduced accurately. The number of major rooms, their relative sizes, adjacency, and connectivity should match the reference.\n\n4. Openings and architectural elements: Doors, windows, stairs, entrances, and other major architectural elements should appear in the correct locations and on the correct walls. The exact CAD symbol style may differ, but the functional placement should be correct.\n\n5. Overall visual and semantic fidelity: The rendered submission should clearly depict the same CAD design as the rendered reference. Major omissions, incorrect geometry, missing key annotations, altered dimensions, or changed layout topology should be treated as errors.\n\nA submission is successful if it passes the renderability check and GPT-5.4 determines that submission.png represents the same CAD layout as reference.png under the criteria above.\n\nThe prompt for the judge is below:\n\nYou are an automatic visual judge for a CAD layout reproduction task.\n\nYou will be given two PNG images:\n\n- reference_png: the PNG rendering of the reference DWG file.\n- submission_png: the PNG rendering of the submitted DWG file.\n\nBoth images were generated using the same dwg_to_png.py rendering script. Your task is to determine whether the submitted DWG faithfully reproduces the reference CAD layout.\n\nDo not require pixel-perfect matching. Minor differences in line thickness, font style, color, or small decorative details are acceptable. However, the submitted image must represent the same CAD design and preserve the important geometry, dimensions, labels, and spatial structure.\n\nEvaluate the submission using the following rubric:\n\n1. Overall boundary and dimensions\n   - Check whether the exterior boundary shape, orientation, and overall proportions match the reference.\n   - Check whether visible major dimension values and dimension segments are reproduced correctly.\n   - Penalize missing, incorrect, or substantially misplaced dimension labels.\n\n2. Grid lines, axes, and annotations\n   - Check whether important grid markers, axis labels, dimension lines, room names, and drafting annotations are present.\n   - The exact text style does not need to match, but labels should be readable and located in the correct areas.\n   - Penalize missing major labels, incorrect labels, or labels placed in clearly wrong locations.\n\n3. Walls, rooms, and spatial topology\n   - Check whether exterior walls, interior walls, partitions, room boundaries, and corridors match the reference.\n   - Check whether the number of major rooms or spaces is correct.\n   - Check whether room adjacency, relative room sizes, and connectivity are preserved.\n   - Penalize missing walls, extra walls, changed room topology, or substantially incorrect proportions.\n\n4. Openings and architectural elements\n   - Check whether doors, windows, stairs, entrances, and other major architectural symbols appear in the correct locations and on the correct walls.\n   - The exact CAD block style may differ, but the functional placement should match the reference.\n   - Penalize missing openings, openings on the wrong wall, incorrect swing direction when clearly important, or misplaced architectural elements.\n\n5. Overall visual and semantic fidelity\n   - Judge whether the two images clearly depict the same CAD layout.\n   - Minor rendering differences are acceptable.\n   - Major omissions, incorrect geometry, missing key annotations, or altered layout meaning should cause failure.\n\nReturn your judgment in the following format:\n\njson {   \"pass\": true or false,   \"overall_score\": 0-10,   \"criteria_scores\": {     \"overall_boundary_and_dimensions\": 0-2,     \"grid_axes_and_annotations\": 0-2,     \"walls_rooms_and_spatial_topology\": 0-3,     \"openings_and_architectural_elements\": 0-2,     \"overall_visual_and_semantic_fidelity\": 0-1   },   \"major_errors\": [     \"List the most important problems, if any.\"   ],   \"minor_errors\": [     \"List minor differences that do not necessarily cause failure.\"   ],   \"rationale\": \"Briefly explain whether the submitted image represents the same CAD layout as the reference.\" } \n\nUse the following passing rule:\n\n- The submission should pass only if overall_score >= 7 and there are no major structural errors.\n- A major structural error includes an invalid exterior boundary, missing major room, incorrect room topology, missing major wall, missing major door/window/opening, or clearly incorrect major dimensions.\n- If the submission is visually similar but misses important dimensions, grid labels, or room labels, reduce the score accordingly and decide whether the missing information changes the meaning of the CAD layout.","software_and_version":"AutoCAD","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"1513ce7b-cb16-4257-8711-55ab8531a10d","filename":"input.png","category":"input_materials","content_type":"image/png","size_bytes":178231},{"file_id":"60e9a497-b709-4244-aeb5-2f2831dbf310","filename":"output.dwg","category":"reference_output","content_type":"application/octet-stream","size_bytes":44684},{"file_id":"df00f6f2-a195-4afc-b3e7-acb0441353d9","filename":"dwg_to_png.py","category":"reference_output","content_type":"text/x-python-script","size_bytes":3073},{"file_id":"47d07c20-37c0-4e78-8274-a453f001717e","filename":"requirements.txt","category":"reference_output","content_type":"text/plain","size_bytes":270}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The task is coherent, professionally scoped, and end-to-end reachable, with only reviewer-tooling uncertainty about the proprietary reference DWG’s model-space contents.\n\n**Completeness Check:**\n- Instruction: Complete — The editable CAD deliverable, required geometry, dimensions, labels, openings, and expected fidelity are clearly defined.\n- Inputs: Complete — `input.png` is a valid 2382×1684 RGBA PNG with valid chunk checksums and no trailing data; its visible plan agrees with the written specification.\n- Toolchain: Complete — AutoCAD or compatible CAD software is allowed, and the renderer’s Python and ODA File Converter dependencies are disclosed.\n- Reference output: Unclear — `output.dwg` has the expected size and a valid AutoCAD 2013–2017 DWG signature, but its entities, layers, and rendered appearance could not be fully inspected without a compatible local DWG parser.\n- Rubric: Complete — The objective visual criteria, scoring weights, major-error definition, renderability gate, and passing threshold are explicit.\n\n**File and Task Reachability:**\nThe attachment set is **partially_reachable**, while the end-to-end task is **reachable**. All four files were inspected: `input.png`, `output.dwg`, `dwg_to_png.py`, and `requirements.txt`. Actual formats and byte counts match the manifest. The PNG was structurally validated and visually inspected; the renderer and dependency file were reviewed as text without execution. The unreviewed content is the DWG’s detailed entity, layer, and model-space inventory because no compatible non-executing DWG parser or renderer was available in the review environment. This is a reviewer-tooling limitation rather than evidence of a defective submission. The manifest correctly separates the PNG as solver-visible input from the DWG and rendering materials as reference/evaluator assets. A solver can reconstruct the plan in compatible CAD software and submit an editable DWG; an evaluator with the documented ODA and Python dependencies can render both DWGs and apply the supplied visual rubric. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. No affirmative evidence supports a special-screening label. Public-source identifiability was not externally checked because external search is outside the authorized evidence boundary.\n\n**Recommended Changes:**\nThe submission provides a coherent target, a valid editable-output contract, and an objective evaluation path. No contributor changes are required. Evaluation deployment should provision the already-documented ODA File Converter dependency before running the renderer.","reviewed_at":"2026-08-14T23:58:23.531241Z"},"claim":null}