{"submission_id":"08e959a7-4bb4-47e7-bf3f-371d43847a7b","submission_revision_id":"999573e4-2cd2-40fa-a3c5-47a07d8b9971","revision_version":2,"submission_status":"under_review","submitted_at":"2026-10-05T20:06:49.473225Z","submitter_name":"Yunhao Luo","submitter_email":"yunhaoluo@ucsb.edu","task":{"status":"submitted","version":2,"extra_fields":{"difficulty_self_test":{"model":"GPT-6-Sol","harness":"Codex","estimated_score":"20–40%"}},"submitted_at":"2026-10-05T20:06:49.473225+00:00","industry_domain":"Graphical Design","task_short_name":"raster_to_svg_1","operating_system":"Windows, Linux, MacOS","task_description":"Objective: Recreate a raster graphic as a visually faithful, editable, and well-organized SVG.\n\nStarting State: A raster reference image (`image.png`) containing structured visual elements such as text, shapes, illustrations, icons, and decorative components.\n\nInstructions & Rules: Recreate the reference image as an SVG (`image.svg`) so that, when rendered, it closely matches the original image. Preserve meaningful elements as independently editable SVG objects and organize them into logical groups that support efficient selection and editing. Text should remain editable where practical. Do not simply embed the original raster image as a full-canvas image inside the SVG. The final SVG should be valid, render correctly, and maintain a useful object hierarchy for common editing operations.","software_licensing":"Commercial","evaluation_criteria":null,"verification_method":"File validity\n- The submission must be a valid, renderable SVG.\n- The SVG must preserve the reference image’s 4:3 aspect ratio and render at exactly 2048 × 1536.\n- The original photograph may not be embedded or referenced in the SVG using an `<image>` element, data URI, external raster reference, CSS background image, or equivalent raster-image embedding.\n- Failure to satisfy any file-validity requirement results in overall failure.\n\nVisual fidelity\n- Rasterize the submitted SVG at exactly 2048 × 1536 using the same fixed renderer for all submissions.\n- Resize the reference photograph to 2048 × 1536 using Lanczos resampling before comparison.\n- Compare the rendered SVG with the resized reference using the following fixed similarity score:\n\n  `RegionScore = 0.70 × SSIM + 0.30 × (1 − NMAE)`\n\n  where:\n  - `SSIM` is structural similarity computed with a data range of 255.\n  - `NMAE` is the mean absolute RGB pixel error divided by 255.\n  - Both values are normalized to the range `[0, 1]`.\n\n- Evaluate similarity over the full image and over the following five fixed reference regions:\n  - the wooden bridge,\n  - the foreground trail,\n  - the foreground/right-side rocks,\n  - the surrounding trees and vegetation,\n  - the visible sky/background.\n\n- Each region must be defined by a fixed binary mask created from the reference image and reused unchanged for every submission.\n- Compute the final visual score as:\n\n  `VisualScore = 0.50 × FullImageScore + 0.15 × BridgeScore + 0.10 × TrailScore + 0.10 × RocksScore + 0.10 × VegetationScore + 0.05 × SkyScore`\n\n- The bridge, trail, rocks, vegetation, and sky must remain in approximately the same location, scale, orientation, and spatial relationship as in the reference.\n- A submission that achieves similar overall colors but substantially omits, relocates, or distorts a major scene component should receive a low score through the corresponding region comparison.\n\nVector reconstruction\n- Visible content must be recreated using SVG vector primitives such as paths, polygons, rectangles, ellipses, strokes, gradients, masks, clipping paths, and groups.\n- A submission consisting primarily of one raster image, one full-scene traced object, or another effectively flattened representation does not satisfy the editability requirement.\n- The following scene components must remain independently selectable:\n  - complete bridge,\n  - bridge railings,\n  - bridge deck,\n  - foreground trail,\n  - foreground/right-side rocks,\n  - sky,\n  - left-side forest/vegetation,\n  - right-side/background forest/vegetation.\n\nOrganization and editability\n- The SVG hierarchy will be evaluated against the following exact selection targets:\n  1. complete bridge,\n  2. bridge railings,\n  3. bridge deck,\n  4. foreground trail,\n  5. foreground/right-side rocks,\n  6. sky,\n  7. left-side forest/vegetation,\n  8. right-side/background forest/vegetation.\n\n- Each target is evaluated independently from an initially collapsed Layers/Object hierarchy.\n- A hierarchy interaction is counted as:\n  - expanding one group: 1 interaction,\n  - selecting one object or group: 1 interaction,\n  - adding another object or group to the current selection: 1 additional interaction.\n- Scrolling and pointer movement do not count.\n- Selecting a parent group counts as a valid single selection only when that group contains all and only the visible elements belonging to the requested target.\n- If the group contains unrelated visible elements, it cannot be used as a valid single-selection solution for that target.\n- For each target, record the minimum number of hierarchy interactions needed to select all and only the requested elements.\n- Reset the hierarchy to the fully collapsed state before evaluating the next target.\n- Compute:\n\n  `SelectionCost = sum of the interaction counts for all 8 targets`\n\n- Excessive unnecessary nesting, placing unrelated components in the same group, or failing to group elements that naturally belong together increases the selection cost.\n\nOutcome criteria\n- **Success**\n  - all file-validity requirements pass,\n  - `VisualScore ≥ 0.60`,\n  - each of the bridge, trail, rocks, and vegetation region scores is at least `0.40`,\n  - `SelectionCost ≤ 14`.\n\n- **Partial success**\n  - all file-validity requirements pass,\n  - `VisualScore ≥ 0.45`,\n  - each of the bridge, trail, rocks, and vegetation region scores is at least `0.30`,\n  - `SelectionCost ≤ 22`,\n  - but one or more Success thresholds are not met.\n\n- **Failure**\n  - any file-validity requirement fails, or\n  - `VisualScore < 0.45`, or\n  - any required major-region score is below `0.30`, or\n  - `SelectionCost > 22`.\n\nA successful submission must therefore both closely reproduce the appearance of the original photograph according to the fixed visual-scoring procedure and preserve a vector hierarchy that allows the specified scene components to be selected and edited efficiently.","software_and_version":"Adobe Illustrator 2026","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"c2a9ebbd-07ed-402a-8a99-472050252bfc","filename":"IMG_5194.JPG","category":"input_materials","content_type":"image/jpeg","size_bytes":5921980},{"file_id":"328ed09c-f23c-49f2-bb3e-b0928dd7ae80","filename":"Screenshot 2026-09-29 at 1.08.44 PM.png","category":"difficulty_self_test_evidence","content_type":"image/png","size_bytes":1256343},{"file_id":"96fc5153-b734-450a-8ecc-f28f1b990f4d","filename":"Screenshot 2026-09-29 at 1.08.54 PM.png","category":"difficulty_self_test_evidence","content_type":"image/png","size_bytes":264316}],"deep_review":{"overall_verdict":"Borderline Accept","review_text":"**Overall Verdict:** Borderline Accept — the task is implementable, but the visual score cannot be reproduced consistently until the renderer and region masks are fixed.\n\n**Completeness Check:**\n- Instruction: Complete — specifies the SVG deliverable, editable scene components, and hierarchy targets.\n- Inputs: Complete — the reference photo is readable and supports the task; its actual 4032 × 3024 dimensions are 4:3, as required.\n- Toolchain: Complete — Illustrator 2026 is specified; a suitable Windows or Mac machine can run it, and the SVG output is compatible with vector workflows.\n- Reference output: Complete — the required `image.svg`, dimensions, and content constraints are stated.\n- Rubric: Unclear — the scoring formula and thresholds are detailed, but no renderer/version or fixed binary region masks are supplied or precisely defined.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. I inspected all three manifest files: `IMG_5194.JPG` (JPEG, 4032 × 3024) and both named PNG difficulty-test screenshots (2848 × 1640 and 1021 × 1893). Their actual byte counts match the manifest, and the files are readable. The screenshots show a vector reconstruction attempt and a later claim of an SVG deliverable, but do not include that SVG, so they do not independently establish its validity or fidelity. No attachment content was left unreviewed.\n\nTask classification: `conditionally_reachable` until the visual evaluation is made reproducible. The task otherwise has a workable path from opening the supplied photo in a vector editor, creating the SVG, and checking its validity, rendered appearance, and object hierarchy. The listed Adobe software is not natively available on Linux, but a Windows or Mac machine is sufficient. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone.\n\n**Recommended Changes:**\nThe reference and requested SVG workflow are clear. Before evaluation, specify the fixed renderer and its version/configuration, and provide the five binary masks or an unambiguous reproducible method for defining them. Consider removing GPS metadata from the solver-visible photo if its location disclosure is unintended.","reviewed_at":"2026-10-06T16:20:57.921124Z"},"claim":null}