{"submission_id":"ce296098-8cbc-4346-9415-3bef26d9ecbe","submission_revision_id":"67e2d810-85ac-496e-8697-4563e8639fac","revision_version":2,"submission_status":"under_review","submitted_at":"2026-10-03T21:44:16.172527Z","submitter_name":"Xin Wang","submitter_email":"xinwang.ca@berkeley.edu","task":{"status":"submitted","version":2,"extra_fields":{"ai_draft":{"source":"submissions_workbench","status":"ready","preview":{"domain":"Organic Chemistry","task_goal":"Extract molecular information from a publication figure.\n\nStarting state: A full reaction figure as an image.\n\nInstructions and rules: Locate relevant information for the two molecules with indices 5m and 5n. Convert the structures into SMILES strings.","short_name":"organic_reasoning_instance_3","input_files":"- molecule_01.sdf (SDF, ~2 KB each (8 files)): raw 3D starting geometry per candidate molecule, input for conformer search\n- charges_multiplicities.tsv (TSV, <1 KB): per-molecule net charge and spin multiplicity required to configure each ORCA input correctly\n- calc_conditions.tsv (TSV, <1 KB): per-molecule DFT functional, basis set, and implicit solvent model assignment controlling the optimization level","output_files":"- key_metrics.tsv (TSV) [graded]: per-molecule final SCF energy, heavy-atom RMSD to a reference minimum geometry, lowest vibrational frequency, and imaginary-frequency count confirming true minima\n- quantum_properties.tsv (TSV) [graded]: per-molecule dipole moment, HOMO/LUMO energies, and HOMO-LUMO gap extracted from the converged DFT output for ranking","operating_system":"Windows, Linux, MacOS","software_licensing":"Free/Open Source","verification_method":"key_metrics.tsv: parse/schema pass; required fields = 100%; records > 0.\nkey_metrics.tsv: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nquantum_properties.tsv: cross-file totals and joins reconcile with key_metrics.tsv; mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (TSV, TSV; declared constraints).","evaluation_criterion":"Method: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_and_version":"ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2","tools_software_details":"Software: ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2\nConformer generation, deduplication, and HF preoptimization screening (Tools: RDKit, ORCA)\nDFT optimization, frequency verification, and descriptor extraction (Tools: ORCA)"},"warnings":["estimated_expert_hours \"\" does not meet the ≥5 hour requirement"],"createdAt":1791062150197,"formPrefill":{"extra_fields":{},"industry_domain":"Organic Chemistry","task_short_name":"organic_reasoning_instance_3","operating_system":"Windows, Linux, MacOS","task_description":"# Task Goal\n\nExtract molecular information from a publication figure.\n\nStarting state: A full reaction figure as an image.\n\nInstructions and rules: Locate relevant information for the two molecules with indices 5m and 5n. Convert the structures into SMILES strings.\n\n# Input Files\n\n- molecule_01.sdf (SDF, ~2 KB each (8 files)): raw 3D starting geometry per candidate molecule, input for conformer search\n- charges_multiplicities.tsv (TSV, <1 KB): per-molecule net charge and spin multiplicity required to configure each ORCA input correctly\n- calc_conditions.tsv (TSV, <1 KB): per-molecule DFT functional, basis set, and implicit solvent model assignment controlling the optimization level\n\n# Output Files\n\n- key_metrics.tsv (TSV) [graded]: per-molecule final SCF energy, heavy-atom RMSD to a reference minimum geometry, lowest vibrational frequency, and imaginary-frequency count confirming true minima\n- quantum_properties.tsv (TSV) [graded]: per-molecule dipole moment, HOMO/LUMO energies, and HOMO-LUMO gap extracted from the converged DFT output for ranking\n\n# Tools / Software Details\n\nSoftware: ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2\nConformer generation, deduplication, and HF preoptimization screening (Tools: RDKit, ORCA)\nDFT optimization, frequency verification, and descriptor extraction (Tools: ORCA)\n\n# Evaluation Criterion\n\nMethod: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_licensing":"Free/Open Source","verification_method":"key_metrics.tsv: parse/schema pass; required fields = 100%; records > 0.\nkey_metrics.tsv: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nquantum_properties.tsv: cross-file totals and joins reconcile with key_metrics.tsv; mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (TSV, TSV; declared constraints).","software_and_version":"ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2"},"draftSession":"a0af562a-3308-411c-9eec-f5a4b267347a"}},"submitted_at":"2026-10-03T21:44:16.172527+00:00","industry_domain":"Organic Chemistry","task_short_name":"organic_reasoning_instance_3","operating_system":"Windows, Linux, MacOS","task_description":"# Task Goal\n\nExtract molecular information from a publication figure.\n\nStarting state: A full reaction figure as an image.\n\nInstructions and rules: Locate relevant information for the two molecules with indices 5m and 5n. Convert the structures into SMILES strings.\n\n# Input Files\n\n- extraction.png: A full reaction figure as an image that contains the information for molecules 5m and 5n.\n\n# Output Files\n\n- extraction.smi [graded]: Daylight SMILES string of the two molecules\n- extract.mol: MDL Molfile V2000 format of the two molecules.\n\n# Tools / Software Details\n\nSoftware: Ketcher v3.18.0\n\n# Evaluation Criterion\n\nMethod: script + rubric review\nMetric: primary reference-output comparison over structure.\nThreshold: pass when the primary reference output has no critical mismatches.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Require structural equivalence across the two outputs. Require correct atom identities, connectivity, bond orders, charges, and product stereochemistry; accept equivalent SMILES notation and harmless Molfile coordinate or atom-order differences. Require V2000 compliance and agreement between each molecule’s two exports.","software_and_version":"Ketcher v3.18.0","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"3ef28f94-107d-45ea-adfb-92a8eabc8f00","filename":"extract.mol","category":"reference_output","content_type":"chemical/x-mdl-molfile","size_bytes":5570},{"file_id":"861b52ad-2eee-4fea-a849-9225ef3d3f74","filename":"extraction.smi","category":"reference_output","content_type":"application/octet-stream","size_bytes":121},{"file_id":"505ce483-ac3c-4f58-ac15-185e7be17fac","filename":"extraction.png","category":"input_materials","content_type":"image/png","size_bytes":569883}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The task is complete and reachable, with readable inputs and structurally consistent references matching both requested products.\n\n**Completeness Check:**\n\n- Instruction: Complete — Identifies 5m and 5n and specifies both required export formats.\n- Inputs: Complete — The reaction figure clearly supplies the shared scaffold, target labels, and substituent definitions.\n- Toolchain: Complete — Ketcher is an open-source editor suitable for constructing and exporting these structures.\n- Reference output: Complete — Both exports contain the correct two molecular components and match each other in atom identities, connectivity, bond orders, and charges. Unspecified absolute stereochemistry is consistent with the figure’s racemic notation.\n- Rubric: Complete — Structural equivalence, V2000 compliance, cross-export agreement, and the critical-mismatch threshold provide objective evaluation criteria.\n\n**File and Task Reachability:**\n\nAttachments: `reachable`. Task: `reachable`.\n\nAll three attachments were fully inspected: extraction.png is a valid 1415 × 1577 RGBA PNG; extraction.smi is ASCII containing two dot-separated structures; extract.mol is an ASCII V2000 record containing 58 atoms, 64 bonds, and two disconnected components. Downloaded sizes match the manifest, and both reference components match their corresponding SMILES graphs and the figure. No attachment content remains unreviewed.\n\nA solver can inspect the figure, construct both products, export the requested files, and undergo structure-based comparison without exact text or coordinate matching. References are categorized separately from solver inputs. Ketcher execution and an independent chemistry-toolkit round trip were not performed; these are review limitations, not demonstrated submission defects.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. No affirmative evidence supports a screening label. Public benchmark or answer availability could not be independently checked because approved search was unavailable.\n\n**Recommended Changes:**\n\nThe submission provides usable chemical evidence, matching references, and appropriate structural evaluation criteria. No changes are required.","reviewed_at":"2026-10-03T21:48:28.655467Z"},"claim":null}