{"claim":{"id":"a501564c-3e36-4a53-8532-a152e0491e87","submission_id":"b908e383-9995-44a6-b43d-bf32fbc69357","submission_revision_id":"ee069d86-842a-4fc4-83ad-128e90f85eb6","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:36:52.652200Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"b908e383-9995-44a6-b43d-bf32fbc69357","submission_revision_id":"ee069d86-842a-4fc4-83ad-128e90f85eb6","revision_version":2,"submission_status":"under_review","submitted_at":"2026-10-03T21:03:40.809619Z","submitter_name":"Xin Wang","submitter_email":"xinwang.ca@berkeley.edu","task":{"status":"submitted","version":2,"extra_fields":{"ai_draft":{"source":"submissions_workbench","status":"ready","preview":{"domain":"Organic Chemistry","task_goal":"Locate and verify the transition state connecting reactant and product for a proposed organic reaction step, then confirm connectivity via IRC.\n\nStarting state: Raw 3D reactant and product geometries plus a proposed reaction scheme and calculation spec are provided; no transition state guess exists yet.\n\nInstructions and rules: Build a reasonable TS guess (e.g., interpolation or manual distortion) from the reactant/product geometries, optimize to a first-order saddle point at the specified DFT level, confirm exactly one imaginary frequency corresponding to the expected bond-forming/breaking motion, and run IRC in both directions to confirm it connects to the given reactant and product minima. Report energetics (activation and reaction energy) and imaginary mode character in a single case-keyed table.","short_name":"ts_irc_organic_reaction","input_files":"- reactant.xyz (XYZ, <5KB): Starting 3D geometry of the reactant minimum, defines one IRC endpoint target\n- product.xyz (XYZ, <5KB): Starting 3D geometry of the product minimum, defines the other IRC endpoint target\n- calc_spec.tsv (TSV, <2KB): DFT functional, basis set, charge, multiplicity, and solvent model to use for optimization and frequency jobs","output_files":"- key_metrics.tsv (TSV) [graded]: Case-keyed table with TS imaginary frequency (cm-1), activation energy (kcal/mol), reaction energy (kcal/mol), and IRC endpoint RMSD to reactant/product\n- ts_optimized.xyz (XYZ): Final optimized saddle-point geometry used to derive the reported energetics and frequency","operating_system":"Windows, Linux, MacOS","software_licensing":"Free/Open Source","verification_method":"key_metrics.tsv: parse/schema pass; required fields = 100%; records > 0.\nkey_metrics.tsv: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nts_optimized.xyz: cross-file totals and joins reconcile with key_metrics.tsv; mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (TSV; declared constraints).","evaluation_criterion":"Method: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_and_version":"ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2","tools_software_details":"Software: ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2\nGenerate TS guess and run frequency-verified saddle-point optimization (Tools: ORCA, Avogadro)\nRun IRC in both directions and confirm endpoint connectivity to reactant/product minima (Tools: ORCA, RDKit)"},"warnings":["estimated_expert_hours \"\" does not meet the ≥5 hour requirement"],"createdAt":1791055222934,"formPrefill":{"extra_fields":{},"industry_domain":"Organic Chemistry","task_short_name":"ts_irc_organic_reaction","operating_system":"Windows, Linux, MacOS","task_description":"# Task Goal\n\nLocate and verify the transition state connecting reactant and product for a proposed organic reaction step, then confirm connectivity via IRC.\n\nStarting state: Raw 3D reactant and product geometries plus a proposed reaction scheme and calculation spec are provided; no transition state guess exists yet.\n\nInstructions and rules: Build a reasonable TS guess (e.g., interpolation or manual distortion) from the reactant/product geometries, optimize to a first-order saddle point at the specified DFT level, confirm exactly one imaginary frequency corresponding to the expected bond-forming/breaking motion, and run IRC in both directions to confirm it connects to the given reactant and product minima. Report energetics (activation and reaction energy) and imaginary mode character in a single case-keyed table.\n\n# Input Files\n\n- reactant.xyz (XYZ, <5KB): Starting 3D geometry of the reactant minimum, defines one IRC endpoint target\n- product.xyz (XYZ, <5KB): Starting 3D geometry of the product minimum, defines the other IRC endpoint target\n- calc_spec.tsv (TSV, <2KB): DFT functional, basis set, charge, multiplicity, and solvent model to use for optimization and frequency jobs\n\n# Output Files\n\n- key_metrics.tsv (TSV) [graded]: Case-keyed table with TS imaginary frequency (cm-1), activation energy (kcal/mol), reaction energy (kcal/mol), and IRC endpoint RMSD to reactant/product\n- ts_optimized.xyz (XYZ): Final optimized saddle-point geometry used to derive the reported energetics and frequency\n\n# Tools / Software Details\n\nSoftware: ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2\nGenerate TS guess and run frequency-verified saddle-point optimization (Tools: ORCA, Avogadro)\nRun IRC in both directions and confirm endpoint connectivity to reactant/product minima (Tools: ORCA, RDKit)\n\n# Evaluation Criterion\n\nMethod: script + rubric review\nMetric: primary reference-output comparison over structure, key values, consistency, and evidence grounding\nThreshold: pass when the primary reference output has no critical mismatch and supporting artifacts show no blocking inconsistency","software_licensing":"Free/Open Source","verification_method":"key_metrics.tsv: parse/schema pass; required fields = 100%; records > 0.\nkey_metrics.tsv: exact match for ids/labels; numeric delta <= 5% or domain tolerance; blocking mismatches = 0.\nts_optimized.xyz: cross-file totals and joins reconcile with key_metrics.tsv; mismatch count = 0.\nOverall: machine-readable = true; constraint violations = 0; formats remain comparable (TSV; declared constraints).","software_and_version":"ORCA 5.0.4, RDKit 2023.09, Avogadro 1.2"},"draftSession":"e450a817-4dba-4a03-9688-9ec9c7179b06"}},"submitted_at":"2026-10-03T21:03:40.809619+00:00","industry_domain":"Organic Chemistry","task_short_name":"organic_reasoning_instance_1","operating_system":"Windows, Linux, MacOS","task_description":"# Task Goal\n\nIdentify the reactant and product of an organic reaction, then convert them to Daylight SMILES strings using Ketcher software.\n\nStarting state: an image of a chemical reaction with reaction conditions specified in a table.\n\nInstructions and rules: Build the molecules from the image; confirm there are no errors, such as wrong labels, unrecognized labels, or layout mistakes; export the reactant and product as MDL Molfile V2000.\n\n# Input Files\n\n- reaction_optimization.png: Figure panel obtained from publication.\n\n# Output Files\n\n- reactant.smi: reactant in Daylight SMILES format.\n- product.smi: product in Daylight SMILES format.\n- reactant.mol: reactant in MDL Molfile V2000 format.\n- product.mol: product in MDL Molfile V2000 format.\n\n# Tools / Software Details\n\nSoftware: Ketcher v3.18.0\n\n# Evaluation Criterion\n\nMethod: script.\nMetric: primary reference-output structure comparison.\nThreshold: pass when the primary reference output has no critical mismatches.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Open the output files as text.  \nRequire structural equivalence across all four outputs. Require correct atom identities, connectivity, bond orders, charges, and product stereochemistry; accept equivalent SMILES notation and harmless Molfile coordinate or atom-order differences. Require V2000 compliance and agreement between each molecule’s two exports.","software_and_version":"Ketcher v3.18.0","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"ff1e6192-04f0-4239-bfca-05f9a5cb3382","filename":"reaction_optimization.png","category":"input_materials","content_type":"image/png","size_bytes":492540},{"file_id":"9690f15a-b7b5-453a-b289-8d7589bd25b3","filename":"reactant.mol","category":"reference_output","content_type":"chemical/x-mdl-molfile","size_bytes":1964},{"file_id":"fdee527c-5e72-4b38-b696-07fa7e4c6f85","filename":"reactant.smi","category":"reference_output","content_type":"application/octet-stream","size_bytes":40},{"file_id":"aedf9132-a7ef-48d0-9da1-3fa98ba171da","filename":"product.mol","category":"reference_output","content_type":"chemical/x-mdl-molfile","size_bytes":1964},{"file_id":"a9b04c3b-bde5-4ed7-9830-d26101d276ac","filename":"product.smi","category":"reference_output","content_type":"application/octet-stream","size_bytes":42}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The submission is complete, reachable, and internally consistent, with references that match the depicted reaction and support objective structural evaluation.\n\n**Completeness Check:**\n\n- Instruction: Complete — Clearly specifies molecule reconstruction in Ketcher and four required exports.\n- Inputs: Complete — The readable reaction image identifies both structures, substituents, chain lengths, and product alkene geometry.\n- Toolchain: Complete — Ketcher is an open-source molecular editor suitable for the requested SMILES and V2000 workflow.\n- Reference output: Complete — All four references are present; each SMILES agrees with its corresponding Molfile, including the product’s E configuration.\n- Rubric: Complete — Specifies atom identities, connectivity, bond orders, charges, stereochemistry, V2000 compliance, and cross-format agreement while accepting equivalent representations.\n\n**File and Task Reachability:**\n\nAttachment classification: `reachable`. Task classification: `reachable`.\n\nInspected every attachment: `reaction_optimization.png`, `reactant.mol`, `reactant.smi`, `product.mol`, and `product.smi`. All byte counts match the manifest. The image is a valid 1150 × 950 RGBA PNG with intact chunks and readable content. The four reference files are ASCII text; both Molfiles have consistent V2000 atom and bond records, each containing 20 atoms and 21 bonds. Structural comparison found matching connectivity and bond orders within each export pair. Product Molfile geometry agrees with the E stereochemistry in its SMILES and the image.\n\nNo attachment content remains unreviewed. A Ketcher GUI round-trip was not tested. Completion requires reconstructing the two depicted molecules, exporting both formats, and comparing parsed structures against evaluator-only references on a single ordinary machine. The manifest separates input material from reference outputs.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. The stated publication origin does not itself establish a public benchmark. Public provenance and answer retrievability could not be independently checked because approved search was unavailable.\n\n**Recommended Changes:**\n\nThe readable input, consistent references, and representation-independent verification criteria provide a usable task. No changes are required.","reviewed_at":"2026-10-03T21:17:20.793736Z"},"claim":{"id":"a501564c-3e36-4a53-8532-a152e0491e87","submission_id":"b908e383-9995-44a6-b43d-bf32fbc69357","submission_revision_id":"ee069d86-842a-4fc4-83ad-128e90f85eb6","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:36:52.652200Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}