{"submission_id":"e69a2c8e-d914-48b0-a300-4d2eb96ec5a4","submission_revision_id":"f80620f5-c371-4b4d-b7de-9de1d91760cd","revision_version":33,"submission_status":"submitted","submitted_at":"2026-08-21T20:44:13.653675Z","submitter_name":"Chase Broderick","submitter_email":"chaseburnsbroderick@gmail.com","task":{"status":"submitted","version":33,"extra_fields":{"difficulty_self_test":{"model":"Gemini 3.1. Pro Preview","harness":"OpenClaw","estimated_score":">60%"}},"submitted_at":"2026-08-21T20:44:13.653675+00:00","industry_domain":"Data & Analytics Engineering","task_short_name":"shakespearean_spatial_scenes","operating_system":"MacOS","task_description":"### Expected Completion Time\n60-90 minutes\n\n**Objective:** \nEngineer a robust data pipeline to extract spatial and semantic data from all scenes of Shakespeare's *Pericles*. The agent must initialize a relational SQLite database, populate it with parsed character trajectories across macro and micro-locations, and execute a SQL query to export a comprehensive continuity CSV.\n\n**Starting State (Input Materials for the Agent):** \n1. A raw text document containing the complete script of Shakespeare's *Pericles* (`pericles_script.txt`). This file acts as the absolute and ONLY source of truth for all stage directions and dialogue.\n\n**Evaluation Assets (Hidden from the Agent):**\n1. The grading script (`grader_script.py`).\n2. The evaluator's reference output (`pericles_golden_continuity_log.csv`). This is the authoritative \"gold standard\" key the grader will test the agent's output against.\n\n**Instructions & Rules:**\nAs a Data Engineer, process `pericles_script.txt` to map every character's presence and action at both the macro and micro-location levels to build a continuous spatial database.\n\n1. **Database Initialization:** You must initialize a local SQLite database named `pericles_spatial.db`.\n2. **Schema Design:** Create normalized relational tables (e.g., `Characters`, `Locations`, `Appearances`) capable of tracking the Character, Domain (macro-city), Zone (micro-room), Action, Scene Number, and Reasoning/Citation.\n3. **Data Extraction & Insertion:** Use the Python Standard Library to parse the text. Insert this data into your SQLite database. \n    *   **Density Expectation:** Track *distinct character presences per scene*. If a collective group (e.g., \"Sailors\", \"Lords\", \"Gentlemen\", \"Train\", \"Inhabitants\") enters or is addressed in the stage directions or dialogue, they must be tracked as a distinct presence using their exact capitalized noun (e.g., \"LORDS\"). If a singular unnamed character enters (e.g., \"a Lady\", \"a Servant\"), track them using their exact capitalized noun (e.g., \"LADY\", \"SERVANT\").\n    *   **Naming Conventions (Exhaustive List):** Use strict naming conventions for Domains. You must only use domains from this exact, exhaustive list: `['Antioch', 'Tyre', 'Tarsus', 'Pentapolis', 'Ephesus', 'Mytilene', 'Sea']`. If a scene occurs \"off\" a location or \"on shipboard\" near a city, assign it to the closest permitted Domain (e.g., \"Mytilene\" or \"Sea\").\n    *   **Zone Canonicalization:** The Zone is defined as the exact text of the micro-location described in the scene header immediately following the Domain (e.g., \"A room in the palace\"). If the header says \"The same\", inherit the Domain from the previous scene and use the new micro-location as the Zone. If the scene header has no micro-location (e.g., just \"SCENE I:\"), the Zone is \"Unspecified\". For multi-line headers, only use the first sentence describing the location. Strip trailing punctuation.\n    *   **Scene Numbering:** Map every explicit `SCENE` header sequentially to a flat integer format prefixed with \"Scene_\". For example, the very first \"SCENE I.\" maps to `Scene_1`. The final scene maps to `Scene_22`.\n4. **Data Export:** Write and execute a SQL query to join your tables and export the comprehensive dataset into a structured CSV file named `pericles_golden_continuity_log.csv`. ","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"### How should we verify success? (Grading Rubric)\nThe automated script (`grader_script.py`) will execute to verify:\n1. **Database Artifact:** Validates the physical existence of `pericles_spatial.db` containing at least two normalized, populated tables.\n2. **Header Schema:** Verifies the exported `pericles_golden_continuity_log.csv` contains exactly the 6 required headers in order.\n3. **Primary Pass/Fail Gate (Recall & Precision):** The script compares the agent's generated CSV against the hidden evaluator's gold standard.\n    *   **Recall:** The agent must successfully find at least 80% of the gold-standard tuples (Character, Domain, Zone, Scene Number).\n    *   **Precision (Anti-Spam):** The script divides the number of correct matches by the *total number of physical rows* generated by the agent. Malformed rows, incomplete rows, or rows lacking meaningful Action/Citation text (len < 4) are counted as incorrect penalty rows. To pass precision, this must be >= 80%. This mathematically penalizes duplicate spamming and hallucinations.\n    Achieving >= 80% in *both* metrics is the sole automated pass/fail gate.\n\n### Difficulty Self-Test Evidence\n**Score:** >60% Accuracy (Near-Term)\n**Notes:** Self-test performed on OpenClaw running `google/gemini-3.1-pro-preview` (using Python 3.12 and SQLite3). The agent successfully demonstrated professional engineering capabilities by designing the relational schema, parsing the text via Python to populate the SQLite database, and exporting the fully compliant 6-column CSV. It successfully matched the vast majority of the required character trajectories against the gold standard, passing the strict 80% recall and 80% precision thresholds. However, because it occasionally derived valid but alternative interpretations of highly ambiguous implicit stage directions hidden deep in the dialogue, its exact empirical tuple-match rate hovered firmly in the >60%-80% tier.","software_and_version":"OS Agnostic (Requires only Python 3.12 Standard Library and SQLite3)","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"6dd93385-77d4-4c63-95ae-a855f3096e1d","filename":"pericles_script.txt","category":"input_materials","content_type":"text/plain","size_bytes":106219},{"file_id":"188ebfe5-6d98-4f34-a58b-d169fa1577ca","filename":"pericles_golden_continuity_log.csv","category":"reference_output","content_type":"text/csv","size_bytes":8670},{"file_id":"de4bfbff-f836-471f-a936-105293c3e8df","filename":"grader_script.py","category":"reference_output","content_type":"text/x-python-script","size_bytes":5362}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — the task is complete, internally consistent, reachable on one machine, and objectively evaluable with the supplied local artifacts.\n\n**Completeness Check:**\n- Instruction: Complete — specifies extraction scope, database/output artifacts, permitted domains, zone canonicalization, scene numbering, and required CSV schema.\n- Inputs: Complete — `pericles_script.txt` is readable ASCII text, 106,219 bytes, and contains 22 explicit scene headers.\n- Toolchain: Complete — Python 3.12 standard library and SQLite3 are sufficient; no credentials or network services are required.\n- Reference output: Complete — the CSV is readable, has the required six-column schema, contains 99 valid unique tuples covering all 22 scenes, and uses only permitted domains.\n- Rubric: Complete — the grader checks database presence and populated tables, exact headers, domain and scene coverage, and 80% recall/precision thresholds.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. Task classification: `reachable`. I inspected the complete contents of `pericles_script.txt`, `pericles_golden_continuity_log.csv`, and `grader_script.py`; all match their declared plain-text formats and sizes. The script, reference CSV, and grader are parseable without execution, and no archives, encryption, unsafe links, credentials, or unreviewed attachment content were present. No reviewer-tooling limitation prevented inspection. The end-to-end path is clear: parse the script, populate `pericles_spatial.db`, export the six-column continuity CSV, then run the local grader against the reference output.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor.  \nConfidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n`public_benchmark` — the input is explicitly the canonical Shakespeare *Pericles* text, making the source material publicly identifiable. The custom continuity reference and exact benchmark publication status were not independently verified because no public search was performed. Keeping evaluation offline mitigates retrieval of externally hosted solutions; the custom reference remains evaluator-side evidence.\n\n**Recommended Changes:**\nThe submission is coherent and ready; no contributor changes are required. Optional evaluator hardening could validate required table columns and relationships rather than only checking that at least two populated tables exist.","reviewed_at":"2026-09-20T17:38:47.314468Z"},"claim":null}