{"submission_id":"a5ca2011-2120-4b15-9c8f-6d39fc598b41","submission_revision_id":"8cfcc449-43b7-4f06-8bf7-61189502edb6","revision_version":1,"submission_status":"submitted","submitted_at":"2026-10-14T00:50:18.565357Z","submitter_name":"Shivanshu Verma","submitter_email":"shivanshu156@gmail.com","task":{"status":"submitted","version":1,"extra_fields":{"difficulty_self_test":{"model":"Claude Opus 5.5","harness":"Custom single-agent harness (shell + Python tools, Docker, 4 vCPU, max reasoning effort)","estimated_score":"10–20%"}},"submitted_at":"2026-10-14T00:50:18.565357+00:00","industry_domain":"Audio, Music & Post-Production Media","task_short_name":"satb_chorale_transcription","operating_system":"Linux","task_description":"# Four-part chorale transcription from archive recordings\n\nA music publisher is re-issuing a collection of short four-part chorales for\nsoprano, alto, tenor and bass (SATB). The engraved masters were lost. All that\nsurvives are the reference recordings that were made from the masters for an\naudio companion. Some were made with a synthesised vocal ensemble, the others\nwith small instrumental ensembles (choir pad, strings, winds or brass). The\nengravers need the exact notes of every voice in every recording so that they\ncan re-engrave the scores. Your job is to produce those note lists.\n\nNothing about these chorales has been published. All you have is the data in\n`input/`.\n\n## Input files\n\n`input/pieces.csv` has one row per piece:\n\n| Column | Meaning |\n| --- | --- |\n| `piece` | piece identifier, e.g. `c01` |\n| `audio` | path of the recording, relative to `input/` |\n| `bpm` | tempo in quarter-note beats per minute (constant for the whole piece) |\n| `beats_per_bar` | quarter-note beats per bar (3 or 4) |\n| `first_onset_s` | time of the first downbeat in the WAV file, s |\n\n`input/audio/<piece>.wav` holds the recording: mono, 44.1 kHz, 16-bit PCM.\nThere are 20 pieces. Each lasts 15 to 29 s, followed by up to 2 s of reverb\ntail.\n\n## What is guaranteed about the music\n\nThese facts hold for every piece:\n\n1. There are exactly four voices: soprano `S`, alto `A`, tenor `T` and bass\n   `B`. Each voice is a single line and never sounds two notes at once.\n2. Voice ranges in MIDI note numbers (60 = middle C = C4): S 60–79 (C4–G5),\n   A 55–74 (G3–D5), T 48–67 (C3–G4), B 40–60 (E2–C4).\n3. Whenever two voices sound at the same time, S is higher than A, A is higher\n   than T and T is higher than B. Voices never cross and never share a pitch.\n4. Each piece is in one major or minor key, with no modulation. The key is not\n   given. Every pitch belongs to the key: in a major key, the seven notes of\n   the major scale; in a minor key, the seven notes of the natural minor scale\n   plus the raised seventh degree (the leading tone). There are no other\n   accidentals.\n5. The tempo is constant and equal to `bpm`. All four voices start together\n   on a downbeat at `first_onset_s`.\n6. Every note starts on the eighth-note grid, that is at\n   `first_onset_s + k × 30 / bpm` seconds for some integer k ≥ 0. Every note\n   lasts a whole number of eighth notes. A note can be held across beats and\n   bar lines; a held note is one note.\n7. The only rests are (a) an optional one-beat rest in all four voices at the\n   same time at the end of a phrase, and (b) the silence after the final chord.\n\n## What the recordings contain\n\n- Each note is played or sung at its score time plus a small random timing\n  error (at most ±15 ms). When a voice repeats the same pitch, the second note\n  is re-attacked after a short break. A held note is not re-attacked.\n- Tuning is equal temperament with A4 = 440 Hz. Each voice drifts by up to\n  about ±20 cents. The synthesised vocal parts also have vibrato.\n- Each recording has moderate room reverberation and a low noise floor.\n- The voices are roughly balanced in loudness.\n\n## Deliverable\n\nFor every piece in `input/pieces.csv`, write `output/<piece>.csv`, for example\n`output/c01.csv`:\n\n```\nvoice,onset_s,pitch\nS,0.500,72\nA,0.500,67\nT,0.500,64\nB,0.500,48\n```\n\n- The header line `voice,onset_s,pitch` is required. Extra columns are\n  allowed and ignored, and the row order does not matter.\n- One row per note. `voice` is `S`, `A`, `T` or `B`. `onset_s` is the note's\n  start time in seconds from the start of the WAV file. `pitch` is the MIDI\n  note number as an integer. Scientific pitch names such as `C4`, `F#3` or\n  `Bb2` are also accepted.\n- A held note is one row, at the time it starts. A re-attacked note is a new\n  row.\n- Durations and note ends are not graded and need not be given.\n- Voice assignment: every note belongs to the voice that sings it. Because the\n  voices never cross or share a pitch, at any moment the sounding notes from\n  highest to lowest belong to S, A, T and B. A correct note filed under the\n  wrong voice counts as one missed note and one wrong note.\n\n`output/METHOD.md` is optional and is not graded. Use it to describe your\nmethod in at most one page.\n\n## How it is graded\n\n- The reference for each piece is the note list of its lost score. Reference\n  onsets are the exact score times on the eighth-note grid, not the slightly\n  shifted performed times.\n- A predicted note matches a reference note if it is in the same voice, has\n  exactly the same MIDI pitch, and its onset is within ±0.050 s of the\n  reference onset. Matching is one-to-one: each reference note can match at\n  most one predicted note, and the grader uses the largest possible set of\n  matches, per voice.\n- For each piece: matches = matched notes over all four voices; precision =\n  matches / number of predicted rows; recall = matches / number of reference\n  notes; F1 = 2 × precision × recall / (precision + recall), and F1 = 0 if\n  there are no matches. Rows with an unreadable voice, onset or pitch count as\n  predicted rows that match nothing.\n- A missing file, or a file without the three required columns, gives F1 = 0\n  for that piece.\n- A piece **passes** if its F1 is at least **0.95**.\n- The **score** is the number of passing pieces divided by 20. The grader also\n  reports every piece's F1, for information only.\n\nYou have no access to the reference, and no grading feedback is available while\nyou work.\n\n## Environment\n\n- Linux with Python 3. You have internet access.\n- You have 4 CPU cores and no GPU. Plan for about 4 hours of wall-clock time in\n  total.\n- You may install any open-source tools and packages you need.\n- Leave `input/` unchanged.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Grading is deterministic and host-side, with no LLM judge. For each of the 20 pieces, the solver's output/<piece>.csv (columns voice,onset_s,pitch) is compared with the hidden reference score for that piece. A predicted note matches a reference note if it is in the same voice, has exactly the same MIDI pitch, and its onset is within ±0.050 s of the reference onset; matching is one-to-one and maximal per voice. Piece F1 is the micro-F1 over the four voices; a piece passes at F1 ≥ 0.95; the task score is the number of passing pieces divided by 20. Run hidden_eval/run_eval.sh <output_dir> from the reference ZIP to reproduce any score; ale/main.py implements the same rule for the harness. All of this is stated verbatim in the task description, so the solver knows exactly how it will be graded.","software_and_version":"Python 3 (open source); any open-source audio tools; no commercial software required","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"d63b3cbc-f83d-4eea-978f-d6cdebc60930","filename":"1_input_materials.zip","category":"input_materials","content_type":"application/zip","size_bytes":39068130},{"file_id":"70b72c70-cb02-434b-a675-c653c65daabb","filename":"2_reference_and_evaluation.zip","category":"reference_output","content_type":"application/zip","size_bytes":39208880},{"file_id":"14ce18c2-18c5-4886-b5f5-27388a4c503d","filename":"3_self_test_evidence.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":162251}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The submission is complete, technically reachable, and objectively evaluable, with consistent references and reproducible difficulty scores.\n\n**Completeness Check:**\n\n- Instruction: Complete — Deliverables, musical constraints, onset tolerance, voice assignment, and scoring are explicit.\n- Inputs: Complete — All 20 recordings are readable mono, 44.1 kHz, 16-bit PCM; the piece table has the required metadata.\n- Toolchain: Complete — Open-source Python and audio tools provide a viable Linux, CPU-only workflow.\n- Reference output: Complete — Twenty reference files contain 1,799 notes satisfying the disclosed grid, ranges, key restrictions, ordering, and rest rules.\n- Rubric: Complete — Deterministic maximum matching and per-piece F1 implement the stated scoring contract; independently checked fixtures reproduce scores of 1.00, 0.00, 0.10, 0.20, and 0.35.\n\n**File and Task Reachability:**\n\nAttachment classification: `reachable`. Task classification: `reachable`.\n\nInspected `1_input_materials.zip`, `2_reference_and_evaluation.zip`, and `3_self_test_evidence.zip`, including `TASK.md`, `pieces.csv`, all 20 WAVs, reference JSONs, evaluator sources, authoring documentation, reports, and fixture outputs. Archive sizes match the manifest; integrity and container-safety checks passed. Text, CSV, and JSON content is readable. Duplicate inputs and fixtures agree, and solver-visible inputs contain no reference answers.\n\nThe completion path is audio analysis followed by 20 transcription CSVs and hidden-reference evaluation. Supplied programs were not executed. Auditory intelligibility and human completion rates were not directly established; complete frontier transcripts and historical kill-test audio were not supplied. The evidence supports computational feasibility within the stated budget, not guaranteed perfect transcription.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. Authoring evidence establishes procedural composition and synthesized recordings rather than actual lost-master recovery, but does not establish AI fabrication. Public provenance could not be independently checked; the supplied evidence does not establish a public benchmark.\n\n**Recommended Changes:**\n\nThe precise specification, validated references, and graded controls provide a usable submission. No changes are required. A human transcription pilot would optionally strengthen difficulty calibration.","reviewed_at":"2026-10-14T00:57:58.245495Z"},"claim":null}