{"submission_id":"b4ba08e6-c277-4c2c-9576-f98a07ec1d3c","submission_revision_id":"efeca181-0ea5-4dc3-ba6e-012bfa4ef8ea","revision_version":2,"submission_status":"submitted","submitted_at":"2026-09-21T22:19:23.042975Z","submitter_name":"Himanshu Gupta","submitter_email":"him141195@gmail.com","task":{"status":"submitted","version":2,"extra_fields":{"difficulty_self_test":{"model":"Claude Opus 4.8","harness":"Claude Code","estimated_score":"5–10%"}},"submitted_at":"2026-09-21T22:19:23.042975+00:00","industry_domain":"Formal Verification and Mathematics","task_short_name":"lean_formal_proof_instance_1","operating_system":"Linux","task_description":"Objective: Write a complete, machine checked proof in Lean 4 with Mathlib of a counting identity about labellings of points on a circle.\n\nThe mathematics. Put n + 1 equally spaced points on a circle and label them 0, 1, ..., n, using each label exactly once. Two labellings count as the same if one is a rotation of the other.\nCall a labelling beautiful if for every four labels a < b < c < d with a + d = b + c, the chord joining the points labelled a and d does not cross the chord joining the points labelled b and c. Let M n be the number of beautiful labellings up to rotation, and let N n be the number of ordered pairs of positive integers (x, y) with x + y at most n and gcd(x, y) = 1. Prove that M n = N n + 1 for every n at least 3.\n\nStarting State: A Lean 4 project pinned to Lean v4.27.0 and Mathlib v4.27.0 is staged in your workspace, with Mathlib already compiled so you never rebuild it and an edit to one file recompiles in seconds. The project contains:\n\n  - task_specification.md, the written statement of the problem, the rules, and the deliverable.\n  - project/OlympiadProj/Model.lean, 41 lines, namespace Circle. This fixes the computable model:\n    the list representation of a labelling, strict cyclic betweenness, chord crossing, the\n    beautiful labelling predicate, the permutation enumeration, and the two counts M and N. It is the authoritative definition of every symbol in the theorem. It is fixed and it is not graded.\n  - project/OlympiadProj/Submission.lean, the file you complete. It imports the model and states the theorem with its proof left as sorry:\n\n        theorem beautiful_labelling_count {n : ℕ} (hn : 3 ≤ n) : Circle.M n = Circle.N n + 1\n\n  - build.sh, a launcher that runs lake build in the project.\n\nInstructions & Rules:\n\n  1. Read task_specification.md and then read Model.lean. The model file, not the prose, is the\n     authoritative statement of what M and N mean.\n  2. Replace the sorry in Submission.lean with a complete, correct proof. Add whatever auxiliary\n     definitions and lemmas you need, above the theorem, in that same file.\n  3. Run the build until the project compiles with no errors and no sorry warning.\n  4. Copy your finished Submission.lean to the output directory. That single file is the only\n     graded artifact.\n\n  Rules:\n  - Do not change the name, the binders, or the statement of beautiful_labelling_count.\n  - Do not restate, shadow, or edit the definitions in Model.lean, and do not edit any other file\n    in the project. Only Submission.lean is graded.\n  - The proof must be genuine. No sorry, no admit, no native_decide, and no added axiom\n    declarations.\n  - Keep Submission.lean to ordinary Lean: definitions, lemmas, and tactic or term proofs. No\n    metaprogramming and no compiler escapes, specifically no macro, macro_rules, elab, elab_rules,\n    syntax, declare_syntax_cat, notation, custom infixl, infixr, infix, prefix or postfix, delab,\n    unexpanders, elaborator or tactic attributes, run_cmd, initialize, #eval, import Lean,\n    open ... Lean, any Lean.* identifier, implemented_by, extern, load_dynlib, unsafe, or\n    set_option toggles affecting the kernel or debugging. The reference proof uses none of these.\n\n  Note on what this task is. There is no formalisation of this identity in Mathlib or in any\n  public Lean library, and Mathlib carries none of the cyclic chord geometry the statement needs,\n  so there is nothing to look up or adapt. You have to reconstruct the mathematical argument and\n  then build the supporting layer in Lean yourself. The reference proof is a few thousand lines.\n","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Verification is done by the Lean 4 kernel and elaborator. It is fully deterministic and binary,\n1.0 or 0.0. There are no tolerances, no fuzzy comparison, and no model judgment anywhere in the\nloop. Two agents submitting different valid proofs both score 1.0, which is correct: the task is\nto prove the theorem, not to reproduce our text.\n\nDo not diff the agent output against our reference proof. The reference is one valid proof among\nmany. It is provided so you can confirm the task is solvable and confirm the grader passes an\nhonest solution.\n\nSet up the grading tree first. Do not grade the agent's own build directory. Build a fresh\ndirectory from the pre-built Mathlib plus the pristine project/ in the reference upload, then\ncopy in exactly two files: the agent's submitted Submission.lean, and Verify.lean from the\nreference upload, placed at OlympiadProj/Verify.lean. Nothing else the agent wrote is read. This\nis what makes a tampered Model.lean, an edited lakefile.toml, or a poisoned .lake build artifact\nunable to affect the result.\n\nThen apply four gates. All four must pass for 1.0. Any single failure is 0.0.\n\n  Gate 1, deliverable present. The agent's Submission.lean exists in the output directory and is\n  readable. If it is missing, the score is 0.0. Treat this as a normal agent failure, not an\n  error.\n\n  Gate 2, source scan. Reject the submission if it uses metaprogramming or compiler escapes:\n  macro, macro_rules, elab, elab_rules, syntax, declare_syntax_cat, notation, custom infixl,\n  infixr, infix, prefix or postfix, delab, unexpanders, elaborator or tactic attributes, run_cmd,\n  initialize, #eval, import Lean, open ... Lean, any Lean.* identifier, implemented_by, extern,\n  load_dynlib, unsafe, or set_option toggles affecting the kernel or debugging. Strip Lean\n  comments, string literals and guillemet quoted identifiers before matching, so that a banned\n  keyword appearing inside a comment does not cause a failure. scripts/verify_outputs.py in the\n  reference upload implements this exactly; scan_banned_syntax(source) returns the list of\n  violations. The reference proof returns zero violations.\n\n  Gate 3, it compiles. lake build succeeds in the grading tree, exit code 0.\n\n  Gate 4, the grader module compiles. Run: lake env lean OlympiadProj/Verify.lean. Exit code 0 is\n  a pass. This is the gate that does the real checking, and it does two independent things.\n\n    First it pins the statement by type. Verify.lean states the intended theorem itself and\n    proves it by applying the agent's theorem:\n\n        theorem reference_lemma (n : ℕ) (hn : 3 ≤ n) : Circle.M n = Circle.N n + 1 :=\n          BeautifulLabel.beautiful_labelling_count hn\n\n    Circle.M and Circle.N here resolve to the grader's own pristine model file. So a weakened\n    statement, a changed binder, a renamed theorem, or a shadowed model is a type mismatch and\n    the compile fails. An agent cannot score by proving something easier.\n\n    Second it audits the axioms. Verify.lean collects the transitive axiom dependencies of the\n    resulting proof term, which covers every lemma the agent's proof rests on, and fails the\n    compile unless they all lie inside the three standard Lean axioms, propext,\n    Classical.choice and Quot.sound. This is what rejects an incomplete proof: sorry appears as\n    sorryAx, native_decide appears as Lean.ofReduceBool, and an agent declared axiom appears\n    under its own name. A known-sorry canary theorem is audited first in the same context, and\n    the compile fails if that canary stops reporting sorryAx, which catches an attempt to\n    disable the audit rather than to fake the proof.\n\nExpected fixture behaviour, which you can use to confirm your setup: the reference proof scores\n1.0, and the untouched sorry stub from the input upload scores 0.0.\n\nmain.py in the reference upload is a working implementation of all of the above for the ale_run toolkit, if you would rather not rebuild it.\n","software_and_version":"Lean 4 v4.27.0, Mathlib v4.27.0 (git tag v4.27.0), Lake (bundled with Lean 4), elan (latest)","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"850f929f-54b2-4b86-b600-7f7c15216010","filename":"upload_input_materials.zip","category":"input_materials","content_type":"application/zip","size_bytes":6630},{"file_id":"ebfd8b2b-78a8-405e-8f66-f1eeefe3d185","filename":"evidence_self_test.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":7864},{"file_id":"4f3d79ad-764c-447d-9d78-97fb0402d5cf","filename":"upload_reference_and_eval.zip","category":"reference_output","content_type":"application/zip","size_bytes":53860}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — the revision is complete, technically coherent, and objectively gradable, provided the declared deny-by-default network policy is enforced to mitigate public reference-answer exposure.\n\n**Completeness Check:**\n- Instruction: Complete — the theorem, authoritative model, permitted edits, prohibited proof mechanisms, build process, and single-file deliverable are explicit.\n- Inputs: Complete — the fixed model, proof stub, specification, launcher, and pinned project configuration are present and mutually consistent.\n- Toolchain: Complete — Lean 4.27.0, Mathlib 4.27.0, Lake, and elan are pinned, legally obtainable, and supported by a documented prebuilt environment.\n- Reference output: Complete — a substantive 2,691-line proof ends with the exact required theorem and contains no raw occurrences of `sorry`, `admit`, `native_decide`, added axioms, or banned metaprogramming constructs.\n- Rubric: Complete — four deterministic hard gates cover delivery, source restrictions, pristine compilation, exact theorem type, and transitive axiom dependencies; incomplete work receives zero.\n\n**File and Task Reachability:**\nAttachment classification: `reachable`. Task classification: `reachable`. All three manifest attachments were inspected: `upload_input_materials.zip`, `evidence_self_test.zip`, and `upload_reference_and_eval.zip`. Their downloaded sizes matched the manifest; each was a valid, unencrypted ZIP with traversal-free relative paths, only regular files and directories, modest expansion, readable ASCII/UTF-8 content, valid JSON where applicable, and no archive integrity errors.\n\nThe inspection covered the specification, build launcher, complete solver project, model and stub; all four self-test evidence files; and the reference proof, hidden verifier, pristine project, environment metadata and installer, task card, review notes, evaluator implementation, source scanner, and scanner tests. No archive member was omitted. The long reference proof received structural, token-level, and representative-content inspection rather than line-by-line mathematical reconstruction. Uploaded programs and scripts were not executed, as required by the review policy; fixture execution results are therefore supported by the submitted logs rather than independently reproduced.\n\nThe end-to-end path is clear: edit the supplied proof stub, compile against the pinned model and Mathlib, submit one Lean file, scan it for prohibited syntax, rebuild it over a pristine model, and compile the hidden verifier. The verifier pins the exact theorem by type and rejects nonstandard transitive axioms. The solver-visible package does not contain the protected proof or verifier, and the input and reference copies of the model and dependency configuration are byte-identical. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n`public_benchmark` — the submission affirmatively discloses that the exact reference proof and verifier remain anonymously retrievable from a public repository. This was not independently rechecked through network search, but the disclosure itself establishes the exposure. The required mitigation is deny-by-default solver egress with only harness and model endpoints allowed; the task card marks this constraint as mandatory and unsafe to omit.\n\n**Recommended Changes:**\nThe revision aligns the scanner with the documented treatment of comments and strings, adds focused scanner tests, preserves a strong kernel-level evaluator, and explicitly declares the contamination mitigation. No contributor changes are required. Deployment must enforce the stated egress restriction; the task should not run in an unrestricted-network environment.","reviewed_at":"2026-09-21T22:28:50.778074Z"},"claim":null}