{"claim":{"id":"8fe42582-6055-493b-875c-5e44843caf4c","submission_id":"040b6e42-a251-483f-8c32-6dfc773ba899","submission_revision_id":"b81de150-b5e8-4b01-b30d-49e2019c4718","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:14:34.503180Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null},"task":{"submission_id":"040b6e42-a251-483f-8c32-6dfc773ba899","submission_revision_id":"b81de150-b5e8-4b01-b30d-49e2019c4718","revision_version":3,"submission_status":"under_review","submitted_at":"2026-08-30T01:48:01.612536Z","submitter_name":"Tarun Raheja","submitter_email":"tarunraheja1234@gmail.com","task":{"status":"submitted","version":3,"extra_fields":{"difficulty_self_test":{"model":"frontier agent (Claude, general-purpose agentic harness)","harness":"agentic, filesystem + Python, ORE installed, no internet lookup of the task","estimated_score":"40–60%"}},"submitted_at":"2026-08-30T01:48:01.612536+00:00","industry_domain":"Quantitative Finance — derivatives pricing and collateral-aware valuation (rates)","task_short_name":"collateral_aware_valuation_ore_instance_1","operating_system":"Linux","task_description":"Objective: Value a five-trade interest-rate book across two netting sets as of\n2016-02-05 in ORE, and hand back the configuration used.\n\nStarting state:\n- input/market/ — the desk's market environment, read-only: 8,222 market quotes\n  (deposits, OIS, tenor swaps, cross-currency basis, FX, normal swaption and\n  cap/floor surfaces, credit), historical fixings, index and instrument\n  conventions, curve definitions, currencies and calendar adjustments.\n- input/docs/termsheets.md — the five trades and the credit support annex for\n  each netting set. Every field is binding, including the notional exchange flags.\n- input/docs/brief.md — what to deliver, what is judged, and the prescribed\n  Bermudan model specification.\n- input/output_contract.json — the deliverable, the ore.xml requirements, the\n  model specification, and the graded quantities.\n\nNetting sets: EUR_CSA (cash collateral in EUR — a 20y vanilla swap, a cap on\nEURIBOR-6M, a Bermudan swaption) and USD_CSA (cash collateral in USD — the same\n20y swap, and a EUR/USD cross-currency basis swap).\n\nDeliverable, eight files:\n  output/runs/EUR_CSA/{ore.xml, portfolio.xml, pricingengine.xml, todaysmarket.xml}\n  output/runs/USD_CSA/{ore.xml, portfolio.xml, pricingengine.xml, todaysmarket.xml}\n\nNo pre-built curve wiring is supplied. todaysmarket.xml must be authored from the\ncurves defined in curveconfig.xml: the discount curve per currency, the\nforwarding curve per index, and the volatility and FX objects in scope.\n\nRules and traps:\n- Discounting must reflect how each netting set is collateralised. Under a cash\n  CSA in a currency other than the trade's own, the discount curve is the\n  cross-currency-basis-adjusted curve, not the trade currency's own OIS curve.\n  The same 20y swap appears in both netting sets and must price differently.\n- The EUR volatility surfaces are quoted normal, not lognormal.\n- The Bermudan swaption has no closed form. Because its NPV is model-dependent,\n  the model is prescribed in the brief and must be used exactly: LGM, Bootstrap\n  calibration, CoterminalATM strategy, HullWhite reversion 0.03, Hagan volatility\n  0.01, ShiftHorizon 0.5, tolerance 1e-4, Grid engine with sx=sy=3.0 and\n  nx=ny=10. CoterminalDealStrike is defensible in practice but is not what is\n  being asked for.\n- Swap indices referenced by the swaption volatility surface (EUR-CMS-1Y,\n  EUR-CMS-30Y) discount on the netting set's own collateral curve.\n- ORE silently drops trades that fail to build; a run pricing four of five trades\n  reports no error.\n\nEverything runs on CPU. No GUI step.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Run the shipped grader:\n\n    PYTHONPATH=scripts python scripts/score_outputs.py <submission_dir> reference\n\nBinary and deterministic. No judge model.\n\nFor each netting set the grader assembles a run directory from the task's own\nmarket environment plus the four files the submission authors, runs ORE, and\nreads npv.csv and flows.csv. The market environment always comes from the task,\nso a submission cannot alter market data — the self-test verifies this by\nsmuggling a corrupted market file into a passing submission and confirming the\nscore does not change.\n\nHard gates, each scoring 0.0: a run directory missing; any of the four authored\nfiles missing; ORE failing to run on the submitted configuration.\n\n1.0 requires, for every trade:\n- NPV within 1e-6 relative of the reference\n- the set of (payment date, amount) cashflow pairs matching exactly to the cent\n- the netting set pricing exactly its own trades\n\nReference NPVs. EUR_CSA: Swap_20y 1,640,812.637217; Cap_EUR -6,881.951398;\nBerm_EUR 84,184.942997. USD_CSA: Swap_20y_USDCSA 1,729,065.395397;\nXCcy_EURUSD -25,336,750.964580. The identical 20y swap differs by EUR 88,252.76\nbetween the two netting sets, purely from the collateral currency.\n\nFixtures, both shipped:\n- fixtures/reference_pass     → 1.0, no failures\n- fixtures/naive_single_curve → 0.0, all five trades mispriced. An engineer who\n  discounts off the forwarding curve, the pre-crisis single-curve approach:\n  EUR 30,927 wrong on the vanilla swap and EUR 9.3 million wrong on the\n  cross-currency swap, with no error anywhere.\n\nThe engine is pinned at open-source-risk-engine==1.8.16.0 and the reference\ngenerated against that pin, because ORE is deterministic within a version but not\nacross versions. The self-test checks the installed version against the pin.\nscripts/selftest.py runs 13 checks; all pass.\n\nChanged in this revision: the Bermudan model was underdetermined. It is now\nprescribed in full in the brief and the output contract, the calibration basket\nis stated, and the swaption surface's swap-index discounting is stated. Two\nchecks were added — one tying the published model settings to the reference\npricing engine field by field, and one confirming the prescribed strategy is\nload-bearing: substituting CoterminalDealStrike gives 83,925.92 against the\nreference 84,184.94 and is correctly rejected.\n\nI prescribed the model rather than widening the tolerance. Loosening enough to\naccommodate the 0.3% spread between the two calibration strategies would also\nhave admitted genuinely wrong configurations.","software_and_version":"ORE (Open Source Risk Engine) 1.8.16.0, Apache-2.0, driven from Python 3.11. pip install open-source-risk-engine==1.8.16.0  The pin matters: ORE is deterministic within a version but not across versions, so the reference must be generated against the pinned build.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"8df19c11-0143-402e-b708-df312af0abc5","filename":"DIFFICULTY.md","category":"difficulty_self_test_evidence","content_type":"text/markdown","size_bytes":3294},{"file_id":"07e9a86c-157e-4fcd-b2d3-6d01fbc20ca1","filename":"ore_input.zip","category":"input_materials","content_type":"application/zip","size_bytes":112773},{"file_id":"ace02771-c35f-414c-92d3-a348bd99c663","filename":"ore_reference.zip","category":"reference_output","content_type":"application/zip","size_bytes":62951}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The submission is complete, professionally scoped, safely reachable, and objectively gradable, with only minor evidence-hygiene improvements advisable.\n\n**Completeness Check:**\n- Instruction: Complete — The valuation date, five trades, CSA-specific discounting, model parameters, required directory structure, and eight authored files are unambiguous.\n- Inputs: Complete — The market data, fixings, conventions, curve definitions, currency/calendar data, term sheets, brief, and output contract are present and readable; the stated 8,222 “quotes” are actually 8,222 physical lines containing 8,126 dated quote records, which is nonmaterial.\n- Toolchain: Complete — Linux, Python 3.11, and the Apache-2.0 ORE 1.8.16.0 package provide a legal, CPU-only, single-machine execution path with no external service dependency.\n- Reference output: Complete — Reference configurations, five NPVs, 212 dated cashflows, passing and negative fixtures, generation logic, and model-consistency checks are supplied and internally consistent.\n- Rubric: Complete — The binary grader explicitly enforces run viability, trade membership, relative NPV tolerance, and cent-rounded dated cashflows; its failure list identifies individual partial errors.\n\n**File and Task Reachability:**\nThe attachment classification is `reachable`, and the end-to-end task classification is `reachable`. All three manifest attachments were inspected: `DIFFICULTY.md`, `ore_input.zip`, and `ore_reference.zip`. The archives matched their declared sizes and formats, passed integrity and path-safety checks, were unencrypted, contained no links or unusual filesystem entries, and expanded to nine input files and 35 reference files. The brief, term sheets, output contract, all market/support files, reference configurations, gold data, fixtures, task documentation, and submitted Python sources were inspected; repeated fixture files were confirmed identical where expected. Large market and XML files were structurally checked and sampled rather than read line by line. All JSON parsed, all files were valid UTF-8, and all 29 XML files passed structural nesting checks without DTD or entity declarations. Uploaded programs were reviewed statically and not executed; the review environment did not provide Python/ORE or an XML schema validator, so the reported 13-check self-test was not independently rerun. The completion path is clear: install the pinned ORE package, author four configurations per CSA, then let the evaluator combine them with immutable market inputs, execute ORE, and compare generated reports with the protected reference. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone — no affirmative evidence supports a policy label; public identifiability was not externally checked under the evidence boundary.\n\n**Recommended Changes:**\nNo material changes are required. Optional hardening would be to remove the exact Bermudan reference and alternative NPVs from the solver-visible output contract while retaining the prescribed strategy, correct “8,222 quotes” to “8,222 lines containing 8,126 dated quote records,” and fix the README’s residual “eleven checks” wording to “thirteen checks.”","reviewed_at":"2026-08-30T01:59:36.972840Z"},"claim":{"id":"8fe42582-6055-493b-875c-5e44843caf4c","submission_id":"040b6e42-a251-483f-8c32-6dfc773ba899","submission_revision_id":"b81de150-b5e8-4b01-b30d-49e2019c4718","builder":"builder-01","status":"active","claimed_at":"2026-10-15T15:14:34.503180Z","delivered_at":null,"ended_at":null,"delivery_note":null,"end_reason":null}},"revision_is_current":true}