# Difficulty self-test — measured, not estimated

Three independent agents attempted the task blind, each in a clean directory
containing only the agent-facing inputs. No agent could see the reference
configuration, the gold file, the grader, or another agent's work. Each was
graded by running ORE on its submitted configuration.

| trial | task version | market wiring | score | what happened |
|---|---|---|---|---|
| A | original | given | 0.0 | 4 of 5 trades exact; missed only the cross-currency swap |
| B | original | given | 0.0 | identical outcome, independently |
| C | hardened | **authored by the agent** | **1.0** | every trade exact, first attempt |

## Trials A and B did not fail on difficulty

Both agents got the collateral-discounting decision right, selected the correct
market configuration for each netting set, explained the reasoning correctly,
handled normal rather than lognormal volatilities, configured LGM for the
Bermudan, and recovered ORE's undocumented engine keys from its structured error
log.

Their single miss was the cross-currency swap, and the cause was a defect in the
term sheet: the reference trade exchanges notional at maturity and the term sheet
never said so. Both agents reasoned "unstated, so ORE's default applies, so no
exchange" — which is what the brief told them to do.

Adding that one field to trial A's portfolio, changing nothing else, moves it
from 0.0 to 1.0. The task was not testing difficulty; it was testing whether the
agent could guess an omission.

## Trial C is the real measurement

The term-sheet defect was fixed and the task was hardened by removing the
pre-built market wiring, so the agent had to author `todaysmarket.xml` from the
curve definitions — deciding for itself that a USD-collateralised EUR trade
discounts on the cross-currency-basis-adjusted `EUR-IN-USD` curve rather than on
EUR OIS.

It scored 1.0. Every NPV exact, every cashflow schedule exact, in 26 tool calls
and about 94,000 tokens.

## Conclusion

**This task is near-term tier, not last-exam.** Collateral-aware discounting is
well documented and frontier models reason about it correctly. ORE's
configuration fiddliness is not a barrier either: the agents iterated against the
error log and converged.

The €7.6 million single-curve mispricing measured during design is real, but it
is a trap for a careless human, not for these models.

## What the agents themselves flagged as uncertain

Worth reading, because it is where genuine difficulty would have to come from —
each is a point where the term sheet does not determine the answer:

1. The Bermudan LGM calibration strategy. Trial C measured `CoterminalATM`
   against `CoterminalDealStrike` at a 0.3% NPV difference and correctly noted
   nothing pins it.
2. Which discount curve the swaption volatility cube's CMS indices should use.
3. Whether the cross-currency swap exchanges notional — the defect above.

Pinning these in the term sheet would make the task well-posed but not harder; it
would only remove ambiguity. Real difficulty would need a different source: a
quote set that admits no arbitrage-free fit and requires a diagnosed choice, a
CSA with thresholds making the discount curve state-dependent, or a calibration
that fails silently under the obvious basket.