# Difficulty report — dde-bifurcation

Consolidated record of the measured agent trial of this task. Trial data comes
from SUBMISSION_SPEC.md section 5 and this package's own self-test log;
threshold, oracle and trap numbers come from `calibration.md` and the JSON
reports in `reference_outputs/`.

## Summary

| model | harness | shots scored | scores | result |
|---|---|---|---|---|
| Codex (GPT-5-based coding agent) | local execution | 3 | 0.0, 0.0, 0.0 | 0/3 |

- **Measured rate: 0/3 (0%)** on the only agent trial run against this task.
- **ALE tier chip: `10-20%`**, taken from SUBMISSION_SPEC.md section 5.
- **0/3 is an upper bound from a three-shot sample, not a claim of a
  sub-1-percent rate.** Three shots cannot resolve a pass rate below roughly
  20 percent: a task a frontier agent clears 20 percent of the time still
  returns 0/3 about half the time. The `<1% (last-exam)` chip is therefore not
  claimed.
- No other model family was trialled. Not run, not estimated.

## Trial protocol

Codex (GPT-5-based coding agent), local execution. Each shot ran from an
isolated copy of the task under `/private/tmp/agent_selftest` with the
solution, the reference output, the baseline output and all build/oracle
artifacts excluded; the hidden reference was passed to the evaluator only after
the submissions had been produced. Three shots, run 2026-08-08. Submissions
were scored by the real grader (`grade_bifurcation` and `grade_doubled` in
`bif_common.py`, the same functions the shipped verifier and `evaluate.py`
call).

The exclusion list removes every file that contains tau*, the orbit samples, or
the oracle source. Because the grader is self-certifying there is no hidden
reference value that could leak through the environment in any case: "the
dominant nontrivial Floquet multiplier equals -1" is recomputed from the
submitted pair alone.

Budget available to the agent per the task definition: 3 h
(`task.toml`: `timeout_sec = 10800`) on 4 vCPU / 8 GB RAM with no network.

## Agent trial

| shot | score |
|---|---|
| 1 | 0.0 |
| 2 | 0.0 |
| 3 | 0.0 |
| **result** | **0/3** |

What each attempt did, from `SELF_TEST.md`:

- The fixed-point attempt failed the amplitude and fundamentality
  requirements. Grader thresholds: amplitude >= 0.4, fundamentality gap >= 0.1.
- Smooth nontrivial guesses had acceptable-looking amplitudes but failed the
  DDE residual by many orders of magnitude, and did not meet the bifurcation
  multiplier condition. Grader thresholds: residual <= 1e-8, and |mu + 1| <= 1e-4
  for full credit or <= 1e-2 for the partial band.
- No genuine doubled branch was produced in any shot, so the 0.5
  partial-credit path (bifurcation point right, branch switch wrong or missing)
  was never reached either.

Self-test verdict recorded in `SELF_TEST.md`: "very hard. It combines
periodic-orbit solving, continuation in delay, Floquet analysis, and branch
switching to a period-doubled orbit."

## Non-agent baselines

Naive-method failures measured during construction and recorded in
`calibration.md` and `reference_outputs/trap_report.json`. These are
construction evidence, not scored agent shots.

1. **Forward simulation alone cannot produce the answer.** The orbit at tau* is
   unstable, so time-stepping actively repels it. The reference solver instead
   continues the orbit downward in tau with a collocation Newton solve at each
   step.
2. **A single accurate orbit is not an answer.** Along this branch
   dmu/dtau ~ -0.168, so the acceptance bound |mu + 1| <= 1e-4 pins tau* to
   about 6e-4 absolute. The measured branch runs monotonically from
   mu = -2.676218 at tau = 22.70 to mu = -0.984631 at tau = 12.70; only a real
   continuation plus root-solve reaches the answer. Secant refinement on
   mu(tau) + 1 then converged in 3 iterations to tau* = 12.795726635219,
   T* = 37.732804942870.
3. **Plain Newton continuation onto the doubled branch fails silently.** The
   first continuation attempt during construction slid onto the period-1 orbit
   written at double period: the residual stayed at 1e-14 while the
   fundamentality gap collapsed to 0.000. That is why the reference solver
   re-seeds from forward simulation at every step instead of Newton-continuing.
4. **The decoy passes every check but one.** Measured in `trap_report.json`:
   at tau_ev = 14.0 the period-1 orbit has period 40.382287016425735; written
   at double period its residual is 4.255e-14, comfortably inside the 1e-8
   gate, and its multiplier is mu_1^2, but its fundamentality gap is 4.44e-16
   against a required 0.1. The genuine doubled orbit has gap
   0.1909290540741806. Only the fundamentality check separates them, and the
   grader rejects the decoy with the recorded reason "submitted 'doubled' orbit
   is the period-1 orbit written at double period".

## What makes it hard

Measured, not asserted:

- **Three dependent numerical layers.** SciPy ships no delay-equation support,
  so the history-interpolating nonlinear integrator, the periodic
  boundary-value (collocation) solver, and the linearised monodromy operator
  must all be written from scratch, and all three must be simultaneously
  correct before the parameter root-solve means anything. The measured trial
  failed at layers 1-2 -- residual off by many orders of magnitude, amplitude
  and fundamentality violations -- and therefore never reached layer 3.
- **The tolerance is set by the grader's own certified accuracy, and it is
  tight enough to force a real Floquet computation.** At the bifurcation orbit
  the grader's two-resolution Richardson estimator (nh = 280/400,
  steps_per_delay = 700/1400) spreads by 2.689e-6 and its trivial-multiplier
  canary -- the exact eigenvalue mu = 1, which any correct discretisation must
  reproduce -- sits at 2.301e-5. So 1e-4 is the tightest bound the grader can
  honestly vouch for, and the oracle clears it by seven orders of magnitude at
  |mu + 1| = 1.609867794627462e-11.
- **The second deliverable is a branch switch guarded by a trap hit during
  construction.** See "Non-agent baselines" items 3 and 4. This is the
  deliverable the measured trial never reached in any shot.
- **Cost of the reference solution.** The oracle needs 1088.6005392074585 s,
  about 18 min, of the 3 h budget for the full continuation plus secant
  refinement, on 4 CPUs with NumPy and SciPy only.

## Limits of this evidence

1. **Three shots.** The entire agent-trial evidence for this task is three
   scored shots. That is a small sample by any standard.
2. **One model family.** Only Codex (GPT-5-based coding agent) was trialled, in
   one harness, on one host. No other model family or harness was run against
   this task, and none is estimated here.
3. **0/3 bounds the rate from above; it does not establish that the rate is
   near zero.** Three shots cannot resolve a pass rate below roughly 20
   percent -- an agent that clears the task 20 percent of the time returns 0/3
   about half the time. The reported chip is therefore `10-20%`, and the
   `<1% (last-exam)` chip is deliberately not claimed.
4. **What is not known.** Whether a stronger model, a longer budget, a
   different scaffold, or access to a delay-equation library would change the
   outcome; and how much of the failure is attributable to the branch-switch
   deliverable versus the bifurcation-point deliverable, since no shot passed
   the first well enough to be scored on the second.
5. **Expert-time figures are estimates, not measurements.** `task.toml` records
   `expert_time_estimate_hours = 18`; the superseded proposal wrote
   "~14-18 hours". Neither was timed.

## Discrepancies found

Where `tb-science/proposal-airtable.md` (now deleted, content carried into
`tb-science/submission-form.md`) disagreed with `calibration.md`,
`calibration.md` was trusted.

1. **Grader wall time.** `README.md` previously stated the TB-Science pytest
   verifier was "pytest verified, 4.4 s". `calibration.md` and `task.toml` both
   state "~90 s". The `~90 s` figure is used everywhere in the rewritten
   documents. For reference, a bare `evaluate.py` run on the reference
   submission on the author's macOS host completed in 1.4 s wall / 12.5 s CPU
   during this normalization pass; the spread is a hardware/threading artefact,
   and the conservative `~90 s` is what the forms quote.
2. **Superseded difficulty paragraph removed.** The old
   `proposal-airtable.md` Complexity section (iii) reported in-house trials run
   by the same agent lineage that built this task. Per SUBMISSION_SPEC.md
   section 5 those are not independent difficulty evidence, so the paragraph
   was deleted rather than softened and survives in no document in this
   package. The only agent-trial evidence anywhere in this package is the
   three-shot Codex self-test reported above.
3. **`instruction.md` omitted the second deliverable -- FIXED.**
   `tb-science/task/instruction.md` previously asked only for `tau_star`,
   `period`, `x` and five `results.json` keys, while
   `tb-science/task/tests/test_outputs.py::test_branch_switch` requires
   `period2` and `x2`, so an agent following the instruction literally could
   not score 1.0. The instruction now specifies both deliverables: the
   `period2` / `x2` entries with their dtype, shape (4096,), sampling
   convention t_j = j*T2/4096 and free starting phase; the fixed evidence
   delay tau_ev = 14.0; the full `results.json` key set including `period2`
   and `resid_linf_2`; the truthfulness cross-checks (tau_star to 1e-9
   relative, resid_linf within a factor of 10); and every acceptance gate the
   doubled orbit must clear (period2 in [70, 95], positivity, amplitude >= 0.4,
   spectral tail <= 1e-10, residual at tau_ev <= 1e-8, fundamentality gap
   >= 0.1). All 22 gates asserted by `test_outputs.py` and `bif_common.py` were
   checked off against the new text one by one; no existing gate was weakened
   or restated, and the original three-paragraph shape is preserved.
4. **The shipped TB-Science `solution/` could not pass its own tests --
   FIXED.** `tb-science/task/solution/find_bifurcation.py` was byte-identical
   to `build/find_bifurcation.py`: it seeded from
   `../../dde-periodic-orbit/reference_outputs/orbit_1.npz`, wrote to
   `../reference_outputs/`, and never produced `period2` / `x2`. It has been
   replaced by a turnkey solver with no cross-package reads, no vendored seed
   data and no reachable-orbit assumption: it captures the period-1 orbit by
   forward simulation at tau = 12.0, where that orbit is still stable, then
   continues UPWARD in tau (12.0, 12.4, 12.8) until mu(tau) brackets -1 and
   secant-refines; it then captures the doubled orbit at tau_ev = 14.0 by
   forward simulation, where the doubled branch is the attracting one, and
   polishes it at 512 then 4096 samples. It writes `/app/solution.npz` (all
   five entries) and `/app/results.json` (all seven keys), which is where
   `task.toml` `[verifier] artifacts` and `tests/test.sh` look. `solve.sh` is
   unchanged and now runs end to end. Verification actually performed is
   recorded in the "Verification of the shipped solution" section below.
5. **Cost of the reference solution, restated.** The 1088.6 s figure quoted
   throughout this package is the genuine wall time of the original oracle run
   that produced everything in `reference_outputs/`, recorded in
   `solver_report.json`, and it stands as the calibration record. The
   replacement turnkey solution reaches the same answer by a shorter path and
   was measured at 292 s. Both numbers are real; they time different
   procedures, and neither has been retro-fitted to the other.
6. **ALE upload set does not match the form's old file list.** The superseded
   ALE form listed `reference_outputs/solution.npz` as the reference output and
   `calibration.md` as an uploaded file. Measured with the shipped grader,
   `solution.npz` scores 0.5 ("branch switch: missing 'period2'"); the complete
   expert result is `reference_outputs/submission.npz`, which scores 1.0. And
   `calibration.md` is not present under `agents-last-exam/upload/`. Both are
   corrected in `agents-last-exam/submission-form.md`, and `calibration.md`
   plus `doubled_orbit_report.json` have since been copied into the upload set
   so document and directory agree.

## Verification of the shipped solution

Run during this pass, on the author's macOS host (Python 3.11.5, pytest 7.4.0),
not inside the task container -- Docker is not available in this environment, so
`environment/Dockerfile` was not built and `tests/test.sh` was not executed as a
script. What was run, and what it produced:

1. **The shipped solution, end to end.**
   `tb-science/task/solution/find_bifurcation.py` executed with its output
   directory pointed at a scratch folder (`BIF_OUT`; the default is `/app`,
   which is what `solve.sh` uses in the container). Run twice: **291.98 s** from
   the shipped `tb-science/task/solution/` tree, and 287.97 s from a scratch
   copy beforehand. Both runs wrote `solution.npz` with all five entries and
   `results.json` with all seven keys, and produced identical metrics. The
   292 s figure quoted elsewhere in this package is the shipped-tree run.
   Recovered tau* = 12.795726630532183 against the reference
   12.795726635219003 -- agreement to 4.7e-9, nine significant figures, by a
   completely different continuation path (upward from tau = 12.0 rather than
   downward from tau = 22.7). |mu + 1| = 7.382121580690182e-10, orbit residual
   1.0e-13, fundamentality gap 0.33298811512479887, amplitude
   0.6146291670326967. The doubled orbit reproduced the reference exactly:
   period2 = 83.42643131772385, residual 3.970955508858509e-14, fundamentality
   gap 0.1909290540741806, amplitude 0.7683666189275312.
2. **The shipped verifier, against that output.**
   `tests/test_outputs.py` run under pytest with `BIF_VDIR` / `BIF_SOL` /
   `BIF_RES` pointed at the scratch folder -- the same three tests, the same
   `bif_common.py`, that `test.sh` invokes. Result: **3 passed in 2.65 s**
   from the shipped tree (2.44 s on the earlier scratch-copy run)
   (`test_bifurcation`, `test_results_truthful`, `test_branch_switch`), which
   is the branch on which `test.sh` writes `1` to
   `/logs/verifier/reward.txt`. **Score: 1.0.**
3. **The ALE grader, against the same output, as a cross-check.**
   `agents-last-exam/upload/evaluate.py` returned **score 1.0**, reason "orbit
   at a certified period-doubling point (mu = -1); genuine period-doubled
   orbit".

Not verified here: the Docker build, `pip install pytest==8.2.0` inside the
verifier container, the `/app` and `/verifier` path wiring, and the 4 vCPU /
8 GB Linux timings. The 291.98 s and 2.65 s figures are macOS-host timings and
are not container measurements; `calibration.md`'s "~90 s" remains the quoted
verifier figure, as recorded in discrepancy 1 above.