# Mindustry sampled video checker — version 3.0.0 Documentation/package revision: `reference-docs-1`. Scoring code and matching assets are unchanged; CHECKER_MANIFEST.json records their SHA-256 hashes. ## Authoritative score The only scored candidate deliverable is `run_recording.mp4`. Success means `check_video.py` returns `status: PASS`, `scope: VIDEO_SAMPLED_FINAL_STATE_60S`, `checker_version: 3.0.0`, `longest_matching_seconds: 60`, and process exit code 0 under the sampling rule below. An old text-checker PASS, a single-frame MATCH, a report, or a correct bit string is not success. This version replaces the previous every-frame evaluation contract. ## Two-stage sampling rule Times are relative to the first decoded video frame, which is time zero. 1. **Coarse search:** select one actual source frame every 3 seconds, targeting 0, 3, 6, ... seconds. Find frames matching the final reactor profile. 2. **Candidate intervals:** for each coarse match at time `t`, include whole seconds from `max(0, floor(t)-3)` through `floor(t)+4` (exclusive). Merge overlapping or touching intervals and clamp them to complete video seconds. The extra context avoids losing the beginning or end of a good run. 3. **Fine check:** only in candidate intervals, target two actual source frames per second: `s` and `s+0.5`, for each integer second `s`. 4. **Accepted second:** both samples must satisfy all visual tests and the retained freeze heuristic. A missing, unreadable, nonmatching, excessively late, or freeze-rejected sample makes that second fail. 5. **PASS:** find 60 consecutive accepted whole-second bins. Stop fine checking when this is established. Separate good intervals are never added together. Sampling selects the first available source frame at or after each grid target, within its time bucket. Fine samples may be at most 0.25 seconds late. It does not create duplicate frames to fill missing slots. A complete second `[s,s+1)` must also fit within the video duration. For example, accepted samples at 0, 0.5, ..., 59, 59.5 count as 60 seconds if the video lasts at least 60 seconds. A 59.75-second video cannot pass. This is a bin-counting convention, not a claim that the first and last sample timestamps are 60 seconds apart. Coarse targets are a subset of fine targets. Therefore a genuine 60-second run of passing fine samples necessarily provides coarse candidates. Fine checks can discover an interval starting before the first matching coarse sample. The interval may appear anywhere, including in the final 112 seconds. There is no startup deadline and no restriction to the beginning of the recording. ## Visual tests and motion Each accepted fine sample must show: - A readable English **Impact Reactor** information panel and **Health** label. - **Power Output: 6300.0**, matching the target glyphs. - A visibly full Health bar, including its right edge. - The designated reactor and its surrounding reference-map battery layout. HUD and map localization remain adaptive: no resolution whitelist, fixed screen position, aspect ratio, or recording-specific coordinates. A cached location is checked first; failed localization falls back to content-based search. Different UI and map scales are supported. Text and landmarks must remain readable; low detail, occlusion, changed themes/language, extreme zoom, or heavy compression can make a sample unreadable. Constant canvas dimensions are required within a single video; different videos may have different dimensions. The earlier freeze heuristic remains, now applied to fine samples: compare the located reactor body with its last sufficiently changed sample. A change means at least 1.5% of pixels have a channel difference greater than 12. Two seconds without sufficient observed change causes sample rejection until motion resumes. This is a heuristic: sampled animation can alias, and this does not prove that footage is live or unedited. ## What changed and what PASS means This checker intentionally examines two samples per candidate second, not every source frame. Brief problems occurring only between samples may go undetected. For this benchmark, two passing samples plus the retained motion check represent an accepted second. This sampling convention defines the score; do not continue claiming verification of every frame in ALE text fields or other attachments. Video PASS is evidence of the sampled final visual outcome. It does not prove knowledge of a solution, authenticate preceding switch actions, GUI compliance, intermediate-state safety, every building's health, indefinite fuel/coolant supply, actual simulation speed, or recording provenance. Those are not hidden additional scoring gates. Visual recognition can have false positives/negatives. ## Performance and progress - Metadata probing does not enumerate every source frame. - FFmpeg selects samples before sending RGB images to Python. It still decodes intervening compressed frames where needed; total runtime is not determined only by the number of samples. - Coarse scan checks roughly `duration / 3` images; fine scan checks two per candidate second and stops once 60 consecutive seconds pass. - Cached HUD and map locations avoid repeated full-image localization. - Progress is printed to stderr about every 3 seconds of processing, including stage, video time, sample count, and best accepted duration. JSON stays on stdout. - No GPU, OCR service, model API, or internet connection is used during evaluation. The frame selection uses FFmpeg's documented [select and showinfo filters](https://ffmpeg.org/ffmpeg-filters.html). ## Install and run (PowerShell) Requirements: Python 3.9+, NumPy, SciPy, Pillow, and native FFmpeg/ffprobe on PATH. The host must provision these on the evaluator machine before validation. The Python requirements do not install ffmpeg/ffprobe executables. See HOST_SETUP.md for installation, preflight checks, and enforced access isolation. ```powershell python -m pip install -r requirements.txt # Host installation of native executables (skip if already provisioned): conda install conda-forge::ffmpeg python validate_reference.py "C:\Users\12097\Pictures\Screenshots\reference_run_recording.mp4" python check_video.py "D:\Benchmark\deliverables\run_recording.mp4" --output evaluation_result.json ``` Run from the extracted `mindustry_video_eval` folder. Keep `hud_reference.png` and `map_reference.png` beside the script. `--quiet` hides progress. `--frame` checks an image only and never awards video success. Ctrl+C interrupts safely; any resulting INCOMPLETE JSON is not a successful evaluation. ## Results - `PASS`, exit 0: 60 consecutive accepted seconds were found. - `FAIL`, exit 1: no qualifying interval was established, without unreadable or missing/late samples preventing a conclusion. - `INCOMPLETE`, exit 2: no PASS and unreadable content, missing/late fine samples, missing files/dependencies, decoding errors, or interruption prevented completion. Read `scope` as well as `status`. `longest_matching_seconds` counts accepted bins; `matching_start_seconds` and `matching_end_seconds_exclusive` describe the winning bin interval. Counts distinguish coarse and fine samples. Fine checking stops at first PASS, so longest duration is the longest measured before stopping, not necessarily the maximum possible duration in the entire video. ## Capture, file separation, and reference evidence The host provisions Mindustry v159, native Windows Computer Use, and OBS Studio on the solver machine; Python dependencies plus native FFmpeg and ffprobe are provisioned in a separate evaluator environment. OBS Studio is the authorized host-operated recorder. Start it before handing the paused map to the agent and finalize the unedited capture after completion. | Role | File | |---|---| | Candidate recording | run_recording.mp4 | | Evaluator-only expert recording | reference_run_recording.mp4 | | Candidate checker result | evaluation_result.json | | Current reference checker result | reference_evaluation.json | | Current reference execution provenance | reference_provenance.json | All checker code, bundled reference assets, answer files, reference videos, and checker test files must stay evaluator-only. The host enforces this using a separate evaluator machine/workspace inaccessible to the solver, or separate OS accounts with access controls verified from the solver identity. Labels and folder names alone are insufficient. See HOST_SETUP.md. Solver-visible inputs contain only the map, generic offline reference, switch legend, and RECORDING_INSTRUCTIONS.txt. No text trace, final string, screenshot, or run report is required for scoring. ## Submitted reference evidence and its version The contributor's latest ALE review explicitly reports a submitted reference MP4 and a valid JSON recording PASS, scope VIDEO_FINAL_STATE_60S, and a matching span of 115.866666 seconds. That is submitted real-video reference evidence for the legacy contract. Its presence is acknowledged; local synthetic tests are a separate category of evidence. The review inspected container/JSON structure, but did not execute the checker or inspect all decoded pixels. The current sampled checker uses VIDEO_SAMPLED_FINAL_STATE_60S and stops at 60 accepted whole seconds. The reported legacy JSON is therefore not a result of this unmodified 3.0.0 checker. Preserve it as historical evidence; do not edit its scope or duration. The current reference must be the unmodified result of this checker on the same MP4. The validate_reference.py command above performs that run in a fresh directory and records UTC times, hashes, runtime versions, exit code, exact result bytes, and logs in reference_provenance.json and companion files. It adds no image-judging or gameplay requirement. REFERENCE_VALIDATION.md records the feedback-derived values and their source, the distinction between the two contracts, and how to preserve execution provenance. Upload the current reference_evaluation.json and reference_provenance.json alongside the original reference_run_recording.mp4. Generated plans, declarations, and synthetic tests do not replace that result. A static documentation revision does not assert that a new execution occurred. ## Tests and submission updates `TEST_RESULTS.md` reports real executions of synthetic/fixture tests, with full synthetic-video diagnostics in `synthetic_video_results.json`. To reproduce: ```powershell python -m unittest -v test_checker python test_video_pipeline.py ``` Replace the previous checker ZIP with this whole package. Replace the ALE Task Description and Verification fields with `Task_Description.txt` and `Verification.txt`; replace separately uploaded EVAL_README.md and RECORDING_INSTRUCTIONS.txt with this package's versions. The map, legend, generic game documentation, and actual source recording do not need changes for this sampling revision. Use the reference validation command above when the submitted report has the legacy scope. No re-recording is required for this documentation reconciliation.