{"submission_id":"a7aff377-7ed6-41ee-aa28-745fd4b92778","submission_revision_id":"0869f2db-4311-452b-834a-79a485c6dcb2","revision_version":3,"submission_status":"under_review","submitted_at":"2026-10-02T04:00:36.120777Z","submitter_name":"Rongjie Huang","submitter_email":"zjurongjiehuang@gmail.com","task":{"status":"submitted","version":3,"extra_fields":{"difficulty_self_test":{"model":"GPT-6 and Opus 5.5","harness":"ALE harness","estimated_score":"<1%"}},"submitted_at":"2026-10-02T04:00:36.120777+00:00","industry_domain":"Video Production / Post-production (finishing and compositing)","task_short_name":"blender_kerry_kayak_finishing_instance_1","operating_system":"Linux","task_description":"Objective: Finish a 30-second Kerry kayak travel-series sting in Blender 5.0.1 by restoring a shuffled voice-over edit, matchmoving and rotoscoping the guides into the supplied plate, matching exposure and timing the approved lower-third, then delivering a re-renderable Blender project and master.\n\nStarting State: A Linux environment with Blender 5.0.1. The task folder is base/, with eight original files in input/ and an empty output/ directory. Input files and software installation instructions are provided in kerry_input_and_software.zip.\n\nInstructions & Rules:\nYou are the finishing editor at a small travel-series post house. Finish the 30-second Kerry kayak sting in Blender 5.0.\n\nTask folder: `base`\n\nInputs (`input/`):\n- `clip_a.mp4`, `clip_b.mp4`, `clip_c.mp4` — three parts of one continuous voice-over piece, shuffled and renamed. The voice-over was cut mid-sentence, so the spoken words are the reliable way to restore the order.\n- `plate_path.mp4` — second-unit plate for the composite: a handheld walk up a footpath, 29.97 fps.\n- `layout_marker_first_frame.png` — the plate at the first composite frame with a cross on the spot of ground where the guides stand (the placement numbers are in `brand_board.md`).\n- `lower_third_kerry.png` — the approved lower-third, a full-frame 1920×1080 overlay.\n- `client_brief.txt` — the producer's note.\n- `brand_board.md` — the finishing spec. Its numbers and rules are the delivery contract.\n\nSoftware: Blender 5.0.1. The supplied reference and standalone evaluator were actually run on Linux.\n\nWhat to deliver in `output/`:\n1. `kerry_kayak_master.mp4` — H.264 video with AAC stereo audio, 1920×1080, 25 fps, built to `brand_board.md`: restored order; the opening shot composited into the plate with the guides planted on the path (matchmoved, including the camera's roll), held out by the wall, the passing post and the reeds in front of the lens, and following the plate's exposure; the lower-third on the spoken county name.\n2. `final.blend` — the Blender project that renders that master headless from the files in `input/` alone. Roto must be done with Blender's own tools (Mask data-blocks, drawn and/or tracked in Blender; keying nodes may help with the occluders); pretrained segmentation or matting models are not allowed.\n3. `posters/composite.png` — the master frame at 00:00:03:00 (frame index 75, counting the first frame as 0).\n4. `posters/lower_third.png` — the first master frame on which the lower-third is visible.\n5. `posters/end_card.png` — the last frame of the master.\n6. `qc_notes.txt` — three lines:\n   - `ORDER: <first file>, <second file>, <third file>`\n   - `LOWER_THIRD_IN: HH:MM:SS:FF` (25 fps timecode of the first lower-third frame, starting from 00:00:00:00)\n   - `COMPOSITE_METHOD: <one sentence on how you tracked and masked the shot>`\n\nA master stitched with ffmpeg or composited outside Blender will not be accepted.\n\nUse the supplied input files unchanged. The project may read only those original media files; added precomposites, embedded generated mattes and altered packed images are not allowed. All 160 composite frames are evaluated.\n\nSource and reference: The three source clips are filmed footage; the footpath plate and camera truth are procedural for this simulated brief. Rongjie Huang confirms that the exact packaged reference deliverables were commissioned from human production personnel. The Blender Masks were manually drawn or rebuilt and checked by a human. The final editable reference project, rendered master, posters, and checking/acceptance were completed as commissioned human work. This production statement is supplied by the named contributor and is tied to the listed file hashes; the production personnel have not been publicly named, and measured working time has not been supplied. Contributor-confirmed manually drawn or rebuilt source-mask spline controls, with human checking. The reproducibility scripts apply bounded classical Sobel normal edge snapping within3px, then native Blender Mask interpolation and rasterization. No pretrained segmentation model or prior mattes are consumed.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Evaluate the delivered output/ folder with the attached standalone_scorer.zip. Its scorer/README.md contains the complete rubric and execution instructions; the scorer was actually tested with Blender 5.0.1 on Linux. This is a task-content/reference/rubric contribution with a standalone evaluator, without a claim of tested ALE framework integration.\n\n### Gates (score 0)\n- Missing, undecodable or off-spec master (not 1920x1080, not 25 fps, no audio, >2 frames off 752), or `final.blend` that does not open headless in Blender 5.0.1 (`--factory-startup --disable-autoexec`), crashes it, or does not finish the probe within 30 minutes.\n- `final.blend` reads any picture file (image or movie) from outside `input/` (external mattes and pre-rendered composites).\n\n- Every decoded frame, including the opening composite, must retain black pillars outside the supplied lower-third PNG. At480x270, left columns0..103 and right376..479 are checked in2x2 cells; maximum per-channel cell mean must be<=8/255. The original PNG nonzero-alpha support, dilated2 small pixels for normal encoding bleed, is excluded only when the actual supplied overlay is detected. A failing frame is a score-zero gate. The small local tolerance permits codec noise while preserving required black pillars.\n\n### Score = composite x QC factor\n- Composite (all 160 frames of the opening shot, against the exact camera/occluder truth of the synthetic plate):\n  - track 30%: the guides' placement in the master is recovered by masked template matching (offset, scale, roll); per frame the mean displacement of the visible guides' pixels from the truth, full at 3 px, 0 at 12 px (full resolution);\n  - roto 30%: candidate foreground = pixels that differ from the plate frame; IoU against the visible guides (0 at 0.55, full at 0.85) outside a +-2 px don't-care band around the guides' edge and the occluders' soft edges, frames with less than 25% of the guides visible skipped;\n  - holdout 25%: inside opaque occluder interiors behind the guides (wall, post, reeds; guide alpha >0.9, occluder alpha >0.85) the share that is closer to the plate than to the guides (0 at 60%, full at 95%);\n  - exposure 15%: guides' luminance ratio to the reference, full within 6%, 0 at 25%.\n  - any of the four below 0.5 marks the shot undeliverable: composite capped at 0.5.\n- QC factor = 0.5 + 0.5 x mean(edit, lower-third, project):\n  - edit: all 592 restored picture frames at 270x270: grayscale SSIM>=0.75 and each RGB channel MAE<=8/255 against original source; the allowed lower-third region is checked by its separate alpha-composite fidelity test. Source audio is globally aligned within60ms and checked over its entire duration using 200ms windows stepped every100ms, including quiet intervals and the last partial window; no sampling gaps. Duration within one frame; aligned boundary samples outside the candidate are treated as zero and checked against the complete reference window, so original endpoint silence is allowed but missing audible samples are measured; voiced windows (source RMS>=0.001) require correlation>=0.70, RMS gain within1dB and normalized error RMS<=0.65; quiet windows require absolute error RMS<=0.003;\n  - lower-third: in-frame 353 (full within 3 frames, 0 at 12), 100-frame hold, fidelity over every detected overlay frame and every pixel with alpha>0. Expected RGB is the supplied unchanged PNG alpha-composited over the corresponding original source picture and black pillars, with premultiplied RGB and alpha box-averaged to480x270. Mean RGB error<=8/255 is full fidelity, >=40/255 is zero, linearly interpolated between;\n  - project: scene settings 20%, complete source movie+sound strips in order in the active timeline or live reachable nested SCENE/META strips 20%, three frames re-rendered from `final.blend` matching the master 30%, posters 10%, QC note 10%, delivery encoding 10% (H.264 video, AAC audio, 2 channels).\n\n### Caps (total <= 0.25)\n- Composite score below 0.10; spoken order not restored (audio windows < 50%); roto not done with Blender Mask data-blocks. At composite frames40,70,150, the evaluator renders each live Mask use bypassed. VSE MASK modifiers are muted; compositor Mask nodes are also replaced temporarily by constant alpha0 and1. Redundant live MASK modifiers directly on the original guide source clip are additionally bypassed collectively while preserving plate holdouts. At least one bypass must satisfy all three averaged checks: changed pixels within24px of the candidate guide foreground cover>=25% of that foreground; >=35% of all changed pixels lie within96px of it; and newly revealed original field background outside the full reference guide silhouette dilated24px covers>=10% of full guide area. A revealed pixel must have RGB distance to the matching plate<=28 before bypass, >28 afterwards, increasing by>=12 (Euclidean RGB in0..255). This directional test verifies that a Blender Mask isolates the guides rather than merely masking plate occluders. Strip, modifier and Mask names do not determine credit; all temporary changes are restored before full replay.\n\n### Infrastructure errors (raised, not scored)\n- Hidden reference missing or malformed (schema, finite numbers, truth rows and mattes are validated before any candidate file is read).\n- Blender 5.0.1 missing or unable to run a one-line factory-startup script on the VM.\n\nOriginal input SHA-256 values are verified against the hidden inventory before candidate evaluation. Linked input/output paths and unapproved media are rejected.\nRestored-edit picture pass fraction below50% caps total at0.25. Unused Blender generated-image data-blocks are not rejected.\nSoft out-of-focus holdout is checked by recovering occluder opacity from master RGB versus the supplied plate and exposed guide inside guide interiors (plate alpha0.15..0.85, RGB contrast>60). Opacity MAE<=0.10 gets full credit,>=0.25 gets zero; holdout credit equals opaque-interior credit times this soft-transition credit.\nThe saved project is independently rendered over its complete timeline. Every decoded frame must match the submitted master (480x270 SSIM>=0.90 and whole-frame RGB MAE<=8/255; visible guide-region RGB MAE<=8/255 in the first160 frames), with matching frame count, resolution and fps. Full native audio must match duration within one frame and have correlation>=0.98 and gain within1dB after at most one frame of alignment. Missing or mismatching native full render is a score-zero gate. The evaluator controls export settings and output location; these exports are evaluator-owned evidence.\nQC notes credit covers the specified ORDER, LOWER_THIRD_IN and a nonempty single-line COMPOSITE_METHOD equally. The text is a delivery-format check; actual masking is verified independently by native render and live mask-effect tests.\nRestored-picture RGB tolerance allows ordinary decoding, scaling and H.264 quantization differences while preserving source colour. Overlay RGB tolerance likewise allows resampling and encoding; all nonzero-alpha support is included, not only opaque text. Timing, hold and fidelity are weighted50/30/20 within the lower-third score. The audio correlation threshold allows source/export AAC differences; RMS gain measures unity gain separately from coding noise. Audio windows cover every reference sample, including silence. Source edit picture/audio credits are their pass fractions, equally weighted.\nRe-rendered stills and posters require grayscale SSIM>=0.90 and RGB MAE<=8/255 against the corresponding master frame. Missing, unreadable or non-file QC notes receive zero QC-text credit rather than crashing candidate evaluation; a malformed candidate delivery is not a hidden-reference infrastructure error.\n\nOverlay fidelity also uses all detected frames at original1920x1080 resolution. An8x8 grid over every nonzero-alpha PNG pixel checks worst local RGB MAE (full<=16/255, zero>=40/255). In tiles containing at least4 support pixels and PNG premultiplied luminance SD>=8/255, normalized correlation of the supplied PNG texture with the rendered overlay after subtracting the source-background contribution is full at>=0.90, zero at<=0.50. This checks small lettering and graphic details that a whole-overlay average can dilute. Per-frame fidelity is the minimum of the global RGB credit, local RGB credit and local texture credit, averaged over all detected overlay frames. PNG-transparent pixels do not enter these tests. Original source RGB is upscaled by3 and alpha-composited at the specified full-frame placement; small local tolerances allow resampling and normal H.264 quantization.\n\nMask proof examines all relevant uses reachable from the active rendering scene, including reachable SCENE/META strips, compositor scene dependencies and nested compositor groups; there is no fixed Mask-use sampling cap. Unrelated unused planning scenes are permitted and cannot suppress evidence of a working source matte. Temporary bypass edits are restored before native export.\nHidden-reference preflight checks the published fixed clip order, section starts and positive lengths (covering752 frames), picture window, composite interval, numerical threshold ranges, lower-third timing and complete unique160-frame truth. Plate indices must conform to0.4s at30000/1001fps sampled on the25fps task timeline. Affine scale, roll and anchor must agree with truth geometry. Malformed types, impossible values, duplicate rows or inconsistent geometry raise ReferenceDataError before any candidate is inspected. These are evaluator-data checks and do not impose additional requirements on the solving agent.\n\nLower-third hold receives full credit only for one uninterrupted run of exactly100 decoded frames. Hold credit decreases linearly to zero at an absolute duration error of25 frames; multiple detected runs halve the hold credit. The separate lower-third onset tolerance remains full within3 frames and zero at12 frames.\n\nSquare pixels are a score-zero delivery gate: the submitted master and trusted native render must have SAR 1:1 (unset SAR defaults to square), and the active Blender render scene must have positive finite pixel_aspect_x/y with ratio1 within1e-6. Equal non-unit values, such as2/2, are equivalent square-pixel settings; unused scenes impose no additional restriction.\n\nNative-project replay also compares the saved project export against the submitted master over the supplied lower-third PNG alpha support in every decoded frame, including frames without a visible overlay. At full resolution the support is divided into8x8 pixel cells; cells with at least4 nonzero-alpha pixels must have mean absolute RGB difference<=8/255. Any failing cell/frame makes the project-replay delivery gate fail (score0). This preserves codec tolerance while detecting missing, additional or changed localized lettering, colour and timing. This compares native export with submitted video; lower-third correctness against the supplied PNG is scored separately.\n\nStereo source preservation is checked on corresponding left and right channels decoded at16000Hz, with one shared alignment from their summed corresponding-channel correlations (search within120ms; credit requires lag within60ms). Each channel uses the same complete200ms windows stepped100ms, duration within one frame, voiced correlation>=0.70, gain within1dB, normalized error RMS<=0.65, and quiet absolute error RMS<=0.003 described above. Stereo audio credit is the lesser left/right window pass fraction; the existing mono window credit is supplemental and final source-audio credit is the minimum of mono and stereo credit. The spoken-order cap uses this final credit. Native-project audio parity compares corresponding left/right channels with one shared alignment of at most one frame and duration within one frame; each voiced channel (reference RMS>=0.001) must have correlation>=0.98 and gain within1dB. A quiet channel must have absolute error RMS<=0.003. Failure in either channel makes native replay fail (score0). Channel order is preserved; opposite-phase errors cannot cancel through downmixing.\n\nMalformed native-probe JSON, including non-object roots/scenes, nonfinite numbers or wrongly typed accessed scene, strip, file and mask-use fields, fails the candidate probe gate with score0. It does not raise a hidden-reference infrastructure error. Valid unused scene data and equivalent positive equal pixel-aspect values remain permitted.\n","software_and_version":"Blender 5.0.1","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"1f5e2218-9595-426f-9ecd-5b02614c2829","filename":"kerry_input_and_software.zip","category":"input_materials","content_type":"application/zip","size_bytes":7469227},{"file_id":"b62ed2bd-e523-4684-ba75-9881e219ef43","filename":"standalone_scorer.zip","category":"reference_output","content_type":"application/zip","size_bytes":28338},{"file_id":"98851b91-8f85-4568-8ace-3d87d9b220c3","filename":"kerry_reference_outputs.zip","category":"reference_output","content_type":"application/zip","size_bytes":47072451},{"file_id":"81d307e6-b1db-4438-ba04-f93cb20db47a","filename":"kerry_evaluator_reference_readable.zip","category":"reference_output","content_type":"application/zip","size_bytes":84006240},{"file_id":"083e0ce2-617e-4031-b8a2-ff270d7f438c","filename":"model_trial_evidence.zip","category":"difficulty_self_test_evidence","content_type":"application/zip","size_bytes":7805},{"file_id":"72089773-d5e6-4c91-806b-1549cbdeab89","filename":"kerry_ale_submission_20261002f.zip","category":"reference_output","content_type":"application/zip","size_bytes":146686306}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — Revision 3 supplies a coherent, professionally substantive finishing task, complete reference artifacts, and an objective evaluation path with no material blocker identified.\n\n**Completeness Check:**\n\n- Instruction: Complete — The finishing spec defines placement, frame-rate conforming, masking, exposure, editorial timing, encoding, and all deliverables.\n- Inputs: Complete — All eight original files are present; their hashes match the hidden inventory and the duplicate task-material package.\n- Toolchain: Complete — Blender 5.0.1/Linux acquisition instructions and pinned scorer dependencies are supplied; execution records support single-machine feasibility.\n- Reference output: Complete — The readable reference contains the project, master, three posters, QC notes, complete 160-frame truth, and 480 reference masks. All 489 reference files match the supplied inventories.\n- Rubric: Complete — The scorer covers every composite frame, restored picture and stereo audio, lower-third fidelity, live-mask behavior, and complete native-project replay, with explicit gates and partial-credit rules.\n\n**File and Task Reachability:**\n\nAttachments: reachable. Task: reachable.\n\nAll six manifest attachments were inspected: `kerry_input_and_software.zip`, `standalone_scorer.zip`, `kerry_reference_outputs.zip`, `kerry_evaluator_reference_readable.zip`, `model_trial_evidence.zip`, and `kerry_ale_submission_20261002f.zip`, including relevant nested archives. Archive safety, byte counts, supplied hashes, structured data, image data, media headers, native-project structure, representative visuals, and evaluator source were checked.\n\nThe completion path is original inputs → native Blender edit/composite → required deliverables → protected evaluator-owned replay and scoring. Solver inputs are separated from hidden answers and construction assets. Supplied execution evidence reports a full reference score and successful 752-frame replay; this was not independently reproduced. Full audiovisual playback and native rendering were unavailable in the review environment, and no uploaded code was executed. Tested ALE integration is explicitly outside the submission’s claim.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\n\nNone. The procedural plate is disclosed, and human reference production and checking are contributor-reported rather than independently authenticated. No affirmative evidence supports AI fabrication or a core synthetic-defect fixture. Public-benchmark status could not be independently checked because approved public search was unavailable.\n\n**Recommended Changes:**\n\nThe complete reference package and extensive replay checks support acceptance; no material changes are required. Optionally qualify the “<1%” difficulty estimate as historical: the attached trials used an earlier evaluator, and one zero score was affected by delivery filenames.","reviewed_at":"2026-10-02T04:13:42.525403Z"},"claim":null}