{"submission_id":"ffb86b91-a5e0-480e-ace0-fda6f4ae652f","submission_revision_id":"b5584976-f121-4bcf-8f77-543785ebb803","revision_version":1,"submission_status":"submitted","submitted_at":"2026-10-10T06:06:44.818129Z","submitter_name":"Ipek Oztas","submitter_email":"ipek_oztas@brown.edu","task":{"status":"submitted","version":1,"extra_fields":{},"submitted_at":"2026-10-10T06:06:44.818129+00:00","industry_domain":"Computer Science (Visual Computing)","task_short_name":"water_pouring_camera_occlusion_instance_1","operating_system":"Linux, MacOS","task_description":"**Objective:** Reconstruct a water-pouring scene and estimate how much water was poured, retained in the cup, and spilled while the camera was looking away.\n\n**Starting State:** You have `reference.mp4`, a six-second video in which the camera turns away for approximately 1.2 seconds during pouring; `scene_facts.json`, containing vessel dimensions and modeling assumptions; and `camera_poses.json`, containing the camera’s position and orientation for each frame.\n\n**Instructions & Rules:** Use the visible water levels before and after the camera movement to estimate the volume changes between frames 66 and 120. Rebuild the scene and animation in Blender using a volume-conserving model. Do not assume that pouring continues at a constant rate while off-screen. Account for cup capacity and overflow, and use the supplied camera path. The reference uses an analytical volume model, so a full fluid simulation is not required. The exact hidden flow timing cannot be uniquely recovered and will not be scored.\n\n**Deliverables:** Submit `answer.json` containing `unobserved_poured_ml`, `unobserved_retained_ml`, `unobserved_spilled_ml`, and the boolean `overflow_occurred`; your reconstructed `match.blend`; and `match.mp4`, exported at 800 × 600, 30 fps, for 180 frames.","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"Verify that the submission contains:\n\n- `answer.json` with finite, nonnegative numeric values for `unobserved_poured_ml`, `unobserved_retained_ml`, and `unobserved_spilled_ml`, plus a JSON boolean `overflow_occurred`.\n- A readable `match.blend`.\n- A decodable `match.mp4` containing exactly 180 frames at 30 fps and 800 × 600 resolution.\n\nCompare the submitted volumes with the private `hidden/answers.json`. All volume changes refer to frames 66–120.\n\n| Criterion | Weight | Full credit | Zero credit |\n|---|---:|---|---|\n| Absolute poured-volume error | 35% | ≤8 mL | ≥32 mL |\n| Absolute retained-volume error | 25% | ≤5 mL | ≥20 mL |\n| Absolute spilled-volume error | 25% | ≤12 mL | ≥40 mL |\n| Conservation residual: \\|poured − retained − spilled\\| | 10% | ≤2 mL | ≥15 mL |\n| Overflow classification | 5% | Matches reference | Does not match |\n\nFor each numerical criterion, interpolate linearly between full and zero credit and clamp the result to [0, 1]. Sum the weighted component scores using `hidden/score.py`.\n\nStrict success requires all deliverable checks to pass and a numerical score of 1.0. Missing or invalid answer fields fail validation.\n\nDo not compare exact hidden flow timing, overflow-onset timing, individual droplets, or pixel-perfect liquid shapes: these are not uniquely recoverable from the supplied observations. The numerical scorer evaluates `answer.json`; file readability and video-format checks must be performed separately.","software_and_version":"MPM-lite 0.1.0, Blender 5.2.1 LTS, Python 3.12.14, NVIDIA Warp 1.10.1 (CPU mode), NumPy 2.3.5, SciPy 1.18.1.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"8579d208-3d35-487a-aaa4-217254dffff7","filename":"reference.mp4","category":"input_materials","content_type":"video/mp4","size_bytes":227770},{"file_id":"c1678170-1677-4ab8-a731-7a66c35d7504","filename":"scene_facts.json","category":"input_materials","content_type":"application/json","size_bytes":2270},{"file_id":"a22b80c3-85a1-47f0-a4f7-195ac15d20f3","filename":"camera_poses.json","category":"input_materials","content_type":"application/json","size_bytes":93922},{"file_id":"6d7430ce-5629-4db6-87ff-e89ebeda6817","filename":"TASK.md","category":"input_materials","content_type":"text/markdown","size_bytes":3345},{"file_id":"b9c470b2-a442-4dd6-a096-3e2ee2d8006a","filename":"score.py","category":"reference_output","content_type":"text/x-python-script","size_bytes":1570},{"file_id":"757624c4-78de-43a6-aba9-1560bb2140dc","filename":"answers.json","category":"reference_output","content_type":"application/json","size_bytes":171}],"deep_review":{"overall_verdict":"Borderline Accept","review_text":"**Overall Verdict:** Borderline Accept — The volume-inference task is coherent and well specified, but reconstruction acceptance criteria need a small clarification.\n\n**Completeness Check:**\n- Instruction: Complete — Defines endpoint frames, conservation assumptions, deliverables, and which unidentifiable timing details must not be scored.\n- Inputs: Complete — Geometry and camera JSON are readable and consistent; the reference video is structurally valid, with visual verification limited as noted below.\n- Toolchain: Complete — Compatible open-source Blender and Python tools provide a viable analytical-model workflow without requiring fluid simulation; exact listed releases were not independently verified.\n- Reference output: Complete — Evaluator-only answers contain valid fields and conserve volume. The scorer’s source agrees with the stated numerical thresholds and weights.\n- Rubric: Unclear — Numerical scoring is objective, but readability and video-format checks alone do not establish that the required scene, camera path, or volume-conserving animation was reconstructed.\n\n**File and Task Reachability:**\nAttachments are **partially_reachable**; the task is **conditionally_reachable** pending reconstruction acceptance criteria.\n\nAll six attachments were inspected: `reference.mp4`, `scene_facts.json`, `camera_poses.json`, `TASK.md`, `score.py`, and `answers.json`. Their byte counts match the manifest. Text files are valid UTF-8, JSON schemas are consistent, and all 180 camera matrices are finite and geometrically valid. Cup capacity matches the supplied dimensions. Inputs and evaluator-only references are separately categorized.\n\nThe self-contained H.264 video has bounded media samples and container metadata indicating 180 frames, 30 fps, 800 × 600, and six seconds. No video decoder was available, so actual decoding, visible water levels, graduation legibility, and reference-to-answer agreement remain unverified. This is a reviewer-tooling limitation, not an established submission defect. Uploaded code was inspected without execution.\n\nThe completion path is endpoint measurement, analytical volume accounting, Blender reconstruction, rendering, numerical scoring, and separate artifact validation. Meaningful reconstruction validation remains underspecified.\n\nLicense keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone. No affirmative evidence supports a policy label. Public-benchmark provenance could not be independently checked because approved search was unavailable.\n\n**Recommended Changes:**\nThe conservation model, endpoint definitions, and exclusion of unknowable hidden timing are well designed.\n\n- Define minimum reconstruction acceptance checks for recognizable scene geometry, supplied camera motion, visible endpoint water levels, and consistency with the reported volume ledger. Keep exact hidden timing, droplets, and pixel-perfect liquid shapes excluded.","reviewed_at":"2026-10-10T06:11:59.352472Z"},"claim":null}