Paper deep dive
PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction
Shuai Shao, Jinchen He, Cunxin Fan, Zihan Guo, Shijian Wang, Qihan Ren, Kangning Zhang, Zunnan Xu, Haoyuan Li, Hao Wang, Jianghao Lin, Weiwen Liu, Weinan Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Building physical simulations from video requires coding agents to translate observations into executable models and refine them through tool use. We introduce PhysCodeGym, a benchmark for physically verifiable 4D reconstruction with $156$ tasks covering rigid-body, liquid, deformable, and wind-driven motion. Agents autonomously inspect videos, write and run Blender programs, and revise their scenes using simulation feedback. Six metrics combine rule-based measurements with rubric-based judgments to assess visual and physical reconstruction quality. Program inspection, scene measurements, and counterfactual interventions test physical consistency under original and modified conditions. Experiments with frontier coding agents reveal a tradeoff between artifact quality and completion across reasoning-effort settings, while harness comparisons and agent traces link verification behavior with reconstruction performance. Case studies show that matching visible objects and events can coexist with physical defects. Counterfactual tests further expose latent contact and support failures in reconstructions that pass ordinary physical checks, highlighting the gap between reproducing observations and building reliable physical models. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.17288v1
- Canonical: https://arxiv.org/abs/2610.17288v1
Trouble viewing inline? Open PDF directly →
Full Text
106,602 characters extracted from source content.
Expand or collapse full text
PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction Shuai Shao 1, * ,† , Jinchen He 1,* , Cunxin Fan 1,* , Zihan Guo 2,3,*,‡ , Shijian Wang 4 , Qihan Ren 1 , Kangning Zhang 1 , Zunnan Xu 5 , Haoyuan Li 2 , Hao Wang 6 , Jianghao Lin 1,‡ , Weiwen Liu 1,‡ , Weinan Zhang 1,‡ 1 Shanghai Jiao Tong University 2 Sun Yat-sen University 3 Shanghai Innovation Institute 4 Southeast University 5 Nanyang Technological University 6 Carnegie Mellon University Flags fluttering in wind Rotating arm triggering a domino cascade 0.00 s3.20 s4.00 s Milk pouring into a glass 0.00 s4.00 s7.97 s Figure 1. PhysCodeGym benchmarks coding agents on156video-to-Blender reconstruction tasks spanning rigid-body, liquid, deformable, and wind-driven motion. Examples pair reference frames (top) with agent reconstructions (bottom). Evaluation measures visual fidelity, dynamic behavior, and physical validity. Abstract Building physical simulations from video requires coding * Equal contribution. All four co-first authors agree that each may list their own name first when presenting this work. † Project lead. shaoshuai.ederson@sjtu.edu.cn. ‡ Co-corresponding authors. Weinan Zhang: wnzhang@sjtu.edu.cn. agents to translate observations into executable models and refine them through tool use. We introduce PhysCodeGym, a benchmark for physically verifiable 4D reconstruction with156tasks covering rigid-body, liquid, deformable, and wind-driven motion. Agents autonomously inspect videos, write and run Blender programs, and revise their scenes us- ing simulation feedback. Six metrics combine rule-based 1 arXiv:2610.17288v1 [cs.AI] 12 Oct 2026 measurements with rubric-based judgments to assess vi- sual and physical reconstruction quality. Program inspec- tion, scene measurements, and counterfactual interventions test physical consistency under original and modified con- ditions. Experiments with frontier coding agents reveal a tradeoff between artifact quality and completion across reasoning-effort settings, while harness comparisons and agent traces link verification behavior with reconstruction performance. Case studies show that matching visible ob- jects and events can coexist with physical defects. Counter- factual tests further expose latent contact and support fail- ures in reconstructions that pass ordinary physical checks, highlighting the gap between reproducing observations and building reliable physical models. Code is available at https:// github.com/ShaoShuai0605/PhysCodeGym. 1. Introduction Physics-aware reconstruction enables the creation of simulation-ready objects and scenes for applications in an- imation [55], gaming [22], manufacturing [57], and robot learning [34]. Specialized methods estimate scene geometry and physical properties within predefined representations and simulation pipelines [9,30,52,53]. More recently, cod- ing agents have emerged as a promising path forward by generating executable scene programs and revising them through execution and visual feedback [49, 61]. Recent frontier models, such as GPT-6 Astra, demon- strate substantially improved capabilities in executable 4D scene construction, creating an urgent need to quantitatively assess their physical fidelity. We argue that a benchmark for this purpose should meet three requirements. First, broad interaction coverage should span diverse processes such as contact, fluid flow, and deformation [5]. Second, hy- brid assessments should combine rule-based checks with rubric-based judgments [59]. Rule-based checks provide precise measurements but are limited to predefined crite- ria. Rubric-based judgments can assess broader task re- quirements [29,43], though they may miss physical viola- tions [32,36]. Third, agreement with observed motion and appearance does not ensure correct behavior under altered physical conditions [6,8,45,49,60,64], motivating factual and counterfactual evaluation. For example, hardcoded motion trajectories may make balls appear to collide even when contact is disabled. While prior benchmarks cover subsets of these design choices, we argue that considering them jointly provides a holistic view of the capabilities and limitations of frontier coding agents in 4D reconstruction. To this end, we introduce PhysCodeGym, a benchmark for evaluating coding agents on physically verifiable 4D re- construction (Fig. 1). Given a reference video, an agent writes a standalone Blender script to create an editable scene and render its animation. Our benchmark comprises 156 tasks spanning rigid-body, liquid, deformable, and wind- driven dynamics, with scenarios involving collisions, sus- pension, pouring, and cloth deformation. To enable systematic evaluation across all 156 tasks (Sec. 4.3), we provide object checklists, 578 timestamped event annotations, and task-specific physical requirements. The evaluation protocol combine these annotations with LLM-based and rule-based judges to compute six metrics. Five assess the submitted video for structural and perceptual similarity, object accuracy, event completion, and motion overlap. To support event completion scoring, we specify failure and uncertainty criteria and aggregate three indepen- dent judgments per submission by majority vote. The sixth metric, physical validity, is assessed through a task-specific measurement that combines code and scene inspection with frame-by-frame analysis of native states, contacts, and forces. The scoring further incorporates counterfactual checks, in which copies of the scene are rerun under altered physical conditions. For example, our protocol redirects a billiard ball toward the table edge to check whether it falls after losing support. To verify that our protocol reliably detects phys- ical violations, we perform manual checks by perturbing submitted scenes and injecting known defects. We evaluate nine coding agents to characterize their strengths and failure modes in 4D physical reconstruction. We find that, while these agents can reconstruct recognizable objects, they often struggle to accurately capture motion and physical interactions (Tab. 2). Moreover, reconstructions with perfect object and event scores can violate physical constraints, including energy conservation (Sec. 5.2). Our harness-level comparison further reveals that, at high effort, Codex achieves higher visual and event scores, while Claude Code has slightly higher motion overlap (Fig. 5). Reasoning effort presents an additional tradeoff: low effort completes more tasks and leads overall under a fixed budget, whereas max effort scores higher among successful tasks (Sec. 5.2). Our contributions are: •A benchmark for coding agents that building physical simulations. PhysCodeGym uses156tasks to test coding agents’ ability to build physics-aware Blender scenes. The tasks cover gravity and support, contact and friction, fluids, deformation, and wind-driven motion. •A protocol for joint visual and physical evaluation. We combine LLM-based and rule-based judges with factual and counterfactual checks. Our diagnostic cases include reconstructions that pass object and event checks but fail native-scene audits or targeted interventions. •An empirical study of coding agents for physical recon- struction. We evaluate leading agents on PhysCodeGym and study how their results vary with harness choice and configured reasoning effort. We highlight their strengths, failure modes and shortcut behaviors, informing future development of physically aware coding agents. 2 Table 1. Comparison of programmatic scene generation and reconstruction benchmarks. Task distinguishes generation (Gen.) from reconstruction (Recon.); Input specifies the conditioning modality and source (Syn.: synthetic). Visual Eval. assesses rendered appearance or spatial layout. Under Behavioral Fidelity, Event Eval. checks target actions or outcomes, and Motion Alignment compares motion against a reference. Under Physical Implementation, Internal Physics Checks inspect scene structures, physical parameters, or simulation states, and Mechanism Dependence uses matched physical interventions to test whether the modeled mechanisms drive the observed behavior. Harness lists reported execution workflows or interfaces.✓included,✓limited scope,✗not evaluated under the column definition. BenchmarkTaskInput Visual Eval. Behavioral FidelityPhysical Implementation Harness Event Eval. Motion Alignment Internal Physics Checks Mechanism Dependence Code4D [67]Gen.Text✓✗Multi-agent code loop 4DBuildBench [32]Gen.Text✓✗✓✗Multi-agent MCP PhysCodeBench [56]Gen.Text✓✗✓✗Direct codegen. / SMRF SceneActBench [68]Recon.Syn. frames✓✗✓✗Shared MCP VisPhyBench [31]Recon.Syn. frames✓✗✓✗Codegen. + repair MPMWorlds [26]Recon.Syn. video✓✗✓✗Scaffolded codegen. BVB [48]Recon.Real video✓✗✓✗Mini-BVB 4DCodeBench [45]Recon. Real + Syn. video ✓✗Native CLIs / Stirrup OursRecon.Video✓Native CLIs (all models) 2. Related Work Video-based Physical Reconstruction.Video-based phys- ical reconstruction requires recovering scene geometry, phys- ical properties, and dynamics from observations [44,62]. Existing methods couple geometry and appearance represen- tations with differentiable continuum simulation, mesh-free reduced-order simulation, or spring–mass dynamics to re- construct simulatable scenes for subsequent prediction and re-simulation [9,21,30,70]. Language models also support physical scene construction:∆YNAMICS generates struc- tured scene configurations, evaluates reconstruction through MuJoCo re-simulation, and supports editing gravity and ini- tial velocities [24]. OVOW recovers instance meshes, poses, and deformations from monocular video, using ground-plane estimation and contact projection to assemble scenes for downstream simulation [10]. These methods study how to recover simulatable scenes from video. PhysCodeGym turns this reconstruction objective into an evaluation task for cod- ing agents, testing whether their generated scene programs satisfy both reference observations and task-specific physical requirements. Programmatic Scene Construction and Evaluation.Ear- lier inverse-graphics systems represented scenes with proba- bilistic programs in Picture and structured scene descriptions in Neural Scene De-rendering [28,54]. Programmatic meth- ods perform 3D generation and editing through code [13,18, 19,27,47,50,58,66], with benchmarks covering graphics editing, text- or image-conditioned 3D modeling, and image- to-scene inverse rendering [12,14,15,33,69]. Thinking in Blender reconstructs single images as editable Blender programs by progressively refining geometry, materials, com- position, and lighting [17]. VIGA extends code iteration and visual feedback to dynamic scenes [61], while Code2Worlds generates dynamic scenes from text and refines them through motion critique and code revision [67]. For physical im- plementation, SimWorlds’ 4DBuildBench separately eval- uates visual output and internal physics, auditing solver configurations, constraints, and animation substitutes [32]; PhysCodeBench evaluates simulation behavior through phys- ical assertions and conservation diagnostics read from engine states [56]. These dynamic generation benchmarks check scenes and physical implementations against textual require- ments; video reconstruction additionally requires agreement with reference observations of appearance and behavior. Video-to-Program Reconstruction Benchmarks.Video- to-program benchmarks evaluate agents’ ability to recover scene structure and dynamic behavior. BVB reconstructs real videos as Blender animations, evaluating spatiotem- poral fact retention and representation-level similarity of layout and dynamics under a protocol that prohibits physics simulation [48]. SceneActBench’s dynamic task evaluates layout and keyframed trajectory recovery with supplied as- sets [68]. VisPhyWorld and MPMWorlds recover executable simulations from synthetic visual observations, studying physical-parameter and motion reconstruction, and future prediction of multi-material dynamics, respectively [26,31]. The closely related 4DCodeBench reconstructs real and syn- thetic videos as executable dynamic scenes, evaluating ap- pearance, geometry, and motion alongside state/event ques- tions and mesh-integrity diagnostics, with direct 3D ground- truth comparisons on its synthetic subset. However, its task permits keyframes, prescribed motion, and physical simulation as alternative dynamic representations, primar- ily evaluating submissions by reconstruction fidelity [45]. Code as Worlds constructs, verifies, and refines executable worlds from text or real videos, supports re-execution un- 3 der changed physical conditions, and uses these worlds for physical-reasoning supervision [49]. Its emphasis is on dis- covering executable worlds and using them for physical reasoning; PhysCodeGym instead makes the physical im- plementation of core motion a requirement of video recon- struction and tests mechanism dependence through indepen- dent evaluation of submitted artifacts. Evaluation combines reference-event and motion alignment, scene and runtime checks, and matched interventions on contacts, support, or actuation, comparing behavior before and after intervention to determine whether reconstructed motion depends on task- required physical mechanisms. Table 1 compares representative benchmarks by task, in- put, visual evaluation, behavioral fidelity, physical imple- mentation evaluation, and execution harness. 3. PhysCodeGym: Dataset and Task 3.1. Dataset The dataset contains156tasks covering rigid-body, liquid, deformable, and wind-driven motion; detailed category defi- nitions and task statistics are provided in Appendix A.1. Design principles.PhysCodeGym rests on four principles. (P1) Visible physical causation: the motion in every refer- ence clip is caused by a visible physical process whose cause can be inferred from the footage, such as falling, rolling, collision, rotation, swinging, liquid pouring and splashing, deformable motion, and wind-driven motion. (P2) Simula- tor expressibility: the process can be expressed within the simulation capabilities of Blender; combustion, cutting, tear- ing and fracture, siphoning and absorption, liquid mixing, and magnetic forces are excluded. (P3) Mechanism-level taxonomy: every task has one mutually exclusive main cate- gory, and rigid-body and liquid tasks are further divided into subcategories by their dominant motion mechanism, so that results can be analyzed per mechanism. (P4) Observation- grounded annotation: annotations describe only what the reference video shows, with timestamped evidence; what it does not show, such as the path of an object while it briefly leaves the frame, is not required. Collection pipeline. Following (P1) and (P2), we collect candidate videos from Pexels [42], Physics-IQ [37], BAAI- DataCube Physics-aware Videos [4], and IRIS [25]. We combine automated checks, visual agent review, and manual selection to identify clearly observable physical processes that can be reconstructed in Blender, prioritizing stable cam- eras, simple backgrounds, unobstructed events and diverse physical phenomena. From videos at least8seconds long, we select a continuous8-second segment with pronounced motion and standardize it to3840× 2160resolution,30FPS, and240frames while preserving the original playback speed. After deduplication and final manual review, we retain156 clips, each defining one task. Annotation. Each task includes three structured annota- tions that specify the objects to reconstruct, the events to reproduce, and the physical constraints to satisfy. (i) The ob- ject checklist describes object categories, counts, appearance attributes, and initial spatial layout. (i) Event annotations identify the participating objects and specify key actions, their temporal order, and the attainment and persistence of final states, supported by timestamped video evidence. (i) Physical requirements are recorded in manually annotated rubrics that define permitted initial conditions, actuation, and equivalent representations, together with criteria for check- ing the physical mechanisms of the reconstructed scene. For selected tasks, counterfactual verification alters forces or con- tact conditions and re-simulates the scene to check whether its response is consistent with the task’s physical require- ments. These annotations support object accuracy, event score, and physical validity, respectively (Sec. 4.3). We draft object checklists with model assistance and event annotations from reference observations, then check both using reference videos and unrelated-video controls. 3.2. Task Definition Every task in PhysCodeGym poses the same problem, recon- structing the physical process recorded in a video, and each submission is scored along two axes, each targeting a distinct capability: visual reconstruction (reproducing what the refer- ence video shows) and physical reconstruction (reproducing the dynamic process through modeled physics). Figure 2 illustrates the reconstruction and verification pipeline. Physical-process reconstruction. A video records only one presentation of a dynamic scene from one viewpoint. To understand why the process unfolds as it does, or how it would change under a different viewpoint or a modified condition, one needs not another similar video but a model of the physical process that can be run, inspected, and modi- fied again. Executable worlds support re-simulation under changed conditions and rendering from new viewpoints [49]. PhysCodeGym therefore sets the reconstruction target to be a program. Given a reference video, a coding agent must write a scene program that starts from an empty Blender scene, builds geometry, materials, cameras, and lighting, and sets the physical properties, initial conditions, constraints, and forces of the objects; executing it yields a native scene that evolves over time, together with its rendered sequence. The submitted reconstruction is the program and the native scene it produces; the rendered sequence is only one pre- sentation of this process from the reference viewpoint. We choose Blender because it provides modeling, rigid-body, cloth, fluid, and force-field simulation, and rendering in a 4 Stage 1 Task InputStage 2 Agent Reconstruction Models Harness coding agent Simulated Video compare scene.py Real Video Reference video import bpy # scene construction ... ... import bpy # scene construction ... ... Reference evidence for evaluation Revise Code Final submission Stage 3 Independent Verification Factual Evaluation Reference LLM as Judge Object + Event Rubrics Counterfactual Evaluation Scene intervention Inspect Copy Edit Same initial state Table support preserved Re- simulate Native-state trajectories t = 3.50 s Contacts enabled Target responds A: incoming ball; B: middle ball; C: target ball Rule-based Judge Measured displacement (scene units, t = 3.50 s) Incoming motion preserved Target response disappears Contact dependence supported Scoped mechanism check 2.9 s AB C BCA ABC Condition AC 0.887 1.349 0 0.581 0 0 scene.py Rule-based Judge Schematic controls - Ball contact: ON + Ball contact: OFF 3.1 s3.5 s Reconstruction Ball contacts disabled Launch field disabled No launch A moves; C stays Overall Scores SSIM + LPIPS + Motion Visual Score SSIM | LPIPS | Object Physical Score Event | Motion | Validity Contact on Contact off Launch off Physical validity evidence + Launch field: OFF - Launch field: ON scene.py Figure 2. Reconstruction and verification pipeline of PhysCodeGym. Stage 1: A reference video defines the reconstruction task and supplies evidence for evaluation. Stage 2: A coding agent iterates between observing the video, writing a Blender scene program, running it, and comparing rendered previews, then submits the scene program. Stage 3: Independent evaluation combines rubric-based object and event judgments with image and motion metrics, while native-scene inspection and controlled re-simulation assess physical validity. In the illustrated contact test, disabling ball contacts preserves incoming motion but removes the target response; disabling the launch field removes the incoming motion as well. Visual and physical scores are reported separately. single environment [7], and because both the scene structure and the simulation state are accessible through its scripting interface, so an agent’s decisions about objects, contacts, and the causes of motion can be inspected directly in the scene. Valid reconstruction.A valid reconstruction must satisfy two conditions: (i) it reproduces the observations, match- ing the appearance, layout, and visible events of the refer- ence video; and (i) the observed motion is produced by the modeled physical process. The same footage can also be reproduced through keyframes, scripted poses, or hidden supports; such a scene may match the frames, but it does not explain the motion and does not respond correctly when conditions change. Prescribed trajectories can achieve high reconstruction scores while leaving physical mechanisms underconstrained [45]. The motion of core objects must therefore come from the physics solver, forces, constraints, and initial conditions. This excludes scripted trajectories and force fields with no counterpart in the video, such as an attractor that steers a ball along the observed path. Ex- plicitly modeled external actuators are allowed, provided that the objects they act on respond through physical interac- tion. Physical parameters are not always identifiable from the observed motion: in a modeled free-fall and inclined- sliding scenario, friction can be inferred while mass remains unobservable [23]. Monocular reconstruction also faces am- biguity in scale and unobserved geometry, for which physical dynamics can provide additional constraints [44]. PhysCode- Gym therefore does not require recovering the true values of these parameters or the exact hidden structure, and accepts any implementation that reproduces the observations and satisfies the physical requirements of the task. Observation and submission. The agent works in a ter- minal environment and uses command-line tools to observe the reference video and operate Blender. The observation tools return basic video metadata, specified frames, frame grids, and clips sampled at a given rate. Neither the source program nor the physical parameters of the reference scene are provided. The build tools run the scene program, inspect the saved scene structure, and render single-frame previews of the simulation state. The agent submits the scene program scene.py. The program must be self-contained, build the entire scene procedurally, and bake the simulation cache be- fore rendering. It outputs the native scene file and a rendered sequence of240frames at30FPS and640× 360, whose 5 duration matches that of the reference video,8seconds. The program may not decode media, access the network, spawn subprocesses, or import modules dynamically, which ensures that the scene is generated entirely by modeling statements. Before submission, a checking tool verifies the format and policy requirements and reports any problems, so that the agent can fix them before the run ends. The unified output specification allows every submission to be re-executed in an independent environment and compared with the reference video frame by frame. Scoring. We report separate visual and physical scores, their component metrics, and the overall score defined in Sec. 4.3. The visual score measures how closely the ren- dered sequence resembles the reference video on screen, and averages three visual metrics: structural similarity (SSIM) and perceptual similarity (LPIPS) compare the frames at the pixel and perceptual levels, and object accuracy checks the rendered objects against the object checklist. The physical score measures the dynamic process of the scene, and aver- ages three physical metrics: the event score checks whether the annotated events occur, the motion score compares where and for how long motion occurs, and physical validity checks the scene against the physical requirements. Except for phys- ical validity, which reads the program and the native scene, all metrics use only the rendered sequence. Each metric is also reported individually, which separates failures that a sin- gle number would conflate [20], such as a scene that matches the frames through scripted motion and a physically valid scene that misses the observed events. Metric definitions are given in Sec. 4.3. 4. Experiments 4.1. Setup We build the execution framework of PhysCodeGym on Har- bor [16], integrating our Blender reconstruction environment and benchmark-specific evaluation pipeline. Appendix A.3 describes this integration and its relationship to the agent harnesses used in our experiments. All agents work in the terminal environment of Sec. 3.2 and typically iterate between observing the reference, writing the program, running the simulation, comparing previews, and revising the scene. Each launch of a Blender process counts as one iteration, and the iteration limit constitutes the generation budget; observation commands do not count as iterations, but they consume running time and context, and the whole run is also bounded by a total time limit. Unless stated otherwise, all agents use high reasoning effort, with a run-time limit of3hours and at most20Blender launches per task. Each model is initially scheduled once per task. Audited serving-infrastructure failures are handled by sepa- rately recorded recovery runs, preserving the original trials; verifier-side failures are repaired by re-executing archived programs. Errors caused by the agent’s own requests, such as exceeding the context window or sending a malformed image, count as failures of the agent. A task that is run more than once is scored by its latest run that produced a rendered sequence, not by its best-scoring run; a task for which no run produced a rendered sequence receives zero. 4.2. Models We evaluate coding agents that autonomously write and run code in a terminal environment. The main compar- ison in Tab. 2 uses three harnesses: (i) Codex [38] for GPT-5.6-Sol and GPT-5.6-Luna [40], GPT-6-Luna [41], and GPT-6-Astra [39]; (i) Claude Code [2] for Claude Sonnet 5 [3], GLM-5.3-Flash [63], DeepSeek-V4.1-Flash [11], and Qwen3.8-Flash [1]; and (i) Kimi Code for Kimi K2.8 Pre- view, requested askimi-for-coding[35]. Prior work explores sampling, revision, and verification strategies for language agents [71], while studies of LLM reasoning show that the effectiveness of test-time computation depends on inference strategy and problem difficulty [46]. We also com- pare DeepSeek-V4.1-Flash at low, high, and max configured reasoning effort in Claude Code, and at high effort in Codex. 4.3. Evaluation Each submission is evaluated with three visual metrics, which compare the rendered sequence with the reference video, and three physical metrics, which assess the dynamic process of the scene (Sec. 3.2). Visual metrics. Structural similarity is the per-frame SSIM [51] between the rendered sequence and the reference video. Perceptual similarity isclip(1−d, 0, 1), wheredis the per-frame LPIPS [65] (AlexNet) distance. Object accuracy summarizes a vision-language model’s judgments against the object checklist as an aggregate pass rate. Structural and perceptual similarity measure closeness at the pixel and per- ceptual levels, while object accuracy focuses on object-level correctness and is insensitive to subtle differences in lighting and texture. Event score. A vision-language model judges the candi- date video against the event annotation item by item and cites time intervals in the candidate video as evidence. When judging, the model sees still images of each role from the ref- erence video, to confirm object identities, together with the candidate video and supplementary temporal frames; it does not receive the action descriptions of the reference video, and decides whether each event occurs based on the candidate video alone. Each candidate is judged independently three times, and the majority verdict is taken per item. The event score is the weighted fraction of passed items; if a key event that defines the task fails, the score is capped at zero. For 6 Table 2. Visual and physical reconstruction scores on PhysCodeGym (higher is better). Metric definitions and aggregation follow Sec. 4.3. Best and second-best scores are bold and underlined. HarnessModel VisualPhysical S tot ↑ S SSIM ↑ S LPIPS ↑ S obj ↑ S vis ↑ S evt ↑ S mot ↑ S val ↑ S phys ↑ Codex GPT-6-Astra0.6100.5610.9090.6930.8050.3240.7680.6330.663 GPT-5.6-Sol0.5500.4480.8350.6110.6870.2100.6900.5290.570 GPT-6-Luna0.5180.3630.6330.5050.3930.1280.5550.3590.432 GPT-5.6-Luna0.5060.3550.6390.5000.4480.1570.5640.3910.446 Claude Code DeepSeek-V4.1-Flash0.5140.3980.6150.5090.4460.2000.5220.3910.450 Claude Sonnet 50.5130.3150.4810.4360.3680.1110.5590.3470.392 Qwen3.8-Flash0.3540.2390.4100.3340.3070.0900.3500.2520.293 GLM-5.3-Flash0.2860.2260.3620.2910.2400.1030.2950.2130.252 Kimi CodeKimi K2.80.4350.3040.4770.4050.3490.1100.4480.3020.354 example, if the ball does not roll around the tray, no credit is given even if the tray layout is correct. Motion score. The motion score adopts the motion mask of Physics-IQ [37]: grayscale frames are Gaussian-blurred and differenced against a running-average background, then thresholded and morphologically processed to obtain the motion region of each frame. We compare the reference and candidate masks with two IoU measures. Spatial IoU compares the regions where motion occurs at any point in the video. Weighted IoU also reflects how long motion lasts: withw p andˆw p the fractions of frames in which pixel pis moving in the reference and the candidate, it equals P p min(w p , ˆw p )/ P p max(w p , ˆw p ) . The motion score is the average of the two. This metric does not require recogniz- ing objects, and complements motion details not covered by the event annotation, such as the decay of swing amplitude or the extent of a liquid stream. Physical validity. An agent verifies physical validity against manually annotated task rubrics. It inspects the sub- mitted program and native scene, executes Blender/Python measurements, and supports each judgment with evidence. The main-table scoreS val evaluates the submitted program and its Blender scene, counting each task once per con- figuration. For selected tasks, counterfactual verification intervenes on forces, contacts, or initial conditions and re- simulates the scene to test the physical dependencies speci- fied by the rubric. Counterfactual checks are scored as crite- ria of the physical-validity rubric. The verification procedure and intervention example appear in Appendix B. Score aggregation.We report visual, physical, and overall scores: S vis = 1 3 (S SSIM + S LPIPS + S obj ), S phys = 1 3 (S evt + S mot + S val ), S tot = 1 2 (S vis + S phys ). (1) whereS SSIM ,S LPIPS , andS obj are structural similarity, per- ceptual similarity, and object accuracy, andS evt ,S mot , and S val are the event score, the motion score, and physical va- lidity. All three summaries are reported together with the component metrics. For a fully measured task,S tot is also the arithmetic mean of the six component metrics. In model comparisons, a task for which no valid artifact is submitted receives zero on every metric and is counted in the denomina- tor. For the provisional validity results, scores use the lower endpoints of the rubric-score intervals; existing-artifact tech- nical gaps are omitted while registered missing-artifact zeros remain included. The physical aggregate is first computed per task and then averaged on exactly the same available- case subset as validity. It therefore does not average the differently covered model-level columns of Tab. 2. For model-aggregate tables,S tot directly averages the displayed visual and physical score columns. The equiva- lence to a six-component mean over tasks requires all six metrics to share the same averaging set; it does not hold for differently covered columns merely by averaging their means. 5. Results & Analysis We first present visual and physical reconstruction scores and resource use, then examine reasoning effort, harness settings, and cases exposing gaps between visual fidelity and physical correctness. 5.1. Benchmark Results Table 2 reports six metrics and aggregate scores for nine model–harness configurations at default high effort, follow- 7 0.050.10.51510 (a) API cost per task (USD, log scale) 0.2 0.4 0.6 GPT-6-Astra GPT-5.6-Sol GPT-6-Luna GPT-5.6-Luna DeepSeek-V4.1-Flash Claude Sonnet 5 Qwen3.8-Flash GLM-5.3-Flash Kimi K2.8 2351020 (b) Tokens per task (M, log scale) 0.2 0.4 0.6 GPT-6-Astra GPT-5.6-Sol GPT-6-Luna GPT-5.6-Luna DeepSeek-V4.1-Flash Claude Sonnet 5 Qwen3.8-Flash GLM-5.3-Flash Kimi K2.8 Overall score S tot ProprietaryOpen-weightPareto Figure 3. Overall scoreS tot versus (a) estimated API cost and (b) recorded token use per task. Greater cost or token use does not consistently yield better reconstruction across models. ing Sec. 4.3. Overall performance. GPT-6-Astra leads in overall, vi- sual, and physical scores (0.663,0.693, and0.633); GPT- 5.6-Sol ranks second overall at 0.570. Performance across dimensions.All nine configurations score higher on objects than events; GPT-5.6-Sol scores 0.835and0.687, respectively. Motion remains weak, with GPT-6-Astra reaching only0.324. Rankings also vary: GPT- 6-Luna has a slightly higher visual score but lower event and motion scores than GPT-5.6-Luna. Resource use and quality. Figure 3 compares quality against estimated API cost and recorded token use. GPT- 5.6-Sol and GPT-6-Astra lie on the estimated-cost Pareto frontier: Sol scores0.570at approximately $3.40 per task, while Astra reaches 0.663 at $11.57, or 3.4 times the cost. Greater token use need not improve reconstruction. DeepSeek-V4.1-Flash records16.98M cumulative tokens per task, nearly3times GPT-5.6-Sol’s5.70M, but scores 0.450; its estimated cost is lower ($0.72 per task). Cumu- lative tokens include repeated context across interactions and cannot directly measure effective reasoning. We next examine how effort and harness settings relate to budget use. 5.2. Analysis We analyze how artifact quality and completion shape aggre- gate scores through DeepSeek-V4.1-Flash effort and harness comparisons, followed by physical failure cases. Reasoning effort and task completion. Figure 4 com- pares low, high, and max with the same limits of3hours and20Blender launches. Low leads overall (0.476) when missing artifacts receive zero. Among produced artifacts, max scores0.547versus low’s0.527, and also has the high- est mean on the shared subset. Gap decomposition links low’s aggregate advantage mainly to completion, while max has longer runtimes and more recorded output tokens. Ap- pendix C.3 details this completion–quality tradeoff under limited budgets. 0.35 0.50 0.65 0.80 0.494 0.378 0.645 (a) S SSIM S LPIPS S obj 0.2 0.4 0.6 0.517 0.192 0.531 (b) S evt S mot S val LowHighMax Reasoning effort 0.35 0.50 0.65 0.506 0.413 0.460 (c) S vis S phys S tot Score (higher is better) Figure 4. Comparison of reasoning effort levels. We evaluate DeepSeek-V4.1-Flash on (a) visual metrics, (b) physical metrics, and (c) aggregate scores. Under the same budget of3hours and 20Blender launches, low effort achieves the highest overall score, largely because it completes more tasks than max effort. Harness and verification. Figure 5 compares DeepSeek- V4.1-Flash at high effort in Claude Code and Codex. Codex scores0.479overall versus0.450, and0.550on events ver- sus0.446, with slightly lower motion overlap. In the rolling- wood and wine-pouring traces, Codex identifies problems, revises, reruns, and inspects outputs; Claude Code’s final revisions lack successful execution verification within the launch budget. This pattern may relate to proactive verifi- cation guidance, check timing, and budget allocation (Ap- pendix D.4). Physical failures beyond visual fidelity.Physical checks reveal flaws even when object and event judgments pass. All 8 00.250.500.751.00 S SSIM S LPIPS S obj S evt S mot S val S vis S phys S tot 0.514 0.398 0.615 0.446 0.200 0.522 0.509 0.391 0.450 0.524 0.391 0.673 0.550 0.187 0.547 0.529 0.428 0.479 Claude Code Codex Figure 5. Claude Code and Codex harness comparison. Visual, physical, and aggregate scores of DeepSeek-V4.1-Flash at high reasoning effort. In this comparison, Codex achieves a higher overall score, with the largest gain in event reconstruction despite slightly lower perceptual similarity and motion overlap. six reconstructions in Appendix D.2 scoreS obj = S evt = 1, yet program inspection and scene measurements expose fail- ures such as unexplained energy increases and interpenetra- tion. Changing support, contact, or initial conditions further exposes latent defects beyond the original trajectories (Ap- pendix D.3). 6. Conclusion We introduced PhysCodeGym, a156-task benchmark for evaluating how coding agents reconstruct physical processes from video as executable Blender programs. Visual and dy- namic metrics assess rendered outcomes; verification agents inspect scene programs and perform measurements against task rubrics, including counterfactual rubric checks for se- lected tasks. Our results highlight a gap between matching visible behavior and constructing physically valid scenes: ob- ject and event checks can pass while native-scene diagnostics identify physical inconsistencies. The current benchmark is limited to processes that Blender can express and the physi- cal checks specified by its rubrics. Future work can extend PhysCodeGym to more simulation engines and cover more physical interactions. References [1]Alibaba Cloud. Qwen3.8-Flash - Alibaba Cloud. https://w w.alibabacloud.com/help/en/model-studio/qwen3-8-flash. 6 [2]Anthropic. Claude Code by Anthropic | AI Coding Agent, Terminal, IDE. https://claude.com/product/claude-code. 6 [3]Anthropic. Introducing Claude Sonnet 5. https://w.anthro pic.com/news/claude-sonnet-5, 2026. 6 [4] BAAI-DataCube. BAAI-DataCube/Physics-Aware-Videos· Datasets at Hugging Face. https://huggingface.co/datasets/ BAAI-DataCube/Physics-aware-videos, 2026. 4, 14 [5] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. VideoPhy: Evaluating Physi- cal Commonsense for Video Generation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. 2 [6] Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, and Christian Wolf. CoPhy: Counterfactual Learning of Phys- ical Dynamics. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. 2 [7]Blender Foundation. Blender Python API. https://docs.blend er.org/api/current/. 5 [8] Kunlin Cai, Rui Song, Jinghuai Zhang, Kaiyuan Zhang, Pranav Bodapati, Alicia Yu, Fnu Suya, Mohammad Ros- tami, Jiaqi Ma, and Yuan Tian. What-If World: A Causal Benchmark for General World Models in Embodied Scenar- ios. arXiv preprint arXiv:2605.27589, 2026. 2 [9]Chuhao Chen, Zhiyang Dou, Chen Wang, Yiming Huang, Anjun Chen, Qiao Feng, Jiatao Gu, and Lingjie Liu. Vid2Sim: Generalizable, Video-Based Reconstruction of Appearance, Geometry and Physics for Mesh-Free Simulation.In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 26545–26555. Computer Vision Foundation / IEEE, 2025. 2, 3 [10]Junhao Chen, Boran Zhang, Mingjin Chen, Henghaofan Zhang, Saining Zhang, Congcong Zhu, Hao Zhao, Ruqi Huang, Zhihao Li, and Yufei Wang. One Video, One World: Turning Monocular Video into Physical 4D Scenes. In Com- puter Vision - ECCV 2026 - 19th European Conference, Malmö, Sweden, September 8-12, 2026, Proceedings, Part X, pages 271–293. Springer, 2026. 3 [11]DeepSeek.Introducing DeepSeek-V4.1-Flash: Smarter, Faster, More Efficient. https://w.deepseek.com/en/ news/deepseek-v4-1-flash/, 2026. 6 [12]Anna C. Doris, Jacob Thomas Sony, Ghadi Nehme, Era Syla, Amin Heyrani Nobari, and Faez Ahmed. CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Gen- eration. arXiv preprint arXiv:2605.10873, 2026. 3 [13]Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingx- uan Wang, Dingjie Song, Bo Li, Yan Hu, and Benyou Wang.BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-Improvement. arXiv preprint arXiv:2412.14203, 2024. 3 [14]Yipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, and Jindong Chen. 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling via Code. arXiv preprint arXiv:2606.01057, 2026. 3 [15]Yunqi Gu, Ian Huang, Jihyeon Je, Guandao Yang, and Leonidas J. Guibas. BlenderGym: Benchmarking Founda- tional Model Systems for Graphics Editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 18574–18583. Computer Vision Foundation / IEEE, 2025. 3 [16]Harbor Framework Team. Harbor: A Framework for Evaluat- ing and Optimizing Agents and Models in Container Environ- ments, 2026. doi:10.5281/zenodo.20953922. 6, 19 [17]Guangzhao He, Rundong Luo, Wei-Chiu Ma, and Hadar Averbuch-Elor. Thinking in Blender: Staged Executable In- verse Graphics with Vision-Language Models. arXiv preprint arXiv:2606.02580, 2026. 3 [18]Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong 9 Yue, David A. Ross, Cordelia Schmid, and Alireza Fathi. SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code. In Forty-First International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, pages 19252–19282. PMLR / OpenReview.net, 2024. 3 [19]Ian Huang, Guandao Yang, and Leonidas J. Guibas. Blender- Alchemy: Editing 3D Graphics with Vision-Language Mod- els. In Computer Vision - ECCV 2024 - 18th European Con- ference, Milan, Italy, September 29-October 4, 2024, Pro- ceedings, Part LXXXIX, pages 297–314. Springer, 2024. 3 [20]Swarnim Jain and Shangzhe Wu. RigidBench: Evaluating Rigid-Body Physics in Video Generation Models. arXiv preprint arXiv:2608.15555, 2026. 6 [21] Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, and Yunzhu Li.PhysTwin: Physics- Informed Reconstruction and Simulation of Deformable Ob- jects from Videos. In IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025, pages 7219–7230. IEEE, 2025. 3 [22] Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, and Chenfanfu Jiang. VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality. In ACM SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024, https://arxiv.org/abs/2401.166 63. 2 [23]Rama Krishna Kandukuri, Jan Achterhold, Michael Möller, and Joerg Stueckler. Learning to Identify Physical Parameters from Video Using Differentiable Physics. In Pattern Recog- nition - 42nd DAGM German Conference, DAGM GCPR 2020, Tübingen, Germany, September 28 - October 1, 2020, Proceedings, pages 44–57. Springer, 2021. 5 [24]Chia-Hsiang Kao, Cong Phuoc Huynh, Chien-Yi Wang, No- ranart Vesdapunt, Stefan Stojanov, Bharath Hariharan, Olek- sandr Obiednikov, and Ning Zhou. Dynamics: Language- Based Representation for Inferring Rigid-Body Dynamics from Videos. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2026, Denver, CO, USA, June 3-7, 2026, pages 42364–42374. Computer Vision Foundation, 2026. 3 [25]Rasul Khanbayov, Mohamed Rayan Barhdadi, Erchin Ser- pedin, and Hasan Kurban.IRIS: A Real-World Bench- mark for Inverse Recovery and Identification of Physical Dy- namic Systems from Monocular Video. In Computer Vision - ECCV 2026 - 19th European Conference, Malmö, Sweden, September 8-12, 2026, Proceedings, Part LVI, pages 366–383. Springer, 2026. 4, 14 [26]Žiga Kova ˇ ci ˇ c and Kevin Ellis. MPMWorlds: Material-Point- Method Simulations for Inferring and Extrapolating Physical Dynamics. arXiv preprint arXiv:2606.01538, 2026. 3 [27]Peter Kulits, Haiwen Feng, Weiyang Liu, Victoria Fernandez Abrevaya, and Michael J. Black. Re-Thinking Inverse Graph- ics with Large Language Models. Transactions on Machine Learning Research, 2024. 3 [28]Tejas D. Kulkarni, Pushmeet Kohli, Joshua B. Tenenbaum, and Vikash Mansinghka. Picture: A Probabilistic Program- ming Language for Scene Perception. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 4390–4399. IEEE Computer Society, 2015. 3 [29]Seongyun Lee, Seungone Kim, Sue Hyun Park, Geewook Kim, and Minjoon Seo. Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and Virtual Meeting, August 11-16, 2024, pages 11286–11315. Association for Computational Linguis- tics, 2024. 2 [30] Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. PAC-NeRF: Physics Augmented Continuum Neural Radi- ance Fields for Geometry-Agnostic System Identification. In The Eleventh International Conference on Learning Rep- resentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 2, 3 [31]Jiarong Liang, Max Ku, Ka-Hei Hui, Ping Nie, and Wenhu Chen.VisPhyWorld: Probing Physical Reason- ing via Code-Driven Video Reconstruction. arXiv preprint arXiv:2602.13294, 2026. 3 [32] Chunjiang Liu, Xiaoyuan Wang, Haoyu Chen, Yizhou Zhao, Ming-Hsuan Yang, and László A. Jeni. SimWorlds: A Multi- Agent System for Dynamic 3D Scene Creation. arXiv preprint arXiv:2607.01766, 2026. 2, 3 [33] Hengyu Liu, Chenxin Li, Zhengxin Li, Yipeng Wu, Wuyang Li, Zhiqin Yang, Zhenyuan Zhang, Yunlong Lin, Sirui Han, and Brandon Y. Feng. IR3D-Bench: Evaluating Vision- Language Model Scene Understanding as Agentic Inverse Rendering. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Pro- cessing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. 3 [34]Haozhe Lou, Mingtong Zhang, Haoran Geng, Hanyang Zhou, Sicheng He, Zhiyuan Gao, Siheng Zhao, Jiageng Mao, Pieter Abbeel, Jitendra Malik, Daniel Seita, and Yue Wang. D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping. In The Fourteenth In- ternational Conference on Learning Representations, 2026, https://openreview.net/forum?id=13jshGCK9i. 2 [35]Moonshot AI. What’s New | Kimi Code Docs. https://w. kimi.com/code/docs/en/kimi-code/whats-new.html. 6 [36]Saman Motamed, Minghao Chen, Luc Van Gool, and Iro Laina. TRAVL: A Recipe for Making Video-Language Mod- els Better Judges of Physics Implausibility. arXiv preprint arXiv:2510.07550, 2025. 2 [37]Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do Generative Video Models Understand Physical Principles? In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2026, Tucson, AZ, USA, March 6-10, 2026, pages 948–958. IEEE, 2026. 4, 7, 14 [38]OpenAI. Codex in ChatGPT | AI Coding Agents for Software Engineering. https://chatgpt.com/codex/. 6 [39] OpenAI. GPT-6 Astra: A New Generation of Intelligence. https://openai.com/index/gpt-6-astra/. 6 [40]OpenAI. GPT-5.6: Frontier Intelligence That Scales with Your Ambition. https://openai.com/index/gpt-5-6/, 2026. 6 [41]OpenAI. Introducing GPT-6 Sol and Luna. https://openai.c om/index/introducing-gpt-6-sol-and-luna/, 2026. 6 [42]Pexels. Free 4K Stock Videos & Full HD Video Clips to Download. https://w.pexels.com/videos/. 4, 14 [43]Atin Pothiraj, Jaemin Cho, Yue Zhang, Elias Stengel-Eskin, and Mohit Bansal. Physics Question Scene Graph: Fine- Grained Evaluation of Physical Plausibility in Text-to-Video Generation. In Computer Vision - ECCV 2026 - 19th Eu- 10 ropean Conference, Malmö, Sweden, September 8-12, 2026, Proceedings, Part IX, pages 301–318. Springer, 2026. 2 [44] Daniel Rho, Jun Myeong Choi, Matthew Thornton, Biswadip Dey, and Roni Sengupta. MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos. arXiv preprint arXiv:2605.30320, 2026. 3, 5 [45]Ruihong Shen, Žiga Kova ˇ ci ˇ c, Peter Kulits, Xingrui Wang, Zizhang Li, Joshua B. Tenenbaum, Alan Yuille, Jieneng Chen, and Jiajun Wu. 4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes. arXiv preprint arXiv:2610.03715, 2026. 2, 3, 5 [46]Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Parameters for Reasoning. In The Thirteenth International Conference on Learning Repre- sentations, ICLR 2025, Singapore, April 24-28, 2025. Open- Review.net, 2025. 6 [47]Chunyi Sun, Junlin Han, Weijian Deng, Xinlong Wang, Zis- han Qin, and Stephen Gould. 3D-GPT: Procedural 3D Mod- eling with Large Language Models. In International Confer- ence on 3D Vision, 3DV 2025, Singapore, March 25-28, 2025, pages 1253–1263. IEEE, 2025. 3 [48]Yolo Y. Tang, Daiki Shimada, Jiayue Meng, Jing Bi, Pinxin Liu, Yicheng Wang, Yunzhong Xiao, Zhangyun Tan, Zeliang Zhang, Chao Huang, Susan Liang, Qianxiang Shen, Luchuan Song, Ali Vosoughi, Mingqian Feng, Melika Filvantorka- man, and Chenliang Xu. BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender. arXiv preprint arXiv:2609.15478, 2026. 3 [49]Hanyang Wang, Yimo Cai, Weiliang Chen, Jiawei Chi, Haowen Sun, Qiyu Dai, Yi-Hsin Hung, Xingzhuo Guo, Jin- shan Ren, Runmao Yao, Ziwei Liu, Mingsheng Long, Yueqi Duan, Jun Gao, Jiangran Lyu, Fangfu Liu, and Jialong Wu. Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning. arXiv preprint arXiv:2608.27549, 2026. 2, 4 [50]Hao Wang, Wenhui Zhu, Shao Tang, Zhipeng Wang, Xu- anzhao Dong, Xin Li, Xiwen Chen, Ashish Bastola, Xinhao Huang, Yalin Wang, and Abolfazl Razi. EzBlender: Efficient 3D Editing with Plan-and-ReAct Agent. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2026 - Workshops, Tucson, AZ, USA, March 6-10, 2026, pages 1343–1352. IEEE, 2026. 3 [51]Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image Quality Assessment: From Error Visibility to Struc- tural Similarity. IEEE Transactions on Image Processing, 13 (4):600–612, 2004. 6 [52]Jiajun Wu, Ilker Yildirim, Joseph J. Lim, Bill Freeman, and Joshua B. Tenenbaum. Galileo: Perceiving Physical Ob- ject Properties by Integrating a Physics Engine with Deep Learning. In Advances in Neural Information Processing Sys- tems 28: Annual Conference on Neural Information Process- ing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 127–135, 2015. 2 [53] Jiajun Wu, Erika Lu, Pushmeet Kohli, Bill Freeman, and Josh Tenenbaum. Learning to See Physics via Visual De- Animation. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Pro- cessing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 153–164, 2017. 2 [54]Jiajun Wu, Joshua B. Tenenbaum, and Pushmeet Kohli. Neu- ral Scene De-Rendering. In 2017 IEEE Conference on Com- puter Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 7035–7043. IEEE Computer Society, 2017. 3 [55] Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang.PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 4389–4398. IEEE, 2024. 2 [56]Tianyidan Xie, Peiyu Wang, Hu Jiaxin, Yuyi Qian, Yuxuan Wang, Shenyi Wang, Rui Ma, Yanlun Peng, Lanjun Wang, Ying Tai, Jian Yang, and Zili Yi. PhysCodeBench: Bench- marking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement. arXiv preprint arXiv:2604.23580, 2026. 3 [57]Xiaoyu Xiong, Changyu Hu, Chunru Lin, Pingchuan Ma, Chuang Gan, and Tao Du. TopoGaussian: Inferring Internal Topology Structures from Visual Clues. In The Thirteenth International Conference on Learning Representations, 2025, https://proceedings.iclr.c/paper_files/paper/2025/hash/f41 b6e5af73421e46ceed9cb036e72e7-Abstract-Conference.h tml. 2 [58]Yutaro Yamada, Khyathi Raghavi Chandu, Bill Yuchen Lin, Jack Hessel, Ilker Yildirim, and Yejin Choi. L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Uncon- ventional Objects. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, NAACL 2025 - System Demonstrations, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, pages 456–469. Association for Computational Linguistics, 2025. 3 [59]Runmao Yao, Kairui Hu, Yukang Cao, Ruisi Wang, Shulin Tian, Ziang Cao, Weichen Fan, Ziqi Huang, Yuhao Dong, Hao Li, Zhaoxi Chen, Zhongang Cai, Lei Yang, and Zi- wei Liu. Apple-π: Benchmarking Thinking with Video to- wards Law-Grounded Physical Intelligence. arXiv preprint arXiv:2607.16401, 2026. 2 [60]Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. CLEVRER: Collision Events for Video Representation and Reasoning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. Open- Review.net, 2020. 2 [61]Shaofeng Yin, Jiaxin Ge, Zora Zhiruo Wang, Chenyang Wang, Xiuyu Li, Michael J. Black, Trevor Darrell, Angjoo Kanazawa, and Haiwen Feng. Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning. In Computer Vision - ECCV 2026 - 19th European Conference, Malmö, Sweden, September 8-12, 2026, Proceedings, Part XXXV, pages 557–575. Springer, 2026. 2, 3 [62]Conghan Yue, Yuanjie Chen, Yue Han, Ya Gao, Yunyan Xiao, WeiYao Zhang, and Zhineng Chen. VideoPhysEdit: Physical Counterfactual Video Editing via Rigid-Body Physical Scene Reconstruction. arXiv preprint arXiv:2609.35134, 2026. 3 [63]Z.ai. GLM-5.3-Flash: Frontier Intelligence, Flash Cost. https: //z.ai/blog/glm-5.3-flash. 6 [64]Jiahan Zhang, Muqing Jiang, Nanru Dai, Taiming Lu, Arda Uzunoglu, Shunchi Zhang, Yana Wei, Jiahao Wang, Vishal M. Patel, Paul Pu Liang, Daniel Khashabi, Cheng Peng, Rama Chellappa, Tianmin Shu, Alan L. Yuille, Yilun Du, and Jieneng Chen. World-in-World: World Models in a Closed- Loop World. In The Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23-27, 2026. proceedings.iclr.c, 2026. 2 11 [65]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 586–595. Computer Vision Foundation / IEEE Computer Society, 2018. 6 [66]Yunzhi Zhang, Zizhang Li, Matt Zhou, Shangzhe Wu, and Jiajun Wu. The Scene Language: Representing Scenes with Programs, Words, and Embeddings. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 24625–24634. Computer Vision Foundation / IEEE, 2025. 3 [67] Yi Zhang, Yunshuang Wang, Zeyu Zhang, and Hao Tang. Code2Worlds: Empowering Coding LLMs for 4D World Generation. arXiv preprint arXiv:2602.11757, 2026. 3 [68] Yifei Zhao, Xiangxin Zhou, Wenhao Yang, Jiaqi Tang, Pu Jian, Huanjin Yao, Jiarui Yao, Haowei Lin, Chunchao Guo, Zhuo Chen, Wenkai Lyu, Jianzhu Ma, Xueqian Wang, and Wenxi Zhu. SceneActBench: Can Agents Act on the 3D Scenes They See? arXiv preprint arXiv:2607.22393, 2026. 3 [69]Yan Zheng and Florian Bordes. VoxelCodeBench: Bench- marking 3D World Modeling through Code Generation. arXiv preprint arXiv:2604.02580, 2026. 3 [70]Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and Simulation of Elastic Objects with Spring- Mass 3D Gaussians. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part I, pages 407–423. Springer, 2024. 3 [71]King Zhu, Hanhao Li, Siwei Wu, Tianshun Xing, Dehua Ma, Xiangru Tang, Minghao Liu, Jian Yang, Jiaheng Liu, Yuchen Eleanor Jiang, Changwang Zhang, Chenghua Lin, Jun Wang, Ge Zhang, and Wangchunshu Zhou. Scaling Test-Time Compute for LLM Agents. arXiv preprint arXiv:2506.12928, 2025. 6 12 PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction Appendix ContentsPage A Dataset and Execution Framework14 A.1Composition and Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 A.2Reference Video Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 A.3Harbor Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 B Evaluation Protocol20 B.1Task Rubrics and Executable Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 B.2Counterfactual Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 C Additional Results and Analysis21 C.1Scores on Aligned Task Subsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 C.2Scores on Produced Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 C.3Reasoning Effort, Completion, and Reconstruction Quality . . . . . . . . . . . . . . . . . . . . .22 D Case Studies23 D.1Metrics and Reconstruction Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 D.1.1Two Pendulums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 D.1.2Rolling Quail Egg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 D.1.3Wine Pouring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 D.1.4Avocado Halves Entering Water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 D.1.5Thai Flag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 D.2Physical Validity Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29 D.3Counterfactual Mechanism Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32 D.3.1A Nearby Obstacle Has No Contact Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 D.3.2Support Extends Beyond the Physical Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 D.3.3A Bearing Hides a Missing Table Collision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.3.4A Different Reconstruction Omits Contact with a Chess Piece . . . . . . . . . . . . . . . . . . . . 35 D.3.5A Solid Token Behaves as if It Were Absent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 D.4Cross-Harness Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37 D.4.1Checking Rolling After Contact Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 D.4.2Checking Accumulation Across a Pour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 D.4.3Harness Instructions and Completed Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 13 A. Dataset and Execution Framework A.1. Composition and Sources The156tasks contain578event items in total (Tab. 3). By main category, there are95rigid-body,40liquid,12deformable, and 9 wind-driven tasks. Table 3. Composition of PhysCodeGym: number of tasks and event items per main category and subcategory. CategorySubcategoryTasks Event items Rigid body Falling and bouncing1671 Rolling and tracks1868 Collision and chain reactions1456 Swinging and suspension1651 Rotation2063 Granular and multi-object piling722 Stacking and static equilibrium413 Liquid Pouring and container filling25109 Jets and surface flow825 Liquid–object interaction730 Deformable–1241 Wind-driven –929 Total156578 Data sources. The reference videos come from Pexels [42], Physics-IQ [37], BAAI-DataCube Physics-aware Videos [4], and IRIS [25]. Table 4 reports the number of retained tasks from each source by main category. Table 4. Data sources of PhysCodeGym. Each retained reference clip contributes one task. SourceRigid bodyLiquidDeformableWind-drivenTotal Pexels693599122 Physics-IQ1953027 BAAI-DataCube Physics-aware Videos50005 IRIS20002 Total9540129156 Rigid-body tasks. Rigid-body tasks fall into seven subcategories. Falling and bouncing tests gravity, collision restitution, and coming to rest after landing. Rolling and tracks covers objects rolling on planes, slopes, trays, or tracks, with paths determined by the track geometry. Collision and chain reactions focuses on momentum transfer between objects, as in billiards, bowling, dominoes, and Newton’s cradles. Swinging and suspension involves swings, pendulums, metronomes, and hanging objects, whose motion is organized by hinges or ropes. Rotation involves turntables, windmills, spinning tops, and rotating platforms. Granular and multi-object piling involves pouring and piling of many objects, such as rice grains, flakes, and paper clips. Stacking and static equilibrium tests whether blocks or stones remain stable or topple under loading. Liquid tasks.Liquid tasks fall into three subcategories. Pouring and container filling is the most common case: liquid flows from a bottle mouth, spout, or pipe into a container, and the liquid level rises accordingly. Jets and surface flow covers jets from hoses, nozzles, and fountains, as well as the spreading and splashing of liquid after it lands on a surface. Liquid–object interaction covers objects falling into liquid, sinking or floating in it, and liquid streams striking objects. Deformable and wind-driven tasks.The deformable and wind-driven categories contain few tasks and are not subdivided. 14 A.2. Reference Video Examples Figures 6 to 9 show32reference clips selected to illustrate the categories and subcategories of PhysCodeGym. Each row shows four frames at approximately0%,33%,67%, and100%of the clip. Full frames are resized without cropping or color adjustment. This selection illustrates the range of phenomena; the task frequencies are given in Appendix A.1. 0%33%67%100% (a) Lemon slice falling (b) Blueberries bouncing (c) Ball rolling in a tray (d) Wooden piece rolling (e) Billiard balls at a pocket (f) Duck and dominoes (g) Rice pouring and piling (h) Paper clips scattering Figure 6. Rigid-body reference examples (1/2): falling and bouncing, rolling and tracks, collision and chain reactions, and granular or multi-object piling. Each row contains four uniformly sampled frames from one reference video. 15 0%33%67%100% (a) Metronome oscillating (b) Two pendulums swinging (c) Hanging dolphin ornament (d) Record rotating on a turntable (e) Spinning top (f) Teapot rotating (g) Stable block stack (h) Unstable block stack Figure 7. Rigid-body reference examples (2/2): swinging and suspension, rotation, and stacking or static equilibrium. Each row contains four uniformly sampled frames from one reference video. 16 0%33%67%100% (a) Milk filling a glass (b) Container overflowing (c) Coffee pouring into a cup (d) Cocktail being poured (e) Water jet from a hose (f) Liquid splashing on a surface (g) Liquid pouring over a duck (h) Strawberry entering liquid Figure 8. Liquid reference examples: pouring and container filling, jets and surface flow, and liquid–object interaction. Each row contains four uniformly sampled frames from one reference video. 17 0%33%67%100% (a) Cloth draping over a teapot (b) Balloon inflating (c) Weight loading a pillow (d) Hammock swaying (e) Single flag fluttering (f) A row of flags in wind (g) Flowers moving in wind (h) Poppy stem swaying Figure 9. Deformable-category and wind-driven reference examples. Rows (a)–(d) show draping, membrane inflation, pillow deformation, and hammock motion; rows (e)–(h) show flag and plant motion in wind. Each row contains four uniformly sampled frames from one reference video. 18 A.3. Harbor Integration Framework organization.PhysCodeGym builds on Harbor [16] v0.22.0, with local compatibility patches for CLI setup and container execution. We use Harbor’s Docker backend with Ubuntu 24.04 containers and the official Blender 4.0.2 distribution. Harbor’s installed-agent adapters launch the native Codex, Claude Code, and Kimi Code CLIs inside these containers (Sec. 4.2). The CLIs are installed in the image or mounted as preinstalled binaries; our Kimi adapter overrides only installation and inherits Harbor’s Kimi Code execution path. Harbor manages container and agent lifecycles, while PhysCodeGym supplies the reconstruction tasks, Blender tools, and evaluation protocol. Reconstruction workflow. Each reconstruction task is packaged as a separate Harbor task containing task instructions, a reference video, and environment configuration. The task image provides the Blender runtime and command-line tools for video observation, scene execution and inspection, and preview rendering. The agent observes, programs, and checks its reconstruction inside the container (Sec. 3.2). Unless otherwise specified, Harbor limits agent execution to3hours per run, and the task environment’s launch interface permits at most20Blender process launches (Sec. 4.1). Scene execution, scene inspection, and preview rendering share this quota. Video observation commands do not consume Blender launches but count toward execution time. Once the launch quota is exhausted, new Blender launch requests are rejected; when the execution time limit is reached, Harbor ends the agent run. Evaluation integration.After agent execution, Harbor collects the submitted program and invokes the benchmark’s checking script in a separate verification container. This stage executes the program, checks the scene and rendered outputs, and records basic image and motion measurements. Subsequent evaluation jobs assess objects and events in the archived video. A verification agent checks the program and scene against manually annotated physical criteria, including counterfactual tests for selected tasks (Appendix B). The component scores are aggregated according to the evaluation protocol in Sec. 4.3. 19 B. Evaluation Protocol This section describes task-level physical verification and counterfactual tests. B.1. Task Rubrics and Executable Verification Task rubrics.We assess physical validity with an agent acting as a verifier, guided by manually annotated physical rubrics for each task. Each rubric specifies the permitted initial setup and actuation, the mechanisms to check, and the evidence required for a judgment. The criteria cover numerical stability, gravity, mass, and actuation, together with task-specific requirements on contact, support, and structural integrity. The same rubric applies to every candidate for a task. Executable verification. The verification agent reads the submitted program and native scene, maps the rubric’s semantic roles to scene objects, and traces how their motion is implemented. It formulates a measurement plan and writes probes adapted to that implementation. The agent executes these probes in Blender to inspect geometry and measure object states, contact or support relations, and motion over time; Python tools analyze the resulting measurements. It inspects the outputs and refines its measurements when further evidence is needed. The final report gives a judgment for each criterion with supporting source references and measured evidence, and the evaluation harness applies the rubric’s aggregation rules. A missed reference event is assessed by the event score and is not by itself a physical violation. B.2. Counterfactual Tests Counterfactual verification evaluates rubric criteria that require controlled interventions. The rubric specifies the physical dependence to test and the expected response under a controlled intervention. The agent locates the corresponding mechanism in the program, creates an independent scene copy, implements the prescribed intervention, and re-simulates it for comparison with the unchanged baseline. Interventions may alter a force, remove a contact, or change an initial condition while holding other relevant conditions fixed. Before interpreting the response, the agent verifies that the intervention takes effect and uses control scenes to check the measurement. For example, when a ball moves a target through collision, removing the ball–target contact before impact while preserving the ball’s incoming motion should eliminate the target’s impact-induced response (Fig. 2). The agent compares the measured response with the rubric’s expectation to determine whether the reconstructed motion depends on the required physical interaction. 20 C. Additional Results and Analysis We compare scores on aligned task subsets, report quality conditional on producing an artifact, and examine the relationship between reasoning effort and reconstruction quality. C.1. Scores on Aligned Task Subsets The main table’s video metrics cover156tasks, whereas physical scores exclude technical gaps. Table 5 recomputes all metrics on the task set used for each configuration’s physical score, aligning the averaging set within each row. Event and motion remain video measurements; validity uses the corresponding reports. Task sets still differ across configurations, rather than forming a common intersection for all models. Table 5. Scores recomputed on each configuration’s physical-evaluation task subset. ModelHarnessS vis ↑S evt ↑S mot ↑S val ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.6940.8060.3250.7680.6330.664 GPT-5.6-SolCodex0.6110.6870.2100.6900.5290.570 GPT-6-LunaCodex0.5070.3920.1290.5550.3590.433 GPT-5.6-LunaCodex0.5030.4500.1570.5640.3910.447 DeepSeek-V4.1-FlashClaude Code0.5110.4520.2000.5220.3910.451 Claude Sonnet 5Claude Code0.4360.3700.1110.5590.3470.392 Qwen3.8-FlashClaude Code0.3350.3160.0900.3500.2520.294 GLM-5.3-FlashClaude Code0.2910.2410.1040.2950.2130.252 Kimi K2.8Kimi Code0.4040.3480.1100.4480.3020.353 All scores within a row use the same task subset. Best and second-best scores are bold and underlined, respectively. For GPT-6-Astra, restricting the visual mean from156to155tasks changesS vis from0.693to0.694andS tot from0.663 to 0.664, without changing individual task scores. C.2. Scores on Produced Artifacts Table 6 averages the same selected artifacts as Tab. 2, conditional on each configuration producing a rendered sequence. Tasks without an output are excluded; zero scores on existing outputs are retained. The five observed metrics use all rendered outputs, while validity and the physical score use those with an available physical judgment from either evidence scope, following Sec. 4.3.S tot averages the reported visual and physical scores, retaining their respective coverage. These output-conditioned means describe reconstruction quality on produced artifacts; the full-task results also account for missing outputs. Table 6. Mean scores on tasks with rendered outputs for the configurations in Tab. 2. Higher is better. Best and second-best scores are bold and underlined. HarnessModel VisualPhysical S tot ↑ S SSIM ↑ S LPIPS ↑ S obj ↑ S vis ↑ S evt ↑ S mot ↑ S val ↑ S phys ↑ Codex GPT-6-Astra0.6100.5610.9090.6930.8050.3240.7680.6330.663 GPT-5.6-Sol0.5530.4510.8400.6150.6920.2110.6940.5320.574 GPT-6-Luna0.5540.3870.6770.5390.4200.1360.5930.3840.462 GPT-5.6-Luna0.5290.3710.6690.5230.4690.1650.5910.4090.466 Claude Code DeepSeek-V4.1-Flash0.599 0.4630.7160.5930.5200.2330.6090.4560.525 Claude Sonnet 50.5680.3490.5320.4830.4070.1220.6190.3840.434 Qwen3.8-Flash0.5640.3810.6530.5320.4890.1420.5660.4070.470 GLM-5.3-Flash0.5510.4360.6960.5610.4620.1990.5720.4140.488 Kimi CodeKimi K2.80.5700.3980.6260.5310.4580.1450.5920.3990.465 21 C.3. Reasoning Effort, Completion, and Reconstruction Quality We separate the benchmark aggregate, which retains missing-output zeros, from reconstruction quality conditional on producing an artifact. All comparisons use the same frozen scores and validity composition as the main results. Table 7 follows the score-aggregation convention in Sec. 4.3. Table 7. Overall scores under different averaging sets. Best and second-best scores are bold and underlined. Averaging setLowHighMax Benchmark aggregate0.4760.4500.460 Each setting’s produced artifacts0.5270.5250.547 Shared tasks with outputs and physical judgments0.5330.535 0.564 Quality conditional on an output.Figure 10 plots the nine scores using each setting’s produced artifacts. Video metrics use those artifacts, while validity and physical scores use the available physical judgments within that subset. These subsets can differ between settings. The overall ordering changes: max scores0.547, compared with0.527for low and0.525for high. Max also has the highest conditional event and physical scores (0.615and0.492). Thus, the benchmark-level advantage of low does not imply better reconstruction quality whenever an output is produced. LowHighMax 0.35 0.50 0.65 0.80 Score (higher is better) 0.589 0.450 0.768 (a) S SSIM S LPIPS S obj LowHighMax 0.2 0.4 0.6 0.615 0.229 0.633 (b) S evt S mot S val LowHighMax 0.35 0.50 0.65 0.602 0.492 0.547 (c) S vis S phys S tot Reasoning effort Figure 10. DeepSeek-V4.1-Flash scores on tasks with rendered outputs (axis ranges differ). Comparison on shared tasks.We also restrict all three settings to the intersection of tasks with both a submitted video and an available physical judgment for every setting, retaining zero scores on existing artifacts. Max again has the highest visual, physical, and overall means on this shared subset. Its event score is0.628, compared with0.551for low and0.535for high. The mean paired, unrounded per-task overall-score difference between max and low is0.030; its paired task-bootstrap95% interval is[−0.001, 0.061]. The bootstrap measures sensitivity to task sampling; repeated-run variability is not estimated here. This interval includes zero, so the observed mean ordering should not be interpreted as a universal effort advantage. Where the aggregate gap arises. On the aligned task set with defined scores for both low and max, asymmetric missing outputs contribute+0.036to the low-minus-max mean per-task score, while tasks with outputs from both settings contribute −0.020. Their net contribution is+0.016. This diagnostic uses the same eligible tasks and unrounded case scores, rather than recombining differently covered headline columns. It identifies completion outcomes as the main source of low’s aggregate lead. Using original-condition-only validity reduces the low–max gap by approximately0.003, making the CF component a secondary contributor. Execution overhead. Max uses2.46×the recorded output tokens of low on average, and mean agent execution time rises from93.3to110.7minutes under the fixed three-hour budget. The additional missing outputs are concentrated in timeouts and final submissions that fail execution or lack required submission files. These results highlight a tradeoff between task completion and reconstruction quality under a fixed execution budget. 22 D. Case Studies These case studies connect evaluation scores with reconstructed behavior and executable evidence. They cover metric interpretation, physical failures in unchanged scenes, counterfactual mechanism checks, and verification across harnesses. D.1. Metrics and Reconstruction Quality Different metrics assess reconstruction appearance, event progression, and physical constraints. The five cases in Figs. 11 to 15 examine differences in visible behavior and physical constraints that remain when individual metric scores are high or equal. Each case pairs synchronized frames from a reference video and six reconstructions with component and summary scores. Table 8. Overview of the five case studies and their diagnostic questions. QuestionCasesMain observation What differs when one metric is high?Two pendulums; rolling quail egg High motion overlap can coexist with incomplete events or physical violations. What differs when one metric is equal?Wine pouring; avocado halvesEqual event or validity scores coexist with differences in appearance and event progression. What does scene inspection reveal?Thai flagAn event score of 1 coexists with cloth self-intersections. Reading the figures and scores. Display times follow the reference event stages and are shared by the reference and all candidates; frames are shown without cropping. Table rows follow the image-row order, identify each execution harness, and report six components together with the visual, physical, and overall scores. The highest and second-highest distinct scores in each column are bold andunderlined, respectively; ties are determined before rounding. Full videos and denser temporal samples accompany the cases. Structural similarity, perceptual similarity, object accuracy, event score, and motion score evaluate submitted videos. Physical validity evaluates the corresponding submitted program and its Blender scene. All table metrics are higher-is-better (↑);S LPIPS is perceptual similarity,clip(1− d, 0, 1), rather than raw LPIPS distanced. Score definitions and aggregation follow Sec. 4.3. Component scores and their evidence identify the properties on which candidates differ. All displayed score intervals are resolved, while validity judgments remain provisional pending semantic and evidence review. Case and model selection.The archived 7 October 2026 pool contains22eligible tasks, each with a GPT-6-Astra video and at least five other configurations with complete observed metrics and resolved physical-validity intervals. Each task retains Astra and five other configurations selected by report availability, with configuration ID breaking ties; score values do not enter selection. We inspect five non-flag candidates under randomized model labels and select four for distinct phenomena and visible differences, retaining one flag example. All configurations use high effort, a three-hour budget, and at most twenty Blender launches. Model sets can differ by task. These targeted cases explain evaluation dimensions; they do not estimate benchmark-wide ranking accuracy or constitute a human-preference experiment. 23 D.1.1. Two Pendulums 0.00 s1.60 s2.67 s3.20 s Reference GPT-6- Astra GPT-6- Luna GPT-5.6- Luna Claude Sonnet 5 Qwen3.8- Flash Kimi K2.8 Component and summary scores. Best and second-best scores are bold and underlined, respectively. Visual metricsPhysical metricsSummary scores ModelHarnessS SSIM ↑S LPIPS ↑S obj ↑S evt ↑S mot ↑S val ↑S vis ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.9050.8011.0001.0000.2421.0000.9020.7470.825 GPT-6-LunaCodex0.7580.4581.0000.0000.1411.0000.7390.3800.559 GPT-5.6-LunaCodex0.7360.4540.8570.0000.3180.5000.6820.2730.478 Claude Sonnet 5Claude Code0.7430.4230.8570.2940.1281.0000.6740.4740.574 Qwen3.8-FlashClaude Code0.9060.7291.0001.0000.306 0.5000.8780.6020.740 Kimi K2.8Kimi Code0.8910.7801.0001.0000.2541.0000.8910.7510.821 Figure 11. Motion overlap and event completion. GPT-5.6-Luna has the highest motion IoU but an event score of0; GPT-6-Astra ranks fourth on motion IoU and scores1on events. Luna’s validity report identifies connector-solid overlap outside the shared pivot throughout frames 55–240, yielding validity 0.5 versus Astra’s 1. Displayed frames cover early motion; scores cover all 8 seconds. 24 D.1.2. Rolling Quail Egg 0.53 s2.67 s5.30 s7.97 s Reference GPT-6- Astra GPT-6- Luna GPT-5.6- Luna DeepSeek- V4.1-Flash Qwen3.8- Flash Kimi K2.8 Component and summary scores. Best and second-best scores are bold and underlined, respectively. Visual metricsPhysical metricsSummary scores ModelHarnessS SSIM ↑S LPIPS ↑S obj ↑S evt ↑S mot ↑S val ↑S vis ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.9190.8641.0001.0000.2251.0000.9270.7420.835 GPT-6-LunaCodex0.8950.7530.5000.0000.0000.4500.7160.1500.433 GPT-5.6-LunaCodex0.8510.7081.0001.0000.0830.4500.8530.5110.682 DeepSeek-V4.1-FlashClaude Code0.9070.8220.5001.0000.2350.5000.7430.5780.660 Qwen3.8-FlashClaude Code0.8730.7050.3331.0000.0010.4500.6370.4840.560 Kimi K2.8Kimi Code0.8960.7960.833 0.6250.1241.0000.8420.5830.712 Figure 12. Motion overlap and scene constraints. DeepSeek-V4.1-Flash exceeds GPT-6-Astra on motion IoU (0.235versus0.225), but its validity is capped at0.5, versus Astra’s1. The validity report identifies persistent interpenetration between two stationary eggs across all 240frames; this violation concerns the resting cluster rather than the moving egg. Astra retains the cluster and speckled appearance, with a timing mismatch. 25 D.1.3. Wine Pouring 0.00 s2.67 s5.30 s7.97 s Reference GPT-6- Astra GPT-6- Luna GPT-5.6- Luna DeepSeek- V4.1-Flash Claude Sonnet 5 Kimi K2.8 Component and summary scores. Best and second-best scores are bold and underlined, respectively. Visual metricsPhysical metricsSummary scores ModelHarnessS SSIM ↑S LPIPS ↑S obj ↑S evt ↑S mot ↑S val ↑S vis ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.7660.5891.0001.0000.2001.0000.7850.7330.759 GPT-6-LunaCodex0.7410.4910.6331.0000.1070.5000.6220.5360.579 GPT-5.6-LunaCodex0.6970.4570.8001.0000.2090.4500.6510.5530.602 DeepSeek-V4.1-FlashClaude Code0.7180.5190.8001.0000.3141.0000.6790.7710.725 Claude Sonnet 5Claude Code0.7740.5110.5331.0000.0560.5000.6060.5190.562 Kimi K2.8Kimi Code0.6850.4420.8001.0000.0971.0000.6420.6990.671 Figure 13. Appearance differences under equal event scores. All six candidates receive an event score of1, despite differences in stream color, glass visibility, and accumulated liquid. Claude Sonnet 5 has the highest SSIM (0.774) despite its pale rendering, but lower perceptual similarity than Astra (0.511versus0.589). Equal event scores thus coexist with appearance differences reflected in the frames and visual metrics. 26 D.1.4. Avocado Halves Entering Water 0.53 s2.67 s5.30 s7.97 s Reference GPT-6- Astra GPT-5.6- Sol GPT-6- Luna GPT-5.6- Luna DeepSeek- V4.1-Flash Kimi K2.8 Component and summary scores. Best and second-best scores are bold and underlined, respectively. Visual metricsPhysical metricsSummary scores ModelHarnessS SSIM ↑S LPIPS ↑S obj ↑S evt ↑S mot ↑S val ↑S vis ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.7710.5631.0001.0000.5161.0000.7780.8390.808 GPT-5.6-SolCodex0.7020.5190.7501.0000.3391.0000.6570.7800.718 GPT-6-LunaCodex0.6520.4800.1250.0000.0001.0000.4190.3330.376 GPT-5.6-LunaCodex0.5800.4070.2500.0000.3320.5000.4120.2770.345 DeepSeek-V4.1-FlashClaude Code0.5730.3830.875 0.0000.2690.5000.6100.2560.433 Kimi K2.8Kimi Code0.4050.5090.0000.4710.1910.1500.3050.2700.288 Figure 14. Equal validity with different reconstruction fidelity. GPT-6-Astra, GPT-5.6-Sol and GPT-6-Luna receive validity1, but event scores of1,1, and0, respectively. Luna’s validity report accepts a single composite rigid body in a homogeneous analytical medium: measured motion responds to effective gravity and linear drag. Passing these checks under the permitted simplification does not establish reproduction of the reference events. 27 D.1.5. Thai Flag 0.00 s2.67 s5.30 s7.97 s Reference GPT-6- Astra GPT-6- Luna GPT-5.6- Luna DeepSeek- V4.1-Flash Qwen3.8- Flash Kimi K2.8 Component and summary scores. Best and second-best scores are bold and underlined, respectively. Visual metricsPhysical metricsSummary scores ModelHarnessS SSIM ↑S LPIPS ↑S obj ↑S evt ↑S mot ↑S val ↑S vis ↑S phys ↑S tot ↑ GPT-6-AstraCodex0.8630.7110.8331.0000.2311.0000.8020.7440.773 GPT-6-LunaCodex0.8530.6340.6670.0000.2260.5000.7180.2420.480 GPT-5.6-LunaCodex0.8610.6330.6671.0000.1741.0000.7200.7250.722 DeepSeek-V4.1-FlashClaude Code0.8590.7250.8331.0000.2720.5000.8060.5910.698 Qwen3.8-FlashClaude Code0.8010.6830.8331.0000.3950.500 0.7720.6320.702 Kimi K2.8Kimi Code0.8570.6950.8331.0000.1910.5000.7950.5640.679 Figure 15. Event completion with cloth self-intersections. Qwen3.8-Flash receives an event score of1, yet physical-validity checks detect self-intersections between nonadjacent cloth regions in14frames, capping validity at0.5. This case shows that reproducing the visible event can coexist with invalid cloth geometry. Displayed frames also show differences in flag extension and cloth phase. 28 D.2. Physical Validity Failures The six selected reconstructions receiveS obj = S evt = 1, yet have explicit physical-validity failures. All displayed validity scores are resolved raw proposals from the original-condition checks; no default-pass assignments are used. This subsection illustrates rubric criteria evaluated on unchanged scenes; counterfactual criteria appear in Appendix D.3. Image pairs use the same reference/submission frame index. Plots identify one failure witness; scores summarize the corresponding original-condition rubric. Pixel fidelity and timing vary; these examples do not estimate failure prevalence. Reference · f85Reference · f236 Submission · f85Submission · f236 85120160200240 Native frame (after release) 0 2 4 6 8 10 12 Energy (mJ) 9.44 mJ > 4.97 mJ Potential energy alone Largest fitted initial energy S obj =1.00, S evt =1.00 Raw S val =0.80 Native energy witness Figure 16. GPT-6-Astra: energy addition in a Newton cradle. After the gripper is clear at frame 85, four fits to the free pre-impact swing estimate initial total energy at approximately4.971mJ; the dashed line is the largest fitted estimate. At frame 236, gravitational potential alone reaches9.441mJ. With fixed supports and no admitted continuing work source, the native trajectory violates the passive energy budget (N3). This witness does not require differentiating impact-frame velocities. Reference · f1Reference · f120 Submission · f1Submission · f120 0.00.51.0 Below-table depth (scene units) E1 E2 E3 E4 E5 E6 E7 E8 E9 0.309 0.313 0.385 0.449 0.723 1.051 1.079 1.224 1.298 S obj =1.00, S evt =1.00 Raw S val =0.50 All 9 resting eggs: 240/240 frames Figure 17. GPT-5.6-Sol: resting eggs embedded in the tabletop. All nine resting eggs intersect the actual evaluated table surface throughout240/240frames (E1). Their deepest below-surface vertices are0.309–1.298scene units beneath the local surface, with persistent material-boundary intersections. The bars describe the resting eggs only, excluding the mover’s isolated initial overlap. 29 Reference · f92Reference · f99 Submission · f92Submission · f99 909599105 Native frame 0 10 20 30 40 50 60 Jaw-centre interior depth (m) 46.3 m Block-local slice f99 S obj =1.00, S evt =1.00 Raw S val =0.50 A gripper finger enters solid material Figure 18. DeepSeek-V4.1-Flash: a gripper passes through a released block. The right jaw’s centre lies inside the blue block on frames 95–100, reaching46.3m from the nearest block face at frame 99 (K1). The inset is a block-local cross-section through that centre, derived from measured coordinates. The plotted quantity is centre interior depth, not a universal surface-penetration depth; the original-condition score also includes an independent guide-attachment failure. Reference · f1Reference · f133 Submission · f1Submission · f133 −0.30.00.3 Scene x (scene units) −0.05 0.00 0.05 Scene z (scene units) 57.47% of crate-floor volume inside the table Table Crate floor S obj =1.00, S evt =1.00 Raw S val =0.68 Measured material slice (unchanged scene) Figure 19. GLM-5.3-Flash: the crate floor occupies the wooden tabletop. The measured solid intervals overlap by0.51864× 0.51864× 0.05000scene units, embedding57.47%of the crate-floor volume in the table (B1). The right panel is a section through the actual axis-aligned box materials, with their common material shaded. Both bodies remain unchanged throughout all240frames; this is a persistent original-state overlap, not a failure inferred by modifying the scene. 30 Reference · f1Reference · f120 Submission · f1Submission · f120 160120180240 Native frame 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 Sampled material–support distance (m) Persistent separation Unchanged scene Persistent support gap S obj =1.00, S evt =1.00 Raw S val =0.72 Native material–support distance Figure 20. Claude Sonnet 5: rotation supported by an oversized collision proxy. Joined top decoration enlarges the spherical collider beyond the ball material, producing persistent remote support/traction (S2). The main curve is the minimum over sampled vertices, edge midpoints and face centroids, not an exact global surface-distance minimum. The unchanged scene continues to support and rotate the ball while sampled material distances remain positive; the dashed line denotes zero material separation. Only original-state measurements are displayed. Reference · f54Reference · f60 Submission · f54Submission · f60 Frame 54Frame 60 0 100 200 300 400 Intersecting nonlocal triangle pairs 41 352 Shared vertices and nearby material neighbours excluded S obj =1.00, S evt =1.00 Raw S val =0.50 Native cloth self-intersection Figure 21. GPT-6-Luna: nonadjacent flag material crosses itself. Exhaustive checks of unchanged Blender-scene frames 54 and 60 find 41and352intersecting nonlocal triangle pairs (F2). Pairs sharing a material vertex or lying within a two-quad material neighbourhood are excluded; a frame-54 witness joins patches separated by18material rows. These are geometric pair counts at the two inspected frames, not independent events or a claim about the first onset of crossing. The raw validity score is the resolved value 0.50. 31 D.3. Counterfactual Mechanism Checks A reconstruction can pass checks along its original trajectory while omitting a physical interaction that the trajectory never exercises. The five selected reconstructions below illustrate this gap: a rolling body does not encounter a nearby dish, the cue ball remains on the table, or a bearing keeps a rotor away from the surface below. Changing the release or connection activates the missing interaction. A failure is established by measured material intrusion or unsupported motion together with a responsive control, rather than by trajectory change alone. Each selected reconstruction scores1.0on its original-condition checks but fails a counterfactual criterion of its physical- validity rubric. The paired reports use identical source and saved-world hashes. The native scene views show the tested intervention states, paired with measurements of geometry and motion. The examples illustrate failures of counterfactual rubric criteria; they do not estimate a failure rate. D.3.1. A Nearby Obstacle Has No Contact Response (a) CF launch, frame 1(b) CF rim crossing, frame 30 −0.3−0.2−0.1 COM x (m) −0.25 −0.20 −0.15 −0.10 −0.05 0.00 0.05 COM y (m) Release Dish footprint (c) Aim the wooden piece at the dish Candidate: shifted release Contact control 50100150200 Frame 0 1 2 3 4 5 Max. vertex intrusion (m) 4.62 m 158 frames inside dish material (d) Measured intrusion into dish material Candidate Contact control Figure 22. GPT-6-Astra: the wooden piece passes through the dish. The views show the shifted release before contact and during rim crossing. The original release avoids the dish. A shifted release starts32.25m clear of its surface but puts piece vertices inside the actual closed dish material for158frames, reaching4.619m depth. The grey disc in (c) is only the dish footprint; the failure witness in (d) measures intrusion into its three-dimensional material. Removing the dish leaves the candidate’s centre-of-mass trajectory unchanged. An independent contact control using the source mesh, mass, inertia and integration scale arrests the approach, with only0.110m maximum local contact intrusion. Thus the issue is absent obstacle response, not merely overlap in the image or the top-down projection. 32 D.3.2. Support Extends Beyond the Physical Table (a) Counterfactual, frame 120 Table edge (b) Finite-support control, frame 120 1.11.21.31.41.51.6 Cue-ball centre y (m) −0.6 −0.4 −0.2 0.0 Cue-ball centre z (m) Table Real edge: y = 1.100 m Candidate stays level Lower floor (not reached) (c) Outward release beyond finite support 115117120123127 Frame −700 −600 −500 −400 −300 −200 −100 0 Cue-ball height (m) Whole ball beyond edge from frame 117 a z = −9.8101 m/s 2 (d) Effective gravity before floor contact Candidate Finite-support control Figure 23. GPT-6-Astra: the cue ball remains suspended beyond the table edge. The views compare the outward-release candidate and finite-table control at frame 120. An outward release moves the entire cue-ball envelope beyond the actual support boundary aty = 1.100m from frame 117. Its centre remains at approximately28.575m height through frame 240 (124samples), despite the absence of table material below it. An explicit infinite-support control reproduces the candidate trajectory exactly. Restricting support to the finite tabletop reproduces the prefix through frame 116 and then givesa z =−9.8101m/s 2 . Only frames 115–127 are shown for that control, before it reaches the lower floor; its later floor-contact behavior is outside this witness. The plotted table patch marks the top support boundary, not a scaled cross-section of the entire table. What the intervention distinguishes.The original-condition checks see rolling supported motion and do not exercise this upper boundary. The intervention tests the spatial extent of the support law against the actual scene material. The finite-support and infinite-support controls separate a missing boundary condition from an ineffective gravity change. The defect is in the reconstructed support mechanism; it is not inferred from the ball leaving the camera view. 33 D.3.3. A Bearing Hides a Missing Table Collision (a) Before release, frame 120(b) After release, frame 125 Released assembly 120122124126128 Frame (bearing released after 120) −300 −200 −100 0 100 200 Moving-body height (m) Table slab (c) Remove bearing: candidate falls through Entire body below table Lowest material Highest material 120122124126128 Frame (bearing released after 120) −300 −200 −100 0 100 200 Moving-body height (m) Table slab (d) Same table with operative contact Body is arrested at table Lowest material Highest material Figure 24. DeepSeek-V4.1-Flash (max effort): releasing the rotor exposes missing table contact. The views show the assembly just before release and intersecting the table at frame 125. The original bearing supports the moving assembly. The intervention retains the exact trajectory through frame 120, then removes the sole base-to-beam hinge. The body intersects the unchanged table on frames 123–126 and lies entirely below the table bottom at frame 127. The shaded red and blue bands show each branch’s measured lowest-to-highest material extent, not a filled-body cross-section. Giving the same table geometry an operative collision path arrests the body at its surface; table-removal and disabled-contact controls reproduce the candidate’s free fall. Falling after a bearing is removed is expected. Passing through the table is the counterfactual failure. Why the control matters.Removing a legitimate support does not itself establish that a reconstruction is invalid. Here, the bearing-retained branch and the unchanged prefix isolate the release, while the table-contact control demonstrates that the available surface can stop the body. The intervention reveals a contact mechanism that the supported original motion did not require. Full source identities, intervention records and trajectory data are retained with the figures. 34 D.3.4. A Different Reconstruction Omits Contact with a Chess Piece (a) CF approach, frame 5(b) CF crossing, frame 35 020406080100120 Branch frame −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 Position along approach (scene units) Knight centre plane Present / removed: position difference = 0 (c) Removing the knight changes no motion Candidate, left release Knight removed Contact control 020406080100120 Branch frame 0.0 0.1 0.2 0.3 0.4 Max. vertex intrusion (scene units) 46 / 29 occupied frames (d) Material is crossed from both sides Left release Right release Contact control (both) Figure 25. GPT-5.6-Sol: the rolling piece passes through the knight. The views follow the left-release branch before and during the crossing. This reconstruction of the same rolling task contains a nearby closed knight assembly whose five material meshes have no collision owner. Two initially separated, table-supported releases put the moving piece inside knight material for46and29frames, with maximum interior vertex depths of0.295and0.105scene units. Depth is the largest measured vertex intrusion in either direction between the two meshes. In (c), the left-release positions and velocities are exactly unchanged when the knight is removed, across all122recorded samples; crossing the centre plane is only a path illustration, while (d) measures actual material intrusion. Adding passive contact to the unchanged target geometry produces a response and avoids material intrusion for both releases. Repeated runs and doubled solver substeps retain the failure. Length values use native scene units. What the original trajectory leaves untested.The original-condition checks certify the rolling body’s rigidity, continuity and table support. They do not require a response from the nearby knight because the original motion never reaches it. The changed releases activate this missing contact path. This extends the dish example in Fig. 22 to a different submitted program and a different obstacle selected from its own scene. 35 D.3.5. A Solid Token Behaves as if It Were Absent (a) CF approach, frame 105 Target token (b) CF crossing, frame 130 Target token 100110120130140150 Frame 0.0 0.5 1.0 1.5 2.0 Moving-body x (scene units) Token centre Target x extent (c) Native contact control stops the approach Candidate Token removed Native contact control 100110120130140150 Frame 0.0 0.2 0.4 0.6 Min. sampled signed distance (scene units) Negative values indicate intrusion 32 intrusion frames for each release (d) Actual token material is penetrated Left release Right release Native contact control Figure 26. GPT-5.6-Luna: the rolling piece traverses a solid wooden token. The views track the left-release branch as the body enters the selected token. The selected target is a filled cylinder with no hole and no physical contact owner. Two target-reaching releases produce mesh intersections on frames 114–145 and 118–149, respectively:32consecutive frames each. The minimum cylinder signed distance over sampled moving-body surface points is approximately−0.110scene units in both branches. For the left release, removing the token changes none of the240sampled positions. A native contact control matches the prefix through frame 111 and then stops the approach without token penetration. Its collider responds before the actual material surface, with a dense measured separation of0.0622scene units; it is a sensitivity control, not an exact surface-contact trajectory. The grey band in (c) is only the target’s projectedxextent; (d) gives the material-distance witness. Scope of the comparison.The original motion avoids this obstacle, so its table-support checks can pass while the obstacle remains physically inactive. The intervention exposes the mismatch between rendered solid material and the program’s operative collision scene. The initial opposite-side placement blocked by another real object remains in the archived test matrix; the plotted opposite-side branch uses a separately measured, target-reaching release. All three reconstructions of this rolling task retain their own geometry and source scale; no common artificial obstacle is inserted to create the failure. 36 D.4. Cross-Harness Verification Harness instructions may influence when an agent checks its reconstruction and how it reserves execution budget for corrections. We explore this possible connection through two DeepSeek-V4.1-Flash tasks where Codex completes execution checks after its final revisions and obtains better event scores than Claude Code. Both agents perform earlier tests; the difference is whether the final repairs are followed by another successful run and output inspection. The galleries show videos generated by the benchmark verifier from the submitted programs, while Appendix D.4.3 examines the agents’ own verification steps and the harness guidance behind them. D.4.1. Checking Rolling After Contact Changes In the wooden-piece task, Claude Code performs prototype measurements but exhausts its Blender allowance before executing the final rewritten program. Its last checks find no saved world or completed frame set, and the finalizer refuses the submission (steps 199–205). Codex also initially produces a stalled object. It detects the stall, tests collision shapes and table resolution, and revises the contact geometry and incline (132–160). Its twentieth launch renders the final program, after which it inspects output frames and diagnostics and finalizes (163–169). Reference0.00 s3.97 s7.97 s Claude Code Codex Figure 27. DeepSeek-V4.1-Flash: wooden-piece rolling. Codex tests and revises the contact setup before completing the final render and checking its motion. Outcome. The benchmark event score rises from0for Claude Code to1for Codex. Appearance remains imperfect: both receiveS obj = 0.286. Codex’s tests are followed by contact changes, a new full run, and recovery of the rolling event. Claude Code’s final revision has static checks and earlier-prototype evidence, but no successful execution after the final changes. This pattern suggests that completing behavioral checks within the launch budget may matter for event fidelity. 37 D.4.2. Checking Accumulation Across a Pour In the wine-pouring task, Claude Code’s final production bake aborts. It then changes the solver substeps, initial-liquid clearance, and resolution, but has no remaining launch budget to execute those fixes; the agent’s output directory remains empty and finalization fails (224–248). Codex inspects sampled frames from a complete rendered sequence, identifies an excessive filling rate, and twice adjusts the inlet and reruns the program (80–90). It then inspects separated frames and checks a saved-scene preview at frame 240 against the delivered frame (91–94). Reference0.00 s3.97 s7.97 s Claude Code Codex Figure 28. DeepSeek-V4.1-Flash: wine pouring. Codex’s checks lead to repeated inlet corrections and a new full render after each correction. Outcome.The event score increases from0to0.857, andS vis from0.521to0.601. Codex still fails the event item requiring the tilted, visibly supported glass; not every metric improves. Its frame-240 preview has zero pixel difference from the corresponding delivered frame, which establishes that single-frame agreement only. The sequence illustrates how checks across time can reveal a filling-rate error and guide further corrections when execution budget remains. 38 D.4.3. Harness Instructions and Completed Verification What the prompts require. The archived Codex 0.150.1 sessions explicitly instruct the agent, in the recordednever approval mode, to “proactively run tests, lint and do whatever you need to ensure you’ve completed the task.” Its task prompt also starts with a reminder to reserve the last Blender launch for final validation. After removing this prefix, the benchmark task text is byte-identical across harnesses for both selected tasks: it asks the agent to build a minimal runnable scene, verify it, refine it, and runfinalize_submission. Thus Claude Code also receives an explicit task-level verification requirement. Version-matched local captures of Claude Code 2.1.274 and 2.1.278 emphasize faithful reporting of failed tests and skipped steps; we did not observe the same broad proactive-testing instruction in those captures. This part of the comparison uses local reproductions because the historical full Claude Code system prompts were not retained. Table 9. Prompt evidence for the selected DeepSeek-V4.1-Flash configurations. EvidenceClaude CodeCodex Benchmark taskExplicitly requires verification and finalization.Same task body; an additional leading reminder re- peats the final-validation budget requirement. Native promptHistorical full prompt unavailable. Version-matched local captures emphasize faithful reporting of tests and skipped steps. Historical session metadata records an explicit proactive-testing instruction under never. InterpretationEarlier checks are present, but the final revisions in these two cases remain unexecuted. The final revisions in these two cases are run and inspected before the attempt ends. The observed verification sequences. Table 10 connects the prompt comparison to concrete checks, corrections, and subsequent runs. The two Codex traces contain no observed bypass of the launch cap, backdating of the submitted source, or substitution of older frames for the final revision. The screenshots and scores describe verifier-stage videos; the table describes what the agents themselves managed to execute and inspect. Table 10. Final-revision verification in the two selected tasks. Step numbers refer to each case’s own agent trace. TaskClaude CodeCodex Wooden pieceRevises the final program after the cap is reached; static checks pass, but execution is blocked and out- put checks fail (166–205). Detects stalling, revises contact settings, executes the final program at launch 20, then checks frames and finalizes (132–169). Wine pouringAfter a bake abort, changes numerical settings with- out a successful subsequent execution; finalizer re- ports missing frames (224–248). Detects excessive accumulation, adjusts the inlet twice, reruns after each edit, inspects final frames, and checks a saved-scene preview (80–94). Possible connection to harness guidance. The event-score differences in these examples may be related to the harness’s emphasis on proactive verification, the timing of behavioral checks, and the allocation of the limited launch budget. Codex’s instructions explicitly encourage verification, and its successful revisions here are followed by simulation and output inspection. The two cases illustrate how such guidance may help an agent turn detected defects into tested corrections and better event reconstruction. 39