Abstract

Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibration, all of which are often ambiguous or unavailable in generated video. We propose a different approach. When two objects in the same scene obey the same physical law, their motions must satisfy predictable relationships, and these relationships hold independent of calibration. We introduce Principia, a benchmark that evaluates Newtonian physics through relational consistency between paired objects. Principia spans eight phenomena – gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation—across translational, rotational, collisional, and oscillatory dynamics, using real-world scenes recorded under controlled protocols. We also introduce a calibration-independent consistency score that quantifies physical violation directly in image space. Across thousands of generations from six state-of-the-art video generators, no model exceeds 0.42 on Principia despite all scoring around 0.8 on VBench. Vision-language models are evaluated on their ability to detect relational physics violations, with the best model achieving only 67% accuracy and most performing near chance level.

1 Introduction

Evaluation and benchmarks have been a strong basis for analyzing and improving modern machine learning models. Today’s video generators have reached a point where they can produce highly realistic clips that are hard to distinguish from real videos. VBench, one of the most widely used benchmarks for video generation, evaluates visual quality along several axes [26]. These have helped drive rapid progress in the field. However, in this paper, we show that visual realism does not imply physical consistency.

To this end, several benchmarks have recently been proposed to evaluate physical consistency in generated videos. Some of these evaluate whether motion appears physically plausible [2, 3]. Others compare generated trajectories against recorded videos [34, 29]. A few also evaluate motion against explicit physical laws [51]. However, plausibility is subjective and can be met by visually convincing but physically incorrect motion. Moreover, trajectory matching penalizes deviation from a single reference even when multiple futures are physically valid, and law-based evaluation often requires recovering or assuming metric quantities. Single-phenomenon studies like free-fall under gravity [42] further reveal targeted failure modes but do not establish whether models preserve physical structure across different physics phenomena.

To make physical evaluation central to modern video generators, especially given their potential use as world models for predicting the consequences of actions [13, 24], we propose Principia, a physics evaluation benchmark that takes into account the relative motion of two objects and measures the extent to which their relative motions satisfy the expected physical relation for eight phenomena (Fig. 1). We evaluate the model’s understanding of gravity in free fall, restitution measured from motion during and after collisions, sliding on inclined planes under friction, rotational inertia and momentum, projectile motion, pendulum swinging under gravity, and spring-mass systems under gravity.

The motivation for using relative motion is easily explained with the following example. Let us take a block moving down an inclined plane. In the case of a single block, as the visual input provides no direct information about material properties or mass, evaluation of the dynamics of sliding can only be carried out qualitatively, such as whether the block is accelerating down the inclined plane.

Consider two blocks of different masses placed under identical conditions (Fig. 2), i.e., having the same starting point on inclined planes with the same angle of inclination and coefficient of friction. Assuming that the two blocks follow the same physical laws, they will reach the bottom of the ramp simultaneously because the acceleration of a block sliding down an inclined plane with friction does not depend on its mass. This equality is an invariant of the underlying physical law; it should not depend on the absolute values of the physical variables. In general, physical laws impose invariants on the motion of multiple objects that can include equalities, ratios, and relative ordering in time and space. Such invariants can be computed from image-space measurements without estimating absolute quantities such as mass, scale, velocity, or acceleration.

For each physical phenomenon, we specify the expected relation between the motions of paired objects and measure how closely the generated motion satisfies that relation. We aggregate these deviations into a continuous invariance score. The test is straightforward: if two objects are released under the same conditions, they must reach the ground at the same time. A systematic deviation from this relationship indicates a violation of the tested physical law, irrespective of whether either individual motion appears plausible.

However, creating such a benchmark is difficult because small geometric, temporal, surface, or release asymmetries may produce signals indistinguishable from genuine physics violations, requiring strict experimental control. We captured hundreds of real-world videos, ensured synchronized release and controlled experiments, and conducted automated and manual validation at all steps to filter out bad videos. The obtained benchmark includes about 500 scenes of eight Newtonian phenomena covering the translational, rotational, collisional, and oscillatory dynamics categories.

The same relational tests can also be applied to VLMs. We additionally build a synthetic testbed in Isaac Sim[35] to generate controlled and counterfactual scenarios at scale. This provides a scalable way to evaluate both video generators and VLMs, and can further be used to develop methods for improving their physical consistency.

• Principia, a benchmark consisting of 500+ real-world scenes capturing 8 physics principles, with calibrated objects, aligned geometry, and coordinated release to facilitate comparison against analytic expectations.

• An invariant score of relational consistency which evaluates consistency relations purely in image space without any dependence on metric scale, velocity, or camera intrinsics.

• A scalable synthetic pipeline implemented in Isaac Sim that generates physical scenarios, both controlled and counterfactual, allowing for evaluation of VLMs and video generators.

• A benchmark evaluating six state-of-the-art video generators and four vision-language models reveals that no model achieves a score higher than $0.42$ on Principia despite achieving scores around 0.8 on standard visual benchmarks, and that increasing the size of a network does not improve its physical consistency.

2 Related Work

Video generative models, world simulators, and physics-guided generation. Recent video diffusion and autoregressive models [25, 33, 9, 46] achieve high visual fidelity and are increasingly described as world models [13]. Probing studies suggest that such models somewhat encode 3D structure and motion [5, 7, 16, 47, 17, 50, 6, 27, 40], but perceptual realism does not imply adherence to physical laws. Several approaches attempt to improve physical realism through reward-based fine-tuning [29, 28], trajectory correction [52], or integration of physics engines [31, 48, 45, 11], all of which depend on how physical correctness is evaluated.

Benchmarking physical reasoning. Existing benchmarks fall into four families (Table 1): simulation-based reasoning in calibrated environments [39, 8, 4, 43, 1, 30]; VQA-based VLM evaluation [12, 18, 15]; generator commonsense scoring via human or VLM judges [2, 3, 23, 32, 22]; and real-video physics evaluation [34, 51, 49, 19, 29]. All require absolute measurements—calibration, scale, or recorded ground-truth parameters—that may be ambiguous in generated video. The closest precursor is the two-object gravity protocol of Thozhiyoor et al. [42], which sidesteps calibration by testing ratios of fall times to isolate Galileo’s principle: unit-free, relational, and quantitative. Principia generalizes this single-phenomenon protocol to eight Newtonian phenomena spanning translational, rotational, collisional, and oscillatory dynamics, and evaluates both video generators and vision-language models.

媒体内容 · 前往原文查看

Work	Unit-Free	Relational	Quant. Physics	Multi-Phys.	Real	Gen	VLM
Simulation-based reasoning
IntPhys [39, 8]	✗	✗	✗	✓	✗	✗	✓
PHYRE/IPHYRE [1, 30], Physion [4, 43]	✗	✗	✗	✓	✗	✗	✗
VQA-based VLM evaluation
PhysBench [12], CausalVQA [18], MVPBench [15]	✗	✗	✗	✓	✓	✗	✓
Generator commonsense & plausibility
VideoPhy [2, 3], T2VPhysBench [23],	✗	✗	✗	✓	✗	✓	✗
PhyGenBench [32], PhyWorldBench [22]
NewtonRewards [28]	✗	✗	✓	✓	✗	✓	✗
Real-video physics evaluation
Physics-IQ [34]	✗	✗	✗	✓	✓	✓	✗
Morpheus [51]	✗	✗	✓	✓	✓	✓	✗
PhysVid [49]	✗	✗	✓	✓	✓	✗	✓
Delfys75 [19]	✗	✗	✓	✓	✓	✗	✗
PISA [29]	✗	✗	✗	✗	✓	✓	✗
Unit-free relational
Galileo’s equivalence [42]	✓	✓	✓	✗	✗	✓	✗
Principia (ours)	✓	✓	✓	✓	✓	✓	✓

3 The Principia Dataset

Principia consists of 500+ real-world paired-object scenes spanning eight Newtonian phenomena (Table 2). Each scene contains two objects whose motions must satisfy a relational invariant independent of camera, scale, or frame rate. All parameters are held fixed except the one varied (mass, height, length, or moment of inertia), isolating the law being tested.

3.1 Construction Protocol

Relational invariants are sensitive to small perturbations: a slight asymmetry in incline angle, a timing skew between paired releases, or a small lateral push on a pendulum can produce a relational signal indistinguishable from a real physics violation. Construction error must be small enough that it does not obscure the violations we aim to detect, which makes data collection the bulk of the work. We recorded approximately 750+ videos in total and applied heavy filtering to retain those meeting the conditions each invariant assumes. The resulting dataset is further augmented using editing models[20] to increase diversity, yielding a final dataset comprising 529 scenes.

We enforce four constraints during data collection. Matched geometry: paired objects share identical contact surfaces, ramps, supports, or springs, machined or assembled to a common specification (for example, the rotational-inertia experiment uses a solid Delrin cylinder and a hollow aluminum cylinder manufactured in-house to match in mass, height, and outer radius within $5\%$). Synchronized release: mechanical alignment guides hold both objects in matched starting poses and release them simultaneously; hand-released objects are avoided except for pendulum swings. Controlled surface material: contact surfaces are validated for each pairing, with no-slip conditions inspected for rolling experiments and contact faces inspected for friction experiments. Minimized external forces: lateral velocity at release is held below visible drift, air resistance is negligible on the relevant timescales, and phenomenon-specific assumptions are verified individually (no visible slip on rolling ramps, collision axis aligned with the motion direction for momentum).

We segment and track all objects with SAM3 [10] using hand-annotated initial points. All subsequent measurements happen in pixel space; the relational consistency score depends only on ratios and equalities of pixel-space quantities, so evaluation requires no knowledge of camera intrinsics, frame rate, or metric scale.

Each session yields multiple takes per scene; we include only those passing manual inspection of the recorded video and the resulting SAM3 trajectory (for example, confirming monotonic descent for ramp scenarios, clean ball-on-ground transitions for restitution, and periodic motion for pendulum and mass-spring scenes). Common excluded failure modes include lateral drift, asymmetric release timing, pendulum motion outside the small-angle regime, ball spin biasing projectile trajectories, and surface anomalies producing slip in rolling experiments.

3.2 Principia-Synth

In addition to the recorded real-world videos, we construct a synthetic dataset for each phenomenon using Isaac Sim [35, 36]. The physics simulator allows us to render both physically correct videos and corresponding anti-physics videos, which are generated by explicitly violating the relational invariant associated with each phenomenon. We use this Principia-Synth dataset to evaluate whether VLMs can identify relational physics failures. Since such anti-physics videos cannot be recorded in the real world, and AI-generated videos that violate physical constraints often contain visual artifacts that could confound the evaluation, we instead rely on simulation to generate controlled anti-physics scenarios. Representative examples of the real-world physics and anti-physics videos for each phenomenon are provided in section 3.3 and Appendix B.1 respectively.

3.3 Phenomena

Principia covers four types of Newtonian dynamics: translational, rotational, collisional, and oscillatory dynamics. Free-fall and projectile dynamics examine translational dynamics under gravity, while inclined-plane dynamics additionally involve friction. The study of rotational inertia involves rotational dynamics and the interaction between translational and rotational motion under no-slip rolling conditions. Restitution and momentum dynamics investigate collisional dynamics and the relationships between the motions of colliding bodies. Pendulum and mass-spring dynamics investigate oscillatory dynamics under gravity and elastic restoring forces. Representative samples from Principia-Synth are shown alongside the description of each phenomenon.

Restitution and Gravity.

Two identical balls are dropped from different heights onto the same surface. The rebound height satisfies $r=e^{2}h$, so the relational invariant is $r_1/h_1=r_2/h_2$. The same recordings yield Galileo’s gravity invariant $h_1/h_2={\left(t_1/t_2\right)}^2$.

Friction.

Two blocks of different mass slide down opposite faces of a tent-shaped ramp with identical surface material and angle. Because acceleration on an incline with friction satisfies $a=g\left(\sin \theta -\mu \cos \theta \right)$, the result is mass-independent. The relational invariant is therefore equality of arrival times, $t_1=t_2$, despite the mass difference between the two blocks.

Rotational Inertia.

A solid and a hollow cylinder of matched mass and radius roll down opposite faces of a no-slip ramp. Pure rolling acceleration is $a=g\sin \theta /\left(1+I/m{R}^2\right)$, so the cylinder with greater moment of inertia arrives later. The relational invariant is the time ratio $t_1/t_2=\sqrt{\left(1+I_1/m{R}^2\right)/\left(1+I_2/m{R}^2\right)}$.

Momentum.

Two identical balls are released from different heights on a single ramp and collide with identical blocks placed at matched distances. The ball released higher imparts greater momentum and pushes its block farther. The relational invariant is ($x\propto h$). In a second setup, identical balls collide with blocks of different masses. The heavier block travels a shorter distance, giving the relational invariant ($x\propto 1/m$).

Projectile Motion.

Two spheres are launched in opposite directions from a central platform at distinct heights with fixed launch angle. Greater height produces greater launch velocity ($v\propto \sqrt{h}$) and therefore greater range ($R\propto v$). The relational invariant is range ordering: the higher-launched sphere lands farther.

Pendulum.

Two pendulums of different length and identical bob mass are released from the same small angle. The period $T=2\pi \sqrt{l/g}$ is independent of bob mass, so the relational invariant is $T_1/T_2=\sqrt{l_1/l_2}$. The longer pendulum therefore oscillates more slowly, with a proportionally larger period.

Mass-Spring.

Two springs with identical spring constants are loaded with masses $m_1$ and $m_2$ and allowed to settle at equilibrium. By Hooke’s law, equilibrium extension is $x=mg/k$, so extension scales linearly with suspended mass. The relational invariant is $x_1/x_2=m_1/m_2$, with proportional extensions.

媒体内容 · 前往原文查看

Scenario	Constants	Variables	Relational Invariant
Restitution	Material, surface	Drop heights $h_1,h_2$	$\sqrt{r_1/h_1}=\sqrt{r_2/h_2}$
Gravity	Drop conditions	Drop heights $h_1,h_2$	$h_1/h_2={\left(t_1/t_2\right)}^2$
Friction	Incline angle $\theta$, friction coeff. $\mu$	Block masses $m_1,m_2$	$t_1=t_2$
Rotational Inertia	Mass $m$, radius $R$, angle $\theta$	Moment of inertia $I_1,I_2$	$\frac{t_1}{t_2}=\sqrt{\frac{1+I_1/m{R}^2}{1+I_2/m{R}^2}}$
Projectile	Launch angle $\theta$, gravity $g$	Launch heights $h_1,h_2$	$R\propto v\propto \sqrt{h}$
Momentum (height)	Block mass $m$, ball mass	Release heights $h_1,h_2$	$x\propto v\propto h$
Momentum (mass)	Release height $h$, ball mass	Block masses $m_1,m_2$	$x\propto \frac{1}{m}$
Pendulum	Release angle $\theta$, gravity $g$	Lengths $l_1,l_2$	$T_1/T_2=\sqrt{\left(l_1/l_2\right)}$
Mass-Spring	Spring constant $k$, gravity $g$	Masses $m_1,m_2$	$x_1/x_2=\left(m_1/m_2\right)$

4 Experiments

4.1 Setup

We evaluate six video generators (Omni [21], Veo-3.1 [38], Wan2.2-5B/14B [44], Cosmos-2.5-2B/14B [41]) and four vision-language models (Gemini-3.1-Pro, Gemini-3-Flash [14], Qwen-32B, Qwen-4B [37]). Each generator is conditioned on a text prompt and the first frame of a real recording, with the experimenter, suspension strings, and visible release mechanisms inpainted out using Nano Banana 2 [20] so the model conditions on physics rather than apparatus. We sample each scenario using multiple random seeds and average the resulting scores across seeds to obtain a sample score. The Principia benchmark assumes that models can generate basic qualitatively correct motion (e.g., a dropped object moves downward or an object on an inclined plane moves down the slope). We therefore filter out non-conforming videos before computing the Principia scores(More details in Appendix A.6). Total inference compute exceeds 2,600 A100-hours across the four open-weights generators (Appendix A.5).

Principia Consistency Score.

For each phenomenon $\varphi$, the invariant defines two scalar quantities ${ℱ}_{\varphi }\left(o_1\right)$ and ${ℱ}_{\varphi }\left(o_2\right)$ that should be equal under correct physics. Depending on the phenomenon, these quantities may correspond either to directly measured values (friction) or to ratios derived from the measured values (gravity, spring, restitution, rotational inertia, pendulum). The specific quantities used for each phenomenon are listed in Table 2. We measure how closely a generated video satisfies the invariant using a normalized consistency score:

$S_{\varphi }=\mathrm{\hspace{0.33em}1}-\frac{\left|{ℱ}_{\varphi }\left(o_1\right)-{ℱ}_{\varphi }\left(o_2\right)\right|}{\left|{ℱ}_{\varphi }\left(o_1\right)\right|+\left|{ℱ}_{\varphi }\left(o_2\right)\right|}.$

$S_{\varphi }$ equals $1$ when the invariant holds exactly and decreases toward $0$ as the violation grows; concretely, $S_{\varphi }=0.95$ corresponds to a roughly $10\%$ relational asymmetry. The projectile and momentum invariants is qualitative – distance ordering rather than equality – and is scored separately as the fraction of scenes where the ordering is satisfied. The normalization makes $S_{\varphi }$ unit-free and bounded in $[0,1]$; per-phenomenon scores reported throughout this paper are calculated by taking mean and standard deviation across sample scores.

4.2 Visual Quality and Physical Fidelity Are Decoupled

媒体内容 · 前往原文查看

Figure 3(a) plots VBench[26] against Principia. All six generators cluster around $0.8$ on VBench but between $0.14$ and $0.42$ on Principia – visual quality and physical fidelity are nearly orthogonal. Models at the visual-quality frontier are no more likely to satisfy physical invariants than substantially lower-quality alternatives.

4.3 Per-Phenomenon Results: Generators

媒体内容 · 前往原文查看

Model	Restitution	Gravity	Friction	Inertia	Projectile	Momentum	Spring	Pendulum	Overall
Veo-3.1 [38]	0.479 $\pm$ 0.354	0.524 $\pm$ 0.378	0.387 $\pm$ 0.400	0.669 $\pm$ 0.380	0.250 $\pm$ 0.370	0.000 $\pm$ 0.000	0.381 $\pm$ 0.160	0.343 $\pm$ 0.309	0.379 $\pm$ 0.186
Cosmos-2.5-2B [41]	0.272 $\pm$ 0.301	0.378 $\pm$ 0.357	0.078 $\pm$ 0.176	0.044 $\pm$ 0.154	0.000 $\pm$ 0.000	0.074 $\pm$ 0.177	0.130 $\pm$ 0.167	0.205 $\pm$ 0.191	0.148 $\pm$ 0.120
Cosmos-2.5-14B [41]	0.179 $\pm$ 0.233	0.365 $\pm$ 0.337	0.319 $\pm$ 0.339	0.171 $\pm$ 0.303	0.019 $\pm$ 0.094	0.015 $\pm$ 0.084	0.147 $\pm$ 0.161	0.380 $\pm$ 0.268	0.199 $\pm$ 0.135
Wan2.2-5B [44]	0.158 $\pm$ 0.243	0.220 $\pm$ 0.262	0.307 $\pm$ 0.357	0.470 $\pm$ 0.371	0.009 $\pm$ 0.067	0.067 $\pm$ 0.249	0.065 $\pm$ 0.125	0.346 $\pm$ 0.244	0.205 $\pm$ 0.150
Wan2.2-14B [44]	0.504 $\pm$ 0.338	0.540 $\pm$ 0.344	0.709 $\pm$ 0.291	0.729 $\pm$ 0.260	0.231 $\pm$ 0.300	0.017 $\pm$ 0.090	0.099 $\pm$ 0.150	0.519 $\pm$ 0.235	0.419 $\pm$ 0.253
Omni [21]	0.277 $\pm$ 0.327	0.295 $\pm$ 0.334	0.852 $\pm$ 0.264	0.784 $\pm$ 0.261	0.148 $\pm$ 0.266	0.054 $\pm$ 0.155	0.308 $\pm$ 0.205	0.554 $\pm$ 0.241	0.409 $\pm$ 0.272

Table 3 reports per-phenomenon results. No generator exceeds an average consistency score of $0.5$, highlighting the difficulty of maintaining relational physical consistency across diverse scenarios. Wan2.2-14B achieves the best overall performance with an average score of $0.419$, narrowly outperforming Omni and Veo-3.1 at $0.409$ and $0.38$ respectively despite their closed-frontier status. Within-family scaling trends are highly uneven. Scaling Cosmos-2.5 from 2B to 14B produces modest gains in friction ($+0.24$), inertia ($+0.13$), and pendulum consistency ($+0.18$), while reducing performance on restitution ($-0.09$), gravity ($-0.01$), and momentum ($-0.06$). Similarly, scaling Wan2.2 from 5B to 14B produces substantial gains across restitution ($+0.35$), gravity ($+0.32$), friction ($+0.40$), inertia ($+0.26$), and projectile ($+0.22$), with smaller gains on spring ($+0.03$) and pendulum ($+0.17$), while momentum consistency slightly decreases ($-0.05$). These results suggest that larger model scale does not uniformly improve physical reasoning, and that different invariants stress distinct failure modes in current video generators. Compute also fails to predict fidelity – Cosmos-2.5-14B uses more compute per video than Wan2.2-14B (74 vs 66 min) yet scores $0.22$ lower (Appendix A.3).

Figure 3(b) visualizes per-model failure profiles. Each polygon exhibits a distinct performance profile rather than uniform weakness. Omni performs particularly well on inertia and friction but struggles with restitution, gravity, and momentum. Wan2.2-14B achieves the strongest overall performance, reaching the performance frontier among evaluated models across all scenarios except spring and momentum. The Cosmos models exhibit smaller polygon areas, indicating weaker physical consistency across phenomena. All models perform poorly on the momentum task, potentially reflecting the greater difficulty of modeling complex multi-object interactions. Notably, no model dominates across all physical phenomena, with none fully covering the radar chart. Qualitative comparisons appear in Figures 4,5.

4.4 Vision-Language Models

媒体内容 · 前往原文查看

Model	Restitution	Gravity	Friction	Inertia	Projectile	Momentum	Spring	Pendulum	Overall
Gemini-3.1-Pro [14]	0.439	0.489	0.498	0.517	0.517	0.536	0.843	0.519	0.545
Gemini-3-Flash [14]	0.594	0.644	0.840	0.465	0.665	0.556	0.895	0.626	0.661
Qwen-32B [37]	0.460	0.553	0.467	0.427	0.517	0.411	0.587	0.579	0.500
Qwen-4B [37]	0.588	0.580	0.642	0.427	0.547	0.450	0.535	0.557	0.541

VLMs are evaluated on whether they can detect relational physics violations rather than generate physically correct motion. We evaluate the models on a dataset comprising both real-world samples and synthetic samples from the Principia-Synth dataset. Principia-Synth consists of rendered videos depicting both real-world physics and physics violations for each phenomenon. The anti-physics videos are constructed by explicitly violating the relational invariant associated with each phenomenon. For every scene, the VLM receives the full video along with a phenomenon-specific PASS/FAIL prompt (Appendix B.2); its response is then compared against the known ground truth. Table 4 reports the agreement scores for each phenomenon. No VLM achieves an average agreement exceeding $0.7$, indicating near-chance performance and suggesting architectural, rather than scale-related, limitations.

5 Discussion and Conclusion

Video generators are increasingly marketed as world models. The results on Principia suggest otherwise: across six state-of-the-art generators, no model exceeds $0.45$ on the continuous metric despite all scoring around $0.8$ on visual quality benchmarks; scaling within an architecture regresses on at least one phenomenon in both the Cosmos and Wan families; and open-weights exceeds closed-frontier (Wan2.2-14B exceeds Veo-3.1 and Omni). Current generators produce highly realistic textures and visually appealing renderings but fail to enforce the relational invariants that physical laws require. They have learned to render the world without learning what holds it together.

Vision-language models are no better at detecting physical consistency than video generators are at producing it. Most models perform near chance level, with even the best-performing model achieving only 67% accuracy in distinguishing physically consistent videos from those containing physics violations.

Scope and limitations.

Principia tests macroscopic Newtonian mechanics; it does not address fluid dynamics, soft-body deformation, or thermodynamic phenomena. Generators sometimes produce videos with hallucinated objects, missing objects, or severe deformations – failures unrelated to physical reasoning. We do not automatically separate these from genuine physics violations; our scope is physical-law adherence in videos that are otherwise interpretable. As generators mature, more of their output will be evaluable under Principia without changes to the benchmark itself.

Resources and outlook.

Final-scale evaluation generated over $\mathrm{4,000}$ videos at approximately $\mathrm{2,600}$ A100-hours, with comparable additional compute consumed during prompt iteration, threshold calibration, and apparatus development. Because the consistency score is computed within each generated video, the benchmark is robust to test-set contamination. The first frames are precisely staged physical setups that current image generators cannot reproduce, but they can be edited cheaply to vary backgrounds and lighting using modern image editing models [20], enabling extension to new visual setups. We release first-frame inputs and scenario prompts; full real-world videos will be held back to preserve benchmark integrity. If video generators are to serve as world simulators, they must preserve the structural constraints that physical laws impose—not merely the appearance of motion. Closing this gap will likely require new training signals or architectural biases for relational invariants.

References

• [1] Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. Phyre: A new benchmark for physical reasoning. Advances in Neural Information Processing Systems, 32, 2019.
• [2] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation, 2024.
• [3] Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation in video generation. arXiv preprint arXiv:2503.06800, 2025.
• [4] Daniel M Bear, Elias Wang, Damian Mrowca, Felix J Binder, Hsiao-Yu Fish Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. arXiv preprint arXiv:2106.08261, 2021.
• [5] Anand Bhattad, Daniel McKee, Derek Hoiem, and David Forsyth. Stylegan knows normal, depth, albedo, and more. Advances in Neural Information Processing Systems, 36:73082–73103, 2023.
• [6] Anand Bhattad, Konpat Preechakul, and Alexei A Efros. Visual jenga: Discovering object dependencies via counterfactual inpainting. arXiv preprint arXiv:2503.21770, 2025.
• [7] Anand Bhattad, James Soole, and David A Forsyth. Stylitgan: Image-based relighting via latent control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4231–4240, 2024.
• [8] Florian Bordes, Quentin Garrido, Justine T Kao, Adina Williams, Michael Rabbat, and Emmanuel Dupoux. Intphys 2: Benchmarking intuitive physics understanding in complex synthetic environments. arXiv preprint arXiv:2506.09849, 2025.
• [9] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024.
• [10] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719, 2025.
• [11] Boyuan Chen, Hanxiao Jiang, Shaowei Liu, Saurabh Gupta, Yunzhu Li, Hao Zhao, and Shenlong Wang. Physgen3d: Crafting a miniature interactive world from a single image. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 6178–6189, 2025.
• [12] Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Campagnolo Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision-language models for physical world understanding. In The Thirteenth International Conference on Learning Representations, 2025.
• [13] Kenneth James Williams Craik. The nature of explanation, volume 445. CUP Archive, 1967.
• [14] Google DeepMind. Gemini 3 pro: Multimodal foundation model. Online model release, 2026.
• [15] Zhuobai Dong, Junchao Yi, Ziyuan Zheng, Haochen Han, Xiangxi Zheng, Alex Jinpeng Wang, Fangming Liu, and Linjie Li. Seeing is not reasoning: Mvpbench for graph-based evaluation of multi-path visual physical cot, 2025.
• [16] Xiaodan Du, Nicholas Kolkin, Greg Shakhnarovich, and Anand Bhattad. Generative models: What do they know? do they know things? let’s find out! arXiv preprint arXiv:2311.17137, 2023.
• [17] Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Abhishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Probing the 3D Awareness of Visual Foundation Models. In CVPR, 2024.
• [18] Aaron Foss, Chloe Evans, Sasha Mitts, Koustuv Sinha, Ammar Rizvi, and Justine T Kao. Causalvqa: A physically grounded causal reasoning benchmark for video models. arXiv preprint arXiv:2506.09943, 2025.
• [19] Alejandro Castañeda Garcia, Jan Warchocki, Jan van Gemert, Daan Brinks, and Nergis Tomen. Learning physics from video: Unsupervised physical parameter estimation for continuous dynamical systems. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 27924–27933, 2025.
• [20] Google DeepMind. Gemini 3.1 flash image. https://deepmind.google/models/gemini-image/flash/, 2026.
• [21] Google DeepMind. Gemini omni flash: Model card. https://deepmind.google/models/model-cards/gemini-omni-flash/, 2026.
• [22] Jing Gu, Xian Liu, Yu Zeng, Ashwin Nagarajan, Fangrui Zhu, Daniel Hong, Yue Fan, Qianqi Yan, Kaiwen Zhou, Ming-Yu Liu, and Xin Eric Wang. "phyworldbench": A comprehensive evaluation of physical realism in text-to-video models, 2025.
• [23] Xuyang Guo, Jiayan Huo, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Jiale Zhao. T2vphysbench: A first-principles benchmark for physical consistency in text-to-video generation, 2025.
• [24] David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2018.
• [25] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633–8646, 2022.
• [26] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
• [27] Sonia Joseph, Quentin Garrido, Randall Balestriero, Matthew Kowal, Thomas Fel, Shahab Bakhtiari, Blake Richards, and Mike Rabbat. Interpreting physics in video world models. arXiv preprint arXiv:2602.07050, 2026.
• [28] Minh-Quan Le, Yuanzhi Zhu, Vicky Kalogeiton, and Dimitris Samaras. What about gravity in video generation? post-training newton’s laws with verifiable rewards. 2025.
• [29] Chenyu Li, Oscar Michel, Xichen Pan, Sainan Liu, Mike Roberts, and Saining Xie. Pisa experiments: Exploring physics post-training for video diffusion models by watching stuff drop. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
• [30] Shiqian Li, Kewen Wu, Chi Zhang, and Yixin Zhu. I-PHYRE: Interactive physical reasoning. In The Twelfth International Conference on Learning Representations, 2024.
• [31] Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics-grounded image-to-video generation. In European Conference on Computer Vision (ECCV), 2024.
• [32] Fanqing Meng, Jiaqi Liao, Xinyu Tan, Quanfeng Lu, Wenqi Shao, Kaipeng Zhang, Yu Cheng, Dianqi Li, and Ping Luo. Towards world simulator: Crafting physical commonsense-based benchmark for video generation. In Forty-second International Conference on Machine Learning, 2025.
• [33] Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world models. arXiv preprint arXiv:2209.00588, 2023.
• [34] Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? arXiv preprint arXiv:2501.09038, 2025.
• [35] NVIDIA. Isaac Sim. https://github.com/isaac-sim/IsaacSim.
• [36] Mihir Prabhudesai, Aryan Satpathy, Yangmin Li, Zheyang Qin, Nikash Bhardwaj, Amir Zadeh, Chuan Li, Katerina Fragkiadaki, and Deepak Pathak. Solving physics olympiad via reinforcement learning on physics simulators, 2026.
• [37] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026.
• [38] Google Research and DeepMind. Veo 3.1: Text-to-video model. Online model release, 2026.
• [39] Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. Intphys: A framework and benchmark for visual intuitive physics reasoning. arXiv preprint arXiv:1803.07616, 2018.
• [40] Ayush Sarkar, Hanlin Mai, Amitabh Mahapatra, Svetlana Lazebnik, David A Forsyth, and Anand Bhattad. Shadows don’t lie and lines can’t bend! generative models don’t know projective geometry… for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 28140–28149, 2024.
• [41] NVIDIA Cosmos Team. Cosmos-predict2: World foundation models for physical ai. https://github.com/nvidia-cosmos/cosmos-predict2, 2025. GitHub repository, Apache-2.0 license.
• [42] Varun Varma Thozhiyoor, Shivam Tripathi, Venkatesh Babu Radhakrishnan, and Anand Bhattad. Objects in generated videos are slower than they appear: Models suffer sub-earth gravity and don’t know galileo’s principle… for now. arXiv preprint arXiv:2512.02016, 2025.
• [43] Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Josh Tenenbaum, Dan Yamins, Judith Fan, and Kevin Smith. Physion++: Evaluating physical scene understanding that requires online inference of different physical properties. Advances in Neural Information Processing Systems, 36:67048–67068, 2023.
• [44] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.
• [45] Chen Wang, Chuhao Chen, Yiming Huang, Zhiyang Dou, Yuan Liu, Jiatao Gu, and Lingjie Liu. Physctrl: Generative physics for controllable and physics-grounded video generation. arXiv preprint arXiv:2509.20358, 2025.
• [46] Thaddäus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, and Robert Geirhos. Video models are zero-shot learners and reasoners. arXiv preprint arXiv:2509.20328, 2025.
• [47] Xiaoyan Xing, Konrad Groh, Sezer Karaoglu, Theo Gevers, and Anand Bhattad. Luminet: Latent intrinsics meets diffusion models for indoor scene relighting. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 442–452, 2025.
• [48] Yu Yuan, Xijun Wang, Tharindu Wickremasinghe, Zeeshan Nadir, Bole Ma, and Stanley Chan. Newtongen: Physics-consistent and controllable text-to-video generation via neural newtonian dynamics. arXiv preprint arXiv: 2509.21309, 2025.
• [49] Guanqi Zhan, Xianzheng Ma, Weidi Xie, and Andrew Zisserman. Inferring dynamic physical properties from video foundation models. arXiv preprint arXiv:2510.02311, 2025.
• [50] Guanqi Zhan, Chuanxia Zheng, Weidi Xie, and Andrew Zisserman. A general protocol to probe large vision models for 3d physical understanding. Advances in Neural Information Processing Systems, 37:43468–43498, 2024.
• [51] Chenyu Zhang, Daniil Cherniavskii, Antonios Tragoudaras, Antonios Vozikis, Thijmen Nijdam, Derck W. E. Prinzhorn, Mark Bodracska, Nicu Sebe, Andrii Zadaianchuk, and Efstratios Gavves. Morpheus: Benchmarking physical reasoning of video generative models with real physical experiments, 2025.
• [52] Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models. arXiv preprint arXiv:2505.23656, 2025.

Appendix

Appendix A Video Generator Evaluation

Video generators are evaluated on their ability to generate videos satisfying the relational constraint of the phenomenon. In this section we provide additional details on the Principia benchmark and evaluation protocol.

A.1 Dataset Statistics:

The dataset consists of 401 real-world scenes captured in-house. For each scene, the first frame is extracted and used along with a text prompt as the conditioning for video generation. The conditioning frames are first edited using image-editing models [20] to remove the human experimenter and experimental apparatus. These edited frames are then further augmented to increase visual diversity, resulting in a total of 529 scenes. The per-scenario composition of the dataset is shown in Fig. 6.

媒体内容 · 前往原文查看

A.2 Evaluation protocol

All quantitative measurements are extracted automatically from the generated videos. Object trajectories are obtained using SAM3[10], which tracks the position(centroid/bottom) of each object throughout the sequence. Phenomenon-specific events (e.g., impact, arrival, turning points) are then detected from the tracked trajectories to compute the quantities required for evaluating the corresponding relational invariant.

Restitution and Gravity.

The centroid trajectory of each ball is tracked using SAM3. Ground impact is detected when the centroid falls below a predefined ground threshold and the vertical velocity changes sign from positive (downward) to negative (upward). The rebound apex is identified as the subsequent frame where the vertical velocity changes from negative to positive. Initial drop height, rebound height, and time of flight are computed from these detected events and used to evaluate the restitution and gravity invariants.

Friction and Rotational Inertia.

For both scenarios, the arrival time of each object is measured as the first frame in which its bottom most pixel(to handle size differences in blocks) crosses a predefined annotated threshold at the end of the incline. These arrival times are used to evaluate the friction and rotational invariants.

Projectile Motion.

The projectile centroid trajectory is tracked using SAM3. Ground impact is detected when the vertical velocity becomes positive after descent and the centroid crosses the ground threshold. The horizontal range is computed as the horizontal distance between the impact location and the annotated launch point at the end of the incline.

Pendulum.

The oscillation period of each pendulum is estimated from the centroid trajectory of the bob. A half-period is detected when the horizontal velocity reverses sign after the bob has crossed the pendulum’s vertical midpoint (anchor line). The full period is obtained by doubling the measured half-period.

Spring.

The maximum extension of each spring is measured from the first oscillation. The oscillation turning point is detected when the vertical velocity changes from positive to negative, and the corresponding vertical displacement is recorded. Object masses are estimated from the cube side lengths assuming constant density, i.e., $m\propto l^3$, and the measured displacement ratio is compared against the expected mass ratio.

Momentum.

The vertical displacement of each target block is measured by tracking the bottom edge of the block. The maximum post-collision displacement is extracted from the trajectory and compared across the two interactions to evaluate the expected ordering of momentum transfer.

媒体内容 · 前往原文查看

Model	Resolution	Frames	FPS	Time
Wan2.2-5B [44]	1280$\times$704	121	24	7 min
Wan2.2-14B [44]	1280$\times$720	81	16	66 min
Cosmos-2.5-2B [41]	1280$\times$704	93	16	15 min
Cosmos-2.5-14B [41]	1280$\times$704	93	16	74 min
Veo-3.1 [38]	1280$\times$720	96	24	API
Omni [21]	1280$\times$720	240	24	API

A.3 Model Configurations.

We use the default configuration of each evaluated model to obtain high-quality generations. Resolution, frame count, and frame rate are reported in Table 5. Inference times are measured on an NVIDIA A100 80GB and refer to wall-clock time per generated video. Veo-3.1 and Omni are accessed through public APIs.

A.4 Prompts Structure

We list the prompts used to generate evaluation videos with Principia. Each video generation model is conditioned on the first frame of the video along with the corresponding prompt.

• Restitution/Gravity: A video showing two identical balls being dropped from two different heights onto the ground. The camera is static and positioned to clearly capture the vertical motion of both balls. Both balls fall naturally under gravity, accelerating freely with no air resistance and hit the ground. The balls bounce a few times on the ground before coming to rest.

• Friction: A video showing two identical inclined planes with the same angle of inclination. Two blocks of the same material are placed on the inclines at the same height from the bottom. The blocks start from rest and slide down the planes. The camera is fixed and motion is strictly along the planes.

• Rotational Inertia: A video showing two identical inclined planes with the same angle of inclination. A solid cylinder is placed on one plane and a hollow cylinder of the same mass and outer radius is placed on the other. Both cylinders start from rest at the same height and roll down their planes without slipping. The camera is fixed and the motion is strictly along the planes.

• Projectile: A video showing two identical inclined planes with the same angle of inclination positioned on an elevated surface. Two identical balls are placed on the inclines. The balls start from rest, roll down the planes, and launch off the edge to the ground. The camera is fixed and captures the complete motion of the balls.

• Momentum: A video from a fixed camera showing a single wide inclined plane facing directly towards the camera. Two identical tennis balls are placed on the incline at different/same starting heights. Both balls start from rest and roll down the plane towards the camera without slipping. Two solid wooden square blocks of same/different masses are placed at the same height from the bottom, directly in the path of each ball. The balls collide head-on with their respective wooden blocks, causing them to move forward. The motion is strictly along the plane and the horizontal direction after the collision.

• Mass-Spring: A video from a fixed camera showing two vertical spring-mass systems suspended side-by-side from a fixed horizontal support. Both springs are entirely identical, having the same unstretched length and the same spring constant. Two wooden cube blocks having different masses are attached to the springs. Initially, both blocks are held elevated right up near the horizontal support, fully compressing the springs. At time zero, both blocks are released simultaneously from rest. The blocks fall under gravity, stretching the springs downwards, and then oscillate vertically about their respective equilibrium positions. The camera is fixed and clearly captures the continuous vertical motion of the system.

• Pendulum: A video from a fixed camera showing two simple pendulums suspended from the same horizontal support. The pendulum bobs are identical in size, shape, and material, but are attached to strings of different lengths. Both pendulums are initially displaced to the same angular position and released simultaneously from rest without any initial push. The pendulums swing freely under gravity in a plane parallel to the camera. The camera is fixed and captures the complete oscillatory motion of both pendulums.

A.5 Evaluation Compute.

Generating the corpus required substantial compute. Wan2.2-14B alone required approximately 1,100 A100-hours (46 days), while Cosmos-2.5-14B required approximately 1300 A100-hours (54 days) to generate the 529 scenes across two seeds. Across all four open-weight models, inference required approximately 2,800 A100-hours, equivalent to 120 days of continuous compute on a single A100.

A.6 Directional Consistency Score.

We define a Directional Consistency Score(DCS) that evaluates the generated object motion trajectory against an expected object motion trajectory to check for basic qualitative motion.

$\mathrm{DCS}=\frac{1}{\left|S\right|}\sum _{i\in S}\frac{\Delta {𝐩}_i^{\top }\widehat{𝐝}}{\Vert \Delta {𝐩}_i\Vert },$

(1)

where $\Delta {𝐩}_i={𝐩}_{i+1}-{𝐩}_i$ is the object’s displacement between consecutive frames, $\widehat{𝐝}$ is the unit vector representing the expected direction of motion, and $S=\{i:\Vert \Delta {𝐩}_i\Vert \ge ϵ\}$ is the set of frames with non-negligible displacement. Since both $\Delta {𝐩}_i/\Vert \Delta {𝐩}_i\Vert$ and $\widehat{𝐝}$ are unit vectors, $\mathrm{DCS}\in \left[-1,1\right]$, where a value of $1$ indicates perfect agreement with the expected direction and $-1$ indicates motion consistently opposite to the expected direction.

In all scenarios except the pendulum, $p_i$ denotes the vertical ($y$) coordinate of the tracked object, with the expected direction $\widehat{𝐝}=+1$ corresponding to downward motion in image coordinates. For the pendulum scenario, $p_i$ denotes the horizontal ($x$) coordinate, and $\widehat{𝐝}$ is defined toward the equilibrium position (the center). We use a Directional Consistency Score threshold of $0.8$ to filter non-conformational videos. Videos with a DCS below this threshold are assigned a Principia Score of $0$, thereby penalizing models that fail to generate the expected qualitative object motion.

A.7 Scaling Within Architecture

Figure 7 compares performance within each model family as parameters increase: Cosmos-2B to Cosmos-14B ($7\times$ scale) and Wan2.2-5B to Wan2.2-14B ($2.8\times$ scale). The annotations show the per-phenomenon delta, with negative values shown in red.

The picture is more nuanced than "scaling helps." Cosmos shows substantial improvement on friction ($+0.29$) and pendulum ($+0.24$), but regresses on restitution ($-0.09$). Wan shows large improvements on friction ($+0.40$), inertia ($+0.26$), and pendulum ($+0.17$), but regresses on momentum ($-0.05$). Critically, in both families at least one phenomenon degrades with scale, suggesting that scaling within a fixed architecture does not uniformly improve physical fidelity—and may actively hurt specific physical regimes that the smaller model handled adequately.

This finding has implications for the path forward: physical fidelity is unlikely to be solved by scale alone within current architectural choices.

媒体内容 · 前往原文查看

A.8 Camera Sensitivity Analysis

Accurate measurement of the relational invariants using SAM masks requires videos with a static camera, as camera motion can introduce apparent object motion and interfere with mask-based measurements. In an uncontrolled scene, deviations from the expected trajectory can arise from several confounding factors, making it difficult to attribute the measured error to physics alone. We therefore explicitly instruct the video generators to use a fixed camera. However, qualitative inspection of the generated videos shows that models do not always follow this instruction. We consequently evaluate the robustness of our evaluation protocol to camera motion. Specifically, we first compute Principia scores on our filtered real-world samples with a static camera. We then apply synthetic translational camera motion to these videos to simulate camera movement and recompute the scores. The results are presented in Tab. 6. Six out of the seven phenomena remain unaffected. The one exception - Projectile is sensitive to horizontal pan, for a reason specific to how it is scored. It is our only invariant evaluated as a horizontal range ordering rather than a continuous ratio, so a horizontal pan compresses one sphere’s apparent range relative to the other and flips the ordering outright, with no partial credit. The other scenarios remain unaffected indicating that the low principia scores for the models are attributable to bad physics and not to camera motion.

媒体内容 · 前往原文查看

Phenomenon	Original	Pan Down	Pan Left	Zoom
		2% / 5% / 10%	2% / 5% / 10%	2% / 5% / 10%
Gravity	0.934	0.934 / 0.936 / 0.937	0.934 / 0.934 / 0.934	0.934 / 0.934 / 0.935
Restitution	0.977	0.960 / 0.960 / 0.961	0.976 / 0.959 / 0.977	0.960 / 0.961 / 0.961
Pendulum	0.901	0.899 / 0.903 / 0.901	0.900 / 0.912 / 0.920	0.903 / 0.904 / 0.902
Inertia	0.936	0.935 / 0.938 / 0.901	0.940 / 0.900 / 0.898	0.936 / 0.933 / 0.900
Friction	0.963	0.964 / 0.964 / 0.964	0.964 / 0.964 / 0.964	0.964 / 0.964 / 0.964
Momentum	1.000	1.000 / 1.000 / 1.000	1.000 / 1.000 / 1.000	0.950 / 1.000 / 0.950
Spring	0.881	0.855 / 0.867 / 0.837	0.844 / 0.851 / 0.850	0.849 / 0.850 / 0.867
Projectile	1.000	1.000 / 1.000 / 1.000	0.861 / 0.722 / 0.417	1.000 / 1.000 / 0.972

Appendix B Vision-Language Model Evaluation

We evaluated four vision-language models—Qwen-4B, Qwen-32B, Gemini-3-Flash, and Gemini-3.1-Pro—on Principia-synth. Unlike video generators, which we score on whether their generated motion preserves relational invariants, VLMs are scored on whether they correctly identify physical violations in videos shown to them.

B.1 Principia-Synth Anti Physics Scenarios

We use Omniverse[35] to create synthetic videos having relational physics violations creating an anti-physics set. Representative examples for each scenario are in Fig.8.

B.2 Evaluation Protocol

For each scene, we present the VLM with the full real world physics or anti-physics video and a phenomenon-specific prompt asking the model to classify the physics as PASS or FAIL with a brief explanation. The exact prompts used are:

• Restitution: Check bouncing physics of the two objects. Both objects are identical. Limit the analysis to the first bounce only. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Gravity: Check free fall physics of the two objects, specifically the time taken to hit the ground. Both objects are identical. Limit the analysis to the initial fall only, before any bouncing occurs. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Friction: Check sliding physics of the two blocks. Both are made of the same material. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Rotational Inertia: Check rolling dynamics of the hollow and solid cylinder. Both have the same mass and radius. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Projectile: Check projectile motion trajectory of the two balls, specifically the horizontal range. Both balls are identical. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Momentum: Check collision physics of the balls with the blocks, specifically the distance travelled by the blocks after the collision. Both balls have the same mass. The blocks are solid and are made of the same material. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Mass-Spring: Check extension behaviour of the springs when mass is placed on them. Both springs have the same spring constant. Both blocks are solid and are made of the same material. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

• Pendulum: Check pendulum motion physics, specifically the time period of the swings. Return ONLY valid JSON (no markdown, no code fences) in the following format: {"verdict":"PASS"|"FAIL","reason":"brief explanation"}.

We score VLM responses by agreement with the ground-truth Principia-synth binary score: a VLM scene receives credit if its PASS/FAIL judgment matches whether the video actually satisfies the relational invariant. The agreement score $A_{\varphi }\in \left[0,1\right]$ is the fraction of scenes in phenomenon $\varphi$ where the VLM and the Principia-synth ground truth agree.

媒体内容 · 前往原文查看

B.3 VLM Scaling

Figure 9 compares within-family scaling for Qwen and Gemini. Scaling does not improve performance, with Gemini and Qwen showing overall performance reductions of $0.12$ and $0.04$, respectively. For both model families, Friction exhibits the largest regression, with decreases of $0.34$ for Gemini and $0.17$ for Qwen.

Appendix C Additional Qualitative Results of Video Generators on Principia

We provide model-wise results on our tasks in Figures10 - 14. We also recommend viewing the videos on the project webpage: https://principiabench.github.io/.