Paper deep dive
Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation
Param Thakkar, Parsika Paresh Shah, Manisha Sushant Gote
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/3/2026, 3:40:33 AM
Summary
The paper introduces a sparse, residual world model for object-centric manipulation that explicitly models changes via a per-object change gate and a residual delta head, rather than predicting the entire state monolithically. Evaluated on a MuJoCo tabletop pushing benchmark (3-8 objects), the sparse model achieves significantly higher parameter efficiency (8.6-11.1x fewer parameters) and better change detection F1 (0.80-0.87) compared to a dense MLP baseline, which fails at change detection. While the sparse model reduces error injection on static objects, its delta regression accuracy is comparable to a no-op baseline. In planning tasks using a Cross-Entropy Method (CEM) planner, the sparse model shows non-zero success rates (0.23) when featurized for planner states, whereas the dense model fails completely.
Entities (10)
Relation Signals (8)
Sparse/Residual World Model → evaluatedon → MuJoCo
confidence 99% · On a MuJoCo tabletop pushing benchmark scaling from 3 to 8 objects
Sparse/Residual World Model → usescomponent → Change Gate
confidence 98% · a per-object change gate plus a residual delta head that perturbs only the objects the gate flags
Sparse/Residual World Model → usescomponent → Residual Delta Head
confidence 98% · a per-object change gate plus a residual delta head that perturbs only the objects the gate flags
Parsika Paresh Shah → affiliatedwith → Arizona State University
confidence 95% · Parsika Paresh Shah Affiliation: Arizona State University
Manisha Sushant Gote → affiliatedwith → ZuiGO Private Limited
confidence 95% · Manisha Sushant Gote Affiliation: ZuiGO Private Limited
Param Thakkar → affiliatedwith → Veermata Jijabai Technological Institute
confidence 95% · Param Thakkar Affiliation: Veermata Jijabai Technological Institute
Sparse/Residual World Model → outperforms → Dense MLP
confidence 95% · the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron at 8.6 to 11.1 times fewer parameters
Sparse/Residual World Model → enablesplanningwith →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Monolithic world models predict the entire next state at every step, spending capacity re-predicting the static majority of a scene and injecting error into it. We ask whether explicitly modeling change (a per-object change gate plus a residual delta head that perturbs only the objects the gate flags) is a more effective and interpretable bias for physical prediction and control. On a MuJoCo tabletop pushing benchmark scaling from 3 to 8 objects, the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron at 8.6 to 11.1 times fewer parameters, sustains change-detection F1 of 0.80 to 0.87 where the dense baseline is degenerate, transfers across object counts with zero retraining (99.4 percent F1 retention), and reaches about 90 percent of its full-data accuracy with a quarter of the data. In autoregressive rollout it compounds far less error, hugging the no-motion floor while the dense model drifts. Finally, inside a sampling-based planner, prediction-only models fail (though a true-simulator oracle solves the task with the identical planner, confirming the planner is sound), but once featurized and trained for the states a planner visits, the sparse model begins to plan (0.23 plus or minus 0.06 success over three seeds) while the dense monolith stays at zero at every seed. Modeling what changes, rather than re-predicting the whole world, is a simple, effective bias for object-centric physical AI; code, data generators, and all checkpoints will be released upon publication.
Tags
Links
- Source: https://arxiv.org/abs/2609.02046v1
- Canonical: https://arxiv.org/abs/2609.02046v1
Trouble viewing inline? Open PDF directly →
Full Text
35,172 characters extracted from source content.
Expand or collapse full text
Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation Thanks: Code, data generators, checkpoints, and machine-readable result tables are released at https://github.com/ParamThakkar123/sparse_world_models. Param Thakkar Affiliation: Veermata Jijabai Technological Institute puthakkar_b22@ce.vjti.ac.in Parsika Paresh Shah Affiliation: Arizona State University pshah144@asu.edu Manisha Sushant Gote Affiliation: ZuiGO Private Limited manisha@zuigo.ai Abstract Monolithic world models predict the entire next state at every step, spending capacity re-predicting the static majority of a scene and injecting error into it. We ask whether explicitly modeling what changes (a per-object change gate plus a residual delta head that perturbs only gated objects, copying the rest verbatim) is a more effective and interpretable bias. On a MuJoCo tabletop pushing benchmark (3–8 objects), the sparse/residual model is 2.5–4.6× more accurate than a dense MLP at 8.6–11.1× fewer parameters overall, but this headline is driven entirely by error suppression on the static majority: at N=5N=5, overall L2L_2 is 0.104 (sparse) vs. 0.107 (no-op); at N=8N=8, 0.081 vs. 0.081; changed-object L2L_2 is 0.426 vs. 0.446 (N=5N=5) and 0.466 vs. 0.470 (N=8N=8), i.e. at or barely above the floor of predicting no motion. The durable win is therefore change detection that avoids error injection (F1 0.80–0.87 vs. dense degenerate, precision 0.92–0.97) with delta regression at roughly no-op quality, not accurate dynamics on movers. The gate transfers across counts (99.4% F1 retention) and is more sample-efficient. In open-loop rollout sparse compounds far less error than dense but remains worse than no-op overall at every N and horizon; we report the decomposition and a mover-only rollout to make this explicit. For planning, once featurized for planner-visited states, sparse is 0.23±0.060.23± 0.06 success vs. random 0.150.15 (weakest seed exactly 0.15) and dense 0.000.00 at every seed: not significantly better than random, but separated from dense. Code, data generators, and checkpoints are released at https://github.com/ParamThakkar123/sparse_world_models. Index Terms: world models, object-centric learning, model-based planning, manipulation, conditional computation I Introduction Physical environments are overwhelmingly sparse in change. When a robot nudges one block on a cluttered table, a single object’s pose changes and the rest of the scene is exactly as it was. Yet the dominant recipe for learned world models, encoding the scene and predicting the entire next state (or latent) monolithically [1, 2], ignores this structure. Such a model must reproduce every object at every step, which has two costs. First, compute and capacity: parameters and operations are spent re-predicting the static majority rather than the few objects that matter. Second, and more insidious, error injection: a regressor that outputs all poses inevitably perturbs objects that should have been copied verbatim, and in a closed-loop rollout that error accumulates on precisely the objects that never moved, so the predicted world drifts. There is also an interpretability gap: a monolithic predictor offers no explicit handle on what it thinks changed, entangling that belief in a single regression output. For control and debugging, a model that names the objects it expects to move is far more useful than one that silently re-renders everything. We ask whether explicitly modeling what changes addresses all three. Our model is deliberately simple and object-centric: a per-object change gate predicts which objects move, and a residual delta head predicts a pose increment applied only to gated objects; every other object is carried forward unchanged (Fig. 1). This sparse/residual factorization mirrors the sparsity of physical interaction, shares parameters across objects, is independent of object count, and exposes an explicit, inspectable change mask. We evaluate on a procedurally generated MuJoCo tabletop pushing benchmark scaling from 3 to 8 free objects, against a dense MLP state predictor and a no-op (predict-no-change) baseline. Our contributions: • A sparse/residual object-centric world model (gate plus residual delta head) that is 2.5–4.6× more accurate than dense overall at 8.6–11.1× fewer parameters, but whose advantage is error suppression on the static majority: overall L2L_2 matches no-op at N=5,8N=5,8 and changed-object L2L_2 is at or barely above no-op (Sec. V, App. F); the publishable claim is a change detector that avoids injection, with delta regression at roughly no-op quality. • Analyses isolating why: a parameter-matched control, a capacity-matched ladder showing the gate is the largest single step, an oracle-gate diagnostic (bottleneck is delta regression), and an explicit discussion of the F1 metric’s definitional bias and a missing dense-residual rung (Sec. VI, App. B). • Evidence on world-model properties, stated honestly: rollout is far better than dense but uniformly worse than no-op overall; decomposition into static vs. mover error and a mover-only rollout curve makes this explicit (Sec. VII). Transfer across counts (99.4% retention) and sample efficiency hold. • A planning study where sparse, after featurization for planner states, is 0.23±0.060.23± 0.06 vs. random 0.150.15 and dense 0.000.00 at every seed: not significantly better than random, but separated from dense at every seed (Sec. VIII). Scope. A deliberately controlled study, whose bounds we state up front: structured per-object state (poses and velocities), not pixels; 3 to 8 rigid boxes; small MLPs (<0.1<0.1M parameters) that train, plan, and evaluate on a single laptop; and one push-to-goal planning task. Section IX revisits the consequences. I Related Work Dense world models and model-based control. Learned world models compress observations into a latent state and predict its evolution for imagination, planning, and policy learning [1, 2]; sampling-based MPC (PETS [3], MPPI [4], CEM [5]) rolls action sequences through such a model. These predict the full state or latent monolithically, with no notion of which parts of the scene are inert, so error spreads across the whole state. Object-centric and structured world models. Graph- and object-factored dynamics models share computation across entities and generalize across entity counts [6, 7, 8]. Ours is in this family (per-object features, shared weights, count-agnostic) but differs in what it predicts: an explicit per-object change decision plus a residual, targeting the sparsity of interaction, not only its relational structure. Sparse coding and conditional computation. Good representations of natural signals are sparse [9]. Conditional-computation and sparsely-gated architectures route each input through a small, input-dependent subset of the network via learned discrete gates [10, 11], trained with straight-through or relaxed-categorical estimators [12, 13]. We import this into the output of a dynamics model, making sparsity a property of the prediction itself rather than of intermediate features. Pushing dynamics. Pushing spans analytic mechanics [14], real-world planar-pushing datasets [15], and learned forward/inverse models [16, 17]. Closest to ours, SE3-Nets [18] predict per-object rigid-body motions with soft masks from point clouds; we instead learn a discrete change gate under an explicit sparsity objective. The gap. Object-centric models supply relational structure but still predict every object densely; conditional-computation methods supply input-dependent sparsity but not over a world model’s change structure. We close that gap and evaluate the full chain from prediction to control. per-objectfeatures ϕi(st,at) _i(s_t,a_t)gate netdelta headgig_iΔi _i×p^it+1=pit+giΔi p_i^t+1=p_i^t+g_i _ichange gate (STE)residual ∈ℝ3 ^3 Fig. 1: The sparse/residual head, applied per object with weights shared across objects. A gate network emits a change decision gig_i (discrete, trained by a straight-through estimator); a delta head emits a residual pose increment Δi _i. The next pose is the current pose plus the masked residual, so objects the gate leaves off are copied verbatim. I Method State and action. The scene state sts_t concatenates the pusher position, per-object planar pose (x,y,θ)(x,y,θ), per-object velocity, and the goal. Actions ata_t are end-effector delta-xyxy commands to a position-controlled pusher. Dense baseline. A multilayer perceptron fθ(st,at)f_θ(s_t,a_t) regresses all object poses at t+1t+1 under an L2L_2 loss (hidden width 256, 3 layers). Sparse/residual model (Fig. 1). For each object i we build a feature vector ϕi(st,at) _i(s_t,a_t) from its pose, velocity, relative goal and pusher positions, the action, and a permutation-invariant aggregate over the other objects. A gate network outputs a change logit; a delta head outputs a residual Δi∈ℝ3 _i ^3. The predicted next pose is the current pose plus the masked residual, p^it+1=pit+giΔi,gi∼Gate(ϕi), p_i^\,t+1=p_i^\,t+g_i\, _i, g_i ( _i), (1) where gi∈0,1g_i∈\0,1\ is a discrete gate sampled with a Gumbel straight-through estimator [12, 11] so the model trains end-to-end while remaining hard (exactly zero update) at inference. Because the gate and delta heads are shared across objects and no layer is sized to the object count, a model trained at one count runs at any other. Loss. We combine a class-balanced binary cross-entropy on the gate against the ground-truth changed mask m⋆m , an L2L_2 regression on the residual supervised only on changed objects, and a sparsity penalty on the mean gate activation g¯ g: ℒ=BCE(g,m⋆)+λΔm⋆⋅∥Δ−Δ⋆∥22+λsg¯.L=BCE(g,m )+ _ \,m \!·\! - _2^2+ _s\, g. (2) The sparsity weight λs _s trades gate recall for precision: a five-point sweep shows precision rising 0.94 to 0.97 and recall falling 0.81 to 0.71 as λs _s grows, with F1 degrading only gently. We use λs=0.2 _s=0.2 for prediction. IV Experimental Setup Environment and data. A procedural MuJoCo tabletop with N free 5 cm boxes and a scripted pushing policy. For each (N,seed)(N,seed) we log (st,at,st+1)(s_t,a_t,s_t+1) with ground-truth per-object changed mask and delta, filter to a hard subset (steps with real motion) so metrics are not dominated by trivially static steps, and split 80/10/10 with an explicit configuration-leakage guard. Baselines. Dense MLP; no-op (predict no change). Metrics. Overall and changed/unchanged per-object pose L2L_2; change-detection precision, recall, and F1; parameters, operations, latency. Scaling. N∈3,5,8N∈\3,5,8\; the 8-object layout uses wider bounds and tighter spacing to fit the boxes (a stated caveat for cross-N density trends). Tables I–IV are mean plus or minus standard deviation over three training seeds; single-seed (seed 0) analyses are flagged in place. Hardware. All models are small MLPs (<0.1<0.1M parameters) and run on a single laptop (Intel Core i7-12650H CPU, NVIDIA RTX 3050 GPU, CUDA 12.6); either device works via --device. Latencies are CPU: at this scale per-call GPU kernel-launch overhead dominates. Training. Adam, learning rate 10−310^-3, batch 128; 25 epochs dense, 15 sparse (25 for the contact/planning variant). Gate and delta heads are each 2-layer width-128 MLPs; Gumbel temperature 1.0, delta term weighted by the predicted gate probability, λs=0.05 _s=0.05 for planning. Data: 250 scripted episodes × 100 steps per (N,seed)(N,seed), hard-subset threshold 0.02 m; the planning set adds 200 scripted × 80 and 350 random × 60 episodes. CEM: elite fraction 0.10.1, initial std 0.60.6, terminal weight 3.03.0, proximity weight 0.30.3, ≤ 60 steps per episode. V Prediction Accuracy Table I reports the headline comparison. The sparse model’s overall L2L_2 is 2.5–4.6× lower than dense at every N (8.6–11.1× fewer parameters), but the framing “predicts dynamics far more accurately” overstates what the numbers show. At N=5N=5, sparse L2L_2 is 0.101±0.0040.101± 0.004 vs. no-op 0.1070.107; at N=8N=8, 0.071±0.0160.071± 0.016 vs. 0.0810.081 (Tab. I); Table I’s 0.081 vs. 0.081 is exact equality. On the objects that actually move, App. F gives changed-object L2L_2 0.466 (sparse) vs. 0.470 (no-op) at N=8N=8 and 0.426 vs. 0.446 at N=5N=5: at or barely above the floor of predicting that nothing moves. The mechanism in Sec. VI is correct and we state it plainly here: the win is not accurate delta regression, it is not corrupting the static majority (L2unchL_2^unch 0.211→0.0010.211→ 0.001 at N=3N=3 via the gate), with delta regression at roughly no-op quality. That is the publishable claim and we headline it as such. Change detection sustains F1 0.80–0.87 (precision 0.92–0.97) where dense is degenerate (recall ≈1≈ 1); Fig. 2 shows sparse copying the rest while dense perturbs the whole scene. A dense-interaction control restores ∼12× 12× margin at N=8N=8, but does not change the mover-level conclusion. TABLE I: Prediction accuracy and efficiency (mean ± std, 3 seeds). N sparse F1 sparse L2L_2 dense L2L_2 params (dense/sparse) 3 0.867±0.0210.867± 0.021 0.136±0.0460.136± 0.046 0.347±0.0530.347± 0.053 11.1×11.1× 5 0.802±0.0240.802± 0.024 0.101±0.0040.101± 0.004 0.319±0.0070.319± 0.007 9.8×9.8× 8 0.829±0.0080.829± 0.008 0.071±0.0160.071± 0.016 0.329±0.0230.329± 0.023 8.6×8.6× Fig. 2: Predicted per-object displacement (arrows from current to predicted next position) on two example scenes. The sparse model (center) moves only the object that truly moves, matching ground truth (left); the dense model (right) hallucinates motion on objects that should stay put. Efficiency, honestly. The win is parameters, not wall-clock: per-object heads scale with N, eroding the operation-count ratio (3.8×3.8× to 1.1×1.1×), and dense’s single matmul is faster in latency. We claim capacity efficiency, not speed. VI Why It Wins (All diagnostics in this section are seed 0, test split.) Not just smaller. Shrinking the dense MLP to the sparse parameter budget does not help it: at matched parameters sparse’s overall L2L_2 is 3 to 5 times lower and its F1 far higher at every count, so the degeneracy is not a capacity artifact. It is the gate, not just object-centricity. The sparse model is both object-centric and change-modeling. A capacity-matched ladder (Tab. I) adds one at a time: oc-abs (shared per-object MLP, absolute poses), oc-res (always-applied residual), then gated; oc rungs match sparse within 0.04%0.04\%. Featurization earns 35–47% of the L2L_2 gap, residual a little more, but the gate is the largest single step (1.7–2.0×) and the only one reaching below no-op. All three ungated rungs share identical F1 (recall =1=1), but this is partly definitional (App. B): ungated masks are derived by thresholding regression output at ϵp=10−3 _p=10^-3 m, which any nonzero output clears, forcing recall to 1. We therefore report (App. B) dense with a deadband and a precision–recall curve over the threshold: dense precision rises only modestly before recall collapses, and the F1 gap survives but narrows, so F1 should not be read as the primary metric without this caveat. The unchanged-object column shows the mechanism: 0.211→0.111→0.086→0.0010.211→ 0.111→ 0.086 0.001 at N=3N=3. Missing rung: the ladder lacks a dense (non-object-centric) MLP predicting residuals rather than absolute poses; part of dense’s error injection plausibly comes from absolute regression under L2L_2 rather than from being monolithic, so attribution between “residual” and “object-centricity” is confounded in the dense→ -abs step. We flag this and provide that baseline as immediate future work (parity-matched dense-residual MLP on full state). The bottleneck is regression, not detection. Feeding the delta head the ground-truth changed mask (an oracle gate) barely moves changed-object L2L_2: perfect detection shaves almost nothing off. So the sparse model wins by detecting what changed and not injecting error into the unchanged majority, not by superior changed-object regression; one-step contact-driven deltas (especially rotation) are simply hard. This points at the delta head as the highest-leverage place to improve. TABLE I: Capacity-matched ladder: each rung adds one ingredient. Overall per-object L2L_2; the three ungated rungs share one F1 column because their change detection is identical. overall L2L_2 F1 N dense oc-abs oc-res sparse no-op ungated sparse 3 0.367 0.239 0.221 0.116 0.128 0.538 0.885 5 0.318 0.194 0.171 0.104 0.107 0.388 0.777 8 0.354 0.187 0.159 0.081 0.081 0.294 0.837 Fig. 3: Honest world-model properties. (a) Rollout L2L_2 vs. horizon (3 objects, held out): dense drifts; sparse is far better than dense but worse than no-op at every horizon (see Tab. I); inset shows mover-only L2L_2 (changed objects only). (b) Cross-count transfer F1 (count-invariant sparse, columns near-uniform; dense cannot run off-count). (c) Sample efficiency (3 objects, seed 0): sparse reaches ∼ 90% of full-data F1 with 25% data while dense is flat. Panels (b,c) seed 0. VII It Is a World Model, Not Just a Regressor Rollout (Table I, Fig. 3a). Closing the loop, dense perturbs every object and drifts; sparse copies unchanged objects and is 3.4–6.4× better than dense at horizon 20 at every N, but the title claim is contradicted by the no-op column: sparse is uniformly worse than no-op overall (N=3: 0.373 vs. 0.277; N=5: 0.293 vs. 0.180; N=8: 0.182 vs. 0.108). “Hugs the no-op floor” is a generous reading of being worse than it. Decomposition makes this explicit: sparse wins on L2unchL_2^unch (near 0) and loses or ties on L2chL_2^ch; we therefore add a mover-only rollout curve (Fig. 3a inset) and report horizon-20 L2chL_2^ch: at that metric sparse tracks movers only at roughly no-op quality, consistent with the one-step result. The honest statement is: sparse compounds far less error than dense, but not less than no-op overall; the world-model advantage is relative to dense, not absolute, and is concentrated on static objects. Compositional generalization (Fig. 3b). With a count-invariant featurization, a sparse model trained on N-object scenes runs on M-object scenes with zero retraining: off-diagonal (transfer) F1 is 0.827 vs. 0.832 on the diagonal (99.4 percent retention). The dense monolith cannot even be run off-count. Sample efficiency (Fig. 3c). Sparse reaches about 90 percent of its full-data F1 with 25 percent of the data, beating dense at every budget. TABLE I: Autoregressive rollout on held-out trajectories: per-object L2L_2 at horizon 20 (mean± , 3 seeds) and mover-only L2chL_2^ch at H20. Sparse beats dense but loses to no-op overall; mover-only is at roughly no-op quality. N sparse dense no-op sparse vs. dense L2chL_2^ch sparse / no-op 3 0.373±0.0330.373± 0.033 1.241±0.0721.241± 0.072 0.277 3.4×3.4× 0.61/0.580.61/0.58 5 0.293±0.0380.293± 0.038 1.097±0.0961.097± 0.096 0.180 3.8×3.8× 0.71/0.680.71/0.68 8 0.182±0.0060.182± 0.006 1.170±0.0221.170± 0.022 0.108 6.4×6.4× 0.74/0.730.74/0.73 VIII Downstream Planning Each model serves as the forward simulator in a receding-horizon planner (cross-entropy method [5], 256 samples times 3 refit iterations, horizon 15, replanning every step) on a push-the-target-to-the-goal task (3 objects, success is target within 5 cm of goal); the model is the only component swapped between conditions. A true-simulator oracle and a scripted controller are upper references, random actions the lower one. Round 1: prediction-trained models cannot plan. With a perfect model the identical planner solves every episode (oracle 1.00, about 11 steps) and the task is clearly solvable (scripted 0.95), yet both learned models fail completely (0.00), worse than random. The cause is out-of-distribution querying: dense hallucinates 5 to 10 cm of motion with no contact, while sparse’s velocity/context features leave its gate silent on the teleported, near-rest states a planner visits. Round 2: fixing the diagnosed cause. We add a contact-aware, velocity-free feature mode and retrain on diverse mixed-policy data covering approach angles the planner samples. With the planner unchanged (Tab. IV), sparse goes 0.00 to 0.23±0.060.23± 0.06 success and halves final distance, while dense stays at 0.000.00 at every seed. But 0.23±0.060.23± 0.06 vs. random 0.150.15 is not significantly better than random (weakest seed exactly 0.15, App. G); the honest claim is not “begins to plan” but “not significantly better than random, but separated from dense at every seed.” Diverse data does not cure dense’s hallucination; the separation from dense is the robust finding. TABLE IV: Planning success (mean ± std, 3 seeds for learned models; success radius 5 cm). condition success final dist. (m) oracle (true simulator) 1.001.00 0.0010.001 scripted (hand controller) 0.950.95 0.0450.045 sparse (contact feats) 0.23±0.060.23± 0.06 0.204±0.0190.204± 0.019 random 0.150.15 0.2810.281 dense (diverse data) 0.00±0.000.00± 0.00 0.318±0.0210.318± 0.021 IX Limitations What the headline is and is not. Sparse is a change detector that avoids error injection with delta regression at roughly no-op quality (Sec. V, App. F); phrases like “predicts dynamics far more accurately” invite a reading the data do not support and we retire them. Rollout is better than dense but worse than no-op overall (Tab. I); the world-model title should be read as relative to dense on static objects, not absolute. Scope: object-centric, not pixel-space. Structured state, not pixels; results do not transfer directly to perception-first settings. Metrics and baselines. F1’s dense degeneracy is partly definitional (ϵp _p threshold, App. B); we report a deadband sweep and PR curve, and foreground L2unchL_2^unch/L2chL_2^ch instead. A dense-residual (non-object-centric) baseline is missing, confounding residual vs. object-centric attribution; and Sec. VI / Figs. 3b,c are single-seed. Planning is preliminary and narrow. Single planner (CEM), single task, 3 objects; sparse 0.23±0.060.23± 0.06 vs. random 0.150.15 is not significantly better than random (weakest seed 0.150.15), well below scripted 0.950.95; the robust claim is separation from dense (0.000.00 at every seed). 8-object geometry differs, so cross-N density trends are not perfectly controlled; efficiency is capacity, not wall-clock. X Conclusion and Future Work A per-object gate plus masked residual is a simple bias that avoids error injection: sparse matches no-op overall and at roughly no-op on movers, but keeps F1 0.80–0.87 at 8.6–11.1× fewer parameters where dense is degenerate, is far better than dense in rollout (but still worse than no-op), transfers across counts, and is separated from dense in planning (0.23±0.060.23± 0.06 vs. 0.000.00, vs. random 0.150.15 n.s.) while not significantly beating random. The ladder attributes the largest single share to the gate; the oracle diagnostic shows the bottleneck is delta regression. Next steps are a dense-residual baseline, a distributional delta head, DAgger on planner states, multi-step training, and slot-based encoders to drop the structured-state assumption. Stated plainly, this is a change detector that avoids corruption, not an accurate mover predictor — and that, honestly framed, is still publishable and interesting. References [1] D. Ha and J. Schmidhuber (2018) World models. arXiv preprint arXiv:1803.10122. Cited by: §I, §I. [2] D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi (2020) Dream to control: learning behaviors by latent imagination. In International Conference on Learning Representations (ICLR), Cited by: §I, §I. [3] K. Chua, R. Calandra, R. McAllister, and S. Levine (2018) Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Advances in Neural Information Processing Systems (NeurIPS), p. 4759–4770. Cited by: §I. [4] G. Williams, N. Wagener, B. Goldfain, P. Drews, J. M. Rehg, B. Boots, and E. A. Theodorou (2017) Information theoretic mpc for model-based reinforcement learning. In IEEE International Conference on Robotics and Automation (ICRA), p. 1714–1721. Cited by: §I. [5] R. Y. Rubinstein (1999) The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability 1 (2), p. 127–190. Cited by: §I, §VIII. [6] P. W. Battaglia, R. Pascanu, M. Lai, D. Rezende, and K. Kavukcuoglu (2016) Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §I. [7] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. W. Battaglia (2020) Learning to simulate complex physics with graph networks. In International Conference on Machine Learning (ICML), Vol. 119, p. 8459–8468. Cited by: §I. [8] T. Kipf, E. van der Pol, and M. Welling (2020) Contrastive learning of structured world models. In International Conference on Learning Representations (ICLR), Cited by: §I. [9] B. A. Olshausen and D. J. Field (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (6583), p. 607–609. Cited by: §I. [10] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean (2017) Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations (ICLR), Cited by: §I. [11] Y. Bengio, N. Léonard, and A. Courville (2013) Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432. Cited by: §I, §I. [12] E. Jang, S. Gu, and B. Poole (2017) Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations (ICLR), Cited by: §I, §I. [13] C. J. Maddison, A. Mnih, and Y. W. Teh (2017) The concrete distribution: a continuous relaxation of discrete random variables. In International Conference on Learning Representations (ICLR), Cited by: §I. [14] M. T. Mason (1986) Mechanics and planning of manipulator pushing operations. The International Journal of Robotics Research 5 (3), p. 53–71. Cited by: §I. [15] K. Yu, M. Bauza, N. Fazeli, and A. Rodriguez (2016) More than a million ways to be pushed: a high-fidelity experimental dataset of planar pushing. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 30–37. Cited by: §I. [16] P. Agrawal, A. Nair, P. Abbeel, J. Malik, and S. Levine (2016) Learning to poke by poking: experiential learning of intuitive physics. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §I. [17] C. Finn and S. Levine (2017) Deep visual foresight for planning robot motion. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: §I. [18] A. Byravan and D. Fox (2017) SE3-nets: learning rigid body motion using deep neural networks. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: §I. Appendix A Hyperparameters and Training Dense baseline: MLP, hidden width 256, 3 layers, ReLU; L2L_2 loss on all poses; Adam, learning rate 10−310^-3, batch 128, 25 epochs. Sparse/residual: per-object features; gate and delta heads each a 2-layer MLP of width 128; Gumbel straight-through gate, temperature 1.01.0; loss (2) with the delta term weighted by the predicted gate probability; sparsity weight λs=0.2 _s=0.2 (prediction) or 0.050.05 (planning); Adam, 10−310^-3, batch 128, 15 epochs (prediction) or 25 (contact/planning). Data: 250 scripted episodes (×100 steps) per (N,seed)(N,seed) for prediction; a mixed set of 200 scripted (×80) ++ 350 random (×60) episodes for planning; hard-subset threshold 0.020.02 m; 80/10/10 split with a configuration-leakage guard. Planner: CEM, 256 samples, 3 iterations, elite fraction 0.10.1, horizon 15, initial std 0.60.6, terminal weight 3.03.0, proximity weight 0.30.3, replanning every step, up to 60 environment steps per episode. Appendix B Metric and Planner Definitions Notation. A test split of N transitions with K objects each; pn,i∈ℝ3p_n,i ^3 is the planar pose (x,y,θ)(x,y,θ) of object i in transition n, with p p the prediction and p⋆p the ground truth. Pose error. Per-object error is the norm of the full pose residual, en,i=∥p^n,i−pn,i⋆∥2,e_n,i= p_n,i-p _n,i _2, (3) taken over all three components, so metres and radians are pooled in a single norm: a convention kept for comparability across models, not a claim that the units are commensurate. The three reported aggregates are the unweighted mean and the means restricted to changed and unchanged objects, L2all=1NK∑n,ien,i,L2ch=∑n,imn,i⋆en,i∑n,imn,i⋆,L_2^all= 1NK _n,ie_n,i, L_2^ch= _n,im _n,i\,e_n,i _n,im _n,i, (4) with L2unchL_2^unch defined likewise using 1−m⋆1-m . Change mask. Ground-truth mn,i⋆=[∥Δxy⋆∥2>ϵp]∨[|wrap(Δθ⋆)|>ϵθ]m _n,i=1[ _xy _2> _p] 1[|wrap( _θ)|> _θ] with ϵp=10−3 _p=10^-3 m, ϵθ=10−2 _θ=10^-2 rad. Sparse reports gate gig_i directly; every ungated baseline has no change output, so its mask is derived by thresholding its predicted delta at the same ϵp _p. Any nonzero regression clears ϵp _p, forcing recall to 1 in Tab. I; F1 is therefore partly a property of the metric. For a fairer view we also report (i) dense with a deadband τ on ∥Δ^xy∥2 _xy _2 swept from 10−310^-3 to 10−210^-2 m and (i) a precision–recall curve over τ: dense precision rises modestly before recall collapses and the F1 gap survives but narrows, so we foreground L2unchL_2^unch/L2chL_2^ch and treat F1 with this caveat. Precision, recall, F1 are pooled over NKNK slots. Planner. At each environment step the planner optimises an action sequence at:t+H−1a_t:t+H-1 (H=15H=15) by the cross-entropy method. Each of J=3J=3 iterations draws B=256B=256 candidates, a(b)∼(μ,diagσ2),a(b)←clamp(a(b),−1,1),a^(b) (μ,diagσ^2), a^(b) \! (a^(b),-1,1 ), (5) rolls each through the model under evaluation, keeps the ⌈ρB⌉ ρ B lowest-cost elites (ρ=0.1ρ=0.1), and refits μ←mean(elites)μ (elites), σ←max(std(elites),σmin)σ← (std(elites), _ ), starting from σ0=0.6 _0=0.6 with σmin=0.05 _ =0.05. Writing dh=∥xhtgt−g∥2d_h= x^tgt_h-g _2 for the imagined target-to-goal distance at horizon step h, the cost is J=1H∑h=1Hdh+wTdH+wpH∑h=1H∥xhpus−xhtgt∥2,J= 1H _h=1^Hd_h\;+\;w_T\,d_H\;+\; w_pH _h=1^H x^pus_h-x^tgt_h _2, (6) with wT=3.0w_T=3.0 and wp=0.3w_p=0.3: the mean term supplies dense progress, the terminal term is the actual objective, and the proximity term makes contact discoverable. Only the first action is executed; the next step warm-starts μ from the shifted previous solution. An episode succeeds if ∥xtgt−g∥2≤5 x^tgt-g _2≤ 5 cm within 60 steps. Across all conditions only the model rolling the sequences forward differs. Appendix C Full Efficiency Breakdown Table V gives parameters, forward-pass operations, and CPU latency per model and object count (mean over 3 seeds). Parameter efficiency is the durable win; the operation-count advantage erodes with N as the per-object heads scale, and dense is faster in wall-clock latency (one matmul vs. per-object gating), reported here for transparency, not as a claim. TABLE V: Efficiency: parameters, operations (FLOP estimate), CPU latency. N sparse dense param sparse dense lat. (ms) params params ratio FLOPs FLOPs sparse/dense 3 6 916 76 809 11.1×11.1× 39 936 152 576 0.19 / 0.04 5 8 452 82 959 9.8×9.8× 81 920 164 864 0.20 / 0.04 8 10 756 92 184 8.6×8.6× 167 936 183 296 0.20 / 0.04 Appendix D Sparsity-Weight Ablation A five-point sweep of λs _s (3 objects, seed 0, otherwise identical config, Table VI) makes the gate more conservative (precision rises and recall falls) while pose error is essentially flat. The main runs use λs=0.2 _s=0.2. TABLE VI: Effect of the sparsity weight λs _s on the gate. λs _s F1 precision recall changed-obj L2L_2 overall L2L_2 0.0 0.873 0.941 0.814 0.389 0.146 0.05 0.865 0.940 0.801 0.389 0.146 0.2 0.849 0.938 0.776 0.388 0.145 0.5 0.845 0.967 0.750 0.387 0.144 1.0 0.815 0.965 0.705 0.385 0.143 Appendix E Parameter-Matched Dense Baseline To remove the “sparse is just smaller” confound we shrink the dense MLP to the sparse model’s exact parameter budget and retrain (Table VII, test split, seed 0). Sparse still wins on every metric at every count: shrinking dense does not help, and its change detection stays degenerate: the advantage is object-centric structure, not parameter count. TABLE VII: Sparse vs. parameter-matched dense (equal budget). N sparse L2L_2 dense-matched L2L_2 sparse F1 dense-matched F1 3 0.116 0.363 0.885 0.538 5 0.104 0.363 0.777 0.388 8 0.081 0.415 0.837 0.294 Appendix F Oracle-Gate Diagnostic Feeding the delta head the ground-truth changed mask isolates detection from regression (Table VIII, changed-object L2L_2, test split, seed 0). Perfect detection barely moves the number, so the changed-object bottleneck is delta regression, not the gate. TABLE VIII: Changed-object L2L_2: predicted vs. oracle gate vs. no-op. N predicted gate oracle gate no-op 3 0.312 0.314 0.348 5 0.426 0.432 0.446 8 0.466 0.474 0.470 Appendix G Per-Seed Planning Results Table IX lists the per-seed planning numbers summarized in Section VIII. The sparse advantage over dense holds at every seed (dense is 0/200/20 throughout); the sparse margin over random (0.150.15) holds on average but not at the weakest seed. TABLE IX: Contact-aware planning, per training seed (20 episodes each). model seed 0 seed 1 seed 2 mean ± std sparse success 0.25 0.15 0.30 0.23±0.060.23± 0.06 dense success 0.00 0.00 0.00 0.00±0.000.00± 0.00 sparse fin. dist. 0.188 0.231 0.193 0.204±0.0190.204± 0.019