Paper deep dive
Resolving State-Representation Mismatch: State-Space Visual Reasoning for Open-Loop VLA Planning
Junhao Xiao, Haoxiang Zhao, Menghao Fang, Jinkui Zhang, Jinghan Yu, Xinyu Huang, Zhiyu Wu, Kaiming Xu, Yi Chen, Youjun Bao, Zhiyuan Ma
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Despite rapid progress in vision-language-action (VLA) models, existing reasoning paradigms still face a fundamental \emph{state-representation mismatch} in open-loop planning. Given only an initial observation, models must internally simulate action-conditioned state transitions, whereas text-, pixel-, and latent-space reasoning can suffer from lossy spatial compression, error-accumulating visual generation, and bypass of intermediate latent tokens, respectively, undermining reliable long-horizon planning. We propose \textbf{State-Space Visual Reasoning} (SSVR), which decouples static visual context, language constraints, and a recurrent latent state. SSVR encodes the initial image and instruction once, then conditions each action prediction on the latent state and updates it with an action-conditioned GRU. Using Qwen2.5-VL as the backbone, SSVR achieves 99.5/99.6, 96.3/98.0, and 83.9/90.6 EM/PR on FrozenLake, Maze, and MiniBehavior, substantially outperforming prior methods. Extensive experiments support the effectiveness of recurrent state modeling for VLA open-loop planning across input transformations and transfer settings. By reusing static visual-textual context and updating a compact recurrent state, SSVR supports efficient multi-step inference, achieving up to $98.58\times$ faster Maze decoding rollouts than the evaluated baselines with the prefix cache prebuilt.
Tags
Links
- Source: https://arxiv.org/abs/2609.33412v1
- Canonical: https://arxiv.org/abs/2609.33412v1
Trouble viewing inline? Open PDF directly →
Full Text
56,133 characters extracted from source content.
Expand or collapse full text
Resolving State‑Representation Mismatch: State‑Space Visual Reasoning for Open‑Loop VLA Planning Junhao Xiao, Haoxiang Zhao, Menghao Fang, Jinkui Zhang, Jinghan Yu Email: mailto:jhxiao26@m.fudan.edu.cnjhxiao26@m.fudan.edu.cnmailto:mzyth@hust.edu.cnmzyth@hust.edu.cn Xinyu Huang, Zhiyu Wu, Kaiming Xu, Yi Chen, Youjun Bao, Zhiyuan Ma Affiliation: FDU HUST TJU CCNU Kuaishou Abstract Despite rapid progress in vision-language-action (VLA) models, existing reasoning paradigms still face a fundamental state-representation mismatch in open-loop planning. Given only an initial observation, models must internally simulate action-conditioned state transitions, whereas text-, pixel-, and latent-space reasoning can suffer from lossy spatial compression, error-accumulating visual generation, and bypass of intermediate latent tokens, respectively, undermining reliable long-horizon planning. We propose State-Space Visual Reasoning (SSVR), which decouples static visual context, language constraints, and a recurrent latent state. SSVR encodes the initial image and instruction once, then conditions each action prediction on the latent state and updates it with an action-conditioned GRU. Using Qwen2.5-VL as the backbone, SSVR achieves 99.5/99.6, 96.3/98.0, and 83.9/90.6 EM/PR on FrozenLake, Maze, and MiniBehavior, substantially outperforming prior methods. Extensive experiments support the effectiveness of recurrent state modeling for VLA open-loop planning across input transformations and transfer settings. By reusing static visual-textual context and updating a compact recurrent state, SSVR supports efficient multi-step inference, achieving up to 98.58×98.58× faster Maze decoding rollouts than the evaluated baselines with the prefix cache prebuilt. Code is publicly available at project page. 11footnotetext: Corresponding author: Zhiyuan Ma. Figure 1: Motivation and evidence for aligning reasoning with dynamic state. Existing paradigms represent evolving state through compressed language, repeatedly generated images, or intermediate latent tokens. SSVR follows the finger-tracing intuition: a static scene is reused, a recurrent latent state is updated by actions through a GRU, and visual attention varies across planning steps. 1 Introduction Vision-language-action (VLA) systems in robotics (Hu et al., 2023), GUI interaction (Agashe et al., 2025) and navigation (Shah et al., 2023) often need to plan before execution because feedback may be delayed or costly and mistakes may be irreversible. In this open-loop setting, the model observes the initial scene and instruction once, internally models how actions change the execution state, and outputs a complete action sequence. Planning reliability therefore depends on maintaining an informative internal state throughout the rollout. Existing reasoning paradigms exhibit a fundamental state-representation mismatch with this requirement. They conflate two components with different temporal roles: high-dimensional scene context that remains largely static and execution state that evolves with actions. Text-space reasoning compresses map topology and dynamic state into language, making critical spatial relations easy to omit or blur; pixel-space reasoning repeatedly generates complete visual states, so lossy frame errors propagate into later decisions; and latent-space reasoning may bypass intermediate visual tokens. In open-loop rollout, textual omissions and pixel-generation errors can compound, while latent bypass can weaken the use of intermediate visual representations. This asymmetry between static context and dynamic state is common in VLA open-loop planning (Hao et al., 2026; Wu et al., 2024a; Ivanitskiy et al., 2023; Jin et al., 2023). In the Maze example in Figure 1, the walls and goal remain fixed while only the agent position changes: prior paradigms repeatedly re-express the evolving state in text, images, or hidden vectors, whereas a human can trace the route by holding the map fixed and moving only a finger. Figure 2 shows the failures: topology hallucination in text-space reasoning, object duplication or disappearance in pixel-space rollout, and bypass of intermediate visual tokens in latent-space reasoning. Together, these observations motivate a reasoning paradigm better matched to open-loop state evolution. We propose State-Space Visual Reasoning (SSVR), which addresses this mismatch by decoupling static visual context, language constraints, and a recurrent latent state while retaining the pretrained MLLM backbone (Bai et al., 2025). The initial image and instruction are encoded once into a reusable KV cache (Jiang et al., 2025; Jiang et al., 2026b); at each step, the latent state token et=zte_t=z_t conditions next-action prediction, and the predicted action updates ztz_t through a learned GRU transition, recursively advancing the latent state-action trajectory. Experiments show three advantages of SSVR: (1) strong planning performance, with 99.5/99.6, 96.3/98.0, and 83.9/90.6 EM/PR on FrozenLake, Maze, and MiniBehavior, respectively, surpassing the strongest pixel baseline; (2) robustness and generalization, with resilience to prompt and image transformations, stronger FrozenLake-to-Maze transfer than VPRL (Xu et al., 2026), and a 22.9-point gain in native Maze VQA optimal-action hit rate without recurrent state conditioning (Table 6), while VQAv211 1 https://huggingface.co/datasets/HuggingFaceM4/VQAv2 (Goyal et al., 2017) accuracy drops by only 3.9 points from native Qwen; and (3) efficient reasoning, with decoding-rollout speedups of 5.81×5.81×, 38.24×38.24×, and 98.58×98.58× over LVR, Monet, and VPRL, respectively, using a prebuilt prefix cache. 2 Related Work Text-space visual reasoning. MLLMs often map image features into language-model space and use text tokens as intermediate reasoning carriers (Liu et al., 2023; Alayrac et al., 2022; Liu et al., 2025c; Venhoff et al., 2025). CoT-style methods externalize spatial inference as symbols (Wu et al., 2024b; Jiang et al., 2026a) or enrich text reasoning with visual abstraction, perception tokens, region distillation, and tool use (Liu et al., 2025b; Bigverdi et al., 2024; Wei et al., 2026; Zheng et al., 2026; Hong et al., 2025). This suits knowledge QA and compactly verbalizable vision tasks, but verbalizing map-like inputs is lossy: text must maintain coordinates, adjacency, reachability, and action history, making critical topology easy to omit or blur as the rollout evolves. SSVR keeps language for goals and constraints, while an action-conditioned GRU maintains the recurrent planning state. Pixel-space visual reasoning. Pixel-space methods construct explicit visual intermediates through drawing, annotation, programmatic editing, generated visual thoughts (Li et al., 2025b; Chern et al., 2025), or future image/state rollouts for VLA and spatial planning (Zhao et al., 2025; Wang et al., 2026; Xu et al., 2026). They preserve spatial structure and improve inspectability, but open-loop tasks often change only low-dimensional variables, such as position, carrying state, or phase, while the scene stays mostly fixed. Full image rollout therefore regenerates mostly static content; more importantly, each generated frame is a lossy state estimate whose rendering errors can propagate and compound when reused by later steps. SSVR preserves visual grounding and decouples static context from recurrent state. Latent-space visual reasoning. Latent-space methods replace pixels with continuous tokens, reducing generation cost and enabling lightweight reasoning in a shared vision-language space (Wang et al., 2025; Li et al., 2025a; Liu et al., 2025a; Yang et al., 2025; Chen et al., 2026; Tang et al., 2026; Guo et al., 2026). Prior analyses report image-to-latent and latent-to-answer disconnections (Li et al., 2026; Yu et al., 2026), alongside representation homogenization, manifold separation, and input-output embedding mismatch under recursive use (Cui et al., 2026; Miao et al., 2026). SSVR uses a continuous recurrent state initialized from visual features, updated by an action-conditioned GRU, and injected as the query for action prediction. 3 Vision-Language-Action Open-Loop Planning We study VLA open-loop planning. Given an initial image I0I_0 and instruction x, the model predicts an action sequence =(a0,…,aH−1)a=(a_0,…,a_H-1). The evaluation environment is defined as =(I0,x,,s0,T,g,H),st+1=T(st,at),at∈, array[]lT=(I_0,x,A,s_0,T,g,H),\\ s_t+1=T(s_t,a_t), a_t , array (1) where s0s_0 is the initial environment state, A is the action set, T is the environment transition, g is the goal predicate, and H is the planning horizon. The evaluator uses T and g to execute and score the plan; success requires g(st)=1g(s_t)=1 for some t≤Ht≤ H. In the default evaluation, H is set to the ground-truth shortest-path length, and execution stops at the first terminal event. Here, I0I_0 provides persistent scene context, x specifies task objectives and constraints, and sts_t denotes the changing environment state. The model maintains a learned latent state ztz_t from the initial visual features and action history. A suitable reasoning paradigm should reuse persistent context while updating its internal state according to predicted actions. Figure 2: Representative failure modes of existing reasoning paradigms. Text-space reasoning can hallucinate Maze topology after compressing the image into language; repeatedly generated pixel-space states may duplicate or erase objects and propagate these errors; latent-space reasoning can bypass both image and intermediate latent tokens when producing the final answer. Figure 2 summarizes the resulting failures across existing paradigms, motivating a better reasoning paradigm. 4 State-Space Visual Reasoning Figure 3 summarizes SSVR, which reuses static visual-textual context while recursively updating a GRU latent state. Figure 3: Overview of SSVR. The initial image and instruction are encoded once and kept as cached visual-textual context. Visual features initialize the latent state z0z_0. At step t, the model appends the state token et=zte_t=z_t, predicts action ata_t, and updates the latent state through an GRU. 4.1 Static Context and Dynamic State Let θ=(θ0,Δθ)θ=( _0, θ) collect the frozen pretrained parameters and trainable LoRA adapters, and let η denote the trainable latent-state parameters. The decoder is FθF_θ, and the visual frontend produces projected feature tokens V0=ℰθvis(I0)∈ℝNv×dV_0=E^vis_θ(I_0) ^N_v× d. We pack these features with the instruction and template tokens into the input embedding sequence C and compute the static prefix cache (C,ℳ)=Packθ0(V0,x),=Fθprefill(C,ℳ).(C,M)=Pack_ _0(V_0,x), =F_θ^prefill(C;M). (2) The metadata ℳM retains the attention mask and position information for decoding. The GRU latent state zt∈ℝdz_t ^d represents the evolving planning state. Its initialization uses the mean visual feature: c0=1Nv∑p=1NvV0,p,z0=fηinit(V0)=LNη(qη+Wηc0),et=zt,c_0= 1N_v _p=1^N_vV_0,p, z_0=f^init_η(V_0)=LN_η(q_η+W_ηc_0), e_t=z_t, (3) where qηq_η, WηW_η, and the LayerNorm parameters are learned jointly with the GRU transition. At step t, SSVR appends ete_t as an independent sequence position after the cached context and computes the decoder readout ut=Fθdecode(,et,ℳ).u_t=F_θ^decode(K,e_t;M). (4) Each decision reuses the same prefix cache and appended-token position. The current state token’s temporary KV entries are discarded after decoding, implementing the sequence C∥etC \|e_t. 4.2 State-Conditioned Action Prediction SSVR predicts one action at a time from the current latent state and cached context. Each a∈a is assigned a distinct single-token verbalizer, such as UP, DOWN, LEFT, or RIGHT. If v(a)v(a) is its token ID, the action logit is ot(a)=LMHeadθ0(ut)v(a),a∈,o_t(a)=LMHead_ _0(u_t)_v(a), a , (5) and the resulting policy is πθ,η(a∣C,zt)=exp(ot(a))∑a′∈exp(ot(a′)). _θ,η(a C,z_t)= (o_t(a)) _a (o_t(a )). (6) This reuses the pretrained language-model head, with probabilities normalized over the task action set. 4.3 Recursive State-Space Inference With greedy decoding, SSVR alternates between state-conditioned action prediction and action-conditioned latent transition: at a_t =argmaxa∈πθ,η(a∣C,zt), = _a _θ,η(a C,z_t), (7) zt+1 z_t+1 =ftransη(zt,at)=LNη(GRUCellη(Eaη(at),zt)),et+1=zt+1. =f^trans_η(z_t,a_t)=LN_η\! (GRUCell_η(E^a_η(a_t),z_t) ), e_t+1=z_t+1. The learned action embedding Eηa(at)E^a_η(a_t) and previous latent state drive the GRU update. Initialization and transition share the LayerNorm module. This recursion produces an H-step plan using the fixed visual-textual context and predicted action history. The evaluator handles environment execution and terminal-event checks. 4.4 Soft-Target Training For each training instance, environment transitions and path-quality rules identify safe actions safe(st)A_safe(s_t) and optimal actions opt(st)A_opt(s_t) offline at the reference decision state sts_t. A reference action history is constructed by BFS from the episode start, and replayed through the GRU to obtain the supervised latent state ztz_t. For opt(st)⊆safe(st)A_opt(s_t) _safe(s_t) and a nonempty optimal set, the soft target is qα(a∣st)=(1−α)Unif(safe(st))(a)+αUnif(opt(st))(a).q_α(a s_t)=(1-α)\,Unif(A_safe(s_t))(a)+α\,Unif(A_opt(s_t))(a). (8) An empty optimal set uses the uniform safe-action target; training retains items with nonempty safe sets. SSVR minimizes ℒSFT=−∑a∈qα(a∣st)logπθ,η(a∣C,zt).L_SFT=- _a q_α(a s_t) _θ,η(a C,z_t). (9) Gradients propagate through the decoder, differentiable prefix cache, and reference-history GRU updates. Training jointly learns the LoRA adapters, latent initializer, action embeddings, and GRU transition while keeping pretrained weights frozen. The coefficient α balances safe-action diversity and optimal-action preference; the main model uses α=0.7α=0.7. Implementation details and pseudocode are provided in Appendix. 5 Experiments We evaluate open-loop rollout planning on FrozenLake (Wu et al., 2024a), Maze (Ivanitskiy et al., 2023), and MiniBehavior (Jin et al., 2023), covering risk avoidance, topological path finding, and phased object interaction, respectively. The model uses the initial observation and instruction, and updates its latent state with its own predicted actions. Metrics are exact match (EM) and progress rate (PR) (Xu et al., 2026), evaluated against shortest optimal action sequences; full implementation, training, and evaluation settings are in the appendix. 5.1 Main Results Table 1: Comparison across reasoning paradigms. Entries report EM/PR (%). VQA Support: native VQA through text-based responses; ∗: official release without training on our datasets. Model VQA Support FrozenLake Maze MiniBehavior Avg. Text Space Gemini 2.0 Flash Direct∗ ✔ 21.2 / 47.6 8.3 / 31.4 0.7 / 29.8 10.1 / 36.3 Gemini 2.0 Flash CoT∗ ✔ 27.6 / 52.5 6.9 / 29.8 4.0 / 31.2 12.8 / 37.8 Gemini 2.5 Pro∗ ✔ 72.0 / 85.0 21.5 / 35.5 37.6 / 59.9 43.7 / 60.1 Qwen2.5-VL-7B Direct∗ ✔ 1.2 / 15.0 0.6 / 14.5 0.3 / 9.8 0.7 / 13.1 Qwen2.5-VL-7B CoT∗ ✔ 8.2 / 29.1 2.3 / 15.2 0.5 / 14.7 3.7 / 19.7 Qwen2.5-VL-7B SFT ✔ 68.6 / 84.4 60.9 / 70.3 31.3 / 56.1 53.6 / 69.9 Pixel Space VPFT ✗ 75.4 / 79.5 59.0 / 64.0 33.8 / 52.2 56.1 / 65.2 VPRL ✗ 91.6 / 93.2 74.5 / 77.6 75.8 / 83.8 80.6 / 84.9 Latent Space LVR∗ ✔ 6.8 / 16.0 5.0 / 10.2 0.0 / 8.0 3.9 / 11.3 Monet∗ ✔ 5.0 / 13.0 2.3 / 5.0 0.0 / 7.5 2.4 / 8.5 Recurrent State (Ours) SSVR ✔ 99.5 / 99.6 96.3 / 98.0 83.9 / 90.6 93.2 / 96.1 Table 1 reports aggregate performance. SSVR achieves the highest EM and PR on all three tasks, averaging 93.2/96.1 and outperforming the strongest pixel baseline, VPRL, by 12.6/11.2 percentage points. The largest improvement occurs on Maze, where EM/PR rises from 74.5/77.6 to 96.3/98.0, a gain of 21.8/20.4 points. SSVR also achieves 99.5/99.6 on FrozenLake and 83.9/90.6 on MiniBehavior, compared with VPRL’s 91.6/93.2 and 75.8/83.8, respectively. These gains are consistent with the benefit of conditioning decisions on an updated recurrent state, although MiniBehavior remains more challenging than the navigation tasks. The baseline pattern is consistent with the proposed state-representation mismatch. In text space, verbalizing a map is a lossy compression: walls, narrow passages, and adjacency relations are easily blurred, and these omissions enter the evolving textual state. Task-specific text SFT reaches 53.6/69.9 EM/PR. In pixel space, each generated frame can introduce errors, including duplicated or disappearing objects in Figure 2. Subsequent steps condition on these generated states, allowing errors to propagate with rollout length. The evaluated latent-space baselines exhibit bypass and weak transfer. On our tasks, Monet ceases to emit latent tokens and falls back to text-only reasoning. During output generation, LVR assigns 3.5% attention to image tokens and 3.8% to intermediate latent tokens, while text tokens dominate (Figure 2), indicating limited direct attention to visual and intermediate latent representations. The released LVR and Monet models average 3.9% and 2.4% EM, respectively, in this transfer setting. SSVR learns an action-conditioned recurrent update and uses the resulting latent state for each action decision. Figure 4: Maze EM and PR across image scale ratios 0.7–1.3, overall and by difficulty level. 5.2 Robustness and Generalization We evaluate SSVR across prompt paraphrasing, image transformations, cross-dataset transfer, and VQA retention. Table 2: Prompt paraphrase robustness (%). Prompt EM PR Legal Default 96.4 98.1 100.0 Paraphrased 96.8 98.3 100.0 Table 3: Native VQAv2 retention. Model Acc. (%) Δ (p) Native Qwen 71.9 0.0 SSVR 68.0 −3.9-3.9 VPRL ✗ ✗ 5.2.1 Input Robustness With semantically equivalent prompt paraphrases (Table 3), EM increases from 96.4 to 96.8 and PR from 98.1 to 98.3, while the legal rate remains 100%. These small changes indicate stable performance under semantically equivalent wording. We resize Maze inputs over ratios 0.7–1.3 to test scale robustness. Figure 4 reveals a difficulty-dependent response: Level 3 maintains 100% EM/PR throughout, and Level 4 EM remains between 98.8% and 99.6%. Shrinking substantially affects harder maps: at scale 0.7, Level-5 EM/PR falls from 95.20/97.03 to 72.00/84.26, and Level-6 EM/PR from 90.80/95.52 to 46.40/68.22, relative to scale 1.0. Performance under enlargement remains close to the native-scale baseline; at scale 1.3, Level-5 and Level-6 EM/PR are 96.40/97.81 and 89.60/94.56, respectively. This asymmetry suggests sensitivity to the loss of fine visual detail on difficult maps under downscaling, together with stable performance under enlargement. We further invert colors and synchronously flip images, evaluator coordinates, and action labels to test sensitivity to appearance and orientation. Table 5 shows that color inversion reduces EM from 96.40 to 94.00 and PR from 98.06 to 96.57, with the legal rate remaining at 99.90%. Vertical and horizontal flips each reduce EM by 0.40 points to 96.00, while flipping both directions reduces it by 1.40 points to 95.00; all flips retain a 100% legal rate. These modest declines indicate robustness to the tested appearance and orientation changes, with greater sensitivity to color inversion. Since these transformations preserve topology, the results are consistent with the use of relational geometry. Table 4: Color/flip robustness. Transformation EM PR Legal Rate Original 96.40 98.06 100.00 Color inversion 94.00-2.40 96.57-1.49 99.90-0.10 Vertical flip 96.00-0.40 97.62-0.44 100.000.00 Horizontal flip 96.00-0.40 97.77-0.29 100.000.00 Dual-direction flip 95.00-1.40 97.00-1.06 100.000.00 Table 5: FrozenLake→ transfer. B+MazeIF uses a Maze-trained interface. Method Maze EM Maze PR Qwen Direct 0.6 14.5 Qwen CoT 2.3 15.2 VPRL 6.8 7.2 SSVR (B+MazeIF) 30.9 36.9 SSVR (Full) 33.6 38.3 5.2.2 Cross-Dataset Transfer Table 6: Native Maze action-choice VQA in generation mode. Level Base Opt. Trained Opt. Base Legal Trained Legal 3 24.00 50.40+26.40 43.60 72.80+29.20 4 22.80 49.20+26.40 49.60 72.80+23.20 5 24.00 44.80+20.80 53.60 72.80+19.20 6 24.40 42.40+18.00 60.00 73.60+13.60 Avg. 23.80 46.70+22.90 51.70 73.00+21.30 FrozenLake→ transfer shifts from avoiding circular holes to parsing linear wall topology. It changes both visual primitives and connectivity cues, testing transferable path finding. Full applies the FrozenLake-trained backbone and GRU module directly to Maze. B+MazeIF combines the FrozenLake-trained backbone adapters with the Maze-trained GRU/action interface, evaluating compatibility across separately trained components. Table 5 shows that both variants substantially outperform Qwen and VPRL. Full raises EM from 2.3% for Qwen-CoT and 6.8% for VPRL to 33.6%. This transfer from holes to wall topology is consistent with learned use of connectivity cues. The gains also persist on hard maps: Full reaches 27.2/32.1 EM/PR on Level 5 and 22.4/29.9 on Level 6, versus VPRL’s 2.4/2.6 and 4.0/4.6. B+MazeIF is slightly weaker than Full, possibly because replacing the GRU interface disrupts the joint calibration learned during training. 5.2.3 VQA Retention We disable the GRU state-conditioning pathway and evaluate the trained backbone through its original LM head on native Maze action-choice VQA and VQAv2, using an LLM judge for VQAv2 semantic correctness. Table 6 shows that average optimal-action and legal-action hit rates rise from 23.8% to 46.7% and from 51.7% to 73.0%, respectively, with gains at every difficulty level. On VQAv2 (Table 3), SSVR retains 68.0% accuracy versus native Qwen’s 71.9%, a 3.9-point decrease; VPRL does not support native text-response VQA. Thus, training with the GRU state module improves the backbone’s native spatial action-choice capability while largely preserving general VQA ability. 5.3 Ablations We analyze two key designs: soft supervision and the action-output interface. 5.3.1 Soft-Target Coefficient Table 7: Soft-target coefficient ablation on Maze. α Maze EM Maze PR Entropy VQAv2 Acc. Paraphrase EM Scale 0.7 EM Scale 1.3 EM 1.0 91.8 95.7 0.0045 67.6 91.7 68.7 90.5 0.7 96.4 98.1 0.4833 68.0 96.8 79.4 96.4 0.4 95.5 97.5 0.7287 71.1 94.7 80.0 95.1 We vary only α to examine the balance between safe-action and optimal-action supervision. Table 7 shows that reducing α from 1.0 to 0.7 increases action entropy from 0.0045 to 0.4833 and improves Maze EM/PR from 91.8/95.7 to 96.4/98.1, with gains in all other evaluated metrics. Further reducing α to 0.4 raises entropy to 0.7287 and VQAv2 accuracy from 68.0 to 71.1, while slightly improving EM at scale 0.7 from 79.4 to 80.0. Maze EM/PR falls to 95.5/97.5, paraphrase EM drops from 96.8 to 94.7, and EM at scale 1.3 decreases from 96.4 to 95.1. Greater safe-action weight thus yields higher policy entropy with differing effects across planning and robustness metrics. We use α=0.7α=0.7 because it achieves the best in-domain EM/PR, paraphrase EM, and enlarged-image EM among the tested settings, while α=0.4α=0.4 favors VQA retention and robustness to image shrinking. 5.3.2 Action Output Table 8: Action-output ablation. Variant Maze EM (%) Maze PR (%) VQAv2 (%) SSVR 96.3 98.0 68.0 SSVR-head 96.4+0.1 98.5+0.5 46.9-21.1 Table 8 shows that replacing the LM action head with a linear softmax classifier, while keeping the GRU fixed, increases Maze EM/PR from 96.3/98.0 to 96.4/98.5, gains of 0.1/0.5 percentage points, and reduces VQAv2 accuracy from 68.0 to 46.9, a loss of 21.1 points. VQAv2 uses native text generation through the original LM head for both variants. Reusing the LM head during planning training therefore retains more general VQA capability in this comparison while achieving comparable planning performance. 6 Discussion 6.1 Interpretability Analysis Figure 5: Gradient-weighted visual attribution from the GRU state-token query to image tokens. As the recurrent latent state changes, attribution highlights next-action-relevant regions: nearby walls in Maze, nearby holes in FrozenLake, and a phase-dependent shift from printer to table in MiniBehavior. We use gradient-weighted attention to visualize attribution from the GRU state-token query to image tokens, conditioned on the selected action logit. Figure 5 shows changing attribution across planning steps: nearby walls in Maze, nearby holes in FrozenLake, and a shift from printer to table across the PICK/DROP phases in MiniBehavior. This analysis measures the current decision’s visual readout with the latent state and prefix cache held fixed, illustrating how static context is read under the evolving recurrent state. 6.2 Efficiency Analysis 6.2.1 Qualitative Analysis SSVR’s efficiency advantage comes from static-context reuse and compact recurrent updates. Let NCN_C be the number of static-context tokens, NE=1N_E=1 the state-token count, NIN_I the number of intermediate visual-frame tokens, and H the planning horizon. With fixed decoder width and depth, the attention cost with a static prefix KV cache is (NC2+H(NENC+NE2)).O (N_C^2+H(N_EN_C+N_E^2) ). (10) The GRU additionally costs (Hd2)O(Hd^2) for input and hidden dimension d. A visual planner that processes one NIN_I-token intermediate frame per step against a fixed prefix has attention cost (NC2+H(NINC+NI2)).O (N_C^2+H(N_IN_C+N_I^2) ). (11) These expressions describe attention over the stated token sequences; total runtime also includes projections, feed-forward layers, and visual processing. Since NE≪NIN_E N_I, SSVR reduces the per-step token workload while reusing the visual-textual prefix. 6.2.2 Quantitative Analysis Table 9: Efficiency on Maze case0. †: KV cache; latency excludes preprocessing and, for SSVR†, prefix-cache construction. Multipliers are relative to SSVR†. Method Step latency (ms) Rollout latency (ms) Throughput (cases/s) Peak memory (GiB) SSVR† 45.611 (1.00x) 182.442 (1.00x) 5.481 (1.00x) 15.918 (1.00x) SSVR 89.159 (1.95x) 356.634 (1.95x) 2.804 (0.51x) 15.937 (1.00x) LVR 529.596 (11.61x) 1059.193 (5.81x) 0.944 (0.17x) 17.609 (1.11x) Monet 1743.966 (38.24x) 6975.866 (38.24x) 0.143 (0.03x) 15.705 (0.99x) VPRL 4496.111 (98.58x) 17984.442 (98.58x) 0.056 (0.01x) 13.331 (0.84x) We benchmark all methods on the same Maze case0 sample over 20 repetitions after warm-up, using CUDA events for latency and the PyTorch CUDA memory API for peak memory. Image/text preprocessing and prefix-cache construction are measured separately; cached SSVR rollout timing covers decoding with the prebuilt cache and recurrent state updates. As shown in Table 9, enabling KV cache reduces SSVR step latency from 89.159 ms to 45.611 ms and decoding-rollout latency from 356.634 ms to 182.442 ms, giving about a 1.95×1.95× speedup with nearly unchanged peak memory. Compared with LVR, Monet, and VPRL, cached SSVR has 5.81×5.81×, 38.24×38.24×, and 98.58×98.58× lower rollout latency under this timing protocol, illustrating the computational benefit of reusing static visual context during recurrent planning. 7 Conclusion This paper identifies state-representation mismatch as an obstacle in VLA open-loop planning: existing paradigms can encode evolving state through lossy textual compression, error-accumulating pixel rollouts, or bypass-prone latent tokens, even though most visual context remains static. SSVR addresses this mismatch by encoding the initial visual-textual context once and recursively updating a learned latent state through an action-conditioned GRU. Across three planning tasks, this design yields strong in-domain performance, robust behavior under linguistic and visual transformations, cross-dataset transfer, and efficient cached decoding while largely retaining general VQA ability. These results support action-conditioned recurrent state modeling as a useful design principle for VLA open-loop planning. AI-Assisted Content Disclosure Generative AI tools assisted with linguistic polishing and technical revision of the manuscript, including method descriptions and mathematical notation. The authors are responsible for verifying the revised text against the implementation and experimental records and for all claims, content, and findings in the paper. References Agashe et al. (2025) S. Agashe, J. Han, S. Gan, J. Yang, A. Li, and X. Wang Agent s: an open agentic framework that uses computers like a human. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 22924–22946. External Links: Link Cited by: §1. Alayrac et al. (2022) J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. Flamingo: a visual language model for few-shot learning. arXiv preprint arXiv:2204.14198. Cited by: §2. Bai et al. (2025) S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin Qwen2.5-vl technical report. External Links: 2502.13923, Link Cited by: §1. Bigverdi et al. (2024) M. Bigverdi, Z. Luo, C. Hsieh, E. Shen, D. Chen, L. G. Shapiro, and R. Krishna Perception tokens enhance visual reasoning in multimodal language models. External Links: 2412.03548, Link Cited by: §2. Chen et al. (2026) C. Chen, Z. Ma, Y. Li, Y. Hu, Y. Wei, W. Li, and L. Nie Reasoning in the dark: interleaved vision-text reasoning in latent space. In Findings of the Association for Computational Linguistics: ACL 2026, p. 39117–39129. Cited by: §2. Chern et al. (2025) E. Chern, Z. Hu, S. Chern, S. Kou, J. Su, Y. Ma, Z. Deng, and P. Liu Thinking with generated images. External Links: 2505.22525, Link Cited by: §2. Cui et al. (2026) J. Cui, X. Long, X. Zhang, Y. Zhang, C. Su, J. Gan, B. Zhao, and P. Ren Retrieve, integrate, and synthesize: spatial-semantic grounded latent visual reasoning. External Links: 2605.07106, Link Cited by: §2. Goyal et al. (2017) Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh Making the v in vqa matter: elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 6904–6913. Cited by: §1. Guo et al. (2026) Z. Guo, R. Liu, X. Chen, and P. A. Heng ATLAS: agentic or latent visual reasoning? one word is enough for both. arXiv preprint. Cited by: §2. Hao et al. (2026) Y. Hao, Y. Chen, C. Fan, and Y. Zhang Simulation to rules: a dual-vlm framework for formal visual planning. External Links: 2510.03182, Link Cited by: §1. Hong et al. (2025) J. Hong, C. Zhao, C. Zhu, W. Lu, G. Xu, and X. Yu DeepEyesV2: toward agentic multimodal model. arXiv preprint arXiv:2511.05271. Cited by: §2. Hu et al. (2023) Y. Hu, F. Lin, T. Zhang, L. Yi, and Y. Gao Look before you leap: unveiling the power of gpt-4v in robotic vision-language planning. External Links: 2311.17842, Link Cited by: §1. Ivanitskiy et al. (2023) M. I. Ivanitskiy, R. Shah, A. F. Spies, T. Räuker, D. Valentine, C. Rager, L. Quirke, C. Mathwin, G. Corlouer, C. D. Behn, and S. W. Fung A configurable library for generating and manipulating maze datasets. External Links: 2309.10498, Link Cited by: §1, §5. Jiang et al. (2026a) Y. Jiang, R. Xing, L. Zhao, B. Plancher, M. Chen, and D. Balkcom ASCII art turns llms into vla controllers. External Links: 2606.21470, Link Cited by: §2. Jiang et al. (2026b) Z. Jiang, K. Chen, K. Li, K. Yin, Y. Zhou, Z. Wang, C. Lv, and S. Zhang Acckv: towards efficient audio-video llms inference via adaptive-focusing and cross-calibration kv cache optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 5494–5502. Cited by: §1. Jiang et al. (2025) Z. Jiang, K. Li, Y. Zhou, S. Liu, Z. Wang, C. lv, and S. Zhang PureKV: plug-and-play kv cache optimization with spatial-temporal sparse attention for vision-language large models. External Links: 2510.25600, Link Cited by: §1. Jin et al. (2023) E. Jin, J. Hu, Z. Huang, R. Zhang, J. Wu, L. Fei-Fei, and R. Martín-Martín Mini-behavior: a procedurally generated benchmark for long-horizon decision-making in embodied ai. External Links: 2310.01824, Link Cited by: §1, §5. Li et al. (2025a) B. Li, X. Sun, J. Liu, Z. Wang, J. Wu, X. Yu, H. Chen, E. Barsoum, M. Chen, and Z. Liu Latent visual reasoning. External Links: 2509.24251, Link Cited by: §2. Li et al. (2025b) C. Li, W. Wu, H. Zhang, Y. Xia, S. Mao, L. Dong, I. Vulić, and F. Wei Imagine while reasoning in space: multimodal visualization-of-thought. External Links: 2501.07542, Link Cited by: §2. Li et al. (2026) Y. Li, C. Chen, Y. Li, F. Zeng, K. Huang, J. Xu, and M. Sun Imagination helps visual reasoning, but not yet in latent space. In Forty-third International Conference on Machine Learning, Cited by: §2. Liu et al. (2025a) C. Liu, Y. Yang, Y. Fan, Q. Wei, S. Liu, and X. E. Wang Reasoning within the mind: dynamic multimodal interleaving in latent space. arXiv preprint arXiv:2512.12623. Cited by: §2. Liu et al. (2025b) D. Liu, Z. Wang, M. Ruan, F. Luo, C. Chen, P. Li, and Y. Liu Thinking with visual abstract: enhancing multimodal reasoning via visual abstraction. External Links: 2505.20164, Link Cited by: §2. Liu et al. (2023) H. Liu, C. Li, Q. Wu, and Y. J. Lee Visual instruction tuning. Advances in neural information processing systems 36, p. 34892–34916. Cited by: §2. Liu et al. (2025c) J. Liu, Z. Liu, Z. Cen, Y. Zhou, Y. Zou, W. Zhang, H. Jiang, and T. Ruan Can multimodal large language models understand spatial relations?. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 620–632. Cited by: §2. Miao et al. (2026) Y. Miao, Y. Sun, D. Wang, M. Zhou, P. Poupart, L. Lv, Q. Zhao, L. Wang, H. Li, X. Jiang, and G. Jiang Fill the gap: a granular alignment paradigm for visual reasoning in multimodal large language models. External Links: 2605.12374, Link Cited by: §2. Shah et al. (2023) D. Shah, B. Osiński, b. ichter, and S. Levine LM-nav: robotic navigation with large pre-trained models of language, vision, and action. In Proceedings of The 6th Conference on Robot Learning, K. Liu, D. Kulic, and J. Ichnowski (Eds.), Proceedings of Machine Learning Research, Vol. 205, p. 492–504. External Links: Link Cited by: §1. Tang et al. (2026) Y. Tang, Y. Yang, T. Bao, and Y. Tong Thinking in latent space: progressive multimodal simplification for visual reasoning. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §2. Venhoff et al. (2025) C. Venhoff, A. Khakzar, S. Joseph, P. Torr, and N. Nanda How visual representations map to language feature space in multimodal llms. arXiv preprint arXiv:2506.11976. Cited by: §2. Wang et al. (2026) C. Wang, W. Bao, S. Gao, B. Xu, Y. Tian, Y. S. Rawat, Y. Ge, and Y. Shang VLA-thinker: boosting vision-language-action models through thinking-with-image reasoning. ArXiv abs/2603.14523. External Links: Link Cited by: §2. Wang et al. (2025) Q. Wang, Y. Shi, Y. Wang, Y. Zhang, P. Wan, K. Gai, X. Ying, and Y. Wang Monet: reasoning in latent visual space beyond images and language. External Links: 2511.21395, Link Cited by: §2. Wei et al. (2026) L. Wei, L. He, J. Lan, L. Dong, Y. Cai, S. Li, H. Zhu, W. Wang, L. Kong, Y. Wang, Z. Zhang, and W. Huang Zooming without zooming: region-to-image distillation for fine-grained multimodal perception. External Links: 2602.11858, Link Cited by: §2. Wu et al. (2024a) Q. Wu, H. Zhao, M. Saxon, T. Bui, W. Y. Wang, Y. Zhang, and S. Chang VSP: assessing the dual challenges of perception and reasoning in spatial planning tasks for vlms. External Links: 2407.01863, Link Cited by: §1, §5. Wu et al. (2024b) W. Wu, S. Mao, Y. Zhang, Y. Xia, L. Dong, L. Cui, and F. Wei Mind’s eye of llms: visualization-of-thought elicits spatial reasoning in large language models. External Links: 2404.03622, Link Cited by: §2. Xu et al. (2026) Y. Xu, C. Li, H. Zhou, X. Wan, C. Zhang, A. Korhonen, and I. Vulić Visual planning: let’s think only with images. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §5. Yang et al. (2025) Z. Yang, X. Yu, D. Chen, M. Shen, and C. Gan Machine mental imagery: empower multimodal reasoning with latent visual tokens. External Links: 2506.17218, Link Cited by: §2. Yu et al. (2026) X. Yu, Z. Chen, Y. He, T. Fu, G. Dong, C. Yang, C. Xu, Y. Ma, X. Hu, Z. Cao, J. Xu, G. Zhang, J. Tao, J. Zhang, S. Ma, K. Feng, H. Huang, Y. Li, R. Chen, H. Wang, C. Wu, Z. Su, X. Xu, K. Yao, K. Wang, C. Gao, Y. Liao, R. Huang, T. Jin, Z. Xue, C. Tan, J. Zhang, W. Ren, Y. Fu, Y. Liu, Y. Wang, X. Yue, Y. Jiang, and S. Yan The latent space: foundation, evolution, mechanism, ability, and outlook. External Links: 2604.02029, Link Cited by: §2. Zhao et al. (2025) Q. Zhao, Y. Lu, M. J. Kim, Z. Fu, Z. Zhang, Y. Wu, Z. Li, Q. Ma, S. Han, C. Finn, A. Handa, T. Lin, G. Wetzstein, M. Liu, and D. Xiang CoT-vla: visual chain-of-thought reasoning for vision-language-action models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 1702–1713. External Links: Document Cited by: §2. Zheng et al. (2026) Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, C. Shen, and X. Yu DeepEyes: incentivizing ”thinking with images” via reinforcement learning. External Links: 2505.14362, Link Cited by: §2. Appendix A Evaluation Metrics We use Exact Match (EM) and Progress Rate (PR) to evaluate open-loop plans. Consider an instance with M shortest optimal action sequences of length n>0n>0, (m)=(a1(m),…,an(m)),m∈1,…,M,a^(m)=(a_1^(m),…,a_n^(m)), m∈\1,…,M\, (12) where aj(m)a_j^(m) is the j-th action of optimal sequence m. Let the predicted action sequence under the evaluation budget H=nH=n be ^=(a^1,…,a^n) a=( a_1,…, a_n); missing actions are treated as mismatches. The instance-level EM is EM=maxm∈1,…,M∏j=1n[a^j=aj(m)].EM= _m∈\1,…,M\ _j=1^nI\! [ a_j=a_j^(m) ]. (13) Thus, EM is one if the prediction exactly follows any shortest optimal action sequence and zero otherwise. The maximum accounts for all equally short optimal action sequences. PR measures the longest optimal prefix: PR=maxm∈1,…,M1n∑j=1n∏k=1j[a^k=ak(m)].PR= _m∈\1,…,M\ 1n _j=1^n _k=1^jI\! [ a_k=a_k^(m) ]. (14) If the first ℓ actions form an optimal prefix and the first error occurs at action ℓ+1 +1, then PR=ℓ/nPR= /n. In the deterministic, unit-cost evaluation environments, an action is optimal when its transition is valid and reduces the task-specific shortest-path distance-to-goal by exactly one; successful termination has distance zero. This distance-based test identifies prefixes of shortest optimal action sequences. Dataset-level EM and PR are the arithmetic means of the corresponding instance-level scores and are reported as percentages in the main paper. Appendix B Training and Inference Algorithms Algorithms 1 and 2 implement the method in Section 4, using θ=(θ0,Δθ)θ=( _0, θ) and latent-state parameters η. The initializer fηinitf^init_η and transition fηtransf^trans_η follow Equations 3 and 7. B.1 Implementation Details Packing and batching. The metadata ℳM contains the prefix attention mask, multimodal position indices, appended-token position, and image-token key indices. Visual pooling includes valid image tokens. Padded reference actions preserve the latent state during batched history replay. Cache handling. An in-place cache is restored to the prefix length NCN_C after each decode, preserving tensors required for training gradients. Training computes fresh differentiable caches for each minibatch. A full-sequence forward on C∥etC \|e_t provides an equivalent formulation in evaluation mode, where dropout is disabled. Action-head ablation. The classifier variant uses (Wωut+bω)a(W_ωu_t+b_ω)_a for the action logit and includes ω in the optimizer and saved model. The GRU update and soft-target supervision follow the main method. B.2 Training Pseudocode The training set D contains tuples (I0,x,<tref,safe,opt)(I_0,x,a^ref_<t,A_safe,A_opt). Each tuple supervises one reachable decision state; an empty history denotes the initial decision. The safe set is the base admissible set after the supervision protocol’s action filters. Algorithm 1 averages the per-item loss in Equation 9 over the minibatch. In the pseudocode, qα(a)q_α(a) denotes the target at the reference decision state of the current item. Algorithm 1 Soft-target training with a GRU latent state 1: D as defined above; frozen pretrained parameters θ0 _0; trainable LoRA parameters Δθ θ; latent initializer fηinitf^init_η; transition fηtransf^trans_η; injective action verbalizer v; α∈[0,1]α∈[0,1] 2: Trained Δθ θ and η 3: Initialize Δθ θ and η; keep θ0 _0 frozen 4: for each training minibatch ℬ⊂B do 5: Clear optimizer gradients; ℒ←0L← 0 6: for each (I0,x,<tref,safe,opt)∈ℬ(I_0,x,a^ref_<t,A_safe,A_opt) do 7: V0←ℰθvis(I0)V_0 ^vis_θ(I_0) 8: (C,ℳ)←Packθ0(V0,x)(C,M) _ _0(V_0,x) 9: ←Fθprefill(C,ℳ)K← F_θ^prefill(C;M) ⊳ retain the training computation graph 10: z←fηinit(V0)z← f^init_η(V_0) 11: for k=0,…,t−1k=0,…,t-1 do 12: z←fηtrans(z,akref)z← f^trans_η(z,a^ref_k) ⊳ differentiable reference-history replay 13: end for 14: e←ze← z 15: u←Fθdecode(,e,ℳ)u← F_θ^decode(K,e;M) ⊳ read-only prefix cache; one appended token 16: for each action a∈a do 17: o(a)←LMHeadθ0(u)v(a)o(a) _ _0(u)_v(a) 18: end for 19: πθ,η(a∣C,z)←softmaxa∈(o(a)) _θ,η(a C,z) _a (o(a)) 20: Construct qαq_α using Equation 8 and its empty-optimal-set fallback 21: ℒ←ℒ−∑a∈qα(a)logπθ,η(a∣C,z)L - _a q_α(a) _θ,η(a C,z) 22: end for 23: Backpropagate ℒ/|ℬ|L/|B| through decoding, prefill, and all replayed GRU transitions 24: Update only (Δθ,η)( θ,η) using the optimizer 25: Discard all minibatch KV caches and recurrent-state computation graphs 26: end for 27: return Δθ,η θ,η B.3 Inference Pseudocode Algorithm 2 uses the horizon and evaluation protocol in Section 3. Greedy selection uses a fixed tie-breaking rule. Every predicted action drives the GRU transition, including actions marked invalid by the evaluator. Algorithm 2 Open-loop GRU rollout with a read-only visual–textual KV cache 1: Initial image I0I_0; instruction x; planning horizon H; trained parameters (θ0,Δθ,η)( _0, θ,η); action verbalizer v 2: Action plan a of length H 3: Set all modules to evaluation mode; disable gradients for ordinary rollout 4: V0←ℰθvis(I0)V_0 ^vis_θ(I_0) 5: (C,ℳ)←Packθ0(V0,x)(C,M) _ _0(V_0,x) 6: ←Fθprefill(C,ℳ)K← F_θ^prefill(C;M) ⊳ compute the static prefix once 7: z←fηinit(V0)z← f^init_η(V_0); ←[]a←[\ ] 8: for t=0,…,H−1t=0,…,H-1 do 9: et←ze_t← z 10: ut←Fθdecode(,et,ℳ)u_t← F_θ^decode(K,e_t;M) ⊳ same prefix and appended position; discard temporary state KV 11: for each action a∈a do 12: ot(a)←LMHeadθ0(ut)v(a)o_t(a) _ _0(u_t)_v(a) 13: end for 14: at←argmaxa∈ot(a)a_t← _a o_t(a) ⊳ use a fixed tie-breaking rule 15: Append ata_t to a 16: z←fηtrans(z,at)z← f^trans_η(z,a_t) ⊳ learned action-conditioned latent update 17: end for 18: return a Appendix C State-Conditioned Visual Attribution Analysis This section defines visual attribution for the GRU latent-state variant under the same projected-image-token and cached-decoding assumptions. At step t, the conditioning state is the recurrent vector ztz_t derived from the initial image features and the predicted action history. C.1 Gradient-Weighted Attribution Method Attention indexing under cached decoding. Let ⊂0,…,NC−1P⊂\0,…,N_C-1\ be the image-token key positions in the packed static prefix. During a one-token cached decode, each decoder layer has one current query and NC+1N_C+1 keys, including the current state token. For decoder layer ℓ and query-attention head j, write the relevant attention matrix as Atℓ,j∈ℝ1×(NC+1).A_t ,j ^1×(N_C+1). (15) The state query has local row index 00 and full-sequence position NCN_C. The state-to-image attention is at,pℓ,j=Atℓ,j[0,p],p∈.a_t,p ,j=A_t ,j[0,p], p . (16) For a full-sequence forward on C∥etC \|e_t, the query row is NCN_C. Image-token indices follow the packed key order, including interleaved special and text tokens. In grouped-query attention, the averaging below uses query-attention heads. Diagnostic forward with gradients. Ordinary rollout uses inference without gradients. For attribution, we snapshot the pre-action latent state ztz_t, retain the same static prefix and selected action, and perform a separate diagnostic one-token decode with gradient tracking enabled. The diagnostic pass holds model parameters and the stored rollout state fixed, with dropout disabled. The state snapshot and prefix KV cache are detached from the rollout graph, and a fresh state-embedding leaf enables gradient tracking for the diagnostic decode. The diagnostic forward exposes the differentiable attention probabilities used in the attention-value product. An explicit attention implementation supports this gradient calculation. Numerical agreement with the ordinary decoder is checked when switching attention implementations, and the explained action remains fixed to the rollout selection. Temporary state-token KV entries are discarded after decoding. Action-conditioned gradient. Let the action selected by the ordinary rollout be a^t=argmaxa∈ot(a). a_t= _a o_t(a). (17) The selected action a^t a_t is held fixed during differentiation. Let otdiag(a^t)o_t^diag( a_t) denote the corresponding pre-softmax action logit in the diagnostic forward. Its derivative with respect to the attention tensor is gt,pℓ,j=[∂otdiag(a^t)∂Atℓ,j]0,p,p∈.g_t,p ,j= [ ∂ o_t^diag( a_t)∂ A_t ,j ]_0,p, p . (18) Computationally, the gradient is obtained with respect to the attention tensor used by the forward pass and is then indexed at [0,p][0,p]. We retain positive gradient-weighted attention and average it over the selected decoder layers ℒL and their query-attention heads: rt(p)=1|ℒ|∑ℓ∈ℒ1Nhℓ∑j=1Nhℓ[at,pℓ,jgt,pℓ,j]+,[z]+=max(z,0).r_t(p)= 1|L| _ 1N_h _j=1^N_h [a_t,p ,jg_t,p ,j ]_+, [z]_+= (z,0). (19) Here NhℓN_h is the number of query-attention heads at layer ℓ ; if it is constant across layers, this reduces to division by |ℒ|Nh|L|N_h. A positive product indicates positive local sensitivity of the selected action logit to the corresponding attention entry, weighted by the attention magnitude. For visualization, the scores are normalized separately at each step: r~t(p)=rt(p)maxp′∈rt(p′)+ϵ,ϵ>0. r_t(p)= r_t(p) _p r_t(p )+ε, ε>0. (20) If every score is zero, the displayed attribution map is zero. The scores are mapped to the spatial grid of the actual decoder-visible image feature tokens, accounting for visual token merging or spatial reordering. They are then bilinearly upsampled to the initial-image resolution and alpha-blended with I0I_0. Per-step normalization displays relative attribution within each panel. Interpretation and scope. All panels within a trajectory use the same initial image, instruction, and static-prefix KV cache. Across steps, the GRU latent state ztz_t, its appended token et=zte_t=z_t, the decoder readout, and the selected action can change. The diagnostic backward pass holds model parameters and the stored rollout state fixed. Conditioned on fixed ztz_t and prefix keys and values, this analysis describes the current decision’s state-to-image-token readout through local sensitivity of the selected action logit. Accordingly, r~t r_t is a latent-state- and action-conditioned attribution map. The first panel of each row shows the unchanged initial observation; subsequent panels display the attribution overlay and selected action. C.2 Qualitative Results across Tasks We next examine complete trajectories from FrozenLake, Maze, and MiniBehavior. These tasks respectively test whether visual attribution follows changing local hazards, wall-constrained topology, and phase-dependent object interactions while the visual input remains fixed. Figure 6: Additional FrozenLake and Maze trajectories. In FrozenLake (left), attribution emphasizes nearby holes and safe local passages as the GRU latent state evolves with the predicted actions. In Maze (right), it moves across the fixed map and concentrates on relevant walls and openings. The shift across steps is consistent with latent-state-conditioned reading of a persistent scene. Figure 6 contains examples with different map sizes, start locations, goals, and path lengths. In FrozenLake, high attribution frequently appears around hazards adjacent to the position reached by the predicted actions, which are relevant to avoiding an immediately failing transition. In Maze, attribution tracks the relevant wall segment or opening as the predicted plan progresses. Highlighted regions can include neighboring obstacles and corridors relevant to the current decision. Figure 7: Additional MiniBehavior trajectories. Each row follows a complete two-phase plan: the agent first navigates to the printer and executes PICK, then navigates to the table and executes DROP. The GRU updates its latent state from the predicted action history. Attribution correspondingly shifts from the printer and its approach path to the table and its approach path. MiniBehavior further tests whether attribution changes with the task phase represented implicitly through the action history. As shown in Figure 7, the pre-pick panels emphasize the printer and navigational constraints on the way to it. After PICK, the visual input remains fixed, and the GRU latent state is updated using the selected action; attribution then moves toward the table and relevant boundaries before DROP. This phase-dependent change provides qualitative evidence that the same scene is re-read according to the evolving latent state. Appendix D Implementation and Experimental Details D.1 Task Interfaces and Data FrozenLake22 2 https://huggingface.co/yixu1/VPRL-7B-FrozenLake and Maze33 3 https://huggingface.co/yixu1/VPRL-7B-Maze use four actions, UP, DOWN, LEFT, and RIGHT. Their model state is a GRU latent vector updated from the action history. MiniBehavior44 4 https://huggingface.co/yixu1/VPRL-7B-MiniBehaviour additionally uses PICK and DROP, with carrying information modeled implicitly in the same recurrent state. Initial-map tokens are decoded once with the MUSE VQ-VAE into 256×256256× 256 images and cached on disk; Qwen’s native image processor then constructs the multimodal input. Rollout uses the initial observation and the recurrent state updated from predicted actions. Table 10: Dataset-specific state and action interfaces. Task Model state Action tokens FrozenLake zt∈ℝ3584z_t ^3584 U/D/L/R Maze zt∈ℝ3584z_t ^3584 U/D/L/R MiniBehavior zt∈ℝ3584z_t ^3584 U/D/L/R/PICK/DROP D.2 Optimization All main GRU-based models initialize the backbone from Qwen2.5-VL-7B-Instruct. The pretrained backbone weights are frozen and adapted using LoRA, while the GRU state module is learned jointly. LoRA is applied to the attention projections and MLP projections listed in Table 11. We use the soft target from the main paper with α=0.7α=0.7. Table 11: Main training and evaluation configuration. The learning-rate scheduler is constructed for ten epochs, while training is stopped after epoch five. Setting Value Backbone Qwen2.5-VL-7B-Instruct Trainable adaptation LoRA + GRU state module LoRA rank / lora_alpha / dropout 32 / 64 / 0.1 LoRA target modules q/k/v/o_proj, gate/down/up_proj Optimizer fused AdamW Peak learning rate 1.5×10−41.5× 10^-4 Schedule / warmup cosine / 10% of scheduled steps Scheduled / executed epochs 10 / 5 Per-device batch size 16 Gradient accumulation 1 Number of training processes 8 Training hardware 8× NVIDIA H800 GPUs Effective global batch size 128 Training precision bfloat16 mixed precision with TF32 enabled Initialization / Trainer seed 2026 / 42 Checkpointing once per epoch Soft-target coefficient α 0.7 GRU input / hidden dimension 3584 / 3584 Image reconstruction resolution 256×256256× 256 Evaluation decoding greedy over task action tokens Evaluation environment dynamics deterministic, no slip All models are trained on eight NVIDIA H800 GPUs with one distributed training process per GPU. The per-device batch size is 16 with one gradient accumulation step, giving an effective global batch size of 128. Training uses bfloat16 mixed precision, fused AdamW, a peak learning rate of 1.5×10−41.5× 10^-4, and a cosine schedule with a 10% warmup. The schedule is parameterized for ten epochs, and the reported checkpoint is saved after epoch five.