Paper deep dive
TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning
Binnan Liu, Yechi Ma, Tian Xie, Wei Hua
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 8/3/2026, 3:10:59 AM
Summary
The paper introduces TraceViT, a looped visual reasoner for the Abstraction and Reasoning Corpus (ARC) that employs grounded trace supervision. Unlike conventional methods that only constrain final outputs, TraceViT uses semantically monotonic transformation chains derived from verified programmatic implementations to supervise intermediate grid states. It utilizes a task reference and object workspace to ground iterations, and soft trace alignment to handle variable chain lengths. The model achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2.
Entities (10)
Relation Signals (8)
TraceViT → achievesscoreon → ARC-AGI-1
confidence 95% · TraceViT achieves 67.8% pass@2 on ARC-AGI-1
TraceViT → achievesscoreon → ARC-AGI-2
confidence 95% · and 24.3% on ARC-AGI-2
TraceViT → uses → Soft Trace Alignment
confidence 90% · soft trace alignment enforces only their ordering, letting the model allocate iterations freely.
TraceViT → usescomponent → Task Reference
confidence 90% · Each iteration is grounded by a task reference derived from the few-shot demonstrations
TraceViT → usescomponent → Object Workspace
confidence 90% · and an object workspace representing the current grid state.
TraceViT → buildson → LoopViT
confidence 85% · LoopViT ... is the architecture our method builds on.
ARC-GEN → sourceof → TraceViT
confidence 80% · ARC-GEN ... provides generators ... To increase diversity, we broaden size and object-count ranges in the ARC-GEN generators
RE-ARC → sourceof → TraceViT
confidence 80% · RE-ARC ... supplies generators ... We therefore use GPT-5.5 ... to decompose these implementations
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few-shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2607.29586v1
- Canonical: https://arxiv.org/abs/2607.29586v1
Trouble viewing inline? Open PDF directly →
Full Text
73,827 characters extracted from source content.
Expand or collapse full text
TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning Binnan Liu1,2, Yechi Ma1,2, Tian Xie1, Wei Hua1,2 Abstract The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input–output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few-shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at https://github.com/LiuBinnan/TraceViT. Introduction A core aspect of reasoning is the ability to infer an abstract rule from a few examples and apply it to a new instance step by step. The Abstraction and Reasoning Corpus (ARC) tests this ability with colored-grid puzzles (Fig. 1, left). Each task provides a few demonstration input–output pairs, typically two to four, that share a hidden transformation, e.g. moving objects, repairing symmetry, or recoloring by size. Given a held-out test input, the solver must produce the output grid from scratch by choosing its dimensions and assigning one of ten colors to every grid cell. A prediction counts as correct only if it matches the target grid exactly (Chollet 2019). Because each evaluation task instantiates a rule never seen during training, the solver must induce the transformation from its demonstrations rather than retrieve a memorized solution (Chollet 2019). This emphasis on generalization has made ARC a central benchmark for abstract reasoning (Chollet et al. 2025). Humans solve tasks from the demonstrations alone, often constructing the answer step by step (LeGris et al. 2025), whereas leading ARC solvers typically depend on large models, explicit program search, or per-task test-time adaptation (Greenblatt 2024; Pourcel, Colas, and Oudeyer 2025; Li et al. 2025b; Sorokin and Puget 2025). Figure 1: Grounded trace supervision. Left: an ARC task provides demonstrations and a test input. Right: TraceViT iterates a shared visual core, decoding a grid at every step. During training, intermediate predictions are aligned with transformation chain milestones. See Fig. 2 for the full pipeline. Compact recurrent models tackle ARC tasks by trading scale for iterative computation. HRM and TRM (Wang et al. 2025; Jolicoeur-Martineau 2025) use recurrent computation over tokenized grid sequences, whereas LoopViT (Shu et al. 2026) repeatedly applies a shared visual core to spatial grid representations, producing a decodable grid prediction at every iteration. Yet all these models anchor supervision to the final answer, applied once in LoopViT or repeated at every iteration in HRM and TRM, so no signal specifies what intermediate predictions should look like. In principle, explicit intermediate targets would decompose a complex transformation into simpler steps that are easier for the model to learn. Providing such targets requires intermediate grids from input to output, but existing sources are insufficient: human solving traces are scarce (Kim et al. 2026; LeGris et al. 2025), while language rationales are not directly usable as grid-state targets (Lehmann et al. 2026). Programmatic task implementations (Hodel 2024a; Moffitt 2025) offer a scalable alternative because they implicitly encode how the output is built. However, running these programs directly does not yield clean intermediate steps: programs may perform several changes at once or skip meaningful intermediate states entirely. We therefore decompose and verify each program so that execution yields semantically monotonic transformation chains—sequences of intermediate grids in which each milestone applies one meaningful action toward the answer without backtracking. These chains define what each iteration should produce. We introduce TraceViT (Fig. 1), which trains a looped reasoner with these chains as intermediate targets. In a looped reasoner such as LoopViT, the hidden state serves both as memory of the input and as the computation workspace. We argue that supervising intermediate states risks disrupting the stored input information, because the same representation must simultaneously preserve it and undergo transformation. To mitigate this, we externalize the memory role through a task reference that encodes all demonstrations into a compact summary and re-supplies it at every iteration, and an object workspace that decomposes the current scene into object-centric components and carries them across iterations. A separate challenge is that the number of chain steps and loop iterations need not match. To address this, soft trace alignment enforces only the ordering of the chain while letting the model decide how many iterations to spend on each step. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2 (Table 1), demonstrating strong performance across both benchmarks. These results provide empirical support for grounded trace supervision as an effective approach to compact visual reasoning. In sum, we contribute: • An ARC dataset with verified trace annotations: we rewrite and validate programmatic task implementations to produce semantically monotonic transformation chains as training targets. Once instrumented, trace annotations are generated alongside sampled instances, avoiding per-instance labeling and supporting scalable resampling. • Grounded trace supervision: an architecture and training objective that grounds every iteration with a task reference and object workspace, then uses soft trace alignment to supervise intermediate predictions while letting the model freely allocate iterations to milestones. • Systematic evaluation: experiments demonstrate strong performance on both ARC-AGI-1 and ARC-AGI-2. A factorial ablation identifies the interaction between trace supervision and grounding, while targeted variants evaluate key alignment choices. Related Work ARC-Related Datasets. ARC-AGI-1 (Chollet 2019) and ARC-AGI-2 (Chollet et al. 2025) serve as the standard benchmarks, with each task presented as a set of input–output grid pairs. Beyond these final pairs, several datasets capture intermediate human solving steps—from action traces to low-level interaction histories (LeGris et al. 2025; Kim et al. 2026; Strandgaard 2026). While these human-collected datasets offer rich process-level supervision, they are difficult to scale. Programmatic methods instead synthesize ARC-like data automatically. RE-ARC (Hodel 2024a), BARC (Li et al. 2025b), NVARC (Sorokin and Puget 2025), and ARC-GEN (Moffitt 2025) generate new input–output pairs through procedural sampling or LLM-driven program remixing, greatly expanding available training data. ARC-TGI (Lehmann et al. 2026) supplements a curated subset with natural-language reasoning chains. Yet no existing resource provides intermediate grid states at the scale needed for training: human traces capture them but are scarce, while programmatic generators emit only final input–output pairs. Approaches to ARC. Program induction and transduction. ARC solvers broadly fall into two paradigms: induction searches for a program that maps inputs to outputs, either through symbolic DSL search (Wind 2020; Hodel 2024b; Alford et al. 2021; Xu, Khalil, and Sanner 2023; Lei, Lipovetzky, and Ehinger 2024; Ouellette 2024) or LLM-guided code generation (Greenblatt 2024; Singhal and Shroff 2025; Pourcel, Colas, and Oudeyer 2025), while transduction directly predicts test outputs without an explicit program, typically by fine-tuning LLMs on text-serialized grids with test-time training (T) (Li et al. 2025b; Cole and Osman 2025; Akyürek et al. 2025; Franzen, Disselhoff, and Hartmann 2025b; Sorokin and Puget 2025). Both rely on expensive DSL search or large LLMs rather than learning a compact network that computes the transformation directly. Recurrence and iterative refinement. Pursuing exactly such compact models, HRM (Wang et al. 2025) couples two recurrent modules at different frequencies, while TRM (Jolicoeur-Martineau 2025) reduces this to a single recursively applied network. Follow-up work adapts or varies the recursive core (Roye-Azar et al. 2026; McGovern 2025; Wang and Reid 2026). Beyond latent-state refinement, ARChitects (Franzen, Disselhoff, and Hartmann 2025a) realizes the same iterative principle through token-level masked diffusion over serialized grids. Across these models the refinement loop is supervised only by the final answer, leaving intermediate iterations without an explicit semantic target. Vision-native ARC. ViTARC (Li et al. 2025a) and VARC (Hu et al. 2025) show that a Vision Transformer (ViT) applied directly to the grid can match text-based LLM solvers. Other recent work explores reasoning itself as a visual modality (Liu and Shang 2026; Zhang et al. 2025). LoopViT (Shu et al. 2026) unifies recurrent refinement with the vision-native setting and is the architecture our method builds on. Concurrent with our work, Loop-OWM (Gao, Schölkopf, and Geiger 2026) recasts the loop as an object-centric world model in which color-prototype slots decompose the grid and a demonstration-conditioned transition model rolls the state forward. As with the looped reasoners above, its supervision anchors only at the final state, without explicit semantic targets for intermediate iterations. Method A looped visual reasoner processes the input over multiple iterations before producing the final output. Under final-state supervision the objective specifies only the endpoint but leaves the trajectory, i.e. the sequence of intermediate states from input to output, unconstrained. We therefore supervise the trajectory, aligning each intermediate state with corresponding intermediate targets. We propose three components to realize this principle (Figure 2): transformation chains define what each iteration should produce; a task reference and object workspace externalize the memory role so that intermediate states can be freely supervised; and soft trace alignment enforces the ordering of the chain while letting the model decide how to allocate its iterations. Figure 2: Overview of the proposed method. Top: the few-shot demonstrations are encoded into a compact task-reference prefix G (left) via cross-attention from learned queries Q to the joint demonstration context X; a looped visual reasoner (center) iterates a shared core F for N steps. Iteration t receives the static reference G and the dynamic slot bank St−1S_t-1; the object workspace (right) extracts StS_t from the updated state for the next iteration when t<Nt<N. Bottom: soft trace alignment computes a cost matrix Ct,kC_t,k between the loop’s decoded predictions and the transformation chain targets. A soft-min DP marginalizes over monotonic paths, supervising the order but not the pacing of intermediate states; both the pairwise costs and final-state loss are change-weighted. The indicator c∈0,1c∈\0,1\ marks whether an instance is traced. Preliminaries: Task Formulation and Backbone An ARC task contains m demonstration pairs (xjd,yjd)j=1m\(x^d_j,y^d_j)\_j=1^m and one or more test inputs. Each grid has at most 30×3030×30 cells drawn from ten colors. Because the test inputs are answered independently, we write one input–target pair as (x,y)(x,y) without loss of generality. We adopt LoopViT (Shu et al. 2026), a looped visual reasoner, as our backbone and extend it with grounded trace supervision. Its encoder E maps the test input and a learned task token τ to an initial state h0h_0. A shared core F then iterates N times, producing a sequence of hidden states ht=F(ht−1+et)h_t=F(h_t-1+e_t), where ete_t is a learned step embedding. A decoder maps each state to a color distribution over every grid cell, pt=D(ht)p_t=D(h_t). The conventional objective supervises only the last prediction, ℒend=CE(pN,y)L_end=CE(p_N,y). Thus, although the decoder can produce a prediction at every iteration, only the final one receives a training signal. To supervise the intermediate predictions as well, we first need targets that define what each iteration should produce. In the base model, the loop is conditioned only on the test input and a task token supplied at initialization. Test-time training can encode the demonstrations indirectly through weight updates, but they remain absent from individual iterations as explicit inputs. Constructing Transformation Chains Trace supervision requires intermediate grids at training scale, but existing resources do not directly provide suitable targets. Human solving traces are scarce (Kim et al. 2026; LeGris et al. 2025), and language rationales describe the rule verbally without producing grid-state targets that can supervise intermediate iterations (Lehmann et al. 2026). Programmatic task implementations offer a scalable alternative because they implicitly encode how the output is built. RE-ARC (Hodel 2024a) supplies generators and deterministic verifiers for all 400 ARC-AGI-1 training tasks, while ARC-GEN (Moffitt 2025) provides generators for all ARC-AGI-1 tasks and 500 ARC-AGI-2 tasks. However, running these programs directly does not yield clean intermediate steps: programs may perform several changes at once or skip meaningful intermediate states entirely. We therefore use GPT-5.5 with xhigh reasoning effort to decompose these implementations into single-action steps. For a traced instance, executing the decomposed program yields x=T0→T1→⋯→TK=y,x=T_0\;→\;T_1\;→\;·s\;→\;T_K=y, (1) where K=|steps|≥1K=|steps|≥ 1 and the final recorded stage is the output. A chain is semantically monotonic when each step performs exactly one action toward the answer, without backtracking or exploration. Here monotonic refers to the progression of actions, not to pixel-level similarity to y. A single indivisible action yields K=1K=1. Instances for which no valid chain can be produced remain untraced. We validate each rewrite in two stages. Automated checks verify that the decomposed program reproduces the original input–output pairs exactly and that every intermediate grid is valid. We then manually review each rewritten program by visualizing sampled chains to ensure every step corresponds to a single meaningful action, and iterate with GPT-5.5 until all checks pass. Correctness alone is insufficient if the rewritten generators collapse the diversity of the training distribution. On the ARC-AGI-1 tasks shared by ARC-GEN and RE-ARC, for example, the median number of distinct output shapes per task is 16 for ARC-GEN and 216 for RE-ARC. The same problem affects ARC-AGI-2. To increase diversity, we broaden size and object-count ranges in the ARC-GEN generators and apply a semantics-preserving color permutation to each chain. Because colors can themselves encode a rule, we manually adjudicate, for each task, whether recoloring preserves its semantics. Examples where recoloring changes the rule are shown in Appendix A.6. For ARC-AGI-1, we annotate RE-ARC’s 400,000 pairs across all 400 training tasks with transformation chains, leaving inputs and outputs unchanged; 270,854 of them (67.7%) are traced. For ARC-AGI-2, we curate a corpus of 877,962 instances across 891 tasks, combining 500 ARC-GEN tasks with 391 tasks also covered by RE-ARC. Full construction and verification details are in Appendix A. Grounding the Loop: Task Reference and Object Workspace Trace supervision requires intermediate states to evolve toward chain targets, but in a looped architecture the same hidden state also serves as the model’s memory of the task. Supervising it toward specific targets therefore risks disrupting stored input information. This concern is further motivated by the base design, where the task rule is captured only indirectly through test-time weight updates (Hu et al. 2025; Shu et al. 2026) and object structure is encoded only implicitly in the patch sequence. We mitigate this by externalizing the memory role into two dedicated structures: a static task reference encoded from the demonstrations and a dynamic object workspace extracted from the evolving recurrent state. Task reference. We process every demonstration pair (xjd,yjd)\(x^d_j,y^d_j)\ through the same encoder E used for the test grid. We concatenate their non-padding patch features into a joint context X, tagging each patch with three additive embeddings: a role embedding indicating whether it comes from an input or output grid, a demonstration-index embedding identifying which pair j it belongs to, and a two-dimensional sinusoidal positional embedding. This joint representation allows cross-attention to compare features across all pairs simultaneously, rather than encoding each pair in isolation. We encode the joint context X into 128 reference tokens using learned queries Q=[Qa;Qf]Q=[Q_a;Q_f]. The 64 queries in QaQ_a are initialized from an 8×88×8 grid of two-dimensional positional anchors, encouraging spatial coverage; the remaining 64 queries are unconstrained and can capture nonlocal or task-specific relations. The queries are refined over two rounds, each applying cross-attention into X, self-attention, and a feed-forward network. The resulting tokens form the reference G, enabling each patch token to selectively attend to relevant demonstration evidence during the loop. G is computed once and re-injected before every loop iteration. Object workspace. Many ARC rules select, count, move, or recolor objects (Xu et al. 2024; Ferré 2024), whereas patch tokens encode such groupings only implicitly. Slot Abstractors (Mondal, Cohen, and Webb 2024) show that decomposing images into object-centric slots benefits abstract visual reasoning. Following this principle, we use Slot Attention (Locatello et al. 2020) to obtain an explicit object-centric summary of an ARC grid. By competing for patch features, the slots iteratively decompose the scene without assuming a fixed object segmentation. Let ϕφ denote this slot update, SinitS_init its learned initial queries, and WsW_s a linear projection to the backbone dimension. The initial workspace is extracted before the loop, with each subsequent workspace initialized from its predecessor: S0=ϕ(h0patch;Sinit),ht=F(ht−1+et+PGG+PSWsSt−1),St=ϕ(htpatch;St−1),t<N. array[]rclS_0&=&φ(h_0^patch;S_init),\\ h_t&=&F (h_t-1+e_t+P_GG+P_SW_sS_t-1 ),\\ S_t&=&φ(h_t^patch;S_t-1), t<N. array (2) Here ete_t is the step embedding of the base loop, and PGP_G and PSP_S write the two grounding sources into their reserved prefix positions. We train ϕφ with the straight-through BO-QSA update (Jia, Liu, and Huang 2023). Initializing each update from St−1S_t-1 encourages consistent slot assignments across iterations, though slots need not maintain a fixed object identity. The two grounding structures have complementary roles: G supplies a fixed description of the rule, while StS_t summarizes the evolving objects. Together they make the task rule and evolving scene available at every iteration, allowing each chain state to be interpreted in context. The remaining question is how to align chain steps with loop iterations. Supervising the Loop: Soft Trace Alignment Given a transformation chain, the most direct supervision strategy assigns milestones to iterations at fixed intervals. However, some steps need more computation than others, so a fixed assignment may be suboptimal. Instead of fixing the assignment, we require only that iterations follow the chain in order and that the final iteration matches the output. For a traced instance, let pt=D(ht)p_t=D(h_t) be the distribution decoded at iteration t and Ωk _k the valid cells of chain state TkT_k in Eq. 1. We construct a pairwise cost by averaging cross-entropy over the valid cells of each milestone: Ct,k=−1|Ωk|∑i∈Ωklogpt(i,Tk(i)).C_t,k=- 1| _k| _i∈ _k p_t (i,T_k(i) ). (3) Collecting these scores over all iterations and milestones gives the pairwise cost matrix C. An alignment path is a sequence π=(π0,…,πN)π=( _0,…, _N) with π0=0 _0=0, 0≤πt−1≤πt≤K0≤ _t-1≤ _t≤ K, and πN=K _N=K. The path can stay at the same step but never go backward. We focus on the K≤NK≤ N case, where the loop has enough iterations to visit every milestone in order and jumps larger than one are disallowed; the K>NK>N regime, which requires skip penalties, is detailed in Appendix B. Selecting only the cheapest path would lock in a single assignment too early in training. For path π, define A(π;C)=∑t=1NCt,πt.A(π;C)= _t=1^NC_t, _t. (4) We marginalize all admissible paths ΠN,K _N,K with a temperature-γ free energy, Fγ(C)=−γlog∑π∈ΠN,Kexp(−A(π;C)/γ).F_γ(C)=-γ \!\! _π∈ _N,K\!\! \! (-A(π;C)/γ ). (5) A smaller γ sharpens the objective toward the single best (Viterbi) path, while a larger γ averages over more assignments. This construction combines the latent monotone alignment used by CTC (Graves et al. 2006) with the differentiable soft-min dynamic programming of Soft-DTW (Cuturi and Blondel 2017), specialized here to ordered grid states and a fixed final state. Writing softminγsj=−γlog∑je−sj/γsoftmin_γ\s_j\=-γ _je^-s_j/γ, the free energy is computed exactly by Vt(k)=Ct,k+softminγVt−1(k−1),Vt−1(k),V_t(k)=C_t,k+softmin_γ \V_t-1(k-1),\,V_t-1(k) \, (6) with V0(0)=0V_0(0)=0, V0(k>0)=+∞V_0(k>0)=+∞, Vt(−1)≡+∞V_t(-1)≡+∞, and Fγ(C)=VN(K)F_γ(C)=V_N(K). Differentiation gives the posterior occupancy qt,k=∂Fγ/∂Ct,k=Pr(πt=k∣C)q_t,k=∂ F_γ/∂ C_t,k= ( _t=k C), so each decoded state receives a posterior-weighted mixture of milestone losses rather than a fixed target assignment. Because the number and cost of admissible paths vary with trajectory length, we subtract the cost under a zero matrix (with identical transition rules) and normalize by N: ℒalign=1N(Fγ(C)−Fγ()).L_align= 1N (F_γ(C)-F_γ(0) ). (7) Let ℒoutL_out denote the mean cross-entropy between the final decoded distribution pNp_N and the ground-truth output y, averaged over valid cells. With c∈0,1c∈\0,1\ marking whether an instance is traced, the training objective is ℒ=λoutℒout+βecℒalign,L= _outL_out+ _e\,c\,L_align, (8) where βe _e is a warmup schedule for the alignment weight. For untraced examples (c=0c=0), only the final-state term applies. The cost Ct,kC_t,k weighs every valid cell equally, but the steps it supervises are far from uniform: the median step in our corpus changes only 7.8% of its cells. The uniform average is therefore dominated by the unchanged majority, while the few cells that do change carry nearly all of the step’s semantic content. We therefore make every per-cell cost change-weighted: matching against milestone TkT_k, cell i receives weight wk(i)=1+α[Tk(i)≠Tk−1(i)],w0(i)=1,w_k(i)=1+α\,[\,T_k(i)≠ T_k-1(i)\,], w_0(i)=1, (9) where α≥0α≥ 0, and replace the uniform cost with Ct,k=−∑i∈Ωkwk(i)logpt(i,Tk(i))∑i∈Ωkwk(i).C_t,k=- _i∈ _kw_k(i) p_t (i,T_k(i) ) _i∈ _kw_k(i). (10) The final-state term is weighted the same way: ℒoutL_out averages its per-cell cross-entropy under w(i)=1+α[y(i)≠x(i)]w(i)=1+α\,[\,y(i)≠ x(i)\,], which takes the test input as the state preceding the output. The objective has a natural starting point: early in training, all admissible pacings contribute roughly equally, and because monotonic paths concentrate near the diagonal, the initial supervision resembles a smoothed uniform schedule before specializing per instance. Experiments Our experiments address four questions, with all controlled analyses conducted on ARC-AGI-1. First, we compare pass@2 accuracy against published ARC solvers on ARC-AGI-1 and ARC-AGI-2. Second, a factorial ablation isolates the interaction between trace supervision and grounding, while targeted variants evaluate change weighting, soft trace alignment, and the supervision schedule. Third, we visualize how intermediate predictions and object-workspace assignments evolve across iterations. Finally, we decompose the oracle–pass@2 gap into candidate coverage and selection errors to identify the dominant remaining test-time bottleneck. Experimental Setup Data and benchmarks. We evaluate on the official public benchmarks ARC-AGI-1 (Chollet 2019) and ARC-AGI-2 (Chollet et al. 2025). For ARC-AGI-1, we retain the same official tasks and RE-ARC input–output pairs used by prior vision-native solvers (Hu et al. 2025; Shu et al. 2026). The transformation chains are the only additional supervision. The source contains 400,000 RE-ARC pairs, of which 129 are removed by the standard 30×3030×30 size filter. We then add 1,718 official examples, yielding 401,589 training records. For ARC-AGI-2, the programmatic resource contributes 877,962 records over 891 of the 1,000 training tasks. We supplement it with 4,308 official examples, including examples from the 109 tasks without a programmatic source, yielding an assembled corpus of 882,270 records. Model. We train two model sizes, both using an eight-block shared core applied for N=6N=6 loop iterations: TraceViT-Medium (width 384, 11M parameters) and TraceViT-Large (width 512, 18M parameters). Architectural details of the backbone, the task-reference encoder, and the object workspace are provided in Appendix C. Training and evaluation. We train a separate model for each of the two benchmarks using the Adam optimizer (Kingma and Ba 2015) with a learning rate of 3×10−43×10^-4 for 100 epochs. Because ARC-AGI-2 is harder and its training tasks partially overlap ARC-AGI-1’s, we warm-start its model from the trained ARC-AGI-1 weights. Our evaluation follows VARC (Hu et al. 2025): each evaluation task receives 100 epochs of test-time training, with demonstration pairs augmented by flips, rotations, and color permutations. At inference, predictions from 510 augmented views of the test input are de-augmented and aggregated by exact-match majority voting (Akyürek et al. 2025), and the two most-voted grids form the submissions scored by the official pass@2 metric. Remaining hyperparameters and training details are in Appendix D. All experiments run on a single node with 8 NVIDIA A100 80GB GPUs. System Params ARC-1 ↑ ARC-2 ↑ [gray]0.92large language models GLM-5.2 744B 77.0 22.8 GPT-5.6 Sol (Max) n/a 96.5 92.5 GPT-5.4 Mini (xHigh) n/a 63.7 18.9 Gemini 3.5 Flash (High) n/a 84.7 72.1 Grok 4 (Thinking) 1.7T 66.7 16.0 [gray]0.92recurrent models HRM 27M 40.3 5.0 TRM 7M 44.6 7.8 [gray]0.92vision models VARC 18M 54.5 8.3 VARC (ensemble) 73M 60.4 11.1 LoopViT (Medium) 11M 63.8 11.5 LoopViT (Large) 18M 65.8 14.2 TraceViT (Medium) 11M 65.5 13.3 TraceViT (Large) 18M 67.8 24.3 Table 1: Comparison on ARC-AGI-1 and ARC-AGI-2 evaluation sets. LLM results are quoted from the ARC-AGI leaderboard (ARC Prize Foundation 2025); compact-solver results are quoted from their respective publications. Comparison with Prior ARC Solvers Table 1 shows that both TraceViT variants outperform prior compact solvers at matched parameter counts on ARC-AGI-1 and ARC-AGI-2. ARC-AGI-2 is substantially harder: every compact solver, ours included, sees a sharp accuracy drop. The scale of the drop is consistent with the benchmark’s design: its tasks compose multiple interacting rules across multiple steps and define symbol meanings within each task (Chollet et al. 2025). A single N=6N=6-iteration pass must realize all of these jointly, and exact-match scoring credits no partial composition. Compact visual reasoning on ARC-AGI-2 therefore remains an open problem. Ablation Studies Grounding structures Trace supervision absent present none (final state only) 62.3 63.9 chain milestones 61.6 65.5 [gray]0.92variants of the full model uniform alignment cost (α=0α=0) 64.3 fixed-interval alignment 64.4 late-stage trace relaxation 63.6 Table 2: Ablation results (pass@2 %, TraceViT-Medium, ARC-AGI-1). The 2×22×2 block varies two factors: whether milestone supervision (ℒalignL_align) is added to the final-state loss (ℒoutL_out), and whether task-reference and object-workspace grounding is enabled. The three additional rows each change one design choice: uniform cost sets α=0α=0 in Eq. 9, fixed-interval alignment replaces soft alignment, and late-stage trace relaxation sets βe=0 _e=0 after epoch 40. Table 2 disentangles the contributions of trace supervision and grounding. The 2×22×2 block crosses two factors: whether training uses only the final-state term ℒoutL_out or adds the milestone-alignment term ℒalignL_align, and whether the task reference and object workspace are jointly enabled. Three additional variants each modify one aspect of the full model: setting α=0α=0 in the milestone cost, replacing soft alignment with a fixed-interval schedule, or setting βe=0 _e=0 after epoch 40. All seven configurations use TraceViT-Medium and the same ARC-AGI-1 training and evaluation protocol. Trace–grounding interaction. The 2×22×2 block reveals a clear asymmetry: trace supervision degrades accuracy without grounding but becomes beneficial once grounding is present. Starting from the ungrounded, final-state-only baseline (62.3%), adding milestone supervision alone lowers pass@2 to 61.6%. Adding the task-reference encoder and Slot Attention workspace alone raises it to 63.9%, consistent with the expected benefit of re-supplying the task rule and scene at every iteration. Combining both yields 65.5%, a substantial gain over either single-factor variant. We hypothesize that this interaction arises because the recurrent state in LoopViT serves both as memory of the input and as the computation workspace. Under milestone supervision, intermediate states are pushed toward partial outputs, which may disrupt the input information that subsequent iterations depend on to complete the transformation. The task reference and object workspace externalize this memory, making the rule and scene available at every iteration so that the recurrent state need not carry them. Once memory is decoupled from prediction, trace supervision guides the transformation without starving later iterations of input information. Data control. The grounding-only and full-model configurations use exactly the same input–output pairs, differing only in whether the chain milestones enter the trace loss. The 1.6-point gap therefore estimates trace supervision’s contribution with training-set size and diversity held constant. Change weighting. Setting α=0α=0 in Eq. 9 restores the uniform milestone cost of Eq. 3 while retaining all other components. Pass@2 drops from 65.5% to 64.3%, indicating that without change weighting the unchanged majority of cells dominates the alignment cost. Alignment rule. Replacing soft alignment with a fixed-interval schedule while keeping the milestones and grounding unchanged lowers pass@2 from 65.5% to 64.4%, confirming the benefit of letting the model decide how to allocate iterations to milestones. Soft alignment also remains applicable when K exceeds the iteration budget, whereas the fixed schedule must omit milestones. Trace-supervision schedule. Setting βe=0 _e=0 after epoch 40 removes the milestone-level constraint for the remainder of training. Pass@2 drops from 65.5% to 63.6%, indicating that trace supervision is not merely an early optimization scaffold but remains beneficial when maintained throughout. Process Visualization Figure 3: Per-step predictions and slot assignments on three tasks. Prediction rows show the input and six loop outputs. Assignment rows show the argmax over eight slots at each step. Evolution of intermediate predictions. Figure 3 shows per-step predictions and slot assignments on three tasks. The model builds its answer incrementally across iterations rather than producing it in one step. In the top example, three colored boxes must each be filled with a pattern determined by a single seed cell in their interior. The model solves this in two visible phases: it first clears the box interiors (steps 1–2), then draws each box’s pattern (steps 3–5). The middle and bottom examples show similar progressive construction, with each iteration refining a different aspect of the output. Evolution of workspace organization. Across all tasks we inspected, two slots play fixed roles: one covers the empty canvas outside the grid and another covers the grid background. The remaining slots carve out task-specific regions that correspond to where the transformation happens. In the top example, the clearing phase shows a coarse spatial bipartition across slots. When the model switches to drawing, one slot expands to cover both box interiors while the others narrow down to box boundaries and pattern-row segments. This suggests that the slots shift from coarse spatial roles in the clearing phase to fine-grained roles once drawing begins. Similar phase-aligned slot transitions appear in the other two examples; see Appendix E for additional visualizations. Error Analysis Coverage and selection errors. For each ARC-AGI-1 evaluation task, we merge identical predictions across augmented views and rank the resulting candidates by the vote count used for submission. Let r denote the rank of the correct grid, with r=∞r=∞ when it is absent. Pass@2 succeeds when r≤2r≤ 2, whereas oracle accuracy succeeds whenever r<∞r<∞. Thus, 2<r<∞2<r<∞ represents a selection gap, while r=∞r=∞ represents a coverage error. pass@2 oracle only total Rank r 1–2 3–10 >>10 absent Oracle acc. Tasks (%) 67.8 5.0 3.5 23.8 76.3 Table 3: Rank of the correct grid in TraceViT-Large’s vote-ranked candidate pool on 400 ARC-AGI-1 tasks. Finite ranks above 2 form the selection gap, whereas “absent” denotes a coverage error. Oracle accuracy sums all finite-rank bins; percentages are rounded. Coverage dominates. As shown in Table 3, 67.8% of tasks place the correct grid in the top two, while another 8.5% generate it at lower ranks, raising oracle accuracy to 76.3%. The correct grid is absent for the remaining 23.8%, so coverage accounts for nearly three times as much residual error as selection, and reranking can recover at most the 8.5-point selection gap. The asymmetry is structural: each view is decoded deterministically, so voting can aggregate correct predictions but never generate new ones. When the correct grid is absent from all 510 views, this reflects a competence gap, not a sampling one. Where the solver does succeed, voting already places the correct grid in the top two for most oracle-accessible tasks (67.8 of 76.3 points). Improving coverage must therefore come from the per-view solver itself, consistent with the single-view bottleneck reported by VARC (Hu et al. 2025). Conclusion Looped visual reasoners produce intermediate predictions at every iteration, yet endpoint-only training imposes no structure on them. We introduced TraceViT to supervise these predictions with explicit intermediate targets: verified, semantically monotonic transformation chains define the milestones; a task reference and object workspace ground every iteration; and soft trace alignment enforces milestone order without fixing the model’s pace. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 reveal a clear interaction: trace supervision alone slightly degrades accuracy, yet combining it with grounding yields a substantial gain over either component in isolation. This confirms that supervising intermediate states requires dedicated structures to preserve input information throughout the loop. An error decomposition further shows that candidate coverage, not selection, is the dominant remaining bottleneck. Grounded trace supervision thus offers a principled way to leverage the intermediate predictions that looped architectures already produce. The current approach assumes that reliable chains can be obtained from programmatic task implementations. Extending grounded trace supervision to tasks where intermediate states must be inferred, such as through learned decomposition or human demonstrations, is an important open direction. More broadly, the principle of supervising iterative computation with structured intermediate targets may generalize to other domains where models refine predictions over multiple steps. References Akyürek et al. (2025) Akyürek, E.; Damani, M.; Zweiger, A.; Qiu, L.; Guo, H.; Pari, J.; Kim, Y.; and Andreas, J. 2025. The Surprising Effectiveness of Test-Time Training for Few-Shot Learning. In Proceedings of the Forty-Second International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. PMLR. Alford et al. (2021) Alford, S.; Gandhi, A.; Rangamani, A.; Banburski, A.; Wang, T.; Dandekar, S.; Chin, J.; Poggio, T.; and Chin, P. 2021. Neural-guided, Bidirectional Program Search for Abstraction and Reasoning. In Complex Networks & Their Applications X, 657–668. Springer. ARC Prize Foundation (2025) ARC Prize Foundation. 2025. ARC-AGI Benchmarking: Leaderboard for the ARC-AGI Benchmark. https://arcprize.org/leaderboard. Accessed: 2026-07-25. Chollet (2019) Chollet, F. 2019. On the Measure of Intelligence. arXiv:1911.01547. Chollet et al. (2025) Chollet, F.; Knoop, M.; Kamradt, G.; Landers, B.; and Pinkard, H. 2025. ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems. arXiv:2505.11831. Cole and Osman (2025) Cole, J.; and Osman, M. 2025. Don’t throw the baby out with the bathwater: How and why deep learning for ARC. arXiv:2506.14276. Cuturi and Blondel (2017) Cuturi, M.; and Blondel, M. 2017. Soft-DTW: A Differentiable Loss Function for Time-Series. In Proceedings of the Thirty-Fourth International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, 894–903. PMLR. Ferré (2024) Ferré, S. 2024. Tackling the Abstraction and Reasoning Corpus (ARC) with Object-centric Models and the MDL Principle. In Advances in Intelligent Data Analysis XXII, volume 14641 of Lecture Notes in Computer Science, 3–15. Springer. Franzen, Disselhoff, and Hartmann (2025a) Franzen, D.; Disselhoff, J.; and Hartmann, D. 2025a. The ARChitects - Technical Report: ARC Prize 2025 Solution Summary. https://lambdalabsml.github.io/ARC2025_Solution_by_the_ARChitects/. Accessed: 2026-06-17. Franzen, Disselhoff, and Hartmann (2025b) Franzen, D.; Disselhoff, J.; and Hartmann, D. 2025b. Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective. In Proceedings of the Forty-Second International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. PMLR. Gao, Schölkopf, and Geiger (2026) Gao, G.; Schölkopf, B.; and Geiger, A. 2026. Slots, Transitions, Loops: Learning Composable World Models for ARC. arXiv:2606.12316. Graves et al. (2006) Graves, A.; Fernández, S.; Gomez, F.; and Schmidhuber, J. 2006. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. In Proceedings of the Twenty-Third International Conference on Machine Learning, volume 148 of ACM International Conference Proceeding Series, 369–376. ACM. Greenblatt (2024) Greenblatt, R. 2024. Getting 50% (SoTA) on ARC-AGI with GPT-4o. https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt. Accessed: 2026-06-17. Hodel (2024a) Hodel, M. 2024a. Addressing the Abstraction and Reasoning Corpus via Procedural Example Generation. arXiv:2404.07353. Hodel (2024b) Hodel, M. 2024b. Domain Specific Language for the Abstraction and Reasoning Corpus. https://github.com/michaelhodel/arc-dsl. Accessed: 2026-06-17. Hu et al. (2025) Hu, K.; Cy, A.; Qiu, L.; Ding, X. D.; Wang, R.; Zhu, Y. E.; Andreas, J.; and He, K. 2025. ARC Is a Vision Problem! arXiv:2511.14761. Jia, Liu, and Huang (2023) Jia, B.; Liu, Y.; and Huang, S. 2023. Improving Object-centric Learning with Query Optimization. In Proceedings of the Eleventh International Conference on Learning Representations. Jolicoeur-Martineau (2025) Jolicoeur-Martineau, A. 2025. Less is More: Recursive Reasoning with Tiny Networks. arXiv:2510.04871. Kim et al. (2026) Kim, S.; Choi, H.; Lee, S.; and Kim, S. 2026. ARCTraj: A Dataset and Benchmark of Human Reasoning Trajectories for Abstract Problem Solving. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2687–2698. New York, NY, USA: ACM. Kingma and Ba (2015) Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the Third International Conference on Learning Representations. LeGris et al. (2025) LeGris, S.; Vong, W. K.; Lake, B. M.; and Gureckis, T. M. 2025. A Comprehensive Behavioral Dataset for the Abstraction and Reasoning Corpus. Scientific Data, 12(1): 1380. Lehmann et al. (2026) Lehmann, J.; Khushbakht, S.; Salehfard, N.; Nishat, N. A. Z.; Bhandiwad, D.; Aioanei, A.; and Vahdati, S. 2026. ARC-TGI: Human-Validated Task Generators with Reasoning Chain Templates for ARC-AGI. arXiv:2603.05099. Lei, Lipovetzky, and Ehinger (2024) Lei, C.; Lipovetzky, N.; and Ehinger, K. A. 2024. Generalized planning for the abstraction and reasoning corpus. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, volume 38, 20168–20175. Li et al. (2025a) Li, W.; Xu, Y.; Sanner, S.; and Khalil, E. B. 2025a. Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects. Transactions on Machine Learning Research, 2025. Li et al. (2025b) Li, W.-D.; Hu, K.; Larsen, C.; Wu, Y.; Alford, S.; Woo, C.; Dunn, S.; Tang, H.; Zheng, W.-L.; Pu, Y.; et al. 2025b. Combining Induction and Transduction for Abstract Reasoning. In Proceedings of the Thirteenth International Conference on Learning Representations. Liu and Shang (2026) Liu, Z.; and Shang, Y. 2026. Reasoning is a Modality. arXiv:2601.13562. Locatello et al. (2020) Locatello, F.; Weissenborn, D.; Unterthiner, T.; Mahendran, A.; Heigold, G.; Uszkoreit, J.; Dosovitskiy, A.; and Kipf, T. 2020. Object-Centric Learning with Slot Attention. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 11525–11538. New York, NY, USA: Curran Associates, Inc. McGovern (2025) McGovern, R. K. 2025. Test-time Adaptation of Tiny Recursive Models. arXiv:2511.02886. Moffitt (2025) Moffitt, M. D. 2025. ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus. arXiv:2511.00162. Mondal, Cohen, and Webb (2024) Mondal, S. S.; Cohen, J. D.; and Webb, T. W. 2024. Slot Abstractors: Toward Scalable Abstract Visual Reasoning. In Proceedings of the Forty-First International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, 36088–36105. PMLR. Ouellette (2024) Ouellette, S. 2024. Towards Efficient Neurally-Guided Program Induction for ARC-AGI. arXiv:2411.17708. Pourcel, Colas, and Oudeyer (2025) Pourcel, J.; Colas, C.; and Oudeyer, P. 2025. Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI. In Proceedings of the Forty-Second International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. PMLR. Roye-Azar et al. (2026) Roye-Azar, A.; Vargas-Naranjo, S.; Ghai, D.; Balamurugan, N.; and Amir, R. 2026. Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute. arXiv:2512.11847. Shu et al. (2026) Shu, W.-J.; Qiu, X.; Zhu, R.-J.; Chen, H. H.; Liu, Y.; and Yang, H. 2026. LoopViT: Scaling Visual ARC with Looped Transformers. arXiv:2602.02156. Singhal and Shroff (2025) Singhal, K.; and Shroff, G. 2025. ConceptSearch: Towards Efficient Program Search Using LLMs for Abstraction and Reasoning Corpus (ARC). In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence, volume 39, 20506–20513. AAAI Press. Sorokin and Puget (2025) Sorokin, I.; and Puget, J.-F. 2025. NVARC solution to ARC-AGI-2 2025. https://github.com/1ytic/NVARC. Accessed: 2026-06-17. Strandgaard (2026) Strandgaard, S. 2026. ARC-Interactive-History-Dataset. https://github.com/neoneye/ARC-Interactive-History-Dataset. Accessed: 2026-06-17. Wang et al. (2025) Wang, G.; Li, J.; Sun, Y.; Chen, X.; Liu, C.; Wu, Y.; Lu, M.; Song, S.; and Yadkori, Y. A. 2025. Hierarchical Reasoning Model. arXiv:2506.21734. Wang and Reid (2026) Wang, W.; and Reid, F. 2026. Tiny Recursive Reasoning with Mamba-2 Attention Hybrid. arXiv:2602.12078. Wind (2020) Wind, J. S. 2020. DSL solution to the ARC challenge. https://github.com/top-quarks/ARC-solution. Accessed: 2026-06-17. Xu, Khalil, and Sanner (2023) Xu, Y.; Khalil, E. B.; and Sanner, S. 2023. Graphs, constraints, and search for the abstraction and reasoning corpus. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence, volume 37, 4115–4122. Menlo Park, Calif.: AAAI Press. Xu et al. (2024) Xu, Y.; Li, W.; Vaezipoor, P.; Sanner, S.; and Khalil, E. B. 2024. LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations. Transactions on Machine Learning Research, 2024. Zhang et al. (2025) Zhang, B.; Zang, Y.; Dong, X.; Cao, Y.; Duan, H.; Lin, D.; and Wang, J. 2025. Think Visually, Reason Textually: Vision-Language Synergy in ARC. arXiv:2511.15703. Appendix Appendix A Chain-Corpus Construction Details This section expands the chain-corpus construction pipeline summarized in the Method section of the main paper. We first describe how the ARC-GEN generators and RE-ARC verifiers are rewritten to expose semantically meaningful intermediate states, followed by automated checks and human review that preserve functional fidelity and step coherence. We then detail constrained recoloring and parameter widening for greater diversity, before presenting the deterministic corpus assembly and final audit. A.1 Source Programs Our corpus builds on two complementary program collections. RE-ARC (Hodel 2024a) pairs each of the 400 ARC-AGI-1 training tasks with a generator that samples fresh instances and a verifier that deterministically derives the output from the input. The released collection contains 1,000 sampled pairs per task and spans diverse sizes, palettes, and layouts, but provides no process supervision and does not guarantee reproduction of the official examples. ARC-GEN (Moffitt 2025) provides generators for 900 tasks: all 400 ARC-AGI-1 training tasks and 500 ARC-AGI-2 training tasks. Under its fidelity contract, each generator reproduces its task’s official examples cell for cell when invoked with a fixed parameter setting. The two collections encode solution structure in different components. ARC-GEN’s generators follow “construct the puzzle, then the output,” whereas RE-ARC’s verifiers follow “derive the output from the puzzle.” We rewrite only these components. Both ordinarily emit the final output in one shot, without exposing intermediate states. Importantly, we never modify RE-ARC’s generators or its 400,000 released pairs. A.2 Rewriting Principles We use GPT-5.5 with xhigh reasoning effort to produce every rewrite. For each task, the model receives the source program, a description of the task rule, and the fixed instructions below. Later batches additionally receive reusable instructions distilled from recurring faults in earlier reviews. • Distribution invariant. The random sampling logic and public interface remain unchanged; only the construction of the output may be reorganized. • Input first, output progressively. The input is built by the original logic; the output starts as a blank grid (or a copy of the input) and is constructed progressively. • One transition, one nameable action. Each successive frame must result from a single human-nameable action (“outline every red frame,” “move the k-th object to the bottom”). We judge the semantic operation rather than the number of changed pixels: placing a one-cell marker may be a valid step, whereas a no-op initial frame is not. • Per-object unrolling with capacity guards. When the object count is sampled, repeated phases unroll one object at a time. Any widened range must be covered by explicit capacity guards; otherwise, overflow can silently drop objects (see Parameter Widening below). • Frame budget. Typical chains span two to nine frames including the final output. A one-frame chain is permitted only for a genuinely atomic transformation and is flagged for review. • No fabricated traces. Tasks without a meaningful transformation chain remain untraced rather than being assigned artificial frames. A traced, one-frame chain records one meaningful atomic action with K=1K=1; an untraced sample receives only direct endpoint supervision. A.3 Automated Verification Gates The rewrite criteria define the desired chain structure; six automated gates then test functional fidelity and equivalence. Any failure returns the rewrite to the model for correction. 1. Fidelity. An ARC-GEN rewrite reproduces its task’s official examples cell for cell at the contract parameters. A RE-ARC rewrite recomputes all 1,000 released pairs for its task; across 400 tasks, this covers 400,000 pairs with no mismatches or execution errors. 2. Old–new equivalence. Across hundreds of seeds, the rewritten and original programs produce identical outputs and identical error and rejection behavior. This tests that the rewrite reorganizes the existing computation rather than changing the data distribution. 3. Structural invariants. Every recorded frame is a rectangular grid of colors 0–9 with dimensions at most 30×3030× 30. For traced examples, the final frame equals the output; untraced examples carry no process targets. 4. Frame-count monitoring. We track the frame-count distribution of every task. Fragmentation (abnormally many frames) or collapse (concentration on a single frame) triggers review. 5. Degeneracy and solvability. We flag constant outputs, identity outputs, and input collisions. An input collision makes the generated relation inconsistent because the same input is paired with two different outputs. Tasks whose rules genuinely permit identity or constant outputs receive explicit exemptions. 6. Independent re-verification. We do not rely on the model’s self-reported test results. Instead, we re-run every gate and fingerprint the codebase between batches to detect out-of-scope edits and silent rollbacks. A.4 Human Review and Iterative Correction Automated gates establish functional correctness but cannot determine whether the intermediate states form a semantically coherent transformation chain. We therefore render every gate-passing rewrite as an input→\,→\,steps→\,→\,output filmstrip, one page per task. To avoid clusters of nearly identical samples, we sample broadly across the variant space. Review asks three questions: should adjacent frames be merged because they fragment a single action; should any transition be split because it fuses several actions; and can each transition be named as a solution action so that the full chain forms a coherent narrative? We judge the rendered frames rather than the model’s accompanying explanations, which can disagree with the program’s actual behavior. Because no automated gate can assess step semantics, human review is the final adjudicator. We record task-specific comments and return them to the model with the current program; every revised program re-enters the full verification battery. Recurring faults become reusable instructions for later batches. For example, per-object phases are the preferred granularity; counting tasks clear distractors before producing the count; long paths are segmented at crossing events; trailing one- or two-cell marks join the final phase rather than forming separate steps; and degenerate puzzles are excluded from sampling when the official examples always contain the triggering elements. Most batches converged after one correction round, with roughly a dozen fixes per 50-task batch. A.5 Diversity: Constrained Recoloring Functional correctness does not by itself guarantee distributional diversity. Among tasks covered by both collections, the median number of distinct output grid shapes per task is 16 for ARC-GEN and 216 for RE-ARC; the corresponding medians for distinct palettes are 14 and 162. ARC-GEN is likewise less diverse in object counts and densities. We therefore augment color diversity with full-trajectory bijections, applying the same mapping to the input, every intermediate frame, and the output while handling foreground and background colors separately. Not every color permutation preserves task semantics. Specific colors may encode the rule, as in a fixed color-to-shape legend, a gray axis, or a black corridor. Remapping such colors can produce an internally consistent trajectory that is nevertheless invalid under the original task rule. We therefore impose task-specific recoloring constraints. Across the 500 ARC-AGI-2 tasks, 80 disable recoloring entirely, whereas 324 freeze at least one load-bearing color: the color is neither replaced nor used as the target of another color. Background constraints are tracked separately. The black background is locked in 45 tasks because it carries content semantics, while 113 tasks have non-black backgrounds that may be recolored. Automated probes help propose these constraints but err in both directions. Treating every color mentioned by a program as fixed is overly restrictive, whereas testing only whether a remapped trajectory is internally consistent misses load-bearing colors. We therefore manually adjudicate the recoloring policy for every task. As review progressed, we refined the background criterion: black is locked only when it carries content semantics in the input. A.6 Recoloring Failure Examples Figure 4 provides three examples. In each case, applying one permutation consistently to the input, intermediate states, and output preserves the appearance of a valid trajectory but changes a color-dependent rule established by the official examples. Figure 4: Failure cases for trajectory-wide recoloring. Official pairs establish the task semantics. Records labeled identity or production preserve those semantics, whereas each improper recolor applies a consistent palette mapping across the full trajectory but changes the rule encoded by the colors. A.7 Diversity: Parameter Widening Beyond recoloring, we increase content diversity by widening generator parameter spaces. We decouple height and width from shared size parameters, broaden size ranges, expose and randomize sizes previously hidden in default values, and parameterize object counts. Every widening must preserve byte-level fidelity along the default parameter path, so official examples remain unchanged and each edited program re-enters the full verification battery. The principal risk is silent object loss. Increasing one dimension can push derived quantities, such as diagonal counts or sub-block counts, beyond the original implementation’s capacity. The program may then produce a structurally valid but rule-incomplete output that no structural gate detects; explicit capacity guards are therefore mandatory. Recoloring policies and parameter-widening edits are both made on a per-task basis, proceed in the same batches, and undergo the same filmstrip review as the step rewrites. A.8 Corpus Assembly and Audit With all available programs rewritten and diversified, we synthesize the corpus in a single deterministic pass. Each task draws from a fixed seed stream, and every record carries complete variant metadata, making the corpus reproducible record by record and resumable after interruption. We deduplicate records by (x,y)(x,y), impose a timeout on each attempt, and assign each task a time budget. Tasks with heavy-tailed runtimes may therefore produce fewer records; every shortfall is recorded in the corpus ledger. Official examples serve only as fidelity targets in the construction pipeline: ARC-GEN generators must reproduce them at the contract parameters, but the examples themselves are not included in the corpus contribution. The RE-ARC portion is pure post-processing over the existing 400,000 pairs: 270,854 instances (67.7%) are traced. The resulting ARC-AGI-2 corpus contains 877,962 records across 891 of the 1,000 training tasks: 500 tasks from ARC-GEN synthesis and 391 ARC-AGI-2 tasks that overlap ARC-AGI-1 and are therefore covered by RE-ARC. The remaining 109 tasks have no programmatic source and are not represented in the contributed corpus. Among the 500 ARC-GEN tasks, 499 yield traces; the remaining task is untraced because it has no meaningful intermediate state. Finally, we audit the complete corpus. The audit re-runs all structural invariants and confirms that the training loader drops no records, reconciles per-task counts with the ledger, re-checks byte-level fidelity of RE-ARC records against their source, revalidates ARC-GEN generators against the official JSON without adding those examples to the corpus, verifies every recoloring constraint on the final records, and confirms that official-example files are not ingested as records in the contributed corpus. Appendix B Alignment When Milestones Outnumber Iterations When a traced chain has more milestones than the loop has iterations, K>NK>N, no non-decreasing path from chain state 0 to chain state K can visit every state in N steps. Skips are therefore unavoidable. We extend the alignment formulation in the main text to this regime by allowing skip transitions and assigning them an explicit penalty. We retain the main text’s endpoint-anchored paths π=(π0,…,πN)π=( _0,…, _N), with π0=0 _0=0, 0≤πt−1≤πt≤K0≤ _t-1≤ _t≤ K, and πN=K _N=K, but now allow an iteration to advance by more than one milestone. Let ΠN,K _N,K denote this enlarged path set. Each transition incurs a penalty proportional to the number of milestones it skips: Δ(π)=λskip∑t=1Nmax(πt−πt−1−1, 0), (π)= _skip _t=1^N ( _t- _t-1-1,\;0 ), (11) where λskip _skip is the cost per skipped milestone. The path energy in this regime is therefore A(π;C)=∑t=1NCt,πt+Δ(π)A(π;C)= _t=1^NC_t, _t+ (π). The first term measures the agreement between each iteration and its assigned milestone, whereas the second makes every omission explicit and costly. Including the skip penalty in each path energy gives Fγ(C)=−γlog∑π∈ΠN,Kexp(−A(π;C)/γ).F_γ(C)=-γ \! _π∈ _N,K \! (-A(π;C)/γ ). (12) Correspondingly, the dynamic-programming recursion must consider every predecessor 0≤j≤k0≤ j≤ k: Vt(k)=Ct,k+softminγ0≤j≤k[Vt−1(j)+λskipmax(k−j−1,0)],V_t(k)=C_t,k+ softmin_γ_0≤ j≤ k [V_t-1(j)+ _skip (k-j-1,0) ], (13) with V0(0)=0V_0(0)=0, V0(k>0)=+∞V_0(k>0)=+∞, and Fγ(C)=VN(K)F_γ(C)=V_N(K). The main-text recursion is the K≤NK≤ N specialization of Eq. 13. In that regime, jumps larger than one are disallowed, so πt−πt−1∈0,1 _t- _t-1∈\0,1\, every term of Δ(π) (π) vanishes, and the predecessor set reduces to k−1,k\k-1,k\. The batched implementation evaluates every 0≤j≤k0≤ j≤ k in both regimes; when skips are disallowed, transitions with j<k−1j<k-1 receive infinite cost. The normalization baseline must use the same transition rules and penalties. Consequently, Fγ()F_γ(0) reflects both path multiplicity and skip penalties rather than path multiplicity alone. The normalized alignment loss remains zero when C=C=0. We use λskip=0.3 _skip=0.3 throughout. In this regime, order-only supervision is necessary if every milestone is to remain eligible without prescribing omissions in advance. A fixed schedule must preselect which milestones to omit. By contrast, a latent monotone alignment—soft or hard—keeps every milestone eligible as a supervision target, lets the path determine which milestones to skip for each instance, and charges every omission. Allowing skips also admits the limiting case corresponding to deep supervision. The path π0=0 _0=0 and πt=K _t=K for all t≥1t≥ 1 is valid, although its first transition incurs a skip penalty. If the posterior occupancy concentrates on this path, every iteration is supervised by the final target. Appendix C Model Architecture Details This section details the two model sizes used in the main paper, TraceViT-Medium (11M) and TraceViT-Large (18M). We first describe their shared backbone and recurrent loop, then the task-reference encoder and the object workspace. Offline training Loss Test-time training (per task) Optimizer Adam Final-state weight λout _out 2.0 Initialization EMA weights of the run Learning rate 3×10−43× 10^-4, cosine Alignment weight β 0.2 (Medium), 0.3 (Large) Epochs 100 Warmup 10 epochs, linear Alignment warmup 5 epochs, linear Batch size 8 Batch size 32 per GPU (256 total) Softmin temperature γ 0.5 Learning rate 3×10−43× 10^-4, cosine Epochs 100 Skip penalty λskip _skip 0.3 Precision, clipping bf16, 1.0 Gradient clipping 1.0 Change weight α 3.0 Objective final-state CE, α=0α=0 Precision bf16 autocast Independent runs 2 Dropout 0.1 Views per test input 51×1051× 10 per run Weight averaging EMA, decay 0.9999 Seed 42 Table 4: Training and evaluation configuration. Every value is shared by the two model sizes and by the two benchmarks except β and the ARC-AGI-2 warm-start changes described in the text. C.1 Backbone and Loop Configuration Canvas and tokenization. Every grid is rendered on a fixed 64×6464× 64 canvas. A grid of at most 30×3030× 30 cells is upscaled by an integer factor using nearest-neighbor interpolation and placed at an offset within the canvas. Training and test-time inference resample the scale and offset on every pass, whereas offline evaluation and demonstration rendering use fixed placements. Positions outside the grid carry a dedicated background index. Every target canvas—the answer or an intermediate chain state—also carries a one-cell border of a second dedicated index along the grid’s right and bottom edges. This border lets the model encode the predicted grid shape directly in the canvas instead of using a separate shape predictor. The resulting vocabulary contains twelve symbols: the ten ARC colors, background, and border. The loss is evaluated only on the grid-and-border region. Within a traced instance, the input, intermediate states, and output use the same scale and offset, keeping the entire trajectory pixel-aligned. Core block. Both model sizes use the pre-norm hybrid block of LoopViT (Shu et al. 2026): x←x+MHSA(RMSNorm(x)),x←x+ConvGLU(RMSNorm(x)). array[]rclx&←&x+MHSA (RMSNorm(x) ),\\ x&←&x+ConvGLU (RMSNorm(x) ). array The ConvGLU branch uses a nominal feed-forward width of 512, giving a gated hidden width of ⌊2⋅512/3⌋=341 2· 512/3 =341. It projects each token to twice this hidden width and splits the result into a gate and a value. For image tokens, the gate is reshaped to the spatial patch grid and processed by a 3×33× 3 depthwise convolution; prefix tokens bypass this convolution. The branch then recombines gate g and value v as GELU(g)⊙vGELU(g) v before the output projection. Loop. The core is unrolled for exactly N=6N=6 iterations. Each iteration performs the following operations in order: 1. add the step embedding ete_t to the whole sequence, prefix included; 2. add the reference G at its reserved positions; 3. add the projected workspace WsSt−1W_sS_t-1 at its reserved positions; 4. apply the eight shared blocks; 5. decode the current prediction; every iteration is decoded during training because the alignment cost matrix requires all six outputs; 6. re-extract the workspace from the updated patch tokens, except after the final iteration, whose slots would never be re-injected. Under the fixed unroll, the prediction decoded at t=Nt=N is the model’s output. Sharing block parameters across iterations yields 6×8=486× 8=48 block applications per forward pass while storing the parameters of only eight blocks. Decoder. The decoder first normalizes the patch tokens and maps each one through Linear→\,→\,GELU→\,→\,Linear, producing 12⋅22=4812· 2^2=48 logits per token. De-patchification converts these values into a 12×64×6412× 64× 64 logit map. The decoder runs at every iteration. Consequently, the intermediate predictions used by the alignment loss and the final answer are produced by an identical decoding path. The two sizes. The two models differ only in embedding width: 384 for TraceViT-Medium and 512 for TraceViT-Large. Both use an eight-block core, six loop iterations, eight attention heads, and a nominal feed-forward width of 512. Their object workspaces contain eight slots, run three internal rounds, and use a 256-wide projection space. Their task-reference encoders produce 128 tokens with a 128-wide, two-round encoder over at most four demonstrations. The per-task token table is the only parameter component that does not transfer. It is discarded and re-initialized for test-time training, and the parameter counts reported in the main paper exclude it. C.2 Task-Reference Encoder The task-reference encoder converts at most four demonstration pairs into the static reference G. Each input and output grid is rendered at a canonical placement on the 64×6464× 64 canvas and processed by the backbone’s shared color, patch, and positional embeddings. After a projection to the encoder width denc=128d_enc=128, a token from demonstration j, stream s∈in,outs∈\in,out\, and patch position ℓ is represented as Xj,s,ℓ=WinHj,ℓs+tagsstream+tagjdemo+posℓ,X_j,s, =W_inH^s_j, +tag^stream_s+tag^demo_j+pos_ , (14) where the stream and demonstration tags are learned and posℓpos_ is a fixed two-dimensional sine–cosine code. Valid input and output tokens from all demonstrations are flattened into the joint context X. Reference extraction and injection. Q=[Qa;Qf]Q=[Q_a;Q_f] contains 128 queries. The 64 queries in QaQ_a are initialized with the two-dimensional sine–cosine code of an 8×88× 8 grid, while the 64 queries in QfQ_f are freely learned. Two four-head rounds update the queries by q←LN(q+CrossAttn(q,X)),q←LN(q+SelfAttn(q)),q←LN(q+FFN(q)), array[]rclq&←&LN (q+CrossAttn(q,X) ),\\ q&←&LN (q+SelfAttn(q) ),\\ q&←&LN (q+FFN(q) ), array (15) where the feed-forward hidden width is 2denc2d_enc and the activation is GELU. A final linear map lifts the refined queries to the backbone width, producing G. The reference is computed once, concatenated into its reserved prefix positions in the initial state, and added again at those positions before every loop iteration. This additive re-injection restores the same demonstration evidence without discarding the prefix state accumulated by the loop. C.3 Object Workspace The object workspace applies Slot Attention (Locatello et al. 2020) to the valid patch features of the recurrent state. Let Z∈B×P×dZ ^B× P× d denote these features on the fixed canvas and m∈0,1B×Pm∈\0,1\^B× P their validity mask. Each extraction projects normalized features once into keys K and values V and maintains eight slots S. Before the loop, the slots are initialized from learned queries; subsequent extractions start from the workspace produced by the preceding loop iteration. Competitive slot update. For each internal round, queries Q=WqLN(S)Q=W_qLN(S) produce scaled slot–patch affinities ℓb,k,n=⟨Qb,k,Kb,n⟩/d _b,k,n= Q_b,k,K_b,n / d. A softmax over the eight slots makes them compete for each patch, after which the validity mask removes positions outside the grid: a~b,k,n=mb,nexp(ℓb,k,n)∑jexp(ℓb,j,n),ub,k=∑na~b,k,n1+∑n′a~b,k,n′Vb,n. array[]rcl a_b,k,n&=&m_b,n\, ( _b,k,n) _j ( _b,j,n),\\ u_b,k&=& _n a_b,k,n1+ _n a_b,k,n \,V_b,n. array (16) The added unit in the denominator suppresses updates to slots that receive little attention. A shared GRU combines u with the previous slot state, followed by a residual MLP. Iterations and loop integration. Each extraction uses three update rounds. Following BO-QSA (Jia, Liu, and Huang 2023), the first two rounds do not record gradients, and S~(2)=sg(S(2))+S(0)−sg(S(0)) S^(2)=sg\! (S^(2) )+S^(0)-sg\! (S^(0) ) provides a straight-through connection to the third, gradient-enabled round. At each loop iteration, the current slots are projected by WsW_s and added at the reserved workspace positions; the next workspace is then extracted from the updated patch states, initialized from the current slots. No extraction follows the final iteration. Appendix D Training and Evaluation Configuration This appendix records the training configuration and the test-time protocol. Table 4 collects the values; the paragraphs below give the parts that a value alone does not specify. Both model sizes share every setting except the alignment weight β. Offline training. The ARC-AGI-1 models train from scratch on the 401,589-record corpus. The alignment term uses the input canvas as trajectory state T0T_0 and is active for the whole run under the warmup schedule βe=βmin(1,e/5) _e=β (1,e/5) with the epoch index e starting at one; untraced instances contribute the final-state term alone. Training runs data-parallel over the eight GPUs with a static graph. ARC-AGI-2 warm start. The ARC-AGI-2 models start from the corresponding ARC-AGI-1 EMA checkpoint and are trained for a further 100 epochs on the 882,270-record ARC-AGI-2 corpus under the same schedule. Only the weights transfer: the per-task token table is re-initialized for the new task set, and the optimizer state and epoch counter are reset, so the run is a fresh cosine cycle rather than a continuation. Test-time training. Following VARC (Hu et al. 2025), each evaluation task is augmented into 51 variants: the original task, plus five geometric views (rotations by 90∘90 , 180∘180 , 270∘270 and the two axis flips), each in its original colors and under nine random color permutations. All variants’ demonstration pairs form the training set of that task, each variant carrying its own task identity, and the per-task token table is re-initialized to that number of entries. Test-time training optimizes the plain cross-entropy of the final canvas only: no chain supervision and no change weighting are used at test time, so the transformation chains are purely an offline resource and every compared system runs the identical test-time objective. Appendix E Additional Process Visualizations Figure 5 complements the three examples in the main paper with three additional tasks, showing how predictions and workspace assignments evolve across loop iterations. Figure 5: Additional per-step predictions and slot assignments on three tasks. Prediction rows show the input and six loop outputs. Assignment rows show the argmax over the eight workspace slots at each iteration. The remaining pages show representative training records from ARC-AGI-1 and ARC-AGI-2 ARC-GEN synthesis and from verifier-traced RE-ARC data. Each record is rendered as an input, its intermediate transformation states, and the final output. See pages - of Figures/dataset-samples-appendix.pdf