Paper deep dive

Training-Free Hidden-State Refinement for Flow-Matching Image Generators

Yuanyi Yan, Xinzhe Rao, Canyu Shen, Yang Chen, Yunlu Chen, Meng Tang, Teng Long, Vincent Tao Hu

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 73

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 9/1/2026, 5:52:46 AM

Summary

The paper introduces a training-free framework for improving frozen flow-matching image generators by adding inference-time computation inside the denoiser without modifying model weights. The method employs internal transformer-layer looping, utilizing Dense and Sparse Token Loops to vary token scope, Sampling-Progress Gating to control active intervals, and Loop Guidance to combine ordinary and looped vector-field predictions. Experiments on Scale-RAE and RAEv2 models demonstrate that this approach improves quality metrics like GenEval and DPG-Bench with competitive efficiency compared to increasing outer sampling steps.

Entities (10)

Dense Token Loop · method · 99%Loop Guidance · method · 99%Sparse Token Loop · method · 99%DPG-Bench · metric · 98%GenEval · metric · 98%Scale-RAE DiT2.4B · model · 98%Scale-RAE DiT9.8B · model · 98%Flow Matching · technique · 95%RAEv2 SigLIP2-B · model · 95%Sampling-Progress Gating · method · 95%

Relation Signals (6)

Loop Guidance → combines → ordinary vector-field predictions

confidence 95% · Loop Guidance combines ordinary and looped vector-field predictions

Sampling-Progress Gating → controls → loop active interval

confidence 95% · Sampling-Progress Gating specifies the active interval along sampling progress

Dense Token Loop → improves → GenEval

confidence 95% · Dense Token Loop (Ours) 0.5422 ... GenEval

Dense Token Loop → improves → DPG-Bench

confidence 95% · Dense Token Loop (Ours) ... 0.8007 DPG

Loop Guidance → improves → RAEv2 SigLIP2-B

confidence 90% · Loop Guidance produces the clearer RAEv2 improvement

Sparse Token Loop → reduces → generation time

confidence 90% · Sparse Token Loop is slightly weaker but more efficient overall

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:We aim to improve frozen flow-matching image generators by adding inference computation inside the denoiser, without changing model weights or the outer sampler. Existing generators usually spend extra test-time computation by increasing the number of sampling steps, which repeatedly evaluates the entire denoiser and couples quality gains to sampler cost. A key challenge is how to use extra computation inside a frozen transformer denoiser: the method must decide which tokens, layers, and sampling times receive repeated updates while preserving the original generation pipeline. We introduce a training-free looping framework that repeatedly applies selected transformer layers inside each denoising call. Dense and Sparse Token Loop vary the token scope; Sampling-Progress Gating and the loop layer range specify when and where looping is active; loop count and strength control the repeated updates; and Loop Guidance combines ordinary and looped vector-field predictions. Across two Scale-RAE model scales, loop variants improve primary and auxiliary quality metrics with competitive quality--efficiency trade-offs. Loop Guidance further improves both primary metrics across all three tested models; on Scale-RAE DiT2.4B, it raises GenEval from 0.4471 to 0.5691 and DPG-Bench from 0.7656 to 0.8053. Code will be released.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

69,360 characters extracted from source content.

Expand or collapse full text

Training-Free Hidden-State Refinement for Flow-Matching Image Generators Yuanyi Yan Xinzhe Rao Canyu Shen Yang Chen Yunlu Chen Meng Tang Teng Long Vincent Tao Hu Abstract We aim to improve frozen flow-matching image generators by adding inference computation inside the denoiser, without changing model weights or the outer sampler. Existing generators usually spend extra test-time computation by increasing the number of sampling steps, which repeatedly evaluates the entire denoiser and couples quality gains to sampler cost. A key challenge is how to use extra computation inside a frozen transformer denoiser: the method must decide which tokens, layers, and sampling times receive repeated updates while preserving the original generation pipeline. We introduce a training-free looping framework that repeatedly applies selected transformer layers inside each denoising call. Dense and Sparse Token Loop vary the token scope; Sampling-Progress Gating and the loop layer range specify when and where looping is active; loop count and strength control the repeated updates; and Loop Guidance combines ordinary and looped vector-field predictions. Across two Scale-RAE model scales, loop variants improve primary and auxiliary quality metrics with competitive quality–efficiency trade-offs. Loop Guidance further improves both primary metrics across all three tested models; on Scale-RAE DiT2.4B, it raises GenEval from 0.4471 to 0.5691 and DPG-Bench from 0.7656 to 0.8053. Code will be released. 1Huazhong University of Science and Technology 2Tongji University 3King Abdullah University of Science and Technology 4University of California, Merced 5University of Amsterdam attr/Border [0 0 0] user/Subtype /Link /A << /S /URI /URI (https://yuanyiyan.com/projects/looped-flow-matching) >>Project Page Introduction Flow matching has become a central formulation for modern generative modeling because it learns a time-dependent vector field that transports a simple base distribution to data (Lipman et al. 2023; Liu et al. 2023). Its progress in image generation has come from a broad design space, including noise schedules and preconditioning (Karras et al. 2022), transformer backbones (Peebles and Xie 2023; Ma et al. 2024; Esser et al. 2024), guidance mechanisms (Dhariwal and Nichol 2021; Ho and Salimans 2022; Hu et al. 2023a; Karras et al. 2024), inference-time scaling (Ma et al. 2025), and latent representations, from latent diffusion autoencoders to RAEs (Rombach et al. 2022; Tong et al. 2026; Singh et al. 2026). These advances raise a practical question: beyond changing the sampler, architecture, or training objective, can a frozen flow-matching generator benefit from additional computation inside its denoiser? In parallel, looped transformer computation has re-emerged as a way to spend extra computation through repeated hidden-state updates. Universal and looped transformers study recurrence as a mechanism for iterative refinement, algorithm learning, and latent reasoning (Dehghani et al. 2019; Giannou et al. 2023; Yang et al. 2024; Saunshi et al. 2025). More recent training-free looped transformers suggest that pretrained pre-norm transformers can sometimes benefit from repeated layer evaluations without updating weights (Chen et al. 2026). However, it remains unclear how such hidden-state looping should be used inside flow-matching image generators, where denoising time, layer depth, token routing, and guidance all interact. Our training-free inference-time approach repeatedly applies selected transformer layers inside a frozen flow-matching denoiser, while leaving the model weights, text conditioning, latent autoencoder, and outer sampler unchanged. We organize the design around four questions: what is recomputed, when looping is active, where it is applied, and how its effect is controlled. Dense Token Loop repeats transformer-layer computation for all tokens, whereas Sparse Token Loop limits repetition to a subset of tokens to reduce computation. Sampling-Progress Gating specifies the active interval along sampling progress, and the loop layer range specifies where looping occurs. Loop count and strength control the repeated updates, while Loop Guidance determines how ordinary and looped vector-field predictions are combined. Figure presents prompt-matched qualitative comparisons and quality–efficiency trade-offs. The examples suggest that our approach primarily corrects subject-level structure: malformed or incomplete shapes become more coherent, spurious parts are suppressed, and object identities, attributes, and simple counts are represented more faithfully. The efficiency plots further show that internal layer looping reaches higher GenEval and DPG-Bench scores than increasing only the number of outer sampling steps, at comparable or lower generation time. Our contributions are: • We formulate internal transformer-layer looping as a training-free mechanism for allocating additional inference compute within frozen flow-matching image generators. • We organize internal loop design by what, when, and where computation is repeated: Dense and Sparse Token Loop operate on all or selected tokens, Sampling-Progress Gating sets the active sampling interval, and the loop layer range sets the repeated layers. • We introduce Loop Guidance as an orthogonal prediction-space mechanism that controls how the loop-induced correction is applied by combining ordinary and looped vector-field predictions. • We evaluate and analyze the loop design on Scale-RAE and RAEv2 through ablations of loop count, layer range, sampling-progress interval, sparsifier design, and guidance. Related Work Flow matching and diffusion-transformer design. Flow matching provides a flexible generative modeling framework by learning vector fields between base and data distributions (Lipman et al. 2023; Liu et al. 2023). In image generation, this formulation has been combined with diffusion-transformer backbones (Peebles and Xie 2023; Ma et al. 2024; Esser et al. 2024), improved schedule and preconditioning choices (Karras et al. 2022), and increasingly strong latent representations (Rombach et al. 2022; Tong et al. 2026; Singh et al. 2026). Prior work also studies latent-space manipulation in transformer-based flow matching (Hu et al. 2024a). Our work does not introduce a new training objective, backbone, sampler, or autoencoder. Instead, it studies whether a frozen flow-matching transformer can use additional inference compute through internal hidden-state re-evaluation. Inference-time control and guidance. A large body of work improves diffusion and flow models at inference time through schedule design, guidance, candidate selection, or prediction-space modification. Classifier guidance, classifier-free guidance, and self-guidance alter the denoising prediction without changing the generator weights (Dhariwal and Nichol 2021; Ho and Salimans 2022; Hu et al. 2023a), while autoguidance combines predictions from models of different quality (Karras et al. 2024). Other approaches scale inference by adding denoising steps, searching over noisy candidates, or applying rejection-style selection (Ma et al. 2025; Na et al. 2024). Token routing and token merging have also been used to improve diffusion training or inference efficiency (Krause et al. 2025; Bolya and Hoffman 2023); our Sparse Token Loop instead applies token selection only at inference inside a frozen denoiser while retaining full-token context. Loop Guidance is related in spirit to prediction-space control, but its direction is defined by two paths through the same frozen model: an ordinary path and a looped path. Figure 1: Overview of training-free internal looping. Sampling-Progress Gating determines when looping is active. Dense Token Loop repeats the selected layer range for all tokens, while the optional Sparse Token Loop restricts repetition to selected tokens and reuses a complement residual initialized once. All weights remain frozen, and the outer sampler is unchanged. Looped transformers. Recurrent transformer computation has been studied in universal transformers and recent looped-transformer models, where repeated application of shared layers can support iterative computation, algorithm learning, or latent reasoning (Dehghani et al. 2019; Giannou et al. 2023; Yang et al. 2024; Saunshi et al. 2025). Training-free looped transformers further show that repeated layer evaluation can be useful even without retraining (Chen et al. 2026). Our method differs by applying looped computation inside frozen flow-matching image denoisers and by separating Dense Token Loop, Sampling-Progress Gating, Sparse Token Loop with cached complement reuse, and prediction-space Loop Guidance. Method Preliminaries Flow matching learns a time-dependent vector field θ(t,t,c)v_θ(x_t,t,c) whose numerical integration transports an initial noise state toward the data distribution (Lipman et al. 2023; Liu et al. 2023); here tx_t is the latent image state, t is flow time, and c is the text condition. In DiT-based image generators, this vector field is predicted by residual transformer layers conditioned on flow time and text (Peebles and Xie 2023; Esser et al. 2024). Each denoising call ordinarily evaluates every transformer layer once. We index the denoiser’s D shape-preserving transformer layers by j∈0,…,D−1j∈\0,…,D-1\ and write jL_j for layer j. We use jh_j for the full token hidden state entering jL_j. Naive Internal Looping. A direct way to add inference compute is to reapply the same transformer layer, or layer range, K times at every denoising call. For one layer, the naive update can be written as (0) ^(0) =j, =h_j, (1) (r) ^(r) =j((r−1);t,c),r=1,…,K, =L_j(Z^(r-1);t,c), r=1,…,K, j+1 _j+1 =(K). =Z^(K). This provides additional hidden-state refinement at fixed (t,t,c)(x_t,t,c), before the vector-field prediction is returned. However, directly repeating full layer updates can produce an excessively large hidden-state displacement, applying the same amount of replay everywhere ignores the different roles of layer depth and sampling progress, and recomputing every token introduces substantial cost. These limitations motivate our controlled formulation, which normalizes each repeated residual and determines where, when, and for which tokens looping is applied. Our Method We add inference compute inside a frozen flow-matching denoiser by repeatedly applying a selected transformer-layer range during chosen denoising calls, an operation we call a loop. Figure 1 organizes the design around four choices. What is repeated is set by token scope: Dense Token Loop covers all tokens, while the optional Sparse Token Loop restricts repetition to selected tokens and reuses a complement residual initialized once. Sampling-Progress Gating determines when looping is active, and the loop layer range determines where. Loop count and strength control repeated updates, while Loop Guidance determines how ordinary and looped vector-field predictions are combined. Calls outside the active interval and layers outside the selected range follow the ordinary path. Dense Token Loop Dense Token Loop keeps the repeated-layer structure in Eq. (1) but controls each round as a residual update. We define the residual map of layer j together with the ordinary update: j(,t,c) _j(Z;t,c) =j(,t,c)−, =L_j(Z;t,c)-Z, (2) j+1 _j+1 =j+j(j,t,c)=j(j,t,c). =h_j+R_j(h_j;t,c)=L_j(h_j;t,c). It then repeats a selected layer for r=1,…,Kr=1,…,K rounds with per-round strength η=λloop/Kη= _loop/K: (0) ^(0) =j, =h_j, (3) (r) ^(r) =(r−1)+ηj((r−1),t,c), =Z^(r-1)+ _j(Z^(r-1);t,c), j+1 _j+1 =(K). =Z^(K). Here K denotes the loop count, equal to the number of evaluations of the frozen layer, and λloop _loop denotes the total loop strength distributed across the K rounds. We set λloop=1 _loop=1 in all subsequent experiments and omit it hereafter, so η=1/Kη=1/K. With K=1K=1, Eq. (3) recovers the ordinary layer output. Let [a,b][a,b] denote the selected layer range a,a+1,…,b\a,a+1,…,b\. Its ordinary map is a:b=b∘⋯∘aG_a:b=L_b ·s _a. We apply Eq. (3) successively from aL_a through bL_b and denote the resulting composition by a:b _a:b. Sampling-Progress Gating Denoising follows a coarse-to-fine trajectory: high-noise calls strongly influence text-aligned content and global layout, whereas low-noise calls mainly remove residual noise and sharpen local details (Balaji et al. 2022; Park et al. 2024; Wang and Vastola 2023). We therefore anchor the loop window at the noise side and vary only its endpoint. For N denoising calls, let tit_i be the flow time and si=i/(N−1)s_i=i/(N-1) the normalized sampling progress, where i=0,…,N−1i=0,…,N-1, so si=0s_i=0 is the initial noise state and si=1s_i=1 is the data endpoint. Given send∈[0,1]s_end∈[0,1], we define the loop-active interval as loop=[0,send]S_loop=[0,s_end]. Sampling-Progress Gating applies b+1=a:b(a;ti,c),si∈loop,a:b(a;ti,c),otherwise.h_b+1= cases _a:b(h_a;t_i,c),&s_i _loop,\\ G_a:b(h_a;t_i,c),&otherwise. cases (4) Calls with si>sends_i>s_end follow the ordinary denoiser path. Sparse Token Loop Sparse Token Loop reduces repeated token computation by looping over a token subset. Let U denote the tokens eligible for sparse replay under the chosen sparsifier, and let ρsel∈(0,1] _sel∈(0,1] denote the fraction selected for repeated computation. At each active denoising call and selected layer jL_j, we uniformly sample X⊆UX U, with |X|=⌈ρsel|U|⌉|X|= _sel|U| , and define X¯=U∖X X=U X. This partition is fixed across the K rounds for that layer. For any token subset A, let j,A(,t,c)R_j,A(Z;t,c) denote the layer residual computed only for A using the full token sequence as context. We write X(0)Z_X^(0) and X¯(0)Z_ X^(0) for the corresponding token states obtained by partitioning (0)Z^(0), and MergeMerge restores their original token order. Initialize once. Before looping, we compute and cache only the complement-token residual: X¯=j,X¯((0),t,c).C_ X=R_j, X(Z^(0);t,c). (5) Loop K times. At round r=1,…,Kr=1,…,K, we evaluate the selected-token residual on the current all-token state and reuse X¯C_ X for the complement: X(r) _X^(r) =X(r−1)+ηj,X((r−1),t,c), =Z_X^(r-1)+ _j,X(Z^(r-1);t,c), (6) X¯(r) _ X^(r) =X¯(r−1)+ηX¯, =Z_ X^(r-1)+ _ X, (r) ^(r) =Merge⁡(X(r),X¯(r)). =Merge\! (Z_X^(r),Z_ X^(r) ). This design has two advantages. First, it reduces repeated computation because only the selected-token residual j,XR_j,X is freshly evaluated after the complement residual is initialized once. Second, both token groups are updated in every round, which keeps them synchronized in loop depth. Consequently, each selected-token attention update uses queries from X and keys and values from both X and X¯ X, whose states have undergone the same number of loop updates, rather than using complement states frozen at initialization. Concretely, Sparse Token Loop avoids repeated query, attention-output, and MLP evaluation for the complement tokens while retaining keys and values from all tokens to preserve global context. Its implementation incurs token-partitioning, routing, and merging overhead, which partially offsets these computational savings. This is related to training-free feature caching for diffusion transformers, but our cache is an intra-layer complement residual tied to the current token partition rather than a cross-timestep feature cache (Zou et al. 2025). Loop Guidance The preceding controls alter hidden-state computation inside one denoiser path. Motivated by prediction-space extrapolation in classifier-free guidance and autoguidance (Ho and Salimans 2022; Karras et al. 2024), Loop Guidance combines an ordinary prediction with a looped prediction produced by either Dense Token Loop or Sparse Token Loop: base _base =θ(t,t,c), =v_θ(x_t,t,c), (7) loop _loop =θloop(t,t,c), =v_θ^loop(x_t,t,c), lg _lg =base+glg(loop−base). =v_base+g_lg\! (v_loop-v_base ). The scale glgg_lg controls the strength of the loop-induced correction in prediction space, with glg>1g_lg>1 extrapolating beyond the looped prediction. Because both predictions are required, Loop Guidance costs more than a single looped path. Experiments Model Method GenEval↑ DPG↑ ImageReward↑ CLIP↑ OpenCLIP↑ PickScore↑ Time (s)↓ w/o Loop 0.4471 0.7656 0.0994 0.3204 0.3490 0.2122 1.3054 Scale-RAE DiT2.4B Dense Token Loop (Ours) 0.5422 0.8007 0.4816 0.3302 0.3622 0.2165 2.1812 Sparse Token Loop (Ours) 0.5082 0.7893 0.2798 0.3249 0.3555 0.2144 1.6884 w/o Loop 0.5321 0.8003 0.4609 0.3285 0.3604 0.2175 1.6853 Scale-RAE DiT9.8B Dense Token Loop (Ours) 0.5977 0.8178 0.7211 0.3344 0.3682 0.2221 4.0384 Sparse Token Loop (Ours) 0.5857 0.8206 0.6365 0.3328 0.3663 0.2205 2.4410 w/o Loop 0.3829 0.7131 -0.3385 0.3076 0.3176 0.2055 0.8025 RAEv2 SigLIP2-B (615M) Dense Token Loop (Ours) 0.3795 0.7226 -0.2538 0.3101 0.3202 0.2062 1.1023 Sparse Token Loop (Ours) 0.3815 0.7152 -0.2931 0.3086 0.3187 0.2059 1.3572 Table 1: Main results across three RAE-based model configurations. w/o Loop denotes ordinary denoising. The best and second-best values for each model are shown in bold and underlined, respectively. Model Method lgg_lg GenEval↑ DPG↑ ImageReward↑ CLIP↑ OpenCLIP↑ PickScore↑ Scale-RAE DiT2.4B w/o Loop 0.0 0.4471 0.7656 0.0994 0.3204 0.3490 0.2122 Dense Token Loop (Ours) 2.8 0.5691 0.8053 0.5622 0.3306 0.3627 0.2171 Scale-RAE DiT9.8B w/o Loop 0.0 0.5321 0.8003 0.4609 0.3285 0.3604 0.2175 Dense Token Loop (Ours) 6.0 0.6432 0.8264 0.7975 0.3332 0.3661 0.2229 w/o Loop 0.0 0.3829 0.7131 -0.3385 0.3076 0.3176 0.2055 RAEv2 SigLIP2-B (615M) Dense Token Loop (Ours) 9.0 0.4532 0.7521 0.0460 0.3159 0.3297 0.2090 Sparse Token Loop (Ours) 6.0 0.4114 0.7429 -0.1296 0.3129 0.3251 0.2074 Table 2: Loop Guidance improves both primary metrics and all auxiliary metrics across the three evaluated generators. Baseline rows use glg=0g_lg=0, and guided rows use the listed scales. Best values within each model are bolded. Experimental Setup We conduct experiments on two Scale-RAE configurations, Qwen1.5B/DiT2.4B and Qwen7B/DiT9.8B, and on RAEv2. For RAEv2, we use the public DiffusionBench RAE-SigLIP2-B checkpoint with a 615M-parameter DDT (Leng et al. 2026). All pretrained components remain frozen throughout. We identify the shared loop configuration on Scale-RAE Qwen1.5B/DiT2.4B and directly transfer it to Qwen7B/DiT9.8B and RAE-SigLIP2-B; the transferred setting remains effective on both target models. We use GenEval (Ghosh et al. 2023) and DPG-Bench (Hu et al. 2024b) as the primary metrics, together with CLIP Score (Radford et al. 2021), OpenCLIP Score (Cherti et al. 2023), PickScore (Kirstain et al. 2023), and ImageReward (Xu et al. 2023). Detailed checkpoints and evaluation protocols are provided in the appendix. Controlled ablations use Scale-RAE Qwen1.5B/DiT2.4B with 25 Euler steps, K=4K=4, loop layer range [12,27][12,27], and si∈[0,0.5]s_i∈[0,0.5]; each ablation changes only the named factor. RAE-SigLIP2-B retains its 50-step Euler schedule. We use 25 outer sampling steps because increasing the schedule to 50 steps substantially increases inference time while yielding only marginal improvements in GenEval and DPG-Bench. Complete configurations are provided in the appendix. Main Results Table 1 compares the unguided loop variants across three frozen generators. Both Dense Token Loop and Sparse Token Loop improve GenEval and DPG-Bench at both Scale-RAE scales. Overall, Dense Token Loop achieves stronger quality gains but requires more generation time, whereas Sparse Token Loop is slightly weaker but more efficient overall. On RAEv2, both variants improve DPG-Bench and all auxiliary metrics but slightly reduce GenEval, so the unguided transfer result is mixed. We therefore avoid claiming a uniformly positive RAEv2 gain from the transferred Scale-RAE hyperparameters. As shown in Table 2, Loop Guidance produces the clearer RAEv2 improvement. Figure 2: Moderate Loop Guidance scales and loop counts capture most quality gains on Scale-RAE (DiT2.4B). Panel (a) varies the Loop Guidance scale, and panel (b) varies the loop count K. Sparse Token Loop. As described in the Sparse Token Loop subsection, the method reduces repeated complement-token computation while preserving full-token attention context. On both Scale-RAE models, it remains above w/o Loop on all reported quality metrics while reducing generation time relative to Dense Token Loop. The reduction is substantially larger on DiT9.8B than on DiT2.4B. On the smaller 615M DDT, however, Sparse Token Loop is slower than Dense Token Loop, suggesting that its additional routing overhead is not offset at this scale. Specifically, Sparse Token Loop reduces generation time by 22.6% on DiT2.4B and 39.6% on DiT9.8B relative to Dense Token Loop. Loop Guidance. We do not exhaustively tune the guidance scale; for each model, Table 2 reports one setting that performs reasonably well in our trials. Combining the ordinary and looped predictions improves both primary metrics across the Scale-RAE DiT2.4B and DiT9.8B checkpoints and RAEv2, together with consistent gains on the auxiliary metrics. In particular, although the unguided improvements on RAEv2 are small, adding Loop Guidance produces clear improvements on both primary metrics. On RAEv2, Loop Guidance raises GenEval from 0.3829 to 0.4532 and DPG-Bench from 0.7131 to 0.7521 while improving every auxiliary metric. This suggests that the vector-field difference introduced by looping, loop−basev_loop-v_base, encodes a small, directionally useful quality correction that Loop Guidance can amplify. Quality–Efficiency Tradeoff As shown in the right column of Figure , internal layer looping uses inference compute more effectively than additional outer sampling steps. A 25-step Dense Token Loop reaches 0.5422 GenEval and 0.8007 DPG-Bench in 2.1812 s, outperforming the 50-step no-loop baseline (0.4742 and 0.7738) while remaining faster than its 2.4032 s runtime. Sparse Token Loop similarly reaches 0.5082 GenEval and 0.7893 DPG-Bench in 1.6883 s, compared with 0.4770 and 0.7673 in 1.7575 s for the 35-step no-loop baseline. These comparisons indicate that the gains arise from allocating computation to internal refinement rather than merely increasing total inference cost. The latency advantage remains despite repeated evaluations of the selected layer range, showing that internal compute can be allocated more selectively than full-denoiser resampling. Figure 3: Mid-to-late layer ranges and earlier loop-active sampling intervals give the strongest DPG-Bench settings on Scale-RAE (DiT2.4B). Panel (a) varies the loop layer range with the loop-active sampling-progress interval fixed; panel (b) varies the loop-active sampling-progress interval with the layer range fixed. The corresponding GenEval panels are provided in the appendix. Loop Configuration Ablations Using Scale-RAE DiT2.4B with 25 outer sampling steps, we ablate the Loop Guidance scale, loop count, loop layer range, and loop-active sampling-progress interval. Unless the named factor is varied, we use a loop count of K=4K=4, loop layer range [12,27][12,27], and loop-active sampling-progress interval si∈[0,0.5]s_i∈[0,0.5]. Loop Guidance Scale. With the loop configuration fixed, Figure 2(a) varies glgg_lg. Moderate settings improve both primary metrics, although their preferred scales differ, while overly large scales weaken the gains. This supports moderate prediction-space extrapolation. Loop Count. With the layer range and sampling-progress interval fixed, Figure 2(b) varies the loop count. Moderate loop counts capture most of the gains, whereas additional loop rounds give diminishing, metric-dependent returns while increasing computation. We therefore use a loop count of K=4K=4 by default. Loop Layer Range. With a loop count of K=4K=4 and the sampling-progress interval fixed, Figure 3(a) compares different layer windows on DPG-Bench. Middle-to-late layer ranges consistently outperform early ranges. Looping over all 32 layers provides little additional benefit over a well-placed 16-layer range while requiring more computation. We therefore use [12,27][12,27] as the default loop layer range. Loop-Active Sampling-Progress Interval. With a loop count of K=4K=4 and the layer range fixed, Figure 3(b) varies when the loop is active along the sampling trajectory on DPG-Bench. Across interval lengths, looping during earlier, higher-noise stages is generally more effective than looping later. Table 3 isolates the no-loop, full-schedule, and default gated policies. Activating the loop over the first half of sampling, si∈[0,0.5]s_i∈[0,0.5], retains most of the full-schedule benefit with less loop computation and is therefore our default. Loop-active interval GenEval↑ DPG↑ Time(s)↓ w/o Loop 0.4471 0.7656 1.3054 [0,1][0,1] 0.5597 0.7987 2.9957 [0,0.5][0,0.5] 0.5422 0.8007 2.1812 Table 3: For Dense Token Loop, [0,1][0,1] denotes full-schedule looping and [0,0.5][0,0.5] denotes Sampling-Progress Gating; the latter preserves most full-schedule Loop quality while reducing generation time on Scale-RAE. Further Exploration Sparsifier Comparison. Since Sparse Token Loop concentrates repeated computation on selected tokens, a natural question is whether prioritizing tokens that are important for refinement can improve generation quality. On Scale-RAE, Table 4 compares random routing with condition-aware and attention-aware routing, which prioritize tokens that respond more strongly to conditioning and attention, respectively. Neither targeted variant clearly improves upon random. On RAEv2, whose DDT operates on a joint sequence of image and conditioning tokens, we additionally compare random routing with an image/condition split; their results remain close. Overall, the specific sparsifier designs are described in the appendix. More effective sparsifiers may exist, but random routing gives the strongest overall balance among the choices tested here. Model Sparsifier Ratio GenEval↑ DPG-Bench↑ w/o Loop – 0.4471 0.7656 random 50% 0.5029 0.7951 condition-aware 50% 0.5039 0.7846 Scale-RAE attention-aware 50% 0.5008 0.7894 random 70% 0.5140 0.7962 condition-aware 70% 0.5175 0.7918 attention-aware 70% 0.5070 0.7918 w/o Loop – 0.3829 0.7131 RAEv2 random 50% 0.4114 0.7429 image/condition – 0.4170 0.7474 Table 4: Sparsifier comparison on Scale-RAE and RAEv2. VAE-Based Generators. Table 5 reports a transfer diagnostic on PixArt-alpha (Chen et al. 2024) and FLUX.2 (Black Forest Labs 2025). For both generators, we apply Loop Guidance with glg=3.0g_lg=3.0. Neither model shows a consistent improvement across both primary metrics. We therefore hypothesize that the benefit observed on RAE-based generators may not transfer directly to VAE-based generators; understanding the underlying reason requires further investigation. Model Method GenEval↑ DPG-Bench↑ w/o Loop 0.4778 0.7038 PixArt-alpha Dense Token Loop 0.4710 0.7170 Loop Guidance 0.3928 0.6986 w/o Loop 0.8459 0.8740 FLUX.2 Dense Token Loop 0.8482 0.8740 Loop Guidance 0.8310 0.8720 Table 5: VAE transfer diagnostic on PixArt-alpha and FLUX.2. Qualitative Correction Effect of Looping. The left column of Figure offers a preliminary view of what looping corrects under matched prompts. Corrections are mainly subject-level: malformed or incomplete shapes become coherent, spurious parts are suppressed, and attributes bind more cleanly. The shown GenEval cases also have clearer identities and simple counts, while Loop Guidance gives cleaner DPG-Bench presentations. Appendix GenEval2 examples suggest that harder counting and placement errors remain. Thus, subject-level correction appears stronger, but its mechanism remains open. Conclusion We show that inference computation can be added inside a frozen flow-matching denoiser by repeatedly applying selected transformer layers, without changing model weights, conditioning, latent representations, or the outer sampler. Organizing this operation by token scope, sampling progress, layer range, and prediction-space control yields Dense Token Loop, Sparse Token Loop, Sampling-Progress Gating, and Loop Guidance. Across both Scale-RAE scales, looping improves the primary and auxiliary quality metrics, while Sparse Token Loop provides a lower-latency alternative to dense repetition. At comparable generation time, internal looping also improves upon allocating computation only to additional outer sampling steps. Loop Guidance further improves GenEval and DPG-Bench on all three tested RAE-based generators. The results also clarify the current scope of the method. Looping most reliably corrects subject formation and attribute binding, but does not consistently repair counting or spatial composition on GenEval2. Its effect depends on layer location and sampling progress, and the tested VAE-based generators do not show a consistent gain. Internal layer looping is therefore a useful but model-dependent inference-time refinement axis. Future work should develop task-aware sparsifiers and adaptive loop policies, and determine which architectural and latent-representation properties make an image generator receptive to repeated hidden-state updates. References Balaji et al. (2022) Y. Balaji, S. Nah, X. Huang, A. Vahdat, J. Song, Q. Zhang, K. Kreis, M. Aittala, T. Aila, S. Laine, B. Catanzaro, T. Karras, and M. Liu eDiff-I: text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324. External Links: Link Cited by: Sampling-Progress Gating. Black Forest Labs (2025) Black Forest Labs FLUX.2: frontier visual intelligence. External Links: Link Cited by: Appendix B, VAE-Based Generators.. Bolya and Hoffman (2023) D. Bolya and J. Hoffman Token merging for fast stable diffusion. In CVPR Workshops, External Links: Link Cited by: Inference-time control and guidance.. Chen et al. (2024) J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Z. Wang, J. Kwok, P. Luo, H. Lu, and Z. Li PixArt-α: fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, External Links: Link Cited by: Appendix B, VAE-Based Generators.. Chen et al. (2026) L. Chen, J. Li, C. Liang, N. Lao, and Q. Liu Training-free looped transformers. arXiv preprint arXiv:2605.23872. External Links: Link Cited by: Introduction, Looped transformers.. Cheng et al. (2022) B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar Masked-attention mask transformer for universal image segmentation. In CVPR, External Links: Link Cited by: Appendix B. Cherti et al. (2023) M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C. Gordon, C. Schuhmann, L. Schmidt, and J. Jitsev Reproducible scaling laws for contrastive language-image learning. In CVPR, External Links: Link Cited by: Appendix B, Experimental Setup. Dehghani et al. (2019) M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser Universal transformers. In ICLR, External Links: Link Cited by: Introduction, Looped transformers.. Dhariwal and Nichol (2021) P. Dhariwal and A. Nichol Diffusion models beat GANs on image synthesis. In NeurIPS, External Links: Link Cited by: Introduction, Inference-time control and guidance.. Esser et al. (2024) P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Muller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, D. Podell, T. Dockhorn, Z. English, and R. Rombach Scaling rectified flow transformers for high-resolution image synthesis. In ICML, External Links: Link Cited by: Introduction, Flow matching and diffusion-transformer design., Preliminaries. Ghosh et al. (2023) D. Ghosh, H. Hajishirzi, and L. Schmidt GenEval: an object-focused framework for evaluating text-to-image alignment. In NeurIPS, External Links: Link Cited by: Appendix B, Experimental Setup. Giannou et al. (2023) A. Giannou, S. Rajput, J. Sohn, K. Lee, J. D. Lee, and D. Papailiopoulos Looped transformers as programmable computers. In ICML, External Links: Link Cited by: Introduction, Looped transformers.. Ho and Salimans (2022) J. Ho and T. Salimans Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. Cited by: Introduction, Inference-time control and guidance., Loop Guidance. Hu et al. (2023a) V. T. Hu, D. W. Zhang, Y. M. Asano, G. J. Burghouts, and C. G. M. Snoek Self-guided diffusion models. In CVPR, External Links: Link Cited by: Introduction, Inference-time control and guidance.. Hu et al. (2024a) V. T. Hu, D. W. Zhang, P. Mettes, M. Tang, D. Zhao, and C. G. M. Snoek Latent space editing in transformer-based flow matching. In AAAI, External Links: Link Cited by: Flow matching and diffusion-transformer design.. Hu et al. (2024b) X. Hu, R. Wang, Y. Fang, B. Fu, P. Cheng, and G. Yu ELLA: equip diffusion models with LLM for enhanced semantic alignment. arXiv preprint arXiv:2403.05135. External Links: Link Cited by: Appendix B, Experimental Setup. Hu et al. (2023b) Y. Hu, B. Liu, J. Kasai, Y. Wang, M. Ostendorf, R. Krishna, and N. A. Smith TIFA: accurate and interpretable text-to-image faithfulness evaluation with question answering. In ICCV, External Links: Link Cited by: Appendix D. Kamath et al. (2025) A. Kamath, K. Chang, R. Krishna, L. Zettlemoyer, Y. Hu, and M. Ghazvininejad GenEval 2: addressing benchmark drift in text-to-image evaluation. arXiv preprint arXiv:2512.16853. External Links: Link Cited by: Appendix B. Karras et al. (2022) T. Karras, M. Aittala, T. Aila, and S. Laine Elucidating the design space of diffusion-based generative models. In NeurIPS, External Links: Link Cited by: Introduction, Flow matching and diffusion-transformer design.. Karras et al. (2024) T. Karras, M. Aittala, T. Kynkänniemi, J. Lehtinen, T. Aila, and S. Laine Guiding a diffusion model with a bad version of itself. In NeurIPS, External Links: Link Cited by: Introduction, Inference-time control and guidance., Loop Guidance. Kirstain et al. (2023) Y. Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy Pick-a-pic: an open dataset of user preferences for text-to-image generation. In NeurIPS, External Links: Link Cited by: Appendix B, Experimental Setup. Krause et al. (2025) F. Krause, T. Phan, M. Gui, S. A. Baumann, V. T. Hu, and B. Ommer TREAD: token routing for efficient architecture-agnostic diffusion training. In ICCV, External Links: Link Cited by: Inference-time control and guidance.. Leng et al. (2026) X. Leng, J. Singh, Z. Liang, E. Smith, M. Bell, A. Saha, Y. Yuan, and L. Zheng DiffusionBench: on holistic evaluation of diffusion transformers. arXiv preprint arXiv:2606.24888. External Links: Link Cited by: Appendix B, Experimental Setup. Li et al. (2022) C. Li, H. Xu, J. Tian, W. Wang, M. Yan, B. Bi, J. Ye, H. Chen, G. Xu, Z. Cao, J. Zhang, S. Huang, F. Huang, J. Zhou, and L. Si mPLUG: effective and efficient vision-language learning by cross-modal skip-connections. In EMNLP, External Links: Link Cited by: Appendix B. Lipman et al. (2023) Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le Flow matching for generative modeling. In ICLR, Cited by: Introduction, Flow matching and diffusion-transformer design., Preliminaries. Liu et al. (2023) X. Liu, C. Gong, and Q. Liu Flow straight and fast: learning to generate and transfer data with rectified flow. In ICLR, Cited by: Introduction, Flow matching and diffusion-transformer design., Preliminaries. Ma et al. (2024) N. Ma, M. Goldstein, M. S. Albergo, N. M. Boffi, E. Vanden-Eijnden, and S. Xie SiT: exploring flow and diffusion-based generative models with scalable interpolant transformers. In ECCV, External Links: Link Cited by: Introduction, Flow matching and diffusion-transformer design.. Ma et al. (2025) N. Ma, S. Tong, H. Jia, H. Hu, Y. Su, M. Zhang, X. Yang, Y. Li, T. Jaakkola, X. Jia, and S. Xie Scaling inference time compute for diffusion models. In CVPR, External Links: Link Cited by: Introduction, Inference-time control and guidance.. Na et al. (2024) B. Na, Y. Kim, M. Park, D. Shin, W. Kang, and I. Moon Diffusion rejection sampling. In ICML, External Links: Link Cited by: Inference-time control and guidance.. Park et al. (2024) B. Park, S. Woo, H. Go, J. Kim, and C. Kim Denoising task routing for diffusion models. In ICLR, External Links: Link Cited by: Sampling-Progress Gating. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In ICCV, Cited by: Introduction, Flow matching and diffusion-transformer design., Preliminaries. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In ICML, External Links: Link Cited by: Appendix B, Experimental Setup. Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer High-resolution image synthesis with latent diffusion models. In CVPR, External Links: Link Cited by: Introduction, Flow matching and diffusion-transformer design.. Saunshi et al. (2025) N. Saunshi, N. Dikkala, Z. Li, S. Kumar, and S. J. Reddi Reasoning with latent thoughts: on the power of looped transformers. In ICLR, External Links: Link Cited by: Introduction, Looped transformers.. Singh et al. (2026) J. Singh, B. Zheng, Z. Wu, R. Zhang, E. Shechtman, and S. Xie Improved baselines with representation autoencoders. arXiv preprint arXiv:2605.18324. External Links: Link Cited by: Appendix B, Introduction, Flow matching and diffusion-transformer design.. Tong et al. (2026) S. Tong, B. Zheng, Z. Wang, B. Tang, N. Ma, E. Brown, J. Yang, R. Fergus, Y. LeCun, and S. Xie Scaling text-to-image diffusion transformers with representation autoencoders. arXiv preprint arXiv:2601.16208. External Links: Link Cited by: Appendix B, Introduction, Flow matching and diffusion-transformer design.. Wang and Vastola (2023) B. Wang and J. J. Vastola Diffusion models generate images like painters: an analytical theory of outline first, details later. arXiv preprint arXiv:2303.02490. External Links: Link Cited by: Sampling-Progress Gating. Xu et al. (2023) J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong ImageReward: learning and evaluating human preferences for text-to-image generation. In NeurIPS, External Links: Link Cited by: Appendix B, Experimental Setup. Yang et al. (2024) L. Yang, K. Lee, R. Nowak, and D. Papailiopoulos Looped transformers are better at learning learning algorithms. In ICLR, External Links: Link Cited by: Introduction, Looped transformers.. Zou et al. (2025) C. Zou, X. Liu, T. Liu, S. Huang, and L. Zhang Accelerating diffusion transformers with token-wise feature caching. In ICLR, External Links: Link Cited by: Loop K times.. Appendix A Method Pseudocode Algorithms 1–4 summarize the four core inference-time operations. All model parameters remain frozen. Algorithm 1 Dense Token Loop for a selected layer range. 1: Frozen layers jj=ab\L_j\_j=a^b, state ah_a, time t, condition c, loop count K, total loop strength λloop _loop 2: Refined state b+1h_b+1 3: η←λloop/Kη← _loop/K 4: for j=a,…,bj=a,…,b do 5: (0)←jZ^(0) _j 6: for r=1,…,Kr=1,…,K do 7: (r)←(r−1)+η⁡[j((r−1),t,c)−(r−1)]Z^(r) ^(r-1)+η[L_j(Z^(r-1);t,c)-Z^(r-1)] 8: end for 9: j+1←(K)h_j+1 ^(K) 10: end for 11: return b+1h_b+1 Algorithm 2 Sparse Token Loop for a selected layer range. 1: Frozen layers jj=ab\L_j\_j=a^b, state ah_a, time t, condition c, loop count K, total loop strength λloop _loop 2: Valid token set U, selected-token ratio ρsel _sel 3: Refined state b+1h_b+1 4: η←λloop/Kη← _loop/K 5: for j=a,…,bj=a,…,b do 6: (0)←jZ^(0) _j; m←⌈ρsel|U|⌉m← _sel|U| 7: Sample X⊆UX U uniformly with |X|=m|X|=m; X¯←U∖X X← U X 8: X¯←j,X¯((0),t,c)C_ X _j, X(Z^(0);t,c) ⊳ initialize once 9: for r=1,…,Kr=1,…,K do 10: X(r)←X(r−1)+ηj,X((r−1),t,c)Z_X^(r) _X^(r-1)+ _j,X(Z^(r-1);t,c) ⊳ full context 11: X¯(r)←X¯(r−1)+ηX¯Z_ X^(r) _ X^(r-1)+ _ X 12: (r)←Merge⁡(X(r),X¯(r))Z^(r) (Z_X^(r),Z_ X^(r)) 13: end for 14: j+1←(K)h_j+1 ^(K) 15: end for 16: return b+1h_b+1 Algorithm 3 Sampling-Progress Gating. 1: State ah_a, time tit_i, condition c, call index i, total calls N, layer range [a,b][a,b], loop count K, total loop strength λloop _loop 2: Parameter: loop-active interval loopS_loop 3: State b+1h_b+1 4: si←i/(N−1)s_i← i/(N-1) 5: if si∈loops_i _loop then 6: b+1←DenseTokenLoop(jj=ab,a,ti,c,K,λloop)h_b+1← DenseTokenLoop(\L_j\_j=a^b,h_a,t_i,c,K, _loop) 7: else 8: b+1←a:b(a;ti,c)h_b+1 _a:b(h_a;t_i,c) ⊳ ordinary path 9: end if 10: return b+1h_b+1 Algorithm 4 Loop Guidance at one denoising call. 1: Frozen denoiser θv_θ, input tix_t_i, time tit_i, condition c, guidance scale glgg_lg 2: Loop configuration ([a,b],K,λloop,loop)([a,b],K, _loop,S_loop) 3: Guided vector-field prediction lgv_lg 4: base←θ(ti,ti,c)v_base _θ(x_t_i,t_i,c) ⊳ ordinary path 5: loop←θgate+loop(ti,ti,c)v_loop _θ^gate+loop(x_t_i,t_i,c) ⊳ Algorithms 1–3 6: lg←base+glg(loop−base)v_lg _base+g_lg(v_loop-v_base) 7: return lgv_lg Appendix B Experimental Details Evaluation Protocol The standard protocol uses all 553 GenEval prompts (Ghosh et al. 2023), 1065 unique DPG-Bench image prompts (Hu et al. 2024b), and all 800 GenEval2 prompts (Kamath et al. 2025). GenEval and DPG-Bench are used for the primary comparisons; GenEval2 is used for the additional qualitative analysis. Every setting generates one image per prompt with seed 42. Within each comparison, the baseline and all loop variants share the exact prompt, initial noise seed, checkpoint, resolution, precision, sampler, outer sampling steps, text conditioning, and autoencoder. These complete fixed-seed benchmark comparisons remove prompt-set and initial-noise differences between methods, but they do not estimate variation across random seeds. We therefore do not attach confidence intervals or make statistical-significance claims. GenEval records are exported to the pinned geneval-main evaluator and scored with its Mask2Former-based dependencies (Cheng et al. 2022). DPG-Bench records are exported to the pinned ELLA-main/dpg_bench evaluator and scored with mPLUG VQA (Li et al. 2022). Scale-RAE images are evaluated at their native 224×224224× 224 resolution; the DPG-Bench protocol uses one image per prompt and a 224224-pixel crop. The RAE-SigLIP2-B proxy uses its native 256×256256× 256 resolution. GenEval2 uses its official 800-prompt soft-TIFA-style protocol. For the four auxiliary metrics, each generated image IjI_j is paired only with its own generation prompt pjp_j. CLIP Score, OpenCLIP Score, and PickScore are computed as the raw cosine similarity sj=fI(Ij)⊤fT(pj)∥fI(Ij)∥2∥fT(pj)∥2.s_j= f_I(I_j) f_T(p_j) f_I(I_j) _2 f_T(p_j) _2. The respective encoders are OpenAI CLIP ViT-B/32 (Radford et al. 2021), OpenCLIP ViT-H/14 pretrained on LAION-2B (s32B/b79K) (Cherti et al. 2023), and PickScore v1 (Kirstain et al. 2023). The reported values do not use the temperature-scaled logits or a batch-wise softmax. ImageReward is the raw scalar sj=R⁡(pj,Ij)s_j=R(p_j,I_j) returned by ImageReward-v1.0 (Xu et al. 2023). For each variant and each metric, the table reports s¯=∑j∈GenEvalsj+∑j∈DPGsj553+1065, s= _j s_j+ _j s_j553+1065, equivalently the sample-count-weighted mean of the two split means. Higher is better for all four metrics. Models and Checkpoints We evaluate two official Scale-RAE configurations with the following exact checkpoint identifiers (Tong et al. 2026): nyu-visionx/Scale-RAE-Qwen1.5B_DiT2.4B nyu-visionx/Scale-RAE-Qwen7B_DiT9.8B Both repositories retain their official FP32 parameter dtype and use model guidance level 1.0. For architecture transfer, we load the public repository diffusion-bench/diffusion-bench (Leng et al. 2026) and its EMA checkpoint t2i-ddt-en28d1152hd72-dn2d2048hd128-rae-siglip2-vit-b-vpred-t4-v1.pt. This 615M-parameter RAE-SigLIP2-B/DDT checkpoint is used as a proxy and is not the unreleased official RAEv2 text-to-image checkpoint (Singh et al. 2026). Its reported runs disable classifier-free guidance. The VAE-based transfer experiments use the exact identifiers PixArt-alpha/PixArt-XL-2-1024-MS (Chen et al. 2024) and diffusers/FLUX.2-dev-bnb-4bit (Black Forest Labs 2025). Both use their checkpoint-provided Diffusers pipeline and scheduler. All pretrained components remain frozen. Table 6 collects the resolution, precision, sampling, and loop settings associated with these checkpoints. Generator Resolution Precision Steps Dense range Sparse range / ratio K/λloopK/ _loop Progress glgg_lg dense/sparse Scale-RAE DiT2.4B 2242224^2 FP32 25 [12,27][12,27] [12,19]/70%[12,19]/70\% 4/14/1 [0,0.5][0,0.5] 2.8/–2.8/-- Scale-RAE DiT9.8B 2242224^2 FP32 25 [12,27][12,27] [12,27]/50%[12,27]/50\% 4/14/1 [0,0.5][0,0.5] 6.0/–6.0/-- RAE-SigLIP2-B proxy 2562256^2 BF16 50 [15,22][15,22] [15,22]/50%[15,22]/50\% 4/14/1 [0,0.5][0,0.5] 9.0/6.09.0/6.0 PixArt-alpha 102421024^2 BF16 20 [14,19][14,19] – 4/14/1 [0,0.5][0,0.5] 3.0/–3.0/-- FLUX.2 102421024^2 BF16/4-bit 28 [32,37][32,37] – 4/14/1 [0,0.5][0,0.5] 3.0/–3.0/-- Table 6: Complete loop configurations used for the main and transfer comparisons. Precision denotes parameter or weight precision. A dash indicates that Sparse Token Loop was not evaluated for that generator. The reported Loop Guidance scales are listed as dense/sparse; a dash indicates that the corresponding guided variant was not reported in the main comparison. Unguided loop variants are equivalent to glg=1g_lg=1. The Scale-RAE DiT9.8B TF32 matmul policy is described in the text, and controlled Scale-RAE ablations vary only the named factor. Runtime Configuration and Timing All reported generation runs use NVIDIA H100 GPUs. Each resident Scale-RAE Qwen7B/DiT9.8B generation worker is allocated two H100 GPUs and loads the checkpoint across them with device_map=auto; this is layer placement rather than tensor parallelism. All other image generators use one H100 per generation worker; FLUX.2 additionally enables model CPU offload. The pinned Scale-RAE, RAE-SigLIP2-B proxy, and PixArt-alpha environment records Python 3.10, CUDA 12.4, and PyTorch 2.5.1. The separate FLUX.2 overlay records PyTorch 2.6.0 with CUDA 12.4 and pins its source-installed Diffusers revision; each official evaluator runs in its own pinned environment. The generation launcher enables TF32 for CUDA and cuDNN matmuls and sets high float32 matmul precision. Scale-RAE nevertheless retains FP32 model parameters and an FP32 diffusion head; the RAE-SigLIP2-B proxy and PixArt use BF16 autocast, and FLUX.2 uses BF16 compute with 4-bit model weights. Generation time is measured with a monotonic wall clock around the model generation call followed by device synchronization. It includes denoiser and sampler computation but excludes model loading, prompt tokenization, image decoding, and artifact writing. Timing tables average the per-image records specified in their captions, and absolute times are compared only within a matched hardware and software source. Detailed Hyperparameters for Main-Paper Tables and Figures The entries below follow their first appearance in the main paper. Checkpoint-level resolution, precision, sampling, and default loop settings are collected in Table 6. Figure 1: Qualitative examples and quality–efficiency trade-off. The left examples use Scale-RAE DiT9.8B with 25 outer steps, layers [12,27][12,27], K=4K=4, λloop=1 _loop=1, and progress [0,0.5][0,0.5]. Dense Token Loop uses glg=1g_lg=1 and Loop Guidance uses glg=6.0g_lg=6.0, under prompt- and seed-matched generation with seed 42. The right plots use DiT2.4B. Their no-loop points use (N,K)∈(15,1),(20,1),(25,1),(35,1),(50,1)(N,K)∈\(15,1),(20,1),(25,1),(35,1),(50,1)\; Dense Token Loop uses (10,8)(10,8), (15,4)(15,4), and (25,4)(25,4); and Sparse Token Loop uses (25,4)(25,4). All looped points use layers [12,19][12,19], λloop=1 _loop=1, and progress [0,0.5][0,0.5]. The sparse point uses random cached-complement routing with ρsel=0.70 _sel=0.70. Figure 2: Method overview. This is an algorithmic schematic rather than the output of an experimental run, so it has no checkpoint-, seed-, sampler-, or dataset-specific hyperparameters. Table 1: Main comparison. For Scale-RAE DiT2.4B, Dense Token Loop uses layers [12,27][12,27] and Sparse Token Loop uses layers [12,19][12,19] with random cached-complement routing and ρsel=0.70 _sel=0.70. For DiT9.8B, both variants use [12,27][12,27], and the sparse variant uses random cached-complement routing with ρsel=0.50 _sel=0.50. For the RAE-SigLIP2-B proxy, both variants use [15,22][15,22], and the sparse variant uses random cached-complement routing with ρsel=0.50 _sel=0.50. The generator-level outer steps and precision follow Table 6. All loop variants use K=4K=4, λloop=1 _loop=1, progress [0,0.5][0,0.5], and the unguided setting glg=1g_lg=1. Scale-RAE uses model guidance level 1.0, whereas the RAE-SigLIP2-B proxy disables classifier-free guidance. Table 2: Loop Guidance. The loop paths retain the preceding dense and sparse configurations. DiT2.4B Dense Token Loop uses glg=2.8g_lg=2.8, DiT9.8B Dense Token Loop uses glg=6.0g_lg=6.0, and the RAE-SigLIP2-B proxy uses glg=9.0g_lg=9.0 for Dense Token Loop and glg=6.0g_lg=6.0 for Sparse Token Loop. Each baseline row is the ordinary denoiser path, denoted by glg=0g_lg=0 in the table. Figure 3: Guidance and loop-count sweeps. Both panels use Scale-RAE DiT2.4B with 25 outer steps, layers [12,27][12,27], λloop=1 _loop=1, and progress [0,0.5][0,0.5]. The guidance panel fixes K=4K=4 and varies glgg_lg over the complete 20-point grid reported in Table 11. The loop-count panel fixes glg=1g_lg=1 and varies K∈1,…,8K∈\1,…,8\; Table 9 reports the complete values. Figure 4: Layer range and active progress. Both panels use Scale-RAE DiT2.4B with 25 outer steps, K=4K=4, λloop=1 _loop=1, and glg=1g_lg=1. The layer-range panel fixes progress [0,0.5][0,0.5] and varies only [a,b][a,b]; the progress panel fixes [a,b]=[12,27][a,b]=[12,27] and varies only the active interval. The exact candidate ranges and intervals are listed in Tables 10 and 8, respectively. Table 3: Sampling-Progress Gating. This DiT2.4B comparison fixes 25 outer steps, layers [12,27][12,27], K=4K=4, λloop=1 _loop=1, and glg=1g_lg=1, and changes only the active interval among no loop, [0,1][0,1], and [0,0.5][0,0.5]. Table 4: Sparsifiers. The Scale-RAE DiT2.4B rows fix 25 outer steps, layers [12,19][12,19], K=4K=4, λloop=1 _loop=1, progress [0,0.5][0,0.5], and glg=1g_lg=1. They compare random, condition-aware, and attention-aware routing at ρsel∈0.50,0.70 _sel∈\0.50,0.70\ with cached complement reuse. Random routing samples the selected-token set uniformly; condition-aware routing selects the strongest condition-dependent AdaLN responses; and attention-aware routing selects the strongest gated attention-output responses within each latent-grid row. The RAE-SigLIP2-B rows fix 50 outer steps, layers [15,22][15,22], K=4K=4, λloop=1 _loop=1, progress [0,0.5][0,0.5], and glg=6.0g_lg=6.0. They compare random 50% routing with an image/condition split in which image tokens form the selected set and conditioning tokens form the complement; both use cached complement reuse. Table 5: VAE transfer. PixArt-alpha uses 20 outer steps, layers [14,19][14,19], K=4K=4, λloop=1 _loop=1, progress [0,0.5][0,0.5], BF16 autocast, and model guidance 4.5. FLUX.2 uses 28 outer steps, layers [32,37][32,37], the same loop settings, BF16 compute, 4-bit weights, model CPU offload, and model guidance 4.0. Dense Token Loop is unguided, and the Loop Guidance row uses glg=3.0g_lg=3.0 for both generators. Appendix C Complete Scale-RAE Ablations All ablations in this subsection use Scale-RAE Qwen1.5B/DiT2.4B. The main-paper sampling-progress, loop-count, and Loop Guidance figures use the same pinned GenEval and DPG-Bench evaluators. Their shared no-loop row uses 25 outer sampling steps. The tables below report the complete scores and per-image generation times. Outer-sampling-step choice. Although the Scale-RAE baselines with 35 and 50 outer sampling steps score higher, we use 25 outer sampling steps as the experimental default to halve the sampling budget relative to the native 50-point schedule. Table 7 reports the official scores and per-image times for all four sampler settings. Outer sampling steps GenEval↑ DPG-Bench↑ Time per image (s)↓ 50 0.4742 0.7738 2.4032 35 0.4770 0.7673 1.7575 25 0.4471 0.7656 1.3054 15 0.4418 0.7645 0.8562 Table 7: Twenty-five outer sampling steps are the default efficiency setting. All rows are unlooped Scale-RAE results. For 15, 35, and 50 outer sampling steps, times average 1,618 generation records per setting; all rows use the same pinned evaluators. Sampling-progress sweep. Table 8 gives the complete comparison of loop-active sampling-progress intervals. Progress Calls GenEval↑ DPG- Bench↑ Time/image (s)↓ baseline 0 0.4471 0.7656 1.3054 0–20% 5 0.5082 0.7942 1.6416 20–40% 5 0.5099 0.7914 1.6408 40–60% 5 0.4820 0.7816 1.6392 60–80% 5 0.4723 0.7761 1.6389 80–100% 5 0.4705 0.7684 1.6385 0–30% 8 0.5182 0.7946 1.8407 18–48% 7 0.5043 0.7933 1.7731 35–65% 7 0.4961 0.7885 1.7727 52–82% 7 0.4773 0.7794 1.7728 70–100% 8 0.4687 0.7750 1.8401 0–40% 10 0.5309 0.7999 1.9771 15–55% 10 0.5286 0.7972 1.9758 30–70% 9 0.4993 0.7916 1.9080 45–85% 10 0.4984 0.7831 1.9759 60–100% 10 0.4784 0.7780 1.9767 0–50% 13 0.5422 0.8007 2.1812 25–75% 13 0.4907 0.7956 2.1856 50–100% 13 0.4946 0.7831 2.1849 0–100% 25 0.5597 0.7987 2.9957 Table 8: Earlier and wider loop-active sampling-progress intervals perform best. Progress runs from the initial noise state (0%) to the data endpoint (100%). All scores use the same pinned evaluators, and times are matched-H100 generation seconds per image. Loop-count sweep. Table 9 reports the complete range of loop counts from K=1K=1 through K=8K=8. Loop Count GenEval↑ DPG-Bench↑ Time per image (s)↓ 1 0.4471 0.7656 1.3054 2 0.5180 0.7977 1.5949 3 0.5278 0.7962 1.8881 4 0.5422 0.8007 2.1812 5 0.5322 0.7999 2.4744 6 0.5308 0.7996 2.7676 7 0.5268 0.7990 3.0607 8 0.5376 0.8037 3.3537 Table 9: Moderate loop counts give the best quality–time balance. GenEval peaks at K=4K=4, DPG-Bench peaks at K=8K=8, and measured time per image increases nearly linearly. Layer-range sweep. Table 10 reports the complete layer-range ablation. #Layers Layer range GenEval↑ DPG-Bench↑ 0 baseline 0.4471 0.7656 4 28–31 0.5090 0.7807 4 18–21 0.4990 0.7889 4 10–13 0.4617 0.7726 4 0–3 0.4605 0.7650 8 16–23 0.5282 0.8009 8 24–31 0.5421 0.7850 8 12–19 0.5113 0.7942 8 8–15 0.4611 0.7767 8 0–7 0.4592 0.7695 16 12–27 0.5422 0.8007 16 16–31 0.5407 0.8049 16 4–19 0.5249 0.7978 16 0–15 0.4735 0.7773 32 0–31 0.5395 0.8087 Table 10: Mid-to-late layer ranges work best. Both metrics are computed with the same pinned evaluators. GenEval layer-range and sampling-progress panels. Figure 4 provides the GenEval counterparts to the main-paper DPG-Bench ablations. Figure 4: GenEval counterparts for the loop layer range and loop-active sampling-progress interval ablations on Scale-RAE (DiT2.4B). Mid-to-late layer ranges and earlier sampling-progress intervals give the strongest gains, consistent with the DPG-Bench results in the main paper. Loop Guidance sweep. Table 11 reports every guidance scale covered by the complete evaluation. glgg_lg GenEval↑ DPG-Bench↑ 0.0 0.4471 0.7656 0.5 0.5131 0.7895 1.0 0.5405 0.8011 1.5 0.5530 0.8065 2.0 0.5582 0.8083 2.4 0.5577 0.8019 2.8 0.5691 0.8053 3.2 0.5534 0.8035 3.6 0.5520 0.8028 4.0 0.5523 0.8037 4.5 0.5425 0.8025 5.0 0.5456 0.8030 5.5 0.5477 0.8035 6.0 0.5399 0.8011 7.0 0.5277 0.7977 8.0 0.5220 0.7960 9.0 0.5176 0.7825 10.0 0.5236 0.7801 11.0 0.5169 0.7759 12.0 0.5093 0.7706 Table 11: Moderate Loop Guidance gives the strongest gains. GenEval peaks at glg=2.8g_lg=2.8, while DPG-Bench peaks at glg=2.0g_lg=2.0; all entries use the same pinned evaluators. Appendix D Other Experimental Results RAE-SigLIP2-B layer-range sweep. Table 12 reports the complete layer-range sweep underlying the direct-loop result in the main paper, and Table 13 reports the corresponding Loop Guidance sweep. We use the DiffusionBench RAE-SigLIP2-B/DDT proxy identified under Models and Checkpoints. All variants use 50 outer sampling steps with Euler updates, disabled CFG, the loop-active sampling-progress interval si∈[0,0.5]s_i∈[0,0.5], and dense token updates. Layer range Loop Count GenEval↑ DPG-Bench↑ No loop 1 0.3829 0.7131 0–3 4 0.0000 0.2382 6–9 4 0.3789 0.7135 12–15 4 0.3764 0.7174 18–21 4 0.3873 0.7172 24–27 4 0.0644 0.4093 0–7 4 0.0021 0.2379 5–12 4 0.3678 0.7195 10–17 4 0.3806 0.7170 15–22 4 0.3795 0.7226 20–27 4 0.0685 0.4048 Table 12: One middle RAE layer range improves both primary benchmarks. Several other middle ranges raise DPG-Bench while slightly lowering GenEval, and extreme early and late ranges collapse; all entries are absolute standard scores from the pinned GenEval and DPG-Bench evaluator protocols. RAE-SigLIP2-B Loop Guidance. The sweep fixes loop layer range [15,22][15,22] with a loop count of K=4K=4. Increasing glgg_lg generally strengthens both metrics: glg=9g_lg=9 maximizes GenEval, while glg=7.5g_lg=7.5 gives the highest DPG-Bench score. glgg_lg GenEval↑ DPG-Bench↑ 0.00 0.3829 0.7131 1.50 0.3787 0.7271 2.25 0.3962 0.7286 3.00 0.3892 0.7350 3.75 0.3983 0.7397 4.50 0.4120 0.7393 5.25 0.4094 0.7495 6.00 0.4217 0.7468 6.75 0.4220 0.7483 7.50 0.4274 0.7526 8.25 0.4308 0.7487 9.00 0.4532 0.7521 Table 13: Loop Guidance substantially improves both DiffusionBench RAE-SigLIP2-B scores. The glg=0g_lg=0 row is the no-loop prediction; all entries use the same pinned evaluators. Additional Qualitative Results Subject-level corrections. Manual review of prompt-matched pairs provides a preliminary characterization of the correction effect. On GenEval and DPG-Bench, whose prompts are dominated by a single subject or a pair of subjects, looping most consistently corrects subject-level appearance: malformed or incomplete shapes become well formed, spurious parts and duplicated features are removed, and attributes such as color, material, and style are more cleanly bound to their intended entity instead of bleeding into the background or a neighboring object. The sample as a whole therefore looks closer to the prompt intent, which is consistent with the GenEval and DPG-Bench score gains. Figure 5 provides additional prompt-matched examples. w/o Loop Dense Token Loop Loop Guidance w/o Loop Dense Token Loop Loop Guidance GenEval DPG-Bench Figure 5: Additional prompt-matched qualitative comparisons on Scale-RAE (DiT9.8B). Dense Token Loop uses loop layer range [12,27][12,27], a loop count of K=4K=4, and loop-active sampling-progress interval si∈[0,0.5]s_i∈[0,0.5]; Loop Guidance uses glg=6.0g_lg=6.0. GenEval2 findings. On GenEval2, whose prompts stress object counts, positional relations, and deliberately unconventional phrasing, Table 14 reports official soft-TIFA-style geometric-mean scores over 800 prompts (Hu et al. 2023b). Dense Token Loop changes the score from 0.18010.1801 to 0.17360.1736, while Loop Guidance reaches 0.14410.1441. Thus, neither variant improves the aggregate GenEval2 score. Nevertheless, manual review of matched triplets (w/o Loop, Dense Token Loop, and Loop Guidance) reveals a structured pattern, summarized in Figure 6. On prompts with one or two subjects, looping gives the same correction benefit as on GenEval and DPG-Bench: objects are better formed and attributes are bound more cleanly. On counting prompts, the requested object count is usually inherited from the no-loop sample rather than corrected. On crowded multi-object prompts, individual objects can remain better formed even when their global arrangement does not improve. Looped samples also sometimes weaken or remove the requested background while purifying the foreground subject. Method GenEval2↑ w/o Loop 0.1801 Dense Token Loop 0.1736 Loop Guidance 0.1441 Table 14: GenEval2 results on Scale-RAE (DiT9.8B) over 800 prompts. Dense Token Loop uses loop layer range [12,27][12,27], a loop count of K=4K=4, and loop-active sampling-progress interval si∈[0,0.5]s_i∈[0,0.5]; Loop Guidance additionally uses glg=6.0g_lg=6.0. Figure 6: GenEval2 qualitative correction study on Scale-RAE (DiT9.8B). Each row inside a panel shows the prompt followed by w/o Loop, Dense Token Loop (loop layer range [12,27][12,27], a loop count of K=4K=4, and sampling progress si∈[0,0.5]s_i∈[0,0.5]), and Loop Guidance (glg=6.0g_lg=6.0). Top left: few-subject prompts improve. Top right: counting prompts are not corrected. Bottom left: objects in crowded multi-object prompts stay intact. Bottom right: subjects become purer with weaker backgrounds. Appendix E Reproducibility Code and Data Supplement. The anonymous Code and Data Supplement contains the backend-neutral loop runtime, adapters for Scale-RAE, the public RAE-SigLIP2-B/DDT proxy, PixArt-alpha, and FLUX.2, exact JSON configurations, complete benchmark prompt snapshots, benchmark generation/export/aggregation utilities, CPU-only tests, and the aggregate values used by the paper tables. The archive excludes pretrained weights, generated answers and image grids, per-prompt evaluator outputs, credentials, caches, logs, and machine-specific paths. After installing the environment described in ENVIRONMENT.md, the commands python scripts/validate_release.py and python -m unittest discover tests validate the archive without loading a generator. REPRODUCE.md then gives the backend integration and benchmark procedure. Model checkpoints and official benchmark inputs are obtained separately using the public identifiers in configs/. Composable method and hyperparameter interface. The released resolver treats the token-loop base and Loop Guidance as orthogonal choices. --token-loop dense|sparse selects Dense Token Loop or Sparse Token Loop, while --loop-guidance may be composed with either base and never changes its token-loop type. Tables 15 and 16 list the complete shared interface for adjustable paper controls. Scale-RAE and RAEv2 expose both token-loop bases with or without Loop Guidance. The PixArt-α and FLUX.2 paper presets expose Dense Token Loop with or without Loop Guidance because Sparse Token Loop was not evaluated for those backends; unsupported combinations are rejected instead of being silently relabeled. Paper control JSON field Shared CLI override Semantics and accepted values Outer sampler steps N generation.num_ inference_steps --outer-steps N Positive integer; number of outer denoising steps. Loop count K num_loops --loop-count K Positive integer; inner updates at each active outer step. Total loop strength λloop _loop lambda_value --lambda-loop VALUE Positive scalar; the runtime applies λloop/K _loop/K per inner round. Loop-active sampling interval start_frac, end_frac --loop-active START END 0≤START≤END≤10≤ START≤ END≤ 1; both endpoints are active. Loop layer range [a,b][a,b] block_indices --loop-layers FIRST LAST Inclusive, contiguous, ascending, nonnegative layer indices. Token-loop base token_operator --token-loop none|dense|sparse Ordinary path, Dense Token Loop, or Sparse Token Loop. Loop Guidance switch loopguidance_enabled --loop-guidance Boolean modifier; valid with either Dense or Sparse Token Loop. Loop Guidance scale glgg_lg loopguidance_weight --loop-guidance- weight VALUE 0≤glg≤180≤ g_lg≤ 18; requires --loop-guidance. Table 15: Released code-facing core hyperparameter interface. The JSON files in configs/ are the exact paper presets. A CLI override is applied to a copy and sets paper_preset=false in the resolved record, preventing a custom run from being mistaken for a reported configuration. Sparse control JSON field Shared CLI override Semantics and accepted values Selector selector --selector NAME all, random, condition_aware, attention_aware, or image_condition_split. Selected-token fraction ρsel _sel selection_ratio --selection-ratio RATIO 0<ρsel≤10< _sel≤ 1 when required by the sparse selector. Sparse-routing seed selector_seed --selector-seed SEED Integer seed for reproducible token routing. Eligible token domain token_domain --token-domain DOMAIN backend_default, all_tokens, image_prefix_only, or image_vs_condition. Table 16: Sparse Token Loop routing controls. Unsupported backend/selector combinations fail validation before model loading. Evaluator and artifact integrity. Every reported Scale-RAE ablation row is rescored from its bound image grid with the same pinned evaluators. GenEval uses the pristine evaluator without pre-upscaling, while DPG-Bench fixes the CSV parser, one image per prompt, native evaluation resolution, and the local mPLUG checkpoint. Incomplete, superseded, or evaluator-incompatible artifacts are excluded from the tables so evaluator drift is not interpreted as a method improvement.

Instant research discovery