Paper deep dive
Rank-Factorized Implicit Neural Bias: Scaling Super-Resolution Transformer with FlashAttention
Dongheon Lee, Seokju Yun, Jaegyun Im, Youngmin Ro
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/13/2026, 12:15:09 AM
Summary
The paper introduces Rank-factorized Implicit Neural Bias (RIB), a novel method to enable FlashAttention in Super-Resolution (SR) Transformers by replacing traditional relative positional bias (RPB). RIB uses low-rank implicit neural representations to decouple spatial priors from pixel content, allowing for efficient dot-product attention. Combined with Convolutional Local Attention (CLA) and a cyclic window strategy, the proposed Scalable SR Transformer (SST) achieves state-of-the-art performance with significantly reduced training and inference costs.
Entities (5)
Relation Signals (3)
RIB → enables → FlashAttention
confidence 98% · RIB, an alternative to RPB that enables FlashAttention in SR Transformers
SST → incorporates → RIB
confidence 95% · Building on our proposals, we introduce a FlashAttention-accelerated SR Transformer, dubbed Scalable SR Transformer (SST).
SST → uses → CLA
confidence 95% · we propose a Convolutional Local Attention (CLA) to focus self-attention on a larger context
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent Super-Resolution~(SR) methods mainly adopt Transformers for their strong long-range modeling capability and exceptional representational capacity. However, most SR Transformers rely heavily on relative positional bias~(RPB), which prevents them from leveraging hardware-efficient attention kernels such as FlashAttention. This limitation imposes a prohibitive computational burden during both training and inference, severely restricting attempts to scale SR Transformers by enlarging the training patch size or the self-attention window. Consequently, unlike other domains that actively exploit the inherent scalability of Transformers, SR Transformers remain heavily focused on effectively utilizing limited receptive fields. In this paper, we propose Rank-factorized Implicit Neural Bias~(RIB), an alternative to RPB that enables FlashAttention in SR Transformers. Specifically, RIB approximates positional bias using low-rank implicit neural representations and concatenates them with pixel content tokens in a channel-wise manner, turning the element-wise bias addition in attention score computation into a dot-product operation. Further, we introduce a convolutional local attention and a cyclic window strategy to fully leverage the advantages of long-range interactions enabled by RIB and FlashAttention. We enlarge the window size up to \textbf{96$\times$96} while jointly scaling the training patch size and the dataset size, maximizing the benefits of Transformers in the SR task. As a result, our network achieves \textbf{35.63\,dB PSNR} on Urban100$\times$2, while reducing training and inference time by \textbf{2.1$\times$} and \textbf{2.9$\times$}, respectively, compared to the RPB-based SR Transformer~(PFT).
Tags
Links
- Source: https://arxiv.org/abs/2603.06738v1
- Canonical: https://arxiv.org/abs/2603.06738v1
Trouble viewing inline? Open PDF directly →
Full Text
73,044 characters extracted from source content.
Expand or collapse full text
Rank-Factorized Implicit Neural Bias: Scaling Super-Resolution Transformer with FlashAttention Dongheon Lee, Seokju Yun, Jaegyun Im, and Youngmin Ro ⋆ Machine Intelligence Laboratory, University of Seoul, Korea dslisleedh, wsz871, imij0522, youngmin.ro@uos.ac.kr Code: https://github.com/dslisleedh/SST Enlarged Receptive FieldEfficient Training/Inference Improved Performance 35.24 31.08 28.71 35.2 31.13 28.75 28.5 29.5 30.5 31.5 32.5 33.5 34.5 35.5 PFTMambaIRV2-L RIN-L+ Urban100×2 Urban100×3 Urban100×4 644.9ms 2308.8ms 1920.8 05001000150020002500 2989MB 5816MB 28869MB 050001000015000200002500030000 Latency Memory Sec/Iter Memory 0.672s 1.21s 1.41s 00.511.5 67.7GB 89.8GB 85.4GB 020406080100 Training Metrics Inference Metrics ×3.6 Faster ×9.7 Lower ×2.1 Faster 24.6% Reduced 35.63 (+0.39) 31.53 (+0.40) 29.06 (+0.31) SST-L+ Fig. 1: Overview of our contributions. We enable FlashAttention for Super- Resolution (SR) Transformers by replacing the commonly used relative positional bias (RPB) with our proposed Rank-factorized Implicit Neural Bias (RIB). The re- sulting efficiency allows us to scale up the SR Transformer significantly, including the self-attention window size and training patch size up to 96×96, and also datasets from DF2K to DFLIP. Consequently, our network achieves remarkable performance while drastically reducing training/inference costs. Abstract. Recent Super-Resolution (SR) methods mainly adopt Trans- formers for their strong long-range modeling capability and exceptional representational capacity. However, most SR Transformers rely heavily on relative positional bias (RPB), which prevents them from leveraging hardware-efficient attention kernels such as FlashAttention. This limi- tation imposes a prohibitive computational burden during both train- ing and inference, severely restricting attempts to scale SR Transform- ers by enlarging the training patch size or the self-attention window. Consequently, unlike other domains that actively exploit the inherent scalability of Transformers, SR Transformers remain heavily focused on ⋆ Corresponding author arXiv:2603.06738v1 [cs.LG] 6 Mar 2026 2D. Lee et al. effectively utilizing limited receptive fields. In this paper, we propose Rank-factorized Implicit Neural Bias (RIB), an alternative to RPB that enables FlashAttention in SR Transformers. Specifically, RIB approxi- mates positional bias using low-rank implicit neural representations and concatenates them with pixel content tokens in a channel-wise manner, turning the element-wise bias addition in attention score computation into a dot-product operation. Further, we introduce a convolutional lo- cal attention and a cyclic window strategy to fully leverage the advan- tages of long-range interactions enabled by RIB and FlashAttention. We enlarge the window size up to 96×96 while jointly scaling the training patch size and the dataset size, maximizing the benefits of Transformers in the SR task. As a result, our network achieves 35.63 dB PSNR on Urban100×2, while reducing training and inference time by 2.1× and 2.9×, respectively, compared to the RPB-based SR Transformer (PFT). Keywords: Super-Resolution· FlashAttention· Implicit Neural Field 1 Introduction Super-Resolution (SR) aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) input and has long been a fundamental problem in com- puter vision. Modeling long-range dependencies is crucial for SR, as distant but correlated patterns, such as repeated textures and edges, can help disambiguate the inherently ill-posed LR inputs. Accordingly, Transformers [58] have recently emerged as a promising architecture for SR tasks because their core operator, self-attention, enables long-range interactions and input-adaptive feature aggre- gation, offering strong representational capacity. However, adopting Transformers for SR in practice still faces several limita- tions that prevent the full leverage of its powerful representational capability: (i) The quadratic cost of self-attention in the number of tokens makes global attention prohibitively expensive for high-resolution feature maps that consist of pixel-level tokens without a patchify stem. As a result, many SR Transformers resort to window-based attention with relatively small windows [32, 35], which restricts the capturing of long-range dependencies across distant but correlated regions. (i) Most methods are limited to training on small cropped patches (e.g., 64× 64), since leveraging larger patches (e.g., 96× 96) substantially increases the training budget, although the richer global context can improve perfor- mance [30,31,62]. (i) Many SR Transformers are still trained on relatively small datasets such as DF2K [1, 57] (3,450 images), despite the availability of much larger datasets like LSDIR [31] (84,991 images) and DiverSeg-IP [42] (526,503 images). Data scaling has driven substantial gains for Transformers in other vision domains [3,16,27,43]. Currently, many SR methods primarily mitigate the limitation (i), while the orthogonal gains from limitations (i) and (i) remain underexplored. We iden- tify SR Transformers’ heavy reliance on Relative Positional Bias (RPB) as the hidden bottleneck underlying this research trend. RPB injects a strong spatial Rank-Factorized Implicit Neural Bias3 prior into self-attention by adding a learnable, distance-dependent bias to the attention logits (S = QK ⊤ ), and is crucial for SR performance. Yet, incorporat- ing RPB typically requires either materializing the score matrix or performing additional indexing and memory reads, making it difficult to use with hardware- efficient self-attention kernels (e.g., FlashAttention [11, 12, 49]), which aim to reduce memory I/O and to avoid materializing the score matrix. This incom- patibility forces existing models to rely on slow, memory-intensive implementa- tions, which in turn hinders the effective scaling of training patch size and data. Consequently, instead of pursuing scalability, previous works have been largely confined to designing complex windowing strategies [7,8,67,69] or sub-quadratic alternatives, such as linear-complexity attention [6,7,65,69], Mamba [21,22,74], and sparse operators [36], to bypass this efficiency wall. In this paper, we propose Rank-factorized Implicit Neural Bias (RIB), a FlashAttention-compatible alternative to conventional RPB, unlocking the win- dow limits for SR Transformers. Specifically, RIB parameterizes positional bias into two low-rank implicit neural representations (Q p , K p ∈R N×R ) generated by a coordinate-based multi-layered perceptron (MLP). Then, resulting spatial priors are injected to self-attention by concatenating them to tokens predicted from pixel contents (Q c , K c ∈R N×D ), thereby memory-efficiently emulating the element-wise bias addition through dot-products (e.g., [Q c , Q p ][K c , K p ] ⊤ = Q c K ⊤ c + Q p K ⊤ p ). This novel design separates the number of bias parameters from the window size and remains compatible with any FlashAttention imple- mentation, unlike traditional RPB. Moreover, compared to Rotary Positional Embedding (RoPE) [23,53], which injects spatial prior via rotating Q c and K c , RIB explicitly decouples the pixel contents and spatial prior, thereby preserv- ing the integrity of pixel-content representations. Finally, to better exploit the larger window self-attention enabled by our RIB and FlashAttention, we pro- pose a Convolutional Local Attention (CLA) to focus self-attention on a larger context and a cyclic window strategy that periodically expands the attention window to balance long-range interaction and multi-scale feature extraction. Building on our proposals, we introduce a FlashAttention-accelerated SR Transformer, dubbed Scalable SR Transformer (SST). With a 64×64 attention window, SST achieves +0.16 dB PSNR on Urban100×2 over HAT, despite us- ing only 12M parameters compared to HAT’s 20M. We further present SST+, which scales both the attention window and the training patch size up to 96×96. Even with 8M fewer parameters, it attains +0.18 dB higher PSNR than ATD on Urban100×2, which is also trained on 96× 96 patches. Finally, by scaling the model to 20M parameters and training on the substantially larger DFLIP datasets, which consist of DIV2K, Flickr2K, LSDIR-train, and DiverSeg-IP, in- stead of the commonly used DF2K, we obtain a striking result: +0.4 dB PSNR on Urban100×3 over prior state-of-the-art (SOTA) methods trained under the same setting. Beyond notable performance improvements, our method is markedly more efficient. While leveraging 96×96 attention windows and training patches, it delivers 2.1× faster training with 24.6% lower memory than prior methods trained with 64× 64 patches. At inference time, it achieves 3.6× lower latency 4D. Lee et al. and 9.7× less memory usage. Detailed ablation studies verify that each proposed component contributes to these gains and delivers substantial efficiency gains, while visual analyses show that our approach not only restores visually pleasing outputs but also effectively leverages a larger receptive field. In summary, by making FlashAttention practical for SR Transformers and fully exploiting the resulting scalability, we achieve substantial efficiency gains and translate them into powerful performance, highlighting large-context and large-data scaling as a promising research direction, as shown in Figure 1. 2 Preliminaries In this section, we briefly review the key components relevant to our method, in- cluding general SR architectures, SR Transformers, window-based self-attention, positional bias, and FlashAttention. 2.1 SR Architecture A typical SR architecture adopts a residual-in-residual design [71] composed of a shallow feature extractor (E s ), a deep feature extractor (E d ), and an upsam- pler (U), as illustrated in Figure 2 (a). The E s is a 3× 3 convolution that maps an input LR image I LR ∈R H×W×3 , where H and W denote the height and width of I LR , respectively, to shallow features F s ∈R H×W×D , where D is the base channel dimension. The E d , instantiated with chosen core operators such as convolution [33], self-attention [32], or Mamba [22], takes F s as input and predicts deep features F d ∈R H×W×D . Finally, the U receives F ∈R H×W×D , obtained by element-wise addition of F s and F d , and outputs the reconstructed image I SR ∈R rH×rW×3 , where r denotes the upscaling factor. Some prior stud- ies further employ an image-wise skip connection [17, 30]; similarly, we add a nearest-neighbor interpolated I LR to the U’s output, treating the U as a resid- ual image predictor [26]. 2.2 SR Transformers and Window-based Self-attention Transformers leverage self-attention to model long-range dependencies from to- ken features X ∈R B×N×D , where N denotes the number of tokens. Unlike high- level vision tasks that typically tokenize image patches into tokens (e.g., 16× 16 patches in ViT) [16], SR commonly treats each pixel as a token to preserve pixel-wise fidelity, leading to a much larger N = HW. To clarify the compu- tational bottleneck induced by pixel-level tokenization, we briefly review scaled dot-product self-attention, considering only a single head. Self-attention projects X into queries, keys, and values: Q = XW q , K = XW k , and V = XW v , where W q , W k , W v ∈R D×D . It then computes the scaled similarity matrix S = QK ⊤ / √ D ∈R B×N×N , applies SoftMax along the token dimension to ob- tain attention weights P = SoftMax(S) ∈R B×N×N , and aggregates values as O = PV∈R B×N×D . Rank-Factorized Implicit Neural Bias5 Upsampler ( 풰 ) 퐼 !" 퐼 #" Deep Feat. Extractor ( ℰ ! ) Shallow Feat. Extractor ( ℰ " ) (a) N Interpolation (b) 퐏w/ RoPE퐏w/o RoPE 퐐 index 퐊index퐊index Fig. 2: Preliminaries and motivation. (a) Overview of the general SR architecture. (b) Comparison of the attention probability matrix P = SoftMax(QK ⊤ / √ D) with (right) and without (left) RoPE [23]. For intuition, P is computed in a toy setting where Q = K is obtained by sampling random two orthogonal 32-dimensional vectors and spatially tiling them to form a 32× 32 feature map. Typical SR Transformers set D to 30 [5,32,76]. With RoPE, similarity between repeated patterns becomes unstable and often attenuates as the spatial offset increases, or vice versa, due to phase-wrapping effects, as highlighted by the red arrow. This input-adaptive global aggregation provides strong representational power, but incurs quadratic computational/memory complexity O(N 2 D). Moreover, the aforementioned pixel-level tokenization makes global attention prohibitively ex- pensive due to the large number of tokens. For example, a ViT with a 224× 224 input and 16× 16 patches processes N = (224/16) 2 = 196 tokens, whereas a ×2 SR Transformer for a 1280× 720 output takes a 640× 360 LR input and processes N = 640× 360 = 230,400 tokens. As a result, most SR Transformers adopt window-based self-attention, computing attention only within local M×M windows (e.g., M ∈8, 16, 32), which reduces complexity to O(NM 2 D). While this windowing strategy substantially improves efficiency, it also limits direct long-range interactions, leaving SR Transformers to balance computational cost against long-range modeling. 2.3 FlashAttention and Positional Bias In language modeling tasks [18,63], where models must process huge numbers of tokens, hardware-efficient attention kernels such as FlashAttention [11,12,49] are widely used to mitigate the memory bottlenecks of long-context self-attention. Technically, FlashAttention improves both memory usage and runtime by using IO-aware tiling and kernel fusion to compute exact self-attention without mate- rializing the full N × N matrices (S and P), thereby notably reducing memory traffic. These optimizations significantly accelerate both training and inference speed compared to the standard implementations, and have been instrumental in scaling Transformers. However, FlashAttention cannot be directly applied to many SR Transform- ers due to the use of RPB [35,47], which provides a strong spatial prior and often yields notable performance gains. Specifically, RPB adds a distance-dependent bias matrix B ∈R 1×N×N to the attention logits, i.e., ̃ S = S + B. Therefore, 6D. Lee et al. 퐊 ! ∈ℝ "×$ 퐐 ! ∈ ℝ " × $ Implicit Neural Field of Positional Bias Inner Loops Outer Loops 퐊=[퐊 ! ,퐊 " ]∈ℝ #×(&'() 퐐 = [ 퐐 ! , 퐐 " ] ∈ ℝ # × ( & ' ( ) Transpose 퐕 ∈ ℝ # × & 퐎∈ℝ "×% Scaling 2D Coordinates Fourier Features MLP w/ ReLU ... ... ... ... Self-Attention with Online SoftMax Spatial Prior (RIB): 퐐 ! ,퐊 ! Pixel Contents: 퐐 & ,퐊 & Tokens from Fig. 3: Overall illustration for proposed Rank-factorized Implicit Neural Bias (RIB). incorporating RPB either requires explicitly materializing an additional N × N matrix or performing extra memory accesses and indexing into a bias table to add the corresponding biases [15, 30]. Such operations break the assumptions behind many fused FlashAttention kernels, and therefore are not supported in a large portion of existing FlashAttention implementations. In contrast, modern Large Language Models (LLMs) often adopt a RoPE [53], which is fully compatible with FlashAttention. RoPE injects positional prior by applying distance-dependent rotations to Q and K, so that relative phase dif- ferences are reflected during the dot-product computation of S. Since RoPE does not require materializing or adding an extra N × N bias, RoPE can be seamlessly used with FlashAttention kernels, and recent works have also ex- plored its adaptation to vision transformers [23]. Nevertheless, as illustrated in Figure 2 (b), RoPE can substantially weaken similarities between repeated pat- terns at large spatial offsets, which is crucial for the SR task, due to phase wrap/aliasing especially under small head dimensions (e.g., ≈ 32). Moreover, since RoPE injects positional information by directly modulating the dot prod- uct with rotated Q/K, the positional effect is entangled with the pixel contents. Consequently, compensating for undesired distance-dependent suppression may require altering the Q/K content representations themselves, consuming fitting capacity and potentially degrading performance. This motivates us to introduce a FlashAttention-compatible bias that explicitly decouples the spatial prior from the pixel content, similar to the RPB. 3 Proposed Methods In this section, we describe our proposed methods, including a Rank-factorized Implicit Neural Bias (RIB), a Convolutional Local Attention (CLA), and a cyclic window strategy. Rank-Factorized Implicit Neural Bias7 3.1 Rank-factorized Implicit Neural Bias The main goal of the proposed RIB is to provide a positional bias for self- attention that explicitly disentangles pixel content from spatial priors, while remaining fully compatible with FlashAttention. To this end, we parameter- ize positional bias using per-token 2D coordinates and their Fourier features. Specifically, given a window with N tokens, we assign each token a normalized 2D coordinate x∈ [−1, 1] N×2 . We augment these coordinates with a Fourier fea- ture mapping [41] to obtain the coordinate embedding r in ∈R N×(2+4L) , which is defined as: r in = γ(x) = h x, sin 2 0 x , cos 2 0 x , ..., sin 2 L−1 x , cos 2 L−1 x i , (1) where [·] denotes concatenation along the feature dimension, and sin(·) and cos(·) are applied element-wise. L is the number of frequency bands in the positional encoding. Next, we feed r in into a lightweight MLP with ReLU activation to produce low-rank positional representations that parameterize an implicit neural field for positional bias. Specifically, we first compute a shared hidden represen- tation h∈R N×d h as: h = ReLU(r in W h + b h ),(2) where W h ∈R (2+4L)×d h and b h ∈R d h . We then linearly project h into a low- rank space of dimension R, as follows: Q p = hW p,q ,K p = hW p,k ,(3) where W p,q , W p,k ∈R d h ×R . In practice, we share the hidden representationh across all attention heads, and obtain head-wiseQ p andK p via separate linear projections from the shared h. Next, unlike vanilla self-attention that directly forms Q = XW q and K = XW k , we explicitly separate content and positional components. Given token features X, we define the content projections Q c = XW q ,K c = XW k ,(4) while keeping the value projection identical to standard attention (V = XW v ). Then, we merge content and positional components by channel-wise concate- nation. Specifically, we scale Q c and Q p by their respective dimensions and construct Q = h Q c / √ D , Q p / √ R i ,K = h K c , K p i .(5) The resulting scaled similarity matrix is obtained as a single dot-product in the augmented channel space: S = QK ⊤ = Q c K ⊤ c / √ D |z content term + Q p K ⊤ p / √ R | z bias term ,(6) which shows that the spatial prior is injected as an additive bias term in the logits while remaining decoupled from the pixel content term in logit space. 8D. Lee et al. The subsequent steps operate identically to standard self-attention (e.g., O = SoftMax(S)V). In summary, RIB is fully compatible with existing FlashAttention imple- mentations, similar to RoPE, because it injects positional priors by augmenting QandKand computing the logits via a single dot-product, without requiring any extra N × Nbias matrix materialization or table indexing. Moreover, un- like RPB, whose number of bias parameters grows with the window size (e.g., (2M− 1) 2 ≈ O(M 2 )), RIB parameterizes the positional bias using a lightweight MLP, making the number of bias parameters independent of the window size (e.g., O(d h (L + R))). Finally, sinceQ p andK p depend only on the window ge- ometry (normalized coordinates) and not on the input pixel contents, they can be precomputed and cached, further reducing inference-time overhead. 3.2 Convolutional Local Attention Due to the low-rank nature of the RIB, it may be less effective for certain highly localized, rapidly varying positional patterns. We therefore further enhance RIB with a CLA that is designed to strengthen short-range interactions and help capture fine details. Technically, given the token features X, we define a pair of reshaping operators F and F −1 that convert between token features and 2D feature maps: F :R B×H×W×D →R B×N×D , F −1 :R B×N×D →R B×H×W×D .(7) We first restore X to an image-shaped 2D feature map ̃ X to enable convolution: ̃ X =F −1 (X)∈R B×H×W×D .(8) CLA then generates a locally informed gating map via a lightweight convolu- tional path consisting of a 3 × 3 depth-wise convolution, a 1 × 1 point-wise convolution, and a sigmoid activation: ̃ G = σ PWConv DWConv 3×3 ( ̃ X) ∈R B×H×W×D .(9) The resulting gating map is then converted back to token form to align with the attention output: G =F( ̃ G)∈R B×N×D .(10) We then modulate the self-attention output O ∈R B×N×D by the gating map via element-wise multiplication: O ′ = O⊙ G,(11) where DWConv 3×3 and PWConv denote depth-wise and point-wise convolu- tions, σ(·) is the sigmoid activation, and ⊙ denotes element-wise multiplication. In summary, CLA selectively emphasizes spatially consistent high-frequency responses, aiding self-attention with RIB to focus on larger and broader patterns. Moreover, compared to the output projection of standard self-attention (Y = OW o ), where W o denotes output projection weight matrix, CLA provides ad- ditional non-linearity before output projection (Y = (O⊙ G)W o ), further en- hancing the representational capability of Transformers [46]. Rank-Factorized Implicit Neural Bias9 3.3 Cyclic Window Strategy Thanks to the memory efficiency enabled by FlashAttention and our proposed RIB, we can employ substantially larger attention windows without incurring prohibitive memory overhead. Nevertheless, prior works have highlighted that combining relatively small windows is beneficial for multi-scale feature extrac- tion [69,70]. Motivated by this, we propose a simple cyclic window strategy that balances local detail modeling and long-range interaction by periodically vary- ing the window size across consecutive self-attentions. Specifically, we cycle the window size from small to large and repeat the cycle twice within a block (e.g. 16, 32, 64, 16, 32, 64). This periodic expansion provides repeated global mixing while preserving local refinements. 4 Experiments We evaluate our method in terms of (i) training and inference efficiency, (i) quan- titative SR performance, (i) ablations, and (iv) qualitative analyses, including local attribution maps (LAM), positional bias visualization, and visual results. We introduce our SR Transformer using a 64× 64 attention window, termed as SST (12M), and a parameter-scaled-up variant SST-L (20M). Models using a larger attention window (up to 96× 96) are denoted with "+". Since SR methods are typically trained on 64× 64 cropped patches, naively applying a 96× 96 window would introduce substantial padding to match the feature size to the window size, which can destabilize optimization. Therefore, we train the "+" variants with 96 × 96 training patches, and this setting is marked with " † ". Despite the larger training patches, our method remains faster and more memory-efficient than prior methods trained with 64× 64 patches, thanks to FlashAttention enabled by RIB, as shown in Section 4.1. Additional implementation details and results on lightweight networks are provided in the supplementary material. 4.1 Comparisons of Training and Inference Costs We first compare the training and inference costs of our method against represen- tative SR baselines to highlight the efficiency gains enabled by FlashAttention. As reported in Table 1, SST-L achieves the lowest training time and memory footprint, as well as lower inference latency and memory, despite leveraging a much larger attention window. This demonstrates that FlashAttention can pro- vide substantial end-to-end benefits. Next, we compare SST-L+ with ATD under the same 96×96 training patch setting. SST-L+ is 1.92× faster in training while using 1.9× less training memory. This result indicates that naively increasing the training patch size without an appropriate hardware-efficient kernel can intro- duce significant overhead, and it reinforces the importance of FlashAttention for scaling SR Transformers. We also compare SST-L+ with PFT. Although SST- L+ requires 5.7× more FLOPs, it uses about 10× less inference memory and 10D. Lee et al. Table 1: Comparison of training and inference costs across SR methods. Training costs are measured using the batch size of 10, which is the largest batch size that does not cause an OOM in all cases. Inference costs are measured while reconstructing a 1280×720 image at scale ×2. All statistics are measured using an H200 GPU at FP32 precision. The best result is bolded. We leverage FlashAttention3 [49] for the hardware-efficient acceleration kernel. For more comparisons across various GPUs and FlashAttention implementations, please refer to the supplementary materials. Method TrainingInference patch size sec/step memory#FLOPs #params latency memory HAT [5]64× 64 0.432 32.4GB5.81T 20.6M 709.9ms 9070MB ATD [68] 96× 96 1.291 132.5GB6.07T 20.1M 1266.8ms 6239MB PFT [36]64× 64 1.410 85.4GB5.03T 19.6M 1920.8ms 28869MB MambaIR [22]64× 64 0.791 50.0GB5.87T 20.4M 1043.8ms 7704MB MambaIRV2-L [21] 64× 64 1.210 89.8GB9.42T 34.1M 2308.8ms 5816MB SST-L64× 64 0.370 31.4GB18.8T 20.3M 608.4ms 2823MB SST-L+ 96× 96 0.672 67.7GB28.7T 20.3M 644.9ms 2989MB achieves roughly 3× lower latency. We argue that this discrepancy stems from the fact that FLOPs reductions do not necessarily reflect practical efficiency gains [4, 37, 64]. Although PFT reduces FLOPs through top-k sparsification, it still needs to materialize N × N and N × K matrices for each window (includ- ing shifted-window attention), such as the previous layer’s similarity matrix S and sparsification index matrix I. As a result, PFT is primarily bottlenecked by memory traffic rather than computation, which leads to higher latency and increased memory consumption. Finally, even when processing high-resolution feature maps (e.g., 640×360 features for 1280×720 reconstruction at×2), SST- L+ remains more efficient in inference than Mamba-based (linear-operator) SR methods, underscoring the practicality and promise of FlashAttention-equipped Transformers. 4.2 Quantitative Results Next, we conduct a quantitative comparison to demonstrate the strong perfor- mance of our method. For this comparison, we crop the boundary of the I SR by the upscaling factor, convert the result to the YCbCr color space, and then compute PSNR and SSIM on the Y channel, following standard convention. Please note that we compare against methods for which both the official code and pretrained weights are publicly available. Results on DF2K Datasets We compare our proposed networks (SST, SST+, SST-L, and SST-L+), trained on the DF2K dataset [1,57], with representative SR methods employing different core operators, including Transformers [5,8,30, 32,36,67,68], a graph neural network (GNN) [56], and Mamba [21,22]. All evalu- ations are conducted on commonly used benchmark datasets, including Set5 [2], Set14 [66], BSD100 [38], Urban100 [25], and Manga109 [39]. As shown in Table 2, Rank-Factorized Implicit Neural Bias11 Table 2: Quantitative comparison (PSNR/SSIM) on benchmark datasets trained on DF2K datasets [1,57]. † denotes the method is trained using 96× 96 patches. The best result is bolded. MethodVenue scale #params Set5Set14BSD100Urban100Manga109 PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM SwinIR [32]ICCVW‘21 ×2 11.8 38.42/0.9623 34.46/0.9250 32.53/0.9041 33.81/0.9427 39.92/0.9797 ESC † [30]ICCV‘2512.5 38.59/0.9630 34.70/0.9259 32.61/0.9052 34.49/0.9466 40.38/0.9809 CAT-A [8]NeurIPS‘2216.5 38.51/0.9626 34.78/0.9265 32.59/0.9047 34.26/0.9440 40.10/0.9805 ART [67]ICLR‘2316.4 38.56/0.9629 34.59/0.9267 32.58/0.9048 34.30/0.9452 40.24/0.9808 HAT [5]CVPR‘2320.6 38.63/0.9630 34.86/0.9274 32.62/0.9053 34.45/0.9466 40.26/0.9809 ATD † [68]CVPR‘2420.1 38.61/0.9629 34.95/0.9276 32.65/0.9056 34.70/0.9476 40.37/0.9810 PFT [36]CVPR‘2519.6 38.68/0.9635 35.00/0.9280 32.67/0.9058 34.90/0.9490 40.49/0.9815 IPG [56]CVPR‘2420.4 38.61/0.9632 34.73/0.9270 32.60/0.9052 34.48/0.9464 40.24/0.9810 MambaIR [22]ECCV‘2420.4 38.57/0.9627 34.67/0.9261 32.58/0.9048 34.15/0.9446 40.28/0.9806 MambaIRV2-S [21] CVPR‘259.638.53/0.9627 34.62/0.9256 32.59/0.9048 34.24/0.9454 40.27/0.9808 MambaIRV2-B [21] CVPR‘2522.9 38.65/0.9631 34.89/0.9275 32.62/0.9053 34.49/0.9468 40.42/0.9810 MambaIRV2-L [21] CVPR‘2534.1 38.65/0.9632 34.93/0.9276 32.62/0.9053 34.60/0.9475 40.55/0.9807 SST–11.7 38.63/0.9629 34.72/0.9258 32.55/0.9042 34.61/0.9474 40.31/0.9808 SST+ † –11.7 38.63/0.9631 34.89/0.9273 32.62/0.9053 34.88/0.9488 40.44/0.9812 SST-L–20.3 38.65/0.9631 34.86/0.9270 32.60/0.9050 34.78/0.9485 40.44/0.9812 SST-L+ † –20.3 38.64/0.9630 34.94/0.9273 32.61/0.9050 35.01/0.9497 40.56/0.9815 SwinIR [32]ICCVW‘21 ×3 11.9 34.97/0.9318 30.93/0.8534 29.46/0.8145 29.75/0.8826 35.12/0.9537 ESC † [30]ICCV‘2512.5 35.14/0.9330 31.10/0.8552 29.53/0.8167 30.23/0.8895 35.60/0.9555 CAT-A [8]NeurIPS‘2216.6 35.06/0.9326 31.04/0.8538 29.52/0.8160 30.12/0.8862 35.38/0.9546 ART [67]ICLR‘2316.6 35.07/0.9325 31.02/0.8541 29.51/0.8159 30.10/0.8871 35.39/0.9548 HAT [5]CVPR‘2320.8 35.07/0.9329 31.08/0.8555 29.54/0.8167 30.23/0.8896 35.53/0.9552 ATD † [68]CVPR‘2420.3 35.11/0.9330 31.13/0.8556 29.57/0.8176 30.46/0.8917 35.63/0.9558 PFT [36]CVPR‘2519.8 35.15/0.9333 31.16/0.8561 29.58/0.8178 30.56/0.8931 35.67/0.9560 IPG [56]CVPR‘2418.3 35.10/0.9332 31.10/0.8554 29.53/0.8168 30.36/0.8901 35.53/0.9554 MambaIR [22]ECCV‘2420.4 35.08/0.9323 30.99/0.8536 29.51/0.8157 29.93/0.8841 35.43/0.9546 MambaIRV2-S [21] CVPR‘259.835.09/0.9326 31.07/0.8547 29.51/0.8157 30.08/0.8871 35.44/0.9549 MambaIRV2-B [21] CVPR‘2523.1 35.18/0.9334 31.12/0.8557 29.55/0.8169 30.28/0.8905 35.61/0.9556 MambaIRV2-L [21] CVPR‘2534.2 35.16/0.9334 31.18/0.8564 29.57/0.8175 30.34/0.8912 35.72/0.9561 SST–11.9 35.10/0.9323 31.00/0.8533 29.46/0.8149 30.35/0.8909 35.56/0.9554 SST+ † –11.9 35.19/0.9335 31.08/0.8553 29.54/0.8170 30.59/0.8941 35.66/0.9560 SST-L–20.7 35.13/0.9328 31.02/0.8544 29.47/0.8152 30.52/0.8934 35.61/0.9557 SST-L+ † –20.7 35.25/0.9337 31.16/0.8550 29.56/0.8174 30.75/0.8962 35.83/0.9568 SwinIR [32]ICCVW‘21 ×4 11.9 32.92/0.9044 29.09/0.7950 27.92/0.7489 27.45/0.8254 32.03/0.9260 ESC † [30]ICCV‘2512.5 33.00/0.9054 29.21/0.7968 27.95/0.7504 27.89/0.8351 32.54/0.9295 CAT-A [8]NeurIPS‘2216.6 33.08/0.9052 29.18/0.7960 27.99/0.7510 27.89/0.8339 32.39/0.9285 ART [67]ICLR‘2316.6 33.04/0.9051 29.16/0.7958 27.97/0.7510 27.77/0.8321 32.31/0.9283 HAT [5]CVPR‘2320.8 33.04/0.9056 29.23/0.7973 28.00/0.7517 27.97/0.8368 32.48/0.9292 ATD † [68]CVPR‘2420.3 33.10/0.9058 29.24/0.7974 28.01/0.7526 28.17/0.8404 32.62/0.9306 PFT [36]CVPR‘2519.8 33.15/0.9065 29.29/0.7978 28.02/0.7527 28.20/0.8412 32.63/0.9306 IPG [56]CVPR‘2417.0 33.15/0.9062 29.24/0.7973 27.99/0.7519 28.13/0.8392 32.53/0.9300 MambaIR [22]ECCV‘2420.4 33.03/0.9046 29.20/0.7961 27.98/0.7503 27.68/0.8287 32.32/0.9272 MambaIRV2-S [21] CVPR‘259.832.99/0.9037 29.23/0.7965 27.97/0.7502 27.73/0.8307 32.33/0.9276 MambaIRV2-B [21] CVPR‘2523.1 33.14/0.9057 29.23/0.7975 28.00/0.7511 27.89/0.8344 32.57/0.9295 MambaIRV2-L [21] CVPR‘2534.2 33.19/0.9062 29.29/0.7982 28.01/0.7521 28.07/0.8383 32.66/0.9304 SST–12.1 33.15/0.9059 29.21/0.7966 27.96/0.7505 28.08/0.8389 32.52/0.9295 SST+ † –12.1 33.00/0.9054 29.21/0.7971 27.96/0.7511 28.23/0.8427 32.66/0.9308 SST-L–21.3 33.08/0.9056 29.25/0.7980 27.96/0.7511 28.23/0.8432 32.64/0.9305 SST-L+ † –21.3 33.12/0.9057 29.23/0.7973 27.98/0.7512 28.39/0.8466 32.85/0.9322 SST achieves superior performance on Urban100×2, outperforming HAT (20M parameters) by 0.16 dB while using only 12M parameters, demonstrating the effectiveness of our 64× 64 large-window attention. Furthermore, when trained with 96 × 96 patches, SST+ (12M parameters) surpasses ATD (20M param- eters) trained under the same setting by 0.18 dB on Urban100×2, indicating that SST+ more effectively exploits the benefits of 96× 96 large-window atten- tion. The parameter-scaled variants up to 20M (SST-L and SST-L+) consistently 12D. Lee et al. Table 3: Quantitative comparison (PSNR/SSIM) on benchmark datasets trained on DFLIP datasets [1,31,42,57]. † denotes the method is trained using 96× 96 patches. The best result is bolded. Methodscale #params Set5Set14BSD100Urban100Manga109DIV2K-val LSDIR-val PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM PSNR/SSIM PFT [36] ×2 19.6 38.67/0.9633 35.11/0.9282 32.71/0.9062 35.24/0.9508 40.64/0.9817 37.17/0.9526 32.92/0.9321 MambaIRV2-L [21]34.2 38.77/0.9636 35.20/0.9287 32.74/0.9065 35.20/0.9508 40.82/0.9820 37.16/0.9524 32.93/0.9321 SST-L20.3 38.70/0.9635 35.06/0.9274 32.69/0.9058 35.35/0.9515 40.64/0.9817 37.21/0.9528 32.97/0.9325 SST-L+ † 20.3 38.78/0.9637 35.28/0.9288 32.73/0.9064 35.63/0.9530 40.86/0.9821 37.32/0.9535 33.12/0.9338 PFT [36] ×3 19.8 35.12/0.9335 31.45/0.8572 29.60/0.8188 31.08/0.8996 35.81/0.9567 33.41/0.9020 29.16/0.8578 MambaIRV2-L [21]34.2 35.23/0.9341 31.56/0.8586 29.66/0.8195 31.13/0.9000 35.98/0.9574 33.43/0.9020 29.23/0.8586 SST-L20.7 35.24/0.9339 31.47/0.8567 29.57/0.8177 31.22/0.9010 35.90/0.9570 33.45/0.9026 29.21/0.8587 SST-L+ † 20.7 35.33/0.9345 31.54/0.8576 29.64/0.8193 31.53/0.9046 36.12/0.9580 33.55/0.9037 29.33/0.8609 PFT [36] ×4 19.8 33.17/0.9071 29.50/0.8005 28.03/0.7537 28.71/0.8515 32.84/0.9323 31.38/0.8573 27.22/0.7950 MambaIRV2-L [21]34.2 33.20/0.9073 29.57/0.8018 28.11/0.7556 28.75/0.8526 32.99/0.9334 31.40/0.8576 27.29/0.7967 SST-L21.3 33.25/0.9080 29.58/0.8016 28.07/0.7541 28.90/0.8549 33.00/0.9334 31.44/0.8584 27.31/0.7972 SST-L+ † 21.3 33.10/0.9067 29.58/0.8016 28.06/0.7541 29.06/0.8583 33.18/0.9345 31.50/0.8594 27.37/0.7991 achieve strong performance on Urban100 and Manga109 across all upscaling fac- tors, showing that our methods are competitive with prior SOTA approaches. Notably, these improvements come with substantially reduced training and in- ference costs, as demonstrated in Section 4.1. Results on DFLIP Datasets Our method exhibits remarkable gains in the mid-size regime (11M parameters), even surpassing prior 20M parameter models. However, when scaling to the 20M parameter regime, our results become largely comparable to existing approaches. We hypothesize that this behavior stems from data limitations in standard SR training: given the strong representational capacity of our network, the commonly used training data may be insufficient, leading to overfitting. To examine this hypothesis, we train our models as well as representative SOTA baselines (PFT and MambaIRV2-L) on the DFLIP training setting, which augments the widely used DF2K dataset with additional large- scale datasets, such as LSDIR [31] and DiverSeg-IP [42]. We adopt the same training setup as in the DF2K experiments to ensure a fair comparison. In ad- dition, to broaden the evaluation scope, we report results on DIV2K-val [1] and LSDIR-val [31], which have been less commonly included in prior comparisons. As shown in Table 3, when trained on DF2K only, SST-L underperforms PFT by 0.12 dB on Urban100×2, whereas when trained on DFLIP, it surpasses PFT by 0.11 dB, highlighting the strong data-scaling capability of our approach. More- over, SST-L+ consistently delivers substantial improvements over prior SOTA methods across all upscaling factors on Urban100, Manga109, DIV2K-val, and LSDIR-val, demonstrating that our scaling design choices, including window size, training patch size, and data size, are highly effective. 4.3 Ablation Studies Next, we conduct ablation studies to verify that our proposal is effective and that the implementation is appropriate. As shown in Table 4, our proposed RIB achieves the second-best performance following FlexAttention with RPB [15,30], Rank-Factorized Implicit Neural Bias13 Table 4: Training/inference costs and performance of attention/bias variants on SST+. Training and inference costs are measured as detailed in Table 1. AttentionBias TrainingInferenceUrban100Manga109 sec/step memory#params latency memoryPSNR/SSIM PSNR/SSIM NaiveRPB [35]--13.6M 11705.3ms 126181MB- / -- / - FlexAttention [15,30] RPB [35]5.048 47.2GB13.6M 2414.0ms 2487MB34.91/0.9489 40.42/0.9812 FlashAttention3 [49] CPE [10]0.625 46.5GB11.3M 537.3ms 2354MBNot Converged FlashAttention3 [49] FlashBias [61]0.473 47.3GB46.7M 455.7ms 2825MBNot Converged FlashAttention3 [49] RoPE-ViT [23] 0.489 47.9GB11.3M 432.7ms 2915MB34.71/0.9479 40.40/0.9811 FlashAttention3 [49] RIB (Ours)0.479 47.1GB11.7M 455.7ms 2825MB34.88/0.9488 40.44/0.9812 Table 5: Ablation analysis of various components in the proposed SST on Urban100 and Manga109 (PSNR/SSIM). (a) Gating Type Gating TypeUrban100 Manga109 Without GNot Converged Only PWConv [46] 34.55/0.9473 40.24/0.9807 CLA34.61/0.9474 40.31/0.9808 (b) Window Strategy Window Strategy Urban100 Manga109 4, 8, 16, 32, 64, 64 34.56/0.9470 40.22/0.9807 64, 64, 32, 16, 8, 4 34.51/0.9468 40.17/0.9804 64, 64, 64, 64, 64, 64 34.54/0.9468 40.19/0.9806 16, 32, 64, 16, 32, 64 34.61/0.9474 40.31/0.9808 (c) Fourier Embedding Fourier Emb. Urban100 Manga109 L=534.20/0.9447 39.96/0.9797 L=1534.48/0.9468 40.17/0.9805 SIREN [52]Not Converged L=1034.61/0.9474 40.31/0.9808 while offering substantially improved efficiency (e.g., 10.5× faster training and 5× faster inference). Moreover, our RIB demonstrates stable results compared with methods that cannot directly incorporate a spatial prior into self-attention, such as conditional positional encoding (CPE [10]) or FlashBias [61] trained from scratch. Finally, RIB demonstrates superior performance over the RoPE-based method [23], suggesting that preserving pixel integrity is effective. In addition, Table 5 presents further ablations on our proposed CLA, the cyclic window strategy, and the number of frequency bands used in RIB (L). First, incorporating CLA yields higher performance than configurations without any G, which tend to diverge, as well as variants using only PWConv, demon- strating that CLA contributes meaningfully to the performance gains. Next, the cyclic window strategy achieves better results than schedules that mono- tonically increase or decrease the window size, and even outperforms using a fixed 64× 64 window throughout, suggesting that it strikes an effective balance between multi-scale feature extraction and long-range modeling. Finally, set- ting L = 10 delivers the best performance compared to other numbers or using SIREN [52] activations. This observation is consistent with findings reported in previous research [41,54] on the novel view synthesis task. 4.4 Positional Bias Visualization To verify whether RIB effectively captures positional bias, we conduct a visual- ization analysis. As shown in Figure 4, the bias generated by RIB assigns high correlation locally to nearby pixels and also induces strong correlations along the vertical (up and down) direction. This indicates that RIB effectively provides a spatial prior to the self-attention. 14D. Lee et al. 퐒 ! = 퐐 ! 퐊 !" / 푅 퐑 ∆ = Avg " , $ : ' " , $ ( ∆ [ 퐒 ) ] "$ Layer 1 (푀=16)Layer 2 (푀=32)Layer 3 (푀=64)Layer 4 (푀=16)Layer 5 (푀=32)Layer 6 (푀=64) Fig. 4: Visualized positional score matrix (S p ) and relative positional bias table (R ∆ ) calculated from the second block of SST. We average the biases that correspond to the same 2D distance, since positional bias predicted by our RIB does not guarantee equivalence across the same relative offsets. LAM Attribution Area of Contribution img030×4 Diffusion Index PFT 25.08 SST-L (Ours) 26.06 LAM Attribution Area of Contribution img055×4 Diffusion Index MambaIRV2-L 16.44 SST-L+ (Ours) 19.95 Bicubic 23.59 MambaIRV2-L 29.03 GT PSNR PFT 29.10 SST-L 29.47 SST-L+ 32.23 img097×2 Bicubic 18.44 MambaIRV2-L 20.66 GT PSNR PFT 20.29 SST-L 21.26 SST-L+ 22.04 img069×4 (a)(b) Fig. 5: Comparisons of (a) visual results and (b) local attribution maps (LAM) [20]. Since padding/cropping for windowing inside the networks hinders accurate compari- son, we separately compare SST-L (512× 512 patch) and SST-L+ (768× 768 patch). 4.5 Local Attribution Map and Visual Results Finally, through a visual comparison, we verify that our networks restore visually superior results and leverage a wider receptive field. As illustrated in Figure 5, our networks restore fine details most faithfully, achieving the highest PSNR while leveraging the broadest receptive field around edges. 5 Conclusion In this paper, we propose a FlashAttention-compatible bias termed RIB for SR Transformers that preserves the integrity of pixel representations, along with CLA and a cyclic window strategy to further strengthen long-range interactions. Leveraging these components, we simply scale up both the self-attention window size and two factors that have been relatively underexplored in prior work— training patch size and dataset size. Consequently, we substantially reduce both training and inference costs while achieving 29.06 dB PSNR on Urban100×4, highlighting the promise of this underexplored direction for SR tasks. Rank-Factorized Implicit Neural Bias15 References 1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: Dataset and study. In: CVPRW (July 2017) 2. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012) 3. Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: ICCV. p. 9650– 9660 (2021) 4. Chen, J., Kao, S.h., He, H., Zhuo, W., Wen, S., Lee, C.H., Chan, S.H.G.: Run, don’t walk: chasing higher flops for faster neural networks. In: CVPR. p. 12021–12031 (2023) 5. Chen, X., Wang, X., Zhou, J., Qiao, Y., Dong, C.: Activating more pixels in image super-resolution transformer. In: CVPR. p. 22367–22377 (2023) 6. Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X.: Recursive generalization trans- former for image super-resolution. In: ICLR (2024) 7. Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X., Yu, F.: Dual aggregation trans- former for image super-resolution. In: ICCV. p. 12312–12321 (2023) 8. Chen, Z., Zhang, Y., Gu, J., Zhang, Y., Kong, L., Yuan, X.: Cross aggregation transformer for image restoration. In: NeurIPS (2022) 9. Choi, H., Lee, J., Yang, J.: N-gram in swin transformers for efficient lightweight image super-resolution. In: CVPR. p. 2071–2081 (2023) 10. Chu, X., Tian, Z., Zhang, B., Wang, X., Shen, C.: Conditional positional encodings for vision transformers. In: ICLR (2023), https://openreview.net/forum?id= 3KWnuT-R1bh 11. Dao, T.: Flashattention-2: Faster attention with better parallelism and work par- titioning. arXiv (2023) 12. Dao, T., Fu, D., Ermon, S., Rudra, A., Ré, C.: Flashattention: Fast and memory- efficient exact attention with io-awareness. NeurIPS 35, 16344–16359 (2022) 13. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convo- lutional networks. IEEE TPAMI 38(2), 295–307 (2015) 14. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV. p. 391–407. Springer (2016) 15. Dong, J., Feng, B., Guessous, D., Liang, Y., He, H.: Flex attention: A programming model for generating optimized attention kernels. arXiv (2024) 16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. ICLR (2020) 17. Du, Z., Liu, J., Tang, J., Wu, G.: Anchor-based plain net for mobile image super- resolution. In: CVPRW. p. 2494–2502 (2021) 18. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Let- man, A., Mathur, A., Schelten, A., Vaughan, A., et al.: The llama 3 herd of models. arXiv (2024) 19. Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint (2023) 20. Gu, J., Dong, C.: Interpreting super-resolution networks with local attribution maps. In: CVPR. p. 9199–9208 (2021) 21. Guo, H., Guo, Y., Zha, Y., Zhang, Y., Li, W., Dai, T., Xia, S.T., Li, Y.: Mambairv2: Attentive state space restoration. CVPR (2025) 16D. Lee et al. 22. Guo, H., Li, J., Dai, T., Ouyang, Z., Ren, X., Xia, S.T.: Mambair: A simple baseline for image restoration with state-space model. In: ECCV (2024) 23. Heo, B., Park, S., Han, D., Yun, S.: Rotary position embedding for vision trans- former. In: ECCV. p. 289–305. Springer (2024) 24. Hu, Q., Tang, Y., Zhang, X.: Large kernel modulation network for efficient image super-resolution. arXiv (2025) 25. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR. p. 5197–5206 (2015) 26. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR. p. 1646–1654 (2016) 27. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: ICCV. p. 4015–4026 (2023) 28. Lee, D., Yun, S., Ro, Y.: Implicit grid convolution for multi-scale image super- resolution. arXiv (2024) 29. Lee, D., Yun, S., Ro, Y.: Partial large kernel cnns for efficient super-resolution. arXiv (2024) 30. Lee, D., Yun, S., Ro, Y.: Emulating self-attention with convolution for efficient image super-resolution. In: ICCV. p. 24467–24477 (October 2025) 31. Li, Y., Zhang, K., Liang, J., Cao, J., Liu, C., Gong, R., Zhang, Y., Tang, H., Liu, Y., Demandolx, D., et al.: Lsdir: A large scale dataset for image restoration. In: CVPRW. p. 1775–1787 (2023) 32. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: Image restoration using swin transformer. In: ICCVW. p. 1833–1844 (2021) 33. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: CVPRW. p. 136–144 (2017) 34. Lin, Z., Garg, P., Banerjee, A., Magid, S.A., Sun, D., Zhang, Y., Van Gool, L., Wei, D., Pfister, H.: Revisiting rcan: Improved training for image super-resolution. arXiv (2022) 35. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: ICCV. p. 10012–10022 (2021) 36. Long, W., Zhou, X., Zhang, L., Gu, S.: Progressive focused transformer for single image super-resolution. In: CVPR. p. 2279–2288 (2025) 37. Ma, N., Zhang, X., Zheng, H.T., Sun, J.: Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: ECCV. p. 116–131 (2018) 38. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV. vol. 2, p. 416–423. IEEE (2001) 39. Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimedia tools and applications 76, 21811–21838 (2017) 40. Meng, C., Long, W., Gu, S.: From local windows to adaptive candidates via in- dividualized exploratory: Rethinking attention for image super-resolution. arXiv (2026) 41. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020) 42. Ohtani, G., Tadokoro, R., Yamada, R., Asano, Y.M., Laina, I., Rupprecht, C., Inoue, N., Yokota, R., Kataoka, H., Aoki, Y.: Rethinking image super-resolution from training data perspectives. In: ECCV. p. 19–36. Springer (2024) Rank-Factorized Implicit Neural Bias17 43. Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. TMLR (2023) 44. Park, K., Cho, N.I.: Partial filter-sharing: Improved parameter-sharing method for single image super-resolution networks. In: WACV. p. 2653–2663. IEEE (2025) 45. Park, K., Soh, J.W., Cho, N.I.: Efficient attention-sharing information distillation transformer for lightweight single image super-resolution. In: AAAI (2025) 46. Qiu, Z., Wang, Z., Zheng, B., Huang, Z., Wen, K., Yang, S., Men, R., Yu, L., Huang, F., Huang, S., et al.: Gated attention for large language models: Non- linearity, sparsity, and attention-sink-free. In: NeurIPS (2025) 47. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21(140), 1–67 (2020) 48. Ray, A., Kumar, G., Kolekar, M.H.: Cfat: Unleashing triangular windows for image super-resolution. In: CVPR. p. 26120–26129 (2024) 49. Shah, J., Bikshandi, G., Zhang, Y., Thakkar, V., Ramani, P., Dao, T.: Flashattention-3: Fast and accurate attention with asynchrony and low-precision. NeurIPS 37, 68658–68685 (2024) 50. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR. p. 1874–1883 (2016) 51. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv (2014) 52. Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. NeurIPS 33, 7462–7473 (2020) 53. Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., Liu, Y.: Roformer: Enhanced trans- former with rotary position embedding. Neurocomputing 568, 127063 (2024) 54. Sun, C., Yuan, Z., Xu, K., Mai, L., N, S., Chen, S., Marina, M.K.: Learning high-frequency functions made easy with sinusoidal positional encoding. In: ICML (2024), https://openreview.net/forum?id=qqPL0DkcrI 55. Sun, L., Pan, J., Tang, J.: Shufflemixer: An efficient convnet for image super- resolution. NeurIPS 35, 17314–17326 (2022) 56. Tian, Y., Chen, H., Xu, C., Wang, Y.: Image processing gnn: Breaking rigidity in super-resolution. In: CVPR. p. 24108–24117 (2024) 57. Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: Ntire 2017 chal- lenge on single image super-resolution: Methods and results. In: CVPRW. p. 114–125 (2017) 58. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. NeurIPS 30 (2017) 59. Wang, H., Chen, X., Ni, B., Liu, Y., jinfan, L.: Omni aggregation networks for lightweight image super-resolution. In: CVPR (2023) 60. Wang, X., Xie, L., Yu, K., Chan, K.C., Loy, C.C., Dong, C.: BasicSR: Open source image and video restoration toolbox. https://github.com/XPixelGroup/BasicSR (2022) 61. Wu, H., Guo, M., Ma, Y., Sun, Y., Wang, J., Matusik, W., Long, M.: Flashbias: Fast computation of attention with bias. In: NeurIPS (2025), https://openreview. net/forum?id=7L4NvUtZY3 62. Xiong, W., Liu, J., Molybog, I., Zhang, H., Bhargava, P., Hou, R., Martin, L., Rungta, R., Sankararaman, K.A., Oguz, B., et al.: Effective long-context scaling of foundation models. In: NAACL. p. 4643–4663 (2024) 18D. Lee et al. 63. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv (2025) 64. Yun, S., Ro, Y.: Shvit: Single-head vision transformer with memory efficient macro design. In: CVPR. p. 5756–5767 (2024) 65. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: efficient attention transformer for high-resolution image restoration. In: CVPR. p. 5728–5739 (2022) 66. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse- representations. In: Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7. p. 711–730. Springer (2012) 67. Zhang, J., Zhang, Y., Gu, J., Zhang, Y., Kong, L., Yuan, X.: Accurate image restoration with attention retractable transformer. In: ICLR (2023) 68. Zhang, L., Li, Y., Zhou, X., Zhao, X., Gu, S.: Transcending the limit of local window: Advanced super-resolution transformer with adaptive token dictionary. In: CVPR. p. 2856–2865 (2024) 69. Zhang, X., Zhang, Y., Yu, F.: Hit-sr: Hierarchical transformer for efficient attention image super-resolution. In: ECCV (2024) 70. Zhang, X., Zeng, H., Guo, S., Zhang, L.: Efficient long-range attention network for image super-resolution. In: ECCV. p. 649–667. Springer (2022) 71. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: ECCV. p. 286–301 (2018) 72. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR. p. 2472–2481 (2018) 73. Zheng, M., Sun, L., Dong, J., Pan, J.: Smfanet: A lightweight self-modulation feature aggregation network for efficient image super-resolution. In: ECCV (2024) 74. Zhou, K., Lin, X., Lu, J.: Tsp-mamba: The travelling salesman problem meets mamba for image super-resolution and beyond. In: CVPR. p. 28134–28143 (2025) 75. Zhou, K., Lin, X., Zhonghang, L., Han, X., Lu, J.: Ups: Unified projection sharing for lightweight single-image super-resolution and beyond. In: NeurIPS (2024) 76. Zhou, Y., Li, Z., Guo, C.L., Bai, S., Cheng, M.M., Hou, Q.: Srformer: Permuted self-attention for single image super-resolution. In: ICCV. p. 12780–12791 (2023) Rank-Factorized Implicit Neural Bias19 Rank-Factorized Implicit Neural Bias: Scaling Super-Resolution Transformer with FlashAttention Supplementary Material Supplementary material includes a summary of related work, implementation and training details, additional results on lightweight Super-Resolution (SR) tasks, comparisons of inference costs across FlashAttention implementations, and comparisons of inference costs across devices. S1 Related Work S1.1 Traditional CNNs Early deep learning-based SR methods predominantly relied on a convolution op- eration, which is well suited for extracting local features from the input [13,14,33, 34,50,71]. Motivated by prior findings that stacking small convolutions can grad- ually expand the effective receptive field while keeping the parameter count rel- atively modest [51], many convolution-based works adopted designs that heavily stack 3×3 convolutions. However, such designs only model long-range dependen- cies indirectly, which limits their ability to capture similar but spatially distant patterns [32]. Moreover, these stacked 3×3 convolutional structures often intro- duce large parameter counts, frequently leading to over-parameterization [44]. S1.2 Transformers To address these limitations, Transformers [58], which leverage self-attention as a core operator, have garnered significant attention in SR tasks. By per- forming input-adaptive global feature aggregation, self-attention offers strong representational capacity and can potentially improve performance with fewer parameters and lower computational cost. Nevertheless, unlike many vision tasks that treat an image patch (e.g., 16×16) as a token, SR commonly operates at the pixel level and processes high-resolution inputs, making it difficult to ap- ply vanilla self-attention directly due to its quadratic complexity. Therefore, many SR Transformers restrict self-attention to local windows to address the quadratic complexity of self-attention [35]. While this makes the computational cost more practical and manageable, it also limits long-range modeling capa- bility. As a result, much recent effort has focused on expanding the receptive field while reducing the number of pixels involved in self-attention computa- tion [5,7,8,32,48,67,76]. 20D. Lee et al. 퐹 ! (a)Deep Feature Extractor(ℰ " ) (b)SST Block (c) Self-Attention of SST Block SST BlockSST BlockSST Block LayerNorm 3 × 3 Conv 퐹 " ... Convolutional Local Attention 퐖 # 퐖 $ 퐖 % FlashAttention withRIB 퐐 퐊 퐕 푋 퐎 퐖 & 퐎 ' 푌 DWConv PWConv σ ℱ ℱ !" 퐆 LayerNorm 3 × 3 Conv Self - Attn LayerNorm ConvFFN ×Layers Fig. S1: Overall illustration for SST architecture. S1.3 Alternative Operators Another actively studied direction is to replace self-attention with sub-quadratic operators. Representative examples include proposing attention-like operators with sub-quadratic complexity [6, 9, 65, 68], adapting Mamba [19], which has attracted substantial interest in sequence/language modeling, to the SR do- main [21, 22, 74], and integrating sparsification into self-attention [36, 40]. For lightweight SR, where efficiency under small model size is a primary goal, leverag- ing large-kernel convolutions [24,29,55,73] and reducing the number of attention- map computations [45,70,75] have also been considered. S1.4 Hardware-Efficient Acceleration Methods However, most of the aforementioned approaches focus mainly on computational complexity and comparatively overlook memory efficiency. In practice, when data movement becomes the bottleneck, cutting floating-point operations (FLOPs) alone often fails to reduce latency and memory usage. For instance, self-attention repeatedly moves large N×N matrices (e.g., S and P) between HBM and SRAM during execution, which can incur substantial latency and memory overhead that FLOPs fail to account for. To directly mitigate this issue, FlashAtten- tion [11, 12, 49] was introduced to reduce memory traffic while still computing exact attention, thereby significantly lowering latency and memory usage. Due to its practical impact, FlashAttention has been rapidly adopted across diverse Transformer applications, and is becoming a key ingredient for scaling Trans- formers in terms of both model size and sequence length. In SR, however, the relative position bias (RPB) that is crucial for performance [35,47] is not compat- ible with the layouts supported by FlashAttention, making it difficult to leverage FlashAttention directly. Several works have attempted to resolve this limita- tion [30, 61]. Yet, FlashBias [61] suffers from severe performance degradation Rank-Factorized Implicit Neural Bias21 Table S1: Network implementation details. Methods D Blocks LayersWindow SizesHeads L d h RConvFFN Exp. SST-light 48 56 [8, 16, 32, 16, 32, 64] 3 10 32 [16, 16, 16, 24, 24, 24]1.5 SST-light+ 48 56 [16, 32, 48, 32, 48, 96] 3 10 32 [16, 16, 16, 24, 24, 24]1.5 SST180 66 [16, 32, 64, 16, 32, 64] 6 10 32 [18, 18, 18, 34, 34, 34]1.25 SST+180 66 [16, 32, 48, 32, 48, 96] 6 10 32 [18, 18, 18, 34, 34, 34]1.25 SST-L192 86 [16, 32, 64, 16, 32, 64] 6 10 32 [16, 16, 16, 32, 32, 32]2 SST-L+ 192 86 [16, 32, 48, 32, 48, 96] 6 10 32 [16, 16, 16, 32, 32, 32]2 Table S2: Training configuration across our networks for ×2 upscaling. For ×3 and ×4, we initialize from the weights of×2 and×3, respectively [28], and halve the number of iterations, learning rate, and schedule. Methods TrainingPatch Dataset BatchSize Iteration Optimizer LR Schedule (γ = 0.5)Loss SST-light64×64DIV2K 64 500000 AdamW 5e-4 [250000, 400000, 450000, 475000, 490000] L1Loss SST-light+96×96or DFLIP SST64×64 DF2K 32 SST+96×96 SST-L64×64DF2K SST-L+96×96or DFLIP when training models from scratch, and FlexAttention [15]-based implementa- tions [30] rely on specific frameworks and low-level compilation, which increases system complexity and slows down training (please refer to Table 4 in the main manuscript). S2 Implementation and Training Details Our network follows the same overall architecture as the standard SR Trans- former [32], and its detailed configuration is provided in Figure S1 and Table S1. On top of the window-based self-attention layer [32], we incorporate only two proposed components: the proposed Rank-factorized Implicit Neural Bias (RIB) and a convolutional local-attention module (CLA). We also adopt the ConvFFN introduced in prior work [76], setting the kernel size to 3 to strengthen high- frequency feature extraction. Inspired by previous work [69], we replace shifted windows with a cyclic window strategy, which periodically increases the window sizes. This design aims to extract multi-scale features while enabling more ag- gressive use of large windows. Among the RIB hyper-parameters (L,d h ,R), we set L = 10 following the ablation results in Table 5 in the main manuscript, and choose d h and R empirically. In particular, for R, we select a value such that the dimensionality (D +R), after combining positional tokens with pixel tokens, is a multiple of 8. This allows us to match the FlashAttention layout without channel-wise zero padding. For our experiments, we use PyTorch version 2.9.1 and implement our method using the widely used BasicSR toolbox [60]. 22D. Lee et al. Table S3: Training/inference costs of attention variants on SST-L+. Training and inference costs are measured as detailed in Table 1. Attention TrainingInference sec/step memorylatency memory FlashAttention2 [11]0.556 47.1GB503.7ms 2825MB FlashAttention3 [49] 0.479 47.1GB455.7ms 2825MB As shown in the training configuration in Table S2, our overall training setup is largely consistent with prior studies. However, to highlight our main focus on scalability, we additionally consider variants trained with larger patch sizes (e.g., 96×96) and variants trained on larger-scale datasets (e.g., DFLIP [1,31,42,57]). S3 Comparisons of Training and Inference Costs Across FlashAttention Implementations In this section, we compare the training and inference costs across different FlashAttention implementations. We consider FlashAttention2 [11], which is widely used in practice, and FlashAttention3 [49], an improved version opti- mized for Hopper GPUs. As shown in Table S3, FlashAttention3 achieves a 20% faster training throughput and a 10% lower inference latency compared to FlashAttention2. These results suggest that as FlashAttention continues to im- prove, our approach can benefit from additional acceleration with only minor changes to the overall method. S4 Comparisons of Inference Costs Across Devices We then evaluate the inference efficiency of our approach against representative SR baselines [5,21,22,36,68] by measuring inference latency and memory usage on a range of GPUs. As summarized in Table S4, our SST variants consistently achieve substantially lower memory consumption and faster inference latency than competing methods across all tested devices, highlighting the practical efficiency of our FlashAttention-based implementation. S5 Comparisons on CLA and PWConv In this section, we compare CLA with the PWConv-only gating module proposed in [46] through qualitative visualizations. As shown in Figure S2, when the gating module relies only on PWConv, the self-attention output (O) is heavily influ- enced by local details, making it difficult to consistently capture repeated edge patterns. In contrast, when CLA is applied, the gating path (G) takes responsi- bility for modeling local details, allowing O to focus more stably and robustly on Rank-Factorized Implicit Neural Bias23 CLA (Ours)CLA (Ours)only PWConvonly PWConv PCA of 퐎 PCA of 퐆 PCA of 퐎 PCA of 퐆 Fig. S2: PCA visualization of the gating coefficient (G) and self-attention output (O) under PWConv and our proposed Convolutional Local Attention (CLA). We visual- ize the first 64×64 window self-attention within the last SST block. To visualize the extracted features, we apply Principal Component Analysis (PCA) to reduce the chan- nel dimension to three components and render them as RGB channels. Compared to PWConv-only variants, the self-attention layer with CLA is less affected by local tex- tures and more robustly extracts structural features across the entire image. repeated structures and edges. These results suggest that, unlike conventional PWConv-only gating modules originally introduced for NLP tasks, CLA better reflects the characteristics required for the SR domain. By incorporating convo- lution into the gating path, CLA helps self-attention capture repeated structures and edges more reliably, which in turn contributes to improved performance. S6 Results on Lightweight Super-Resolution Tasks Although our primary goal is to improve performance through scaling, the prac- tical advantages of FlashAttention also make our approach highly effective in lightweight regimes. As shown in Tables S5 and S6, our method substantially improves both efficiency and reconstruction quality over existing RPB-based Transformer approaches. Notably, by leveraging scaling, our model achieves an impressive PSNR of 34.31 dB on Urban100×2 even under a parameter budget of fewer than 1M parameters. These results highlight the practicality of our approach and demonstrate its potential for a wide range of applications. 24D. Lee et al. Table S4: Latency and memory usage across GPUs. We use FlashAttention3 for an H200 GPU and FlashAttention2 for RTX4090 and A6000 GPUs. We report the measured median speed when restoring a 1280×720 image 10 times in FP32. Memory usage is measured using PyTorch’s max_memory_allocated function. MethodsScale RTX4090A6000H200 latency memorylatency memorylatency memory HAT ×2 1722.8ms 9047MB3068.5ms 9047MB709.7ms 9070MB ATD 2665.8ms 6216MB5064.8ms 6216MB1266.8ms 6239MB PFTOOM OOM12990.1ms 28847MB1920.8ms 28869MB MambaIR1570.8ms 6883MB3349.4ms 7680MB1043.8ms 7704MB MambaIRV23373.6ms 5749MB9814.8ms 5792MB2308.8ms 5816MB SST956.7ms 2675MB1996.8ms 2675MB428.9ms 2675MB SST+1046.0ms 2825MB2125.3ms 2825MB455.8ms 2825MB SST-L1439.2ms 2836MB3051.8ms 2824MB608.4ms 2824MB SST-L+1552.7ms 2990MB3212.9ms 2990MB644.9ms 2990MB HAT ×3 799.8ms 4040MB1377.0ms 4040MB317.8ms 4063MB ATD1210.3ms 2795MB2220.6ms 2795MB550.0ms 2819MB PFT2088.5ms 13525MB4646.3ms 13525MB892.7ms 13549MB MambaIR668.4ms 3125MB1358.8ms 3458MB465.1ms 3481MB MambaIRV21434.5ms 2589MB4161.6ms 2632MB1000.3ms 2655MB SST401.1ms 1296MB739.2ms 1296MB194.2ms 1296MB SST+446.8ms 1527MB799.9ms 1527MB207.1ms 1527MB SST-L609.1ms 1404MB1112.5ms 1391MB276.3ms 1391MB SST-L+667.0ms 1639MB1191.6ms 1639MB293.4ms 1639MB HAT ×4 510.8ms 2428MB821.3ms 2428MB195.8ms 2452MB ATD756.1ms 1692MB1248.9ms 1692MB339.0ms 1716MB PFT1237.4ms 7302MB3168.1ms 7302MB493.3ms 7326MB MambaIR356.5ms 1818MB733.9ms 1990MB269.7ms 2014MB MambaIRV2848.9ms 1574MB2188.1ms 1616MB604.5ms 1640MB SST199.4ms 764MB401.9ms 764MB111.5ms 764MB SST+228.9ms 914MB420.6ms 914MB120.8ms 914MB SST-L308.6ms 907MB556.3ms 919MB158.3ms 941MB SST-L+346.0ms 1003MB613.1ms 1003MB170.4ms 1015MB Rank-Factorized Implicit Neural Bias25 Table S5: Comparisons of lightweight SR methods trained on the DIV2K dataset. We reuse the statistics reported in prior work [30] and additionally include our results, measured following their procedure. The best result is bolded. MethodScale Latency (ms) Mem (MB) #FLOPs (G) #params (K) PSNR / SSIM Set5Set14B100Urban100Manga109 SwinIR-lt [32] ×2 1409.8 1287 244.291038.14/0.9611 33.86/0.9206 32.31/0.9012 32.76/0.9340 39.12/0.9783 ELAN-lt [70]94.5 887 203.162138.17/0.9611 33.94/0.9207 32.30/0.9012 32.76/0.9340 39.11/0.9782 OmniSR [59] 120.3 1031 194.577238.22/0.9613 33.98/0.9210 32.36/0.9020 33.05/0.9363 39.28/0.9784 SRFormer-lt [76]1456.3 1184 236.385338.23/0.9613 33.94/0.9209 32.36/0.9019 32.91/0.9353 39.28/0.9785 ATD-lt [68] 733.5 2839 380.075338.29/0.9616 34.10/0.9217 32.39/0.9023 33.27/0.9375 39.52/0.9789 HiT-SRF [69]268.1 1804 226.584738.26/0.9615 34.01/0.9214 32.37/0.9023 33.13/0.9372 39.47/0.9787 ASID-D8 [45] 131.2 999 190.573238.32/0.9618 34.24/0.9232 32.40/0.9028 33.35/0.9387- / - MambaIR-lt [22] 277.1 1695 334.290538.13/0.9610 33.95/0.9208 32.31/0.9013 32.85/0.9349 39.20/0.9782 MambaIRV2-lt [21]580.4 2824 286.377438.26/0.9615 34.09/0.9221 32.36/0.9019 33.26/0.9378 39.35/0.9785 RDN [72]279.3 2058 5096.2 2212338.24/0.9614 34.01/0.9212 32.34/0.9017 32.89/0.9353 39.18/0.9780 RCAN [71]299.3 626 3529.7 1544538.27/0.9614 34.12/0.9216 32.41/0.9027 33.34/0.9384 39.44/0.9786 ESC [30]120.9 831 592.094738.35/0.9619 34.11/0.9223 32.41/0.9027 33.46/0.9395 39.54/0.9790 SST-lt (Ours)191.9 755 2057.689338.35/0.9620 34.14/0.9227 32.41/0.9028 33.54/0.9399 39.48/0.9785 SST-lt+ (Ours)249.6 804 4903.389338.35/0.9619 34.22/0.9232 32.44/0.9031 33.79/0.9417 39.59/0.9788 SwinIR-lt [32] ×3 331.7 596 110.891834.62/0.9289 30.54/0.8463 29.20/0.8082 28.66/0.8624 33.98/0.9478 ELAN-lt [70]32.5 399 90.162934.61/0.9288 30.55/0.8463 29.21/0.8081 28.69/0.8624 34.00/0.9478 OmniSR [59]41.2 476 88.478034.70/0.9294 30.57/0.8469 29.28/0.8094 28.84/0.8656 34.22/0.9487 SRFormer-lt [76]530.5 537 105.486134.67/0.9296 30.57/0.8469 29.26/0.8099 28.81/0.8655 34.19/0.9489 ATD-lt [68]274.4 1258 168.076034.74/0.9300 30.68/0.8485 29.32/0.8109 29.17/0.8709 34.60/0.9506 HiT-SRF [69]124.9 1464 101.685534.75/0.9300 30.61/0.8475 29.29/0.8106 28.99/0.8687 34.53/0.9502 ASID-D8 [45]61.9 460 86.473934.84/0.9307 30.66/0.8491 29.32/0.8119 29.08/0.8706- / - MambaIR-lt [22]109.3 760 148.591334.63/0.9288 30.54/0.8459 29.23/0.8084 28.70/0.8631 34.12/0.9479 MambaIRV2-lt [21]259.0 1250 126.778134.71/0.9298 30.68/0.8483 29.26/0.8098 29.01/0.8689 34.41/0.9497 RDN [72]146.5 985 2281.2 2230834.71/0.9296 30.57/0.8468 29.26/0.8093 28.80/0.8653 34.13/0.9484 RCAN [71]85.1 560 1586.1 1562934.74/0.9299 30.65/0.8482 29.32/0.8111 29.09/0.8702 34.44/0.9499 ESC [30]41.4 385 267.695534.84/0.9308 30.74/0.8493 29.34/0.8118 29.28/0.8739 34.66/0.9512 SST-lt (Ours) 69.2 359 954.190034.79/0.9305 30.68/0.8489 29.31/0.8113 29.34/0.8748 34.55/0.9507 SST-lt+ (Ours)98.1 432 2397.790034.80/0.9308 30.71/0.8495 29.34/0.8122 29.50/0.8774 34.67/0.9514 SwinIR-lt [32] ×4 222.9 351 63.693032.44/0.8976 28.77/0.7858 27.69/0.7406 26.47/0.7980 30.92/0.9151 ELAN-lt [70]18.0 241 54.164032.43/0.8975 28.78/0.7858 27.69/0.7406 26.54/0.7982 30.92/0.9150 OmniSR [59]22.5 273 50.979232.49/0.8988 28.78/0.7859 27.71/0.7415 26.64/0.8018 31.02/0.9151 SRFormer-lt [76]287.2 329 62.887332.51/0.8988 28.82/0.7872 27.73/0.7422 26.67/0.8032 31.17/0.9165 ATD-lt [68]189.7 753 100.176932.63/0.8998 28.89/0.7886 27.79/0.7440 26.97/0.8107 31.48/0.9198 HiT-SRF [69]82.1 1331 58.086632.55/0.8999 28.87/0.7880 27.75/0.7432 26.80/0.8069 31.26/0.9171 ASID-D8 [45]61.8 265 49.674832.57/0.8990 28.89/0.7898 27.78/0.7449 26.89/0.8096- / - MambaIR-lt [22]55.8 438 84.692432.42/0.8977 28.74/0.7847 27.68/0.7400 26.52/0.7983 30.94/0.9135 MambaIRV2-lt [21]153.4 748 75.679032.51/0.8992 28.84/0.7878 27.75/0.7426 26.82/0.8079 31.24/0.9182 RDN [72]66.0 791 1309.2 2227132.47/0.8990 28.81/0.7871 27.72/0.7419 26.61/0.8028 31.00/0.9151 RCAN [72]52.2 540 917.6 1559232.63/0.9002 28.87/0.7889 27.77/0.7436 26.82/0.8087 31.22/0.9173 ESC [30]21.9 215 149.296832.68/0.9011 28.93/0.7902 27.80/0.7447 27.07/0.8144 31.54/0.9207 SST-lt (Ours)36.5 205 517.090832.62/0.9007 28.93/0.7896 27.79/0.7446 27.12/0.8158 31.48/0.9200 SST-lt+ (Ours)53.0 253 1329.790832.65/0.9003 28.94/0.7908 27.81/0.7454 27.23/0.8195 31.63/0.9219 26D. Lee et al. Table S6: Comparisons of lightweight SR methods trained on the DFLIP dataset. We reuse the statistics reported in prior work [30] and additionally include our results, measured following their procedure. The best result is bolded. MethodScale Latency (MB) Mem (mb) #FLOPs (G) #params (K) PSNR / SSIM Set5Set14B100Urban100Manga109 SRFormer-lt [76] ×2 1838.1 1184 236.385338.24/0.9615 34.13/0.9218 32.42/0.9026 33.37/0.9386 39.36/0.9787 ATD-lt [68]733.5 2839 380.075338.29/0.9616 34.30/0.9230 32.43/0.9027 33.62/0.9401 39.60/0.9791 HiT-SRF [69] 268.1 1804 226.584738.31/0.9616 34.31/0.9230 32.45/0.9031 33.58/0.9404 39.69/0.9793 ESC [30]120.9 831 592.094738.34/0.9618 34.42/0.9235 32.50/0.9036 33.86/0.9424 39.73/0.9795 SST-lt (Ours)191.9 755 2057.689338.39/0.9622 34.50/0.9245 32.50/0.9038 34.03/0.9436 39.72/0.9795 SST-lt+ (Ours)249.6 804 4903.389338.42/0.9622 34.62/0.9250 32.53/0.9042 34.31/0.9452 39.90/0.9798 SRFormer-lt [76] ×3 668.3 537 105.486134.67/0.9297 30.75/0.8484 29.30/0.8108 29.10/0.8701 34.26/0.9498 ATD-lt [68]274.4 1258 168.076034.71/0.9300 30.77/0.8493 29.33/0.8116 29.42/0.8743 34.61/0.9509 HiT-SRF [69]124.9 1464 101.685534.69/0.9298 30.81/0.8493 29.32/0.8115 29.28/0.8729 34.72/0.9511 ESC [30]41.4 385 267.695534.85/0.9312 30.97/0.8511 29.41/0.8135 29.70/0.8799 34.94/0.9525 SST-lt (Ours)69.2 359 954.190034.87/0.9310 30.92/0.8508 29.38/0.8131 29.74/0.8807 34.79/0.9520 SST-lt+ (Ours)98.1 432 2397.790034.93/0.9316 30.97/0.8514 29.41/0.8140 29.95/0.8840 34.99/0.9528 SRFormer-lt [76] ×4 327.8 329 62.887332.49/0.8993 28.89/0.7887 27.76/0.7429 26.90/0.8086 31.25/0.9189 ATD-lt [68]189.7 753 100.176932.52/0.8995 28.93/0.7896 27.79/0.7443 27.18/0.8150 31.47/0.9208 HiT-SRF [69]82.1 1331 58.086632.55/0.8997 28.96/0.7897 27.77/0.7443 27.07/0.8130 31.59/0.9208 ESC [30]21.9 215 149.296832.79/0.9025 29.06/0.7927 27.85/0.7466 27.45/0.8229 31.87/0.9239 SST-lt (Ours)36.5 205 517.090832.70/0.9017 29.03/0.7918 27.83/0.7459 27.41/0.8225 31.64/0.9224 SST-lt+ (Ours)53.0 253 1329.790832.71/0.9019 29.04/0.7921 27.86/0.7472 27.58/0.8270 31.92/0.9241