Paper deep dive

Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Hanxu Yang, Yuhuan Zhao, Xiaodong He, Zhao Kang

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 94

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 9/29/2026, 12:46:24 PM

Summary

The paper introduces Spectral Reverse Prompt (SRP), a parameter-efficient graph prompting framework that addresses 'spectral bias' in pre-trained Graph Neural Networks (GNNs). Spectral bias causes pre-trained models to over-represent dominant singular value directions while under-utilizing low-energy directions. SRP rebalances these contributions using a learnable soft-thresholding mask in the spectral domain to suppress dominant directions and amplify weaker ones, alongside a null-space augmentation module to capture variation in directions with minimal activation. Experiments show SRP achieves state-of-the-art performance with minimal parameters.

Entities (6)

Graph Neural Networks · model · 99%Spectral Reverse Prompt · method · 98%Spectral Bias · phenomenon · 96%Null-Space Augmentation · component · 92%Soft-Thresholding Mask · component · 92%Downstream-Pretraining Misalignment Score · metric · 90%

Relation Signals (6)

Spectral Reverse Prompt → addresses → Spectral Bias

confidence 95% · SRP applies a learnable soft-thresholding mask... to down-weight dominant directions... counteracting singular value bias

Spectral Reverse Prompt → usescomponent → Soft-Thresholding Mask

confidence 93% · SRP applies a learnable soft-thresholding mask in the spectral domain

Spectral Reverse Prompt → usescomponent → Null-Space Augmentation

confidence 93% · SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation

Spectral Bias → affects → Graph Neural Networks

confidence 90% · we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias

Soft-Thresholding Mask → operatesin → Spectral Domain

confidence 88% · SRP applies a learnable soft-thresholding mask in the spectral domain

Downstream-Pretraining Misalignment Score → quantifies → Spectral Bias

confidence 85% · We identify and quantify a form of spectral bias... using a Downstream–Pretraining Misalignment Score (DPMS)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

90,245 characters extracted from source content.

Expand or collapse full text

Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting Hanxu Yang †thanks: Equal contribution. Yuhuan Zhao11footnotemark: 1 Xiaodong He Zhao Kang †thanks: Corresponding author. Affiliation: School of Computer Science and Engineering, Affiliation: University of Electronic Science and Technology of China, Chengdu, Sichuan, China Affiliation: yang_hanxu@std.uestc.edu.cn zkang@uestc.edu.cn Abstract Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation. The code is available at https://github.com/keris-yang/Spectral-Reversal. 1 Introduction Graph prompt tuning Sun et al. (2023b); Fu et al. (2025c) has emerged as a parameter-efficient alternative to fine-tuning for adapting pre-trained Graph Neural Networks Kipf and Welling (2017); Veličković et al. (2018); Xu et al. (2019); Hamilton et al. (2017); Brody et al. (2022) (GNNs). Existing approaches typically introduce learnable tokens Fang et al. (2023); Fu et al. (2025a); Li et al. (2025) or modify readout layers Sun et al. (2022); Yu et al. (2024b); Yu et al. (2024a), while keeping the backbone encoder frozen. However, these methods implicitly treat the pre-trained model as a black-box feature extractor, overlooking its internal structure. In particular, little attention has been paid to the spectral properties of the learned transformations within pre-trained GNNs. During self-supervised pre-training, optimization tends to align representations with dominant modes of variation, which are often associated with large singular values of the effective transformation. We refer to this phenomenon as spectral bias, where dominant spectral components are emphasized while lower-energy directions remain underutilized. While these dominant directions capture prevalent patterns in pre-training data, they may not align with the requirements of downstream tasks Huang et al. (2024), especially under inter-class inequality. This observation suggests a different perspective on parameter-efficient adaptation: rather than further reinforcing dominant directions, it may be more effective to reweight the spectral contributions of the frozen encoder, allocating capacity to directions that were under-explored during pre-training. To this end, we propose Spectral Reverse Prompt (SRP), a framework that explicitly rebalances spectral contributions through two complementary mechanisms. First, a spectral channel applies a learnable soft-thresholding mask to modulate singular directions, suppressing dominant components while amplifying weaker ones. Second, a null-space channel extracts informative variation from directions that are minimally expressed by the frozen encoder, further expanding the effective representation space. By focusing adaptation on underutilized spectral components, SRP improves parameter efficiency while enhancing robustness to distribution shift. Empirical results across diverse benchmarks demonstrate that this spectral rebalancing leads to consistent performance gains over existing graph prompting methods across node-level and graph-level benchmarks. Our main contributions can be summarized as follows: • We identify and quantify a form of spectral bias in pre-trained graph models using a Downstream–Pretraining Misalignment Score (DPMS), showing that dominant singular directions are disproportionately aligned with pre-training objectives, while lower-energy directions remain underutilized for downstream tasks. . • We propose SRP, a parameter-efficient graph prompting framework that rebalances spectral contributions of frozen encoders via two complementary components: (i) a learnable soft-thresholding spectral mask that suppresses dominant directions and amplifies weaker ones, and (i) a null-space augmentation module that captures variation in directions minimally expressed by the encoder. . • We show that emphasizing underutilized spectral components leads to more effective and robust adaptation, particularly in low-resource settings. 2 Related Work GNNs and pre-training. Numerous studies propose training powerful GNNs via self-supervised learning, which is broadly categorized into two paradigms: contrastive and generative methods. Contrastive methods aim to maximize agreement between augmented instances. Representative works include DGI Veličković et al. (2019) and InfoGraph Sun et al. (2020), which focus on mutual information across different structural scales; GraphCL You et al. (2020), GRACE Zhu et al. (2020), and SimGRACE Xia et al. (2022) generate contrastive views via structural augmentation, feature masking, or parameter perturbation. Other paradigms avoid negative sampling through bootstrapping Thakoor et al. (2021) or exploit domain-specific motifs Zhang et al. (2021). Conversely, generative methods reconstruct specific graph properties: GraphMAE and GraphMAE2 Hou et al. (2022); Hou et al. (2023) focus on masked node features, while earlier strategies Hu et al. (2020b) leverage edge prediction. This paradigm of pre-training then fine-tuning has become the core of research in graph domain adaptation Fang et al. (2025), graph foundation models (Wang et al., 2024; Wang et al., 2025; Yuan et al., 2025; He et al., 2026a; He et al., 2026b) and graph reasoning mechanisms Xie et al. (2026) in recent years. Graph tuning methods. Graph prompt tuning and low-rank adaptation, or LoRA, offer parameter-efficient alternatives by optimizing minimal components alongside the backbone. GraphLoRA Yang et al. (2025) extends traditional LoRA Hu et al. (2022) by incorporating structure-aware contrastive losses. Meanwhile, graph prompting methods typically inject learnable tokens into various stages while preserving fixed message-passing rules. These approaches target different levels: node-feature prompting via GPF Fang et al. (2023), edge-level adaptation through EdgePrompt Fu et al. (2025a), and topology modifications in GraphTOP(Fu et al., 2025b), UniPrompt Huang et al. (2025) and CurvPrompt Wang et al. (2026a). Further research explores representation-level prompting such as GraphPrompt Liu et al. (2023) and GPPT Sun et al. (2022), as well as specialized designs: virtual-node prompting Tan et al. (2023) and instance-aware schemes Li et al. (2025). Despite their success, these approaches primarily adapt inputs or structures without explicitly analyzing the consensus between prompting mechanisms and weight metrics. This gap motivates our work: we rebalance spectral contributions and utilize the null space to enable universal parameter efficient, task-driven adaptation. 3 Notations and Preliminaries General Graphs. Let =(,ℰ,,)G=(V,E,X,Y) be a graph with N nodes, where =v1,…,vNV=\v_1,…,v_N\ and ℰ⊆×E ×V define the node and edge sets, respectively. The node attributes are captured by a feature matrix ∈ℝN×FX ^N× F, with each row iX_i representing the F-dimensional feature of node viv_i. Structural connectivity is denoted by the adjacency matrix ∈0,1N×NA∈\0,1\^N× N, where ij=1A_ij=1 indicates an existing edge between viv_i and vjv_j. Furthermore, each node viv_i corresponds to a specific label yiy_i from the label space Y. Note that P⁡(⋅)P(·) represents a probability distribution, which is utilized conceptually in our context rather than for strict mathematical derivations. Graph Prompt Tuning. Traditional pretrain-finetuning paradigms jointly update a pretrained graph encoder fθf_θ and a task-specific head gϕg_φ using downstream data. In contrast, graph prompt learning freezes fθf_θ and optimizes only a newly introduced set of prompt parameters Ψ . The generalized objective for such prompt-based adaptation on a downstream dataset D is formulated as: maxΨ⁡1||∑(,,y)∈∑i=1Nlog⁡P⁡(yi∣PredictΨ(,,vi,fθ)), _ 1|D| _(A,X,y) _i=1^N P (y_i _ (A,X,v_i;f_θ ) ), (1) where =(,,y)D=\(A,X,y)\. Here, PredictΨ(⋅)Predict_ (·) serves as a generic prediction module that leverages the structural topology A, node features X, the target node viv_i, and the frozen backbone fθf_θ to estimate the corresponding label yiy_i. Depending on the architecture, the prompt Ψ can be injected at various stages: modifying the raw topological and feature inputs (input-level), embedding into intermediate network layers (layer-wise), or adjusting the final output embeddings before the classifier (representation-level). Singular Value Decomposition. For any given weight matrix ∈ℝm×nW ^m× n with rank r, its compact Singular Value Decomposition (SVD) and the induced right null space are given by =⊤,ker⁡()=span⁡()⟂=∈ℝn∣⊤=0.W=U V , (W)=span(V) = \x ^n x=0 \. (2) In this factorization, ∈ℝm×rU ^m× r and ∈ℝn×rV ^n× r consist of orthonormal columns known as the left and right singular vectors, respectively. The term ∈ℝr×r ^r× r is a diagonal matrix containing the sorted, positive singular values σii=1r\ _i\_i=1^r of W in descending order. Accordingly, the right null space of W is the orthogonal complement of the subspace spanned by its right singular vectors. 4 The Spectral Bias of Pre-trained GNNs To establish our theoretical framework, we investigate a frozen GNN encoder with a weight matrix W∈ℝdout×dinW ^d_out× d_in at an arbitrary layer. For downstream adaptation, we assume Wdown∈ℝdin×CW_down ^d_in× C represents the optimal linear classifier trained directly on raw node features. Origins of Spectral Bias. The spectral bias in pre-trained GNNs fundamentally stems from two compounded phenomena: the nature of gradient descent and the graph message-passing mechanism. During pre-training, W is optimized over T steps of gradient descent. Each step applies a rank-1 update ΔW(t)=−ηtg(t)(h(t))⊤ W^(t)=- _tg^(t)(h^(t)) based on the upstream gradient g(t)g^(t) and layer input h(t)h^(t). For a GNN, let h(t)=ϕ⁡(A^W~(t)h~(t))h^(t)=φ( A W^(t) h^(t)) and let the adjacency matrix be eigendecomposed as A^=QΛQ⊤ A=Q Q , the cumulative update is ΔW=W(T)−W(0)=−∑t=1Tηtg(t)(h(t))⊤=−∑t=1Tηtg(t)[ϕ(QΛQ⊤W~(t)h~(t))]⊤ W=W^(T)-W^(0)=- _t=1^T _tg^(t)(h^(t)) =- _t=1^T _tg^(t) [φ\! (Q Q W^(t) h^(t) ) ] (3) Since each update deposits energy along h(t)h^(t), the spectral structure of ΔW W is governed by the input covariance Σh=t[h(t)(h(t))⊤] _h=E_t[h^(t)(h^(t)) ]. Let Σh=∑iλieiei⊤ _h= _i _ie_ie_i (with λ1≥λ2≥⋯ _1≥ _2≥·s) and Pr=∑i=1reiei⊤P_r= _i=1^re_ie_i be the projection onto the top-r principal directions. Proposition 4.1 (Spectral Alignment and Concentration). If the training inputs satisfy ‖(I−Pr)h(t)‖2≤δ‖h(t)‖2\|(I-P_r)h^(t)\|_2≤δ\|h^(t)\|_2 for all t, and the cross-step gradient terms vanish, then the cumulative gradient update inherits this concentration: ‖PrΔW⊤‖F2‖ΔW‖F2=1−‖(I−Pr)ΔW⊤‖F2‖ΔW‖F2≥1−δ2. \|P_r W \|_F^2\| W\|_F^2=1- \|(I-P_r) W \|_F^2\| W\|_F^2≥ 1-δ^2. (4) The detailed proof is provided in the Appendix B. Proposition 4 reveals that when inputs are confined to a low-dimensional subspace, gradient updates restrict their energy to the same principal subspace. Crucially, this gradient-induced bias is compounded by GNN message passing h(ℓ)=ϕ⁡(A^W(ℓ)h(ℓ−1))h^( )=φ( A\,W^( )h^( -1)), which acts as a graph-spectral filter. Over L layers, scaling frequency components by λiL(A^) _i^L( A) causes an anisotropic contraction of the covariance matrix. This asymptotically forces node representations to collapse toward a dominant vector, severely skewing the spectral distribution of the pre-trained weights. Downstream Capacity and Misalignment. For a frozen encoder W, the downstream classifier is strictly confined to the spectral reachable set ℋ⁡(W):=h↦a⊤(Wh):a∈ℝdoutH(W):=\h a (Wh):a ^d_out\. Since col(W⊤)=col(V)col(W )=col(V), the encoder restricts any downstream head to a k-dimensional subspace (where k=rank(W)k=rank(W)). The minimal approximation error against the optimal downstream classifier WdownW_down is: ε2(W):=minA⁡‖Wdown−W⊤A‖F2=‖PWdown‖F2, ^2(W):= _A\|W_down-W A\|_F^2=\|P_N\,W_down\|_F^2, (5) which is achieved at A∗=UΣ−1V⊤WdownA^*=U ^-1V W_down, leaving an irreducible residual defined by the null space projection P=I−VV⊤P_N=I-V . We further find that applying a computationally inexpensive PCA to the null space leads to substantially higher information density than the Encoder. As shown in Figure 1, using only 8 dimensions, the resulting representation achieves classification performance comparable to that of a 128 dimensional Encoder obtained through costly training. To explicitly quantify the tension between downstream demand and pre-training capacity, we introduce the Downstream-Pretraining Misalignment Score for each singular direction vjv_j. By partitioning the singular values into quintile groups Qqq=15\Q_q\_q=1^5, we further define the aggregate net gap (Δq _q): DPMSj:=‖Wdown⊤vj‖2σj2,Δq:=∑j∈Qq‖Wdown⊤vj‖2∑j‖Wdown⊤vj‖2−∑j∈Qqσj2∑jσj2.DPMS_j:= \|W_down v_j\|^2 _j^2, _q:= _j∈ Q_q\|W_down v_j\|^2 _j\|W_down v_j\|^2- _j∈ Q_q _j^2 _j _j^2. (6) Figure 1: Left: DPMS and Δq _q in the singular values of the quintices of WdownW_down. Right: Information density of encoder and null space. Theorem 4.2 (Spectral Reversal). The interaction between pre-trained capacity and downstream demand exhibits the following properties: • Reachability Limits: Target signals within col(V)col(V) are fully recoverable via aj=σj−1uja_j= _j^-1u_j, whereas signals in the null space ker⁡(W) (W) yield an irreducible approximation error ε2(W)=‖PWdown‖F2 ^2(W)=\|P_NW_down\|_F^2. • Capacity Misalignment: DPMSjDPMS_j grows inversely with the pre-trained capacity σj2 _j^2. Consequently, severe spectral concentration, where σi≫σj _i _j, forces the ratio DPMSj/DPMSiDPMS_j/DPMS_i to diverge, which is regionally quantified by the net gap Δq _q. The theorem above formalizes the fundamental tension between the pre-training and fine-tuning phases. The first property indicates that a frozen backbone inherently introduces blind spots: any downstream feature reliance that falls into the null space of the pre-trained weights simply cannot be recovered, regardless of how the linear head is optimized. The second property further reveals that even within the reachable subspace, there is a severe mismatch in resource allocation. The metric DPMSjDPMS_j acts as a demand-to-supply ratio. Due to spectral bias, the pre-trained model over-allocates its capacity to dominant directions, leading to a low DPMS and a negative net gap representing redundant supply. Conversely, the tail directions, which are often crucial for specific downstream tasks, are starved of capacity, resulting in a critically high DPMS and a positive net gap. This structural misalignment strongly motivates the need for targeted spectral adaptation. Theorem 4.3 (PAC-Bayes Generalization Bound). Consider a learnable adaptor Θ restricted to a bottleneck rank rsr_s across exactly |w||V_w| weak spectral directions, yielding a parameter count of dΘ=⁡(|w|rs)d_ =O(|V_w|r_s). Let R^(Θ^) R( ) and R⁡(Θ^)R( ) be the empirical and expected risks on n samples. Under McAllester’s bound with a Gaussian prior bounded by the backbone’s condition number κ⁡(W):=σmax/σminκ(W):= _max/ _min, and an analytically optimized posterior, with probability ≥1−δ≥ 1-δ: R⁡(Θ^)≤R^(Θ^)+⁡(|w|rslog⁡κ⁡(W)+log⁡(n/δ)n).R( )\;≤\; R( )+O ( |V_w|r_s κ(W)+ (n/δ)n ). (7) Proof Sketch and Motivation. By Theorem 4.2, the effective signal scale per target direction is restricted by 1/σmin1/ _min, which strictly bounds the parameter norm as ‖Θ^‖2=⁡(dΘ/σmin2)\| \|^2=O(d_ / _min^2). Optimizing the posterior variance collapses the KL divergence to ⁡(dΘlog⁡κ(W))O(d_ κ(W)), yielding Equation (7). Crucially, this bound demonstrates that the generalization gap depends explicitly on the number of activated weak directions |w||V_w|, rather than the massive original feature dimensions dindoutd_ind_out. This offers a profound theoretical foundation: by deliberately restricting trainable capacity to only the highly misaligned subspace, one can rectify the spectral bias while inherently securing a significantly tighter generalization guarantee than full fine-tuning. This structural insight directly motivates our algorithmic design in the subsequent section. Figure 2: Overall framework of Spectral Reverse Prompt (SRP). SRP decomposes each frozen GNN layer weight to expose spectral bias, then uses a spectral channel to suppress dominant directions and amplify weak ones, and a null-space channel to exploit information that is not captured by the frozen encoder. The two channels are fused into layer-wise prompts injected into the frozen GNN for downstream tuning. 5 Method: Spectral Reverse Prompt Motivated by the spectral reversal principle, we design our method, Spectral Reverse Prompt (SRP). Our approach introduces a layer-wise graph prompt that counteracts singular values to better align the pretrained models with downstream few-shot tasks. SRP adds prompting vector to node feature: z~(ℓ)=A^W(ℓ)h(ℓ−1)+b(ℓ)+p(ℓ)(h(ℓ−1)). z^( )= A\,W^( )h^( -1)+b^( )+p^( )(h^( -1)). (8) Each prompt consists of two complementary channels on orthogonal subspaces of ℝdinR^d_in: a spectral channel that reverses the encoder’s spectral weighting, and a null-space channel that extracts signal from directions invisible to W. 5.1 Spectral Channel: Soft-Threshold Reversal Motivated by Theorem 4.2, this channel amplifies weak directions while suppressing dominant ones. Node features are projected onto the right-singular basis: hspec=V⊤h∈ℝkh_spec=V h ^k (the spectral coordinates of h). A learnable soft-threshold mask is then applied: τ=sigmoid⁡(τ0),μj=sigmoid⁡((τσmax−σj)⋅γ),τ=sigmoid( _0), _j=sigmoid\! ((τ _max- _j)·γ ), (9) where τ0 _0 is a scalar threshold parameter, σmax _max the largest singular value of W, γ is a pre-modulatable temperature coefficient that can be freely adjusted according to specific experiments, and μj∈(0,1) _j∈(0,1) the mask weight for direction j. By construction μj≈0 _j≈ 0 for strong directions and μj≈1 _j≈ 1 for weak ones. The masked coordinates are linearly projected: zspec=(hspec⊙μ)Asz_spec=(h_spec μ)\,A_s, where As∈ℝrs×kA_s ^r_s× k is a learnable bottleneck adaptor of rank rsr_s. 5.2 Null-Space Channel When din>kd_in>k, the null space ker⁡(W) (W) has dimension din−kd_in-k. Features in this subspace are completely invisible to the frozen encoder and inaccessible to the spectral channel. We project the feature matrix X onto ker⁡(W) (W) and extract the top-m principal components: ⟂=⁡(I−VV⊤),znull=An(V⟂h),X_ =X(I-V ), z_null=A_n\,(V_ h), (10) where V⟂∈ℝm×dinV_ ^m× d_in is the orthonormal PCA basis of the null-space-projected data ⟂X_ , and An∈ℝrs×mA_n ^r_s× m is a learnable adaptor. By construction WV⟂=0WV_ =0 and V⊤V⟂=0V V_ =0: the two channels operate on orthogonal subspaces. Both V⟂V_ and V are frozen; only AnA_n is learned. Figure 1 and proofs given by Appendix B confirm that NS-PCA directions carry higher per-dimension accuracy than the encoder output and that PCA ordering substantially outperforms random null-space selection. 5.3 Channel Fusion and Output The two channels are summed in a shared bottleneck, then projected to the GNN activation space: p⁡(h)=Q⁡[As(VTh⊙μ)⏟spectral channel+An(V⟂h)⏟null-space channel]+bp,p(h)=Q\, [ A_s\,(V^Th μ)_spectral channel\;+\; A_n\,(V_ h)_null-space channel ]+b_p, (11) where Q and bpb_p are the shared projection and bias. This fusion is lossless since V⊤V⟂⊤=0V V_ =0. When din≤k+md_in≤ k+m, SRP uses a projective fallback p⁡(h)=(hweakP)Q+bpp(h)=(h_weakP)Q+b_p. Here, hweak=h−(hV⊙(1−μ))V⊤h_weak=h-(hV (1-μ))V directly preserves all weak spectral and null-space signals. Thus, a single adaptor P∈ℝdin×rsP ^d_in× r_s becomes mathematically isomorphic to the dual-channel architecture (Appendix B.5). 5.4 Complexity Analysis The proposed SRP exhibits exceptional efficiency, characterized by strictly linear time complexity with respect to the graph size and a trainable parameter footprint drastically smaller than standard LoRA-style adaptations. Time Complexity. Initial SVD and null-space computations are performed once offline. During training, by pre-computing the heavy input projections, SRP’s forward-pass overhead is reduced to ⁡(||rs(k+m+h))O(|V|r_s(k+m+h)). Since rs≪hr_s h and k,m≪dink,m d_in, this linear cost is asymptotically negligible compared to the backbone’s heavy message-passing complexity of ⁡(|ℰ|din)O(|E|d_in). Space Complexity. SRP introduces a minimal set of trainable parameters. For a standard n-layer network, combining the initial dual-channel layer and n−1n-1 projective fallback layers yields exactly rs(k+m+(2n−1)h)+nh+nr_s(k+m+(2n-1)h)+nh+n trainable parameters. This scales strictly as ⁡(nrsh)O(nr_sh), avoiding input-dimension (dind_in) dependencies, making it far smaller than full fine-tuning or LoRA-style baselines. Additionally, frozen buffers (V,V⟂⊤V,V_ ) require only ⁡(din(k+m)+nhk)O(d_in(k+m)+nhk) memory, while activation memory remains ⁡(||din)O(|V|d_in), identical to the unmodified backbone. 6 Experiments 6.1 Experimental Setup Datasets. We evaluate the effectiveness of SRP on ten benchmark datasets spanning both node- and graph-level classification tasks. For node classification (5-shot), we consider five widely used benchmarks, including the citation networks Cora (Yang et al., 2016), CiteSeer (Yang et al., 2016), and PubMed (Yang et al., 2016), as well as the large-scale datasets Flickr (Zeng et al., 2020) and ogbn-arxiv (Hu et al., 2020a). For graph classification (50-shot), we use five molecular and biological graph datasets from TUDataset (Morris et al., 2020): ENZYMES, D, NCI1, NCI109, and Mutagenicity. These benchmarks exhibit substantial diversity in graph scale, topology, and feature dimensionality. Detailed dataset statistics are provided in Appendix A. Pre-training strategies and Baselines. To evaluate the compatibility and generality of SRP across different self-supervised paradigms, we conduct experiments under two representative pre-training strategies: GraphCL (You et al., 2020) and SimGRACE (Xia et al., 2022). We compare SRP against ten representative baselines, including: Classifier Only (a linear classifier trained on frozen features), GPPT (Sun et al., 2022) (link-prediction-based prompting), GraphPrompt (Liu et al., 2023) (readout-level prompting), All-in-one (Sun et al., 2023a) (multi-task prompting), GPF and GPF-plus (Fang et al., 2023) (feature-level prompting), EdgePrompt and EdgePrompt-plus (Fu et al., 2025a) (edge-level prompting), and GraphLoRA (Yang et al., 2025) (structure-aware low-rank adaptation). For fair comparison, all methods use the same backbone architecture, pre-trained checkpoints, and evaluation protocol following Prog (Zi et al., 2024). Table 1: Performance on node-level benchmarks (5-seed avg.). Bold = best, underline = second best. Pre-training Tuning Methods Cora CiteSeer PubMed ogbn-arxiv Flickr GraphCL Classifier Only 53.05±4.7653.05_± 4.76 38.62±3.4338.62_± 3.43 64.28±4.5164.28_± 4.51 21.15±1.6421.15_± 1.64 24.32±2.9324.32_± 2.93 GPPT 50.96±6.6750.96_± 6.67 39.50±1.6739.50_± 1.67 60.47±4.7560.47_± 4.75 17.99±1.1417.99_± 1.14 24.35±1.8424.35_± 1.84 GraphPrompt 55.71±4.6255.71_± 4.62 40.81±2.1140.81_± 2.11 63.47±2.2363.47_± 2.23 21.03±1.9221.03_± 1.92 26.08±3.4426.08_± 3.44 ALL-in-one 38.00±4.1738.00_± 4.17 40.27±2.0940.27_± 2.09 58.61±3.4958.61_± 3.49 16.42±2.9816.42_± 2.98 25.08±3.4425.08_± 3.44 GPF 58.52±4.0758.52_± 4.07 43.55±2.8043.55_± 2.80 67.67±3.1467.67_± 3.14 21.73±1.7521.73_± 1.75 23.98±1.7123.98_± 1.71 GPF-plus 52.24±4.5952.24_± 4.59 38.47±3.2738.47_± 3.27 64.30±4.5864.30_± 4.58 21.03±1.9621.03_± 1.96 25.32±2.0225.32_± 2.02 EdgePrompt 58.60±4.4658.60_± 4.46 43.31±3.2343.31_± 3.23 67.76±3.0167.76_± 3.01 21.90±1.7121.90_± 1.71 24.83±2.7824.83_± 2.78 EdgePrompt+ 62.88±6.4362.88_± 6.43 46.20±0.9946.20_± 0.99 67.41±5.2567.41_± 5.25 23.18±1.2623.18_± 1.26 25.57±3.0425.57_± 3.04 GraphLoRA 63.35±4.3263.35_± 4.32 45.98±3.9745.98_± 3.97 67.21±3.1567.21_± 3.15 23.26±1.8923.26_± 1.89 25.71±3.1625.71_± 3.16 SRP (Ours) 68.11±2.29 68.11_± 2.29 49.02±2.26 49.02_± 2.26 68.81±3.92 68.81_± 3.92 24.75±1.79 24.75_± 1.79 26.39±2.11 26.39_± 2.11 SimGRACE Classifier Only 52.27±2.7452.27_± 2.74 40.45±3.5540.45_± 3.55 56.72±3.8056.72_± 3.80 20.75±2.9220.75_± 2.92 25.53±3.9825.53_± 3.98 GPPT 52.07±7.6552.07_± 7.65 40.25±3.2940.25_± 3.29 58.65±5.1258.65_± 5.12 17.76±1.8017.76_± 1.80 23.37±4.6623.37_± 4.66 GraphPrompt 51.42±2.8051.42_± 2.80 41.74±2.2241.74_± 2.22 55.98±2.9455.98_± 2.94 20.48±2.5720.48_± 2.57 25.88±3.8125.88_± 3.81 ALL-in-one 34.64±4.0634.64_± 4.06 38.95±2.3538.95_± 2.35 54.18±4.7054.18_± 4.70 16.72±2.9016.72_± 2.90 27.68±4.5827.68_± 4.58 GPF 58.23±4.1958.23_± 4.19 44.87±4.3544.87_± 4.35 61.55±3.7961.55_± 3.79 21.86±2.9121.86_± 2.91 26.51±4.6926.51_± 4.69 GPF-plus 52.27±3.3452.27_± 3.34 41.02±3.4941.02_± 3.49 56.95±3.8656.95_± 3.86 21.44±3.7721.44_± 3.77 28.35±5.5028.35_± 5.50 EdgePrompt 58.37±4.5158.37_± 4.51 43.94±4.1543.94_± 4.15 61.10±3.6961.10_± 3.69 21.85±2.5421.85_± 2.54 30.12±5.04 30.12_± 5.04 EdgePrompt+ 62.40±7.9762.40_± 7.97 46.62±2.5346.62_± 2.53 64.91±5.5864.91_± 5.58 22.74±2.3422.74_± 2.34 28.50±4.0828.50_± 4.08 GraphLoRA 63.06±3.64¯ 63.06_± 3.64 46.39±6.1546.39_± 6.15 64.33±5.2364.33_± 5.23 23.22±2.9723.22_± 2.97 28.02±3.5428.02_± 3.54 SRP (Ours) 67.24±7.31 67.24_± 7.31 51.37±5.24 51.37_± 5.24 67.32±4.10 67.32_± 4.10 24.01±2.91 24.01_± 2.91 28.48±3.2628.48_± 3.26 Table 2: Performance on graph-level benchmarks (5-seed avg.). Bold = best, underline = second best. Pre-training Tuning Methods ENZYMES D NCI1 NCI109 Mutagenicity GraphCL Classifier Only 30.50±1.1630.50_± 1.16 62.89±2.1962.89_± 2.19 62.49±1.9562.49_± 1.95 61.68±0.9361.68_± 0.93 66.62±1.8766.62_± 1.87 GraphPrompt 27.83±1.6127.83_± 1.61 64.33±1.7964.33_± 1.79 63.19±1.7163.19_± 1.71 62.18±0.4862.18_± 0.48 67.62±0.6567.62_± 0.65 ALL-in-one 25.92±0.5525.92_± 0.55 66.54±1.8266.54_± 1.82 57.52±2.6157.52_± 2.61 62.74±0.7862.74_± 0.78 63.43±2.5363.43_± 2.53 GPF 30.08±1.2530.08_± 1.25 64.54±2.2264.54_± 2.22 62.66±1.8362.66_± 1.83 62.29±0.9062.29_± 0.90 66.54±1.8566.54_± 1.85 GPF-plus 31.00±1.5031.00_± 1.50 67.26±2.2967.26_± 2.29 64.56±1.1064.56_± 1.10 62.84±0.2262.84_± 0.22 66.82±1.6366.82_± 1.63 EdgePrompt 29.50±1.5729.50_± 1.57 64.16±2.1364.16_± 2.13 63.05±2.1163.05_± 2.11 62.59±0.9362.59_± 0.93 66.87±1.8866.87_± 1.88 EdgePrompt+ 34.00±1.2534.00_± 1.25 67.98±2.0567.98_± 2.05 66.30±2.5466.30_± 2.54 66.52±0.9166.52_± 0.91 67.47±2.3767.47_± 2.37 GraphLoRA 36.05±1.8936.05_± 1.89 68.33±1.9368.33_± 1.93 66.54±2.7166.54_± 2.71 66.40±0.9666.40_± 0.96 67.12±2.1167.12_± 2.11 SRP (Ours) 36.32±2.65 36.32_± 2.65 69.71±1.72 69.71_± 1.72 66.93±1.81 66.93_± 1.81 68.18±0.55 68.18_± 0.55 69.35±1.80 69.35_± 1.80 SimGRACE Classifier Only 27.07±1.0427.07_± 1.04 61.77±2.4061.77_± 2.40 61.27±3.6461.27_± 3.64 62.12±1.1062.12_± 1.10 67.36±0.7167.36_± 0.71 GraphPrompt 26.87±1.4726.87_± 1.47 62.58±1.8462.58_± 1.84 62.45±1.5262.45_± 1.52 62.41±0.6962.41_± 0.69 68.03±0.7868.03_± 0.78 ALL-in-one 25.73±1.1825.73_± 1.18 65.16±1.4765.16_± 1.47 58.52±1.5958.52_± 1.59 62.01±0.6662.01_± 0.66 64.43±1.0064.43_± 1.00 GPF 28.53±1.7628.53_± 1.76 65.64±0.7065.64_± 0.70 61.45±3.1361.45_± 3.13 61.90±1.2661.90_± 1.26 67.19±0.7467.19_± 0.74 GPF-plus 27.33±2.0127.33_± 2.01 67.20±1.5667.20_± 1.56 61.61±2.8961.61_± 2.89 62.84±0.2362.84_± 0.23 67.69±0.6467.69_± 0.64 EdgePrompt 29.33±2.3029.33_± 2.30 63.97±2.1463.97_± 2.14 62.02±3.0262.02_± 3.02 62.02±1.0362.02_± 1.03 67.55±0.8567.55_± 0.85 EdgePrompt+ 32.67±2.5332.67_± 2.53 67.72±1.6267.72_± 1.62 67.07±1.96 67.07_± 1.96 66.53±1.3066.53_± 1.30 68.31±1.3668.31_± 1.36 GraphLoRA 35.17±1.4235.17_± 1.42 67.02±1.9767.02_± 1.97 66.83±2.3966.83_± 2.39 65.33±1.0465.33_± 1.04 67.58±1.6767.58_± 1.67 SRP (Ours) 35.38±1.88 35.38_± 1.88 68.12±1.97 68.12_± 1.97 66.53±2.9466.53_± 2.94 67.07±0.83 67.07_± 0.83 68.92±2.73 68.92_± 2.73 6.2 Main Results We compare the node-level and graph-level classification accuracy of our methods and other baselines. Table 1 and Table 2 report results under GraphCL and SimGRACE pre-training. We make several key observations. First, SRP consistently achieves the best or second best performance across almost all dataset. Second, the gains are most pronounced on node classification benchmarks with high input dimension, where the null-space channel can extract substantial discriminative signal from the frozen encoder. Third, on graph classification benchmarks where the input dimension is small, SRP operates via its projective fallback (Section 5.3), yet remains superior to the strongest baselines, confirming that the spectral reversal mask provides meaningful gains. These results validate the central thesis of this work: inverting the spectral priority of frozen GNN weights is a broadly effective strategy for graph prompt tuning. 6.3 Ablation Study To disentangle the contributions of different components in SRP, we evaluate four ablation variants under both GraphCL and SimGRACE pre-training settings: SRP-NM removes the null-space channel, SRP-NS removes the spectral channel, SRP-NR disables spectral reversal by adopting uniform weighting across all singular directions (i.e., μj≡1 _j≡ 1), and SRP-Bi replaces the reverse weighting strategy with a bidirectional mask that amplifies both dominant and trailing spectral directions. The results in Table 3 reveal a consistent trend across both pre-training frameworks. Effectiveness of the spectral and null-space channels. Both the spectral channel and the null-space channel play essential roles in extracting discriminative information for downstream adaptation. Removing either component leads to clear performance degradation. In particular, SRP-NM and SRP-NS consistently underperform the full SRP model across all datasets and pre-training settings, demonstrating that both weak spectral directions and null-space information contribute substantially to effective prompt learning. Effectiveness of the spectral reversal principle. SRP improves adaptation by reversing the pretrained spectral bias and emphasizing weak singular directions. When this mechanism is removed (SRP-NR) or replaced with bidirectional amplification (SRP-Bi), performance consistently declines relative to the full SRP model. These results suggest that selectively emphasizing trailing singular directions is more effective than uniformly weighting or jointly amplifying both spectral extremes for few-shot graph adaptation. Table 3: Ablation Study of Different Variants on GraphCL and SimGRACE. Variant GraphCL SimGRACE Cora CiteSeer Avg. Δ Cora CiteSeer Avg. Δ SRP (full) 68.1±2.3 68.1_± 2.3 49.0±2.3 49.0_± 2.3 −0.0-0.0 67.2±7.3 67.2_± 7.3 51.4±5.2 51.4_± 5.2 −0.0-0.0 SRP-NM (w/o null channel) 63.4±2.863.4_± 2.8 41.1±3.841.1_± 3.8 −6.3-6.3 60.3±5.960.3_± 5.9 43.5±2.943.5_± 2.9 −7.4-7.4 SRP-NS (w/o spectral channel) 64.4±2.164.4_± 2.1 45.6±3.245.6_± 3.2 −3.6-3.6 63.5±6.663.5_± 6.6 47.7±4.347.7_± 4.3 −3.7-3.7 SRP-NR (w/o reverse) 64.7±2.764.7_± 2.7 48.0±4.048.0_± 4.0 −2.2-2.2 63.4±4.463.4_± 4.4 49.9±4.949.9_± 4.9 −2.7-2.7 SRP-Bi (bidirectional) 64.7±3.364.7_± 3.3 47.9±3.347.9_± 3.3 −2.3-2.3 63.0±4.563.0_± 4.5 50.0±4.850.0_± 4.8 −2.8-2.8 6.4 Additional Evaluation Label sensitivity We analyzed the impact of the number of labels for each class during prompt tuning in downstream tasks, as shown in Figure 3. We found that the performance of node classification tasks fluctuates significantly with changes in the number of samples when the shot number is extremely few. In contrast, graph classification tasks are less affected by the number of labels. Parameter sensitivity. We study the sensitivity of SRP to its two key hyperparameters: the null-space PCA dimension m and the adaptor bottleneck rank rsr_s and numbers of labels. m controls how many low-amplification singular directions are used to construct the spectral reparameterization subspace, rsr_s determines the capacity of the lightweight adapter, shot number reflects the ability to fine tune the model with a limited number of labels. Figure 3 reports 5-shot node classification accuracy under GraphCL as we vary rs∈16,32,64,128r_s∈\16,32,64,128\ and m∈4,8,16,32m∈\4,8,16,32\ independently. Performance remains stable across a wide range of values, with accuracy plateauing at the default settings (rs=32r_s=32, m=16m=16), confirming that SRP is robust to the choice of these hyperparameters and does not require extensive tuning in practice. Convergence Speed. We also investigate the convergence speed of our method compared with other strong baselines. Figure 4 illustrates the accuracy curves of our method and the baselines under two pre-training strategies. According to Figure 4, we can observe that SRP can generally converge faster than other methods, which means extra efficiency by adopting fewer epoch numbers in tuning the whole pre-trained models. Figure 3: Label Sensitivity and Parameter Sensitivity of NS-PCA and bottleneck rank. Figure 4: Convergence speed of different methods. 7 Conclusion In this work, we identified spectral bias as a key bottleneck in adapting pre-trained GNNs, where self-supervised objectives overemphasize dominant singular directions while underutilizing lower-energy components. Using the DPMS, we empirically and theoretically showed that downstream utility is often concentrated in these underexplored spectral directions. To address this misalignment, we introduced SRP, a parameter-efficient framework that rebalances spectral contributions of frozen encoders via soft-thresholding and null-space augmentation. By reallocating capacity toward underutilized components, SRP achieves state-of-the-art performance across diverse benchmarks while using less than 7% of backbone parameters. Our results highlight a simple but important principle: in parameter-efficient adaptation, which directions are adapted can matter more than how many parameters are introduced. Limitations and Future Works. While SRP remains robustness across different graph homophily levels and backbone architectures according to our additional results C, its spectral reweighting currently relies on a shared singular basis and a global threshold, which may not fully capture region-specific frequency heterogeneity. In addition, the effectiveness of the null-space channel depends on the relationship between input dimensionality and encoder rank, and can be limited when the frozen transformation has little or no null space. Future work includes instance- or structure-adaptive spectral reweighting driven by downstream feedback, as well as extending the spectral perspective to multi-modal graph representation learning. Acknowledgments and Disclosure of Funding This work was supported by the National Natural Science Foundation of China (No. 62276053); Center for HPC, University of Electronic Science and Technology of China; Sichuan Province Science and Technology Support Program (No. 2025ZDZX0016). References [1] X. Sun, J. Zhang, X. Wu, H. Cheng, Y. Xiong, and J. Li (2023) Graph prompt learning: a comprehensive survey and beyond. arXiv preprint arXiv:2311.16534. Cited by: §1. [2] X. Fu, Z. Wang, Z. Chen, J. Li, Y. Zhu, Z. Lei, C. Shen, Y. Ye, C. Zhang, and J. Li (2025) Graph prompting for graph learning models: recent advances and future directions. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD ’25, New York, NY, USA, p. 6010–6020. External Links: ISBN 9798400714542, Link, Document Cited by: §1. [3] T. N. Kipf and M. Welling (2017) Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, Cited by: §1. [4] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio (2018) Graph attention networks. In International Conference on Learning Representations, Cited by: §1. [5] K. Xu, W. Hu, J. Leskovec, and S. Jegelka (2019) How powerful are graph neural networks?. In International Conference on Learning Representations, Cited by: §1. [6] W. Hamilton, Z. Ying, and J. Leskovec (2017) Inductive representation learning on large graphs. In Advances in neural information processing systems, Cited by: §1. [7] S. Brody, U. Alon, and E. Yahav (2022) How attentive are graph attention networks?. In International Conference on Learning Representations, Cited by: §1. [8] T. Fang, Y. Zhang, Y. Yang, C. Wang, and L. Chen (2023) Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems. Cited by: §1, §2, §6.1. [9] X. Fu, Y. He, and J. Li (2025) Edge prompt tuning for graph neural networks. In International Conference on Learning Representations, Cited by: §1, §2, §6.1. [10] J. Li, J. Li, and C. Zhang (2025) Instance-aware graph prompt learning. Transactions on Machine Learning Research. Cited by: §1, §2. [11] M. Sun, K. Zhou, X. He, Y. Wang, and X. Wang (2022) Gppt: graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §1, §2, §6.1. [12] X. Yu, C. Zhou, Y. Fang, and X. Zhang (2024) MultiGPrompt for multi-task pre-training and prompting on graphs. In Proceedings of the ACM on Web Conference 2024, Cited by: §1. [13] X. Yu, Y. Fang, Z. Liu, and X. Zhang (2024) Hgprompt: bridging homogeneous and heterogeneous graphs for few-shot prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §1. [14] R. Huang, J. Xu, X. Jiang, C. Pan, Z. Yang, C. Wang, and Y. Yang (2024) Measuring task similarity and its implication in fine-tuning graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §1. [15] P. Veličković, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm (2019) Deep graph infomax. In International Conference on Learning Representations, Cited by: §2. [16] F. Sun, J. Hoffmann, V. Verma, and J. Tang (2020) Infograph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, Cited by: §2. [17] Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen (2020) Graph contrastive learning with augmentations. Advances in neural information processing systems. Cited by: §2, §6.1. [18] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang (2020) Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131. Cited by: §2. [19] J. Xia, L. Wu, J. Chen, B. Hu, and S. Z. Li (2022) Simgrace: a simple framework for graph contrastive learning without data augmentation. In Proceedings of the ACM Web Conference 2022, Cited by: §2, §6.1. [20] S. Thakoor, C. Tallec, M. G. Azar, R. Munos, P. Veličković, and M. Valko (2021) Bootstrapped representation learning on graphs. In ICLR Workshop on Geometrical and Topological Representation Learning, Cited by: §2. [21] Z. Zhang, Q. Liu, H. Wang, C. Lu, and C. Lee (2021) Motif-based graph self-supervised learning for molecular property prediction. Advances in Neural Information Processing Systems. Cited by: §2. [22] Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang (2022) Graphmae: self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §2. [23] Z. Hou, Y. He, Y. Cen, X. Liu, Y. Dong, E. Kharlamov, and J. Tang (2023) Graphmae2: A decoding-enhanced masked self-supervised graph learner. In Proceedings of the ACM on Web Conference, p. 737–746. Cited by: §2. [24] W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, and J. Leskovec (2020) Strategies for pre-training graph neural networks. In International Conference on Learning Representations, Cited by: §2. [25] R. Fang, B. Li, Z. Kang, Q. Zeng, N. H. Dashtbayaz, R. Pu, B. Wang, and C. Ling (2025) On the benefits of attribute-driven graph domain adaptation. In The Thirteenth International Conference on Learning Representations, Cited by: §2. [26] Z. Wang, Z. Zhang, N. V. Chawla, C. Zhang, and Y. Ye (2024) GFT: graph foundation model with transferable tree vocabulary. Advances in neural information processing systems. Cited by: §2. [27] S. Wang, B. Wang, Z. Shen, B. Deng, et al. (2025) Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment. In Forty-second International Conference on Machine Learning, Cited by: §2. [28] H. Yuan, Q. Sun, J. Shi, X. Fu, B. Hooi, J. Li, and P. S. Yu (2025) How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees. In Forty-second International Conference on Machine Learning, Cited by: §2. [29] X. He, H. He, R. Fang, M. Sun, and Z. Kang (2026) Structure-centric graph foundation model via geometric bases. In Proceedings of the 43rd International Conference on Machine Learning, Cited by: §2. [30] X. He, X. Wang, and Z. Kang (2026) SCGFM-art: amortized relational transport for structure-centric graph foundation models. arXiv preprint arXiv:2609.20419. Cited by: §2. [31] X. Xie, Z. Guo, B. Li, X. Yu, Z. Liao, Z. Kang, and Y. Fang (2026) Clustering as reasoning: a k-means interpretation of chain-of-thought graph learning. In Proceedings of the 43rd International Conference on Machine Learning, Cited by: §2. [32] Z. Yang, J. Han, C. Wang, and H. Liu (2025) Graphlora: Structure-aware contrastive low-rank adaptation for cross-graph transfer learning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, p. 1785–1796. Cited by: §2, §6.1. [33] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: §2. [34] X. Fu, Z. Lei, Z. Chen, B. Zhang, C. Zhang, and J. Li (2025) GraphTOP: graph topology-oriented prompting for graph neural networks. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §2. [35] Y. Huang, J. Zhao, D. He, X. Wang, Y. Li, Y. Huang, D. Jin, and Z. Feng (2025) One prompt fits all: universal graph adaptation for pretrained models. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §2. [36] Q. Wang, X. Xie, B. Li, X. Yu, S. Wang, R. Fang, and Z. Kang (2026) Dynamic graph prompting via topology-routed mixed-curvature experts. arXiv preprint arXiv:2608.06031. Cited by: §2. [37] Z. Liu, X. Yu, Y. Fang, and X. Zhang (2023) Graphprompt: unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, Cited by: §2, §6.1. [38] Z. Tan, R. Guo, K. Ding, and H. Liu (2023) Virtual node tuning for few-shot node classification. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §2. [39] Z. Yang, W. Cohen, and R. Salakhudinov (2016) Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, Cited by: §6.1. [40] H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V. Prasanna (2020) GraphSAINT: graph sampling based inductive learning method. In International Conference on Learning Representations, Cited by: §6.1. [41] W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec (2020) Open graph benchmark: datasets for machine learning on graphs. Advances in neural information processing systems. Cited by: §6.1. [42] C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann (2020) Tudataset: a collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663. Cited by: §6.1. [43] X. Sun, H. Cheng, J. Li, B. Liu, and J. Guan (2023) All in one: multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §6.1. [44] C. Zi, H. Zhao, X. Sun, Y. Lin, H. Cheng, and J. Li (2024) ProG: a graph prompt learning benchmark. arXiv preprint arXiv:2406.05346. Cited by: §6.1. [45] H. Pei, B. Wei, K. C. Chang, Y. Lei, and B. Yang (2020) Geom-gcn: geometric graph convolutional networks. In International Conference on Learning Representations, External Links: Link Cited by: Appendix A. [46] S. Wang, X. Wang, Q. Wang, B. Wu, B. Wang, S. Huang, B. Deng, H. Liu, R. Fang, Z. Xu, B. Wang, and Z. Kang (2026) The post-GCN decade revisited: curvature-stratified evaluation of relational learning. arXiv preprint arXiv:2606.06397. Cited by: Appendix A. [47] R. A. Horn and C. R. Johnson (2013) Matrix analysis. 2 edition, Cambridge University Press. Cited by: §B.3. [48] D. A. McAllester (1999) Some pac-bayesian theorems. Machine Learning 37 (3), p. 355–363. Cited by: §B.4. [49] B. Li, E. Pan, and Z. Kang (2024) PC-Conv: unifying homophily and heterophily with two-fold filtering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. Cited by: §C.3. Appendix A Dataset Statistics Tables 4 and 5 summarize the basic statistics of all ten benchmark datasets used in our main experiments and three benchmark in our extension to heterophilic graphs [45]. Table 4: Node-level benchmark datasets. #Feat. is the raw input feature dimension; #Classes is the number of downstream classes. Dataset #Nodes #Edges #Feat. #Classes Task Cora 2,708 10,556 1,433 7 Node CiteSeer 3,327 9,104 3,703 6 Node PubMed 19,717 88,648 500 3 Node ogbn-arxiv 169,343 1,166,243 128 40 Node Flickr 89,250 899,756 500 7 Node Cornell 183 298 1703 5 Node Squirrel 5201 217073 2089 5 Node Chameleon 2277 36101 2277 5 Node Table 5: Graph-level benchmark datasets (TUDataset). Avg. Nodes and Avg. Edges are computed over the graph collection. Dataset #Graphs Avg. Nodes Avg. Edges #Feat. #Classes Task ENZYMES 600 32.63 124.27 3 6 Graph D 1,178 284.32 1,431.32 89 2 Graph NCI1 4,110 29.87 64.60 37 2 Graph NCI109 4,127 29.68 64.26 38 2 Graph Mutagenicity 4,337 30.32 61.54 14 2 Graph Our next objective is to investigate how variations in curvature [46], an important structural property in graph representation learning, may affect the performance of our method. Appendix B Proofs of Theoretical Results In this section, we provide the rigorous mathematical proofs for the theoretical claims presented in Section 4. B.1 Proof of Proposition 4 (Spectral Concentration) Statement: Given the rank-1 update ΔW(t)=−ηtg(t)(h(t))⊤ W^(t)=- _t\,g^(t)(h^(t)) and the spectral alignment condition t[‖(I−Pr)h(t)‖2/‖h(t)‖2]≤δ2E_t [\|(I-P_r)h^(t)\|^2/\|h^(t)\|^2 ]≤δ^2, the cumulative gradient update satisfies: ⁡[‖PrΔW⊤‖F2]⁡[‖ΔW‖F2]≥ 1−δ2. E [\|P_r\, W \|_F^2 ]E [\| W\|_F^2 ]\;≥\;1-δ^2. Proof. Consider the transpose of the gradient update at a single step t: (ΔW(t))⊤=−ηth(t)(g(t))⊤.( W^(t)) =- _t\,h^(t)(g^(t)) . (12) We can decompose the input vector h(t)h^(t) into two orthogonal components using the projection matrix PrP_r and its orthogonal complement (I−Pr)(I-P_r): h(t)=Prh(t)+(I−Pr)h(t).h^(t)=P_rh^(t)+(I-P_r)h^(t). (13) The Frobenius norm squared of the update transposed is determined by the outer product of these vectors. Utilizing the property ‖ab⊤‖F2=‖a‖22‖b‖22\|ab \|_F^2=\|a\|_2^2\|b\|_2^2, we have: ‖(ΔW(t))⊤‖F2=ηt2‖h(t)‖22‖g(t)‖22.\|( W^(t)) \|_F^2= _t^2\|h^(t)\|_2^2\|g^(t)\|_2^2. (14) Similarly, the energy of the update residing outside the top-r principal subspace is: ‖(I−Pr)(ΔW(t))⊤‖F2=ηt2‖(I−Pr)h(t)‖22‖g(t)‖22.\|(I-P_r)( W^(t)) \|_F^2= _t^2\|(I-P_r)h^(t)\|_2^2\|g^(t)\|_2^2. (15) Assuming that the magnitude of the upstream gradient ‖g(t)‖22\|g^(t)\|_2^2 is independent of the normalized directional projection of h(t)h^(t) over the training trajectory, we can take the expectation over t: ⁡[‖(I−Pr)ΔW⊤‖F2]=⁡[‖(I−Pr)h(t)‖22‖h(t)‖22⋅ηt2‖h(t)‖22‖g(t)‖22].E [\|(I-P_r) W \|_F^2 ]=E [ \|(I-P_r)h^(t)\|_2^2\|h^(t)\|_2^2· _t^2\|h^(t)\|_2^2\|g^(t)\|_2^2 ]. (16) Given the spectral alignment condition t[‖(I−Pr)h(t)‖2/‖h(t)‖2]≤δ2E_t [\|(I-P_r)h^(t)\|^2/\|h^(t)\|^2 ]≤δ^2, it directly follows that: ⁡[‖(I−Pr)ΔW⊤‖F2]≤δ2[‖ΔW⊤‖F2]=δ2[‖ΔW‖F2].E [\|(I-P_r) W \|_F^2 ]≤δ^2\,E [\| W \|_F^2 ]=δ^2\,E [\| W\|_F^2 ]. (17) By the Pythagorean theorem for the Frobenius norm, ‖PrΔW⊤‖F2+‖(I−Pr)ΔW⊤‖F2=‖ΔW‖F2\|P_r W \|_F^2+\|(I-P_r) W \|_F^2=\| W\|_F^2. Substituting the upper bound yields the final concentration result: ⁡[‖PrΔW⊤‖F2]≥(1−δ2)[‖ΔW‖F2],E [\|P_r\, W \|_F^2 ]≥(1-δ^2)\,E [\| W\|_F^2 ], (18) which concludes the proof. B.2 Proof of Theorem 4.2 (Reachability and Approximation Limits) Statement: For a frozen encoder W=UΣV⊤W=U V and an optimal downstream classifier WdownW_down, the minimum approximation error using any linear head A is ε2(W)=‖PWdown‖F2 ^2(W)=\|P_NW_down\|_F^2, achieved at A∗=UΣ−1V⊤WdownA^*=U ^-1V W_down. Proof. The objective is to minimize the approximation error ℒ⁡(A)=‖Wdown−W⊤A‖F2L(A)=\|W_down-W A\|_F^2 over all possible downstream projection heads A∈ℝdout×CA ^d_out× C. Substituting the compact SVD of the transposed encoder W⊤=VΣU⊤W =V U into the objective yields: ℒ⁡(A)=‖Wdown−VΣU⊤A‖F2.L(A)=\|W_down-V U A\|_F^2. (19) Since the columns of V are orthonormal, VV⊤V forms the orthogonal projection onto col(V)col(V), and P=I−VV⊤P_N=I-V is the projection onto the null space ker⁡(W) (W). We can orthogonally decompose the target matrix WdownW_down as: Wdown=VV⊤Wdown+PWdown.W_down=V W_down+P_NW_down. (20) Because the term VΣU⊤AV U A lies entirely within the column space col(V)col(V), it is orthogonal to PWdownP_NW_down. By applying the Pythagorean theorem for the Frobenius norm, we can decouple the objective function: ℒ⁡(A)=‖VV⊤Wdown−VΣU⊤A‖F2+‖PWdown‖F2.L(A)=\|V W_down-V U A\|_F^2+\|P_NW_down\|_F^2. (21) The second term, ‖PWdown‖F2\|P_NW_down\|_F^2, is entirely independent of A and represents the irreducible error (the fundamental blind spot). To minimize ℒ⁡(A)L(A), we must force the first term to zero: VV⊤Wdown=VΣU⊤A.V W_down=V U A. (22) Left-multiplying both sides by V⊤V (noting that V⊤V=IV V=I) gives: V⊤Wdown=ΣU⊤A.V W_down= U A. (23) Since Σ is a diagonal matrix of strictly positive singular values, it is invertible. We can explicitly solve for A by left-multiplying by UΣ−1U ^-1: A∗=UΣ−1V⊤Wdown.A^*=U ^-1V W_down. (24) Substituting A∗A^* back into ℒ⁡(A)L(A) eliminates the first term, formally proving that the minimum reachable error is exactly: ε2(W)=minA⁡ℒ⁡(A)=‖PWdown‖F2. ^2(W)= _AL(A)=\|P_NW_down\|_F^2. (25) This establishes both the optimal projection head structure and the irreducible nature of downstream targets residing in the null space. B.3 Null-Space PCA Optimality (Proof of Theorem B.1) Theorem B.1 (Optimality of Null-Space PCA). Let ΣX=⁡[xx⊤] _X=E[x ] be the input feature covariance and P=I−VV⊤P_N=I-V the orthogonal projector onto ker⁡(W) (W). Among all orthonormal matrices No∈ℝdin×mN_o ^d_in× m satisfying WNo=0WN_o=0, the null-space PCA basis uniquely solves: No∗=argmaxNotr(No⊤ΣXNo)|No⊤No=Im,WNo=0.N_o^*= *arg\,max_N_o \tr\! (N_o _XN_o )\; |\;N_o N_o=I_m,\;WN_o=0 \. (26) Standard PCA (unconstrained) is generically infeasible for (26) and therefore suboptimal. Proof. The constraint WNo=0WN_o=0 is equivalent to V⊤No=0V N_o=0 (since ker⁡(W)=ker⁡(V⊤) (W)= (V )). Writing No=PCN_o=P_NC for an arbitrary matrix C, the constraint V⊤No=0V N_o=0 is automatically satisfied, and (26) reduces to: maxC⊤PC=Im⁡tr⁡(C⊤PΣXPC). _ subarraycC P_NC=I_m subarraytr(C P_N _XP_NC). By the Ky Fan maximum theorem [47], the maximum equals ∑j=1mλj(PΣXP) _j=1^m _j(P_N _XP_N), attained uniquely by the top-m eigenvectors of PΣXP_N _XP_N—the null-space PCA basis. An unconstrained PCA solution lies generically outside ker⁡(W) (W) (a measure-zero coincidence would be needed for it to satisfy V⊤No=0V N_o=0), so it is infeasible and thus suboptimal. ∎ B.4 PAC-Bayes Generalization Bound (Proof of Theorem B.2) Theorem B.2 (PAC-Bayes Generalization Bound). Let Θ denote SRP’s learnable parameters with dΘ=|w|rs+mrs+rsdout+dout+1d_ =|V_w|r_s+mr_s+r_sd_out+d_out+1. Let R^(Θ) R( ) be the empirical risk on n labeled examples, R⁡(Θ)R( ) the expected risk, and δ∈(0,1)δ∈(0,1). For any prior π and posterior ρ concentrated at Θ , with probability ≥1−δ≥ 1-δ: R⁡(Θ^)≤R^(Θ^)+KL(ρ∥π)+log(2n/δ)2n.R( )\;≤\; R( )+ KL(ρ\|π)+ (2 n/δ)2n. (27) Setting π=⁡(0,κ2IdΘ)π=N(0,κ^2I_d_ ) and ρ=⁡(Θ^,σ^2IdΘ)ρ=N( , σ^2I_d_ ) with κ=κ⁡(W):=σmax/σminκ=κ(W):= _max/ _min, one obtains: R⁡(Θ^)≤R^(Θ^)+|w|rslog⁡κ⁡(W)+log⁡(2n/δ)2n,R( )\;≤\; R( )+ |V_w|r_s κ(W)+ (2n/δ)2n, (28) where dΘ=O⁡(|w|rs)d_ =O(|V_w|r_s) is the dominant parameter count (spectral adaptor A∈ℝ|w|×rsA ^|V_w|× r_s). Proof. Equation (27) is McAllester’s PAC-Bayes bound [48]. For the chosen Gaussian prior and posterior, the KL divergence evaluates to: KL(ρ∥π)=12[dΘlogκ2σ^2−dΘ+dΘσ^2+‖Θ^‖2κ2].KL(ρ\|π)= 12\! [d_ κ^2 σ^2-d_ + d_ σ^2+\| \|^2κ^2 ]. (29) The effective signal scale per direction satisfies μj/σj≤μj/σmin≤1/σmin _j/ _j≤ _j/ _min≤ 1/ _min, so ‖Θ^‖2=O⁡(dΘ/σmin2)\| \|^2=O(d_ / _min^2). Setting κ=σmax/σminκ= _max/ _min and optimizing σ^2 σ^2 yields KL(ρ∥π)=O(dΘlogκ(W))KL(ρ\|π)=O(d_ κ(W)). Substituting into (27) with dΘ≈|w|rsd_ ≈|V_w|r_s gives (28). Complexity interpretation. The bound depends on |w||V_w| (number of under-served directions targeted by SRP) rather than on k (total singular directions) or dindoutd_ind_out (full-weight dimension). Since DPMS-guided prompting restricts capacity to the weak subspace, the effective model complexity is |w|rs≪krs≪dindout|V_w|r_s kr_s d_ind_out, directly linking the DPMS-based design philosophy to a tighter generalization bound. ∎ B.5 Algebraic Equivalence of the Projective Fallback In our SRP architecture, we introduce a projective fallback path when the input dimension is small, specifically when din≤k+md_in≤ k+m. In this section, we rigorously prove that this fallback is not a heuristic deviation, but rather an exact algebraic simplification of the dual-channel architecture. Theorem B.3 (Algebraic Isomorphism of the Fallback). Let the pre-trained weight have rank k, and let V∈ℝdin×kV ^d_in× k be its right singular vectors. Let μ∈ℝkμ ^k denote the soft-threshold mask. When din−k≤md_in-k≤ m, the projective fallback parameterized by a single adaptor P∈ℝdin×rsP ^d_in× r_s acting on hweakh_weak spans the exact same hypothesis space as the dual-channel prompt parameterized by As∈ℝk×rsA_s ^k× r_s and An∈ℝ(din−k)×rsA_n ^(d_in-k)× r_s. Proof. First, we analyze the structure of the dual-channel prompt. The hidden representation h can be orthogonally decomposed into the reachable subspace h∥=hVV⊤h_ =hVV and the exact null space h⟂=h⁡(I−VV⊤)=hPh_ =h(I-V )=hP_N. In the dual-channel mode, the target response ZdualZ_dual is the sum of the masked spectral channel and the null-space channel: Zdual=(hVdiag(μ))As+(hV⟂⊤)An,Z_dual=(hVdiag(μ))A_s+(hV_ )A_n, (30) where V⟂⊤V_ acts as the PCA basis matrix of the null space. Crucially, when din−k≤md_in-k≤ m, the required PCA dimension m exceeds or equals the dimension of the entire null space. Thus, the columns of V⟂⊤V_ simply form a complete orthonormal basis for P_N, meaning V⟂⊤V⟂=PV_ V_ =P_N. Next, we analyze the input to the projective fallback, defined as hweak=h−(hVdiag(1−μ))V⊤h_weak=h-(hVdiag(1-μ))V . We can expand h using the orthogonal decomposition h=hVV⊤+hPh=hVV +hP_N: hweak h_weak =(hVV⊤+hP)−hVdiag(1−μ)V⊤ =(hVV +hP_N)-hVdiag(1-μ)V =hV(I−diag⁡(1−μ))V⊤+hP =hV (I-diag(1-μ) )V +hP_N =hVdiag(μ)V⊤+hP. =hVdiag(μ)V +hP_N. (31) Equation (31) reveals a critical property: hweakh_weak flawlessly preserves both the masked weak spectral signals and the entirety of the null-space signals, seamlessly superimposing them in the original ambient space ℝdinR^d_in. Applying the single linear adaptor P to hweakh_weak yields: Zfallback=hweakP=(hVdiag(μ))(V⊤P)+(hP)P.Z_fallback=h_weakP= (hVdiag(μ) )(V P)+(hP_N)P. (32) To prove the isomorphism, we must show that there exists a matrix P such that Zfallback=ZdualZ_fallback=Z_dual for any arbitrary AsA_s and AnA_n. By comparing Eq. (32) with Eq. (30), and using the identity hPP=hV⟂⊤(V⟂P)hP_NP=hV_ (V_ P), this equivalence requires: V⊤P=AsandV⟂P=An.V P=A_s V_ P=A_n. (33) Since V and V⟂⊤V_ span mutually orthogonal complementary subspaces, the concatenated matrix [V,V⟂⊤]∈ℝdin×din[V,V_ ] ^d_in× d_in is an orthogonal square matrix, and thus strictly invertible. Consequently, we can uniquely solve for P: P=VAs+V⟂⊤An.P=VA_s+V_ A_n. (34) Equation (34) demonstrates that optimizing the single matrix P over hweakh_weak is mathematically identical to independently optimizing AsA_s and BnB_n in their respective sub-manifolds. Therefore, when din≤k+md_in≤ k+m, the projective fallback explicitly circumvents the redundant PCA calculations while strictly maintaining the theoretical capacity and spectral-reversal philosophy of SRP. ∎ Appendix C Additional Experiments and Analysis C.1 Generalization to GraphGPS and Graphormer To evaluate whether SRP extends beyond conventional message-passing GNNs, we replace the default backbone with GraphGPS and Graphormer and evaluate on two node-level datasets (Cora and CiteSeer) and two graph-level datasets (NCI1 and ENZYMES). We follow the same pre-training objective, data split, few-shot setting, training budget, and evaluation protocol as in the main experiments. Classifier-only, GraphLoRA, and SRP use the same frozen encoder and downstream splits. We retain the default SRP configuration with adaptor dimension r=32r=32 and PCA dimension m=16m=16. Results are averaged over five random seeds. Table 6: Performance of SRP with GraphGPS and Graphormer backbones. Results are reported as mean ± standard deviation over five runs. Backbone Dataset Classifier-only GraphLoRA SRP GraphGPS Cora 48.17±6.1248.17± 6.12 64.33±6.8864.33± 6.88 66.62±5.2766.62± 5.27 GraphGPS CiteSeer 39.13±4.5739.13± 4.57 46.80±3.3846.80± 3.38 58.74±5.0158.74± 5.01 GraphGPS NCI1 63.32±1.5163.32± 1.51 62.49±1.9362.49± 1.93 66.81±2.1366.81± 2.13 GraphGPS ENZYMES 35.08±2.8235.08± 2.82 45.00±3.2545.00± 3.25 47.75±2.7147.75± 2.71 Graphormer Cora 51.13±4.0451.13± 4.04 62.88±5.4962.88± 5.49 65.69±5.9765.69± 5.97 Graphormer CiteSeer 39.01±1.7439.01± 1.74 47.13±1.7247.13± 1.72 54.89±1.6054.89± 1.60 Graphormer NCI1 56.75±2.7656.75± 2.76 60.54±1.0360.54± 1.03 59.72±2.7559.72± 2.75 Graphormer ENZYMES 26.75±2.4126.75± 2.41 39.67±2.4939.67± 2.49 38.92±2.1338.92± 2.13 SRP improves over Classifier-only in all eight settings, with an average absolute gain of 12.4812.48 percentage points. It achieves the best result in six settings and remains within 0.820.82 percentage points of GraphLoRA in the other two. These results indicate that spectral rebalancing is not specific to standard message-passing GNNs and can also be applied to hybrid local–global and attention-based graph encoders. C.2 Role and Sensitivity of the Temperature Coefficient Recall that the spectral mask is defined as μj=sigmoid⁡((τσmax−σj)γ), _j=sigmoid\! ((τ _ - _j)γ ), (35) where τ determines the spectral cutoff and γ controls the sharpness of the transition around it. The interval μj∈[0.1,0.9] _j∈[0.1,0.9] corresponds to a transition width of approximately 4.39/γ4.39/γ in singular-value units. Thus, γ controls the smoothness of the partition without changing the cutoff determined by τ. We evaluate γ from an extremely soft mask to an approximately discrete partition. Results are reported as mean accuracy over five seeds. Table 7: Sensitivity to the temperature coefficient γ. Mask setting Cora CiteSeer NCI1 ENZYMES γ=0.001γ=0.001 64.55 47.64 66.63 36.42 γ=0.1γ=0.1 64.88 48.03 66.52 36.92 γ=1γ=1 67.80 48.81 66.96 36.50 γ=10γ=10 68.10 48.99 66.80 36.67 γ=100γ=100 68.06 48.87 66.84 36.25 Hard threshold (STE) 67.95 48.90 66.89 35.67 At γ=0.001γ=0.001, the mask is nearly constant around 0.50.5, with entropy close to one bit, and therefore provides little distinction between strong and weak directions. Increasing γ to 1010 improves accuracy by 3.553.55 points on Cora and 1.351.35 points on CiteSeer. For γ≥1γ≥ 1, performance becomes stable. At γ=10γ=10, the mask entropy decreases to 0.0040.004–0.0640.064 bits, and only 0.3%0.3\%–3.6%3.6\% of directions remain in the transition region. Further sharpening therefore affects only a small number of boundary directions. We use γ=10γ=10 for all datasets without dataset-specific tuning. To separate spectral selection from generic dimensionality reduction, we further construct masks with exactly the same cardinality as the learned γ=10γ=10 mask, while selecting the strongest, weakest, or random singular directions. Table 8: Matched-cardinality comparison of different spectral selections. Dataset Strong-k Random-k Weak-k Hard (STE) Soft γ=10γ=10 Cora 64.25 64.44 67.80 67.95 68.10 CiteSeer 47.10 47.46 48.78 48.90 48.99 NCI1 66.42 66.45 66.87 66.89 66.80 Weak-k consistently outperforms Strong-k and Random-k, particularly on Cora and CiteSeer, showing that the gain is not explained by dimensionality reduction alone. The similar performance of the soft and hard masks is also expected when few singular values lie near the cutoff. The sigmoid mask provides a differentiable relaxation during optimization while becoming nearly discrete after convergence. C.3 Extension to Heterophilic Graphs Although the main benchmarks are predominantly homophilic, the SRP architecture is not restricted to homophilic graphs. The connection can be understood from the spectral filtering induced by message passing. Consider Z=A^W⊤H,W=UΣV⊤,Z= AW H, W=U V , (36) and let the normalized graph Laplacian be L=I−A^sym=ΦΛΦ⊤.L=I- A_sym= . (37) For a graph signal s, its graph frequency can be measured by its normalized Dirichlet energy, freq⁡(s)=s⊤Ls⊤s.freq(s)= s Lss s. (38) Since A^symϕi=(1−λi)ϕi A_sym _i=(1- _i) _i, repeated propagation scales the i-th graph Fourier component by (1−λi)K(1- _i)^K after K propagation steps. Standard message passing therefore favors low-frequency components while attenuating high-frequency variation. During pre-training, the features used to update the weights have already passed through this propagation operator. Consequently, the feature covariance observed by optimization is biased toward graph-frequency components preserved by message passing. This provides a graph-specific connection between the graph spectrum and the singular spectrum analyzed in our method. SRP introduces its prompt after neighborhood aggregation: H(ℓ+1)=ϕ⁡(A^W(ℓ)⊤H(ℓ)+p(ℓ)(H(ℓ))).H^( +1)=φ\! ( AW^( ) H^( )+p^( )(H^( )) ). (39) The prompt p(ℓ)p^( ) is therefore not filtered again by the same A A in the current layer, providing a complementary pathway for node-local and high-frequency information suppressed by the frozen message-passing branch. We further evaluate SRP on three heterophilic datasets using GraphCL pre-training and the same hyperparameter configuration as in the main experiments. Across both homophilic and heterophilic graph data, prior approaches typically rely on carefully designed filters to balance the learned graph representations [49], whereas our method demonstrates robustness to varying levels of heterophily without requiring such specialized designs. Table 9: Performance on heterophilic graph benchmarks. Dataset Classifier-only GraphLoRA SRP SRP vs. GraphLoRA Cornell 32.22±8.3532.22± 8.35 45.56±14.3445.56± 14.34 51.67±5.7251.67± 5.72 +6.11+6.11 Squirrel 23.08±1.9823.08± 1.98 23.06±1.4023.06± 1.40 25.12±1.4725.12± 1.47 +2.06+2.06 Chameleon 31.74±1.8831.74± 1.88 31.43±1.4431.43± 1.44 33.58±2.7633.58± 2.76 +2.15+2.15 SRP outperforms GraphLoRA on all three datasets. The improvement is particularly clear on Squirrel and Chameleon, where GraphLoRA falls below the Classifier-only baseline while SRP remains above it. These results support the applicability of SRP beyond homophilic settings. C.4 Scalability and Runtime The number of trainable SRP parameters is independent of the number of graph nodes and edges, while the dominant computation remains the frozen GNN backbone. Thus, SRP remains applicable to large graphs. In our experiments, including ogbn-arxiv with nearly 170,000170,000 nodes, all runs were conducted on a single NVIDIA RTX 3090 GPU with 24 GB memory. We additionally compare runtime against GPF, which has the lowest empirical per-epoch runtime among the prompt-learning baselines in our main experiments. Table 10: Per-epoch runtime in seconds. Lower is better. Dataset GPF SRP Cora 0.121 0.077 CiteSeer 0.131 0.084 PubMed 0.678 0.508 ENZYMES 0.221 0.292 D 0.191 0.136 NCI1 0.342 0.297 SRP is faster than GPF on five of the six evaluated datasets, indicating that its spectral operations do not introduce substantial training-time overhead. C.5 Difference from GraphLoRA GraphLoRA and SRP both introduce low-dimensional trainable components around a frozen GNN, but they address different adaptation objectives and construct different adaptation subspaces. GraphLoRA focuses on feature and structural distribution shifts between source and target graphs through a structure-aware low-rank adaptation branch. Its low-rank correction remains within the message-passing pathway: ΔZGraphLoRA(ℓ)=A^ΔW(ℓ)⊤H(ℓ). Z_GraphLoRA^( )= A\, W^( ) H^( ). (40) SRP instead targets the spectral capacity allocation of the frozen transformation. For W(ℓ)=U(ℓ)Σ(ℓ)V(ℓ)⊤,W^( )=U^( ) ^( )V^( ) , (41) it explicitly constructs prompts from two complementary subspaces. The spectral channel can be written as Pspec=(HV⊙μ⁡(Σ))AsQ,μj=sigmoid⁡((τσmax−σj)γ),P_spec= (HV μ( ) )A_sQ, _j=sigmoid\! ((τ _ - _j)γ ), (42) while the null-space channel is Pnull=HNAnQ,col⁡(N)⊆ker⁡(W).P_null=HNA_nQ, (N) (W). (43) The resulting layer is H(ℓ+1)=ϕ⁡(A^W(ℓ)⊤H(ℓ)+Pspec+Pnull).H^( +1)=φ\! ( AW^( ) H^( )+P_spec+P_null ). (44) Hence, SRP does not merely impose a low-rank parameter budget: it explicitly selects where adaptation is concentrated according to the singular geometry of the frozen transformation. GraphLoRA learns a flexible low-rank correction from graph-structural objectives, whereas SRP directs adaptation toward weak singular and null-space directions. The position of the adaptation branch also differs. If HΔW⊤=∑iϕici⊤,H W = _i _ic_i , (45) then a GraphLoRA-style correction inside message passing becomes A^ΔW⊤H=∑i(1−λi)ϕici⊤, A W H= _i(1- _i) _ic_i , (46) and is therefore filtered by the propagation operator. In contrast, ZSRP=A^W⊤H+p⁡(H),Z_SRP= AW H+p(H), (47) injects p⁡(H)p(H) after aggregation, so its graph Fourier components are not multiplied again by (1−λi)(1- _i) within the same layer. This gives SRP a complementary pathway for information suppressed by the frozen message-passing branch. C.6 Difference from Standard LoRA Standard LoRA and SRP are both parameter-efficient adaptation methods, but differ in their adaptation target, trainable-subspace geometry, and interaction with graph propagation. LoRA freezes W and learns an additive low-rank update Weff=W+ΔW,ΔW=BA,rank⁡(ΔW)≤r.W_eff=W+ W, W=BA, ( W)≤ r. (48) For a message-passing GNN layer, ZLoRA=A^(W+BA)⊤H=A^W⊤H+A^A⊤B⊤H.Z_LoRA= A(W+BA) H= AW H+ AA B H. (49) Thus, although the pretrained parameters remain frozen, LoRA changes the effective linear operator and parameterizes a low-rank subspace in weight-update space. SRP instead leaves the original message-passing operator unchanged and learns a representation-space prompt: ZSRP=A^W⊤H+p⁡(H),Z_SRP= AW H+p(H), (50) where p⁡(H)=[(HV⊙μ⁡(Σ))As+(HN)An]Q+b,col⁡(N)⊆ker⁡(W).p(H)= [(HV μ( ))A_s+(HN)A_n ]Q+b, (N) (W). (51) The two channels therefore do not factorize a weight update; rather, they determine which information from the hidden representation is injected through the prompt. Another distinction lies in the adaptation subspace. The low-rank constraint in LoRA controls the dimensionality of the update but does not explicitly determine how its energy is distributed across the singular directions of the frozen weight matrix. SRP instead derives its prompt-reading directions from the singular geometry of W and redirects prompt capacity toward weak singular directions and ker⁡(W) (W). This distinction is particularly relevant for GNNs. Let L=I−A^=ΦΛΦ⊤.L=I- A= . (52) If A⊤B⊤H=ΦCA B H= C, the LoRA correction becomes ΔZLoRA=Φ⁡(I−Λ)C, Z_LoRA= (I- )C, (53) and is therefore filtered by the same graph propagation operator. By contrast, the SRP prompt is added after aggregation and is not multiplied again by I−ΛI- in the current layer. In summary, LoRA provides generic low-rank operator adaptation, whereas SRP preserves the frozen operator and explicitly targets weak singular and null-space directions through a post-aggregation prompt. C.7 Clarification of the Singular-Value Groups in Figure 1 Let σ1≥σ2≥⋯≥σk>0 _1≥ _2≥·s≥ _k>0 (54) denote the nonzero singular values of a frozen layer W, sorted in descending order. We define the five singular-value groups by bq=⌊qk5⌋,Qq=bq−1+1,…,bq,q=1,…,5,b_q= qk5 , Q_q=\b_q-1+1,…,b_q\, q=1,…,5, (55) with b0=0b_0=0. Thus, Q1Q_1 contains the directions with the largest singular values and Q5Q_5 contains the weakest nonzero singular directions; group sizes differ by at most one. Exact zero-singular-value directions are treated separately by the null-space channel. The quintile partition depends only on the spectrum of the frozen weight W. The downstream classifier WdownW_down enters only through the downstream-demand term in DPMS and does not affect the group construction. C.8 Why SRP Shows Larger Gains on Node-Level Tasks The difference between node- and graph-level performance can be attributed mainly to input dimensionality, null-space capacity, and graph-level readout. Larger null-space capacity. For a first-layer transformation W∈ℝh×din,W=UΣV⊤,W ^h× d_in, W=U V , (56) if rank⁡(W)=krank(W)=k, then dimker⁡(W)=din−k≥din−h. (W)=d_in-k≥ d_in-h. (57) Hence, when din≫hd_in h, the first frozen layer necessarily contains a large right null space. The node-classification datasets have input dimensions of 14331433 (Cora), 37033703 (CiteSeer), 500500 (PubMed), 500500 (Flickr), and 128128 (ogbn-arxiv), whereas the graph-classification datasets contain only 33–8989 input dimensions. With h=128h=128, the null-space dimensions of the first-layer weights on Cora and CiteSeer are at least 13051305 and 35753575, respectively; on Cora we indeed observe a 13051305-dimensional null space. In contrast, when din≤hd_in≤ h, the first layer may have full column rank and dimker⁡(W)=0. (W)=0. (58) Graph-level datasets therefore provide substantially less capacity for the null-space channel, and SRP relies more heavily on spectral reversal or the projective fallback. Graph-level readout. Node classification predicts directly from each node representation: y^i=C⁡(zi+pi), y_i=C(z_i+p_i), (59) so a node-local prompt directly affects the prediction of node i. Graph classification instead aggregates node representations. With mean pooling, g=1Ng⊤H,Δg=1Ng⊤P.g= 1N_g1 H, g= 1N_g1 P. (60) If P=ΦCP= C, then Δg=1Ng∑j(⊤ϕj)cj⊤. g= 1N_g _j(1 _j)c_j . (61) For the combinatorial Laplacian of a connected graph, ⊤ϕj=0,j>1.1 _j=0, j>1. (62) Mean readout therefore preserves the constant component while cancelling many nonstationary node-local variations. Sum pooling has the same directional effect up to scaling, which can reduce the contribution of local information recovered by SRP. Cross-graph consistency. For node classification on a single graph, define the graph frequency of the j-th feature direction by qj=(Hvj)⊤L(Hvj)‖Hvj‖22.q_j= (Hv_j) L(Hv_j)\|Hv_j\|_2^2. (63) This quantity has a consistent meaning on the same graph. For graph classification, however, qg,j=(Hgvj)⊤Lg(Hgvj)‖Hgvj‖22,q_g,j= (H_gv_j) L_g(H_gv_j)\|H_gv_j\|_2^2, (64) and the same vjv_j can correspond to different graph frequencies across different graphs. Large variation in qg,jq_g,j weakens the correspondence between a shared singular direction and complementary graph-frequency information. These factors help explain why the empirical gains are generally larger on node-level benchmarks. C.9 Why the Gains Are Larger on Some Smaller Datasets The observed trend is better explained by spectral complement capacity than by dataset size alone, because input dimension and graph size are confounded in the current benchmarks. Cora and CiteSeer contain 2,7082,708 and 3,3273,327 nodes but have input dimensions of 1,4331,433 and 3,7033,703, respectively. In contrast, ogbn-arxiv contains 169,343169,343 nodes with only 128128 input dimensions, while Flickr contains 89,25089,250 nodes with 500500 input dimensions. The exact right-null-space dimension is dimker⁡(W)=din−rank⁡(W), (W)=d_in-rank(W), (65) which depends on the input dimension and weight rank rather than directly on the number of nodes. Cora and CiteSeer therefore provide substantially greater capacity for the null-space channel. A second factor is local frequency heterogeneity. SRP currently uses a shared right-singular basis V and a global spectral threshold, μj=sigmoid⁡((τσmax−σj)γ). _j=sigmoid\! ((τ _ - _j)γ ). (66) For a graph partitioned into regions ℛ1,…,ℛMR_1,…,R_M, define xr,j=Hℛrvj,qr,j=xr,j⊤Lℛrxr,jxr,j⊤xr,j.x_r,j=H_R_rv_j, q_r,j= x_r,j L_R_rx_r,jx_r,j x_r,j. (67) When Varr⁡(qr,j)Var_r(q_r,j) is large, the same singular direction can represent different graph-frequency characteristics across regions. A single global threshold can therefore become less precise as local frequency heterogeneity increases. Accordingly, the observed performance trend is more closely associated with input dimensionality, null-space capacity, and local frequency heterogeneity than with graph size itself. Appendix D Computational Complexity D.1 Compute Resources Usage. All experiments are conducted on a workstation running the Ubuntu 22.04 LTS operating system. The hardware configuration includes a 13th Gen Intel(R) Core(TM) i9-13900K CPU and an NVIDIA GeForce RTX 3090 GPU equipped with 24GB of memory. Furthermore, the software environment is built using CUDA 12.1 and Python 3.10.19, utilizing PyTorch 2.5.1 and PyTorch Geometric 2.7.0 for the model implementations. D.2 Complexity Analysis. SRP’s per-layer forward pass: (i) V-projection O⁡(||kdin)O(|V|kd_in) (pre-computable); (i) mask O⁡(||k)O(|V|k); (i) A-projection O⁡(||krs)O(|V|kr_s); (iv) N-projection O⁡(||mdin)O(|V|md_in) (pre-computable); (v) B-projection O⁡(||mrs)O(|V|mr_s); (vi) Q-projection O⁡(||rsdout)O(|V|r_sd_out). Trainable-path cost: O⁡(||(k+m)rs+||rsdout)O(|V|(k+m)r_s+|V|r_sd_out), negligible vs. backbone’s O⁡(|ℰ|din)O(|E|d_in). We analyze the time and space complexity of SRP under a two-layer GCN backbone with L=2L=2 frozen layers. Let N denote the number of nodes, |ℰ||E| the number of edges, d the raw input feature dimension, h the hidden (and output) dimension, k=rank⁡(W)≤hk=rank(W)≤ h the numerical rank of each frozen weight W∈ℝh×dinW ^h× d_in, m the null-space PCA dimension (null_pca_dim, default m=64m=64), and r the shared bottleneck rank (r_shared, default r=8r=8). Initialisation. The initialisation of SRP involves two SVD computations and one null-space projection, all performed once before training. For each frozen weight WiW_i, the reduced SVD Wi=UiΣiVi⊤W_i=U_i _iV_i costs ⁡(k2din,i)O(k^2d_in,i) time (thin SVD of an h×din,ih× d_in,i matrix). For the first layer, when d>k+md>k+m holds and the dual-channel path is activated, SRP additionally projects the node-feature matrix X∈ℝN×dX ^N× d onto the null space via Xnull=X−XV0V0⊤X_null=X-XV_0V_0 , costing ⁡(Ndk)O(Ndk), followed by a truncated SVD of the centred XnullX_null to obtain the top-m null-space PCA basis Nmat∈ℝd×mN_mat ^d× m, which costs ⁡(min⁡(N,d)2max⁡(N,d))O( (N,d)^2 (N,d)) but is performed on CPU once and cached as a frozen buffer. Summing across layers, the total initialisation time is ⁡(k2d+Ndk+min⁡(N,d)2max⁡(N,d))O(k^2d+Ndk+ (N,d)^2 (N,d)), which for typical datasets satisfies Nd≪min⁡(N,d)2max⁡(N,d)Nd (N,d)^2 (N,d), so the dominant term is the one-time null-space PCA. This cost is paid once before any gradient step and does not affect training throughput. Per-forward-pass time complexity. During training and inference, get_prompt is invoked once per GNN layer per forward pass. For the first layer operating in the dual-channel mode (d>k+md>k+m), the three dominant operations are hv=hW0−V0∈ℝN×kh_v=hW_0^-V_0 ^N× k at cost ⁡(Ndk)O(Ndk), hnull=hW0−Nmat∈ℝN×mh_null=hW_0^-N_mat ^N× m at cost ⁡(Ndm)O(Ndm), and the two bottleneck projections zspec=(hv⊙)A∈ℝN×rz_spec=(h_v )A ^N× r and znull=hnullB∈ℝN×rz_null=h_nullB ^N× r at cost ⁡(N⁡(kr+mr))O(N(kr+mr)), followed by the output projection zQ+zQ+b at cost ⁡(Nrh)O(Nrh). Since k,m≪dk,m d and r≪k,mr k,m, the total for layer 0 is ⁡(Nd(k+m))O(Nd(k+m)). For the second layer operating under the projective fallback (din=h≤k+md_in=h≤ k+m), the cost is ⁡(Nh2+Nhr)O(Nh^2+Nhr), which collapses to ⁡(Nh2)O(Nh^2). The backbone GCN message-passing itself costs ⁡(|ℰ|h+Ndh)O(|E|h+Ndh) across the two layers. Combining prompt and backbone, the overall per-forward-pass complexity is ⁡(Nd(k+m)+Nh2+|ℰ|h)=⁡(Ndh+|ℰ|h),O\! (Nd(k+m)+Nh^2+|E|h )\;=\;O\! (Ndh+|E|h ), where the equality uses k≤hk≤ h and m≪hm h. Thus SRP adds no asymptotic overhead beyond the backbone GNN’s own message-passing and projection costs. Parameter count and space complexity. SRP introduces only a small set of trainable parameters. For layer 0 (dual-channel): A0∈ℝk×rA_0 ^k× r, B0∈ℝm×rB_0 ^m× r, Q0∈ℝr×hQ_0 ^r× h, 0∈ℝhb_0 ^h, and the scalar threshold τ0 _0, contributing kr+mr+rh+h+1=r⁡(k+m+h)+h+1kr+mr+rh+h+1=r(k+m+h)+h+1 parameters. For layer 1 (projective fallback): P1∈ℝh×rP_1 ^h× r, Q1∈ℝr×hQ_1 ^r× h, 1∈ℝhb_1 ^h, and τ1 _1, contributing 2rh+h+12rh+h+1 parameters. The total trainable parameter count is therefore r⁡(k+m+3h)+2h+2r(k+m+3h)+2h+2. The additional memory consists of frozen buffers: V0∈ℝd×kV_0 ^d× k and Nmat∈ℝd×mN_mat ^d× m for layer 0, and V1∈ℝh×kV_1 ^h× k for layer 1, totalling d⁡(k+m)+hkd(k+m)+hk floating-point values. The activation memory during a forward pass is ⁡(Nd)O(Nd) for the null-space projections of the full feature matrix, which is at most ⁡(Nd)O(Nd)—identical to the GCN input tensor. Overall, SRP’s space overhead is ⁡(d⁡(k+m)+hk)O(d(k+m)+hk) in parameters and buffers, with the number of trainable parameters scaling as ⁡(r⁡(k+m+h))=⁡(rh)O(r(k+m+h))=O(rh), far smaller than any full fine-tuning or LoRA baseline of comparable expressiveness.

Instant research discovery