Paper deep dive
MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval
Xintao Zong, Wenxuan Liu, Jianhao Ding, Zhaofei Yu, Tiejun Huang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/10/2026, 1:50:16 AM
Summary
The paper proposes MSGAT (Multi-Head Spiking Graph Attention Network), a novel framework for Image-Text Retrieval (ITR) that leverages Spiking Neural Networks (SNNs) for energy-efficient computation. MSGAT utilizes dynamic attention heads to capture structural semantic relationships via spike-driven graph reasoning. To address the challenge of fusing sparse discrete spike representations with continuous global representations, the authors introduce Sim-Fuse, a similarity-space fusion strategy that aligns coarse- and fine-grained matching relations without direct feature-level fusion. Experiments on Flickr30K and MSCOCO demonstrate that MSGAT outperforms existing SNN baselines and matches ANN performance while reducing theoretical energy consumption by 55%.
Entities (10)
Relation Signals (9)
MSGAT → uses → Spiking Neural Networks
confidence 98% · we propose a Multi-head Spiking Graph Attention Network (MSGAT)... Spiking neural networks (SNNs) offer an energy-efficient computing paradigm
MSGAT → employs → Sim-Fuse
confidence 96% · we propose a Multi-head Spiking Graph Attention Network (MSGAT)... Therefore, we introduce Sim-Fuse
MSGAT → evaluatedon → Flickr30K
confidence 95% · Experiments on Flickr30K and MSCOCO show our method outperforms
MSGAT → evaluatedon → MSCOCO
confidence 95% · Experiments on Flickr30K and MSCOCO show our method outperforms
MSGAT → outperforms → existing SNN retrieval baselines
confidence 95% · Experiments on Flickr30K and MSCOCO show our method outperforms... existing SNN retrieval baselines
MSGAT → reducesenergyby → 55%
confidence 94% · reducing theoretical module-level energy by 55%
MSGAT → usesfeaturesfrom → BERT
confidence 92% · a pre-trained BERT (Devlin et al. 2019) to extract word features
MSGAT → usesfeaturesfrom → Faster R-CNN
confidence 92% · we use pre-extracted visual region features by Faster-RCNN
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spiking neural networks (SNNs) offer an energy-efficient computing paradigm through sparse event-driven computation, showing great potential for efficient multimodal learning. However, applying SNNs to high-level multimodal tasks, such as image-text retrieval (ITR), remains challenging, since sparse spike representations make it difficult to capture semantic structures required for cross-modal alignment. Existing spiking ITR methods rely on local alignment and additional soft-label supervision during training, while lacking awareness of structural and multi-granularity relationships. To address these issues, we propose a Multi-head Spiking Graph Attention Network (\textbf{MSGAT}) for structural modeling and equip it with dynamic attention heads to capture complementary relational patterns and enable spike-driven graph reasoning and aggregation. However, within a two-branch multi-granularity fusion framework, the fine-grained spike representations generated by MSGAT are sparse and discrete, whereas the global representations are continuous, making conventional feature-level fusion susceptible to interference across heterogeneous representations. Therefore, we introduce \textbf{Sim-Fuse}, a similarity-space fusion alignment strategy integrating coarse- and fine-grained matching relations while avoiding direct fusion of heterogeneous representations. Experiments on Flickr30K and MSCOCO show our method outperforms ANN methods under matched settings and existing SNN retrieval baselines. Moreover, with only two time steps, our SNN achieves comparable or superior performance to its ANN counterpart while reducing theoretical module-level energy by 55\%. The code is provided in the Supplementary Materials.
Tags
Links
- Source: https://arxiv.org/abs/2610.11526v1
- Canonical: https://arxiv.org/abs/2610.11526v1
Trouble viewing inline? Open PDF directly →
Full Text
48,752 characters extracted from source content.
Expand or collapse full text
MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval Xintao Zong Affiliation: Peking University Wenxuan Liu Affiliation: Peking University Jianhao Ding Affiliation: Peking University Zhaofei Yu Affiliation: Peking University Tiejun Huang Affiliation: Peking University Abstract Spiking neural networks (SNNs) offer an energy-efficient computing paradigm through sparse event-driven computation, showing great potential for efficient multimodal learning. However, applying SNNs to high-level multimodal tasks, such as image-text retrieval (ITR), remains challenging, since sparse spike representations make it difficult to capture semantic structures required for cross-modal alignment. Existing spiking ITR methods rely on local alignment and additional soft-label supervision during training, while lacking awareness of structural and multi-granularity relationships. To address these issues, we propose a Multi-head Spiking Graph Attention Network (MSGAT) for structural modeling and equip it with dynamic attention heads to capture complementary relational patterns and enable spike-driven graph reasoning and aggregation. However, within a two-branch multi-granularity fusion framework, the fine-grained spike representations generated by MSGAT are sparse and discrete, whereas the global representations are continuous, making conventional feature-level fusion susceptible to interference across heterogeneous representations. Therefore, we introduce Sim-Fuse, a similarity-space fusion alignment strategy integrating coarse- and fine-grained matching relations while avoiding direct fusion of heterogeneous representations. Experiments on Flickr30K and MSCOCO show our method outperforms ANN methods under matched settings and existing SNN retrieval baselines. Moreover, with only two time steps, our SNN achieves comparable or superior performance to its ANN counterpart while reducing theoretical module-level energy by 55%. The code is provided in the Supplementary Materials. Introduction Image-text retrieval (ITR) aims to identify semantically corresponding samples across visual and textual modalities (Chen et al. 2021). Recent ANN-based methods have achieved remarkable performance by global semantic alignment (Zhang et al. 2024), fine-grained region-word correspondence (Pan et al. 2023), multi-level matching strategies (Fu et al. 2023), and large-scale pre-trained models (Huang et al. 2024). However, existing methods typically rely on deep architectures, dense floating-point operations, and complex cross-modal interactions (Lee et al. 2018), incurring considerable computational and energy costs that limit scalability in large-scale retrieval scenarios. Figure 1: Motivation. Sparse, discrete SNN representations and continuous representations produced by ANN-based GPO (Generalized Pooling Operator) lie in heterogeneous spaces, making conventional feature-level fusion ineffective. Spiking neural networks (SNNs) provide a promising alternative for efficient multimodal learning through sparse spike activations and event-driven computation. Recently, CMSF (Zong et al. 2026) established the feasibility of SNNs for cross-modal retrieval. Nevertheless, CMSF primarily relies on local spike encoding and additional soft-label supervision during training, leaving semantic relation reasoning and global semantic alignment underexplored, although both are important for multimodal understanding and alignment. High-level cross-modal alignment requires modeling structural relationships beyond isolated visual regions and textual tokens. These semantic units are inherently connected through visual and linguistic dependencies, which provide important cues for distinguishing fine-grained semantic correspondence. Therefore, spiking ITR calls for an effective spike-driven graph reasoning mechanism that can capture structural dependencies and enable semantic propagation. Fine-grained local features and coarse-grained global features provide complementary information for multi-granularity image-text alignment. Existing ANN-based ITR methods commonly integrate these representations through feature-level fusion. However, such a strategy introduces another challenge in SNNs, as illustrated in Fig. 1: fine-grained representations from SNNs exhibit sparse and discrete characteristics, while global representations from GPO (Chen et al. 2021) remain continuous. Conventional feature-level fusion across these heterogeneous representation spaces may introduce interference and destabilize joint optimization. To address these challenges, we propose a Multi-head Spiking Graph Attention Network (MSGAT) with Similarity Space Fusion (Sim-Fuse) alignment strategy for ITR. Specifically, MSGAT constructs spiking visual and textual graphs and performs spike-driven graph reasoning and aggregation, enabling fine-grained structural semantic modeling. By employing multiple dynamic attention heads, MSGAT captures complementary relational patterns from different representation subspaces while preserving the event-driven nature of SNNs. Furthermore, we introduce Sim-Fuse, which integrates coarse- and fine-grained matching relations at the similarity level, avoiding feature-level fusion between the discrete and continuous representations produced by MSGAT and GPO, respectively. This similarity-space fusion enables multi-granularity semantic integration. Together, MSGAT and Sim-Fuse form an efficient spiking retrieval framework that improves multi-granularity semantic alignment while retaining the computational advantages of SNNs. Under comparable feature-extraction settings, experiments on Flickr30K and MSCOCO demonstrate that our method outperforms ANN-based methods and existing SNN retrieval baselines. Moreover, with only two time steps, our proposed MSGAT achieves performance comparable or even superior to that of its ANN counterpart while reducing the theoretical inference energy of the graph-reasoning module by approximately 55%. The main contributions are summarized as follows: • We propose MSGAT, a novel spike-driven multi-head graph attention network that introduces dynamic structure-aware reasoning into image-text retrieval. By exploiting graph relationships and complementary attention heads, MSGAT enables fine-grained semantic graph propagation under sparse event-driven computation. • We introduce Sim-Fuse, a similarity-space fusion alignment strategy for SNNs that integrates coarse- and fine-grained matching relations while bypassing heterogeneous feature representation spaces, effectively improving multi-granularity semantic alignment. • Extensive experiments on Flickr30K and MSCOCO demonstrate that the proposed framework outperforms state-of-the-art methods. With only two time steps, our model achieves competitive performance with ANN counterparts while reducing energy consumption by 55%. Related Works Image-Text Retrieval Recent image-text retrieval methods have explored graph structures to model semantic relations beyond isolated regions or tokens. VSRN (Li et al. 2019) performs graph convolutional reasoning over bottom-up image regions to obtain relation-enhanced visual features. GSMN (Liu et al. 2020a) constructs structured visual and textual graphs whose nodes correspond to objects, relations, or attributes, and learns fine-grained phrase correspondence through node-level and structure-level matching. CSGM (Cheng et al. 2022) formulates retrieval as object-level and relationship-level matching between visual and textual scene graphs. HREM (Fu et al. 2023) further models fragment-level and instance-level semantic relations with a cross-embedding association graph to improve hard negative discrimination. CORA (Pham et al. 2024) represents captions as scene graphs and uses graph attention networks to encode object-attribute and object-object relations in a dual-encoder framework. CHAN (Pan et al. 2023) emphasizes discriminative hard alignments to reduce noisy fine-grained matching. CMSF (Zong et al. 2026) first applies SNNs to ITR, demonstrating their feasibility for semantically demanding complex multimodal tasks. However, it relies on soft-label supervision and suffers from interference by semantically irrelevant nodes in self-attention. Our proposed MSGAT mitigates these limitations via spike-driven graph reasoning, graph aggregation, multi-head dynamic attention, and multi-granularity fusion. Spiking Neural Networks Early work such as (Cao et al. 2015) explored converting deep CNNs into SNNs by interpreting ANN activations as firing rates. (Lv et al. 2023) proposed a two-stage pipeline combining ANN-to-SNN conversion with fine-tuning. Parallel efforts investigated direct training with surrogate gradients (Wu et al. 2018) . Spikformer (Zhou et al. 2023) and its variants (Zhou et al. 2024; Yao et al. 2024) proposed a spike-driven self-attention mechanism, marking a milestone in the development of SNNs. Recently, SNNs have achieved notable progress in unimodal visual tasks, such as image classification (Song et al. 2024), object detection (Li et al. 2025; Luo et al. 2024), semantic segmentation (Lei et al. 2025), and saliency detection (Liu et al. 2025). Some works have extended SNNs to graph-structured learning. SpikingGCN (Zhu et al. 2022) encodes graph structures into spike sequences with graph convolution operation, GSAT (Wang and Jiang 2022) learns spike-based graph attention for robust representation learning, SpikeGCL (Li et al. 2024) introduces graph contrastive learning into SNNs to obtain compact binary representations, and SpikeNet (Li et al. 2023a) explores spiking neurons for dynamic representation learning. SpikingHAN (Cao et al. 2026) further applies SNNs to heterogeneous graph learning with a shared convolution layer. Our MSGAT is tailored to multimodal ITR and differs from prior graph static-attention methods by introducing multi-head dynamic attention with softmax-free spike aggregation to capture structural semantic representations. Proposed Method An overview of our MSGAT is shown in Fig. 2. We first extract floating-point image and text node features and then construct spiking visual and textual graphs. MSGAT captures fine-grained structural semantics through spike-driven graph reasoning, whereas GPO preserves coarse-grained information. The resulting similarity matrices are adaptively fused, and the stop-gradient fused similarity supervises the two pre-fusion matrices via KL-divergence alignment. Figure 2: Framework of MSGAT for ITR. Left: Replaceable extractors produce floating-point node features. Top and bottom: GPO derives global features, Spike Encoder converts node features into spikes for Spiking Graph construction. MSGAT performs spike-driven graph reasoning and aggregation to obtain neighborhood-aware spiking fine-grained features. Middle: Coarse- and fine-grained similarities are fused into a detached multi-granularity target, which in turn aligns and optimizes the two branches. Spiking Neuron As the fundamental unit of SNNs, a spiking neuron receives input currents X[t]X[t] and accumulates membrane potentials, which are compared to a threshold to determine firing. The dynamics of the leaky integrate-and-fire (LIF) neuron model (Maass 1997) are: H[t] H[t] =V[t−1]+1τ(X[t]−(V[t−1]−Vreset)), =V[t-1]+ 1τ (X[t]- (V[t-1]-V_reset ) ), (1) S[t] S[t] =1,if H[t]≥Vth,0,otherwise, = cases1,&if H[t]≥ V_th,\\ 0,&otherwise, cases (2) V[t] V[t] =H[t](1−S[t])+VresetS[t]. =H[t] (1-S[t] )+V_resetS[t]. (3) where τ is the membrane time constant. When H[t]H[t] exceeds the threshold VthV_th, the neuron emits a spike S[t]S[t]; if spiking occurs, V[t]V[t] resets to VresetV_reset, otherwise it remains H[t]H[t]. Spiking Graph Construction Following the standard practice in ITR tasks (Lee et al. 2018; Faghri et al. 2018; Chen et al. 2021), we use pre-extracted visual region features by Faster-RCNN (Ren et al. 2017) as model inputs: v=[1v,2v,…,Nv]⊤∈ℝN×2048,X^v=[x^v_1,x^v_2,…,x^v_N] ^N× 2048, and a pre-trained BERT (Devlin et al. 2019) to extract word features: w=[1w,2w,…,Lw]⊤∈ℝL×768X^w=[x^w_1,x^w_2,…,x^w_L] ^L× 768. iwx^w_i and ivx^v_i denote the embedding of the i-th textual node and visual node respectively. Subsequently, a linear layer maps them to a shared embedding dimension of D. Spiking Textual Graph We construct a textual graph by combining explicit syntactic dependencies and latent semantic correlations. To capture explicit linguistic structures, we use Stanford CoreNLP (Manning et al. 2014) to extract syntactic dependency relations among textual nodes. Based on the parsed dependency relations, we construct a syntactic adjacency matrix: syw∈0,1L×L.A^w_sy∈\0,1\^L× L. Since syntactic dependencies may not cover all latent semantic correlations, we further construct a semantic graph: sew ^w_se =(cosine(iw,jw)>θw)∈0,1L×L, =I(cosine(x^w_i,x^w_j)>θ^w)∈\0,1\^L× L, (4) where (⋅)I(·) denotes the indicator function, which assigns a value of 1 if the cosine similarity cosine(⋅)cosine(·) between two nodes exceeds the threshold θwθ^w, and 0 otherwise. Finally, the syntactic dependency matrix and semantic similarity matrix are combined as: w=((syw+sew)>0)∈0,1L×L. ^w=I((A^w_sy+A^w_se)>0)∈\0,1\^L× L. (5) Spiking Visual Graph The regions detected by Faster R-CNN may contain multiple instances of the same object category, such as several "people" within a single image. Since these instances typically exhibit similar embeddings, it is reasonable to establish edges between the corresponding nodes. Accordingly, we construct the adjacency matrix of the visual graph based on the pairwise cosine similarity: v ^v =(cosine(iv,jv)>θv)∈0,1N×N, =I(cosine(x^v_i,x^v_j)>θ^v)∈\0,1\^N× N, (6) where θvθ^v is the threshold of visual node-level similarity. Spike Encoder To meet the spatiotemporal requirements of SNNs, we employ a Spike Encoder to convert floating-point node features into spiking pattern embeddings (Lv et al. 2024). Given an input floating-point tensor XfX_f, we first apply layer normalization and then replicate T time steps. Xst=(γt⊙LN(Xf)+βt),t=1,…,T. X_s^\,t=SN (γ^t (X_f )+β^t ),t=1,…,T. (7) where γtγ^t and βtβ^t are learnable scale and bias parameters associated with the t-th time step, respectively, and ⊙ denotes element-wise multiplication, (⋅)SN(·) is the spiking neuron. Multi-head Spiking Graph Attention Graph attention networks (Veličković et al. 2018) provide an effective mechanism for modeling relational dependencies by adaptively aggregating information from neighboring nodes. GATv2 (Brody et al. 2022) improves the original attention formulation by allowing the attention ranking to dynamically vary with different source-target node pairs, enabling more expressive graph reasoning. Inspired by the dynamic source-target interaction in GATv2, we propose a Multi-head Spiking Graph Attention Network (MSGAT). Given spiking node features s∈0,1T×N×DX_s∈\0,1\^T× N× D, MSGAT first generates spiking source (Q), target (K), and value (V) matrix though linear projections and LIF neuron respectively: (,,)=LIF(BN(s(q,k,v))), (Q,K,V)=LIF (BN (X_s(W_q,W_k,W_v) ) ), (8) where q,kW_q,W_k and vW_v are projection weights. Then, multi-head attention mechanism is adopted to split the spike tensor into H sub-space. For the h-th head, they are reshaped as: h,h,h ^h,k^h,v^h ∈0,1T×N×d, ∈\0,1\^T× N× d, d d =D/H. =D/H. (9) To compute dynamic source-target attention, MSGAT first constructs a vector-valued edge response for each pair of nodes i and j. The response for head h is defined as: eijh e_ij^h =1T∑t=1T(qh⊙ih[t]+kh⊙jh[t])+bhd, = 1T _t=1^T (a_q^h _i^h[t]+a_k^h _j^h[t] )+b^h1d, (10) where qha_q^h and kha_k^h are learnable head-specific attention vectors, and bhb^h is a learnable broadcast scalar bias, ⊙ denotes Hadamard product. Since ih[t]q_i^h[t]and jh[t]k_j^h[t] are binary, their element-wise products with the attention vectors can be computed by accumulating the corresponding weights. MSGAT applies ReLU before feature reduction, allowing target attention rankings to vary across source nodes, and subsequently incorporates the binary adjacent matrix ∈0,1N×NA∈\0,1\^N× N to preserve only valid graph neighborhoods: e¯ijh e_ij^h =ij⊙∑r=1dReLU(eij,rh)∈ℝ≥0h×N×N, =A_ij _r=1^dReLU (e_ij,r^h ) _≥ 0^h× N× N, (11) where ReLU(⋅·) can be implemented by simple thresholding operations in neuromorphic systems. To avoid the exponential computation introduced by SoftMax, MSGAT adopts row-wise normalization: e~ijh e_ij^h =e¯ijh∑j∈(i)e¯ijh+ϵ∈[0,1)h×N×N, = e_ij^h _j (i) e_ij^h+ε∈[0,1)^h× N× N, (12) where ϵε is a small constant that prevents numerical instability when the denominator becomes zero due to sparse responses. The spike-driven neighborhood aggregation is formulated as: h ^h =∑j=1Ne~ijhjh∈[0,1)T×h×N×d. = _j=1^N e_ij^hv_j^h∈[0,1)^T× h× N× d. (13) Finally, the complementary information from all source-target dynamic attention heads is concatenated as: =1∥2∥…∥H∈[0,1)T×N×D. =o^1\|o^2\|…\|o^H∈[0,1)^T× N× D. (14) where "∥\|" indicates concatenation. The aggregated spiking representation is further converted into spike outputs through a LIF neuron: ^=LIF()∈0,1T×N×D O=LIF (O )∈\0,1\^T× N× D. Similarity-Space Fuse Alignment Strategy As illustrated in Fig. 1, fine-grained representations from the MSGAT exhibit sparse and discrete characteristics, which are misaligned with the continuous GPO representations. This representation discrepancy makes feature-level fusion suboptimal, as verified in Fig. 4. To address this issue, we propose Similarity-Space Fuse Alignment Strategy (Sim-Fuse), which bypasses representation-level fusion and performs multi-granularity alignment in the similarity space. For fine-grained branch, we compute the fine-grained similarity tensor: (v,w)∈ℝB×B×N×Ls(X^v,X^w) ^B× B× N× L, B denotes the batch size. Then, we aggregate similarities from the word-to-region direction by selecting the most relevant region for each word (Pan et al. 2023), and the most relevant word for each region along the reverse direction: w→r=1L∑l=1Lmaxnn,l,r→w=1N∑n=1Nmaxln,l. _w→ r= 1L _l=1^L _ns_n,l,\;\;s_r→ w= 1N _n=1^N _ls_n,l. (15) Finally, we combine the fine-grained similarities, defined as: f=w→r+r→w2∈ℝB×B. _f= s_w→ r+s_r→ w2 ^B× B. (16) For coarse-grained branch, we use Generalized Pooling Operator (GPO) to aggregate region and word nodes into global image and text embeddings: vg^v and w∈ℝB×Dg^w ^B× D , and the coarse-grained similarity is computed as: c=cosine(v,w).S_c=cosine(g^v,g^w). The fused similarity matrix fuseS_fuseis obtained by a learnable weighted combination of fS_f and cS_c: fuse=λff+λcc.S_fuse= _fS_f+ _cS_c. (17) Next, each row and column of the similarity matrices is converted into matching probability distribution R and C using temperature-scaled softmax. The image-to-text alignment loss is defined as: ℒKLi2t=DKL(¯fuse∥f)+DKL(¯fuse∥c),L_KL^i2t=D_KL( R_fuse\|R_f)+D_KL( R_fuse\|R_c), (18) where ¯ denotes the stop-gradient operation. Similarly, the text-to-image alignment loss is defined as: ℒKLt2i=DKL(¯fuse∥f)+DKL(¯fuse∥c).L_KL^t2i=D_KL( C_fuse\|C_f)+D_KL( C_fuse\|C_c). (19) The final similarity alignment loss is: ℒKL=12(ℒKLi2t+ℒKLt2i).L_KL= 12 (L_KL^i2t+L_KL^t2i ). (20) Through this objective, fuseS_fuse integrates complementary coarse- and fine-grained matching cues and provides guidance for optimizing fS_f and cS_c. In turn, the refined branch similarities further enhance the fused similarity, forming a collaborative refinement process. Training Objectives We improve the hubness-aware loss (Liu et al. 2020b) as our training objective. For the similarity matrix S, we define the Sii,i∈[1,B]S_i,i∈[1,B] as positive pairs, SijS_ij and Sji,j∈[1,i)∪(i,B]S_ji,j∈[1,i)∪(i,B] as negative pairs. The loss function is formulated as: ℒ =1B∑i=1B(1μlog(1+∑j=1,j≠iBeμ(¯ij−γ))CLOSE = 1B _i=1^B( 1μ (1+ _j=1,j≠ i^Be^μ( S_ij-γ)) OPEN+1μlog(1+∑j=1,j≠iBeμ(¯ji−γ))−log(1+ii)), + 1μ (1+ _j=1,j≠ i^Be^μ( S_ji-γ))- (1+S_i)), (21) where μ and γ are temperature scale and margin, respectively. Node-level Objective Sparse spike representations are sensitive to noise, and unreliable activations introduced at early stages may be propagated and amplified during subsequent spiking graph reasoning. To stabilize the inputs of MSGAT, we impose an auxiliary node-level objective before Spike Encoder. Given the visual nodes XvX^v and textual nodes XwX^w, the similarity nodeS_node is computed by Eq.16, and according to Eq.21, ℒn=ℒ(node).L_n=L (S_node ). Structure-level Objective After MSGAT reasoning, each node aggregates information from its graph neighborhood and encodes relation-enhanced structural semantics. The structure-level objective is defined as ℒs=ℒ(f)+ℒ(c)L_s=L (S_f )+L (S_c ) to capture local and global structural information. According to Eq.17, the loss function of fused similarity matrix is defined as: ℒf=ℒ(fuse)L_f=L (S_fuse ). It encourages the model to jointly optimize fine-grained branch and coarse-grained branch with learnable weights. And the final total objective is formulated as: ℒtotal=ℒf+λ1ℒn+λ2ℒs+λ3ℒKL _total=L_f+ _1L_n+ _2L_s+ _3L_KL (22) where λ1 _1, λ2 _2 and λ3 _3 are weights to balance different losses. Experiments Datasets Metrics and Settings We evaluate MSGAT on two widely used benchmarks: Flickr30K (Young et al. 2014) and MSCOCO (Lin et al. 2014). Flickr30K contains 31,783 images and MSCOCO includes 123,287 images, each paired with five captions. Following the standard evaluation protocol (Lee et al. 2018; Zhang et al. 2024), Flickr30K is split into 29,000/1,000/1,000 images for training/validation/testing, while MSCOCO uses 113,287/1,000/5,000 images. The commonly used evaluation metrics for ITR are Recall@K (K=1,5,10), which depict the percentage of ground truth being retrieved at top 1, 5, 10 results, respectively. To show the overall matching performance, we also compute the sum of all the Recall values (R@Sum). For a fair comparison, we employ Faster R-CNN and BERT as feature extractors. MSGAT is implemented with the SpikingJelly (Fang et al. 2023) framework and trained on a single NVIDIA RTX4090 GPU. More parameter and model setting details are provided in the Supplementary Materials. Main Experimental Results We compare MSGAT with state-of-the-art ANN-based methods and the existing SNN retrieval baseline on Flickr30K and MSCOCO, as shown in Tab. 1 and Tab. 2. Methods are grouped according to their feature extraction backbones, and unlike SCAN (Lee et al. 2018), VSRN (Li et al. 2019), and VSRN++ (Li et al. 2023b), which boost performance via multi-model ensembling, MSGAT reports single-model results like MMCA (Wei et al. 2020) and NAAF (Zhang et al. 2022). On the MSCOCO 1K (5-fold) and Flickr30K 1K test sets, MSGAT achieves R@Sum scores of 539.0 and 529.8, respectively, outperforming all compared methods. Compared with CMSF, MSGAT improves R@Sum by 5.2 and 5.9 points on the two benchmarks, demonstrating the effectiveness of spike-driven graph reasoning and similarity-space fusion. Moreover, with only two time steps, MSGAT surpasses its ANN counterpart by 2.0 and 2.6 R@Sum points, showing the potential of SNNs for efficient cross-modal retrieval. On the more challenging MSCOCO 5K test set (Tab. 2), MSGAT achieves the best R@Sum of 452.0 and consistently outperforms its ANN counterpart across most metrics with T=2. Although MaxMatch (Alomari et al. 2025) achieves competitive performance, it relies on Hungarian-based matching (Kuhn 1955) with additional computational complexity. In contrast, MSGAT provides efficient structure-aware reasoning and similarity-level alignment under sparse spike computation. Compared with the SNN baseline CMSF (Zong et al. 2026), MSGAT is slightly lower on text-to-image R@5 and R@10, but achieves much better overall performance with a simpler architecture and training pipeline. Method Venue S MSCOCO 1K (5-fold) Test Set Flickr30K 1K Test Set Image-to-Text Text-to-Image R@ Sum Image-to-Text Text-to-Image R@ Sum R@1 R@5 R@10 R@1 R@5 R@10 R@1 R@5 R@10 R@1 R@5 R@10 Faster R-CNN + BiGRU SCAN∗ ECCV18 72.7 94.8 98.4 58.8 88.4 94.8 507.9 67.4 90.3 95.8 48.6 77.7 85.2 465.0 VSRN∗ ICCV19 76.2 94.8 98.2 62.8 89.7 95.1 516.8 71.3 90.6 96.0 54.7 81.8 88.2 482.6 VSE∞ CVPR21 78.5 96.0 98.7 61.7 90.3 95.6 520.8 76.5 94.2 97.7 56.4 83.4 89.9 498.1 NAAF CVPR22 78.1 96.1 98.6 63.5 89.6 95.3 521.2 79.6 96.3 98.3 59.3 83.9 90.2 507.6 CMSF CVPR26 ✓ 78.9 96.2 98.7 63.6 91.0 96.3 524.7 80.7 95.0 97.6 61.3 85.9 91.3 511.8 MSGAT Ours (T=2) ✓ 80.5 96.5 98.4 64.4 91.0 96.2 527.0 82.2 95.6 97.9 61.8 86.5 92.5 516.5 Faster R-CNN + BERT MMCA CVPR20 74.8 95.6 97.7 61.6 89.8 95.2 514.7 74.2 92.8 96.4 54.8 81.4 87.8 487.4 VSE∞ CVPR21 79.7 96.4 98.9 64.8 91.4 96.3 527.5 81.7 95.4 97.6 61.4 85.9 91.5 513.5 VSRN++∗ TPAMI23 77.9 96.0 98.5 64.1 91.0 96.1 523.6 79.2 94.6 97.5 60.6 85.6 91.4 508.9 CHAN CVPR23 81.4 96.9 98.9 66.5 92.1 96.7 532.6 80.6 96.1 97.8 63.9 87.5 92.6 518.5 HREM CVPR23 81.1 96.6 98.9 66.1 91.6 96.5 530.7 83.3 96.0 98.1 63.5 87.1 92.4 520.4 USER TIP24 82.8 96.8 98.8 66.1 90.6 95.6 530.5 82.7 97.0 98.3 63.1 86.7 92.1 519.9 CORA CVPR24 82.4 96.8 98.8 66.2 91.9 96.6 532.7 83.7 96.6 98.3 62.3 87.1 92.6 520.1 MaxMatch ACL25 83.0 96.9 98.9 66.4 91.9 96.6 533.8 84.2 96.1 97.9 63.2 87.3 92.2 520.8 CMSF CVPR26 ✓ 81.9 96.7 98.9 67.1 92.4 96.8 533.8 82.1 96.3 98.0 65.9 88.4 93.2 523.9 MSGAT Ours (ANN) 84.8 97.1 98.8 67.4 92.4 96.5 537.0 85.7 97.3 98.7 65.4 87.4 92.7 527.2 MSGAT Ours (T=2) ✓ 85.8 97.1 99.0 67.8 92.5 96.8 539.0 86.1 98.4 99.0 65.9 87.8 92.6 529.8 Table 1: Results on the MSCOCO and Flickr30K 1K test sets. Best and second-best results are marked in bold and underline, respectively. ∗ denotes ensemble results, and S indicates spike-driven methods. For fair comparison, we report only Faster R-CNN- and BERT/BiGRU-based baselines; results with pretrained VLMs are provided in the Supplementary Material. Methods MSCOCO 5K Test Set Image-to-Text Text-to-Image R@ Sum R@1 R@5 R@10 R@1 R@5 R@10 Faster R-CNN + BiGRU SCAN∗ 50.4 82.2 90.0 38.6 69.3 80.4 410.9 VSRN∗ 53.0 81.1 89.4 40.5 70.6 81.1 415.7 VSE∞ 56.6 83.6 91.4 39.3 69.9 81.1 421.9 NAAF 58.9 85.2 92.0 42.5 70.9 81.4 430.9 CMSF 58.5 85.3 92.5 42.5 72.0 82.2 432.0 MSGATT=2 60.4 85.6 92.6 43.4 72.9 83.4 438.3 Faster R-CNN + BERT MMCA 54.0 82.5 90.7 38.7 69.7 80.8 416.4 VSE∞ 58.3 85.3 92.3 42.4 72.7 83.2 434.2 VSRN++∗ 54.7 82.9 90.9 42.0 72.2 82.7 425.4 CHAN 59.8 87.2 93.3 44.9 74.5 84.2 443.9 HREM 62.3 87.6 93.4 43.9 73.6 83.3 444.1 USER 63.7 87.4 93.5 44.8 73.4 82.7 445.5 CORA 62.4 86.8 92.6 44.2 73.6 83.9 443.6 MaxMatch 63.3 87.9 93.2 44.2 73.9 83.9 446.5 CMSF 61.5 86.7 92.8 45.1 75.0 84.6 445.8 MSGATANN 64.7 87.9 93.5 45.1 73.9 83.6 448.8 MSGATT=2 65.2 88.7 93.9 45.5 74.5 84.3 452.0 Table 2: Results on MSCOCO 5K Test Set. Best results are in bold, and the second-best scores are underlined. T=2T=2 indicates the number of time steps in MSGAT, while ANN denotes the ANN counterpart of MSGAT. Efficiency Analysis of MSGAT Figure 3: Efficiency Analysis. SSA: spike self-attention; AC: accumulate; MAC: multiply-accumulate operation. As shown in Fig. 3, we compare MSGAT with other methods in terms of parameters, MACs, ACs, measured by (Chen et al. 2023), and theoretical inference energy consumption estimated following (Lv et al. 2024). The reported cost excludes the replaceable feature extractors, i.e., Faster R-CNN and BERT. SSA, MSGAT, and GATv2 have the same number of parameters, 45.49M, since they all use four linear projection layers in attention block. For SNN-based methods, the MACs mainly come from projecting floating-point features into the same dimension D, leading SGCN, SSA, and MSGAT to share the same MACs cost of 198.39M. For ACs, SGCN only requires 0.13M because its weight tensor passes through the neuron once simply, while MSGAT also reduces 10.5% ACs compared with SSA. Overall, MSGAT consumes only 0.985 mJ, reducing theoretical inference energy by 54.9% compared with GATv2 and demonstrating strong energy efficiency. Ablation Studies Ablation on MSGAT Module Tab. 3 evaluates different fine-grained branch designs. Removing this branch gives an R@Sum of 512.7, confirming its importance. SSA and SGCN improve performance to 520.4 and 524.7, respectively; however, SGCN cannot model dynamic attention among nodes, while SSA may be disturbed by irrelevant node interactions. MSGAT achieves the best R@Sum of 529.8, outperforming w/o fine-grained matching, SSA, and SGCN by 17.1, 9.4, and 5.1 points, respectively. It also performs best on most recall metrics and surpasses the GAT-ANN counterpart by 2.6 R@Sum. These results show that MSGAT provides stronger structure-aware fine-grained matching under sparse spike representations. We further attribute this advantage partly to Sim-Fuse, which is better suited to heterogeneous features with different distributions than continuous ANN features. Ablation on Sim-Fuse Strategy Fig. 4 compares coarse- and fine-grained fusion strategies on the Flickr30K 1K test set. GPO-only and MSGAT-only achieve R@Sum scores of 512.7 and 517.2, respectively, demonstrating the benefit of structure-aware fine-grained matching. However, feature-level fusion provides limited gains: Add-Fuse decreases R@Sum to 516.3 due to interference between discrete MSGAT and continuous GPO representations, while Concat-Fuse reaches only 517.4, with its marginal improvement potentially arising from the doubled feature dimensionality rather than effective cross-granularity interaction. In contrast, Sim-Fuse bypasses this feature-space mismatch and integrates both branches directly in the similarity space, substantially improving R@Sum to 529.8. Ablation on Training Objectives Tab. 4 investigates the effect of different training objectives. Using only the fused loss ℒfL_f achieves an R@Sum of 522.2, while adding the node-level objective ℒnL_n further improves it to 525.0 by providing stable spike representations before MSGAT reasoning. Since fuseS_fuse is derived from fine- and coarse-grained similarities fS_f and cS_c , introducing structure-level supervision ℒsLs enhances individual similarity and increases R@Sum to 526.4. Finally, ℒKLL_KL aligns fS_f and cS_c with the detached fused similarity distribution, achieving the best performance of 529.8. These results verify the effectiveness of each objective. Ablation on Time Steps Figure 4: Effectiveness of Similarity-Space Fuse strategy on Flickr30K 1K test set. Add-Fuse and Concat-Fuse denote features addition and concatenation, respectively. Figure 5: Training Performance of MSGAT with 1 to 5 Time Steps. The upper set of curves corresponds to Sim-Fuse, whereas the lower set corresponds to without Sim-Fuse. The time steps ablation in Fig. 5 shows that MSGAT achieves the best performance at T=2, reaching a R@Sum of 529.8. This result outperforms the ANN-based GATv2 and other spiking variants, indicating that a small number of time steps is sufficient for effective spiking graph representation learning. Although increasing the time step beyond T=2 does not further improve the best result, it shows that MSGAT is robust to different temporal configurations. Without Sim-Fuse, the performance at T=1 decreases substantially to 509.7, suggesting that Sim-Fuse plays an important role in stabilizing training. Overall, T=2 provides the best balance between accuracy and efficiency. Ablation on Attention Heads Tab. 5 analyzes the effect of the number of attention heads in MSGAT. Multi-head decomposition enables complementary structural modeling across feature subspaces, with 8 heads achieving the best R@Sum of 529.8. More heads reduce subspace dimensionality and effective spike activations, weakening per-head representation capacity, whereas fewer heads limit subspace diversity and complementary information. In particular, the 2-head setting degenerates into all-zero spike attention outputs after thresholding, preventing effective neighborhood aggregation and graph information propagation. These results indicate that 8 heads provide the most stable trade-off between subspace diversity and per-head spike representation capacity. Additional sensitivity analyses of the thresholds θvθ^v and θwθ^w, ablations on Spike Encoder and the energy estimation formula, etc., are presented in the Supplementary Material. Fine-grained Branch Image-to-Text Text-to-Image R@ Sum R@1 R@5 R@10 R@1 R@5 R@10 w/o 82.5 95.0 98.0 61.5 85.0 90.7 512.7 SSAT=2 81.9 96.7 98.5 64.1 87.2 92.1 520.4 SGCNT=2 84.1 97.4 99.0 64.8 87.0 92.3 524.7 MSGATT=2 86.1 98.4 99.0 65.9 87.8 92.6 529.8 GATv2-ANN 85.7 97.3 98.7 65.4 87.4 92.7 527.2 Table 3: Ablation of fine-grained branch on Flickr30K 1K test set. Best results are in bold. Loss function Image-to-Text Text-to-Image R@ Sum ℒfL_f ℒnL_n ℒsL_s ℒKLL_KL R@1 R@5 R@10 R@1 R@5 R@10 ✓ 84.1 97.1 98.5 64.2 86.3 92.0 522.2 ✓ ✓ 84.2 97.5 99.0 64.9 86.9 92.4 525.0 ✓ ✓ ✓ 85.6 97.8 99.0 64.8 87.0 92.2 526.4 ✓ ✓ ✓ ✓ 86.1 98.4 99.0 65.9 87.8 92.6 529.8 Table 4: Influence of loss function on Flickr30K 1K test set. Best results are in bold. Heads Image-to-Text Text-to-Image R@ Sum R@1 R@5 R@10 R@1 R@5 R@10 2 - - - - - - - 4 83.6 97.0 98.2 64.6 86.4 92.5 522.3 8 86.1 98.4 99.0 65.9 87.8 92.6 529.8 16 84.6 97.4 98.9 64.2 87.1 92.3 524.5 32 84.0 97.3 98.7 64.2 86.9 92.4 523.0 Table 5: Effect of the number of MSGAT dynamic attention heads on Flickr30K 1K test set. Best results are in bold. Conclusion In this work, we present MSGAT, a spiking framework for image-text retrieval that combines structure-aware graph reasoning with similarity-space alignment. The proposed Multi-head Spiking Graph Attention Network performs spike-driven adaptive neighborhood aggregation over visual and textual graphs, enabling fine-grained relational modeling under sparse event-driven computation. To address the distribution mismatch between discrete spike representations and continuous global representations produced by MSGAT and GPO, respectively, Sim-Fuse integrates coarse- and fine-grained matching relations in similarity space. Experiments on Flickr30K and MSCOCO demonstrate that MSGAT outperforms existing SNN retrieval baseline and surpasses state-of-the-art ANN-based methods. With only two time steps, MSGAT exceeds its ANN counterpart GATv2, while reducing theoretical energy consumption by approximately 55%. Overall, this work advances structure-aware multimodal reasoning in SNNs and provides a promising, accurate, and energy-efficient solution for cross-modal retrieval. References Alomari et al. (2025) H. Alomari, A. Sivakumar, A. Zhang, and C. Thomas Maximal matching matters: preventing representation collapse for robust cross-modal retrieval. In Proc. Annu. Meet. Assoc. Comput. Linguist., p. 31769–31785. Cited by: Main Experimental Results. Brody et al. (2022) S. Brody, U. Alon, and E. Yahav How attentive are graph attention networks?. In Proc. Int. Conf. Learn. Represent., Cited by: Multi-head Spiking Graph Attention. Cao et al. (2026) B. Cao, Q. Peng, X. Xie, L. Chen, M. Shi, and J. Liu Spiking heterogeneous graph attention networks. In Proc. AAAI Conf. Artif. Intell., S. Koenig, C. Jenkins, and M. E. Taylor (Eds.), p. 19853–19861. External Links: Link, Document Cited by: Spiking Neural Networks. Cao et al. (2015) Y. Cao, Y. Chen, and D. Khosla Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. 113 (1), p. 54–66. Cited by: Spiking Neural Networks. Chen et al. (2023) G. Chen, P. Peng, G. Li, and Y. Tian Training full spike neural networks via auxiliary accumulation pathway. arXiv preprint arXiv:2301.11929. Cited by: Efficiency Analysis of MSGAT. Chen et al. (2021) J. Chen, H. Hu, H. Wu, Y. Jiang, and C. Wang Learning the best pooling strategy for visual semantic embedding. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 15789–15798. Cited by: Introduction, Introduction, Spiking Graph Construction. Cheng et al. (2022) Y. Cheng, X. Zhu, J. Qian, F. Wen, and P. Liu Cross-modal graph matching network for image-text retrieval. ACM Trans. Multimedia Comput. Commun. Appl. 18 (4). External Links: ISSN 1551-6857, Link, Document Cited by: Image-Text Retrieval. Devlin et al. (2019) J. Devlin, M. Chang, K. Lee, and K. Toutanova BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. Conf. North Am. Chapter Assoc. Comput. Linguist., p. 4171–4186. Cited by: Spiking Graph Construction. Faghri et al. (2018) F. Faghri, D. J. Fleet, J. R. Kiros, and S. Fidler VSE++: improving visual-semantic embeddings with hard negatives. In Proc. Brit. Mach. Vis. Conf., p. 12. Cited by: Spiking Graph Construction. Fang et al. (2023) W. Fang, Y. Chen, J. Ding, Z. Yu, T. Masquelier, D. Chen, L. Huang, H. Zhou, G. Li, and Y. Tian SpikingJelly: an open-source machine learning infrastructure platform for spike-based intelligence. Science Advances 9 (40), p. eadi1480. Cited by: Datasets Metrics and Settings. Fu et al. (2023) Z. Fu, Z. Mao, Y. Song, and Y. Zhang Learning semantic relationship among instances for image-text matching. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 15159–15168. Cited by: Introduction, Image-Text Retrieval. Huang et al. (2024) H. Huang, Z. Nie, Z. Wang, and Z. Shang Cross-modal and uni-modal soft-label alignment for image-text retrieval. In Proc. AAAI Conf. Artif. Intell., p. 18298–18306. Cited by: Introduction. Kuhn (1955) H. W. Kuhn The hungarian method for the assignment problem. Nav. Res. Logist. Q. 2 (1-2), p. 83–97. Cited by: Main Experimental Results. Lee et al. (2018) K. Lee, X. Chen, G. Hua, H. Hu, and X. He Stacked cross attention for image-text matching. In Proc. Eur. Conf. Comput. Vis., p. 212–228. Cited by: Introduction, Spiking Graph Construction, Datasets Metrics and Settings, Main Experimental Results. Lei et al. (2025) Z. Lei, M. Yao, J. Hu, X. Luo, Y. Lu, B. Xu, and G. Li Spike2former: efficient spiking transformer for high-performance image segmentation. In Proc. AAAI Conf. Artif. Intell., Vol. 39, p. 1364–1372. Cited by: Spiking Neural Networks. Li et al. (2025) C. Li, W. Liu, G. Gong, X. Ding, and X. Zhong SU-yolo: spiking neural network for efficient underwater object detection. Neurocomputing 644, p. 130310. Cited by: Spiking Neural Networks. Li et al. (2023a) J. Li, Z. Yu, Z. Zhu, L. Chen, Q. Yu, Z. Zheng, S. Tian, R. Wu, and C. Meng Scaling up dynamic graph representation learning via spiking neural networks. In Proc. AAAI Conf. Artif. Intell., p. 8588–8596. Cited by: Spiking Neural Networks. Li et al. (2024) J. Li, H. Zhang, R. Wu, Z. Zhu, B. Wang, C. Meng, Z. Zheng, and L. Chen A graph is worth 1-bit spikes: when graph contrastive learning meets spiking neural networks. In Proc. Int. Conf. Learn. Represent., Cited by: Spiking Neural Networks. Li et al. (2019) K. Li, Y. Zhang, K. Li, Y. Li, and Y. Fu Visual semantic reasoning for image-text matching. In Proc. IEEE/CVF Int. Conf. Comput. Vis., p. 4653–4661. Cited by: Image-Text Retrieval, Main Experimental Results. Li et al. (2023b) K. Li, Y. Zhang, K. Li, Y. Li, and Y. Fu Image-text embedding learning via visual and textual semantic reasoning. IEEE Trans. Pattern Anal. Mach. Intell. 45 (1), p. 641–656. Cited by: Main Experimental Results. Lin et al. (2014) T. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick Microsoft coco: common objects in context. In Proc. Eur. Conf. Comput. Vis., p. 740–755. Cited by: Datasets Metrics and Settings. Liu et al. (2020a) C. Liu, Z. Mao, T. Zhang, H. Xie, B. Wang, and Y. Zhang Graph structured network for image-text matching. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 10918–10927. Cited by: Image-Text Retrieval. Liu et al. (2020b) F. Liu, R. Ye, X. Wang, and S. Li HAL: improved text-image matching by mitigating visual semantic hubs. In Proc. AAAI Conf. Artif. Intell., p. 11563–11571. Cited by: Training Objectives. Liu et al. (2025) W. Liu, Y. Deng, K. Chen, X. Zhong, Z. Yu, and T. Huang SOTA: spike-navigated optimal transport saliency region detection in composite-bias videos. In Proc. Int. Joint Conf. Artif. Intell., Cited by: Spiking Neural Networks. Luo et al. (2024) X. Luo, M. Yao, Y. Chou, B. Xu, and G. Li Integer-valued training and spike-driven inference spiking neural network for high-performance and energy-efficient object detection. In Proc. Eur. Conf. Comput. Vis., p. 253–272. Cited by: Spiking Neural Networks. Lv et al. (2024) C. Lv, Y. Wang, D. Han, X. Zheng, X. Huang, and D. Li Efficient and effective time-series forecasting with spiking neural networks. In Proc. Int. Conf. Mach. Learn., Cited by: Spike Encoder, Efficiency Analysis of MSGAT. Lv et al. (2023) C. Lv, J. Xu, and X. Zheng Spiking convolutional neural networks for text classification. In Proc. Int. Conf. Learn. Represent., Cited by: Spiking Neural Networks. Maass (1997) W. Maass Networks of spiking neurons: the third generation of neural network models. Neural Networks 10 (9), p. 1659–1671. Cited by: Spiking Neuron. Manning et al. (2014) C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel, S. Bethard, and D. McClosky The stanford corenlp natural language processing toolkit. In Proc. Annu. Meet. Assoc. Comput. Linguist., p. 55–60. Cited by: Spiking Textual Graph. Pan et al. (2023) Z. Pan, F. Wu, and B. Zhang Fine-grained image-text matching by cross-modal hard aligning network. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 19275–19284. Cited by: Introduction, Image-Text Retrieval, Similarity-Space Fuse Alignment Strategy. Pham et al. (2024) K. Pham, C. Huynh, S. Lim, and A. Shrivastava Composing object relations and attributes for image-text matching. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 14354–14363. External Links: Link, Document Cited by: Image-Text Retrieval. Ren et al. (2017) S. Ren, K. He, R. B. Girshick, and J. Sun Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39 (6), p. 1137–1149. Cited by: Spiking Graph Construction. Song et al. (2024) X. Song, A. Song, R. Xiao, and Y. Sun One-step spiking transformer with a linear complexity. In Proc. Int. Joint Conf. Artif. Intell., p. 3142–3150. Cited by: Spiking Neural Networks. Veličković et al. (2018) P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio Graph attention networks. In Proc. Int. Conf. Learn. Represent., Cited by: Multi-head Spiking Graph Attention. Wang and Jiang (2022) B. Wang and B. Jiang Spiking gats: learning graph attentions via spiking neural network. CoRR abs/2209.13539. External Links: Link, Document, 2209.13539 Cited by: Spiking Neural Networks. Wei et al. (2020) X. Wei, T. Zhang, Y. Li, Y. Zhang, and F. Wu Multi-modality cross attention network for image and sentence matching. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 10938–10947. Cited by: Main Experimental Results. Wu et al. (2018) Y. Wu, L. Deng, G. Li, J. Zhu, and L. Shi Spatio-temporal backpropagation for training high-performance spiking neural networks. Front. Neurosci. 12, p. 331. Cited by: Spiking Neural Networks. Yao et al. (2024) M. Yao, J. Hu, T. Hu, Y. Xu, Z. Zhou, Y. Tian, B. Xu, and G. Li Spike-driven transformer v2: meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips. In Proc. Int. Conf. Learn. Represent., Cited by: Spiking Neural Networks. Young et al. (2014) P. Young, A. Lai, M. Hodosh, and J. Hockenmaier From image descriptions to visual denotations: new similarity metrics for semantic inference over event descriptions. Trans. Assoc. Comput. Linguistics 2, p. 67–78. Cited by: Datasets Metrics and Settings. Zhang et al. (2022) K. Zhang, Z. Mao, Q. Wang, and Y. Zhang Negative-aware attention framework for image-text matching. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog., p. 15640–15649. Cited by: Main Experimental Results. Zhang et al. (2024) Y. Zhang, Z. Ji, D. Wang, Y. Pang, and X. Li USER: unified semantic enhancement with momentum contrast for image-text retrieval. IEEE Trans. Image Process. 33, p. 595–609. Cited by: Introduction, Datasets Metrics and Settings. Zhou et al. (2024) C. Zhou, H. Zhang, Z. Zhou, L. Yu, L. Huang, X. Fan, L. Yuan, Z. Ma, H. Zhou, and Y. Tian QKFormer: hierarchical spiking transformer using q-k attention. In Adv. Neural Inform. Process. Syst., Cited by: Spiking Neural Networks. Zhou et al. (2023) Z. Zhou, Y. Zhu, C. He, Y. Wang, S. Yan, Y. Tian, and L. Yuan Spikformer: when spiking neural network meets transformer. In Proc. Int. Conf. Learn. Represent., Cited by: Spiking Neural Networks. Zhu et al. (2022) Z. Zhu, J. Peng, J. Li, L. Chen, Q. Yu, and S. Luo Spiking graph convolutional networks. In Proc. Int. Joint Conf. Artif. Intell., p. 2434–2440. External Links: Link, Document Cited by: Spiking Neural Networks. Zong et al. (2026) X. Zong, X. Zhong, W. Liu, J. Ding, Z. Yu, and T. Huang Brain-inspired multimodal spiking neural network for image-text retrieval. ArXiv abs/2603.26787. External Links: Link Cited by: Introduction, Image-Text Retrieval, Main Experimental Results.