Paper deep dive
MambaCSP: Hybrid-Attention State Space Models for Hardware-Efficient Channel State Prediction
Aladin Djuhera, Haris Gacanin, Holger Boche
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 99%
Last extracted: 4/27/2026, 1:32:40 AM
Summary
MambaCSP is a hybrid-attention state space model (SSM) designed for efficient Channel State Prediction (CSP) in wireless networks. It addresses the quadratic scaling and high latency issues of Transformer-based Large Language Models (LLMs) by using a Mamba-based backbone with lightweight, periodic patch-mixer attention layers. This architecture enables the capture of long-range dependencies in complex CSI sequences while maintaining near-linear computational complexity. Simulations on MISO-OFDM systems show that MambaCSP outperforms LLM-based approaches in prediction accuracy (9-12% improvement) and significantly enhances hardware efficiency, offering higher throughput and lower VRAM/inference latency.
Entities (7)
Relation Signals (4)
MambaCSP → isahybridof → Mamba
confidence 100% · We propose MambaCSP, a hybrid-attention SSM architecture that replaces LLM-based prediction backbones with a linear-time Mamba model.
MambaCSP → performs → Channel State Prediction
confidence 100% · MambaCSP: Hybrid-Attention State Space Models for Hardware-Efficient Channel State Prediction
MambaCSP → uses → Patch-mixer attention
confidence 100% · we introduce lightweight patch-mixer attention layers that periodically inject cross-token attentions
MambaCSP → solveslimitationsof → LLM
confidence 90% · MambaCSP improves prediction accuracy over LLM-based approaches by 9-12%, while delivering up to 3.0x higher throughput, 2.6x lower VRAM usage, and 2.9x faster inference.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent works have demonstrated that attention-based transformer and large language model (LLM) architectures can achieve strong channel state prediction (CSP) performance by capturing long-range temporal dependencies across channel state information (CSI) sequences. However, these models suffer from quadratic scaling in sequence length, leading to substantial computational cost, memory consumption, and inference latency, which limits their applicability in real-time and resource-constrained wireless deployments. In this paper, we investigate whether selective state space models (SSMs) can serve as a hardware-efficient alternative for CSI prediction. We propose MambaCSP, a hybrid-attention SSM architecture that replaces LLM-based prediction backbones with a linear-time Mamba model. To overcome the local-only dependencies of pure SSMs, we introduce lightweight patch-mixer attention layers that periodically inject cross-token attentions, helping with long-context CSI prediction. Extensive MISO-OFDM simulations show that MambaCSP improves prediction accuracy over LLM-based approaches by 9-12%, while delivering up to 3.0x higher throughput, 2.6x lower VRAM usage, and 2.9x faster inference. Our results demonstrate that hybrid state space architectures provide a promising direction for scalable and hardware-efficient AI-native CSI prediction in future wireless networks.
Tags
Links
- Source: https://arxiv.org/abs/2604.21957v1
- Canonical: https://arxiv.org/abs/2604.21957v1
Trouble viewing inline? Open PDF directly →
Full Text
33,413 characters extracted from source content.
Expand or collapse full text
MambaCSP: Hybrid-Attention State Space Models for Hardware-Efficient Channel State Prediction †thanks: This work acknowledges support from the German BMFTR through the 6G-life (16KISK002), 6GEM+ (16KIS2409K), GEM-X (16KISS004K), VICTOR6G (16KIS2547), QD-CamNetz (16KISQ077), QuaPhySI (16KIS1598K), QUIET (16KISQ093), and AISAC (16KIS2462) projects, as well as from the German DFG through the CeTI project (Germany’s Cluster of Excellence – EXC 2050/2, ID 390696704). Aladin Djuhera1, Haris Gacanin2, Holger Boche1 Abstract Recent works have demonstrated that attention-based transformer and large language model (LLM) architectures can achieve strong channel state prediction (CSP) performance by capturing long-range temporal dependencies across channel state information (CSI) sequences. However, these models suffer from quadratic scaling in sequence length, leading to substantial computational cost, memory consumption, and inference latency, which limits their applicability in real-time and resource-constrained wireless deployments. In this paper, we investigate whether selective state space models (SSMs) can serve as a hardware-efficient alternative for CSI prediction. We propose MambaCSP, a hybrid-attention SSM architecture that replaces LLM-based prediction backbones with a linear-time Mamba model. To overcome the local-only dependencies of pure SSMs, we introduce lightweight patch-mixer attention layers that periodically inject cross-token attentions, helping with long-context CSI prediction. Extensive MISO-OFDM simulations show that MambaCSP improves prediction accuracy over LLM-based approaches by 9–12%, while delivering up to 3.0x higher throughput, 2.6x lower VRAM usage, and 2.9x faster inference. Our results demonstrate that hybrid state space architectures provide a promising direction for scalable and hardware-efficient AI-native CSI prediction in future wireless networks. I Introduction and Motivation Large language models (LLMs) have gained strong capabilities in coding, math, and complex reasoning, demonstrating their transformative potential across diverse domains. In future 6G networks, they can serve as foundational building blocks for embedding agentic capabilities across communication, connectivity, and artificial intelligence (AI) layers, thereby replacing or augmenting existing workflows [1, 2]. However, not all communication layers can readily benefit from such AI-native integration. Wireless signal processing systems operate under strict real-time constraints, which have traditionally limited the adoption of AI due to high computational and latency requirements [3]. LLMs, in particular, contain billions of parameters and require substantial compute, making them unsuitable for real-time deployment on resource-constrained devices and in latency-sensitive applications where time to first token (TTFT) is critical. Overcoming these constraints is essential for enabling practical AI-native integration. Among such latency-sensitive applications is channel state prediction (CSP), which forecasts future channel state information (CSI) from historical observations, thereby reducing overall pilot overhead. However, acquiring CSI via conventional channel estimation methods is challenging, particularly in high-mobility scenarios where shortened channel coherence times significantly increase the estimation overhead. Recent works [4, 5, 6] have explored LLM-based architectures for CSP, leveraging attention mechanisms to effectively capture long-range, non-local dependencies across time and frequency. Despite their strong performance, transformer-based models exhibit quadratic scaling in sequence length, resulting in substantial computational and memory overhead, as well as increased inference latency due to key-value (KV) caching. This makes them impractical for long-context CSI prediction and real-time deployment in wireless systems. Selective state space models (SSMs), such as Mamba [7], offer a hardware-efficient alternative to LLMs by modeling sequences without KV caching, resulting in linear complexity. In particular, state transitions are input-dependent and include a continuous-time inductive bias, making SSMs better suited for modeling wireless channel variations. However, pure SSMs process tokens recursively, which can limit their ability to capture non-local dependencies in complex CSI sequences. In this paper, we address these limitations and propose MambaCSP, a hybrid-attention state space architecture for hardware-efficient CSP. Our approach employs a Mamba-based backbone with lightweight patch-mixer attention layers that periodically inject cross-token attention, enabling modeling of long-range dependencies across CSI sequences while preserving the efficiency of SSMs. To this end, we adapt and generalize LLM-based CSP pipelines into a unified framework that enables efficient processing of CSI sequences across both model architectures. Our key findings are: • Prediction Performance: MambaCSP consistently outperforms LLM-based and other neural baselines across diverse mobility scenarios in both TDD and FDD settings. • Hardware Efficiency: MambaCSP achieves up to 3.0× higher throughput, 2.6× lower memory usage, and 2.9× faster inference compared to LLM-based prediction. • Hybrid-Attention Design: MambaCSP’s patch-mixer attention is particularly beneficial in FDD scenarios, where non-local, cross-frequency dependencies require explicit token interactions beyond recursive state-space modeling. This paper is organized as follows. Section I presents the system model and formalizes the problem statement. Section I introduces the CSI prediction pipeline and MambaCSP. Section IV describes the experiment setup. Section V discusses the simulation results, and Section VI concludes the paper. I System Model and CSI Prediction Task In this section, we formally define the wireless system model and the corresponding channel state prediction task. I-A Wireless System Model We consider a single-cell MISO-OFDM link, where a base station (BS) equipped with a uniform planar array (UPA) with NtN_t antennas serves U single-antenna user equipments (UEs). We follow a 5G NR FR1 OFDM numerology with carrier frequency fc=2.4f_c=2.4 GHz, subcarrier spacing Δf=15 f=15 kHz, and Nsc=12N_sc=12 subcarriers per resource block (RB), such that each RB has a bandwidth of BRB=NscΔf=180B_RB=N_sc f=180 kHz. We consider two contiguous frequency bands of equal bandwidth for uplink (UL) and downlink (DL), each consisting of KUL=KDL=48K_UL=K_DL=48 RBs, resulting in a per-band bandwidth of 8.648.64 MHz. Time is organized in slots of duration Tslot=1T_slot=1 ms and users move with velocities in the range of v∈[10,100]v∈[10,100] km/h. For each UE, we consider a sequence of Nslot=P+LN_slot=P+L consecutive slots, where the first P slots form a history window and the remaining L slots form a prediction window. For the UL band, let k∈1,…,KULk∈\1,…,K_UL\ denote the RB index and s∈1,…,Nslots∈\1,…,N_slot\ the slot index. The corresponding narrowband MISO channel vector on RB k and slot s is denoted as k,sUL∈ℂNt.h^UL_k,s ^N_t\ . (1) Similarly, for the DL band, we define k,sDL∈ℂNth^DL_k,s ^N_t. We model the wireless channel as frequency-selective and time-varying, incorporating multipath propagation and Doppler effects induced by user mobility. The corresponding parametric representation of the UL channel is thus given by k,sUL=∑ℓ=1LpαℓBS(θℓ,ϕℓ)e−j2πfkτℓej2πfD,ℓts,h^UL_k,s= _ =1^L_p _ a_BS( _ , _ )e^-j2π f_k _ e^j2π f_D, t_s\ , (2) where αℓ _ , τℓ _ , and (θℓ,ϕℓ)( _ , _ ) denote the complex gain, delay, and angles of departure of path ℓ , BS(⋅)a_BS(·) is the BS array response, and fD,ℓf_D, is the Doppler shift. We consider both time-division duplex (TDD) and frequency-division duplex (FDD) modes. I-B CSI-RS and DMRS Configuration Channel estimation in 5G NR follows the Demodulation Reference Signal (DMRS) pilot pattern, where pilots are transmitted on a sparse subset of RBs and symbols. Formally, let CSI⊆1,…,KULK_CSI \1,…,K_UL\ denote the set of RBs carrying pilot signals and let CSI⊆1,…,PS_CSI \1,…,P\ denote the corresponding time indices. The observed CSI is given by ~k,sUL=k,sUL+k,s,(k,s)∈CSI×CSI, h^UL_k,s=h^UL_k,s+n_k,s\ , (k,s) _CSI×S_CSI\ , (3) where k,s∼(0,σn2)n_k,s (0, _n^2I) models estimation noise. Note that different DMRS pilot patterns can be configured depending on the scenario, controlling the corresponding pilot density. I-C Channel State Prediction Task Given the UL history window over P slots, we aim to predict the future DL CSI over the next L slots across all RBs. To this end, let the UL history be defined as ℋhisUL=~k,sUL∣k∈CSI,s∈CSIH_his^UL=\ h^UL_k,s k _CSI,s _CSI\\ (4) with corresponding DL prediction target ℋpreDL=k,sDL∣k∈1,…,KDL,s∈P+1,…,P+L,H_pre^DL=\h^DL_k,s k∈\1,…,K_DL\,s∈\P+1,…,P+L\\\ , (5) This defines a sequence-to-sequence prediction of the form ℋ^preDL=fΘ(ℋhisUL), H_pre^DL=f_ (H_his^UL)\ , (6) where fΘf_ denotes a parameterized model with learnable parameters Θ , whose prediction performance is evaluated using the normalized mean square error (NMSE), i.e., NMSE=[∑k,s‖^k,sDL−k,sDL‖22∑k,s‖k,sDL‖22].NMSE=E [ _k,s\| h^DL_k,s-h^DL_k,s\|_2^2 _k,s\|h^DL_k,s\|_2^2 ]. (7) In our work, we require fΘf_ to not only accurately predict future DL CSI, but also to minimize the computational complexity and latency, enabling real-time deployment. I MambaCSP: Hybrid-Attention State Space Models for CSI Prediction Figure 1: CSI prediction pipeline: Historical UL CSI is converted into frequency/delay components, normalized, rearranged, and partitioned into temporal patches. CSI token mixer blocks then produce token embeddings for the sequence model. A final prediction head maps the outputs back to the complex CSI domain. In this section, we introduce a unified framework for CSI prediction that is compatible with both LLM and SSM sequence models. We then present the proposed hybrid MambaCSP architecture, which combines the efficiency of SSMs with the expressivity of attention mechanisms. I-A Unified CSI Prediction Framework Building on prior works [4, 5], we generalize attention-based CSP pipelines into a unified framework that supports arbitrary sequence modeling architectures, including SSMs (see Fig. 1). The key components are outlined as follows. I-A1 Data Preprocessing CSI data is complex-valued and needs to be converted into real-valued embeddings first. To this end, we extract frequency- and delay-domain representations, normalize the data, and partition it into temporal patches. We denote the historical UL CSI over P slots and K RBs as f∈ℂK×PH_f ^K× P and its delay-domain representation as τ=KHfH_τ=F_K^HH_f, where KF_K denotes the K-point DFT matrix. We then separate real and imaginary components and obtain f,τ∈ℝ2×K×PX_f,X_τ ^2× K× P. To ensure stable training across varying SNR conditions, we normalize each representation and reshape the tensors by merging feature dimensions, yielding ~f,~τ∈ℝ2K×P X_f, X_τ ^2K× P. To further reduce sequence length and computational complexity, we partition the temporal dimension into non-overlapping patches of size N (see Fig. 1a), resulting in fp,τp∈ℝ2K×N×P′,X_f^p,X_τ^p ^2K× N× P , (8) where P′=⌈P/N⌉P = P/N denotes the number of patches. I-A2 CSI Tokenization Preprocessed CSI patches need to be further tokenized into embeddings that are compatible with the input format of language models. To this end, each patch is processed by a CSI Token Mixer (see Fig. 2), which adaptively reweights features to emphasize informative multipath components while attenuating less relevant or noisy ones. Let i∈ℝ2K×N×P′X_i ^2K× N× P denote the input tensor of a token mixer block. First, local feature extraction via convolutional Conv(⋅)Conv(·) and ReLU(⋅)ReLU(·) layers is performed to capture joint temporal and frequency-domain dependencies, i.e., feat=Conv(ReLU(Conv(i))).X_feat=Conv(ReLU(Conv(X_i)))\ . (9) To model the relative importance of each token, a channel-wise gating mechanism with sigmoid activation σ(⋅)σ(·) is applied gate=σ(FC(ReLU(FC(GAP(feat))))),X_gate=σ(FC(ReLU(FC(GAP(X_feat)))))\ , (10) where GAP(⋅)GAP(·) and FC(⋅)FC(·) denote global average pooling and fully connected layers. The resulting weights gate∈ℝ1×1×P′X_gate ^1× 1× P are then used to reweight each token, i.e., sca[:,:,i]=gate[i]⋅feat[:,:,i],X_sca[:,:,i]=X_gate[i]·X_feat[:,:,i]\ , (11) before applying a residual connection, which preserves the original features and stabilizes gradient flow during training o=sca+i.X_o=X_sca+X_i\ . (12) Next, to jointly capture spectral correlations and multipath structure, we cascade multiple CSI Token Mixer blocks over the frequency/delay representations, and combine them as tok=TM(N)(fp)+TM(N)(τp),X_tok=TM^(N)(X_f^p)+TM^(N)(X_τ^p)\ , (13) where TM(N)(⋅)TM^(N)(·) denotes N cascaded token mixer blocks. In practice, we employ N∈[2,4]N∈[2,4] to balance modeling capacity and computational efficiency. The resulting tensor is rearranged into ~tok∈ℝ2KN×P′ X_tok ^2KN× P and projected into the model embedding dimension, yielding ¯tok∈ℝF×P′ X_tok ^F× P . Figure 2: CSI token mixer module. Furthermore, to preserve the temporal ordering, we follow [4] and add sinusoidal positional encodings defined as PE(i,j)=sin(j10000i/F),ieven,cos(j10000(i−1)/F),iodd,X_PE(i,j)= cases ( j10000^i/F )\ ,&i\ even\ ,\\ ( j10000^(i-1)/F )\ ,&i\ odd\ , cases (14) where i indexes the feature dimension and j the token position. The final CSI token embeddings are then obtained as emb=¯tok+PE.X_emb= X_tok+X_PE\ . (15) I-A3 Sequence Model Backbone To model spatio-temporal dependencies across CSI token sequences, we leverage the pre-trained backbone’s capacity to attend to complex multi-scale interactions rather than its learned semantic world knowledge. During training, we therefore keep the backbone’s core layers frozen while only adapting lightweight task-specific components. In LLMs, for example, this corresponds to freezing the multi-head attention and feed-forward blocks, while only fine-tuning LayerNorm and Addition layers [4, 5]. This significantly reduces the overall training overhead and yields contextualized token representations given by seq=fΘb(emb)∈ℝF×P′,X_seq=f_ _b(X_emb) ^F× P \ , (16) where fΘb(⋅)f_ _b(·) denotes the sequence model backbone, and seqX_seq captures the learned spatio-temporal dependencies. I-A4 Prediction Head In the final step, seqX_seq is mapped back to its original domain. We first apply a linear projection to obtain ^∈ℝ2K×L X ^2K× L and reshape the predicted tensor into real/imaginary components of the form ^∈ℝ2×K×L X ^2× K× L. We then de-normalize to recover the original CSI scale, i.e., ^de=σ^+μ, X_de=σ\, X+μ\ , (17) where μ and σ denote the mean and standard deviation used during preprocessing. Finally, the predicted CSI is obtained as ^DL=^de[1,:,:]+j^de[2,:,:]. H^DL= X_de[1,:,:]+j\, X_de[2,:,:]\ . (18) I-B Hybrid Mamba Architecture Mamba [7] extends classical SSMs by introducing input-dependent state transitions. Instead of fixed matrices A (state transition), B (input projection), and C (output projection), Mamba parameterizes these operators as functions of the input: t=(t)t−1+(t)t,t=(t)t.h_t=A(x_t)h_t-1+B(x_t)x_t\ , _t=C(x_t)h_t\ . (19) This allows Mamba to dynamically control how information is propagated and updated at each time step, improving its ability to model non-stationary sequences. In addition, Mamba incorporates a continuous-time inductive bias via learned step sizes Δt t, enabling adaptive temporal scaling of the state updates. This is particularly well-suited for Doppler-induced channel variations, which evolve continuously over time. However, Mamba propagates information recursively through the hidden state. As a result, information from distant tokens must be compressed into th_t, which can lead to degradation over long horizons. Consequently, Mamba does not explicitly model pairwise interactions across the entire sequence as attention-based models do. This can be limiting for CSI prediction, particularly in a) high-mobility scenarios, where rapid channel variations introduce non-local dependencies across time and frequency, and in b) FDD settings, where predicting DL CSI from UL CSI requires modeling complex cross-frequency relationships spanning distant tokens. To address these limitations, we propose MambaCSP, a hybrid-attention extension of the Mamba backbone (see Fig. 3). Specifically, we introduce sparse patch-mixer attention layers, which operate on the tokenized CSI patches every k blocks and perform lightweight cross-token interactions using a small number of attention heads H (typically 2–4). Given emb∈ℝF×P′X_emb ^F× P , patch-mixer attention is computed as Attn(emb)=softmax(⊤dh)⊤,Attn(X_emb)=softmax\! ( Q K d_h )V \ , (20) where dhd_h denotes the head dimension and Q, K, and V are the query, key, and value projections obtained from embX_emb via learned linear mappings. The resulting representations are concatenated and projected back via O∈ℝF×FW_O ^F× F, i.e., mix=Concat(Attn1,…,AttnH)O.X_mix=Concat\! (Attn_1,…,Attn_H )W_O\ . (21) The attention output is then inserted sparsely every k Mamba blocks through a residual update, i.e., (ℓ+1)=(ℓ)+mix(ℓ),ℓ∈k,2k,….X^( +1)=X^( )+X_mix^( )\ , ∈\k,2k,…\\ . (22) This restores global context modeling while maintaining near-linear complexity. During training, patch-mixer attentions are learned jointly with projection and LayerNorm layers, while the majority of the Mamba backbone remains frozen. I-C Computational Complexity and Efficiency Plain Mamba’s state-space updates process sequences one step at a time with recurrence. Thus, for a token sequence of length P′P and hidden dimension F, compute scales linearly as (P′F)O(P F). Moreover, since Mamba does not construct a full pairwise attention matrix and does not require KV caching, its memory usage also scales linearly with sequence length. For our hybrid MambaCSP, patch-mixer attentions are inserted only intermittently and use a small number of heads. Thus, for LML_M Mamba blocks and ⌊LM/k⌋ L_M/k patch-mixer layers, where in practice ⌊LM/k⌋≪LM L_M/k L_M, the overall complexity is (LMP′F)+(⌊LMk⌋P′2F)≈(LMP′F),O(L_MP F)+O\! ( L_Mk P 2F )\;≈\;O(L_MP F)\ , (23) such that MambaCSP retains near-linear scaling overall. In contrast, LLM backbones require self-attention in every layer. As a result, the forward computational cost scales quadratically as (P′2F)O(P 2F), while memory is dominated by the attention matrix of size P′×P′P × P , i.e., (P′2)O(P 2). Although freezing transformer layers reduces the number of trainable parameters during training, it does not remove the quadratic self-attention computation in the forward pass. Therefore, MambaCSP provides a more favorable complexity–efficiency trade-off for long CSI histories and low-latency deployment. Figure 3: Hybrid-attention MambaCSP architecture at layer l. IV Experimental Setup To evaluate MambaCSP, we generate a comprehensive CSI dataset with QuaDRiGa [8] for the 3GPP TR 38.901 UMa NLOS channel. The OFDM numerology and all remaining parameters follow our system model in Sec. I. The BS is placed at (0,0,30)(0,0,30) m and is equipped with a 4×44× 4 dual-polarized UPA. For each UE, we simulate a frame of Nslot=20N_slot=20 slots, where the first P=16P=16 slots form the history window and the remaining L=4L=4 slots form the prediction window. For each speed realization, we randomly place 1010 UEs in an annulus between 20 m and 50 m around a cluster center at (200,0,1.5)(200,0,1.5) m and generate linear user tracks. For DMRS, we employ a frequency-selective pilot pattern with temporal spacing ΔtDMRS=2Δt t_DMRS=2 t and frequency spacing Δk=1 k=1 RB. In total, we generate 8000 training, 2000 validation, and 10000 test samples for both TDD and FDD111Training code and data at: https://github.com/aladinD/MambaCSP. We benchmark our hybrid MambaCSP against plain Mamba, LLM-based prediction [4], and classical neural baselines including CNN [9], RNN [10], and LSTM [11]. To ensure fairness, we employ a 130M parameter size Mamba 2 model [12], whose scale is comparable to the 125M parameter size GPT-2 [13] model used in [4]. We further note that increasing the backbone model size or changing the architecture, e.g., to GPT-OSS [14] or other larger models, yields at most marginal gains for CSP, since the task does not primarily rely on semantic world knowledge or reasoning abilities (see Table I). We train each model for 10 epochs using Adam with β=(0.9,0.999)β=(0.9,0.999), batch size 256, and learning rate 10−310^-3. V Results and Discussions We evaluate MambaCSP in terms of prediction accuracy and hardware efficiency, and perform additional ablations. V-A Prediction Accuracy Fig. 4 and Fig. 5 compare the NMSE of all considered baselines for the TDD and FDD settings, respectively, showing that MambaCSP outperforms LLM-based CSP in both duplex modes. For TDD, MambaCSP achieves 5% lower NMSE than plain Mamba and 9% lower NMSE than the LLM, indicating that state-space modeling already captures most relevant temporal structures in TDD, while patch-mixer attention provides a moderate but non-negligible gain. For FDD, the relative benefit of MambaCSP becomes more pronounced. It consistently outperforms plain Mamba by 17% and the LLM by 12% across the full velocity range, showing that patch-mixer attention is particularly beneficial in FDD when non-local cross-frequency dependencies arise due to the UL-to-DL prediction gap, which causes plain Mamba to underperform, particularly at higher velocities. Furthermore, MambaCSP, plain Mamba, and the LLM significantly outperform CNN, RNN, and LSTM baselines, suggesting that sequence models are better at predicting spatio-temporal CSI dependencies. V-B Throughput Fig. 6 shows the training throughput as the sequence length P′P increases. As expected, throughput decreases noticeably as the input history becomes longer. Overall, LLM-based CSP degrades much faster than Mamba-based variants. Specifically, MambaCSP achieves 2.7×2.7×, 2.9×2.9×, and 3.0×3.0× higher throughput than the LLM at P′∈32,48,64P ∈\32,48,64\, respectively, which is similar to plain Mamba. This shows that the proposed patch-level attention introduces only a minor throughput overhead. V-C Memory Consumption Fig. 7 reports the peak memory usage when increasing the sequence length P′P . In line with our theoretical observations, the LLM exhibits a much steeper increase in memory consumption, driven by the quadratic attention matrix and KV caching. For short input histories up to P′=8P =8, the memory gap between LLM and Mamba-based models remains approximately constant, but it widens steadily as P′P grows. Specifically, the LLM requires 2.0×2.0×, 2.3×2.3×, and 2.6×2.6× more VRAM than the Mamba variants for P′∈32,48,64P ∈\32,48,64\. In contrast, plain Mamba and MambaCSP remain close and scale linearly in memory, substantiating our analysis that patch-level attention incurs only a negligible memory overhead. V-D Inference Latency Fig. 8 compares the inference latency per forward pass across sequence lengths. Similar to memory usage, latency increases with P′P for all models. MambaCSP is about 2.0×2.0×, Figure 4: NMSE results for TDD. Across all approaches, the NMSE increases with increasing velocity. MambaCSP outperforms plain Mamba and the LLM. Figure 5: NMSE results for FDD. Across all approaches, the NMSE remains approximately flat. MambaCSP outperforms plain Mamba and the LLM. Figure 6: Throughput for different sequence lengths. MambaCSP achieves up to 3×3× more samples/s for longer CSI patches, comparable to plain Mamba. Figure 7: Memory consumption for different sequence lengths. LLM grows faster with sequence length due to KV caching. Mamba backbones remain linear. Figure 8: Latency for different sequence lengths. Mamba backbones scale linearly with longer sequence lengths compared to LLM (up to 3×3× slower). 2.6×2.6×, and 2.9×2.9× faster than the LLM at P′∈32,48,64P ∈\32,48,64\, respectively, while remaining close to plain Mamba, thereby preserving the latency benefits of the state-space backbone. V-E NMSE vs Input Sequence Length Table I reports the FDD NMSE for MambaCSP and LLM backbones as the input sequence length P′P increases. Overall, a longer CSI history improves prediction (with diminishing returns at P′=32P =32) as additional past observations expose more spatio-temporal structure that is useful for inferring future DL CSI. This supports our motivation for increasing P′P , especially in FDD, which benefits from longer-range CSI dependencies. Similar trends can be observed for TDD. V-F Ablations on MambaCSP Patch-Mixer Attention Layers Table I ablates the impact of the patch-mixer attention layers in MambaCSP. Overall, injecting cross-attention with H=2H=2 heads every k=4k=4 Mamba blocks is sufficient to restore non-local sequence modeling capabilities for complex CSI patches, while providing a favorable trade-off between prediction accuracy and efficiency. Further increasing the number of heads or inserting attention more frequently yields only marginal additional gains at the cost of increased complexity. In summary, our results demonstrate that MambaCSP achieves a substantially better accuracy–efficiency trade-off than LLM-based CSP, particularly for longer CSI histories. Its lower latency and memory footprint make it especially attractive for real-time deployment on low-resource devices. Further reducing memory and compute through quantization and model pruning is an interesting direction for future work. VI Conclusion In this paper, we proposed MambaCSP, a hybrid-attention state space model architecture as a hardware-efficient alternative to LLM-based CSI prediction. MambaCSP replaces the LLM backbone in existing CSP pipelines with a Mamba-based sequence model and augments it with sparse patch-mixer attention layers to better capture long-range CSI token dependencies while preserving the linear-time efficiency of state space models. Our experiments show that MambaCSP achieves better accuracy than LLM-based prediction in both TABLE I: NMSE versus sequence length P′P for LLM and MambaCSP (FDD). Increasing P′P improves prediction with diminishing returns at larger contexts. Seq. Length P′P LLM NMSE MambaCSP NMSE Relative Gain 4 0.503 0.468 7.0% 8 0.462 0.422 8.7% 16 0.425 0.366 13.9% 32 0.392 0.331 15.6% 48 0.381 0.318 16.5% 64 0.374 0.309 17.4% TABLE I: Ablation of the patch-mixer attention layers in MambaCSP. Sparse attention improves FDD more than TDD. In practice, we use k=4k=4 and H=2H=2 as a favorable accuracy-efficiency trade-off. Patch-Mixer Interval k Heads H TDD NMSE ↓ FDD NMSE ↓ None – – 0.0374 0.4222 Yes 4 2 0.0346 0.3750 Yes 4 4 0.0345 0.3736 Yes 2 4 0.0345 0.3731 TABLE I: Comparison of different backbones. Larger models provide only marginal gains for CSP despite substantially higher memory requirements. Backbone Params VRAM (GB) TDD NMSE ↓ FDD NMSE ↓ GPT-2 Small 124M 6.8 0.050 0.425 GPT-OSS 1.3B 18.9 0.049 0.423 Mamba2-130M 130M 5.1 0.049 0.422 Mamba2-780M 780M 11.8 0.048 0.420 TDD and FDD settings, while providing substantially higher throughput, lower memory consumption, and lower inference latency. Our findings thus point to a key takeaway: LLMs are not the final architectural choice for long-context CSI prediction, and combining efficient state space models with lightweight attention is a promising path toward scalable and real-time AI-native CSI prediction in future wireless systems. References [1] F. Jiang, C. Pan, K. Wang, P. Michiardi, O. A. Dobre, and M. Debbah, “From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications,” IEEE JSAC, 2026. [2] A. Djuhera, F. Koch, and A. Binotto, “Joint Partitioning and Placement of Foundation Models for Real-Time Edge AI,” IEEE ICNC, 2026. [3] J. Guo, Y. Cui, S. Jin, and J. Zhang, “Large AI Models for Wireless Physical Layer,” IEEE Communications Magazine, 2026. [4] B. Liu, X. Liu, S. Gao, X. Cheng, and L. Yang, “LLM4CP: Adapting Large Language Models for Channel Prediction,” Journal of Communications and Information Networks, vol. 9, no. 2, p. 113–125, 2024. [5] S. Fan, Z. Liu, X. Gu, and H. Li, “CSI-LLM: A novel Downlink Channel Prediction Method Aligned with LLM Pre-Training,” in IEEE Wireless Communications and Networking Conference, 2025. [6] Y. Cui, J. Guo, C.-K. Wen, S. Jin, and E. Tong, “Exploring the Potential of Large Language Models for Massive MIMO CSI Feedback,” arXiv preprint arXiv:2501.10630, 2025. [7] A. Gu and T. Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” in COLM, 2024. [8] S. Jaeckel, L. Raschkowski, K. Börner, and L. Thiele, “QuaDRiGa: A 3-D Multi-Cell Channel Model with Time Evolution for Enabling Virtual Field Trials,” IEEE Transactions on Antennas and Propagation, 2014. [9] M. S. Safari, V. Pourahmadi, and S. Sodagari, “Deep UL2DL: Data-Driven Channel Knowledge Transfer from Uplink to Downlink,” IEEE Open Journal of Vehicular Technology, 2019. [10] W. Jiang and H. D. Schotten, “Neural Network-Based Fading Channel Prediction: A Comprehensive Overview,” IEEE Access, 2019. [11] —, “Deep Learning for Fading Channel Prediction,” IEEE Open Journal of the Communications Society, 2020. [12] T. Dao and A. Gu, “Mamba-2 130M Model Weights,” https://huggingface.co/state-spaces/mamba2-130m, 2024. [13] A. Radford et al., “Language Models are Unsupervised Multitask Learners,” OpenAI Blog, 2019. [14] OpenAI, “GPT-OSS-120B & GPT-OSS-20B Model Card,” 2025. [Online]. Available: https://arxiv.org/abs/2508.10925