Paper deep dive
TFGformer: Multivariate Time Series Forecasting via Time-Frequency Graph Learning and Covariate Fusion
Yu Sun, Yuan Chang, Xiaohou Shi, Yan Sun
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/3/2026, 3:04:00 AM
Summary
The paper introduces TFGformer, a multivariate time series forecasting framework that integrates time-frequency graph structure learning with covariate-aware representation fusion. It utilizes Short-Time Fourier Transform (STFT) and Mahalanobis distance to learn dynamic inter-variable dependencies and employs MLP-based modules to fuse historical and future covariates, demonstrating state-of-the-art performance on benchmarks like Electricity, Traffic, and Weather.
Entities (13)
Relation Signals (11)
TFGformer → evaluatedon → Traffic
confidence 95% · The results demonstrate that our approach consistently achieves state-of-the-art performance ... on seven widely used benchmark datasets spanning the ... traffic
TFGformer → evaluatedon → Electricity
confidence 95% · The results demonstrate that our approach consistently achieves state-of-the-art performance ... on seven widely used benchmark datasets spanning the electricity
TFGformer → evaluatedon → ETT
confidence 95% · We conduct extensive experiments on four real-world multivariate time series datasets: ETT
TFGformer → evaluatedon → Weather
confidence 95% · The results demonstrate that our approach consistently achieves state-of-the-art performance ... on seven widely used benchmark datasets spanning the ... weather domains.
TFGformer → uses → Time-Frequency Graph (TFG) Module
confidence 95% · The proposed model consists of three main components: the Time–Frequency Graph (TFG) module, the Covariate Feature Fusion (CFM) module, and a Transformer-based sequence modeling module.
Time-Frequency Graph (TFG) Module → uses → Short-Time Fourier Transform (STFT)
confidence 95% · The Time-Frequency Graph (TFG) module leverages the Short-Time Fourier Transform (STFT) to capture complementary temporal–spectral patterns.
TFGformer → uses → Covariate Feature Fusion (CFM) Module
confidence 95% · The CFM module is divided into two sub-modules: the Input Covariate Fusion (I-CFM) ... and the Output Covariate Fusion (O-CFM)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large-scale multivariate time series from heterogeneous IoT sensors demand accurate long-term forecasting for resource scheduling and predictive maintenance. While recent time series foundation models exhibit strong generalization, they rely on static parametric knowledge and lack dynamic access to external historical patterns during inference. Retrieval-Augmented Generation (RAG) offers a potential remedy, yet its application to time series forecasting is challenged by magnitude variations across heterogeneous sources and the mismatch between historical similarity and future consistency. We propose CrossRAG, a retrieval-augmented forecasting framework that integrates Shape-Aware Memory (SAM) with RevIN normalization for magnitude-robust shape-level retrieval, Future-Consistent Contrastive (FCC) learning to distinguish informative references from hard negatives with similar history but divergent futures, and Cross-Attention Temporal Fusion (CATF) to fuse retrieved historical--future reference pairs into the backbone's representations at the representation level. Experiments on seven public benchmarks show that CrossRAG consistently outperforms both parametric-only baselines and existing retrieval-augmented forecasting methods.
Tags
Links
- Source: https://arxiv.org/abs/2607.29459v1
- Canonical: https://arxiv.org/abs/2607.29459v1
Trouble viewing inline? Open PDF directly →
Full Text
29,566 characters extracted from source content.
Expand or collapse full text
TFGformer: Multivariate Time Series Forecasting via Time-Frequency Graph Learning and Covariate Fusion †thanks: This work is supported by the National Natural Science Foundation of China under Grants 62272052 Yu Sun Yuan Chang Xiaohou SHI Yan Sun Abstract Multivariate time series forecasting requires modeling complex temporal dynamics and inter-variable dependencies. However, in practice, noisy correlations often introduce interference, while crucial contextual covariates such as calendar features and external events are frequently underutilized. This highlights the need for models that selectively identify meaningful relationships while effectively incorporating auxiliary information. To address these challenges, we propose a unified framework integrating time–frequency graph structure learning with covariate-aware representation fusion. The Time-Frequency Graph (TFG) module leverages the Short-Time Fourier Transform (STFT) to capture complementary temporal–spectral patterns. It then learns adaptive inter-variable correlations via a weighted Mahalanobis distance and employs Gumbel–Softmax sampling to derive sparse, dynamic graphs that suppress irrelevant dependencies. In parallel, MLP-based modules fuse historical and future covariates into the main sequence representation, enabling the model to exploit trend priors and contextual signals.Extensive experiments on electricity, traffic, and meteorology benchmarks demonstrate that our approach consistently outperforms state-of-the-art Transformer-based models, confirming its effectiveness in modeling selective variable interactions and leveraging covariates for improved forecasting accuracy. I Introduction Transformer-based models have achieved remarkable success in NLP [1] and vision [2], spurring the development of foundation models for multivariate time series forecasting (MTSF). MTSF uncovers dynamic patterns from historical data, playing a crucial role in various real-world applications. As datasets grow complex, inter-variable dependencies become more intricate. Since long-term forecasting naturally suffers from error accumulation [3], effectively capturing these time-varying relationships while leveraging external covariates remains a key challenge. Existing approaches attempt to address inter-variable dependencies in various ways. For example, PatchTST [4] segments sequences into patches and employs channel-independent processing to mitigate interference from weakly correlated variables; however, it lacks explicit cross-variable modeling and interpretability. iTransformer [5] introduces channel-mixing self-attention to capture variable-level dependencies, yet it overlooks fine-grained temporal dynamics within individual variables. TimerXL [6] improves upon this by jointly modeling intra- and inter-variable patterns, but it still assumes static associations across all variable pairs, failing to account for the strength and temporal evolution of these dependencies. This limitation becomes particularly problematic in scenarios where relationships are sparse or change over time. Meanwhile, covariates such as calendar attributes, device metadata, and event-driven signals carry valuable prior knowledge that can significantly enhance forecasting accuracy. Yet, most existing models incorporate them through simple concatenation or by treating them as extra channels, which often introduces noise and overlooks their inherent structure. Although models like TiDE [7] and ChronosX [8] highlight the benefits of dedicated covariate fusion, TiDE is primarily an MLP-based architecture, while ChronosX is tailored for univariate, token-based forecasting. Consequently, neither is ideally suited for explicitly modeling dynamic inter-variable dependencies in a multivariate, patch-based setting. To address these limitations, this paper proposes a novel multivariate time series forecasting framework that integrates time–frequency graph learning with effective covariate fusion. Our main contributions are threefold: • We design a novel time-frequency graph structure learning module that leverages Short-Time Fourier Transform (STFT) and Mahalanobis distance to capture dynamic and nonlinear dependencies among variables. By estimating pairwise relational probabilities in the joint time–frequency domain, the module adaptively constructs an attention mask that guides the attention mechanism, enabling more accurate modeling of complex inter-variable interactions. • We introduce an MLP-based covariate fusion module that integrates both historical and future covariates with the primary sequence representation through residual connections, without losing the original data characteristics. This design allows the model to fully exploit prior trend knowledge and external dynamic context. • We conduct comprehensive experiments on seven widely used benchmark datasets spanning the electricity, traffic, and weather domains. The results demonstrate that our approach consistently achieves state-of-the-art performance across multiple forecasting horizons and evaluation metrics, validating its effectiveness in modeling dynamic inter-variable relationships and leveraging covariate information. I Related Works I-A Transformer-Based Time Series Forecasting Transformers show strong potential in MTSF. Early works like Informer and Autoformer [8] improved attention for long-term dependencies. However, effectively modeling complex inter-variable relationships remains a significant challenge. Existing Transformer-based models for multivariate time series generally fall into two categories. Channel-independent [10] approaches, such as PatchTST, model each variable separately, focusing on intra-variable temporal patterns. Although effective, they do not explicitly capture interactions between variables. Channel-mixing [11] approaches like iTransformer treat each variable as a token and apply attention across the variable dimension, thereby modeling inter-variable relationships but potentially overlooking fine-grained temporal dynamics within each variable. To leverage the strengths of both directions, recent hybrid designs such as TimerXL combine patching strategies with channel-mixing mechanisms, enabling simultaneous modeling of intra- and inter-variable patterns. Nevertheless, these models implicitly assume that all variable pairs share equal levels of correlation. They are unable to distinguish the varying strengths or temporal evolution of relationships among variables, which may introduce noise when dependencies are sparse or highly dynamic. I-B Frequency-Domain Analysis in Time Series Modeling Recent research has revealed that frequency-domain features offer natural advantages in depicting multivariate synchrony and periodic consistency. Motivated by this, several methods have shifted from pure time-domain modeling to frequency-domain representations [12], leveraging amplitude and phase information [13] to characterize inter-variable relationships. Nevertheless, most existing approaches rely on global Fourier transforms, which map the entire sequence into holistic frequency components, thereby overlooking temporal locality. As a result, they struggle to capture the non-stationary and time-varying correlation structures that are common in real-world multivariate time series. I-C Covariate Utilization in Time Series Forecasting In practical forecasting tasks, auxiliary covariates such as weather conditions, calendar features, and holiday indicators provide valuable prior information. Despite this, mainstream forecasting models typically adopt relatively basic fusion strategies. One common approach encodes covariates in a simple manner and then concatenates them with the main series. Another approach directly treats covariates as additional variables and processes them together with the primary inputs. However, because heterogeneous covariates exhibit distinct temporal patterns and statistical properties, such naive concatenation or joint modeling often introduces noise and prevents the model from effectively extracting useful covariate information.As an attempt to improve this, models like ChronosX have introduced a fusion module. However, it is noteworthy that its design is targeted at univariate scenarios and operates on a token-based framework, rather than patch-based. I Methodology I-A Problem Definition This study focuses on the task of multivariate time series forecasting. Given a historical observation sequence X=x1,x2,…,xL∈ℝL×CX=\x_1,x_2,…,x_L\ ^L× C, where L is the sequence length and C denotes the number of variables.We additionally consider historical covariates Chistory∈ℝL×CzC_history ^L× C_z,and future covariates Cfuture∈ℝS×CzC_future ^S× C_z, which provide auxiliary information such as calendar features, weather indicators, or other exogenous factors. The forecasting model aims to generate future series values based on the historical observations and covariates: Y^=f(X,Chis,Cfut) Y=f(X,C_his,C_fut), where Y^∈ℝS×C Y ^S× C denotes the predicted multivariate sequence for the next S time steps. I-B Structure Overview Figure 1: The overall architecture of TFGformer. Our model architecture is illustrated in Fig. 1 The proposed model consists of three main components: the Time–Frequency Graph (TFG) module, the Covariate Feature Fusion (CFM) module, and a Transformer-based sequence modeling module. Given a multivariate time series along with historical and future covariates, the TFG module first analyzes temporal and spectral patterns to learn dynamic inter-variable relationships, producing an attention mask that guides cross-variable interactions. The CFM module is divided into two sub-modules: the Input Covariate Fusion (I-CFM) integrates historical covariates with the original time series before entering the Transformer, while the Output Covariate Fusion (O-CFM) incorporates future covariates after Transformer processing to further refine the hidden representations. The enriched sequence, containing relational information from TFG and covariate-aware features from I-CFM, is fed into the Transformer backbone. The attention mask generated by the TFG module is applied during attention computation to explicitly model relevant inter-variable dependencies. After Transformer processing, future covariates are fused via O-CFM, and the final forecasting head produces the multivariate predictions for the target horizon. I-C Basic Components • Normalization Per-channel Z-score [14] normalization is applied along the temporal dimension: x^i,t=xi,t−μiσi x_i,t= x_i,t- _i _i (1) • Patching The input sequence is split into fixed-length temporal patches using a sliding window: N=L−ls+1N= L-ls+1 (2) • Output Head A linear layer maps the Transformer outputs to the prediction sequence: Y^=HW+b Y=HW+b (3) I-D Time–Frequency Graph Learning Module For each variable xc x_c in the multivariate sequence, the module first computes its short-time Fourier transform (STFT) to obtain the magnitude spectrogram. Where nfft n_ fft denotes the FFT size that determines the number of frequency bins F=nfft2+1F= n_fft2+1, hop hop is the hop length controlling the temporal stride of successive frames, and win win specifies the window length used in the analysis. Subsequently, the time-frequency dimensions are flattened to form the channel feature vectors Z. where Nfeat=F⋅TN_feat=F· T is the dimensionality of the flattened time-frequency feature space. Mc=|STFT(xc,nfft,hop,win)|∈ℝF×TM_c= |STFT(x_c,n_fft,hop,win) | ^F× T (4) Z=Flatten(Mc)∈ℝB×C×NfeatZ=Flatten(M_c) ^B× C× N_feat (5) To adaptively discover the interrelationships among channels, we adopt a learnable Mahalanobis distance [15] metric to evaluate the pairwise distances over the feature vectors ZZ. This metric moves beyond simple weighted distances by learning a matrix that captures the covariance structure of the time-frequency feature space. The distance Di,jD_i,j between the feature vectors of channel i (ZiZ_i) and channel j (ZjZ_j) is defined as a quadratic form: Di,j=(Zi−Zj)TQ(Zi−Zj)D_i,j=(Z_i-Z_j)^TQ(Z_i-Z_j) (6) where D∈ℝB×C×CD ^B× C× C is the resulting distance matrix, and Q∈ℝNfeat×NfeatQ ^N_feat× N_feat is a learnable symmetric positive semi-definite (PSD) matrix.We then convert the distance into a similarity (or unnormalized probability) using the inverse relationship and mask out the diagonal elements (distance from self to self): P~i,j=1Di,j+ϵif i≠j0if i=j P_i,j= cases 1D_i,j+ε&if i≠ j\\ 0&if i=j cases (7) To obtain the final probabilistic matrix P∈[0,1]B×C×CP∈[0,1]^B× C× C, we perform a normalization step. We utilize max normalization on the non-diagonal elements to stabilize the gradients and preserve relative distances, followed by re-inserting the diagonal elements. P=P~max(P~,dim=−1)+ϵ+IP= P ( P,dim=-1)+ε+I (8) To transform the continuous probabilistic matrix PP into a discrete binary adjacency matrix suitable for downstream attention operations, we treat each element PijP_ij as the success probability of a Bernoulli random variable. Specifically, we sample a binary matrix G∈[0,1]B×C×CG∈[0,1]^B× C× C according to Gij∼Bernoulli(Pij)G_ij (P_ij) (9) where a higher probability PijP_ij corresponds to a greater likelihood that Gij=1G_ij=1, indicating a stronger inferred dependency between channel i i and channel j j. To allow gradient propagation through this sampling process, we employ the Gumbel–Softmax [16] reparameterization trick. G=GumbelSoftmax(Gij)G=GumbelSoftmax(G_ij) (10) The resulting binary mask GG effectively filters out noise and weak correlations, ensuring that subsequent computations focus only on the most relevant inter-channel paths learned from the time-frequency domain. To integrate this learned spatial dependency with the temporal dynamics, we first define a causal time mask Mtime∈0,1N×NM_time∈\0,1\^N× N as a lower triangular matrix. The final Attention Mask MattentionM_attention is then constructed by applying the Kronecker [17] Product (⊗ ) between the expanded channel mask GG and the time mask MtimeM_time: Mattention=G⊗MtimeM_attention=G M_time (11) The computational complexity of the TFG module scales efficiently with the sequence length L and the variable count C. First, the frequency feature extraction requires (C⋅LlogL)O(C· L L) time. Second, for graph structure learning, we utilize a learnable diagonal weight vector optimized via tensor broadcasting. This specific design reduces the relation computation complexity to (C2⋅L)O(C^2· L). I-E Covariate Fusion Module To effectively incorporate external signals into the forecasting pipeline, the model introduces a covariate fusion module consisting of two components: the Input Covariate Fusion Module (I-CFM) and the Output Covariate Fusion Module (O-CFM). Both modules rely on MLP-based transformations to fuse covariates with the model’s internal representations, enabling the network to adjust its learned features according to historical and future contextual factors. I-CFM operates at the input stage and injects past covariate information into the patch embeddings. Let XembX_emb denote the patch embedding obtained from the original input sequence X. This embedding XembX_emb and the projected past covariates ChisC_ his are concatenated, processed through a ReLU activation, and then passed through an MLP-based feedforward network. The output is added to the original embedding XembX_emb through a residual path to obtain a covariate-aware input representation: Zin=Xemb+FFN(ReLU([WxX;WcChis]))Z_in=X_emb+FFN\! (ReLU\! ([\,W_xX\,;\,W_cC_his\,] ) ) (12) O-CFM operates at the decoder side and fuses future covariates with the Transformer-generated hidden states. The hidden state HH is first projected through a linear layer, concatenated with projected future covariates CfutC_ fut, and passed through a ReLU activation followed by the MLP. The fused output is added to the projected hidden state to modulate the decoder representation: Zout=WhH+FFN(ReLU([WhH;WfCfut]))Z_out=W_hH+FFN\! (ReLU\! ([\,W_hH\,;\,W_fC_fut\,] ) ) (13) In practical forecasting, future covariates CfutureC_future may contain missing values. To ensure the robustness of the O-CFM, we introduce a preprocessing step: missing continuous covariates are forward-filled, while categorical ones are zero-padded. This ensures the MLP-based fusion remains stable without propagating noise into the hidden states. I-F Transformer Backbone The Transformer architecture is employed to process the covariate-enhanced input patches Zin∈ℝB×(C⋅N)×DZ_in ^B×(C· N)× D. Multi-head self-attention is employed. To enforce the relevant channel interactions learned by the TFG module and to ensure causality, we apply the attention mask additively before the softmax operation. Scores=Q⋅K⊤dk⊙M+(1−M)⊙(−∞)Scores= Q·K d_k M+(1-M) (-∞) (14) Out=Softmax(Scores)⋅VOut=Softmax(Scores)· V (15) IV Experiment IV-A Experimental Settings TABLE I: MULTIVARIATE LONG-TERM FORECASTING RESULTS. Dataset Horizon TFGformer iTransformer TiDE PatchTST Autoformer FEDformer DLinear MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE ETTh1 96 0.380 0.399 0.386 0.405 0.383 0.403 0.414 0.419 0.449 0.459 0.376 0.419 0.386 0.400 192 0.432 0.437 0.441 0.436 0.437 0.433 0.460 0.445 0.500 0.482 0.420 0.448 0.437 0.432 336 0.474 0.455 0.487 0.458 0.455 0.453 0.501 0.466 0.521 0.496 0.459 0.465 0.481 0.459 720 0.485 0.474 0.503 0.491 0.490 0.489 0.500 0.488 0.514 0.512 0.506 0.507 0.519 0.516 Avg 0.442 0.441 0.454 0.447 0.441 0.445 0.469 0.454 0.496 0.487 0.440 0.460 0.456 0.452 ETTh2 96 0.294 0.346 0.297 0.349 0.301 0.350 0.302 0.348 0.346 0.388 0.358 0.397 0.333 0.387 192 0.378 0.392 0.380 0.400 0.384 0.402 0.388 0.400 0.456 0.452 0.429 0.439 0.477 0.476 336 0.399 0.430 0.428 0.432 0.420 0.434 0.426 0.433 0.482 0.486 0.496 0.487 0.594 0.541 720 0.430 0.446 0.427 0.445 0.433 0.450 0.431 0.446 0.515 0.511 0.463 0.474 0.831 0.657 Avg 0.375 0.403 0.383 0.407 0.385 0.409 0.387 0.407 0.450 0.459 0.437 0.449 0.559 0.515 ETTm1 96 0.321 0.354 0.334 0.368 0.332 0.370 0.329 0.367 0.505 0.475 0.379 0.419 0.345 0.372 192 0.364 0.380 0.377 0.391 0.380 0.401 0.367 0.385 0.553 0.496 0.426 0.441 0.380 0.389 336 0.384 0.401 0.426 0.420 0.398 0.408 0.399 0.410 0.621 0.537 0.445 0.459 0.413 0.413 720 0.442 0.437 0.491 0.459 0.447 0.437 0.454 0.439 0.671 0.561 0.543 0.490 0.474 0.453 Avg 0.377 0.393 0.407 0.410 0.389 0.404 0.387 0.400 0.588 0.517 0.448 0.452 0.403 0.407 ETTm2 96 0.170 0.247 0.180 0.264 0.173 0.256 0.175 0.259 0.255 0.339 0.203 0.287 0.193 0.292 192 0.240 0.297 0.250 0.309 0.245 0.310 0.241 0.302 0.281 0.340 0.269 0.328 0.284 0.362 336 0.298 0.337 0.311 0.348 0.304 0.341 0.305 0.343 0.339 0.372 0.325 0.366 0.369 0.427 720 0.394 0.393 0.412 0.407 0.406 0.398 0.402 0.400 0.433 0.432 0.421 0.415 0.554 0.522 Avg 0.275 0.318 0.288 0.332 0.282 0.326 0.281 0.326 0.327 0.371 0.305 0.349 0.350 0.401 Weather 96 0.162 0.209 0.174 0.214 0.185 0.226 0.177 0.218 0.266 0.336 0.217 0.296 0.196 0.255 192 0.218 0.249 0.221 0.254 0.237 0.265 0.225 0.259 0.307 0.367 0.276 0.336 0.237 0.296 336 0.271 0.288 0.278 0.296 0.282 0.305 0.278 0.297 0.359 0.395 0.339 0.380 0.283 0.335 720 0.348 0.347 0.358 0.347 0.359 0.353 0.354 0.348 0.419 0.428 0.403 0.428 0.345 0.381 Avg 0.249 0.273 0.258 0.278 0.266 0.287 0.259 0.281 0.338 0.382 0.309 0.360 0.265 0.317 Electricity 96 0.139 0.227 0.148 0.240 0.174 0.271 0.181 0.270 0.201 0.317 0.193 0.308 0.197 0.282 192 0.160 0.249 0.162 0.253 0.186 0.280 0.188 0.274 0.222 0.334 0.201 0.315 0.196 0.285 336 0.171 0.263 0.178 0.269 0.207 0.298 0.204 0.293 0.231 0.338 0.214 0.329 0.209 0.301 720 0.213 0.298 0.225 0.317 0.249 0.329 0.246 0.324 0.254 0.361 0.246 0.355 0.245 0.333 Avg 0.171 0.259 0.178 0.270 0.204 0.295 0.205 0.290 0.227 0.338 0.214 0.327 0.212 0.300 Traffic 96 0.390 0.259 0.395 0.268 0.450 0.289 0.462 0.295 0.613 0.388 0.587 0.366 0.650 0.396 192 0.412 0.273 0.417 0.276 0.458 0.292 0.466 0.296 0.616 0.382 0.604 0.373 0.598 0.370 336 0.424 0.277 0.433 0.283 0.478 0.301 0.482 0.304 0.622 0.377 0.621 0.383 0.605 0.373 720 0.462 0.297 0.467 0.302 0.497 0.313 0.514 0.322 0.660 0.408 0.626 0.382 0.645 0.394 Avg 0.422 0.276 0.428 0.282 0.471 0.299 0.481 0.304 0.628 0.389 0.610 0.376 0.625 0.383 We conduct extensive experiments on four real-world multivariate time series datasets: ETT (comprising four subsets ETTh1, ETTh2, ETTm1, ETTm2), ECL (321 clients), Traffic (862 road sensors), and Weather (21 meteorological indicators). Following established practice, we split the data into training, validation, and test sets with a ratio of 6:2:2 for the ETT datasets and 7:1:2 for the remaining datasets. Evaluation protocol. We fix the input length to 96 and evaluate prediction lengths L∈96,192,336,720L∈\96,192,336,720\. Mean Squared Error (MSE) and Mean Absolute Error (MAE) are used as core evaluation metrics. We train our model using the Adam optimizer with a learning rate tuned from 1e−4,5e−4,1e−3\1e-4,5e-4,1e-3\ for each dataset, combined with learning rate annealing and early stopping based on validation performance. The checkpoint achieving the lowest validation error is used for final testing. All experiments are conducted on a single NVIDIA A800-SXM4-40GB GPU. Baseline setting. We select six representative forecasting models as baselines: iTransformer, TiDE, PatchTST, Autoformer , FEDformer, and DLinear . IV-B Main Results and Analysis The comprehensive multivariate long-term forecasting results are presented in Table I. The best results are highlighted in bold, and the second-best results are underlined.It is evident that our TFGformer demonstrates outstanding predictive performance, achieving state-of-the-art (SOTA) results by consistently outperforming all baselines. TFGformer achieves the best average MSE and MAE on 6 out of 7 datasets (ETTh2, ETTm1, ETTm2, Weather, Electricity, and Traffic), and remains highly competitive on the final ETTh1 dataset.Compared to recent strong baselines, TFGformer shows significant improvements. On average across all seven datasets, TFGformer reduces MSE by 3.6% compared to iTransformer, 6.4% compared to PatchTST, 5.2% compared to TiDE, and 16.4% compared to the Transformer-based FEDformer. This indicates that TFGformer successfully enhances the long-term forecasting ability for complex multivariate time series (MTS) prediction problems. IV-C Ablation Study Figure 2: The ablation study results of TFGformer To validate the effectiveness of our model’s key components, specifically the TFG and CFM modules, we conducted a comprehensive ablation study. We compare our Full Model (Ours) against two variants: (i) W/o TFG, which removes the TFG module, and (i) W/o CFM, which removes the CFM module.The results on the Weather and ECL datasets are presented in Fig.2 The experimental results clearly demonstrate the contribution of each component. Our Full Model consistently achieves the best performance (lowest MSE and MAE) across all experiments.Quantitatively, removing the CFM module (W/o CFM) resulted in an average MSE increase of 3.0% compared to the full model across the two datasets. The impact of the TFG module was even more significant; removing it (W/o TFG) caused a larger performance degradation, with an average MSE increase of 6.0%. These results confirm that both modules are integral to our model’s architecture, and each part contributes effectively to enhancing the forecasting performance. IV-D Visualization of Variable Correlations Figure 3: Heatmap of learned variable correlations on the ETTh1 dataset We visualize the learned probability matrix P extracted from the ETTh1 dataset. As illustrated in Fig.3, the heatmap explicitly displays the learned dependency structure among the 7 variables. The TFG module successfully assigns higher relational probabilities to highly correlated variable pairs, while automatically filtering out noisy or irrelevant connections by pushing their weights near zero. This visualization confirms that TFGformer not only achieves strong forecasting accuracy but also provides interpretable insights into the underlying variable interactions. IV-E Sensitivity Analysis Table I shows the lowest MSE on the Weather dataset is achieved at τ=0.5τ=0.5. Performance degrades at extreme temperatures: lower values (e.g., τ=0.1τ=0.1) over-sparsify the graph and filter out useful dependencies, while higher values (τ≥1.0τ≥ 1.0) reintroduce noise. This confirms the necessity of learning a balanced, sparse graph. Although τ=0.5τ=0.5 is a robust default, the optimal τ varies with dataset characteristics (e.g., dimensionality) and is thus treated as a tunable hyperparameter in practice. TABLE I: Sensitivity analysis of Gumbel-Softmax temperature (τ) on the Weather dataset (Avg. MSE). Temperature (τ) 0.1 0.5 1.0 2.0 5.0 MSE 0.173 0.162 0.168 0.182 0.187 MAE 0.211 0.209 0.224 0.231 0.232 V Conclusion We propose TFGformer, a novel framework for multivariate time series forecasting. It features a Time-Frequency Graph module that uses STFT and a learnable Mahalanobis distance to build sparse, dynamic dependency graphs for guiding attention and filtering noise, alongside an MLP-based Covariate Fusion Module to integrate historical and future contexts. Experiments on seven benchmarks demonstrate TFGformer achieves superior performance on six datasets. Future work will extend this architecture to anomaly detection, imputation, and large-scale foundation models. References [1] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30. [2] Dosovitskiy A. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929, 2020. [3] H. Hewamalage, C. Bergmeir, and K. Bandara, “Recurrent neural networks for time series forecasting: Current status and future directions,” International Journal of Forecasting, vol. 37, no. 1, p. 388–427, 2021. [4] Nie Y. A Time Series is Worth 64Words: Long-term Forecasting with Transformers[J]. arXiv preprint arXiv:2211.14730, 2022 [5] Liu Y, Hu T, Zhang H, et al. itransformer: Inverted transformers are effective for time series forecasting[J]. arXiv preprint arXiv:2310.06625, 2023. [6] Liu Y, Qin G, Huang X, et al. Timer-xl: Long-context transformers for unified time series forecasting[J]. arXiv preprint arXiv:2410.04803, 2024. [7] Das A, Kong W, Leach A, et al. Long-term forecasting with tide: Time-series dense encoder[J]. arXiv preprint arXiv:2304.08424, 2023. [8] Arango S P, Mercado P, Kapoor S, et al. ChronosX: Adapting pretrained time series models with exogenous variables[J]. arXiv preprint arXiv:2503.12107, 2025. [9] Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(12): 11106-11115. [10] Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting?[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9): 11121-11128. [11] Zhang Y, Yan J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting[C]//The eleventh international conference on learning representations. 2023. [12] Zhou T, Ma Z, Wen Q, et al. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting[C]//International conference on machine learning. PMLR, 2022: 27268-27286. [13] Wu H, Hu T, Liu Y, et al. Timesnet: Temporal 2d-variation modeling for general time series analysis[J]. arXiv preprint arXiv:2210.02186, 2022. [14] Kim T, Kim J, Tae Y, et al. Reversible instance normalization for accurate time-series forecasting against distribution shift[C]//International conference on learning representations. 2021. [15] Qiu X, Wu X, Lin Y, et al. Duet: Dual clustering enhanced multivariate time series forecasting[C]//Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 2025: 1185-1196. [16] Jang E, Gu S, Poole B. Categorical reparameterization with gumbel-softmax[J]. arXiv preprint arXiv:1611.01144, 2016. [17] Jin G, Liang Y, Fang Y, et al. Spatio-temporal graph neural networks for predictive learning in urban computing: A survey[J]. IEEE transactions on knowledge and data engineering, 2023, 36(10): 5388-5408.