Paper deep dive
TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting
Yangze Li, Zhengyang Zhou, Qihe Huang, Pengkun Wang, Binwu Wang, Yang Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Long-term time series forecasting requires temporal dependency modeling that remains stable and generalizable under temporal distribution shifts. However, real-world time series are often noisy and non-stationary, making deep learning models prone to fitting incidental temporal interactions in the training data. A key question is whether there exist temporal relationship structures that recur across different temporal positions and variables. We analyze temporal relationship matrices on commonly used time series datasets and find that samples from different temporal positions and different variables exhibit shared dominant temporal relationship structures. This indicates that temporal relationship structures contain patterns that recur stably across samples and variables. To learn such shared structures, we propose \textbf{T}emporal \textbf{Struct}ure Network (\textbf{TStruct}), a simple yet effective network for directly learning shared and generalizable temporal dependencies. Specifically, TStruct directly parameterizes a set of shared temporal dependency matrices and generates normalized combination weights using dynamic sample information and static variable priors. These weights represent a soft allocation of each sample-variable pair over shared temporal dependency structures, enabling the model to generate adaptive temporal dependency matrices under shared structural constraints, rather than fitting incidental interactions in a fully unconstrained manner. Notably, this simple structure-aware design consistently outperforms several strong baselines across all datasets, achieving superior accuracy while maintaining high computational efficiency.
Tags
Links
- Source: https://arxiv.org/abs/2610.17864v1
- Canonical: https://arxiv.org/abs/2610.17864v1
Trouble viewing inline? Open PDF directly →
Full Text
93,950 characters extracted from source content.
Expand or collapse full text
TStruct: Learning Shared Temporal Structures for Long-term Time-Series Forecasting Yangze Li 1 , Zhengyang Zhou 1,2,B , Qihe Huang 1 , Pengkun Wang 1,2 , Binwu Wang 1,2 , Yang Wang 1,2,B 1 University of Science and Technology of China (USTC), Hefei, China 2 Suzhou Institute for Advanced Research, USTC, Suzhou, China liyangze@mail.ustc.edu.cn Abstract Long-term time series forecasting requires temporal dependency modeling that remains stable and generalizable under temporal distribution shifts. However, real-world time series are often noisy and non-stationary, making deep learning models prone to fitting incidental temporal interactions in the training data. A key question is whether there exist temporal relationship structures that recur across different temporal positions and variables. We analyze temporal relation- ship matrices on commonly used time series datasets and find that samples from different temporal positions and different variables exhibit shared dominant tem- poral relationship structures. This indicates that temporal relationship structures contain patterns that recur stably across samples and variables. To learn such shared structures, we propose Temporal Structure Network (TStruct), a sim- ple yet effective network for directly learning shared and generalizable temporal dependencies. Specifically, TStruct directly parameterizes a set of shared tem- poral dependency matrices and generates normalized combination weights using dynamic sample information and static variable priors. These weights represent a soft allocation of each sample-variable pair over shared temporal dependency structures, enabling the model to generate adaptive temporal dependency matrices under shared structural constraints, rather than fitting incidental interactions in a fully unconstrained manner. Notably, this simple structure-aware design consis- tently outperforms several strong baselines across all datasets, achieving superior accuracy while maintaining high computational efficiency. Our code is available at https://github.com/yangzeli/TStruct/. 1 Introduction Long-term time series forecasting (LTSF) has widespread applications in real-world scenarios such as energy scheduling, traffic management, weather forecasting, and financial risk control [46,39,44,28]. The goal of LTSF is to accurately model future dynamics over a long prediction horizon. The challenge lies not only in fitting historical observations but also in learning temporal dependency structures that remain effective across future time distributions. However, real-world time series data often have low signal-to-noise ratios (SNR), non-stationarity, and temporal distribution shifts, and local fluctuations or incidental interactions in the training data may not generalize well for future predictions [8,7,38,37]. Therefore, a central question in LTSF is: how can we identify and model temporal dependency structures that remain stable across shifting temporal distributions? B Yang Wang and Zhengyang Zhou are corresponding authors. Preprint. arXiv:2610.17864v1 [cs.LG] 13 Oct 2026 (b) Shared Structures Across Samples (c) Shared Structures Across Variables (a) Patch-level Temporal Relationship Matrix Sample 1 Sample 2 Variable 1 Variable 2 Variable C Normalized Singular Value Normalized Singular Value ... Singular Value IndexSingular Value Index Spectral Analysis Spectral Analysis Sample B ... fixing a variable fixing a sample Representation TStruct VS Complexity Generalization Complexity Generalization Patch N Patch 1Patch 2... Patch and Normalize Patch-level Gram matrix Samples Attention Across samples Across variables (d) Spectral Concentration with Null Controls Vectorization and Stacking Vectorization and Stacking ETTh1 5.59 2 . 8 0 2.80 2 . 6 1 2.61 ETTh2 5.73 2 . 6 8 2.68 2 . 6 1 2.61 ETTm1 4.74 2 . 4 1 2.41 2 . 6 1 2.61 ETTm2 4.27 2 . 4 3 2.43 2 . 6 1 2.61 Weather 5.92 2 . 4 4 2.44 2 . 6 1 2.61 Electricity 90.00 8.04 2.61 Traffic 64.29 6.12 2.61 Real data Patch-shuffled IID Gaussian ETTh1 1.75 1 . 1 7 1.17 1 . 0 0 1.00 ETTh2 1.40 1 . 0 0 1.00 1 . 0 0 1.00 ETTm1 1.40 1 . 0 0 1.00 1 . 0 0 1.00 ETTm2 1.40 1 . 0 0 1.00 1 . 0 0 1.00 Weather 2.33 1 . 1 1 1.11 1 . 1 1 1.11 Electricity 64.20 12.84 1.66 Traffic 287.33 8.62 2.90 Real data Patch-shuffled IID Gaussian Figure 1: Patch-level temporal relationship structures. (a) Construction of patch-level relationship matrices and qualitative comparison between attention-based and TStruct-based temporal dependency modeling. (b,c) Concentrated singular value spectra across samples and variables. (d) Higher spectral concentration in real data than in patch-shuffled and IID Gaussian controls supports reusable temporal structures. In recent years, deep learning methods have driven significant progress in long-term time series forecasting [46,39,47,44,28,45,11,10,17,1,18,33,24,12,27,14,26]. One category of methods implicitly models temporal dependencies through model architectures. For example, recurrent models pass historical information via hidden states [22,2], convolutional and TCN models use shared kernels to extract local or multi-scale temporal patterns [23,35,6], Transformer-based models estimate pairwise interactions between time tokens explicitly using attention mechanisms [28,45,4,29], while Linear/MLP methods achieve efficient prediction by mixing time or feature dimensions [44,43,36,20,13]. Despite their strengths in expressive power and efficiency, these methods still face a trade-off between adaptive temporal dependency modeling and forecasting generalization. Highly flexible interaction mechanisms may fit incidental relationships in low-SNR data, while overly fixed transformations struggle to capture heterogeneous temporal dependencies across different samples and variables [18,17,37]. A recent line of work directly parameterizes stable temporal patterns, most notably periodicity, through global temporal priors, suggesting that certain temporal structures need not be entirely learned implicitly by generic neural architectures [19,21,42,25,3]. However, such explicit parameterization has mainly focused on periodic or time-indexed patterns in the input sequence, leaving open whether temporal dependency structures themselves also contain shared and explicitly learnable patterns. To investigate whether temporal dependencies contain shared structures, we analyze patch-level temporal relationship matrices across commonly used time series datasets. As shown in Figure 1(a), for each sample-variable sequence, we divide the input window into temporal patches and construct a matrix that captures the relationships between relative patch positions. This representation shifts the analysis from raw value fluctuations to relationship patterns among relative patch positions, allowing us to examine whether these patterns recur across samples and variables. We then collect these matrices separately across samples from different temporal positions and across variables for spectral analysis. Figure 1(b,c) shows clear spectral concentration in both settings. This consistent spectral concentration indicates the presence of shared dominant temporal relationship patterns across samples and variables. Figure 1(d) further examines these patterns using patch-shuffled and IID Gaussian controls under the same analysis pipeline. Independent patch shuffling preserves each matrix’s relationship values and spectrum while disrupting the alignment of relative patch positions across samples and variables. The resulting decrease in spectral concentration indicates that shared relationship patterns recur at corresponding relative temporal positions. Compared with IID Gaussian inputs, real data also exhibit stronger concentration, indicating shared regularities beyond the random-input reference. Further details are provided in Appendix A. These observations motivate learning reusable temporal structures and adapting their contributions to each sample and variable. We therefore propose the Temporal Structure Network (TStruct). Specifically, TStruct directly parameterizes a set of shared temporal dependency matrices and uses 2 dynamic sample information and static variable priors to generate normalized combination weights. These weights serve as a soft allocation of each sample-variable pair over the shared dependency structures, combining the shared matrices into a sample- and variable-specific temporal dependency matrix. In this way, TStruct models heterogeneous temporal dependencies under the constraint of shared structures, balancing generalizable structure and adaptive flexibility. In addition, we introduce the Parameter-efficient Channel Adaptation Head (PCAHead), which utilizes a shared low-rank weight generation mechanism to dynamically produce specific prediction heads for each variable. The contributions of this paper are summarized as follows: • We reveal that patch-level temporal relationship matrices in real world time series contain dominant structures shared across samples from different temporal positions and across variables. •We propose TStruct, a concise and effective shared-adaptive temporal dependency modeling method that learns shared temporal dependency matrices and uses dynamic sample informa- tion and static variable priors to construct sample- and variable-specific dependencies. • We conduct extensive experiments to validate the effectiveness of TStruct. The results demonstrate that this simple design consistently outperforms several strong baselines on standard long-term forecasting benchmarks and remains effective under diverse generaliza- tion and robustness evaluations, while also exhibiting high computational efficiency. 2 Related Work 2.1 Temporal Dependency Modeling for Time Series Forecasting Temporal dependency modeling aims to characterize how information interacts across different temporal positions, and is a central problem in long-term time series forecasting. In recent years, Transformer-based and Linear/MLP-based methods have received increasing attention due to their strong performance in long-range dependency modeling and efficient forecasting. Transformer-based methods, such as Informer, Autoformer, PatchTST, Crossformer, and Pathformer [46,39,28,45,4], usually use attention mechanisms to dynamically estimate pairwise interactions among temporal tokens or patches for each input, thereby modeling long-range temporal dependencies. Their highly flexible input-dependent interactions provide strong adaptivity, but may also capture sample-specific incidental relations in low-SNR time series. Linear/MLP-based methods, such as DLinear, TSMixer, FiTS, SparseTSF, and TimeBase [44,5,43,20,13], achieve simple and efficient forecasting through shared temporal or feature-wise mixing. These methods demonstrate the strong competitiveness of fixed shared transformations in LTSF, but their adaptability to heterogeneous temporal dependencies remains limited. In contrast, TStruct neither relies on a single fixed mixing rule nor freely estimates interactions for each input. Instead, it generates normalized information aggregation weights through shared temporal dependency structures and sample-variable-conditioned weights, enabling adaptive temporal dependency modeling under shared structural constraints. 2.2 Explicit Temporal Structure Parameterization Recent studies attempt to explicitly parameterize stable structures in time series, rather than relying entirely on generic neural networks to learn them implicitly from data. CycleNet, TQNet, TimeEmb, MOFO, and GTR [19,21,42,25,3] mostly model recurring periodic patterns in data through global learnable periodic representations. These studies suggest that directly parameterizing reusable temporal structures can help improve the generalization ability of time series forecasting. However, existing methods mainly focus on periodic patterns in the input sequence or temporal representations. Temporal dependency structures can contain periodicity, but are not equivalent to periodicity itself; they may also characterize more general relationships among relative temporal positions and patch- level interaction patterns [9,28,40]. Unlike these methods, TStruct shifts the explicit parameterization target from input-level periodic patterns to temporal dependency structures themselves. 3 3 Methodology Problem Definition.The problem of LTSF can be formulated as follows.LetX = x t−L+1 ,..., x t ∈ R L×C denote the sequence of historical observations, whereLrepresents the lookback window length andCdenotes the number of variables. At each time stept, the observa- tion vector is defined asx t =x (1) t ,...,x (C) t , wherex (c) t represents the value of thec-th variable. Our objective is to learn a mapping functionF θ parameterized byθto predict the future values for the nextHtime steps, denoted asY =x t+1 ,..., x t+H ∈ R H×C . In this work, we focus on how to learn temporal dependency structures withinF θ that remain stable and generalizable across different samples and variables. 3.1 Overview As shown in Figure 2, TStruct contains three components. Following RevIN [15], TStruct applies instance-wise normalization before encoding and restores the original scale after prediction. Patch Embedding first converts each univariate sequence into patch-level representations, enabling local pattern extraction and reducing the length of temporal modeling [28,41]. The Temporal Structure Encoder (TSEncoder) then learns shared temporal dependency structures and adaptively combines them using dynamic sample information and static variable priors. The resulting dependency scores are normalized into patch-level aggregation weights, allowing the model to capture sample- and variable-specific temporal dependencies under shared structural constraints. Finally, PCAHead provides a parameter-efficient prediction head that accounts for variable heterogeneity by generating variable-aware forecasting projections. Softmax (a) Temporal Structure Mixer Mean MatMul Softmax MatMul ... Temporal Dependency Matrix Router Channel Priors Sample Dynamics ... Temporal Structure Patch Embedding Instance Norm. PCAHead Instance DeNorm. Temporal Structure Mixer MLP Output Input TStruct TSEncoder ... ... (b) PCAHead ... ... Softmax MatMul Channel Specific Channel Shared Input Output Input Output Figure 2: Overall architecture of TStruct. (a) The Temporal Structure Mixer (TSM) generates sample- and variable-specific temporal dependencies by adaptively combining shared temporal dependency structures with dynamic sample information and static variable priors. (b) The PCAHead handles variable heterogeneity by generating variable-aware prediction weights in a parameter-efficient manner. 3.2 Patch Embedding Patch-based representation has been widely used to capture local temporal patterns and reduce the effective sequence length [28]. Given the input sequence of thec-th variablex (c) ∈ R L , we first segment it intoNnon-overlapping patchesx (c) p ∈ R N×P , wherePis the patch length and N = L/Pis the number of patches. For simplicity, we assume thatLis divisible byP. Subsequently, a learnable linear projectionW emb ∈ R P×D , shared across all variables, maps these patches into a D-dimensional latent representation: x (c) p = Unfold(x (c) ), E (c) = x (c) p W emb ,(1) 4 whereE (c) ∈ R N×D denotes the patch-level representation of thec-th variable. This representation provides the basis for TSEncoder to model temporal dependencies among relative patch positions. 3.3 Temporal Structure Encoder The TSEncoder is designed to model patch-level temporal dependencies in a shared yet adaptive manner. Given the patch-level representationE∈ R B×C×N×D , whereB,C,N, andDdenote the batch size, number of variables, number of patches, and hidden dimension, respectively, TSEncoder learns temporal dependencies among theNrelative patch positions. We first parameterize a set of globally shared temporal dependency structures: B =B k K k=1 , B k ∈ R N×N ,(2) whereKis the number of shared temporal structures. EachB k represents a learnable patch-level dependency score pattern. We initialize the vectorized structure matrices orthogonally, providing distinct initial directions with unit Frobenius norm. This controls the initial scale of the dependency scores before softmax normalization. The importance of initialization is also consistent with findings on learning dynamics in deep linear networks [32]. A detailed initialization study is provided in Appendix C.2. These shared structures are then combined using sample-variable-specific weights. Specifically, we introduce a learnable static variable priorQ static ∈ R C×K , which captures variable-specific temporal preferences. A complementary dynamic query is generated by averaging each sample-variable representation along the patch dimension and passing the result through a lightweight router: Q dynamic = Router (Mean N (E)),(3) whereQ dynamic ∈ R B×C×K . The static and dynamic queries are then fused to obtain normalized combination coefficients: α b,c = Softmax Q (c) static + Q (b,c) dynamic ,(4) whereα b,c ∈ R K and the softmax is applied over theKshared structures. The coefficients are nonnegative and sum to one, so the resulting dependency score matrix is a convex combination of the shared bases. For each sample-variable pair, we compose a temporal dependency score matrix from the shared structures: S b,c = K X k=1 α b,c,k B k , S b,c ∈ R N×N .(5) The composed scores are converted into normalized temporal aggregation weights: A b,c = Softmax row (S b,c ).(6) Each row ofA b,c assigns nonnegative weights summing to one to the source patches. Temporal aggregation therefore forms weighted averages of projected historical features, constraining the operator to select and combine information without arbitrary signed amplification. Finally, the normalized temporal dependency matrix is used to aggregate patch-level features: e E b,c = A b,c (E b,c W proj ),(7) whereW proj ∈ R D×D is a learnable feature projection. The aggregated representation is further processed by a feed-forward network with residual connections: Z b,c = e E b,c + MLP e E b,c .(8) Together, the shared bases and normalized aggregation weights restrict temporal mixing to a common family of dependency patterns learned across samples and variables. This provides an inductive bias toward stable, reusable temporal structures, while allowing their contributions to be adjusted to the input context. 5 3.4 Parameter-efficient Channel Adaptation Head PCAHead maps the encoded features to the prediction horizon while balancing variable heterogeneity and parameter efficiency. A fully shared prediction head is efficient but may be insufficient for variable-specific forecasting patterns, whereas assigning an independent head to each variable leads to a parameter cost proportional toC. PCAHead therefore adopts a shared base projection with variable-specific low-rank residual adaptation. Given the encoder output, we flatten the patch and feature dimensions intoZ∈ R B×C×D in , where D in = ND. A shared base projection gives Y base = ZW base , W base ∈ R D in ×H .(9) For variable-specific adaptation, we define shared low-rank projection banksU k ∈ R D in ×r and V k ∈ R r×H , wherer ≪ D in . For each variablec, PCAHead learns normalized coefficients α c = Softmax(s (c) ) and assembles W (c) down = K X k=1 α c,k U k , W (c) up = K X k=1 α c,k V k .(10) The final prediction is ˆ Y (c) = Y (c) base + Z (c) W (c) down W (c) up .(11) This reduces the head complexity fromO(CD in H)toO(D in H + Kr(D in + H) + CK). We zero-initialize the up-projection bank so that the residual branch starts from zero, allowing the model to first rely on the shared base prediction and then gradually learn variable-specific corrections. 4 Experiment 4.1 Experiment Setup Following prior studies, we use Mean Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics. All methods are evaluated under prediction lengthsF = 96, 192, 336, 720. The datasets and baseline models used for evaluation are summarized below. Dataset statistics are provided in Appendix A.1, while implementation details and hyperparameter settings are provided in Appendix B. Datasets. We evaluate TStruct on seven widely used long-term forecasting benchmarks, including Weather, Traffic, Electricity [40], and four ETT subsets [46]. These datasets cover different do- mains, variable scales, and temporal patterns, providing a comprehensive evaluation of forecasting performance and generalization ability. Baselines. We compare TStruct with eight strong and recent forecasting baselines, covering different temporal dependency modeling paradigms. Specifically, PatchTST [28] and PhaseFormer [29] represent Transformer-based methods; DLinear [44] represents simple linear forecasting methods; CycleNet [19], TQNet [21], TimeEmb [42], and GTR [3] explicitly model temporal patterns or periodic priors; and DUET [31] is included as a recent strong forecasting baseline. For all baselines, we report the best results reported in their papers or accompanying code repositories, with the corresponding experimental configurations detailed in Appendix B. 4.2 Main results Table 1 reports the average results over all prediction horizons, with detailed comparisons in Table 10. TStruct achieves the best average MSE on all seven benchmarks and the best average MAE on six, ranking first in 13 of the 14 dataset-metric comparisons. These results demonstrate the effectiveness of shared-adaptive temporal dependency modeling against strong Transformer-based, Linear-based, and explicit temporal-pattern modeling baselines. On ETT and Weather, TStruct shows clear advantages over existing methods. These datasets often contain non-stationary temporal dynamics and distribution variations, where overly flexible models may fit incidental temporal interactions and overly fixed transformations may be insufficiently 6 Table 1: Average multivariate time series forecasting results over all prediction horizons. The best results are marked in bold red, and the second-best results are marked inblueunderline. Methods TStruct (ours) DUET [2025] PhaseFormer [2026] GTR [2026] TimeEmb [2025] TQNet [2025] CycleNet [2024] PatchTST [2023] DLinear [2023] DatasetsMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE ETTh10.3900.4150.398 0.4190.4030.4150.415 0.4190.418 0.4330.420 0.4280.415 0.4260.417 0.4310.425 0.439 ETTh20.3310.3810.3340.3830.347 0.3890.346 0.3940.353 0.3960.358 0.3990.345 0.3940.351 0.3950.470 0.468 ETTm10.3370.3670.3380.3690.347 0.3740.345 0.3770.345 0.3760.353 0.3880.352 0.3780.349 0.3810.356 0.378 ETTm20.2430.3040.2480.3070.251 0.3140.255 0.3150.248 0.3080.258 0.3180.249 0.3110.256 0.3140.259 0.324 Weather0.2130.2490.2180.2520.223 0.2610.227 0.2640.218 0.2570.230 0.2660.237 0.2770.224 0.2620.242 0.293 Electricity0.1510.2440.1570.2480.161 0.2500.160 0.2540.158 0.2530.162 0.2560.157 0.2500.162 0.2540.167 0.264 Traffic0.3810.2520.393 0.2560.3870.2500.397 0.2740.399 0.2720.404 0.2820.406 0.2800.396 0.2660.418 0.287 1st Count760002000000000000 Table 2: Ablation study on the main components of TStruct. Results are averaged over all prediction horizons. Detailed results for each prediction horizon are provided in Table 11. DatasetETTh1ETTh2ETTm1ETTm2WeatherElectricityTraffic MetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE ①0.4250.4440.3500.3950.3760.3910.2590.3180.2290.2650.1620.2540.4030.270 ②0.4030.4240.3410.3840.3470.3720.2480.3080.2180.2540.1530.2460.3820.253 ③0.3940.4170.3390.3860.3410.3680.2480.3070.2150.2510.1520.2450.3820.253 TStruct0.3900.4160.3310.3810.3370.3670.2430.3030.2130.2490.1510.2440.3810.252 adaptive. The consistent improvements on these datasets indicate that TStruct can better capture stable temporal dependency structures while preserving sample-specific adaptability. On Electricity and Traffic, Notably, without explicit cross-variable interactions, TStruct achieves the best average MSE on both datasets, with the best average MAE on Electricity and the second-best on Traffic. This result is consistent with our spectral analysis in Appendix A.2: Electricity and Traffic exhibit stronger spectral concentration in patch-level temporal relationship matrices across both samples and variables, suggesting a higher degree of shared temporal dependency structures. Such structures can be effectively captured by TStruct through shared temporal dependency matrices and sample-variable-conditioned adaptation. 4.3 Ablation Study We evaluate the effectiveness of three key components in TStruct. The full model is compared against three variants:①TSM→MHA replaces the Temporal Structure Mixer with multi-head attention; ②TSM→Linear replaces it with a shared linear temporal mixer; and③PCAHead→Linear replaces PCAHead with a shared linear forecasting head. All TStruct variants in the ablation study are trained with the same objective and training protocol. The summary of these ablation results is reported in Table 2. On ETT and Weather, TStruct clearly outperforms both①and②on these datasets. These datasets exhibit stronger temporal distribution variations, where forecasting requires both shared structures for generalization and adaptive modeling for sample- and variable-specific heterogeneity. MHA provides flexible input-dependent interactions but may fit incidental relations, while a shared linear mixer is stable but insufficiently adaptive. The superior performance of TStruct indicates that shared-adaptive temporal dependency modeling is especially beneficial in such settings. On Electricity and Traffic, The gap between TStruct and②becomes smaller on these datasets, while TStruct still clearly outperforms①. This is consistent with our spectral analysis in Appendix A: Electricity and Traffic exhibit stronger cross-sample and cross-variable spectral concentration in patch-level temporal relationship matrices, suggesting more pronounced shared temporal dependency 7 20%40%60%80%100% Timestamp-ordered Training Ratio 0.40 0.44 0.48 0.52 0.56 MSE MHA Linear TSM (a) Earliest samples. 20%40%60%80%100% Random Training Samples Ratio (%) 0.40 0.42 0.44 0.46 0.48 0.50 0.52 MSE MHA Linear TSM (b) Random samples. Figure 3: Generalization analysis on ETTh1. (a) Models are trained with the earliest timestamp- ordered training samples, evaluating temporal distribution shift. (b) Models are trained with randomly sampled training instances, evaluating sample efficiency. structures. Therefore, simple shared temporal mixing is already highly competitive on these datasets, whereas fully input-dependent attention remains less stable. Finally, replacing PCAHead with a shared linear head leads to moderate but consistent degradation, showing that variable-aware prediction further improves performance under variable heterogeneity. 4.4 Generalization Analysis To evaluate generalization, we conduct two reduced-training experiments on ETTh1. We compare the proposed TSM with two variants that replace TSM with Linear or MHA, keeping all other components and experimental settings identical. The earliest sample setting evaluates temporal distribution shift, while random sample setting evaluates sample efficiency. Temporal Distribution Shift. As shown in Figure 3(a), models are trained using the earliest 20%, 40%, 60%, 80%, 100%of timestamp-ordered training samples, while validation and test sets remain unchanged. MHA consistently performs worse than Linear and TSM, indicating that fully input-dependent attention is more sensitive to temporally biased training data. TSM is close to Linear when only the earliest20%samples are used, but clearly outperforms Linear from40%onward, showing that shared-adaptive temporal dependency modeling improves generalization once sufficient shared structures can be estimated. Sample Efficiency. As shown in Figure 3(b), TSM achieves the best performance even with only20% randomly sampled training data and quickly converges around40%. Notably, TSM trained with40% data already outperforms MHA and Linear trained with the full training set. This suggests that shared temporal structures improve data efficiency by reducing the need to relearn temporal interactions from individual samples. 4.5 Effect of Look-back Length 4896192336528720 Look-back Length 0.40 0.42 0.44 0.46 MSE CycleNet TQNet TimeEmb GTR TStruct Figure 4: Effect of look-back length on ETTh1. To examine how different methods utilize historical context, we vary the look-back length from 48 to 720 and compare TStruct with explicit temporal-pattern parameterization methods, including CycleNet [19], TQNet [21], TimeEmb [42], and GTR [3]. These baselines mainly rely on global periodic representa- tions, which can remain effective even with short input windows because the periodic priors are largely input- independent. However, as the look-back length in- creases, their improvements become limited and may even degrade when overly long histories introduce noisy or less relevant observations. 8 Var 14 Structure #2Structure #1 Var 16 (a) Learned static prior Structure #2Structure #1 (c) Representative learned structures Var 16: periodic ACF Var 14: fast-decaying ACF (b) ACF of selected variables Figure 5: Interpretability analysis on the Weather dataset. Different variables exhibit different ACF patterns and assign different weights to the learned shared temporal structures throughQ static . The representative learned structures show distinct patch-level dependency patterns, including broader recurrent relations and local diagonal-dominant continuity. In contrast, TStruct consistently benefits from longer look-back windows. This suggests that TStruct does not merely rely on fixed global periodic patterns; instead, it explicitly models patch-level temporal dependency structures, which become richer as more historical context is available. This further indicates that patch-level temporal dependency structures can provide a more extensible modeling target than fixed global periodic priors when longer historical context is available. 4.6 Interpretability of Learned Temporal Structures To examine whether the learned structures are interpretable, we visualize the learned static prior, the autocorrelation functions (ACF) of representative variables, and the corresponding temporal dependency structures on the Weather dataset. As shown in Figure 5, Var 16 exhibits clear periodic autocorrelation, while Var 14 shows rapidly decaying autocorrelation, indicating that these two variables have different temporal characteristics. The learned static priorQ static assigns different preferences to these variables over the shared structures. Specifically, Var 16 places higher weight on Structure #1, which captures broader recurrent relations among patches, whereas Var 14 assigns higher weight to Structure #2, which shows strong diagonal dominance and reflects local temporal continuity. This suggests thatQ static captures variable-specific temporal preferences rather than using the same dependency pattern for all variables. Overall, these observations indicate that TStruct learns reusable temporal dependency structures and adapts them to variables with different temporal characteristics through the static prior. For additional visualizations of learned structure matrices on other datasets, please refer to Appendix D. 5 Conclusion In this paper, we studied LTSF from the perspective of temporal dependency structures. By analyzing patch-level temporal relationship matrices, we found that temporal dependencies contain shared dominant structures across samples and variables. This observation suggests that stable temporal relationships can be explicitly modeled as reusable structures rather than treated as fully input-specific interactions. Based on this finding, we proposed TStruct, which learns shared temporal dependency structures and generates sample-variable-specific dependency scores within the shared structural space. These scores are further converted into normalized soft allocation weights for temporal information aggregation, enabling adaptive dependency modeling under shared structural constraints. Experiments show that TStruct consistently improves forecasting accuracy over strong baselines. 9 Ablation and generalization analyses further verify that the gains come from shared-adaptive temporal dependency modeling, while look-back length and interpretability analyses show that TStruct can exploit longer contexts and learn meaningful temporal structures. References [1] Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shub- ham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, and Bernie Wang. Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=gerNCVqqtR. [2]Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xLSTM: Ex- tended long short-term memory. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=ARAxPPIAhq. [3]Fanpu Cao, Lu Dai, Jindong Han, and Hui Xiong. Enhancing multivariate time series forecasting with global temporal retrieval. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=QUJBPSfyui. [4]Peng Chen, Yingying ZHANG, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=lJkOCMP2aW. [5]Si-An Chen, Chun-Liang Li, Sercan O Arik, Nathanael Christian Yoder, and Tomas Pfister. TSMixer: An all-MLP architecture for time series forecast-ing. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URLhttps://openreview.net/forum?id= wbpxTuXgm0. [6]Luo donghao and wang xue. ModernTCN: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=vpJMJerXHU. [7]Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pages 424–438, 1969. [8] C.W.J. Granger and P. Newbold. Spurious regressions in econometrics. Journal of Econometrics, 2(2):111–120, 1974. ISSN 0304-4076. doi: https://doi.org/10.1016/0304-4076(74)90034-7. URL https://w.sciencedirect.com/science/article/pii/0304407674900347. [9]James D Hamilton. Time series analysis, volume 2. Princeton Princeton University Press, 1994. [10]Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. IEEE Trans. Knowl. Data Eng., 36(11):7129–7142, 2024. [11]Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. CrossGNN: Confronting noisy multivariate time series via cross interaction refinement. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=xOzlW2vUYc. [12]Qihe Huang, Zhengyang Zhou, Yangze Li, Kuo Yang, Binwu Wang, and Yang Wang. Many minds, one goal: Time series forecasting via sub-task specialization and inter-agent cooperation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=Uon41HfqR3. [13]Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power of minimalism in efficient long-term time series forecasting. In Forty-second International Conference on Machine Learning, 2025. URLhttps://openreview.net/ forum?id=GhTdNOMfOD. 10 [14]Qihe Huang, Zhengyang Zhou, Yangze Li, Jiaming Ma, Kuo Yang, Binwu Wang, Xu Wang, and Yang Wang. STORM: Synergistic cross-scale spatio-temporal modeling for weather forecasting. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=JLF6XDnscF. [15]Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2022. URLhttps://openreview. net/forum?id=cGDAkQo1C0p. [16]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015. [17]Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping, 2026. URL https://arxiv.org/abs/2305.10721. [18]Zida Liang, Jiayi Zhu, and Weiqiang Sun. Why attention fails: The degeneration of transformers into mlps in time series forecasting, 2025. URL https://arxiv.org/abs/2509.20942. [19] Shengsheng Lin, Weiwei Lin, Xinyi HU, Wentai Wu, Ruichao Mo, and Haocheng Zhong. Cyclenet: Enhancing time series forecasting through modeling periodic patterns. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=clBiQUgj4w. [20]Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. SparseTSF: Modeling long-term time series forecasting with *1k* parameters. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=54NSHO0lFe. [21]Shengsheng Lin, Haojun Chen, Haijie Wu, Chunyun Qiu, and Weiwei Lin. Temporal query net- work for efficient multivariate time series forecasting. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=e24CueVty2. [22]Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang. Segrnn: Segment recurrent neural network for long-term time-series forecasting. IEEE Internet Things J., 13(5):9861–9871, 2026. [23]Minhao LIU, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia LAI, Lingna Ma, and Qiang Xu. SCINet: Time series modeling and forecasting with sample convolution and interaction. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URLhttps://openreview.net/forum?id= AyajSjTAzmg. [24] Peiyuan Liu, Beiliang Wu, Yifan Hu, Naiqi Li, Tao Dai, Jigang Bao, and Shu-Tao Xia. Time- bridge: Non-stationarity matters for long-term time series forecasting. In Forty-second Inter- national Conference on Machine Learning, 2025. URL https://openreview.net/forum? id=pyKO0Z5lz. [25]Jiaming Ma, Binwu Wang, Qihe Huang, Guanjun Wang, Pengkun Wang, Zhengyang Zhou, and Yang Wang. Mofo: Empowering long-term time series forecasting with periodic pattern modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=sbvLts2HqR. [26]Jiaming Ma, Guanjun Wang, Sheng Huang, Kuo Yang, Binwu Wang, Pengkun Wang, and Yang Wang. Spatiotemporal causal decoupling model for air quality forecasting. In ICASSP 2025- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025. [27]Jiaming Ma, Qihe Huang, Haofeng Ma, Guanjun Wang, Sheng Huang, Zhengyang Zhou, Pengkun Wang, Xu Wang, Binwu Wang, and Yang Wang. PHAT: Modeling period heterogeneity for multivariate time series forecasting. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=lr4RlISR6x. 11 [28]Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview.net/forum? id=Jbdc0vTOcol. [29] Yiming Niu, Jinliang Deng, and Yongxin Tong. Phaseformer: From patches to phases for efficient and effective time series forecasting. In The Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=Lk9SqMQzhX. [30]Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. TFB: towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9):2363– 2377, 2024. [31]Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. DUET: dual clustering enhanced multivariate time series forecasting. In KDD (1), pages 1185–1196. ACM, 2025. [32] Mark Rhee, Dhruva Karkada, and Jamie Simon. Deep linear networks are a surprisingly useful toy model of weight-space dynamics. Learning Mechanics, 2026. URLhttps:// learningmechanics.pub/deep-linear-nets. [33]Mingtian Tan, Mike A Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. Are language models actually useful for time series forecasting? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview. net/forum?id=DV15UbHCY1. [34]Hao Wang, Lichen Pan, Yuan Shen, Zhichao Chen, Degui Yang, Yifei Yang, Sen Zhang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. FreDF: Learning to forecast in the frequency domain. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=4A9IdSa1ul. [35] Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. MICN: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview. net/forum?id=zt53IDUR1U. [36]Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URLhttps: //openreview.net/forum?id=7oLshfEIC2. [37]Zheng Wang, Kaixuan Zhang, Wanfang Chen, Xiaonan Lu, Longyuan Li, and Tobias Schla- genhauf. Characteristic root analysis and regularization for linear time series forecasting. In The Fourteenth International Conference on Learning Representations, 2026.URL https://openreview.net/forum?id=JTtwGRACte. [38]Gerhard Widmer and Miroslav Kubat. Learning in the presence of concept drift and hidden contexts. Mach. Learn., 23(1):69–101, 1996. [39]Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=J4gRj6d5Qm. [40] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview.net/forum? id=ju_Uqw384Oq. [41]Xingjian Wu, Xiangfei Qiu, Hanyin Cheng, Zhengyu Li, Jilin Hu, Chenjuan Guo, and Bin Yang. Enhancing time series forecasting through selective representation spaces: A patch perspective. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=BirE0jYKt0. 12 [42]Mingyuan Xia, Chunxu Zhang, Zijian Zhang, Hao Miao, Qidong Liu, Yuanshao Zhu, and Bo Yang. Timeemb: A lightweight static-dynamic disentanglement framework for time series forecasting. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=sLfMvrkn6T. [43] Zhijian Xu, Ailing Zeng, and Qiang Xu. FITS: Modeling time series with $10k$ parameters. In The Twelfth International Conference on Learning Representations, 2024. URLhttps: //openreview.net/forum?id=bWcnvZ3qMb. [44]Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI, pages 11121–11128. AAAI Press, 2023. [45] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=vSVLM2j9eie. [46] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, pages 11106–11115. AAAI Press, 2021. [47] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 27268–27286. PMLR, 2022. 13 Appendix Catalogue 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 Temporal Dependency Modeling for Time Series Forecasting . . . . 3 2.2 Explicit Temporal Structure Parameterization . . . . . . . . . . . . . . . . . . 3 3. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2 Patch Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3 Temporal Structure Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.4 Parameter-efficient Channel Adaptation Head . . . . . . . . . . . . . . . . . . 6 4. Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.3 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.4 Generalization Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.5 Effect of Look-back Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.6 Interpretability of Learned Temporal Structures . . . . . . . . . . . . . . . . 9 5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 A. Dataset and Temporal Structure Analysis . . . . . . . . . . . . . . . . . . . . . 15 B. Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 C. Additional Experimental Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 D. Additional Interpretability Visualizations . . . . . . . . . . . . . . . . . . . . . 24 E. Detailed Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 F. Limitations and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 14 A Dataset and Temporal Structure Analysis A.1 Dataset Details We evaluate TStruct on seven widely used long-term forecasting benchmarks, including four ETT sub- sets, Weather, Electricity, and Traffic. The statistics of these datasets are summarized in Table 3. The ETT datasets [46] contain temperature and power load records collected from electricity transformers in two regions of China from 2016 to 2018. They include two temporal granularities: hourly records, denoted as ETTh1 and ETTh2, and 15-minute records, denoted as ETTm1 and ETTm2. The Weather dataset [40] contains 21 meteorological variables recorded at 10-minute intervals in 2020, including air temperature, humidity, visibility, and other weather-related indicators. The Electricity dataset [40] consists of hourly electricity consumption records from 321 clients, measured in kilowatt-hours (kWh), and is derived from the UCI Machine Learning Repository. The Traffic dataset [40] records hourly road occupancy rates collected by 862 sensors on freeways in the San Francisco Bay Area, reflecting large-scale traffic flow dynamics. Table 3: Detailed descriptions of datasets. DatasetVariables Timestamps Split Prediction Length Sampling Information ETTh17144006:2:296,192,336,720HourlyElectricity ETTh27144006:2:296,192,336,720HourlyElectricity ETTm17576006:2:296,192,336,72015minElectricity ETTm27576006:2:296,192,336,72015minElectricity Weather21526967:1:296,192,336,72010minWeather ECL 321263047:1:296,192,336,720HourlyElectricity Traffic862175447:1:296,192,336,720HourlyTransport A.2 Temporal Structure Analysis Patch-level temporal relationship matrix.Real-world time series often have low-SNR, and tem- poral interactions observed in individual windows may reflect incidental fluctuations. We hypothesize that these observations nevertheless contain relatively stable temporal structures associated with the underlying evolution, which can be extracted from recurring relationships across windows and variables. To examine this hypothesis, we construct patch-level temporal relationship matrices that describe relationships among relative temporal positions. For each dataset, we first standardize each variable and randomly sampleBinput windows of lengthL. Given theb-th input window X b ∈ R L×C , we apply instance-level normalization along the temporal dimension: e X b = X b − Mean L (X b ) Std L (X b ) + ε ,(12) where Mean L (·) and Std L (·) are computed along the temporal dimension for each variable. Then, for each variablec, the normalized sequence is divided intoNnon-overlapping patches with patch length P : X p b,c = Unfold( e X (c) b )∈ R N×P ,(13) where N = L/P for non-overlapping patches, and we assume L is divisible by P . Each patch vector is further normalized by its ℓ 2 -norm: X p b,c [i] = X p b,c [i] ∥X p b,c [i]∥ 2 + ε , i = 1,...,N.(14) We then construct the patch-level temporal relationship matrix as G b,c =X p b,c X p b,c ⊤ ∈ R N×N ,(15) whereG b,c [i,j]measures the normalized relationship strength between thei-th andj-th relative patch positions. 15 Shared temporal structures across samples and variables.To examine shared temporal structures, we stack the vectorized relationship matrices across samples and across variables. For the sample- side analysis, each variablecis fixed and the matrices from allBwindows are collected; for the variable-side analysis, each sample b is fixed and the matrices from all C variables are collected: G (sample) c = [vec(G 1,c ),..., vec(G B,c )] ⊤ ∈ R B×N 2 , G (var) b = [vec(G b,1 ),..., vec(G b,C )] ⊤ ∈ R C×N 2 . (16) Before performing SVD, the rows of both matrices are normalized to compare temporal relationship patterns on a common scale: G[j] = G[j] ∥G[j]∥ 2 + ε ,(17) whereGdenotes either stacked matrix andjindexes its rows. This normalization removes differences in vector magnitude while preserving the directions of the temporal relationship representations. Letσ (sample) c,i i andσ (var) b,i i denote the ordered singular values ofG (sample) c andG (var) b , respectively. The averaged spectral energies are defined as ̄e (sample) i = 1 C C X c=1 σ (sample) c,i 2 , ̄e (var) i = 1 B B X b=1 σ (var) b,i 2 .(18) These spectra indicate the extent to which temporal relationship patterns are concentrated in a small number of directions across samples and variables, respectively. Spectral concentration metrics. We quantify spectral concentration by measuring how many singular components are required to explain 95% of the spectral energy. Given an averaged spectral energy sequence ̄e i R i=1 , the 95% energy rank is defined as r 95 = min ( r : P r i=1 ̄e i P R i=1 ̄e i ≥ 0.95 ) .(19) Here R denotes the reference dimension of the full vectorized representation: R = ( min(C,N 2 ),forG (var) b , min(B,N 2 ), forG (sample) c . (20) To make the degree of spectral concentration more intuitive, we report the rank compression factor: κ 95 = R r 95 .(21) A largerκ 95 indicates that 95% of the spectral energy is captured by a smaller fraction of the reference dimension, corresponding to stronger spectral concentration. However, spectral concentration can also arise from the Gram construction itself: symmetry introduces redundant coordinates in the vectorized representation, while patch normalization produces a common unit diagonal across matrices. A reference level is therefore needed to assess the additional concentration associated with temporal structure. To this end, we evaluate two null controls through the same analysis pipeline and compare their κ 95 values with those of the real data. Quantitative analysis. Table 4 reports the spectral concentration statistics underB = 2000, L = 720, andP = 24(N = 30), together with two null controls evaluated through the same pipeline. (1) Patch shuffle evaluates the role of temporal ordering by independently permuting patch positions for each sample-variable pair, givingG shuffle b,c = Π b,c G b,c Π ⊤ b,c . This preserves the spectrum and similarity values of each Gram matrix while disrupting the correspondence between relative patch positions across matrices. Comparing the resulting concentration with that of real data therefore assesses whether the observed concentration depends on temporal relationships recurring at aligned patch positions. (2) IID Gaussian provides a complementary reference by replacing the observed windows with independent Gaussian noise. Applying the same analysis to these inputs gives the spectral concentration that can arise without temporal dependencies, against which the additional concentration in real data can be assessed. 16 Table 4: Shuffle denotes independent patch permutations and IID denotes independent Gaussian windows. Results are averaged over repeated runs of each null control. (a) Sample side: across temporal windows DatasetPatch size PPatch number N Variables CRealShuffleIIDReal/Shuffle Real/IID ETTh1243075.592.802.612.00×2.14× ETTh2243075.732.682.612.14×2.20× ETTm1243074.742.412.611.96×1.82× ETTm2243074.272.432.611.76×1.64× Weather2430215.922.442.612.43×2.27× Electricity243032190.008.042.6111.20×34.50× Traffic243086264.296.122.6110.50×24.64× (b) Variable side: across variables within a window DatasetPatch size PPatch number N Variables CRealShuffleIIDReal/Shuffle Real/IID ETTh1243071.751.171.001.50×1.75× ETTh2243071.401.001.001.40×1.40× ETTm1243071.401.001.001.40×1.40× ETTm2243071.401.001.001.40×1.40× Weather2430212.331.111.112.11×2.11× Electricity243032164.2012.841.665.00×38.60× Traffic2430862287.338.622.9033.33×99.00× Table 4 reveals four findings: (i) Shared structures at aligned temporal positions. Across all seven datasets, Real/Shuffle is substantially greater than one on both the sample and variable sides. The reduction in concentration after shuffling supports shared temporal relationship patterns that recur at corresponding relative patch positions across windows and variables. (i) Concentration beyond the random-input reference. Real/IID is also substantially greater than one across all datasets on both sides. Temporal relationships in real time series are therefore more concentrated than those obtained from independent random inputs under the same analysis pipeline, supporting their representation through a compact set of shared patterns. (i) Larger gains in Electricity and Traffic. Electricity and Traffic exhibit substantially larger Real/Shuffle and Real/IID gains than ETT and Weather on both sides. Their temporal relationships show more pronounced concentration relative to their respective controls, with the larger Real/Shuffle gains indicating a greater dependence of this concentration on the original patch-position alignment. (iv) Shared regularities preserved under shuffling. For Electricity and Traffic, shuffled-data concentration remains well above the corresponding IID reference on both sides, whereas the two controls yield closer values for ETT and Weather. This suggests that shared regularities preserved under patch permutation coexist with reusable temporal relationships tied to the original relative positions. From temporal relationships to dependency parameterization. The above analysis provides evidence of shared temporal relationship structures across windows and variables. To connect these historical relationships with forecasting, we consider an idealized model in which historical and future patch representations are related by a shared temporal evolution operator. For a sample-variable contexti, letX i ∈ R N×d andY i ∈ R M×d denote row-normalized historical and future patch representations in a common state space, whereN = L/PandM = H/P, assuming both lengths are divisible by P . Suppose Y i = TX i +ε i ,E[ε i | X i ] = 0,T∈ R M×N ,(22) whereTis shared across the contexts under consideration andε i captures residual variation. The historical auto-relation and history–future cross-relation then satisfy G i = X i X ⊤ i ,Γ i = Y i X ⊤ i = TG i +ε i X ⊤ i ,(23) 17 and hence E[Γ i | X i ] = TG i . Motivated by the observed spectral concentration, suppose the historical relation family admits an approximate representation through a mean structure and r shared directions: G i = μ G + r X ℓ=1 β i,ℓ Q ℓ + ∆ i ,(24) where ∆ i denotes the approximation residual. It follows that Γ i = Tμ G + r X ℓ=1 β i,ℓ (TQ ℓ ) + T∆ i +ε i X ⊤ i .(25) Thus, under shared temporal evolution, the historical directionsQ ℓ give rise to shared directions TQ ℓ in the history–future relation family. When the residual terms are small, the same coefficients β i,ℓ organize both families. This provides a conditional link between reusable historical relationships and a compact description of relationships relevant to forecasting. Motivated by this connection, TStruct parameterizes temporal aggregation through shared structure matrices: A i = Softmax row K X k=1 α i,k B k ! , e Z i = A i Z i ,(26) whereZ i denotes projected historical features. Learned jointly across training observations under forecasting supervision, the shared matricesB k define a compact family of temporal aggregation operatorsA i on historical patch coordinates. This parameterization is intended to capture stable temporal dependency patterns relevant to forecasting, while the forecasting head maps the aggregated features to the future horizon. The connection motivates the shared structural parameterization of A i without requiring equality withG i orΓ i , or inheritance of their spectra. By drawing on patterns shared across observations, TStruct aims to reduce reliance on incidental relationships in noisy time series and improve forecasting generalization. These intended benefits are evaluated in Section 4.4 and Appendix C.4. B Implementation Details All experiments performed in this study are conducted on a workstation equipped with NVIDIA RTX PRO 6000 GPUs with 96GB memory. TStruct is implemented in PyTorch 2.8.0. We fix the random seed to2026for our runs. Following the standard evaluation protocol [30], we set drop_last=False for the data loader during testing to ensure that all test samples are evaluated. For the main forecasting experiments, TStruct uses a fixed input look-back length ofL = 720. We train TStruct using the Adam optimizer [16] for at most 25 epochs with an early stopping patience of 10 epochs. The best checkpoint is selected according to the validation loss. Baseline configurations. To compare the best performance of each method within the evaluated configurations, we retain the training objectives used by the respective methods and allow look-back selection where specified below. The sources and configurations of the results in Table 1 are as follows: TStruct. We report our results using the Huber+FFT objective described below and a fixed look-back length ofL = 720. DUET. We use the paper reported results obtained with the L1 objective, selecting the best performing look-back length fromL∈96, 192, 336, 512, 720for each dataset horizon setting. PhaseFormer. We use the paper reported results obtained with the Huber objective andL = 720. Other baselines. We run their released implementations with their native training objectives and select the best performing look-back length fromL ∈ 96, 192, 336, 512, 720for each dataset horizon setting. Training objective. We optimize TStruct with a hybrid objective that combines Huber loss and FFT loss [34]. Given the prediction ˆ Y and ground truth Y, the Huber loss is defined as L Huber = 1 |Ω| X i∈Ω ℓ δ (ˆy i − y i ),(27) 18 whereΩdenotes all predicted entries and we setδ = 0.1in all experiments. The element-wise Huber loss is defined as ℓ δ (e) = 1 2 e 2 ,|e|≤ δ, δ |e|− 1 2 δ , |e| > δ. (28) The FFT loss measures the discrepancy between predictions and ground truth in the frequency domain: L FFT = rFFT( ˆ Y, dim = 1)− rFFT(Y, dim = 1) 1 ,(29) whererFFT(·, dim = 1)denotes the real-valued fast Fourier transform along the temporal dimension, and∥·∥ 1 is implemented as the mean absolute value over all frequency-domain entries. The final training objective is L =L Huber + λ FFT L FFT ,(30) where λ FFT is selected from0.0, 0.1, 0.2, 0.3. Evaluation metrics. Following prior LTSF studies [46,39], we use Mean Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics: MSE = 1 |Ω| X i∈Ω (ˆy i − y i ) 2 ,MAE = 1 |Ω| X i∈Ω |ˆy i − y i |.(31) Hyperparameter search space. We tune the hyperparameters over the following search space. The patch size is selected fromP ∈ 24, 48, 72, 120, 180, 360, the number of encoder layers from1, 3, the hidden dimension fromD ∈128, 256, 512, and the number of shared temporal structures fromK ∈1, 2, 4, 5, 8, 16, 32. For regularization, the MLP dropout rate is selected from 0.1, 0.2, 0.3, 0.4, the TSM dropout rate from0.0, 0.1, 0.2, 0.3, 0.4, and the PCAhead dropout rate from0.0, 0.1, 0.2. For optimization, the batch size is selected from16, 64, 256, 512, and the learning rate η is selected from0.001, 0.0015, 0.002, 0.005, 0.015. C Additional Experimental Analyses C.1 Random Seed Stability Table 5: Stability analysis under different random seeds. Seed20242025202620272028MeanStd DatasetMetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE ETTh1 960.345 0.3800.344 0.3800.345 0.3810.344 0.3900.345 0.3800.345 0.3820.001 0.004 1920.382 0.4040.383 0.4060.383 0.4050.384 0.4060.384 0.4060.383 0.4050.001 0.001 3360.407 0.4210.405 0.4210.405 0.4200.407 0.4220.407 0.4210.406 0.4210.001 0.001 7200.424 0.4530.426 0.4530.431 0.4560.428 0.4560.424 0.4500.427 0.4540.003 0.003 Avg.0.390 0.4150.390 0.4150.391 0.4160.391 0.4190.390 0.4140.390 0.4160.001 0.002 Weather 960.137 0.1790.137 0.1790.137 0.1810.138 0.1800.138 0.1800.137 0.1800.001 0.001 1920.181 0.2250.182 0.2250.181 0.2260.180 0.2250.181 0.2250.181 0.2250.001 0.000 3360.231 0.2670.231 0.2680.230 0.2670.231 0.2690.230 0.2680.231 0.2680.001 0.001 7200.306 0.3250.305 0.3250.304 0.3240.305 0.3250.305 0.3260.305 0.3250.001 0.001 Avg.0.214 0.2490.214 0.2490.213 0.2500.214 0.2500.214 0.2500.214 0.2490.000 0.000 To examine the stability of TStruct under different random initializations and stochastic training processes, we repeat experiments with five random seeds, i.e.,2024, 2025, 2026, 2027,and2028. Table 5 reports the results on ETTh1 and Weather, including the mean and standard deviation across seeds. The results show that TStruct is highly stable across different random seeds. The standard deviation is consistently very small, mostly ranging from0.000to0.002for both MSE and MAE. This indicates that the performance of TStruct is not sensitive to random initialization or stochastic optimization, and the observed improvements are reproducible rather than caused by favorable random seeds. 19 Table 6: Sensitivity analysis under different initialization strategies. InitializationRandomOrthogonalXavierKaiming DatasetMetricMSEMAEMSEMAEMSEMAEMSEMAE ETTh1 960.366 0.3980.345 0.3800.345 0.3800.345 0.380 1920.421 0.4390.383 0.4050.383 0.4050.383 0.405 3360.426 0.4420.407 0.4200.407 0.4200.407 0.420 7200.497 0.5030.433 0.4560.434 0.4590.430 0.458 Avg.0.428 0.4460.392 0.4150.392 0.4160.391 0.416 Weather 960.142 0.1870.137 0.1810.137 0.1800.137 0.180 1920.187 0.2300.181 0.2260.182 0.2260.181 0.225 3360.247 0.2820.230 0.2670.231 0.2670.230 0.268 7200.327 0.3470.304 0.3240.304 0.3250.305 0.325 Avg.0.226 0.2620.213 0.2500.214 0.2500.213 0.250 C.2 Initialization Study We study the effect of different initialization strategies for the shared temporal structuresB. Specifi- cally, we compare Random, Orthogonal, Xavier, and Kaiming initialization, as reported in Table 6. The results show that the Random baseline yields noticeably higher forecasting errors on both ETTh1 and Weather, whereas Orthogonal, Xavier, and Kaiming achieve consistently strong and comparable performance. All initialization strategies are applied directly to theK×N×Nbasis tensor. Orthogonal initialization gives the vectorized bases mutually orthogonal directions with unit Frobenius norm, while Xavier and Kaiming scale the sampling variance according to the tensor dimensions. These strategies control the initial magnitude of the dependency scores, which are subsequently converted into aggregation weights through softmax. This provides a plausible explanation for their favorable performance: appropriately scaled initial scores moderate arbitrary preferences over patch positions, allowing temporal dependency patterns to develop through forecasting supervision. The role of initialization scale is also consistent with observations on learning dynamics in deep linear networks [32]. We use Orthogonal initialization as the default. It provides diverse initial structure directions with a common scale, reducing initial redundancy among the shared bases while achieving consistently strong empirical performance. C.3 Training Objective Comparison We investigate the effect of the training objective on forecasting performance by evaluating TStruct, DUET, and PhaseFormer under the same two objectives, Huber+FFT and plain MSE. The remaining experimental settings follow Appendix B. Table 7 shows that TStruct ranks first in MSE on all seven datasets and in MAE on six under both objectives, with PhaseFormer leading in MAE on Traffic. Huber+FFT further improves TStruct’s MSE and MAE across all seven datasets compared with plain MSE. Its strong performance under plain MSE supports the effectiveness of shared temporal structure modeling, while the hybrid objective provides additional gains. C.4 Noise Injection Robustness To evaluate robustness under local abnormal perturbations, we conduct noise injection experiments on ETTh1. For each normalized input window, whose values are standardized to approximately zero mean and unit variance, we randomly select a continuous segment and inject Gaussian noise into this segment. The noise ratio controls the length of the perturbed segment relative to the input window, while the noise strength controls the magnitude of the injected Gaussian noise. The start position of the noisy segment is randomly sampled within each input window. We compare TSM and MHA across different noise ratios and strengths, with the results summarized in Table 8. TSM achieves lower MSE and MAE throughout both experiments. As the noise ratio 20 Table 7: Forecasting performance under two training objectives withL = 720for all methods. The lowest MSE and MAE for each dataset within each objective are highlighted in bold. LossHuber+FFTPlain MSE MethodTStructDUETPhaseFormerTStructDUETPhaseFormer DatasetsMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE ETTh10.390 0.4150.430 0.4500.4170.4250.410 0.4320.425 0.4470.4310.437 ETTh20.331 0.3810.364 0.4100.3740.4010.349 0.4000.358 0.4050.3580.402 ETTm10.337 0.3670.432 0.4260.3510.3780.345 0.3730.382 0.4090.3580.384 ETTm20.243 0.3040.282 0.3420.2530.3140.246 0.3110.279 0.3380.2660.329 Weather0.213 0.2490.218 0.2520.2310.2710.217 0.2580.249 0.2930.2310.275 Electricity0.151 0.2440.158 0.2510.1590.2490.157 0.2510.158 0.2560.1610.252 Traffic0.381 0.2520.387 0.2670.3870.2490.385 0.2640.388 0.2700.3930.259 increases from0.05to0.30, its MSE ranges from0.392to0.400, remaining0.042–0.047below MHA. At a noise strength of8, the MSE gap reaches0.075, with TSM obtaining0.404compared with 0.479 for MHA. Table 8: Noise injection robustness on ETTh1 under varying noise ratios (left) and strengths (right). ∆ denotes the error reduction of TSM relative to MHA. SettingVarying noise ratioSettingVarying noise strength MethodTSMMHA∆MethodTSMMHA∆ RatioMSEMAEMSEMAEMSEMAEStrengthMSEMAEMSEMAEMSEMAE 0.050.392 0.4170.434 0.4490.042 0.03320.393 0.4170.437 0.4520.044 0.035 0.100.394 0.4180.441 0.4550.047 0.03630.394 0.4180.441 0.4550.047 0.036 0.200.397 0.4250.443 0.4570.046 0.03350.396 0.4200.434 0.4490.038 0.029 0.300.400 0.4270.442 0.4580.042 0.03180.404 0.4260.479 0.4730.075 0.047 These results support the robustness of shared temporal structure modeling under local perturbations. By aggregating temporal information through structures learned across observations, TSM provides a shared basis for processing corrupted inputs, which may help explain its consistently lower forecasting errors across the tested noise conditions. C.5 Hyperparameter Sensitivity We further analyze the sensitivity of TStruct to two key hyperparameters: the number of shared temporal structuresKand the number of patchesN. Experiments are conducted on the ETT benchmarks under different prediction horizons. Figure 6 reports the forecasting performance under different values ofK. Overall, TStruct remains stable across a wide range ofK. A very smallKmay limit the diversity of shared temporal structures, making it difficult to represent heterogeneous temporal dependencies. In contrast, an overly large Kmay introduce redundant structures and increase optimization difficulty. The results show that a moderate number of shared structures is sufficient for strong performance, and we useK = 4as the default setting in our main experiments. Figure 7 shows the sensitivity to the number of patchesN, which controls the resolution of the patch-level temporal dependency matrix. TStruct achieves robust performance within a broad middle range, especially whenN ∈ [10, 30]. WhenNis too small, the temporal relationship matrix becomes overly coarse and may underfit fine-grained temporal dependencies. WhenNis too large, each patch contains fewer time points, which may make the learned dependency structures more sensitive 21 to local noise and increase the risk of overfitting. These results indicate that TStruct is robust to reasonable patch-level resolutions while benefiting from a balanced choice of N . C.6 Efficiency and Complexity Analysis We provide an additional efficiency analysis on the Electricity dataset to evaluate the accuracy- efficiency trade-off of TStruct. Since TStruct models temporal dependencies at the patch level, we compare it with representative patch-based Transformer baselines, including Crossformer and PatchTST. All models are evaluated under the same setting with input lengthL = 720, patch size and strideP = S = 48, and hidden dimensionD = 512. We report forecasting accuracy, parameter number, multiply-accumulate operations (MACs), GPU memory usage, training time per step, and inference time per batch in Table 9. Complexity analysis.The efficiency of TStruct mainly comes from its structured temporal depen- dency modeling. In attention-based patch-level temporal mixing, the dependency score matrix is computed from input-dependent query-key interactions, which introduces a score estimation cost ofO(N 2 D), whereNis the number of patches andDis the hidden dimension. In contrast, TSM constructs temporal dependency scores by combiningKshared temporal structures, with a score construction cost ofO(N 2 K). SinceK ≪ D, TSM provides a more compact way to estimate temporal dependency scores and avoids fully input-dependent pairwise attention computation. After obtaining the normalized dependency matrix, TSM still performs patch-level feature aggregation, so the overall temporal mixing remains quadratic inN. The key difference is that TSM replaces unconstrained input-dependent attention score estimation with shared-adaptive structure composition. This makes temporal dependency modeling more structured and computationally efficient while preserving sample- and variable-specific adaptability. Empirical efficiency. As shown in Table 9, TStruct achieves the best MSE while requiring fewer parameters, lower MACs, less GPU memory, and faster training and inference than the patch-based Transformer baselines. Compared with PatchTST, TStruct reduces parameters from5.496M to 3.324M and MACs from23.134G to11.779G, while reducing MSE from0.171to0.151. Compared with Crossformer, TStruct also achieves substantially lower memory usage and faster runtime. These results show that TStruct improves forecasting accuracy with a favorable efficiency profile by replacing fully input-dependent attention with shared-adaptive temporal dependency modeling. Table 9: Efficiency comparison on the Electricity dataset. Crossformer [45] and PatchTST [28] are included as representative patch-based Transformer baselines. All models are evaluated under the same setting. We report MSE, parameters, MACs, GPU memory, training time per step, and inference time per batch. Bold indicates the best result, and underlineindicates the second-best result. ModelMSEParams (M)MACs (G)Memory (GB)Train Time (s)Infer Time (ms) Crossformer0.17136.93647.54322.9320.18475.474 PatchTST0.1715.49623.13412.8480.269326.133 TStruct0.1513.32411.7797.1430.05956.059 Improvement (↓)1.13×1.65×1.96×1.80×4.56×1.35× 22 124581632 K 0.300 0.320 0.340 MSE ETTh1 H=96 MSEMAE 124581632 K 0.340 0.360 0.380 MSE H=192 MSEMAE 124581632 K 0.350 0.375 0.400 0.425 MSE H=336 MSEMAE 124581632 K 0.390 0.420 0.450 MSE H=720 MSEMAE 124581632 K 0.240 0.255 0.270 MSE ETTh2 H=96 MSEMAE 124581632 K 0.280 0.300 0.320 0.340 MSE H=192 MSEMAE 124581632 K 0.320 0.340 0.360 0.380 MSE H=336 MSEMAE 124581632 K 0.325 0.350 0.375 0.400 MSE H=720 MSEMAE 124581632 K 0.240 0.255 0.270 0.285 MSE ETTm1 H=96 MSEMAE 124581632 K 0.280 0.300 0.320 0.340 MSE H=192 MSEMAE 124581632 K 0.320 0.340 0.360 MSE H=336 MSEMAE 124581632 K 0.350 0.375 0.400 0.425 MSE H=720 MSEMAE 124581632 K 0.140 0.150 0.160 MSE ETTm2 H=96 MSEMAE 124581632 K 0.180 0.195 0.210 MSE H=192 MSEMAE 124581632 K 0.225 0.240 0.255 0.270 MSE H=336 MSEMAE 124581632 K 0.300 0.320 0.340 0.360 MSE H=720 MSEMAE 0.378 0.380 0.382 0.384 MAE 0.402 0.405 0.408 MAE 0.416 0.424 0.432 MAE 0.440 0.460 0.480 MAE 0.328 0.330 0.332 MAE 0.360 0.368 0.376 0.384 MAE 0.390 0.400 0.410 MAE 0.416 0.424 0.432 0.440 MAE 0.321 0.324 0.327 MAE 0.344 0.352 0.360 0.368 MAE 0.376 0.380 0.384 MAE 0.405 0.410 0.415 MAE 0.235 0.240 0.245 0.250 MAE 0.276 0.282 0.288 0.294 MAE 0.318 0.320 0.322 0.324 MAE 0.368 0.376 0.384 MAE Figure 6: Hyperparameter sensitivity analysis for the number of shared temporal structuresKon the ETT benchmarks. Bar plots denote MSE and line plots denote MAE under different prediction horizons. 60301510642 N 0.300 0.320 0.340 0.360 MSE ETTh1 H=96 MSEMAE 60301510642 N 0.325 0.350 0.375 0.400 MSE H=192 MSEMAE 60301510642 N 0.350 0.375 0.400 0.425 MSE H=336 MSEMAE 60301510642 N 0.400 0.440 0.480 MSE H=720 MSEMAE 60301510642 N 0.240 0.255 0.270 MSE ETTh2 H=96 MSEMAE 60301510642 N 0.280 0.300 0.320 MSE H=192 MSEMAE 60301510642 N 0.320 0.340 0.360 MSE H=336 MSEMAE 60301510642 N 0.325 0.350 0.375 0.400 MSE H=720 MSEMAE 60301510642 N 0.240 0.260 0.280 MSE ETTm1 H=96 MSEMAE 60301510642 N 0.280 0.300 0.320 0.340 MSE H=192 MSEMAE 60301510642 N 0.320 0.340 0.360 MSE H=336 MSEMAE 60301510642 N 0.360 0.390 0.420 MSE H=720 MSEMAE 60301510642 N 0.140 0.150 0.160 MSE ETTm2 H=96 MSEMAE 60301510642 N 0.180 0.195 0.210 MSE H=192 MSEMAE 60301510642 N 0.225 0.240 0.255 0.270 MSE H=336 MSEMAE 60301510642 N 0.300 0.320 0.340 0.360 MSE H=720 MSEMAE 0.360 0.375 0.390 0.405 MAE 0.390 0.400 0.410 0.420 MAE 0.405 0.420 0.435 0.450 MAE 0.400 0.440 0.480 0.520 MAE 0.320 0.328 0.336 MAE 0.365 0.370 0.375 MAE 0.396 0.402 0.408 MAE 0.400 0.420 0.440 0.460 MAE 0.318 0.324 0.330 0.336 MAE 0.350 0.355 0.360 MAE 0.375 0.380 0.385 MAE 0.400 0.410 0.420 MAE 0.232 0.240 0.248 0.256 MAE 0.280 0.285 0.290 MAE 0.318 0.320 0.322 MAE 0.366 0.372 0.378 0.384 MAE Figure 7: Hyperparameter sensitivity analysis for the number of patchesNon the ETT benchmarks. TStruct remains stable within a broad middle range of patch-level resolutions, while overly small or overly large N may lead to underfitting or overfitting. 23 D Additional Interpretability Visualizations We provide additional visualizations of the learned temporal structures across all benchmark datasets. Figures 8 to 14 directly visualize the learned shared temporal structure matrices before softmax normalization, as the concentration of weights after softmax can make finer structural differences difficult to distinguish visually. Red and blue indicate higher and lower relative aggregation pref- erences, respectively, rather than positive and negative dependencies. The visualizations show that the learned structures are not random mixing patterns. Instead, they exhibit structured and reusable temporal dependency patterns across datasets, including diagonal-dominant local continuity, repeated stripe-like interactions, smooth long-range transitions, and more abstract patch-level dependency patterns. Several observations can be made. First, many structures contain clear diagonal or near-diagonal patterns, indicating that neighboring patches often preserve strong local temporal continuity. Sec- ond, repeated stripe-like or checkerboard patterns appear in several datasets, suggesting recurrent interactions between relative patch positions. Third, some structures exhibit smooth low-to-high transitions across rows or columns, reflecting broader temporal dependency tendencies beyond local neighborhoods. These patterns show that TStruct does not rely on a single predefined temporal prior, such as periodicity alone. The learned structures also vary across datasets. The ETT datasets often contain a mixture of local continuity, alternating recurrent patterns, and smooth transition structures, reflecting heterogeneous temporal dynamics. Weather, Electricity, and Traffic exhibit more regular banded or repeated structures, indicating that TStruct can capture dataset-specific temporal regularities through shared dependency matrices. Overall, these visualizations provide qualitative evidence that the temporal structures learned by TStruct are interpretable, structured, and adaptive to different datasets. 02468 Patch Number N 0 2 4 6 8 Patch Number N Layer 0 Time Structure #1 02468 Patch Number N 0 2 4 6 8 Patch Number N Time Structure #2 02468 Patch Number N 0 2 4 6 8 Patch Number N Time Structure #3 02468 Patch Number N 0 2 4 6 8 Patch Number N Time Structure #4 1.0 0.5 0.0 0.5 1.0 Figure 8: Learned Temporal Structures on ETTh1. 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 0 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Figure 9: Learned Temporal Structures on ETTh2. 24 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 0 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 1 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 2 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Figure 10: Learned Temporal Structures on ETTm1. 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 0 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 1 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 2 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #5 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Figure 11: Learned Temporal Structures on ETTm2. 25 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 0 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 1 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 2 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 2 1 0 1 2 Figure 12: Learned Temporal Structures on Weather. 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 0 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 1 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 036912 Patch Number N 0 3 6 9 12 Patch Number N Layer 2 Time Structure #1 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #2 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #3 036912 Patch Number N 0 3 6 9 12 Patch Number N Time Structure #4 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Figure 13: Learned Temporal Structures on Electricity. 26 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Layer 0 Time Structure #1 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #2 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #3 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #4 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Layer 1 Time Structure #1 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #2 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #3 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #4 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Layer 2 Time Structure #1 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #2 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #3 0510152025 Patch Number N 0 5 10 15 20 25 Patch Number N Time Structure #4 1.0 0.5 0.0 0.5 1.0 Figure 14: Learned Temporal Structures on Traffic. 27 E Detailed Experimental Results This section provides the complete experimental results corresponding to the summarized tables in the main paper. While the main text reports average results over all prediction horizons for compactness, here we present the detailed results under each prediction lengthH ∈ 96, 192, 336, 720. These results provide a more fine-grained comparison across horizons and further support the consistency of TStruct under different forecasting settings. E.1 Detailed Forecasting Results Table 10 reports the full forecasting results of TStruct and all baseline methods on seven benchmarks. The results include both MSE and MAE under each prediction horizon, providing the detailed counterpart to the averaged results reported in the main paper. Table 10: Full multivariate time series forecasting results. The best results are marked in bold red, and the second-best results are marked inblueunderline. Methods TStruct (ours) DUET [2025] PhaseFormer [2026] GTR [2026] TimeEmb [2025] TQNet [2025] CycleNet [2024] PatchTST [2023] DLinear [2023] DatasetsMetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE ETTh1 960.3450.3800.3520.3840.359 0.3820.367 0.3910.372 0.4000.371 0.3930.374 0.3960.375 0.3990.379 0.403 1920.3830.4050.398 0.4090.3970.4040.409 0.4200.413 0.4270.410 0.4180.406 0.4150.414 0.4210.408 0.419 3360.4070.4210.414 0.4260.4250.4240.432 0.4340.438 0.4430.432 0.4330.431 0.4300.431 0.4360.440 0.440 7200.4270.4540.429 0.4550.4310.4500.451 0.4630.449 0.4620.466 0.4680.450 0.4640.449 0.4660.471 0.493 Avg.0.3900.4150.398 0.4190.4030.4150.415 0.4190.418 0.4330.420 0.4280.415 0.4260.417 0.4310.425 0.439 ETTh2 960.2670.3300.2700.3360.275 0.3380.274 0.3410.290 0.3480.290 0.3490.271 0.3370.274 0.3370.300 0.364 1920.3220.3710.3320.3740.341 0.3760.340 0.3850.347 0.3870.367 0.3970.332 0.3800.348 0.3840.387 0.423 3360.3570.4000.3530.3970.369 0.4050.363 0.4090.376 0.4110.363 0.4060.362 0.4080.377 0.4160.490 0.487 7200.3810.4260.3820.4250.402 0.4360.405 0.4390.399 0.4390.412 0.4430.415 0.4490.406 0.4410.704 0.597 Avg.0.3310.3810.3340.3830.347 0.3890.346 0.3940.353 0.3960.358 0.3990.345 0.3940.351 0.3950.470 0.468 ETTm1 960.2750.3250.2790.3330.293 0.3440.283 0.3370.293 0.3460.294 0.3450.299 0.3480.289 0.3430.300 0.345 1920.3200.3540.3200.3580.323 0.3610.323 0.3650.326 0.3660.336 0.3720.334 0.3670.329 0.3680.336 0.366 3360.3550.3780.3480.3770.358 0.3810.355 0.3840.356 0.3820.368 0.3920.364 0.3870.362 0.3900.367 0.386 7200.4000.4110.4050.4080.412 0.4100.420 0.4220.405 0.4100.412 0.4430.410 0.4110.416 0.4230.419 0.416 Avg.0.3370.3670.3380.3690.347 0.3740.345 0.3770.345 0.3760.353 0.3880.352 0.3780.349 0.3810.356 0.378 ETTm2 960.1530.2410.1610.2480.163 0.2560.161 0.2520.163 0.2510.166 0.2550.159 0.2470.165 0.2550.164 0.255 1920.2090.2820.2140.2860.219 0.2930.221 0.2920.219 0.2900.222 0.2940.214 0.2860.221 0.2930.224 0.304 3360.2640.3210.2670.3210.269 0.3260.278 0.3260.2650.3200.277 0.3370.268 0.3260.276 0.3270.277 0.337 7200.3440.3720.348 0.3740.351 0.3790.359 0.3880.3430.3720.366 0.3860.353 0.3840.362 0.3810.371 0.401 Avg.0.2430.3040.2480.3070.251 0.3140.255 0.3150.248 0.3080.258 0.3180.249 0.3110.256 0.3140.259 0.324 Weather 960.1370.1810.1460.1910.148 0.1950.147 0.1990.143 0.1930.155 0.2010.164 0.2200.150 0.2000.170 0.230 1920.1810.2260.1880.2310.193 0.2370.192 0.2400.188 0.2370.200 0.2420.209 0.2580.191 0.2390.216 0.273 3360.2300.2670.2340.2680.242 0.2780.243 0.2810.236 0.2740.248 0.2840.255 0.2920.242 0.2790.258 0.307 7200.3040.3240.3050.3190.309 0.3320.324 0.3370.306 0.3250.318 0.3350.320 0.3380.312 0.3300.323 0.362 Avg.0.2130.2490.2180.2520.223 0.2610.227 0.2640.218 0.2570.230 0.2660.237 0.2770.224 0.2620.242 0.293 Electricity 960.1230.2160.1280.2190.129 0.2210.127 0.2220.129 0.2250.129 0.2240.128 0.2230.130 0.2220.140 0.237 1920.1410.2320.1450.2350.148 0.2380.147 0.2420.145 0.2410.151 0.2450.143 0.2370.148 0.2400.154 0.251 3360.1550.2500.163 0.2550.165 0.2570.165 0.2600.161 0.2570.166 0.2610.1590.2540.167 0.2610.169 0.268 7200.1850.2790.1930.2810.201 0.2850.199 0.2920.197 0.2890.201 0.2940.197 0.2870.202 0.2910.204 0.301 Avg.0.1510.2440.1570.2480.161 0.2500.160 0.2540.158 0.2530.162 0.2560.157 0.2500.162 0.2540.167 0.264 Traffic 960.3520.2350.360 0.2380.3610.2380.367 0.2570.374 0.2680.362 0.2600.381 0.2660.367 0.2510.395 0.275 1920.3680.2440.383 0.2490.3730.2430.382 0.2660.387 0.2680.384 0.2690.394 0.2730.385 0.2590.407 0.280 3360.3830.2530.395 0.2590.3850.2480.399 0.2750.401 0.2750.423 0.2970.406 0.2790.398 0.2650.417 0.286 7200.4220.2770.435 0.2780.4280.2700.441 0.2980.434 0.2750.447 0.3030.441 0.3000.434 0.2870.454 0.308 Avg.0.3810.2520.393 0.2560.3870.2500.397 0.2740.399 0.2720.404 0.2820.406 0.2800.396 0.2660.418 0.287 1st Count32233507001200000000 E.2 Detailed Ablation Results Table 11 reports the full ablation results under different prediction horizons. These results complement the averaged ablation table in the main paper and show that the benefits of the proposed Temporal Structure Mixer and PCAHead remain consistent across forecasting horizons. 28 Table 11: Complete ablation results of TStruct under different prediction horizons. TSM→MHA replaces the Temporal Structure Mixer with multi-head attention, TSM→Linear replaces it with a shared linear temporal mixer, and PCAHead→Linear replaces the proposed prediction head with a linear head. DatasetsETTh1ETTh2ETTm1ETTm2WeatherElectricityTraffic MethodMetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE TSM → MHA 960.367 0.3990.285 0.3500.298 0.3400.191 0.2720.143 0.1890.131 0.2250.377 0.255 1920.421 0.4400.360 0.3950.344 0.3690.222 0.2930.188 0.2320.148 0.2400.390 0.262 3360.434 0.4470.360 0.4020.424 0.4260.266 0.3220.260 0.2950.167 0.2580.405 0.272 7200.478 0.4910.393 0.4330.437 0.4290.356 0.3830.326 0.3440.203 0.2910.439 0.289 Avg.0.425 0.4440.350 0.3950.376 0.3910.259 0.3180.229 0.2650.162 0.2540.403 0.270 TSM → Linear 960.364 0.3930.271 0.3270.278 0.3310.158 0.2430.140 0.1850.125 0.2180.352 0.235 1920.395 0.4130.338 0.3730.334 0.3620.218 0.2850.187 0.2320.143 0.2340.368 0.245 3360.418 0.4280.365 0.4040.359 0.3820.266 0.3220.236 0.2730.157 0.2500.383 0.254 7200.433 0.4610.391 0.4310.418 0.4130.351 0.3810.309 0.3270.187 0.2800.423 0.277 Avg.0.403 0.4240.341 0.3840.347 0.3720.248 0.3080.218 0.2540.153 0.2460.382 0.253 PCAHead → Linear 960.348 0.3820.268 0.3310.275 0.3250.160 0.2480.137 0.1790.124 0.2170.353 0.236 1920.385 0.4050.330 0.3740.323 0.3540.212 0.2830.184 0.2270.142 0.2320.369 0.245 3360.411 0.4230.364 0.4050.357 0.3790.267 0.3220.230 0.2680.156 0.2500.383 0.254 7200.431 0.4590.393 0.4320.408 0.4120.354 0.3760.310 0.3280.186 0.2790.423 0.277 Avg.0.394 0.4170.339 0.3860.341 0.3680.248 0.3070.215 0.2510.152 0.2450.382 0.253 TStruct 960.345 0.3800.267 0.3300.275 0.3250.155 0.2410.137 0.1810.123 0.2160.352 0.235 1920.383 0.4050.322 0.3710.320 0.3540.209 0.2820.181 0.2260.141 0.2320.368 0.244 3360.407 0.4210.357 0.4000.355 0.3780.265 0.3200.230 0.2670.155 0.2490.383 0.253 7200.427 0.4540.381 0.4260.400 0.4110.344 0.3720.304 0.3240.185 0.2790.422 0.277 Avg.0.390 0.4160.331 0.3810.337 0.3670.243 0.3030.213 0.2490.151 0.2440.381 0.252 29 F Limitations and Future Directions TStruct models temporal dependencies through a compact set of shared temporal structure matrices, with controlled initialization and softmax normalized composition and aggregation. This parameter- ization provides an effective way to learn reusable temporal structures for LTSF. Their adaptation across patch resolutions and transfer across datasets and domains at scale remain directions for further exploration. Patch resolution dependency. The learned temporal structures are tied to a specific patch resolution. Since each temporal dependency matrix is parameterized overNpatch positions, changing the patch size or look-back length can alter the matrix dimensions or the temporal meaning of its coordinates. Although TStruct remains effective under different look-back lengths, directly sharing temporal structures across different patch resolutions remains non-trivial. A promising direction is to develop resolution-adaptive temporal structures that can be transferred, interpolated, or composed across different patch scales. Temporal structure foundation models. Even after fine-tuning, time series foundation models can struggle to outperform small models trained on individual datasets. One possible explanation is the difficulty of learning input representations that generalize across heterogeneous time series, motivating the exploration of temporal dependency structures as an alternative focus of pretraining. Our analysis provides evidence of reusable temporal relationships across temporal positions and variables, while their transfer and generalization across datasets and domains at scale remain untested. A potential direction is to develop a time series foundation model built around a pretrained dictionary of shared temporal dependency structures, with downstream forecasting tasks selecting and composing these structures to form temporal aggregation operators. 30