Paper deep dive
Addressing Spatial Indistinguishability in Spatiotemporal Prediction via Optimal Transport-Guided Masking
Guangyu Wang, Jiawei Tong
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spatiotemporal prediction aims to learn discriminative representations from correlated temporal signals over spatial structures for accurate future inference. A central challenge is \emph{spatial indistinguishability}: different nodes may share similar historical patterns yet evolve toward divergent futures, severely degrading forecasting performance in real-world sensor networks. Existing embedding-based and graph neural network (GNN)-based approaches can partially detect such ambiguous nodes but rely on historical similarity, struggling to capture \emph{future behavioral divergence}. We propose \textbf{STOT} (\textbf{S}patio\textbf{T}emporal \textbf{O}ptimal \textbf{T}ransport), a self-supervised framework that resolves spatiotemporal ambiguity via structured masking guided by optimal transport. Our key idea treats indistinguishability as a \emph{disambiguation} problem: future states are inferred by exploiting concurrent spatial correlations and their time-varying similarity. We design a similarity-aware metric for dynamic inter-node relationships and an optimal transport-based masking strategy to emphasize ambiguous positions during pre-training. A batch consistency constraint preserves semantic coherence, while a random-walk masking mechanism promotes structured context exploration. Experiments on six real-world datasets show that STOT performs competitively with state-of-the-art baselines on the evaluated benchmarks and improved interpretability through transport-plan visualizations.
Tags
Links
- Source: https://arxiv.org/abs/2609.35021v1
- Canonical: https://arxiv.org/abs/2609.35021v1
Trouble viewing inline? Open PDF directly →
Full Text
64,862 characters extracted from source content.
Expand or collapse full text
Addressing Spatial Indistinguishability in Spatiotemporal Prediction via Optimal Transport-Guided Masking Journal: Pattern Recognition Guangyu Wang Affiliation: Data Science and Artificial Intelligence, Dongbei University of Finance and Economics, Dalian, Liaoning, China Jiawei Tong Affiliation: Institute of Systems, Molecular & Integrative Biology, University of Liverpool, Liverpool, United Kingdom Corresponding author: Jiawei Tong (Jiawei.Tong@liverpool.ac.uk) is the corresponding author. Abstract Spatiotemporal prediction aims to learn discriminative representations from correlated temporal signals over spatial structures for accurate future inference. A central challenge is spatial indistinguishability: different nodes may share similar historical patterns yet evolve toward divergent futures, severely degrading forecasting performance in real-world sensor networks. Existing embedding-based and graph neural network (GNN)-based approaches can partially detect such ambiguous nodes but rely on historical similarity, struggling to capture future behavioral divergence. We propose STOT (SpatioTemporal Optimal Transport), a self-supervised framework that resolves spatiotemporal ambiguity via structured masking guided by optimal transport. Our key idea treats indistinguishability as a disambiguation problem: future states are inferred by exploiting concurrent spatial correlations and their time-varying similarity. We design a similarity-aware metric for dynamic inter-node relationships and an optimal transport-based masking strategy to emphasize ambiguous positions during pre-training. A batch consistency constraint preserves semantic coherence, while a random-walk masking mechanism promotes structured context exploration. Experiments on six real-world datasets show that STOT performs competitively with state-of-the-art baselines on the evaluated benchmarks and improved interpretability through transport-plan visualizations. Keywords: Spatiotemporal Prediction , Masked Autoencoder , Deep Learning , Optimal Transport Constraint , Spatial Indistinguishability , Traffic Prediction 1 Introduction Spatiotemporal data prediction through sensor networks is a classic multivariate time series forecasting task [40], with broad applications in earth system forecasting [43], traffic flow prediction [31], and energy prediction [20]. Prediction accuracy plays a crucial role in effective decision-making and minimizing socioeconomic impacts across these tasks. However, for traditional machine learning methods such as Support Vector Regression [2], Random Forest [34], and Gradient Boosting Decision Trees [21], achieving high prediction accuracy remains challenging because of the complex spatiotemporal relationships and nonlinear patterns inherent in these systems. Notably, deep learning advances in spatiotemporal prediction have revolutionized this field with increasingly effective solutions, especially after identifying spatial indistinguishability as the key insight for model design. In the early stages, although Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN)-based methods showed promise in capturing temporal and spatial patterns [33], their limitations in incorporating topological structures still hindered prediction performance. To address this limitation, Graph Neural Networks (GNNs) overcame topological limitations by capturing complex relationships between spatial nodes [26], yet their architectural complexity with uncertain performance gains still posed significant challenges for spatiotemporal prediction. A key breakthrough came with the specific illustration of spatial indistinguishability [10] – nodes with similar historical patterns exhibiting divergent future trends – which directly led to a paradigm shift in spatiotemporal prediction. This insight inspired new directions in model design, where embedding-based methods [37] tackled spatial indistinguishability more directly through simpler architectures by constructing spatially distinguishable embeddings, avoiding the performance uncertainties of complex GNN-based methods [46]. Currently, predicting spatially indistinguishable nodes poses a critical challenge for accurate spatiotemporal prediction, as these nodes exhibit similar historical patterns but divergent future trends. This challenge is particularly significant given that such nodes dominate over 50% of sensor networks in real-world applications [36]. Although existing embedding-based methods [37] have attempted to improve performance by distinguishing node identities, this approach fails to capture the future behavioral characteristics of these nodes: while distinguishing identities may prevent interference with stable nodes, it does not directly contribute to predicting their future states. Such limitations persist in other approaches as well; GNN-based methods isolate these nodes through topological structures, yet still fail to capture their inherent predictive characteristics. We provide a more detailed discussion of their limitations in Section 2. In this paper, we propose a novel theoretical framework to address the fundamental challenge of predicting spatially indistinguishable nodes in spatiotemporal systems. Our key insight is that while these nodes may exhibit weak temporal autocorrelation, their future states can be effectively inferred through concurrent spatial correlations. This theoretical foundation is supported by empirical observations across various spatiotemporal domains, where local perturbations propagate through the system with spatially and temporally varying effects, creating predictable patterns of influence. Leveraging this theoretical framework, we introduce several methodological innovations. First, we develop a quantitative metric for dynamically evaluating spatial indistinguishability, providing a rigorous foundation for identifying and analyzing these challenging nodes. Second, inspired by recent advances in self-supervised learning, particularly masked autoencoders (MAE), we design a specialized pre-training paradigm that focuses on reconstructing the states of spatially indistinguishable nodes using information from their distinguishable counterparts. To enhance the effectiveness of this approach, we incorporate two complementary constraints: a batch consistency policy that maintains semantic coherence across training instances and a random walk mechanism that ensures comprehensive exploration of the representation space. These components are unified within an optimal transport framework, leading to our proposed method: SpatioTemporal Optimal Transport (STOT). The main contributions of this paper can be summarized as follows: 1. We present a systematic theoretical analysis of spatial indistinguishability in spatiotemporal prediction, revealing the critical importance of predicting indistinguishable nodes’ future states. 2. We introduce a novel timestep-level metric for quantifying spatial indistinguishability, enabling a more precise characterization of spatiotemporal dynamics. 3. We develop an optimal transport constraint framework that ensures both semantic continuity and diversity through batch consistency and random walk strategies. 4. We validate our theoretical framework through extensive experiments on six real-world datasets, demonstrating competitive or superior performance on the evaluated benchmarks and providing insights into spatiotemporal prediction mechanisms. 2 Related Work This section reviews the mainstream approaches for spatiotemporal prediction and discusses their limitations in accurately predicting future states of spatially indistinguishable nodes. 2.1 Graph-based Methods Graph-based methods leverage the topological relationships among sensor nodes to represent spatial interactions in time series data, enabling the design of advanced spatiotemporal models. For instance, STGCN [44] integrates graph convolutional networks with gated temporal convolutions to capture comprehensive spatiotemporal features, while STNN [18] further refines this approach by incorporating spatial and temporal attention mechanisms to address region-based dependencies and external factors. Recently, a prevalent view has emerged that predefined graphs may introduce biases or prove inadequate in certain scenarios. In response, GTS [35] proposes learning latent graph structures jointly with spatiotemporal dynamics, and DFDGCN [27] combines static graphs with dynamically learned adjacency matrices. More recently, LvSTformer [29], a transformer-based approach, has been introduced to capture evolving multi-scale spatiotemporal patterns across sensors. From a data-driven perspective, the superior modeling of spatiotemporal relationships by latent graph structures cannot be solely attributed to capturing time-varying interaction patterns. We argue that the key lies in addressing spatially indistinguishable nodes. In predefined graphs, these nodes are connected to others, and their inherently unpredictable future patterns can induce information confusion during message passing. Latent graphs, by contrast, effectively sever these problematic connections, which leads to consistently better performance, a view supported by [36]. Clearly, GNN-based methods struggle to predict the future states of spatially indistinguishable nodes when they remain connected to other nodes [22], and empirical evidence shows that disconnecting these nodes enhances overall predictive accuracy. 2.2 Embedding-based Methods Embedding-based methods address spatial indistinguishability by designing spatially distinguishable models. Further examining the mechanisms of representative approaches, STID [37] augments nodes with identity information in both temporal and spatial dimensions, enabling node differentiation. STID provides a more direct solution that achieves similar results to models that learn latent graphs, disconnecting nodes that are spatially indistinguishable from each other. Building upon this, STAEformer [30] further integrates these identity-aware embeddings with transformer architectures to enhance performance. Another line of work focuses more on capturing location-specific temporal dynamics. ST-WA [9] learns location-specific encodings to capture unique patterns across time series, while ST-Norm [10] achieves a similar effect through carefully designed normalization methods, eliminating the need for additional embeddings. More recently, HiMNet [12] learns heterogeneous meta-parameters from clustered spatial and temporal embeddings, which captures node-level behavioral differences without relying on explicit identity inputs. A parallel and more recent trend concerns generative modeling for spatiotemporal systems. For instance, Net-Ev2 [39] simulates network event evolution with a generative framework, and Diffusion-TS [45] provides an interpretable diffusion model for general time series generation. Embedding-based methods inherently rely on node differentiation, similar to latent graph learning, as they disconnect spatially indistinguishable nodes from others. However, they fail to effectively predict the future states of these spatially indistinguishable nodes. 2.3 Pre-trained Methods Pre-trained methods enable models to receive longer historical inputs, which has received increasing attention in spatiotemporal forecasting, as extended input sequences enhance a model’s robustness to sudden events and improve its generalization to unseen data [14]. To obtain superior hidden representations, researchers have explored using pre-training techniques on time series data. One promising technique is the masked autoencoder (MAE), which masks out random input patches and reconstructs the original input based on the unmasked patches. This approach has been successfully applied in various domains, with masked pre-training yielding significant improvements in downstream tasks in computer vision [4] and natural language processing [24]. Several studies have explored the application of MAE in spatiotemporal forecasting. For instance, STEP [38] employs MAE to reconstruct long historical time series, enhancing the model’s prediction capabilities. However, Ti-MAE [28] noted the inconsistency between contrastive learning during pre-training and downstream prediction tasks and proposed using MAE as an auxiliary task to address this issue. Similarly, PITS [25] tackled this problem by combining MAE with comparative learning to avoid conflicts between pre-training and downstream tasks. Building upon these works, STD-MAE [14] considered the practical challenges of spatiotemporal forecasting and employed two MAEs in both temporal and spatial dimensions to reconstruct historical sequences, concatenating the spatiotemporal representations as inputs for the downstream predictor. Taking a different approach, SimMTM [11] connects masked modeling with manifold learning to learn deeper representations. However, these methods overlook the inherent nature of spatiotemporal forecasting and fail to strengthen training for spatial indistinguishability, diminishing pre-training performance and reducing efficiency. In contrast, our proposed method, STOT, introduces a carefully designed spatial indistinguishability-guided masking strategy during pre-training. By reconstructing indistinguishable nodes through distinguishable ones, STOT more effectively captures the complex interactions within spatiotemporal data. 3 Problem Formulation We formulate the spatiotemporal system as a graph G=(V,E,A)G=(V,E,A), where V=vii=1,2,…,NV=\v_i\_i=1,2,…,N represents the set of N nodes in the system. The edge set E=eijE=\e_ij\ captures the connectivity between node pairs, with the adjacency matrix A∈0,1N×NA∈\0,1\^N× N encoding these relationships such that Aij=1A_ij=1 if nodes viv_i and vjv_j are connected (eij∈Ee_ij∈ E), and Aij=0A_ij=0 otherwise. The system dynamics are characterized by a feature matrix X∈ℝN×Tlong×CX ^N× T_long× C, where TlongT_long represents the temporal dimension of historical observations in the pre-training stage, which is much longer than the historical window T used in end-to-end prediction and C is the feature dimension. At each timestep t, the state of the system is captured by xt∈ℝNx_t ^N. 4 Methodology 4.1 Overall Architecture of STOT As shown in Figure 1, STOT contains two stages: pre-training (a)-(e) and downstream prediction (f). In pre-training, TlongT_long timesteps pass through an embedding layer that combines raw features with temporal and spatial positional encodings, and then through a Transformer encoder-decoder with an SI-Mask targeting spatially indistinguishable positions; the learned SI-enhanced representations are integrated with downstream predictors. Unlike random-mask MAE models, STOT needs only one pre-training phase without contrastive learning, achieving comparable representation quality more efficiently than STD-MAE [14] and PITS [25]. Figure 1: Architecture of STOT: (a) the embedding layer aggregates raw long-term traffic data with temporal and spatial positional features; (b) the encoder learns spatially indistinguishable node representations through masked reconstruction; (c) the decoder reconstructs masked positions from SI-enhanced representations; (d) mask generation uses SI-Mask, Batch Consistency, and Random Walk, where SI-Mask guides spatial indistinguishability and the latter two constrain representation continuity and richness; (e) patch embedding segments spatiotemporal data into local-pattern units; and (f) the STOT-enhanced downstream predictor can be integrated with various downstream architectures. 4.2 Embedding Layer The embedding layer produces position-aware representations through patch segmentation, high-dimensional feature projection, and temporal/spatial positional embeddings. 4.2.1 Patch Segmentation Following [32], the input Xpre∈ℝTlong×N×CX_pre ^T_long× N× C is divided into P non-overlapping patches of length Tp=Tlong/PT_p=T_long/P: Xpatch=[0(Xpre);1(Xpre);…;P−1(Xpre)],X_patch=[P_0(X_pre);P_1(X_pre);...;P_P-1(X_pre)], (1) where i(⋅)P_i(·) extracts the i-th segment, yielding Xpatch∈ℝTp×N×(CP)X_patch ^T_p× N×(CP). 4.2.2 Positional Embeddings Temporal Positional Embedding (TPE) encodes temporal position p via sinusoidal functions: Et(p,q,k)=sin(p/100004k/D),if k is evencos(p/100004k/D),if k is odd,E_t(p,q,k)= cases (p/10000^4k/D),&if k is even\\ (p/10000^4k/D),&if k is odd cases, (2) yielding Et∈ℝTp×N×d/2E_t ^T_p× N× d/2. Spatial Positional Embedding (SPE) follows the same formulation with spatial position q in place of p, yielding Es∈ℝTp×N×d/2E_s ^T_p× N× d/2. 4.2.3 Embedding Output Raw data is projected via a fully connected layer EF=FC(Xpatch)∈ℝP×N×dE_F=FC(X_patch) ^P× N× d. The final embedding concatenates all three components: Z=EF|Et|Es∈ℝTp×N×2d.Z=E_F E_t E_s ^T_p× N× 2d. (3) 4.3 Spatiotemporal Masked Pre-training Following the MAE architecture, the encoder masks spatially indistinguishable positions and the decoder reconstructs them; our masking strategy targets such positions, balances semantic richness with computational efficiency, and learns transferable spatiotemporal representations. 4.3.1 Encoder Layer As shown in Figure 1(b), the encoder masks positions with high spatial indistinguishability. Unlike prior dataset-level measurements [36], our dynamic assessment captures time-varying node relationships and guides mask generation through an optimal transport (OT) formulation. Similarity Matrix Computation We compute cosine similarity matrices for both historical and future timesteps: St,i,jP=Xt,iP⋅Xt,jP‖Xt,iP‖‖Xt,jP‖,St,i,jF=Xt,iF⋅Xt,jF‖Xt,iF‖‖Xt,jF‖,i,j∈1,…,N.S^P_t,i,j= X^P_t,i· X^P_t,j\|X^P_t,i\|\|X^P_t,j\|, S^F_t,i,j= X^F_t,i· X^F_t,j\|X^F_t,i\|\|X^F_t,j\|, i,j∈\1,…,N\. (4) Spatial Indistinguishability Computation Given thresholds ϵu _u (high similarity) and ϵl _l (low similarity), the indistinguishability score for node i at timestep t is: It,i=∑j≠i(St,i,j<ϵl)∑j≠i(St,i,j>ϵu)+1,I_t,i= _j≠ i1(S_t,i,j< _l) _j≠ i1(S_t,i,j> _u)+1, (5) which is high when node i’s similarity to others is uniformly distributed. We adopt this ratio between confidently dissimilar and confidently similar neighbors because it is directional, growing precisely when a node lacks reliable similar anchors yet is surrounded by dissimilar ones, which is the configuration that makes its future hard to infer. A dispersion summary such as the entropy of the similarity distribution is direction-agnostic and assigns the same value to an anchor-rich node and to an anchor-poor one, so it cannot separate easy nodes from genuinely ambiguous ones. The same construction also keeps the score insensitive to the exact thresholds, since it reflects the gap between the two groups rather than their absolute counts, and Figure 2 confirms that nodes with It,i>0I_t,i>0 remain consistently harder to predict as ϵl _l varies. The overall score aggregates across both time periods and nodes: Ii=α⋅1Tp∑t=1TpIt,iP+(1−α)⋅1Tpred∑t=1TpredIt,iF,Tscore=1N∑i=1NIi,I_i=α· 1T_p _t=1^T_pI^P_t,i+(1-α)· 1T_pred _t=1^T_predI^F_t,i, Tscore= 1N _i=1^NI_i, (6) where α∈[0,1]α∈[0,1] balances historical and future contributions. Generating SI-Mask Position Directly selecting the top-K highest TscoreTscore positions has two drawbacks: dispersed within-batch masks hinder coherent learning, and fixed masks for identical inputs limit semantic diversity. We address both by formulating mask generation as an OT problem. Positions with high TscoreTscore are assigned lower transport cost via a log transformation, and a batch consistency penalty discourages overly dispersed masks: Cbase=−log(Tscore+1),Cfinal=Cbase+λ⋅|Cbase−1B∑b=1BCb|,C_base=- (Tscore+1), C_final=C_base+λ· |C_base- 1B _b=1^BC_b |, (7) where λ controls the batch consistency weight. The Sinkhorn algorithm [15] solves the OT problem by iteratively normalizing rows and columns to produce a doubly stochastic transport matrix (Pseudocode 1): Algorithm 1 Sinkhorn Algorithm for Mask Position Generation 1: Cost matrix CfinalC_final, number of iterations K, regularization parameter ϵε 2: Optimal transport plan M (mask positions) 3: Initialize M(0)=exp(−Cfinal/ϵ)M^(0)= (-C_final/ε) 4: for k=1k=1 to K do 5: Q(k)=M(k−1)Q^(k)=M^(k-1) ⊳ Store previous iteration 6: M(k)=Diag(u(k))⋅Q(k)⋅Diag(v(k))M^(k)=Diag(u^(k))· Q^(k)·Diag(v^(k)) ⊳ Update transport plan 7: u(k)=a⊘(Q(k)v(k−1))u^(k)=a (Q^(k)v^(k-1)) ⊳ Row normalization 8: v(k)=b⊘((Q(k))Tu(k))v^(k)=b ((Q^(k))^Tu^(k)) ⊳ Column normalization 9: end for 10: return M(K)M^(K) ⊳ Final transport plan as mask positions Mask positions are selected by choosing the Mtotal=⌊Tp⋅rmask⌋M_total= T_p· r_mask entries that receive the largest transport mass per sample: ℳb,s=1,if s∈topk(Mb,Mtotal)0,otherwise,M_b,s= cases1,&if s (M_b,M_total)\\ 0,&otherwise cases, (8) where topk(⋅,Mtotal)topk(·,M_total) returns the indices of the MtotalM_total largest transport values for sample b. The cost is evaluated per sample and candidate timestep, so Cfinal∈ℝB×TpC_final ^B× T_p with Cfinal,b,sC_final,b,s the cost of sample b at timestep s, and MbM_b is the b-th row of the transport plan M(K)M^(K) returned by Algorithm 1, holding the mass that sample b assigns to each candidate position. Since M(0)=exp(−Cfinal/ϵ)M^(0)= (-C_final/ε) maps low cost to high mass, selecting the largest-mass entries of MbM_b and setting ℳb,s=1M_b,s=1 turns the transport plan into the binary SI-Mask, while the batch-consistency term and column normalization keep this selection coupled across samples. Random Walk Mechanism Controlled by wratiow_ratio, random walk replaces Nreplace=⌊Mtotal⋅wratio⌋N_replace= M_total· w_ratio positions in the OT-generated mask by sampling a new timestep tnewt_new from unmasked positions ∖ℳbatchT _batch and releasing an existing mask position poldp_old; to maintain batch consistency, the operation is applied uniformly across samples: ℳb,pold=0,ℳb,tnew=1,∀b∈1,…,B,M_b,p_old=0, _b,t_new=1, ∀ b∈\1,…,B\, (9) yielding the final binary mask ℳ=[ℳb,1,…,ℳb,Tp]M=[M_b,1,…,M_b,T_p] that combines OT-guided positions with random walk diversity. Encoder with Mask Given the masked input where [MASK][MASK] tokens replace positions with ℳb,t=1M_b,t=1: x~b,t=[MASK],if ℳb,t=1xb,t,if ℳb,t=0, x_b,t= cases[MASK],&if M_b,t=1\\ x_b,t,&if M_b,t=0 cases, (10) the encoder applies standard Transformer layers—multi-head self-attention, feed-forward networks, and layer normalization—to produce the SI-enhanced representation after L layers: ℛSI=EncoderStackL(X~patch,ℳ)∈ℝB×Tp×D.R^SI=EncoderStack_L( X_patch,M) ^B× T_p× D. (11) 4.3.2 Decoder Layer The decoder uses the same Transformer architecture as the encoder; after padding to uniform dimensionality, LdL_d decoder layers reconstruct masked positions from ℛSIR^SI and yield X^patch∈ℝB×Tp×D X_patch ^B× T_p× D with reconstructed values YpreY_pre where ℳb,t=1M_b,t=1. 4.4 Downstream Spatiotemporal Prediction The pre-trained encoder processes TlongT_long timesteps to produce ℛSIR^SI, which is fused with the short-term representation from downstream predictor F0F_0 via an MLP: ℛfused=MLP([ℛSI;F0(T)]).R^fused=MLP([R^SI;F_0(T)]). (12) In this work we use GWNet [42] as the primary predictor, with additional experiments on LSTM and ASTGCN [16] to validate generalizability across architectures. Since this fusion runs after representation learning, where the pre-trained encoder already supplies informative spatial indistinguishability representations, we keep it as a simple MLP that preserves their integrity while remaining accurate, following classic models such as STAEformer [30]. 4.5 Loss Function STOT uses two losses across its two training phases, with the encoder frozen during prediction: ℒpre=1B∑b=1B∑t=1Tpℳb,t‖Ypreb,t−Xpatchb,t‖22,ℒpred=1B∑b=1B‖F0(ℛfused)−Y‖22,L_pre= 1B _b=1^B _t=1^T_pM_b,t||Y_pre^b,t-X_patch^b,t||_2^2, _pred= 1B _b=1^B||F_0(R^fused)-Y||_2^2, (13) where ℒpreL_pre is the masked reconstruction loss over pre-training and ℒpredL_pred is the RMSE for downstream forecasting. 5 Experiments We answer seven research questions: (I) STOT’s performance against SOTA methods; (I) the effect of each mask pre-training component; (I) performance under high spatial indistinguishability; (IV) hyperparameter sensitivity across datasets; (V) computational efficiency versus other pre-training models; (VI) interpretability from OT visualization; and (VII) the effectiveness of the random walk mechanism compared with variants that change its unit or position. 5.1 Experimental Settings 5.1.1 Datasets Table 1: Dataset description and statistics. Datasets #Sensors #Edges #Time Period #Type #Place PeMS03 358 574 2018/09–2018/11 Flow North Central California, USA PeMS04 307 340 2018/01–2018/02 Flow San Francisco Bay Area, USA PeMS07 883 866 2017/05–2017/08 Flow Los Angeles area, USA PeMS08 170 295 2016/07–2016/08 Flow San Bernardino area, USA PeMS-BAY 325 2369 2017/01–2017/06 Speed San Francisco Bay Area, USA METR-LA 207 1515 2012/03–2012/06 Speed Los Angeles area, USA We conducted extensive experiments using six real-world traffic datasets, which are classic and widely used examples of spatiotemporal data [1, 8, 7], with details provided in Table 1. The datasets are briefly described as follows: PeMS03 provides traffic flow data from 358 sensors in California’s District 3, spanning September to November 2018; PeMS04 provides traffic flow data from 307 sensors in California’s District 4, collected between January and February 2018; PeMS07 provides traffic flow measurements from 883 sensors in California’s District 7, covering May to August 2017; PeMS08 provides traffic flow data from 170 sensors in California’s District 8, obtained during July and August 2016; METR-LA provides traffic speed measurements from 207 sensors in Los Angeles, California, collected between March and June 2012; and PeMS-BAY provides traffic speed data from 325 sensors in the San Francisco Bay Area, California, spanning January to June 2017. The traffic flow and speed datasets are partitioned into training, validation, and test subsets to facilitate model evaluation. For the traffic flow data, the ratio of the training, validation, and test subsets is 6:2:2, while for the traffic speed data, the ratio is 7:1:2. 5.1.2 Baselines We compare STOT with 19 representative baselines covering classical statistical methods and recent SOTA deep learning models: ARIMA [41] and VAR [6]; spatiotemporal graph neural networks including DCRNN [26], STGCN [44], ASTGCN [16], GWNet [42], STGODE [13], DSTAGNN [23], ASTGNN [17], and AGCRN [3]; spatially enhanced or embedding/normalization-based models including ST-WA [9], ST-Norm [10], and STID [37]; transformer-based or attention-driven PDFormer [19] and STAEformer [30]; pre-training and masked-reconstruction STEP [38] and STD-MAE [14]; and heterogeneity or decomposition models HimNet [12] and STDN [5], ensuring comprehensive and fair assessment across modeling paradigms. 5.1.3 Parameter Settings Following the BasicTS heterogeneity analysis protocol [36], we set the similarity thresholds to ϵu=0.9 _u=0.9 and ϵl=0.5 _l=0.5. We also use a batch size of 8, 100 pre-training epochs, and at most 300 forecasting epochs. The specific hyperparameters are summarized in Table 2. Table 2: Detailed configurations of STOT and main baselines. Setting STOT STD-MAE STEP ST-WA STID ST-Norm Learning Rate 0.001 0.001 0.001 0.001 0.002 0.002 Hidden Dimension 96 96 96 128 32 – Number of Heads 4 4 4 – – – Similarity Thresholds (ϵu,ϵl)( _u, _l) (0.9, 0.5) – – – – – Mask Ratio 0.25 0.25 0.75 – – – Sinkhorn Temperature ϵε 0.8 – – – – – Sinkhorn Iteration Count K 3 – – – – – Batch Consistency Weight λ 0.7 – – – – – We further examined whether the low similarity threshold affects the validity of the spatial indistinguishability score. Because the score in Equation 5 is computed as a ratio between confidently dissimilar and confidently similar neighbors, its value depends on the overall separation of the similarity distribution and stays stable when the thresholds are perturbed within a reasonable range. To verify this property on a concrete model, we conducted a control analysis on the GWNet backbone [42]. For each node sample, we computed It,iI_t,i and split the samples into one set with It,i=0I_t,i=0 and one set with It,i>0I_t,i>0, then compared their forecasting MAE distributions under ϵl∈0.40,0.45,0.50,0.55 _l∈\0.40,0.45,0.50,0.55\. As shown in Figure 2, the set with It,i>0I_t,i>0 consistently attains larger MAE, which indicates that the score captures prediction difficulty and that the default threshold remains reliable. Figure 2: KDE distributions of node-sample forecasting MAE grouped by the spatial indistinguishability score in Equation 5. Dashed lines indicate the group-wise mean MAE. 5.1.4 Evaluation Metrics We use three standard metrics (yiy_i: ground truth, y^i y_i: prediction): MAE =1n∑|yi−y^i|= 1nΣ|y_i- y_i|, RMSE =1n∑(yi−y^i)2= 1nΣ(y_i- y_i)^2, and MAPE =1n∑|yi−y^iyi|×100= 1nΣ | y_i- y_iy_i |× 100. 5.1.5 Implementation Details We conducted experiments on 8 RTX A6000 GPUs and implemented our proposed model based on BasicTS [36], which provides a unified framework for data loading and evaluation metrics to ensure fair comparisons. The shared training protocol and method-specific hyperparameters are detailed in the Parameter Settings subsection and summarized in Table 2. For a fair comparison, all pre-training methods, including STEP, STD-MAE, and STOT, were run with the same pre-training length of Tlong=288T_long=288 steps, even though their original papers adopt longer sequences. 5.2 Main Results (RQ1) 5.2.1 Comparison Results Tables 3 and 4 report six-dataset prediction results, with bold values marking the best performance; STOT consistently outperforms baselines on traffic speed and flow, and Figure 3 shows that it surpasses STID and ST-Norm across 1- to 12-step forecasts on PeMS03/04/07/08, demonstrating its advantage in handling spatial indistinguishability. The observations are that conventional statistical methods struggle with nonlinear and non-stationary data, yielding lower accuracy than deep learning approaches; GNN-based methods such as AGCRN capture spatiotemporal correlations but suffer from RNN-based multi-step error accumulation, while attention-based PDFormer mitigates this issue yet still underperforms because of insufficient spatial modeling. Their message passing also aggregates signals over fixed neighborhoods, so the confusion carried by spatially indistinguishable nodes propagates to neighbors and reinforces ambiguous representations, which is the core difficulty that our masking strategy targets. Spatially-aware models such as STID and ST-Norm generalize better by directly addressing spatial distinguishability, outperforming GNN-based methods in accuracy and efficiency, while pre-trained STEP and STD-MAE learn from long historical sequences and focus on hard-to-predict nodes, with STD-MAE’s dual autoencoders further exploiting spatial structure to achieve near-optimal results. STOT’s gains come from long-term inputs with temporal periodic features for richer spatiotemporal representations, adaptive spatial representations learned through mutual reconstruction guided by spatial distinguishability, and OT-based batch consistency that improves semantic coherence and generalization. The gain is smaller on PeMS-BAY because its nodes share very similar long-term levels and form few strongly correlated pairs, so the dataset carries less spatial indistinguishability for the optimal transport masking to exploit. Table 3: Performance comparison on PeMS03, PeMS04, PeMS07, and PeMS08 benchmarks. Model PeMS03 PeMS04 PeMS07 MAE RMSE MAPE MAE RMSE MAPE MAE RMSE MAPE ARIMA [41] 35.31 47.59 33.78 33.73 48.80 24.18 38.17 59.27 19.46 VAR [6] 23.65 38.26 24.51 23.75 36.66 18.09 75.63 115.24 32.22 DCRNN [26] 18.18 30.31 18.91 24.70 38.12 17.12 25.30 38.58 11.66 STGCN [44] 17.49 30.12 17.15 22.70 35.55 14.59 25.38 38.78 11.08 ASTGCN [16] 17.69 29.66 19.40 22.93 35.22 16.56 28.05 42.57 13.92 GWNet [42] 19.85 32.94 19.31 25.45 39.70 17.29 26.85 42.78 12.12 STGODE [13] 16.50 27.84 16.69 20.84 32.82 13.77 22.99 37.54 10.14 DSTAGNN [23] 15.57 27.21 14.68 19.30 31.46 12.70 21.42 34.51 9.01 ST-WA [9] 15.17 26.63 15.83 19.06 31.02 12.52 20.74 34.05 8.77 ASTGNN [17] 15.07 26.88 15.80 19.26 31.16 12.65 22.23 35.95 9.25 AGCRN [3] 16.06 28.49 15.85 19.83 32.26 12.97 21.29 35.12 8.97 ST-Norm [10] 15.32 25.93 14.37 19.21 32.30 13.05 20.59 34.86 8.61 STID [37] 15.30 27.26 16.47 18.40 29.98 12.93 19.61 32.83 8.33 STEP * [38] 14.22 24.55 14.42 18.20 29.71 12.48 19.32 32.19 8.12 PDFormer [19] 14.94 25.39 15.82 18.32 29.97 12.10 19.83 32.87 8.53 STAEformer [30] 15.35 27.55 15.18 18.22 30.18 11.98 19.14 32.60 8.01 HimNet [12] 15.34 27.50 N/A 18.24 30.12 N/A 19.30 32.77 N/A STDN [5] 15.54 27.28 N/A 18.40 30.22 N/A 20.65 34.77 N/A STD-MAE* [14] 14.01 25.79 14.30 18.10 29.68 11.97 19.25 32.27 8.04 STOT (Our Study) 13.91 25.71 13.56 18.08 29.67 11.89 18.95 31.63 7.98 Table 4: Performance comparison on PeMS08, METR-LA, and PeMS-BAY benchmarks. Model PeMS08 METR-LA PeMS-BAY MAE RMSE MAPE MAE RMSE MAPE MAE RMSE MAPE ARIMA [41] 31.09 44.32 22.73 5.15 10.45 12.70 2.33 4.76 5.40 VAR [6] 23.46 36.33 15.42 5.28 9.06 12.50 2.24 4.96 4.83 DCRNN [26] 17.86 27.83 11.45 3.59 7.61 10.44 1.96 4.59 4.68 STGCN [44] 18.02 27.83 11.40 3.60 7.50 10.56 1.99 4.51 4.66 ASTGCN [16] 18.61 28.16 13.08 3.57 7.19 10.32 1.86 4.07 4.27 GWNet [42] 19.13 31.05 12.68 4.90 9.70 14.75 2.71 6.25 6.69 STGODE [13] 16.81 25.97 10.62 4.73 7.60 11.71 1.77 3.89 4.02 DSTAGNN [23] 15.67 24.77 9.94 3.32 6.68 9.31 1.89 4.11 4.26 ST-WA [9] 15.41 24.62 9.94 3.65 7.56 10.35 2.01 4.57 4.70 ASTGNN [17] 15.98 25.67 9.97 N/A N/A N/A N/A N/A N/A AGCRN [3] 15.95 25.22 10.09 3.68 7.74 10.82 2.01 4.64 4.71 ST-Norm [10] 15.39 24.80 9.91 3.56 7.48 10.20 1.92 4.49 4.54 STID [37] 14.20 23.29 9.33 3.55 7.55 10.94 1.90 4.41 4.44 STEP* [38] 14.00 23.41 9.50 3.37 6.99 9.61 1.79 4.20 4.18 PDFormer [19] 13.58 23.51 9.05 3.62 7.47 10.91 1.91 4.43 4.51 STAEformer [30] 13.46 23.25 8.88 3.34 7.04 9.78 1.85 4.31 4.33 HimNet [12] 13.56 23.36 N/A N/A N/A N/A N/A N/A N/A STDN [5] 14.25 24.39 N/A N/A N/A N/A N/A N/A N/A STD-MAE* [14] 13.45 22.47 8.78 3.42 7.08 9.62 1.79 4.21 4.17 STOT (Our Study) 13.37 21.90 8.49 3.27 6.75 9.15 1.83 4.20 4.16 1. * Due to the use of fewer pre-training steps, these metrics may differ from those reported in their paper. Figure 3: Multi-step prediction comparison on PeMS03, PeMS04, PeMS07, and PeMS08. 5.2.2 Reconstruction Results and Visualization Figure 5 visualizes reconstruction accuracy on a randomly selected PeMS node, where STOT closely tracks the ground truth even under abrupt changes, such as the significant PeMS03 fluctuation during t∈[850,1100]t∈[850,1100], because its adaptive masking emphasizes spatially indistinguishable, high-variation timesteps. 5.2.3 Forecasting Results and Visualization Figure 6 shows forecasting accuracy on a randomly selected node, where predicted and true values align closely; the gain over the baseline without pre-training confirms that pre-training focuses the model on challenging nodes and improves its capture of complex spatiotemporal patterns. 5.3 Ablation Study (RQ2) This section first evaluates the mask pre-training strategy in Section 5.3.1, then assesses STOT’s generalization ability in Section 5.3.2. 5.3.1 Mask Ablation To investigate each component of the designed mask pre-training strategy, we define five variants: w/o Random walk mask removes the post-OT random walk step and reduces mask-generation randomness; w/o Optimal transport constraints directly uses the most spatially indistinguishable positions as masks; w/o Batch consistency constraints allows masks at any position without penalty; w/o Spatial indistinguishability guidance removes this guidance and generates fully random masks; and w/o Mask applies no masked pre-training. Figure 4: Mask pre-training component ablation on PeMS04 and PeMS08. Figure 5: STOT reconstruction visualization on four datasets during pre-training. Figure 6: STOT forecasting visualization on four datasets after pre-training. Figure 7: Mask pre-training component ablation on PeMS03 and PeMS07. The ablation results in Figure 4 show that removing the final random walk reduces mask diversity and creates static masks for identical sequences, hindering spatial dependency capture and confirming its role in spatial understanding; OT-based softening with a temperature parameter enables simultaneous temporal and spatial learning, replacing separate pre-training stages with an efficient single-step process; removing batch consistency causes severe degradation, which may indicate that semantic continuity is needed for coherent representations, accuracy, and robustness; eliminating spatial consistency supervision yields fully random masks and what may be trivial semantics, possibly reducing pre-training effectiveness and underscoring the need for guided mask generation; and removing masking strategies makes the model depend entirely on the predictor, highlighting the role of pre-training in accuracy, robustness, and generalization. 5.3.2 Predictor Ablation To demonstrate that STOT enhances diverse backbones, we use it as a pre-trained feature extractor for LSTM, ASTGCN, and GWNet. The results in Figure 7 show that STOT benefits downstream predictors despite architectural differences: before STOT, their performance differs substantially, whereas after integration the gap narrows, indicating stronger feature extraction, deeper spatiotemporal dependency modeling, and more accurate and robust prediction. 5.4 High Spatial Indistinguishability Prediction Performance (RQ3) To evaluate performance under high spatial indistinguishability, we test STOT on PeMS04-s, a PeMS04 subset whose nodes have above-average spatial indistinguishability (Table 5). Table 6 summarizes the averaged results from three random experiments for all models, where STOT consistently outperforms baselines in longer-step prediction, captures long-term dependencies better, and maintains robust, stable performance under high indistinguishability, demonstrating reliable modeling of complex spatiotemporal dynamics when spatial patterns are difficult to discern. Table 5: Dataset segmentation details. Datasets #Sensors #Edges #Training #Validation #Testing PeMS04 (IN) 307 340 [10173, 12, 307,1] [3375, 12, 307,1] [3375, 12, 307,1] PeMS04 (OUT) 307 340 [10173, 12, 307,1] [3375, 12, 307,1] [3375, 12, 307,1] PeMS04-s (IN) 161 188 [10173, 12, 161,1] [3375, 12, 161,1] [3375, 12, 161,1] PeMS04-s (OUT) 161 188 [10173, 12, 161,1] [3375, 12, 161,1] [3375, 12, 161,1] Table 6: High spatial indistinguishability prediction on PeMS04. Model Metrics #1 step #3 steps #6 steps #12 steps STID MAE 16.88±0.03 18.12±0.02 19.12±0.00 21.10±0.25 RMSE 27.29±0.05 29.31±0.03 30.82±0.01 33.32±0.27 MAPE (%) 11.19±0.25 12.07±0.23 12.54±0.11 13.76±0.05 ST-Norm MAE 16.26±0.02 18.23±0.03 19.58±0.07 21.66±0.14 RMSE 26.20±0.02 30.00±0.03 32.38±0.07 35.37±0.17 MAPE (%) 7.18±0.21 8.01±0.23 8.41±0.27 9.55±0.32 STEP MAE 16.88±0.02 18.13±0.01 19.13±0.00 20.88±0.10 RMSE 27.25±0.00 29.31±0.01 30.85±0.01 33.11±0.07 MAPE (%) 11.66±0.16 12.46±0.10 12.97±0.08 14.50±0.35 STD-MAE MAE 16.86±0.03 18.10±0.03 19.00±0.00 20.40±0.03 RMSE 27.30±0.03 29.27±0.03 30.68±0.04 32.62±0.11 MAPE (%) 11.34±0.37 12.19±0.22 12.95±0.16 13.86±0.17 STOT MAE 16.54±0.01 17.73±0.01 18.57±0.01 19.89±0.05 RMSE 26.85±0.00 28.87±0.00 30.23±0.03 31.97±0.11 MAPE (%) 10.79±0.09 11.75±0.11 12.17±0.11 13.30±0.09 5.5 Hyperparameter Sensitivity (RQ4) We investigate STOT’s sensitivity to learning rate, hidden dimension, and encoder depth. For Learning Rate, we test 1e-3, 3e-3, 5e-3, 1e-2, and Figure 8(a) shows that accuracy decreases as the rate rises from 1e-3 to 1e-2, with 1e-3 best balancing convergence speed and stability. For Hidden Dimension, we evaluate 32, 48, 96, 128, and Figure 8(b) shows peak performance at 96, while larger dimensions slightly degrade results, suggesting that overly high-dimensional representations may introduce noise. For Number of Encoder Layers, we vary depth over 1, 2, 3, 4, and Figure 8(c) shows that 2 layers perform best, indicating sufficient capacity for essential spatiotemporal dependencies while deeper encoders provide limited additional benefit. Figure 8: STOT parameter sensitivity on PeMS04 under MAE, RMSE, and MAPE. 5.6 Computation Cost (RQ5) Although STOT’s autoencoder-based spatiotemporal representation learning may raise efficiency concerns, its dynamic mask strategy mitigates them. Figure 9 reports per-sample pre-training and forecasting time on four datasets, showing that STOT is more efficient than other pre-training models, reduces spatial representation learning cost, and runs nearly twice as fast as STD-MAE while balancing competitive performance on the evaluated benchmarks and practical feasibility. For reproducibility, the timing comparison in Figure 9 was measured on a single NVIDIA RTX A6000 GPU with a batch size of 8 for every model, matching the hardware reported in our experimental settings. Figure 9: Time cost of different approaches. We further analyzed how mask generation scales with the number of sensor nodes, since the Sinkhorn solver is only one substep of the full mask generation pipeline. Table 7 reports, for each dataset, the node count together with the time of the complete mask generation and of the Sinkhorn substep alone, measured per batch on a single A6000 GPU. The full mask generation grows with the node count and reaches about 5.04 ms per batch on PeMS07 with 883 nodes, while the Sinkhorn substep stays close to 0.214 to 0.216 ms across all datasets. This stability arises because the optimal transport is solved on the temporal patch dimension after the node dimension has already been aggregated into the TscoreTscore, so the solver cost is largely independent of the network size. These measurements indicate that the masking stays practical for large-scale urban sensor networks. Table 7: Mask generation time and the Sinkhorn substep time, in milliseconds per batch, across datasets with different numbers of nodes. Dataset Nodes Mask generation Sinkhorn PeMS08 170 1.3466±0.08521.3466± 0.0852 0.2138±0.00360.2138± 0.0036 METR-LA 207 1.4644±0.12381.4644± 0.1238 0.2150±0.00350.2150± 0.0035 PeMS04 307 1.3837±0.08911.3837± 0.0891 0.2137±0.00090.2137± 0.0009 PeMS-BAY 325 1.3890±0.09631.3890± 0.0963 0.2149±0.00340.2149± 0.0034 PeMS03 358 1.4294±0.05561.4294± 0.0556 0.2154±0.00290.2154± 0.0029 PeMS07 883 5.0419±0.35145.0419± 0.3514 0.2157±0.00380.2157± 0.0038 5.7 OT Constraint Visualization (RQ6) To interpret STOT’s OT effectiveness, we visualize the cost matrix, time-varying features, OT-generated masks, and batch consistency effects. Figure 10(a) and Figure 10(b) show the cost matrix before and after batch consistency constraints, with input timesteps on the horizontal axis, a batch size of 8 on the vertical axis, and color intensity indicating the cost magnitude that controls the likelihood of masking each position. The original cost matrix in Figure 10(a) has a highly concentrated color distribution; after applying batch consistency in Figure 10(b), the distribution remains concentrated but gains more overlapping regions. These regions encourage masks at similar, though not identical, positions within a batch, which may enhance semantic understanding and improve downstream performance. Figure 10: Visualization of OT and batch consistency: (a) and (b) cost matrices before and after batch consistency for the first 72 timesteps (6 patches), (c) and (d) the final cost matrix and OT matrix, (e) transport-matrix heatmap, and (f) batch-mask correlation matrix showing the effect of batch consistency on mask generation. Figure 10(c) shows that spatial indistinguishability has clear temporal periodicity: the lowest costs occur during morning and evening peaks, indicating the most severe indistinguishability and the greatest prediction challenge, while weaker off-peak fluctuations are less likely to be bottlenecks. This illustrates how SI-guided masking emphasizes challenging peak-hour segments, helping STOT learn richer spatiotemporal dependencies and improve accuracy. Figure 10(d) shows that transport-matrix peaks align with the low-cost regions in Figure 10(c), validating the Sinkhorn algorithm and matching our design expectations. The peak positions also vary slightly across batches, with three distinct peaks in the time range [40, 60], which may reflect STOT’s batch-adaptive mask adjustment for more accurate prediction. The transport-matrix heatmap in Figure 10(e) intuitively shows preferred mask locations, where masks are continuous within batches and periodic across timesteps. Figure 10(f) further shows high inter-batch correlation among generated masks, which may facilitate optimization and enhance overall performance. Together, these visualizations support the effectiveness of STOT’s OT process: transport peaks align with low-cost regions, mask positions adjust dynamically across batches, masks remain continuous and periodic, and high inter-batch correlation improves optimization and prediction accuracy. These findings validate STOT’s design choices, clarify its internal behavior, and motivate future improvements for spatiotemporal prediction. We further quantified how the batch consistency weight λ in Equation 7 trades off consistency against diversity. For a fixed training batch we regenerated the masks across a range of λ without retraining, measured consistency as the average pairwise Jaccard similarity among the batch masks, used its complement as diversity, and combined the normalized diversity with the mean TscoreTscore of the selected patches into a balance score. As reported in Figure 11, the balance score stays near its peak while λ remains small and declines once λ exceeds 0.7, while consistency keeps rising and diversity keeps shrinking. The default λ=0.7λ=0.7 therefore sits at the point where the masks stay diverse enough to cover varied scenarios while the consistency term already aligns them across the batch, and this behavior holds on all six datasets. Figure 11: Consistency, diversity, and balance scores as functions of the batch consistency weight λ on the six datasets. The dashed line marks the default λ=0.7λ=0.7. 5.8 Random Walk Mechanism for Mask Generation (RQ7) Figure 12: Comparison of random walk mechanism variants on PeMS04. To assess the random walk design, we test alternative implementations that operate on different positions or aspects and could substitute for the original mechanism: 1. w/o Patch-based: The smallest unit for the random walk is a single time step, as opposed to a patch of time steps. 2. w/o Batch-based: The random walk does not apply simultaneously to the entire batch, but rather to individual samples within the batch. 3. w/o After OT: The random walk is performed before the OT process, instead of after it. Figure 12 reports training and testing losses for different random walk designs, leading to the following conclusions: 1. w/o Patch-based: Applying random walk at the timestep level simplifies inter-timestep semantics and makes deep representation learning harder; however, because batch consistency remains, optimization is not strongly affected, and the model resembles the T-MAE design in STD-MAE, achieving relatively good results among variants. 2. w/o Batch-based: Disrupting batch consistency within one optimization step produces fragmented and possibly less meaningful semantics, so this design may underperform the Patch-based approach that preserves semantic continuity. 3. w/o After OT constraint: Performing random walk before OT makes randomness depend on cost-matrix softening; temperature adjustment partly mitigates this, but mask-generation randomness is still greatly reduced, possibly leading to redundant semantics and the lowest performance. Overall, all three variants converge to nearly identical training losses, which may indicate that they can almost fully reconstruct the sequence, while their performance differences arise from where random walk is applied. Our design avoids the above issues and enables comprehensive spatiotemporal modeling. 6 Conclusion This paper introduced STOT, a self-supervised spatiotemporal prediction framework that improves representation learning under the common but underexplored challenge of spatial indistinguishability. Instead of treating masking as random or heuristic, STOT uses an optimal transport (OT)-based strategy to construct training signals aligned with intrinsic spatial structure. Across multiple real-world benchmarks, STOT improves over strong baselines, suggesting that explicitly modeling ambiguity among spatially similar regions may offer a forecasting advantage. A key strength of our approach is that the OT-based masking mechanism provides both effectiveness and interpretability. Empirically, the strongest gains appear during peak hours, where demand patterns are volatile and prediction becomes most difficult, suggesting that the proposed masking strategy is especially beneficial when the model must discriminate among multiple plausible spatial explanations. The transport matrices offer an interpretable lens into how the model perceives "substitutable" or low-cost spatial regions, and the observed alignment between low-cost regions and high transport mass supports the intuition that OT can serve as a principled tool for constructing challenging but semantically meaningful self-supervised objectives. The batch consistency constraint further strengthens the learned representations by encouraging coherent semantics while allowing sufficient diversity, helping to stabilize training and improve generalization. Since the reconstruction objective draws on a longer context and a direct target, it is easier to optimize than forecasting, so the pre-training losses of different variants settle at close values, and the downstream gains instead come from masking that concentrates the reconstruction on the spatially indistinguishable nodes. However, our work has several limitations. The computational overhead introduced by OT can become a bottleneck when the number of spatial units grows or when extremely long sequences are used, raising scalability concerns for real-world deployments on city-scale or nation-scale grids. Additionally, STOT focuses primarily on endogenous spatiotemporal signals and may not fully capture the causal drivers of variation when exogenous variables (e.g., weather, holidays, accidents) dominate the dynamics. Finally, the current analysis does not fully characterize when OT-based masking is most beneficial, leaving open questions about robustness under distribution shifts. In addition, the optimal transport objective is easier to optimize under a moderate mask ratio, so STOT does not exploit the high mask ratios that benefit single-stage masking methods such as STEP [38], which leaves room for transport formulations that stay stable under heavier masking. Beyond performance, STOT offers two practical contributions: it provides a general framework for self-supervised objectives that address spatial ambiguity in traffic and environmental monitoring, with OT-based masking usable as a plug-in strategy under limited labels; and its transport matrices act as interpretable diagnostics for spatial similarity, data quality assessment, and decision policy design. Future work includes faster OT approximations for large-scale graphs, exogenous covariates for robustness, and extensions to anomaly detection, imputation, and cross-city transfer. CRediT authorship contribution statement Guangyu Wang: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Visualization. Jiawei Tong: Conceptualization, Methodology, Writing – review & editing, Supervision, Project administration. Acknowledgements The authors would like to thank Chen Yu for language polishing and proofreading, and Yujie Chen for assistance with manuscript formatting and typesetting. References [1] T. Afrin and N. Yodo (2020) A survey of road traffic congestion measures towards a sustainable and resilient transportation system. Sustainability, p. 4660. Cited by: §5.1.1. [2] M. Awad, R. Khanna, M. Awad, and R. Khanna (2015) Support vector regression. Efficient learning machines: Theories, concepts, and applications for engineers and system designers, p. 67–80. Cited by: §1. [3] L. Bai, L. Yao, C. Li, X. Wang, and C. Wang (2020) Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Information Processing Systems 33, p. 17804–17815. Cited by: §5.1.2, Table 3, Table 4. [4] H. Bao, L. Dong, S. Piao, and F. Wei (2021) BEiT: bert pre-training of image transformers. In International Conference on Learning Representations, Cited by: §2.3. [5] L. Cao, B. Wang, G. Jiang, Y. Yu, and J. Dong (2025) Spatiotemporal-aware trend-seasonality decomposition network for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 39 (11), p. 11463–11471. Cited by: §5.1.2, Table 3, Table 4. [6] S. R. Chandra and H. Al-Deek (2009) Predictions of freeway traffic speeds and volumes using vector autoregressive models. Journal of Intelligent Transportation Systems 13 (2), p. 53–72. Cited by: §5.1.2, Table 3, Table 4. [7] J. Choi, H. Choi, J. Hwang, and N. Park (2022) Graph neural controlled differential equations for traffic forecasting. In AAAI, Cited by: §5.1.1. [8] R. Cirstea, T. Kieu, C. Guo, B. Yang, and S. J. Pan (2021) EnhanceNet: plugin neural networks for enhancing correlated time series forecasting. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), p. 1739–1750. Cited by: §5.1.1. [9] R. Cirstea, B. Yang, C. Guo, T. Kieu, and S. Pan (2022) Towards spatio-temporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), p. 2900–2913. Cited by: §2.2, §5.1.2, Table 3, Table 4. [10] J. Deng, X. Chen, R. Jiang, X. Song, and I. W. Tsang (2021) ST-norm: spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, p. 269–278. Cited by: §1, §2.2, §5.1.2, Table 3, Table 4. [11] J. Dong, H. Wu, H. Zhang, L. Zhang, J. Wang, and M. Long (2023) SimMTM: a simple pre-training framework for masked time-series modeling. In Advances in Neural Information Processing Systems, Cited by: §2.3. [12] Z. Dong, R. Jiang, H. Gao, H. Liu, J. Deng, Q. Wen, and X. Song (2024) Heterogeneity-informed meta-parameter learning for spatiotemporal time series forecasting. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 631–641. Cited by: §2.2, §5.1.2, Table 3, Table 4. [13] Z. Fang, Q. Long, G. Song, and K. Xie (2021) Spatial-temporal graph ode networks for traffic flow forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, p. 364–373. Cited by: §5.1.2, Table 3, Table 4. [14] H. Gao, R. Jiang, Z. Dong, J. Deng, Y. Ma, and X. Song (2024) Spatial-temporal-decoupled masked pre-training for spatiotemporal forecasting. Cited by: §2.3, §4.1, §5.1.2, Table 3, Table 4. [15] A. Genevay, G. Peyré, and M. Cuturi (2018) Learning generative models with sinkhorn divergences. In International Conference on Artificial Intelligence and Statistics, p. 1608–1617. Cited by: §4.3.1. [16] S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan (2019) Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33, p. 922–929. Cited by: §4.4, §5.1.2, Table 3, Table 4. [17] S. Guo, Y. Lin, H. Wan, X. Li, and G. Cong (2021) Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering. Cited by: §5.1.2, Table 3, Table 4. [18] Z. He, C. Chow, and J. Zhang (2021) STNN: a spatio-temporal neural network for traffic predictions. IEEE Transactions on Intelligent Transportation Systems 22 (12), p. 7642–7651. Cited by: §2.1. [19] J. Jiang, C. Han, W. X. Zhao, and J. Wang (2023) PDFormer: propagation delay-aware dynamic long-range transformer for traffic flow prediction. In AAAI, Cited by: §5.1.2, Table 3, Table 4. [20] Z. Jiang, C. Liu, A. Akintayo, G. P. Henze, and S. Sarkar (2017) Energy prediction using spatiotemporal pattern networks. Applied Energy 206, p. 1022–1039. Cited by: §1. [21] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T. Liu (2017) Lightgbm: a highly efficient gradient boosting decision tree. Advances in neural information processing systems 30. Cited by: §1. [22] K. N. Kumar, D. Roy, T. A. Suman, C. Vishnu, and C. K. Mohan (2024) TSANet: forecasting traffic congestion patterns from aerial videos using graphs and transformers. Pattern Recognition 155, p. 110721. External Links: ISSN 0031-3203 Cited by: §2.1. [23] S. Lan, Y. Ma, W. Huang, W. Wang, H. Yang, and P. Li (2022) Dstagnn: dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In International conference on machine learning, p. 11906–11917. Cited by: §5.1.2, Table 3, Table 4. [24] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut (2019) Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. Cited by: §2.3. [25] S. Lee, T. Park, and K. Lee (2024) Learning to embed time series patches independently. External Links: 2312.16427 Cited by: §2.3, §4.1. [26] Y. Li, R. Yu, C. Shahabi, and Y. Liu (2018) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In International Conference on Learning Representations, Cited by: §1, §5.1.2, Table 3, Table 4. [27] Y. Li, Z. Shao, Y. Xu, Q. Qiu, Z. Cao, and F. Wang (2023) Dynamic frequency domain graph convolutional network for traffic forecasting. External Links: 2312.11933 Cited by: §2.1. [28] Z. Li, Z. Rao, L. Pan, P. Wang, and Z. Xu (2023) Ti-mae: self-supervised masked time series autoencoders. External Links: 2301.08871 Cited by: §2.3. [29] J. Lin, Q. Ren, X. Lv, H. Xu, and Y. Liu (2025) When multi-view meets multi-level: A novel spatio-temporal transformer for traffic prediction. Information Fusion 117, p. 102801. External Links: ISSN 15662535 Cited by: §2.1. [30] H. Liu, Z. Dong, R. Jiang, J. Deng, J. Deng, Q. Chen, and X. Song (2023) Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, p. 4125–4129. Cited by: §2.2, §4.4, §5.1.2, Table 3, Table 4. [31] Q. Lv, L. Liu, R. Yang, and Y. Wang (2025) Multimodal urban traffic flow prediction based on multi-scale time series imaging. Pattern Recognition 164, p. 111499. External Links: ISSN 0031-3203 Cited by: §1. [32] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam (2023) A time series is worth 64 words: long-term forecasting with transformers. External Links: 2211.14730 Cited by: §4.2.1. [33] S. Pal, L. Ma, Y. Zhang, and M. Coates (2021) RNN with particle flow for probabilistic spatio-temporal forecasting. In International Conference on Machine Learning, p. 8336–8348. Cited by: §1. [34] S. J. Rigatti (2017) Random forest. Journal of Insurance Medicine 47 (1), p. 31–39. Cited by: §1. [35] C. Shang, J. Chen, and J. Bi (2021) Discrete graph structure learning for forecasting multiple time series. External Links: 2101.06861 Cited by: §2.1. [36] Z. Shao, F. Wang, Y. Xu, W. Wei, C. Yu, Z. Zhang, D. Yao, T. Sun, G. Jin, X. Cao, G. Cong, C. S. Jensen, and X. Cheng (2025) Exploring progress in multivariate time series forecasting: comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering 37 (1), p. 291–305. External Links: Document Cited by: §1, §2.1, §4.3.1, §5.1.3, §5.1.5. [37] Z. Shao, Z. Zhang, F. Wang, W. Wei, and Y. Xu (2022) Spatial-temporal identity: a simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, p. 4454–4458. Cited by: §1, §1, §2.2, §5.1.2, Table 3, Table 4. [38] Z. Shao, Z. Zhang, F. Wang, and Y. Xu (2022) Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, p. 1567–1577. Cited by: §2.3, §5.1.2, Table 3, Table 4, §6. [39] G. Wang and Z. Wang (2026) Net-ev2^2: a generative simulator for network event evolution. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, p. 4836–4847. External Links: Document Cited by: §2.2. [40] J. Wang, J. Ji, Z. Jiang, and L. Sun (2023) Traffic Flow Prediction Based on Spatiotemporal Potential Energy Fields. IEEE Transactions on Knowledge and Data Engineering 35 (9), p. 9073–9087. External Links: ISSN 1041-4347, 1558-2191, 2326-3865 Cited by: §1. [41] B. M. Williams, P. K. Durvasula, and D. E. Brown (1998) Urban freeway traffic flow prediction: application of seasonal autoregressive integrated moving average and exponential smoothing models. Transportation Research Record 1644 (1), p. 132–141. Cited by: §5.1.2, Table 3, Table 4. [42] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang (2019) Graph wavenet for deep spatial-temporal graph modeling.. In IJCAI, Cited by: §4.4, §5.1.2, §5.1.3, Table 3, Table 4. [43] L. Xu, N. Chen, Z. Chen, C. Zhang, and H. Yu (2021) Spatiotemporal forecasting in earth system science: methods, uncertainties, predictability and future directions. Earth-Science Reviews 222, p. 103828. External Links: ISSN 0012-8252 Cited by: §1. [44] B. Yu, H. Yin, and Z. Zhu (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, p. 3634–3640. Cited by: §2.1, §5.1.2, Table 3, Table 4. [45] X. Yuan and Y. Qiao (2024) Diffusion-TS: interpretable diffusion for general time series generation. In The Twelfth International Conference on Learning Representations, Cited by: §2.2. [46] G. Zou, Z. Zhou, R. Weibel, Y. Li, T. Wang, Z. Liu, W. Ding, and C. Fu (2026) Multi-graph spatio-temporal network for traffic accident risk forecasting. Pattern Recognition 172, p. 112784. External Links: ISSN 0031-3203 Cited by: §1.