Paper deep dive
Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training
Wei Chen, Junle Chen, Yuqian Wu, Yuxuan Liang, Xiaofang Zhou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/20/2026, 8:37:22 PM
Summary
The paper introduces ST-Prune, a dynamic sample pruning framework for spatio-temporal forecasting that accelerates training by identifying and retaining informative, complex samples while pruning redundant ones. It addresses specific challenges like the 'Averaging Masking Effect' and 'Long-tail Stationarity Distribution' through a complexity-informed scoring metric and stationarity-aware gradient rescaling, demonstrating improved efficiency and performance on datasets like PeMS08 and UrbanEV.
Entities (10)
Relation Signals (9)
ST-Prune → addresses → Long-tail Stationarity Distribution
confidence 95% · To rectify this distribution shift, we propose a Stationarity-Aware Rescaling strategy.
ST-Prune → addresses → Averaging Masking Effect
confidence 95% · To address this, we propose Spatio-Temporal Complexity Scoring... ensuring that samples with high spatial heterogeneity... are preserved
ST-Prune → evaluatedon → PEMS08
confidence 95% · We evaluate our method on representative ST benchmarks: PEMS08... and UrbanEV
ST-Prune → evaluatedon → UrbanEV
confidence 95% · We evaluate our method on representative ST benchmarks: PEMS08... and UrbanEV
ST-Prune → improves → Training Efficiency
confidence 95% · ST-Prune significantly accelerates the training speed while maintaining or even improving the model performance
ST-Prune → appliedto → GWNet
confidence 90% · We employ GWNet (Wu et al., 2019) as the default backbone
ST-Prune → appliedto → STAEformer
confidence 90% · alongside STAEformer (Liu et al., 2023)... for cross-architecture evaluation
ST-Prune → appliedto → OpenCity
confidence 90% · and the foundation model OpenCity (Li et al., 2025b) to assess scalability.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spatio-temporal forecasting is fundamental to intelligent systems in transportation, climate science, and urban planning. However, training deep learning models on the massive, often redundant, datasets from these domains presents a significant computational bottleneck. Existing solutions typically focus on optimizing model architectures or optimizers, while overlooking the inherent inefficiency of the training data itself. This conventional approach of iterating over the entire static dataset each epoch wastes considerable resources on easy-to-learn or repetitive samples. In this paper, we explore a novel training-efficiency techniques, namely learning from complexity with dynamic sample pruning, ST-Prune, for spatio-temporal forecasting. Through dynamic sample pruning, we aim to intelligently identify the most informative samples based on the model's real-time learning state, thereby accelerating convergence and improving training efficiency. Extensive experiments conducted on real-world spatio-temporal datasets show that ST-Prune significantly accelerates the training speed while maintaining or even improving the model performance, and it also has scalability and universality.
Tags
Links
- Source: https://arxiv.org/abs/2602.19113v2
- Canonical: https://arxiv.org/abs/2602.19113v2
Trouble viewing inline? Open PDF directly →
Full Text
72,728 characters extracted from source content.
Expand or collapse full text
Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training Wei Chen 1,2 , Junle Chen 2 , Yuqian Wu 1 , Yuxuan Liang 1 ∗ , Xiaofang Zhou 2 ∗ 1 HKUST(GZ), 2 HKUST Spatio-temporal forecasting is fundamental to intelligent systems in transportation, climate science, and urban planning. However, training deep learning models on the massive, often redundant, datasets from these domains presents a significant computational bottleneck. Existing solutions typically focus on optimizing model architectures or optimizers, while overlooking the inherent inefficiency of the training data itself. This conventional approach of iterating over the entire static dataset each epoch wastes considerable resources on easy-to-learn or repetitive samples. In this paper, we explore a novel training-efficiency techniques, namely learning from complexity with dynamic sample pruning,ST-Prune, for spatio-temporal forecasting. Through dynamic sample pruning, we aim to intelligently identify the most informative samples based on the model’s real-time learning state, thereby accelerating convergence and improving training efficiency. Extensive experiments conducted on real-world spatio-temporal datasets show thatST-Prunesignificantly accelerates the training speed while maintaining or even improving the model performance, and it also has scalability and universality. Correspondence: onedeanxxx@gmail.com, ∗ yuxliang@outlook.com, ∗ zxf@cse.ust.hk Date: Jan 28, 2026 1 Introduction With the rapid advancement of sensing technologies, massive volumes of spatio-temporal data are being collected and leveraged in data-driven forecasting scenarios, enabling critical services ranging from traffic management to weather forecasting and power grid operations (Avila and Mezić, 2020; Nguyen et al., 2023). To capture the complex nonlinear dynamics inherent in such data, spatio-temporal neural networks (Jin et al., 2023, 2024) have become the dominant and most powerful paradigm. However, while research community has largely focused on designing increasingly sophisticated architectures for marginal performance gains (Shao et al., 2024), a fundamental and costly bottleneck has been overlooked: the training process itself. Standard spatio-temporal training protocols require iterating over the entire samples in every training epoch (Liu et al., 2024b). As a result, the widely adopted benchmarks (Li et al., 2018; Song et al., 2020) are typically restricted to limited spatial regions and temporal spans, severely limiting the scalability of spatio-temporal nueral networks training and inflating their computational cost. This raises a natural research question: do we really need to compute over all available spatio-temporal samples during training stage? To answer this question, we first conduct a detailed analysis of the widely used spatio-temporal benchmark (PeMS08). As shown in Figure 1, the left panel visualizes the Pearson correlation matrix across spatial nodes, revealing that the vast majority exhibit high similarity (≥0.8). Even node pairs with lower similarity still display recurring periodic temporal patterns, as illustrated in the middle panel. Principal component analysis (PCA) along both spatial and temporal dimensions (right panel) further shows that a small number of components are sufficient to reconstruct most of the variance. Together, these statistics reveal a high degree of spatio-temporal redundancy, which opens up new opportunities for more efficient utilization of training data. Recent data-centric acceleration research (Zha et al., 2025) have devoted significant effort to identifying unbiased or representative subsets within training datasets. Typical techniques include data pruning (Raju et al., 2021; Wang et al., 2023; Qin et al., 2024; Zhang et al., 2024b,a; Wang et al., 2025), dataset distillation (Nguyen et al., 1 arXiv:2602.19113v2 [cs.LG] 2 Mar 2026 020406080100120140160 Spatial Node Index 0 20 40 60 80 100 120 140 160 Spatial Node Index Spatial Correlation Between Nodes 0.2 0.4 0.6 0.8 1.0 Pearson Correlation Mon.Tue.Wed.Thu.Fri.Sat.Sun. Date 0 100 200 300 400 500 600 700 800 Traffic Flow Periodic Repeating Temporal Patterns Line 1 Line 2 0255075100125150175 # Principal Components 0.825 0.850 0.875 0.900 0.925 0.950 0.975 1.000 Cumulative Explained Variance PCA in Spatial and Temporal Dimensions Spatial PCA Temporal PCA 90% variance Figure 1 The spatio-temporal data redundancy characteristics and statistical properties along the spatial and temporal dimensions, exemplified by the PeMS08 (Song et al., 2020) dataset. For more statistical information on other datasets, please refer to Appendix A.1. Spatial Averag e Temporal Average Avg: 2899.05 Std: 438.08 Avg: 204.63 Std: 114.17 Average Spatial Averag e Temporal Average Avg: 2918.55 Std: 559.39 Avg: 206.01 Std: 145.75 Average Global Hardness: 17.05 Global Hardness: 17.16 ≈ Spatial Complexity Temporal Complexity (a) Illustration of the Averaging Masking Effect. STErrorMapofSampleB STErrorMapofSampleA ≠ ≠ (b) Distribution of Dynamic Intensity & Patterns of Selected Samples Figure 2 Further analysis of spatio-temporal data insights. (a) Averaging Masking Effect: Low-error nodes dilute critical localized anomalies, necessitating a structural scoring mechanism beyond simple mean error. (b) Long-tail Stationarity Distribution: The dominance of stationary patterns motivates our stationarity-aware rescaling to prevent distribution shift and maintain representativeness. 2021; Wang et al., 2022; Cazenavette et al., 2022; Zhao and Bilen, 2023; Li et al., 2023; Zhang et al., 2024c), and coreset selection (Har-Peled and Mazumdar, 2004; Chen, 2009; Toneva et al., 2018; Shim et al., 2021), which respectively retain, synthesize, or select a smaller yet information-rich subset from the original data. Despite their promise, these methods are primarily developed for computer vision and natural language processing tasks and fail to fully exploit the unique redundancy characteristics of spatio-temporal data highlighted above. To address this critical gap, we proposeST-Prune, a novel dynamic sample pruning framework specifically tailored for spatio-temporal training. Distinct from traditional paradigms that passively process the entire dataset,ST-Pruneactively curates high-value data subsets to optimize training efficiency. Our fundamental insight is that directly applying generic dynamic pruning strategies (Raju et al., 2021; Qin et al., 2024; Moser et al., 2025) to spatio-temporal data proves ineffective. This failure stems from two specific phenomena: the Averaging Masking Effect (Figure 2 a) , where critical localized failures are obscured by low global errors, and the inherent Long-tail Stationarity Distribution (Figure 2 b), where standard pruning induces severe distribution shifts. To this end,ST-Pruneoperationalizes these insights via two innovative components: complexity-informed scoring metric, which incorporates a spatio-temporal heterogeneity penalty to identify structural samples that appear ”globally trivial yet locally intractable“; and stationarity-aware gradient rescaling, which adaptively adjusts weights based on dynamic intensity. This ensures that the preserved samples maintain an unbiased representation of the original data distribution while discarding substantial redundant stationary samples. Ultimately, this design not only significantly mitigates computational overhead but also ensures the robust capture of diverse and complex spatio-temporal patterns. In summary, our contributions are: •We proposeST-Prune, a dynamic sample pruning method of spatio-temporal training, that shifts the spatio-temporal research focus from solely optimizing the model to intelligently optimizing the data flow 2 during training. •We design a novel framework consisting of a complexity-informed difficulty metric to assess sample informativeness in real-time and an stationarity-aware distribution rescaling to ensure stable and effective training. •Extensive experiments on multiple real-world spatio-temooral datasets demonstrate thatST-Prunedrastically reduces training time while maintaining or even improving the predictive accuracy of various backbones, proving its effectiveness, efficiency, and universality. 2 Related Work Efficient Spatio-Temporal Training. Spatio-temporal forecasting tasks typically rely on neural networks that integrate spatial and temporal operators (Chen and Liang, 2025a,b) to capture spatial dependencies and temporal dynamics. However, the large number of nodes and extended time horizons (Liu et al., 2024b; Yin et al., 2025) result in substantial computational costs during training. Previous studies have sought efficiency improvements through model-level optimizations, such as prior structure Shao et al. (2022); Cini et al. (2023); Fu et al. (2025), graph sparsification (Wu et al., 2025; Ma et al., 2025), and model distillation (Tang et al., 2024; Chen et al., 2025), or sample-level transformations, including subgraph (Jiang et al., 2023; Liu et al., 2024a; Wang et al., 2024; Weng et al., 2025; Zhao et al., 2026) and input window (Fang et al., 2025; Ockerman et al., 2025; Shao et al., 2025) sampling. While these approaches reduce computational demand at the model or optimization level, they rarely provide explicit control over the number or distribution of training samples and often tightly couple network component design. In contrast, we aim to develop a general dynamic spatio-temporal sample pruning strategy that accelerates the training of arbitrary spatio-temporal neural networks while maintaining competitive performance. Data-Centric Acceleration Strategy. Data-centric acceleration strategies (Zha et al., 2025) have drawn considerable attention because they enhance model training and inference efficiency from a data management perspective. Representative approaches include data distillation (Lei and Tao, 2023; Yu et al., 2023), data pruning, and data selection (Moser et al., 2025). Data distillation seeks to compress the original data distribution into a smaller yet information-equivalent synthetic dataset to reduce training cost, whereas data pruning and selection reduce computational burden by removing redundant samples or selecting the most representative subsets. However, these methods are primarily designed for vision (Wang et al., 2018) or language (Albalak et al., 2024) tasks and struggle to exploit the high redundancy and strong correlations inherent in spatio-temporal data. Moreover, they are often static and sensitive to sample size (Guo et al., 2022), which may limit generalization under dynamically evolving spatio-temporal distributions. Notably, our method exploits the complexity of ST samples and dynamically adjusts sample selection to maximize computational efficiency. 3 Preliminaries Probelm Definition (Spatio-Temporal Forecasting). From an optimization perspective, spatio-temporal forecasting aims to solve an empirical risk minimization problem. Given a large-scale spatio-temporal training datasetD=(X i ,Y i ) |D| i=1 (and an optional graph structure priorG), the objective is to find the optimal model parameters θ ∗ by minimizing a global loss function J (θ;D): min θ J (θ;D) = 1 |D| |D| X i=1 L(f θ (X i ;G),Y i ),(1) wheref θ is the forecasting spatio-temporal neural network, andLis a loss metric (e.g., MAE). (X i ∈ R N×T p ×F ,Y i ∈ R N×T f ×F ) represent the sensor readings ofCmeasurement features atNspatial locations overT p past andT f future consecutive time steps, respectively. This formulation treats every sample (X i ,Y i ) equally and implies that the gradient for each training epoch is computed over the entire, often redundant, datasetD. This exhaustive summation is the primary source of computational inefficiency in the spatio-temporal training process. 3 Task Definition (Dynamic Sample Pruning). We formulate dynamic sample pruning as an optimization problem aimed at approximate the convergence of the objective in Equation 1. Instead of using the full datasetDat each epoche, we seek to identify an optimal training subsetD ∗ e ⊂ D with a constrained size |D e |=k e <|D|. The model parameters are then updated based only on this subset:θ e =Update(θ e−1 ,D e ). Ideally, the optimal subsetD ∗ e should be the one that provides the steepest descent direction for the true, underlying data distribution P: D ∗ e =argmin D e ⊂D,|D e |=k e E (X,Y )∼P [L(f θ e (X;G),Y )].(2) However, minimizing this objective is intractable as the true distributionPis unknown. However, we have empirically shown above that this distribution exhibits a significant low-rank property, that is, the training samples are highly redundant. Thus, the core challenge of dynamic sample pruning is to design a tractable and efficient proxy strategy to select an informative subset D e at each epoch. 4 Methodology Initialize: Model 휃, Score Memory ℋ, Weights 푊 Prune Easy Samples Update Weights for Unbiasedness Set Active Batch 퐼 ! Full-Sample Supervised Annealing Forward & Compute Loss Update Score Memory Compute Weighted Objective No Phase 1: Complexity-Informed Pruning Phase 2: Stability-Guided Optimization Backward & Update 휃 EasySamples Easy Samples Hard Samples C Unbiased Expectation Rescaling Ensures unbiased gradient expectation: 퐸∇푗≈∇̃횥 Sampled & Rescaled (After) After Pruning Yes HardSamples Before Pruning Random Sampling (p) (a) OverviewofST-Pruneworkflow Reweighting (×푤 ! ) Epoch ≤훿⋅퐸 (b) Complexity-Informed Pruning Figure 3 The overall workflow of the ST-Prune for efficient data pruning during spatio-temporal training. 4.1 Complexity-Informed Pruning 4.1.1 Beyond The Averaging Masking Effect Before elaborating on our proposed framework, we first analyze the limitations of standard dynamic pruning techniques (Katharopoulos and Fleuret, 2018; Raju et al., 2021; Qin et al., 2024) when applied to spatio- temporal domains. Existing methods typically quantify sample difficulty using a global loss metric,ℓ i = ∥f θ (X i )−Y i ∥, where a lowerℓ i indicates an “easy” sample. However, in spatio-temporal forecasting, the scalar lossℓ i is an aggregate metric averaged overNspatial nodes andTtime steps. This aggregation introduces a critical pathology we term the Averaging Masking Effect.We empirically demonstrate this phenomenon in Figure 2 (a), which visualizes the error landscapes of two distinct samples from a real-world traffic dataset: •Sample A (Global Noise): The error is distributed relatively uniformly across the spatio-temporal field. The global hardness (mean MAE) is calculated as 17.05. •Sample B (Local Anomaly): As highlighted by the red circle in Figure 2(a), this sample contains critical localized failures (e.g., severe congestion spikes at specific hubs), while the remaining nodes exhibit low errors. Surprisingly, its global hardness is 17.16. Despite the fundamental difference in structural information, standard pruning methods viewing only the global hardness (≈17.1) would treat these two samples as identical. Consequently, the informative Sample B, which drives the learning of local spatial dynamics, may risks being pruned as “easy” data. This observation confirms that magnitude-based metrics fail to distinguish structural complexity from background noise. To address this, we propose Spatio-Temporal Complexity Scoring, a metric designed to capture the non-uniformity (σ) of the error distribution, ensuring that samples with high spatial heterogeneity (like Sample B) are preserved even if their global mean is low. 4 4.1.2 Spatio-Temporal Complexity Scoring Formally, we proposeST-Prune. As shown in Fig. 3, during the forward propagation process, each sample retains a score related to its loss value. The average of these scores is set as the pruning threshold. In each epoch, a corresponding proportion of samples with low scores are pruned. To operationalize the insight from Figure 2(a), we propose a composite scoring function that explicitly penalizes performance heterogeneity. LetE (i) t ∈ R N×T denote the absolute error matrix for thei-th sample at epoch t. We define the structural informativeness score H t (i) as: H t (i) = μ(E (i) t ) |z Global Hardness +λ· σ space (E (i) t ) + σ time (E (i) t ) | z Spatio-Temporal Complexity ,(3) whereμ(·) denotes the global mean. The termsσ space andσ time represent the standard deviations calculated along the spatial and temporal dimensions, respectively. 4.1.3 Randomized Pruning Policy To reduce computational overhead while maintaining data diversity, we categorize samples into an Informative SetS inf and a Redundant SetS red at the beginning of each epocht. Let ̄ H t be the average score of the entire dataset. z i ∈ ( S inf , if H t (i)≥ ̄ H t S red , if H t (i) < ̄ H t (4) Unlike static pruning which permanently discardsS red , we adopt a randomized “soft” pruning strategy. All samples inS inf are retained. For samples inS red , we retain them with a probabilityp∈(0,1), determined by the target pruning ratio. This ensures that even currently ”easy“ samples have a chance to be revisited, preventing the model from catastrophic forgetting of basic patterns. 4.2 Stability-Guided Optimization 4.2.1 Stationarity-Aware Gradient Rescaling While the scoring metric identifies which samples to prune, a naive removal of “easy” samples would inevitably alter the training data distribution. As highlighted in Figure 2(b), spatio-temporal datasets exhibit a long-tail distribution of dynamic intensity: the vast majority of samples are stationary (low temporal variance), while high-dynamic events are rare. Standard pruning techniques Qin et al. (2024), which apply a uniform rescaling weightw= 1 1−r (whereris the pruning rate), fail to account for this imbalance. By disproportionately removing stationary samples, they shift the training distribution towards the “tail" (non-stationary events), causing the model to overfit to extreme dynamics and lose robustness on regular patterns. To rectify this distribution shift, we propose a Stationarity-Aware Rescaling strategy. We first quantify the dynamic intensityδ i of each sampleias the temporal variance of its ground truth targets:δ i =Var t (Y i ). For the subset of retained “inf" samplesS ′ inf , we assign an adaptive weightw i that is inversely proportional to their dynamic intensity: w i = 1 1− r · ̄ δ D δ i + ε α |z Stationarity Correction ,(5) where ̄ δ D is the global average intensity, andα≥0 controls the correction strength. This mechanism ensures that preserved stationary samples (low δ i ) receive higher weights, effectively representing the population of pruned stationary data. Conversely, highly dynamic samples (high δ i ) have been naturally retained through our scoring metric and will receive standard weights. Formally, the modified objective ̃ J(θ) for the current epoch becomes: ̃ J (θ) = X i∈S red L(x i ,y i ;θ) + X j∈S ′ inf w j ·L(x j ,y j ;θ).(6) 5 This reweighting guarantees that the gradient expectation remains unbiased not only in magnitude but also in terms of dynamic regime distribution, ensuring convergence consistency with the full dataset. 4.2.2 Training Schedule with Annealing While the unbiased estimator holds in expectation, the variance of the gradients inevitably increases due to downsampling. To mitigate this effect and ensure stable convergence in the final phase of training, we introduce a deterministic annealing strategy. Given a total training budget ofEepochs and a cutoff ratio δ(e.g.,δ= 0.9), we perform the pruning strategy only for the firstδ· Eepochs. For the remaining epochs t > δ· E, we revert to full-dataset training. This allows the model to fine-tune on all samples, eliminating any residual variance and ensuring that the final model performance is strictly lossless compared to the baseline. 5 Experiments In this section,we conduct extensive experiments to answer the following research questions (RQs): •RQ1: CanST-Pruneoutperform existing data pruning methods in various ST datasets? (Effectiveness) • RQ2: How does the efficiency of ST-Prune compare with that of the existing baselines? (Efficiency) •RQ3: How scalable isST-Prunewith respect to large-scale datasets and foundation models? (Scalability) • RQ4: Is ST-Prune effective across different types of backbones, optimizers, and tasks? (Universality) •RQ5: How doesST-Prunework? Which components play the key roles, and are they sensitive to parameter settings or component designs? (Mechanism & Robustness) 5.1 Experimental Setup Dataset and Evaluation Protocol. We evaluate our method on representative ST benchmarks: PEMS08 (Song et al., 2020) from the traffic domain and UrbanEV (Li et al., 2025a) from the energy domain, alongside the large-scale benchmark LargeST (Liu et al., 2024b) for scalability analysis. Following standard protocols (Shao et al., 2024; Jin et al., 2023), datasets are split chronologically (6:2:2) to forecast the next 12 steps given the past 12. To ensure rigorous efficiency comparison, we disable early stopping and train for a fixed 100 epochs. Each experiment was repeated five times and the mean was reported. Performance under different pruning levels is assessed via MAE, RMSE, and MAPE across data retention ratios of 10%, 30%, 50%, 70%. Further details of datasets and protocols in Appendix A.1. Baselines and Parameter Settings. We benchmark against a comprehensive suite of pruning strategies adapted for spatio-temporal tasks, categorized into: (1) Static methods: Random (Hard Random), geometry- based (CD, Herding, K-Means), uncertainty-based (Least Confidence, Entropy, Margin), loss-based (Forgetting, GraNd), decision boundary-based (Cal), bi-level (Glister), and submodular (GraphCut, FaLo); and (2) Dynamic methods: Random(Soft Random), uncertainty-based (ε-greedy, UCB), and loss-based (InfoBatch). We employ GWNet (Wu et al., 2019) as the default backbone, alongside STAEformer (Liu et al., 2023) and STID (Shao et al., 2022) for cross-architecture evaluation, and the foundation model OpenCity (Li et al., 2025b) to assess scalability. To ensure fair comparison, all models are trained using SGD (momentum 0.9, weight decay 1e-4) with a cosine annealing scheduler. More details on baselines, models, and parameters are provided in Appendix A.2. 5.2 Effectiveness Study (RQ1) Table 1 compares the MAPE results ofST-Pruneagainst representative data pruning and selection methods. The best results are highlighted in bold pink, and the second-best inunderlined in blue. Besides, we report the relative performance change ( ↓degradation or ↑improvement ) compared to the whole dataset baseline. We observe that:❶Dynamic pruning strategies generally outperform static ones. Across various baselines, dynamic methods achieve superior performance by leveraging training dynamics to better assess sample importance. Interestingly, both Hard Random and Soft Random remain competitive at low retention rates, mirroring phenomena observed in vision datasets (Guo et al., 2022). We hypothesize that for strategy-based 6 Table 1 Performance comparison to state-of-the-art dataset selection and pruning methods when remaining 10%,30%,50%,70%of the full training set. All methods are trained using same ST backbone. (Additional MAE and RMSE results are provided in Appendix B) DatasetPems08 ( MAPE (%)↓)UrbanEV ( MAPE (%)↓) Remaining Ratio %1030507010305070 Static Hard Random13.93 ↓26.87% 14.80 ↓34.79% 12.07 ↓9.93% 12.06 ↓9.84% 35.89 ↓20.32% 31.90 ↓6.94% 31.52 ↓5.67% 30.69 ↓2.88% CD (Agarwal et al., 2020)14.55 ↓32.51% 12.25 ↓11.57% 11.96 ↓8.93% 11.83 ↓7.74% 38.37 ↓28.63% 32.20 ↓7.95% 30.40 ↓1.91% 31.01 ↓3.96% Herding (Welling, 2009) 15.79 ↓43.81% 12.11 ↓10.29% 11.58 ↓5.46% 12.84 ↓16.94% 34.63 ↓16.09% 30.97 ↓3.82% 30.94 ↓3.72% 30.97 ↓3.82% K-Means (Sener and Savarese, 2018)15.71 ↓43.08% 12.24 ↓11.48% 12.48 ↓13.66% 11.52 ↓4.92% 38.45 ↓28.90% 31.85 ↓6.77% 30.96 ↓3.79% 30.69 ↓2.88% Least Confidence (Coleman et al., 2019)14.31 ↓30.33% 12.62 ↓14.94% 11.74 ↓6.92% 11.61 ↓5.74% 34.23 ↓14.75% 36.87 ↓23.60% 30.98 ↓3.86% 30.55 ↓2.41% Entropy (Coleman et al., 2019)13.22 ↓20.40% 12.82 ↓16.76% 12.83 ↓16.85% 11.28 ↓2.73% 36.42 ↓22.09% 31.20 ↓4.59% 30.65 ↓2.75% 30.49 ↓2.21% Margin (Coleman et al., 2019)14.06 ↓28.05% 12.55 ↓14.30% 11.60 ↓5.65% 11.38 ↓3.64% 37.21 ↓24.74% 31.89 ↓6.91% 32.73 ↓9.72% 30.89 ↓3.55% Forgetting (Toneva et al., 2018)14.92 ↓35.88% 12.19 ↓11.02% 11.46 ↓4.37% 11.29 ↓2.82% 32.84 ↓10.09% 31.36 ↓5.13% 30.63 ↓2.68% 30.69 ↓2.88% GraNd (Paul et al., 2021) 13.93 ↓26.87% 13.09 ↓19.22% 12.33 ↓12.30% 11.50 ↓4.74% 35.26 ↓18.20% 31.71 ↓6.30% 30.90 ↓3.59% 30.20 ↓1.24% Cal (Margatina et al., 2021) 14.79 ↓34.70% 12.40 ↓12.93% 12.64 ↓15.12% 12.40 ↓12.93% 33.77 ↓13.21% 32.25 ↓8.11% 31.45 ↓5.43% 30.81 ↓3.29% Glister (Killamsetty et al., 2021) 14.48 ↓31.88% 13.23 ↓20.49% 12.51 ↓13.93% 11.74 ↓6.92% 33.70 ↓12.97% 33.65 ↓12.81% 32.78 ↓9.89% 31.26 ↓4.79% GraphCut (Iyer et al., 2021) 15.07 ↓37.25% 12.26 ↓11.66% 11.84 ↓7.83% 11.27 ↓2.64% 35.73 ↓19.78% 31.49 ↓5.56% 30.76 ↓3.12% 31.23 ↓4.69% FaLo (Iyer et al., 2021)15.95 ↓45.26% 12.56 ↓14.39% 12.13 ↓10.47% 11.40 ↓3.83% 35.07 ↓17.57% 31.72 ↓6.34% 31.23 ↓4.69% 31.01 ↓3.96% Dynamic Soft Random15.00 ↓36.61% 12.41 ↓13.02% 12.01 ↓9.38% 11.34 ↓3.28% 33.55 ↓12.47% 31.34 ↓5.06% 30.48 ↓2.18% 30.35 ↓1.74% ε-greedy (Raju et al., 2021)13.82 ↓25.87% 12.21 ↓11.20% 11.84 ↓7.83% 11.24 ↓2.37% 33.25 ↓11.46% 30.98 ↓3.86% 30.46 ↓2.11% 30.30 ↓1.58% UCB (Raju et al., 2021) 13.80 ↓25.68% 12.41 ↓13.02% 11.76 ↓7.10% 11.28 ↓2.73% 33.24 ↓11.43% 31.08 ↓4.19% 30.43 ↓2.01% 30.29 ↓1.54% InfoBatch (Qin et al., 2024)13.66 ↓24.41% 12.12 ↓10.38% 11.83 ↓7.74% 11.33 ↓3.19% 32.02 ↓7.34% 30.78 ↓3.18% 30.24 ↓1.37% 29.83 ↓0.00% ST-Prune (Our)12.13 ↓13.21 11.75 ↓7.01 11.11 ↓5.56 11.19 ↓2.82 29.56 ↑0.91 29.42 ↑1.37 29.15 ↑2.28 29.05 ↑2.61 Whole Dataset10.98 ±0.23 29.83 ±0.09 100%90%70%50%30%10% Per Epoch Time (UrbanEV) 3.6 3.8 4.0 4.2 4.4 4.6 4.8 MAE Vanilla H-Random Margin Herding Forgetting S-Random Cal InfoBatch Glister ST-Prune 100%90%70%50%30%10% Per Epoch Time (UrbanEV) 6.5 7.0 7.5 8.0 8.5 9.0 RMSE Vanilla H-Random Margin Herding Forgetting S-Random Cal InfoBatch Glister ST-Prune 100%90%70%50%30%10% Per Epoch Time (UrbanEV) 28 30 32 34 36 38 40 MAPE(%) Vanilla H-Random Margin Herding Forgetting S-Random Cal InfoBatch Glister ST-Prune Figure 4 The trade-off between per-epoch time and performance in UrbanEV. Specifically, we report the test performance when methods achieve per epoch times of 10%, 30%, 50%, 70%, 90% of the full dataset training time. “Vanilla” denotes the full dataset training result. methods, specific algorithmic inductive biases may introduce a harmful sampling expectation bias.❷Dataset redundancy varies across domains. We observe that PEMS08 is more sensitive to pruning than UrbanEV, indicating differing redundancy levels. For example, at 10% retention (excluding our method), performance degradation on PEMS08 (20.40%∼45.26%) significantly exceeds that of UrbanEV (7.34%∼28.63%). However, performance recovers rapidly as the retention ratio increases; at a 50% pruning level, degradation across both datasets remains within single digits.❸ ST-Pruneconsistently excels across all pruning ratios. ST-Pruneoutperforms both static and dynamic baselines in all settings. Even at 10% retention rate, it keeps practical utility with minimal degradation. Notably,ST-Prunesurpasses whole dataset performance on UrbanEV. We attribute this to UrbanEV ’s low redundancy and high signal-to-noise ratio, where our method effectively filters noise to boost performance. 5.3 Efficiency Study (RQ2) Figure 4 illustrates the efficiency comparison. We observe the following:❹ ST-Prunestrikes a superior balance between training time and performance, significantly outperforming other baselines. Notably,ST-Pruneachieves nearly 2×acceleration (≈50% reduction in per-epoch time) with negligible performance loss across different 7 metrics.❹Even under an aggressive 10×speedup,ST-Pruneincurs only marginal degradation, maintaining acceptable performance where competing methods deteriorate rapidly. 5.4 Scalability Study (RQ3) To assess the scalability of our method, we conduct evaluations from both data and model perspectives. Specifically, we validate performance using the representative large-scale spatio-temporal dataset LargeST (Liu et al., 2024b), which comprises the SD, GBA, and GLA subsets. Additionally, we employ the spatio-temporal foundation model OpenCity (Li et al., 2025b) across the Mini, Base, and Plus scales. Table 2 Comparison ofST-Prunewith the fastest dynamic pruning method Soft Random on LargeST. TIME: Total wall-clock time. Dataset SD (#716)GBA (#2352)GLA (#3834) MAERMSE MAPE (%) TIMEMAERMSE MAPE (%) TIMEMAERMSE MAPE (%) TIME Whole Dataset24.13 ±1.14 38.03 ±1.32 15.77 ±0.87 2.07h27.34 ±1.04 42.11 ±1.19 24.11 ±0.48 13.05h26.26 ±0.84 41.12 ±1.47 16.06 ±0.63 30.53h Soft Random (10%)30.4147.9620.430.52h33.0548.7139.302.97h29.7445.9820.977.25h ST-Prune (10%)24.0137.7315.510.61h27.3241.9424.054.92h26.2240.8815.9513.38h ST-Prune (1%)24.7638.1916.510.31h28.1642.7723.642.84h26.9541.5717.627.12h Improvement↑ 21.05%↑ 21.33%↑ 24.08%×6.68↑ 17.33%↑ 13.90%↑ 38.80%×4.60↑ 11.84%↑ 11.09%↑ 23.94%×4.29 Large-Scale Datasets. Table 2 presents comparative performance on the LargeST benchmark. We highlight the following observations:❶Competitive efficiency-performance trade-off. At a 10% data retention rate, heuristic baselines (i.e., Soft Random) achieve the fastest dynamic pruning speeds but suffer significant performance degradation. In contrast, our method achieves relative performance gains of 11.09%∼38.80% over heuristics and even slightly outperforms full-data training, despite the additional computational overhead incurred by our customized policy design.❷Robustness across spatio-temporal scales. Unlike heuristic baselines (i.e., Soft Random) that exhibit marked performance drops on smaller datasets (e.g., SD), our ST-Prunemaintains consistent performance advantages across varying spatio-temporal scales.❸Superior efficiency and scalability. Even in extreme 1% retention scenarios, our model retains competitive performance while drastically reducing computational costs. By cutting training time from days to hours, it demonstrates significant potential for data scalability and large-scale deployment. Figure 5 Performance and efficiency trade-offs between w/o and w/ST-Pruneat different scales of the ST-foundation model OpenCity. ST-Foundation Models. Following (Li et al., 2025b) settings, we further validate model scalability by integratingST-Prunewith the OpenCity foundation model series. Figure 5 illustrates the trade-off between per-epoch pre-training time and downstream performance, revealing the following:❶ Simultaneous gains in efficiency and capability.ST-Pruneachieves a “win-win” outcome, with green arrow consistently pointing towards the “most sample-efficient” region. Across all model scales,ST-Prunenot only significantly reduces the pre-training time (shifting left) but also generally improves the prediction accuracy (shifting upwards). This suggests that our approach effectively purifies the pre-training corpus by eliminating noisy or redundant samples that hinder convergence.❷Democratizing large-scale model training. For the computation-intensive Plus scale,ST-Prunesignificantly curtails per-epoch training time (e.g., from∼350 to 250 mins) while 8 outperforming the original baseline. Notably,ST-Prunereduces the training cost of the Base scale to a level smaller than the Mini scale, thus effectively training larger and more powerful models under constrained computational resources. 5.5 Universality Study (RQ4) Whole - GWNet (29.83%) Whole - STID (26.45%) Whole - STAEformer (24.92%) 10%30%50%70% Remaining Ratio 25 26 27 28 29 30 MAPE (%) Cross-Architecture GWNet STID STAEformer Whole - SGD (29.83%) Whole - Adam (23.63%) Whole - Muon (23.44%) 10%30%50%70% Remaining Ratio 23 24 25 26 27 28 29 30 MAPE (%) Cross-Optimizer SGD Adam Muon Whole - Short (29.83%) Whole - Medium (25.84%) Whole - Long (37.19%) 10%30%50%70% Remaining Ratio 26 28 30 32 34 36 38 MAPE (%) Cross-Task Short Medium Long Figure 6 Performance variations across distinct architectures, optimizers, and tasks under data retention rates of 10%, 30%, 50%, 70%. To evaluate the universality of our method, we conduct evaluations on UrbanEV across different backbones, optimizers, and tasks. Specifically, we validate performance using representative ST architectures including GWNet, STID, and STAEformer; distinct optimizers such as SGD, Adam (Kingma and Ba, 2015), and Muon (Jordan et al., 2024); and prediction horizons covering short-term (i.e., 12→12), medium-term (i.e., 24→24), and long-term (i.e., 96→96) tasks. From each subgraph in Figure 6, we observed that: Cross-Architecture Evaluation.❶Universal performance gains. Notably, across all three backbones, mod- els trained onST-Prune-selected subsets (dashed lines) consistently outperform their full-dataset counterparts (solid lines), achieving lower MAPE at most retention levels.❷Architectural robustness. In low-retention regimes, our method yields pronounced gains for lightweight architectures like STID and GWNet. However, even for the advanced, over-parameterized STAEformer, our model can also push performance boundaries as the retention ratio increases. Cross-Optimizer Evaluation.❶Robust adaptability to optimization process.ST-Prunemaintains ef- fectiveness regardless of the chosen optimizer. Notably, with SGD,ST-Pruneachieves substantial gains, outperforming the full-dataset baseline even at low retention rates.❷Enhancing advanced optimizers. Even with state-of-the-art optimizers like Adam and the recent Muon, which already yield low error rates (<24%), ST-Prunefurther pushes performance boundaries. At retention rates above 30%,ST-Prunesuccessfully filters detrimental noise, enabling both optimizers to surpass their full-dataset baselines, thereby confirming the robustness of our method ST-Prune across diverse optimization landscapes. Cross-Task Evaluation.❶Short-to-medium term robustness. For short and medium horizons,ST-Prunecon- sistently outperforms full-dataset baselines across all retention ratios, confirming that immediate temporal dependencies are effectively captured within compact, denoised subsets.❷Long-term sensitivity. Although long-term involves more complex dependencies and higher sensitivity to data scarcity, our method remains robust; performance successfully recovers to match full-dataset levels at a 50% retention ratio. 5.6 Mechanism & Robustness Study (RQ5) Ablation Study. To validate the effectiveness of each component inST-Prune, we compare against four variants: removing the spatio-temporal complexity score (w/o STC), the score rescaling strategy (w/o Res.), and the annealing scheduler (w/o Anne.). As shown in Figure 7 Left,ST-Pruneconsistently yields the lowest errors across all retention ratios. We observe that:❶Necessity of scheduling with annealing. The performance degradation of w/o Anne. is the most significant, especially under low retention rates, which illustrates its positive role in variance correction for sparse scenarios.❷Importance of Scoring Components. Removing STC leads to noticeable error increases, indicating that both the intrinsic difficulty of samples and the historical consistency are critical for robust sample valuation. 9 Remaining Ratio: 10% 29.0 29.5 30.0 30.5 31.0 31.5 32.0 MAPE(%) 29.56 29.82 30.10 31.59 Remaining Ratio: 30% 29.0 29.5 30.0 30.5 31.0 31.5 32.0 29.42 29.85 29.84 30.43 Remaining Ratio: 50% 28.5 29.0 29.5 30.0 30.5 31.0 MAPE(%) 29.15 29.85 29.37 30.55 Remaining Ratio: 70% 28.5 29.0 29.5 30.0 30.5 31.0 29.05 29.43 29.44 30.66 ST-Prunew/o STCw/o Res.w/o Anne. 0.20.40.60.81.0 Complexity 29.0 29.2 29.4 29.6 MAPE (%) 10% 30% 50% 70% 0.800.850.900.951.00 Annealing 29.0 29.5 30.0 30.5 31.0 31.5 MAPE (%) 10% 30% 50% 50% Figure 7 Ablation and parameter sensitivity analysis on UrbanEV dataset under training data retention rates of 10%, 30%, 50%, 70%. Parameter Sensitivity. We further investigate the sensitivity ofST-Pruneto key hyperparameters: the complexity weight and the annealing rate .❶Impact of Complexity Weight . Figure 7 (Middle) exhibits a convex trend for , where extreme values (too small or too large) hamper performance. The optimal range lies between 0.4 and 0.6, suggesting that balancing the raw loss with spatio-temporal structural complexity is essential for identifying informative samples.❷Impact of Annealing Rate . Figure 7 (Right) reveals that performance is stable whenδ ∈[0.8,0.95] but deteriorates rapidly asδapproaches 1, i.e. w/o Anne., consistent with previous observations in the ablation study. Whole Training set Hard Random 20%40%60%80% Info Batch 20%40%60%80% Soft Random ST-Prune (Our) 20%40%60%80%20%40%60%80% Figure 8 Visualization of t-SNE fixed-corner clustering of the PEMS08 distribution with the full training set and a retention rate of 20%. The right panel shows the evolution of selected subsets through different pruning strategies at different training progresses (20% to 80%). Interpretability Study. To intuitively understand howST-Pruneselects spatio-temporal sample compared to baselines, we visualize the latent distribution of the selected subsets using t-SNE in Figure 8. We can observe that:❶Dynamic alignment with training dynamics. While heuristic baselines (e.g., Hard/Soft Random) produce static, sparse distributions via uniform sampling, ST-Prune exhibits a clear evolutionary trajectory, balancing the preservation of representative centroids with the exploration of global diversity.❷Restoration of intrinsic data topology. Surpassing InfoBatch,ST-Prunerapidly reconstructs the approximate manifold topology of the original distribution during training, thereby ensuring robust generalization performance. 6 Conclusion and Future Work In this paper, we investigate the redundancy inherent in spatio-temporal training datasets and explore effective approaches for efficient dynamic data pruning. We proposeST-Prunethat uses complexity-informed scoring with stationarity-aware gradient rescaling to prioritize informative samples during training. Extensive experiments confirm its effectiveness. In future work, we aim to explore how to extend this method to continual spatio-temporal forecasting where the spatial topology evolves over training time. 10 References Amit Agarwal, Bhumika Gupta, Gaurav Bhatt, and Ankush Mittal. Construction of a semi-automated model for faq retrieval via short message service. InProceedingsofFIRE, 2015. Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. Contextual diversity for active learning. InECCV, pages 137–153. Springer, 2020. Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, et al. A survey on data selection for language models.Transactionson MachineLearningResearch, 2024. Allan M Avila and Igor Mezić. Data-driven analysis and forecasting of highway traffic dynamics.Naturecommunications, 11(1):2090, 2020. Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting.Advancesinneuralinformationprocessingsystems, 33:17804–17815, 2020. George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPattern Recognition, pages 4750–4759, 2022. Ke Chen. On coresets for k-median and k-means clustering in metric and euclidean spaces and their applications. SIAMJournalonComputing, 39(3):923–947, 2009. Min Chen, Guansong Pang, Wenjun Wang, and Cheng Yan. Information bottleneck-guided mlps for robust spatial- temporal forecasting. InForty-secondInternationalConferenceonMachineLearning, 2025. Wei Chen and Yuxuan Liang. Expand and compress: Exploring tuning principles for continual spatio-temporal graph forecasting. InTheThirteenthInternationalConferenceonLearningRepresentations, 2025a. Wei Chen and Yuxuan Liang. Learning with calibration: Exploring test-time computing of spatio-temporal forecasting. AdvancesinNeuralInformationProcessingSystems, 2025b. Andrea Cini, Ivan Marisca, Daniele Zambon, and Cesare Alippi. Taming local effects in graph-based spatiotemporal forecasting.AdvancesinNeuralInformationProcessingSystems, 36:55375–55393, 2023. Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. InICLR, 2019. Yuchen Fang, Yuxuan Liang, Bo Hui, Zezhi Shao, Liwei Deng, Xu Liu, Xinke Jiang, and Kai Zheng. Efficient large-scale traffic forecasting with transformers: A spatial data management perspective. InProceedingsofthe31stACM SIGKDDConferenceonKnowledgeDiscoveryandDataMiningV.1, pages 307–317, 2025. Yisong Fu, Fei Wang, Zezhi Shao, Boyu Diao, Lin Wu, Zhulin An, Chengqing Yu, Yujie Li, and Yongjun Xu. On the integration of spatial-temporal knowledge: A lightweight approach to atmospheric time series forecasting. InThe Thirty-ninthAnnualConferenceonNeuralInformationProcessingSystems, 2025. Chengcheng Guo, Bo Zhao, and Yanbing Bai. Deepcore: A comprehensive library for coreset selection in deep learning. InInternationalConferenceonDatabaseandExpertSystemsApplications, pages 181–195. Springer, 2022. Sariel Har-Peled and Soham Mazumdar. On coresets for k-means and k-median clustering. InProceedingsofthe thirty-sixthannualACMsymposiumonTheoryofcomputing, pages 291–300, 2004. Rishabh Iyer, Ninad Khargoankar, Jeff Bilmes, and Himanshu Asanani. Submodular combinatorial information measures with applications in machine learning. InAlgorithmicLearningTheory, pages 722–754. PMLR, 2021. Renhe Jiang, Zhaonan Wang, Jiawei Yong, Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. Spatio-temporal meta-graph learning for traffic forecasting. InProceedingsof theAAAIconferenceonartificialintelligence, volume 37, pages 8078–8086, 2023. Guangyin Jin, Yuxuan Liang, Yuchen Fang, Zezhi Shao, Jincai Huang, Junbo Zhang, and Yu Zheng. Spatio-temporal graph neural networks for predictive learning in urban computing: A survey.IEEETransactionsonKnowledgeand DataEngineering, 2023. Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I Webb, Irwin King, and Shirui Pan. A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection. IEEETransactionsonPatternAnalysisandMachineIntelligence, 2024. 11 Keller Jordan, Yuchen Jin, Vlado Boza, You Jiacheng, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https://kellerjordan.github.io/posts/muon/. Angelos Katharopoulos and François Fleuret. Not all samples are created equal: Deep learning with importance sampling. InInternationalconferenceonmachinelearning, pages 2525–2534. PMLR, 2018. Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Generalization based data subset selection for efficient and robust learning. InProceedingsoftheAAAIConferenceonArtificial Intelligence, 2021. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. Shiye Lei and Dacheng Tao. A comprehensive survey of dataset distillation.IEEETransactionsonPatternAnalysis andMachineIntelligence, 46(1):17–32, 2023. Han Li, Haohao Qu, Xiaojun Tan, Linlin You, Rui Zhu, and Wenqi Fan. Urbanev: An open benchmark dataset for urban electric vehicle charging demand prediction.ScientificData, page 523, 2025a. ISSN 2052-4463. doi: 10.1038/s41597-025-04874-4. Xinglin Li, Kun Wang, Hanhui Deng, Yuxuan Liang, and Di Wu. Attend who is weak: Enhancing graph condensation via cross-free adversarial training.arXivpreprintarXiv:2311.15772, 2023. Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting.arXivpreprintarXiv:1707.01926, 2017. Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. InInternationalConferenceonLearningRepresentations, 2018. Zhonghang Li, Long Xia, Lei Shi, Yong Xu, Dawei Yin, and Chao Huang. Open spatio-temporal foundation models for traffic prediction.ACMTransactionsonIntelligentSystemsandTechnology, 2025b. Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio- temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. InProceedingsofthe32nd ACMinternationalconferenceoninformationandknowledgemanagement, pages 4125–4129, 2023. Xu Liu, Yuxuan Liang, Chao Huang, Hengchang Hu, Yushi Cao, Bryan Hooi, and Roger Zimmermann. Reinventing node-centric traffic forecasting for improved accuracy and efficiency. InJointEuropeanConferenceonMachine LearningandKnowledgeDiscoveryinDatabases, pages 21–38. Springer, 2024a. Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting.AdvancesinNeural InformationProcessingSystems, 36, 2024b. Jiaming Ma, Binwu Wang, Guanjun Wang, Kuo Yang, Zhengyang Zhou, Pengkun Wang, Xu Wang, and Yang Wang. Less but more: Linear adaptive graph learning empowering spatiotemporal forecasting.AdvancesinNeural InformationProcessingSystems, 2025. Katerina Margatina, Giorgos Vernikos, Loïc Barrault, and Nikolaos Aletras. Active learning by acquiring contrastive examples.arXivpreprintarXiv:2109.03764, 2021. Brian B Moser, Arundhati S Shanbhag, Stanislav Frolov, Federico Raue, Joachim Folz, and Andreas Dengel. A coreset selection of coreset selection literature: Introduction and recent advances.arXivpreprintarXiv:2505.17799, 2025. Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks.AdvancesinNeuralInformationProcessingSystems, 34:5186–5198, 2021. Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. InInternationalConferenceonMachineLearning, pages 25904–25938. PMLR, 2023. Seth Ockerman, Amal Gueroudji, Tanwi Mallick, Yixuan He, Line Pouchard, Robert Ross, and Shivaram Venkataraman. Pgt-i: Scaling spatiotemporal gnns with memory-efficient distributed training. InProceedingsoftheInternational ConferenceforHighPerformanceComputing,Networking,StorageandAnalysis, pages 217–236, 2025. Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training.AdvancesinNeuralInformationProcessingSystems, 34:20596–20607, 2021. Ziheng Qin, Kai Wang, Zangwei Zheng, Jianyang Gu, Xiangyu Peng, Daquan Zhou, Lei Shang, Baigui Sun, Xuansong Xie, Yang You, et al. Infobatch: Lossless training speed up by unbiased dynamic data pruning. InTheTwelfth InternationalConferenceonLearningRepresentations, 2024. 12 Ravi S Raju, Kyle Daruwalla, and Mikko Lipasti. Accelerating deep learning with dynamic data pruning.arXiv preprintarXiv:2111.12621, 2021. Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. InICLR, 2018. Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. InProceedingsofthe31stACMinternationalconferenceoninformation& knowledgemanagement, pages 4454–4458, 2022. Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis.IEEETransactionsonKnowledgeandDataEngineering, 2024. Zezhi Shao, Yujie Li, Fei Wang, Chengqing Yu, Yisong Fu, Tangwen Qian, Bin Xu, Boyu Diao, Yongjun Xu, and Xueqi Cheng. Blast: Balanced sampling time series corpus for universal forecasting models. InProceedingsofthe 31stACMSIGKDDConferenceonKnowledgeDiscoveryandDataMiningV.2, pages 2502–2513, 2025. Jae-hun Shim, Kyeongbo Kong, and Suk-Ju Kang. Core-set sampling for efficient neural architecture search.arXiv preprintarXiv:2107.06869, 2021. Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. InProceedingsoftheAAAIconferenceonartificial intelligence, volume 34, pages 914–921, 2020. Jiabin Tang, Wei Wei, Lianghao Xia, and Chao Huang. Easyst: A simple framework for spatio-temporal prediction. InProceedingsofthe33rdACMInternationalConferenceonInformationandKnowledgeManagement, pages 2220–2229, 2024. Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning.arXivpreprintarXiv:1812.05159, 2018. Binwu Wang, Pengkun Wang, Zhengyang Zhou, Zhe Zhao, Wei Xu, and Yang Wang. Make bricks with a little straw: Large-scale spatio-temporal graph learning with restricted gpu-memory capacity. InIJCAI, pages 2388–2396. Jeju, South Korea, 2024. Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. InProceedingsoftheIEEE/CVFConference onComputerVisionandPatternRecognition, pages 12196–12205, 2022. Kun Wang, Yuxuan Liang, Xinglin Li, Guohao Li, Bernard Ghanem, Roger Zimmermann, Huahui Yi, Yudong Zhang, Yang Wang, et al. Brave the wind and the waves: Discovering robust and generalizable graph lottery tickets.IEEE TransactionsonPatternAnalysisandMachineIntelligence, 2023. Shaobo Wang, Jiaming Wang, Jiajun Zhang, Cong Wang, Yue Min, Zichen Wen, Fei Huang, Huiqiang Jiang, Junyang Lin, Dayiheng Liu, et al. Winning the pruning gamble: A unified approach to joint sample and token pruning for efficient supervised fine-tuning.arXivpreprintarXiv:2509.23873, 2025. Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation.arXivpreprint arXiv:1811.10959, 2018. Max Welling. Herding dynamical weights to learn. In ICMLg, pages 1121–1128, 2009. Wenchao Weng, Hanyu Jiang, Mei Wu, Xiao Han, Haidong Gao, Guojiang Shen, and Xiangjie Kong. Let’s group: a plug-and-play subgraph learning method for memory-efficient spatio-temporal graph modeling. InProceedingsof theThirty-FourthInternationalJointConferenceonArtificialIntelligence, pages 3471–3479, 2025. Hao Wu, Haomin Wen, Guibin Zhang, Yutong Xia, Yuxuan Liang, Yu Zheng, Qingsong Wen, and Kun Wang. Dynst: Dynamic sparse training for resource-constrained spatio-temporal forecasting. InProceedingsofthe31stACM SIGKDDConferenceonKnowledgeDiscoveryandDataMiningV.1, pages 2682–2692, 2025. Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling.arXivpreprintarXiv:1906.00121, 2019. Du Yin, Hao Xue, Arian Prabowo, Shuang Ao, and Flora Salim. Xxltraffic: Expanding and extremely long traffic forecasting beyond test adaptation. InProceedingsofthe33rdACMInternationalConferenceonAdvancesin GeographicInformationSystems, pages 511–521, 2025. 13 Ruonan Yu, Songhua Liu, and Xinchao Wang. Dataset distillation: A comprehensive review.IEEEtransactionson patternanalysisandmachineintelligence, 46(1):150–170, 2023. Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey.ACMComputingSurveys, 57(5):1–42, 2025. Guibin Zhang, Haonan Dong, Zhixun Li, Dingshuo Chen, Kai Wang, Tianlong Chen, Yuxuan Liang, Dawei Cheng, Kun Wang, et al. Gder: Safeguarding efficiency, balancing, and robustness via prototypical graph pruning.Advances inNeuralInformationProcessingSystems, 37:50285–50312, 2024a. Guibin Zhang, Kun Wang, Wei Huang, Yanwei Yue, Yang Wang, Roger Zimmermann, Aojun Zhou, Dawei Cheng, Jin Zeng, and Yuxuan Liang. Graph lottery ticket automated. InTheTwelfthInternationalConferenceonLearning Representations, 2024b. URL https://openreview.net/forum?id=nmBjBZoySX. Yuchen Zhang, Tianle Zhang, Kai Wang, Ziyao Guo, Yuxuan Liang, Xavier Bresson, Wei Jin, and Yang You. Navigating complexity: Toward lossless graph condensation via expanding window matching.arXivpreprintarXiv:2402.05011, 2024c. Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. InProceedingsoftheIEEE/CVFWinter ConferenceonApplicationsofComputerVision, pages 6514–6523, 2023. Yiji Zhao, Zihao Zhong, Ao Wang, Haomin Wen, Ming Jin, Yuxuan Liang, Huaiyu Wan, and Hao Wu. Fast: Efficient and effective long-horizon forecasting for large-scale spatial-temporal graphs via mixture-of-experts. InProceedings ofthe32thACMSIGKDDConferenceonKnowledgeDiscoveryandDataMining, 2026. 14 Appendix Table of Contents A Experimental Details16 A.1 Dataset and Protocol Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 A.1.1 Datasets Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 A.1.2 Protocol Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 A.2 Baseline and Parameter Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 A.2.1 Baseline Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 A.2.2 Parameter Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 B More Results19 C More Discussion20 C.1 Current Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 C.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Go to Appendix Index15 A Experimental Details A.1 Dataset and Protocol Details A.1.1 Datasets Details Our experiments are carried out on five real-world datasets from diffrent domain. The statistics of these spatio-temporal datasets are shown in Table A.1. Table A.1 Summary of datasets used for our experiments. Degree: the average degree of each node. Data Points: multiplication of nodes and frames. M: million (10 6 ). SourceDatasetNodesTime RangeFrames Sampling Rate Data Points (Song et al., 2020) PEMS08170 07/01/2016 – 08/31/2016 17,856 5 minutes3.04M (Li et al., 2025a) UrbanEV275 09/01/2022 – 02/28/2023 43441 hour1.19M LargeST (Liu et al., 2024b) GLA 3,834 01/01/2019 – 12/31/2019 35,040 15 minutes134.3M GBA 2,352 01/01/2019 – 12/31/2019 35,040 15 minutes88.7M SD 716 01/01/2019 – 12/31/2019 35,040 15 minutes25.1M We adopt the standard approach widely used in prior works (Li et al., 2017; Bai et al., 2020) to determine the graph structure prior. Specifically, we build the adjacency matrixAusing a thresholded Gaussian kernel, formulated as: A [ij] = exp − d 2 ij σ 2 , if exp − d 2 ij σ 2 ≥ r and i̸= j, 0,otherwise, whered ij denotes the geographical distance between sensorsiandj,σis chosen as the standard deviation of all pairwise distances, andris a predefined similarity threshold used to sparsify the graph. For the specific parameter settings, we follow the paper corresponding to the dataset. A.1.2 Protocol Details We use different metrics such as MAE, RMSE, MAPE, and CORR. Formally, these metrics are formulated as following: MAE = 1 n n X i=1 |y i − ˆy i |,RMSE = v u u t 1 n n X i=1 (y i − ˆy i ) 2 , MAPE = 100% n n X i=1 ˆy i − y i y i ,Corr = P n i=1 (y i − ̄y)(ˆy i − ̄ ˆy) p P n i=1 (y i − ̄y) 2 q P n i=1 (ˆy i − ̄ ˆy) 2 , wherenrepresents the indices of all observed samples,y i denotes thei-th actual sample, andˆy i is the corresponding prediction. A.2 Baseline and Parameter Details A.2.1 Baseline Details In this subsection, we describe in detail the advanced dataset pruning and selection methods and different spatiotemporal backbone models that we use in our default evaluation. Go to Appendix Index16 050100150200250 Spatial Node Index 0 50 100 150 200 250 Spatial Node Index Spatial Correlation Between Nodes -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 Pearson Correlation Mon.Tue.Wed.Thu.Fri.Sat.Sun. Date 5 10 15 20 25 Occupancy Ratio Periodic Repeating Temporal Patterns Line 1 Line 2 050100150200250 # Principal Components 0.6 0.7 0.8 0.9 1.0 Cumulative Explained Variance PCA in Spatial and Temporal Dimensions Spatial PCA Temporal PCA 90% variance Figure A.1 The spatio-temporal redundancy characteristics and statistical properties of UrbanEV (Li et al., 2025a). 0100200300400500600700 Spatial Node Index 0 100 200 300 400 500 600 700 Spatial Node Index Spatial Correlation Between Nodes -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 Pearson Correlation Mon.Tue.Wed.Thu.Fri.Sat.Sun. Date 0 100 200 300 400 500 Traffic Flow Periodic Repeating Temporal Patterns Line 1 Line 2 0100200300400500600700 # Principal Components 0.75 0.80 0.85 0.90 0.95 1.00 Cumulative Explained Variance PCA in Spatial and Temporal Dimensions Spatial PCA Temporal PCA 90% variance Figure A.2 The spatio-temporal redundancy characteristics and statistical properties of SD (Liu et al., 2024b). 0500100015002000 Spatial Node Index 0 500 1000 1500 2000 Spatial Node Index Spatial Correlation Between Nodes -0.2 0.0 0.2 0.4 0.6 0.8 1.0 Pearson Correlation Mon.Tue.Wed.Thu.Fri.Sat.Sun. Date 0 100 200 300 400 500 Traffic Flow Periodic Repeating Temporal Patterns Line 1 Line 2 0500100015002000 # Principal Components 0.75 0.80 0.85 0.90 0.95 1.00 Cumulative Explained Variance PCA in Spatial and Temporal Dimensions Spatial PCA Temporal PCA 90% variance Figure A.3 The spatio-temporal redundancy characteristics and statistical properties of GBA (Liu et al., 2024b). 0500100015002000250030003500 Spatial Node Index 0 500 1000 1500 2000 2500 3000 3500 Spatial Node Index Spatial Correlation Between Nodes -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 Pearson Correlation Mon.Tue.Wed.Thu.Fri.Sat.Sun. Date 0 100 200 300 400 500 600 700 Traffic Flow Periodic Repeating Temporal Patterns Line 1 Line 2 05001000150020002500300035004000 # Principal Components 0.800 0.825 0.850 0.875 0.900 0.925 0.950 0.975 1.000 Cumulative Explained Variance PCA in Spatial and Temporal Dimensions Spatial PCA Temporal PCA 90% variance Figure A.4 The spatio-temporal redundancy characteristics and statistical properties of GLA (Liu et al., 2024b). Go to Appendix Index17 Data Pruning and Selection Methods. To ensure fair use of these baselines, we use a popular coreset selection benchmark libraries:https://github.com/PatrickZH/DeepCoreand adapt the respective methods to ST forecasting tasks. • Static Dataset Pruning: † Random-based: ‡ Hard Random: Random selection before training. † Geometry-based: ‡CD (Agarwal et al., 2015): Adapted the diversity metric from image feature space to spatio- temporal feature space, using spatio-temporal embedding vectors to calculate inter-sample distances for selecting representative patterns. ‡Herding (Welling, 2009): Modified prototype selection from classification labels to regression targets, finding core samples in spatio-temporal feature space that represent different trends. ‡K-Means (Sener and Savarese, 2018): Adapted K-Means clustering from image features to spatio-temporal sequence features, clustering based on temporal patterns and spatial correlations in samples for center selection. † Uncertainty-based: ‡ Least Confidence (Coleman et al., 2019):Replaced classification probability confidence with re- gression prediction uncertainty estimation, selecting spatio-temporal samples with maximum prediction variance. ‡ Entropy (Coleman et al., 2019): Adapted entropy calculation from classification probability distributions to regression prediction distributions, selecting spatio-temporal sequence samples with maximum uncertainty-based information content. ‡Margin (Coleman et al., 2019): Changed from classification decision boundary distances to regression prediction interval widths, selecting spatio-temporal samples with maximum prediction confidence intervals. † Loss-based: ‡ Forgetting (Toneva et al., 2018): Replaced classification accuracy forgetting statistics with regression error change statistics, tracking fluctuation patterns of spatio-temporal sequence samples’ prediction errors during training. † Decision boundary-based: ‡GraNd (Paul et al., 2021): Adapted gradient norm calculation from classification loss to regression loss, selecting samples with greatest impact on the model based on spatio-temporal prediction model parameter gradient magnitudes. † Bi-level optimization-based: ‡Glister (Killamsetty et al., 2021): Replaced classification loss with regression loss in the bi-level optimization framework, selecting most valuable training samples through gradient matching for spatio-temporal prediction tasks. † Submodular function-based: ‡ GraphCut (Iyer et al., 2021): Replaced similarity-based construction with spatio-temporal sequence similarity-based construction, performing partitioning based on spatio-temporal correlations to select representative samples. ‡FaLo (Iyer et al., 2021): Adapted the facility location problem from classification to regression tasks, finding facility points in spatio-temporal feature space that cover different patterns as core samples. • Dynamic Dataset Pruning: † Random-based: ‡ Soft Random: Random selection during training. † Uncertainty-based: Go to Appendix Index18 ‡ ε-greedy (Raju et al., 2021): Adapted greedy selection strategy from classification models to regression model parameter changes, dynamically selecting samples most beneficial for spatio-temporal prediction model training. ‡ UCB (Raju et al., 2021): Adapted multi-armed bandit classification reward mechanism to regression task rewards, dynamically balancing exploration and exploitation in sample selection based on prediction error reduction. † Loss-based: ‡InfoBatch (Qin et al., 2024): Adapted dynamic batch selection from classification loss-based to spatiotemporal regression loss-based selection, dynamically adjusting training batch composition according to the information content of spatio-temporal sequence samples. Spatio-Temporal Neural Network. To ensure fair use of these models, we use an implementation of the popular spatio-temporal neural network benchmark library:https://github.com/liuxu77/LargeST, adapted to the above data pruning and selection repository. • Graph-based: † GWNet (Wu et al., 2019): GWNet is a graph-based deep learning framework tailored for spatio- temporal forecasting. It leverages graph convolutional networks to model complex spatial correlations among nodes and employs dilated causal convolutions to capture long-range temporal dependencies, enabling effective representation of spatial-temporal dynamics. • MLP-based: †STID (Shao et al., 2022): STID serves as a lightweight MLP-based baseline for spatio-temporal prediction. By explicitly incorporating spatial and temporal identity embeddings, it alleviates the indistinguishability problem in spatio-temporal data. Its parameter-efficient MLP architecture achieves competitive forecasting accuracy with low computational overhead. • Transformer-based: †STAEformer (Liu et al., 2023): STAEformer is a effective transformer-based model that adapts the vanilla transformer architecture to the spatio-temporal domain. It introduces adaptive spatial-temporal embeddings and attention mechanisms to dynamically capture spatial dependencies and temporal evolution, pushing transformer models to high-competitive performance for spatio-temporal forecasting tasks. Spatio-Temporal Foundation Model. We also applied our method to the recently popular spatio- temporal foundation models to evaluate scalability. We followed and used the open-source code repository and pre-trained datasets of OpenCity (Li et al., 2025b) from: https://github.com/HKUDS/OpenCity. †OpenCity (Li et al., 2025b): OpenCity is a large-scale foundation model designed for general-purpose spatio-temporal learning in urban environments. It is pre-trained on massive, heterogeneous urban data spanning multiple cities and domains, enabling strong cross-city and cross-task generalization. A.2.2 Parameter Details Detailed hyperparameters settings are shown in Table A.2. We use the same parameter configurations for our ST-Prune, along with the other baseline methods according to the recommendation of previous papers (Guo et al., 2022; Liu et al., 2024b). All experiments are conducted on a Linux server equipped with a 1× Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz (512GB memory) and 8× NVIDIA A100 (80GB memory) GPUs. To carry out benchmark testing experiments, all baselines and our method are set to run 100 epochs by default without early stopping mechanism to ensure fairness of efficiency experiments. B More Results We also provide MAE and RMSE comparisons of the performance of our methodST-Prunewith the state-of- the-art dataset pruning and selection methods when10%,30%,50%,70%of the full set remains, as shown Go to Appendix Index19 Table A.2 Hyperparameters setting. Training epochs100 Selection epochs10 Batch size256 Data Fraction 0.1 / 0.3 / 0.5 / 0.7 OptimizerSGD Learning rate1e-3 Minimum Learning rate1e-4 Momentum0.9 Weight Decay 1e-4 SchedulerCosineAnnealingLR Gamma0.5 Step Size50 Loss FunctionMAE λ0.5 δ 0.9 in Tables B.1 and B.2. Table B.1 Performance comparison to state-of-the-art dataset pruning methods when remaining10%,30%,50%,70% of the full set. All methods are trained using GWNet, and the reported metric MAE represent the average of five runs. DatasetPems08 (MAE↓)UrbanEV (MAE↓) Remaining Ratio %1030507010305070 Static Hard Random19.92 ↓16.90% 18.75 ↓10.04% 18.24 ↓7.04% 17.92 ↓5.16% 4.23 ↓18.82% 3.96 ↓11.24% 3.82 ↓7.30% 3.76 ↓5.62% CD (Agarwal et al., 2020)19.99 ↓17.31% 18.50 ↓8.57% 18.12 ↓6.34% 17.86 ↓4.81% 4.34 ↓21.91% 3.95 ↓10.96% 3.81 ↓7.02% 3.79 ↓6.46% Herding (Welling, 2009) 20.14 ↓18.19% 18.51 ↓8.63% 18.16 ↓6.57% 17.84 ↓4.69% 4.37 ↓22.75% 3.92 ↓10.11% 3.88 ↓8.99% 3.74 ↓5.06% K-Means (Sener and Savarese, 2018)20.21 ↓18.60% 18.59 ↓9.10% 18.28 ↓7.28% 17.81 ↓4.52% 4.33 ↓21.63% 4.00 ↓12.36% 3.81 ↓7.02% 3.77 ↓5.90% Least Confidence (Coleman et al., 2019)20.61 ↓20.95% 19.48 ↓14.32% 18.14 ↓6.46% 18.11 ↓6.28% 4.22 ↓18.54% 4.47 ↓25.56% 3.82 ↓7.30% 3.71 ↓4.21% Entropy (Coleman et al., 2019) 20.65 ↓21.19% 18.49 ↓8.51% 18.10 ↓6.22% 17.88 ↓4.93% 4.32 ↓21.35% 3.96 ↓11.24% 3.81 ↓7.02% 3.76 ↓5.62% Margin (Coleman et al., 2019)20.42 ↓19.84% 18.62 ↓9.27% 18.13 ↓6.40% 17.97 ↓5.46% 4.25 ↓19.38% 4.02 ↓12.92% 4.00 ↓12.36% 3.72 ↓4.49% Forgetting (Toneva et al., 2018)19.77 ↓16.02% 18.84 ↓10.56% 18.16 ↓6.57% 17.81 ↓4.52% 4.24 ↓19.10% 3.97 ↓11.52% 3.80 ↓6.74% 3.74 ↓5.06% GraNd (Paul et al., 2021) 20.07 ↓17.78% 18.48 ↓8.45% 18.30 ↓7.39% 17.85 ↓4.75% 4.24 ↓19.10% 3.99 ↓12.08% 3.85 ↓8.15% 3.78 ↓6.18% Cal (Margatina et al., 2021) 19.71 ↓15.67% 18.66 ↓9.51% 18.77 ↓10.15% 18.14 ↓6.46% 4.22 ↓18.54% 3.99 ↓12.08% 3.83 ↓7.58% 3.75 ↓5.34% Glister (Killamsetty et al., 2021)20.71 ↓21.54% 19.10 ↓12.09% 18.87 ↓10.74% 18.40 ↓7.98% 4.29 ↓20.51% 4.13 ↓16.01% 3.90 ↓9.55% 3.78 ↓6.18% GraphCut (Iyer et al., 2021)20.21 ↓18.60% 18.55 ↓8.86% 18.01 ↓5.69% 17.78 ↓4.34% 4.29 ↓20.51% 3.95 ↓10.96% 3.84 ↓7.87% 3.75 ↓5.34% FaLo (Iyer et al., 2021)19.97 ↓17.19% 18.57 ↓8.98% 18.19 ↓6.75% 17.95 ↓5.34% 4.36 ↓22.47% 3.96 ↓11.24% 3.85 ↓8.15% 3.79 ↓6.46% Dynamic Soft Random21.57 ↓26.58% 18.86 ↓10.68% 18.49 ↓8.51% 17.57 ↓3.11% 4.25 ↓19.38% 3.86 ↓8.43% 3.74 ↓5.06% 3.69 ↓3.65% ε-greedy (Raju et al., 2021)20.25 ↓18.84% 18.82 ↓10.45% 18.43 ↓8.16% 17.97 ↓5.46% 4.15 ↓16.57% 3.84 ↓7.87% 3.85 ↓8.15% 3.77 ↓5.90% UCB (Raju et al., 2021)20.24 ↓18.78% 18.95 ↓11.21% 18.34 ↓7.63% 18.05 ↓5.93% 4.14 ↓16.29% 3.83 ↓7.58% 3.82 ↓7.30% 3.71 ↓4.21% InfoBatch (Qin et al., 2024)19.03 ↓11.68% 18.55 ↓8.86% 18.07 ↓6.04% 17.92 ↓5.16% 4.03 ↓13.20% 3.99 ↓12.08% 3.67 ↓3.08% 3.66 ↓2.81% ST-Prune (Our)18.32 ↓7.51 17.91 ↓5.11 17.63 ↓3.46 17.63 ↓3.46 3.63 ↓1.97 3.62 ↓1.69 3.58 ↓0.56 3.56 ↑0.00 Whole Dataset17.04 ±0.23 3.56 ±0.09 C More Discussion C.1 Current Limitation In this paper, we thoroughly investigate methods for accelerating spatio-temporal training via dynamic data pruning. . Based on the analysis of the characteristics of spatiotemporal data, we propose aST-Prune. While we have made some progress in this area, some limitations remain to be considered: ❶ Additional Computational Overhead. Our pruning strategy introduces a small but not negligible additional overhead. While this overhead is negligible when training large-scale spatio-temporal datasets or large backbone models (e.g., Transformers or deep GNNs), it can offset the efficiency gains when applying the model to extremely lightweight models (e.g., simple MLPs) or small datasets. Go to Appendix Index20 Table B.2 Performance comparison to state-of-the-art dataset pruning methods when remaining10%,30%,50%,70% of the full set. All methods are trained using GWNet, and the reported metric RMSE represent the average of five runs. DatasetPems08 (RMSE↓)UrbanEV (RMSE↓) Remaining Ratio %1030507010305070 Static Hard Random31.29 ↓16.28% 29.24 ↓8.66% 28.72 ↓6.73% 28.29 ↓5.13% 8.24 ↓22.07% 7.73 ↓14.52% 7.35 ↓8.89% 7.24 ↓7.26% CD (Agarwal et al., 2020) 31.31 ↓16.35% 29.04 ↓7.92% 28.61 ↓6.32% 28.10 ↓4.42% 8.42 ↓24.74% 7.69 ↓13.93% 7.37 ↓9.19% 7.28 ↓7.85% Herding (Welling, 2009)31.15 ↓15.76% 29.16 ↓8.36% 28.48 ↓5.83% 28.04 ↓4.20% 8.60 ↓27.41% 7.60 ↓12.59% 7.52 ↓11.41% 7.14 ↓5.78% K-Means (Sener and Savarese, 2018) 31.50 ↓17.06% 29.23 ↓8.62% 28.74 ↓6.80% 28.05 ↓4.24% 8.41 ↓24.59% 7.78 ↓15.26% 7.38 ↓9.33% 7.21 ↓6.81% Least Confidence (Coleman et al., 2019)32.36 ↓20.25% 30.11 ↓11.89% 28.54 ↓6.06% 28.43 ↓5.65% 8.29 ↓22.81% 8.65 ↓28.15% 7.36 ↓9.04% 7.07 ↓4.74% Entropy (Coleman et al., 2019) 32.46 ↓20.62% 29.01 ↓7.80% 28.48 ↓5.83% 28.18 ↓4.72% 8.39 ↓24.30% 7.68 ↓13.78% 7.37 ↓9.19% 7.28 ↓7.85% Margin (Coleman et al., 2019) 32.18 ↓19.58% 29.20 ↓8.51% 28.55 ↓6.09% 28.32 ↓5.24% 8.20 ↓21.48% 7.83 ↓16.00% 7.56 ↓12.00% 7.10 ↓5.19% Forgetting (Toneva et al., 2018)30.91 ↓14.86% 29.66 ↓10.22% 28.66 ↓6.50% 28.08 ↓4.35% 8.22 ↓21.78% 7.74 ↓14.67% 7.38 ↓9.33% 7.21 ↓6.81% GraNd (Paul et al., 2021) 31.08 ↓15.50% 29.01 ↓7.80% 28.70 ↓6.65% 28.22 ↓4.87% 8.23 ↓21.93% 7.76 ↓14.96% 7.42 ↓9.93% 7.31 ↓8.30% Cal (Margatina et al., 2021)30.57 ↓13.60% 29.34 ↓9.03% 29.20 ↓8.51% 28.48 ↓5.83% 8.29 ↓22.81% 7.74 ↓14.67% 7.36 ↓9.04% 7.20 ↓6.67% Glister (Killamsetty et al., 2021)32.32 ↓20.10% 30.20 ↓12.23% 29.83 ↓10.85% 29.30 ↓8.88% 8.23 ↓21.93% 8.06 ↓19.41% 7.60 ↓12.59% 7.34 ↓8.74% GraphCut (Iyer et al., 2021)31.35 ↓16.50% 29.19 ↓8.47% 28.37 ↓5.43% 28.11 ↓4.46% 8.47 ↓25.48% 7.67 ↓13.63% 7.44 ↓10.22% 7.15 ↓5.93% FaLo (Iyer et al., 2021)31.04 ↓15.35% 29.22 ↓8.58% 28.59 ↓6.24% 28.32 ↓5.24% 8.40 ↓24.44% 7.63 ↓13.04% 7.46 ↓10.52% 7.26 ↓7.56% Dynamic Soft Random32.71 ↓21.55% 29.44 ↓9.40% 28.82 ↓7.10% 27.86 ↓3.53% 8.29 ↓22.81% 7.45 ↓10.37% 7.19 ↓6.52% 7.24 ↓7.26% ε-greedy (Raju et al., 2021) 29.94 ↓11.26% 29.64 ↓10.14% 28.74 ↓6.80% 27.96 ↓3.90% 8.18 ↓21.19% 7.47 ↓10.67% 7.51 ↓11.26% 7.33 ↓8.59% UCB (Raju et al., 2021) 29.93 ↓11.22% 29.87 ↓11.00% 28.69 ↓6.61% 28.04 ↓4.20% 8.15 ↓20.74% 7.46 ↓10.52% 7.47 ↓10.67% 7.30 ↓8.15% InfoBatch (Qin et al., 2024)30.16 ↓12.08% 29.48 ↓9.55% 28.78 ↓6.94% 28.48 ↓5.83% 7.54 ↓11.70% 7.46 ↓10.52% 7.02 ↓4.00% 6.95 ↓2.96% ST-Prune (Our)28.37 ↓5.46 27.89 ↓3.64 27.61 ↓2.60 27.71 ↓2.97 6.91 ↓2.37 6.89 ↓2.07 6.75 ↓0.00 6.75 ↑0.00 Whole Dataset26.91 ±0.23 6.75 ±0.09 ❷Assumptions of Static Spatial Topology. Current structure scoring mechanisms rely on a consistent set of spatial nodesNto compute spatial variance. For dynamic graph scenarios where nodes frequently appear or disappear (e.g., the changing demand for ride-hailing services within an expanding region), the definition of ”spatial heterogeneity“ becomes ambiguous, potentially requiring a redesign of the scoring function. C.2 Future Work One lesson learned from our experiments is that the definition of “hardness” in spatio-temporal sample is not static but evolves during the training process. While our current method relies on handcrafted heuristics (variance and stationarity intensity) to define importance, we believe that an automated, learnable scoring policy could further adapt to the model’s changing needs. Thus, we view exploring reinforcement Learning or meta-learning to automatically learn the optimal scoring and reweighting functions as a long-term goal for future work. Go to Appendix Index21