Paper deep dive

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

Taehyung Kwon, Yeonje Choi, Yeongho Kim, Kijung Shin

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 100

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%

Last extracted: 3/13/2026, 1:10:29 AM

Summary

STemDist is a novel dataset distillation method designed for spatio-temporal time series forecasting. Unlike existing methods that only compress the temporal dimension, STemDist performs bi-dimensional compression by reducing both spatial and temporal dimensions. It utilizes a location encoder to maintain model applicability across different numbers of locations, employs cluster-level distillation for efficiency, and uses subset-based granular distillation to maintain forecasting performance. Empirical results show significant improvements in training speed, memory efficiency, and prediction accuracy.

Entities (5)

STemDist · method · 100%Dataset distillation · technique · 95%Spatio-temporal time series forecasting · task · 95%Location encoder · component · 90%STGNN · model-architecture · 90%

Relation Signals (3)

STemDist → compresses → Spatio-temporal time series

confidence 95% · A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner

STemDist → improves → Spatio-temporal time series forecasting

confidence 95% · datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective

STemDist → uses → Location encoder

confidence 95% · we integrate a location encoder module that produces embeddings independent of the number of locations

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multidimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective (with up to 12% lower prediction error).

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

99,730 characters extracted from source content.

Expand or collapse full text

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression Taehyung Kwon1, Yeonje Choi1, Yeongho Kim, Kijung Shin Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea taehyung.kwon, yeonjechoi, yeongho, kijungs@kaist.ac.kr Abstract Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multi-dimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset-distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6×) (2) more memory-efficient (up to 8×), and (3) more effective (with up to 12% lower prediction error). 11footnotetext: Equal contribution. I Introduction Figure 1: Synthetic data compressed in both temporal and spatial dimensions in a balanced manner yields significantly lower training time and GPU memory usage compared to datasets of the same size compressed only along the temporal dimension. See Preliminary Experiment 1 for details. Spatio-temporal time series datasets are universal, playing a key role in real-world applications, including traffic prediction [22, 14, 11, 30, 32] and weather forecasting [1, 39]. These datasets consist of observations from multiple locations over time, naturally represented as multi-dimensional arrays where different dimensions capture spatial and temporal information along with one or more measured features (e.g., wind, temperature, and pressure in weather datasets). Since such time series are typically collected from numerous locations over extended periods [22, 14, 41], they are often massive in scale. A central task using spatio-temporal time series is forecasting [39, 11, 53], which aims to predict future time‑series values across locations based on past observations. To this end, diverse deep learning methods [60, 58, 4, 57, 46] have been developed. However, as dataset sizes and model complexities continue to scale in real-world scenarios, training these models leads to significant storage demands and computation costs. Dataset distillation has emerged as a promising approach to mitigate the heavy computational and storage costs of training deep learning models. The key idea is to create a small and informative synthetic dataset that can replace original data in the training process while preserving comparable model performance. Several well-known approaches to it include gradient matching [64], trajectory matching [6], and meta-learning [49]. Gradient matching minimizes the discrepancy between gradients computed on original and synthetic data, while trajectory matching aligns the training dynamics of models trained on both datasets. Meta-learning focuses on improving the transferability of models trained on synthetic data to original datasets. Although dataset distillation has been mainly studied for image and graph data [28, 27, 19, 35], recent work has explored its applications to time series [37, 13, 40]. However, since existing dataset distillation methods typically compress only a single dimension, they are less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. In particular, time-series distillation techniques compress only the temporal dimension, while leaving the spatial dimension (i.e., the number of locations) unchanged, which can lead to substantial training time and GPU memory usage, as evidenced by the preliminary experiment below. Preliminary Experiment 1 (Necessity of Joint Reduction). In Figure 1, we compare two random sampling strategies with the same overall reduction ratio: (1) reducing only the temporal dimension by 0.5%, and (2) jointly reducing both dimensions by 5% (spatial) and 10% (temporal). After applying the strategies to five real-world spatio-temporal time-series datasets (see Section V-A), we measure the training time per outer iteration and GPU memory usage of MTGNN [56], a representative deep learning model for spatio-temporal forecasting. Figure 1 shows that, even with the same overall reduction ratio, jointly reducing the spatial and temporal dimensions significantly reduces training time and GPU memory usage, compared to temporal-only reduction. Refer to Section V-A for detailed experimental settings, and see also Theorems 1 and 2 in Section IV-F for related analyses. In this work, we propose STemDist (Spatio-Temperal Dataset Distillation), the first dataset distillation method specialized for spatio-temporal time series. A crucial step needed to compress the spatial dimension is to design a surrogate model that can be trained on a smaller number of locations (i.e., spatially compressed synthetic data) yet remains applicable for inference on a larger number of locations, which is not supported by most existing spatio-temporal forecasting models. To this end, we integrate a location encoder module that produces embeddings independent of the number of locations into Spatio-Temporal Graph Neural Networks (STGNNs), which are widely-used deep-learning models for spatio-temporal forecasting [66, 38, 55, 54, 8, 25, 10]. Using the surrogate model with the location encoder module, we build STemDist upon gradient matching (described above) with two enhancements. First, for efficiency, we reduce distillation time by performing distillation at the cluster level. Specifically, locations in the original dataset are grouped into clusters, and the averaged data within each cluster, rather than data from individual locations, are distilled. Second, to offset potential quality loss from this coarse-grained approach, we employ subset-based granular distillation. This technique distills data using varying subsets of clusters, ensuring that all parts (i.e., subsets) of the data are adequately reflected and thereby enhancing the quality of the output synthetic dataset. Using five real-world spatio-temporal datasets and nine baseline approaches, we conduct a comprehensive evaluation of STemDist, which reveals the following advantages over the baselines under the same compression ratio: • Faster: Model training is up to 6×6× faster on synthetic data distilled by STemDist than on data distilled by the baselines. • Memory-efficient: Model training requires up to 8×8× less GPU memory on synthetic data distilled by STemDist than on data distilled by the baselines. • Effective: Models trained on synthetic data distilled by STemDist achieve up to 12% lower prediction error compared to those trained on data distilled by the baselines. The rest of the paper is organized as follows. We present related works in Section I, preliminaries in Section I, and our proposed method in Section IV. We share experimental results in Section V and conclude our paper in Section VI. Reproducibility: The code and datasets are available at https://github.com/kbrother/STemDist. I Related Works Spatio-temporal time series forecasting: Spatio-temporal time series forecasting refers to the task of predicting future values of time series over a set of locations based on their historical observations [2]. It is a key task in time series analysis that requires effectively capturing both temporal dynamics and spatial dependencies. Recent advances largely rely on Spatio-Temporal Graph Neural Networks (STGNNs) [26, 33, 12, 44, 30], which jointly model spatio-temporal dependencies and achieve state-of-the-art forecasting performance. Previous works on STGNNs can be categorized into: (1) Prior-based models [15, 16, 60, 34], which model spatial dependencies between locations using a predefined adjacency matrix, i.e., explicit structural information, and (2) Structure-learning-based models [45, 4, 58, 9, 56], which infer the adjacency matrix end-to-end through a dedicated graph learning module. In this work, we adopt structure-learning-based models for dataset distillation due to their flexibility and scalability. Since the adjacency matrix is learned dynamically, neither predefined nor stored graph structures are needed during training or distillation. However, most STGNNs cannot be directly employed as surrogate models in our distillation framework, since they require the number of locations to be identical during training and testing, as elaborated in Section IV-A. As a particularly relevant STGNN-based approach, Zhou et al. [68] proposed an inductive approach that infers embeddings for unobserved locations using an explicit graph structure and embeddings for observed locations. This is related to our location encoder (Section IV-C), which also yields location embeddings. While their embeddings transfer knowledge from observed to unobserved locations, ours operate in the conceptually opposite direction by enabling knowledge transfer from a synthetic dataset to a real dataset without graph structure. Dataset distillation for general data: Dataset distillation aims to transform original datasets into smaller-scale synthetic datasets that serve as effective substitutes for the original datasets in machine-learning model training [43]. Existing approaches can be categorized into: (1) Coreset selecting methods [7, 50, 17, 35, 42], which select a few representative samples from the original dataset, (2) Gradient matching based methods [64, 19, 62, 52, 18], which reduce the discrepancy between the gradients of models when trained on the original and synthetic datasets, (3) Trajectory matching based methods [6, 21], which minimize the discrepancy between the training trajectories of model parameters obtained on the original and synthetic datasets, (4) Meta-learning based methods [69], which directly optimize the transferability of models trained on the synthetic dataset to the original dataset, and (5) Distribution matching based methods [63, 65], which directly align the output distributions of models initialized with random parameters. Our proposed distillation method, STemDist, builds upon gradient matching due to its simplicity and superior empirical effectiveness over alternatives on spatio-temporal time series datasets (refer to Section V-B for empirical comparisons). Dataset distillation for time series: Several dataset distillation methods have been developed specifically for time series datasets. CondTSF [13] adapts general dataset distillation techniques to time series forecasting by incorporating a plugin that refines the synthetic dataset based on the outputs of a model trained on the original dataset. TimeDC [40] combines decomposition-driven frequency matching with trajectory matching. CondTSC [37] enhances the matching process through frequency-aware data augmentation, jointly leveraging temporal and frequency-domain information. It is worth noting, however, that these methods are designed for uni- or multi-variate time series rather than spatio-temporal datasets, which are the focus of our work. Thus, when applied to spatio-temporal data, they compress only the temporal dimension while leaving the spatial dimension (i.e., the number of locations) unchanged, leading to substantial training costs even after distillation (see Preliminary Experiment 1 in Section I). Moreover, they are designed for and applied to models [51, 67] that do not explicitly capture correlations across different locations. In addition, CondTSC [37] is developed for classification, whereas our work targets forecasting. These gaps motivate the development of dataset distillation methods specialized for spatio-temporal forecasting. Dataset distillation for graphs: Several studies [28, 27, 61, 36, 48] have explored graph dataset distillation. They focus on reducing the number of nodes in a predefined graph topology. Our problem differs from theirs in that spatio-temporal time series datasets lack a predefined graph structure, and we aim to compress both spatial and temporal dimensions. I Preliminaries and problem definition TABLE I: Descriptions of frequently used notations Symbol Description Z∈ℝM×F×NZ ^M× F× N Spatio-temporal time series zt∈ℝF×Nz_t ^F× N t-th observation in Z M Total length of time series F Number of (input) features N Number of locations Xt∈ℝLin×F×NX_t ^L_in× F× N Input time series for time t Yt∈ℝLout×Fout×NY_t ^L_out× F_out× N Target time series for time t LinL_in, LoutL_out Lengths of input and target windows FoutF_out Number of target features =(,)T=(X_T,Y_T) Spatio-temporal time series dataset ∈ℝM×Lin×F×NX_T ^M_T× L_in× F× N_T List of input time series in T ∈ℝM×Lout×Fout×NY_T ^M_T× L_out× F_out× N_T List of target time series in T M_T Number of input-target pairs in T N_T Number of locations in T =(,)S=(X_S,Y_S) Synthetic time series dataset ∈ℝM×Lin×F×NX_S ^M_S× L_in× F× N_S List of input time series in S ∈ℝM×Lout×Fout×NY_S ^M_S× L_out× F_out× N_S List of target time series in S M_S Number of input-target pairs in S N_S Number of locations in S mat∈ℝN×(MLinF)X^mat_T ^N_T×(M_TL_inF) Reshaped version of X_T C Clustered time series dataset K Number of location sets Dkk=0K−1\D_k\_k=0^K-1 Divided location sets This section presents the basic concepts and the formal definition of the problem, with a brief overview of gradient matching. Frequently used notations are listed in Table I. I-A Basic Concepts We first define a spatio-temporal time series as follows: Definition 1 (Spatio-temporal Time Series). A spatio-temporal time series Z=(z0,⋯,zM−1)Z=(z_0,·s,z_M-1) is a sequence of M observations, where each observation zt∈ℝF×Nz_t ^F× N contains F features from N spatial locations. Following prior works [56, 60, 4, 58], we define the spatio-temporal time series forecasting problem as predicting the future values of target features from a window of past observations, where the target features are typically a subset of the input features, as follows: Problem 1. (Spatio-temporal Time Series Forecasting) • Given: a past input spatio-temporal time series Xt=(zt−Lin,⋯,X_t=(z_t-L_in,·s, zt−1)∈ℝLin×F×Nz_t-1) ^L_in× F× N of length LinL_in, • Predict: the future target spatio-temporal time series Yt=(zt′,⋯,Y_t=(z _t,·s, zt+Lout−1′)∈ℝLout×Fout×Nz _t+L_out-1) ^L_out× F_out× N of length LoutL_out, where zt′∈ℝFout×Nz _t ^F_out× N is the FoutF_out target feature values at time t. Using the input XtX_t and target YtY_t defined above, we formally define the spatio-temporal time series dataset as follows. Definition 2 (Spatio-temporal Time Series Dataset). A spatio-temporal time series dataset is defined as =(,)=((Xt)t=0M−1,(Yt)t=0M−1),T=(X_T,Y_T)= ((X_t)_t=0^M_T-1,\,(Y_t)_t=0^M_T-1 ), where =(Xt)t=0M−1∈ℝM×Lin×F×NX_T=(X_t)_t=0^M_T-1 ^M_T× L_in× F× N_T denotes a list of input spatio-temporal time series, =(Yt)t=0M−1∈ℝM×Lout×Fout×NY_T=(Y_t)_t=0^M_T-1 ^M_T× L_out× F_out× N_T denotes a list of target spatio-temporal time series, M_T denotes the number of input and target spatio-temporal time-series pairs, and N_T denotes the number of locations in the dataset. An example of creating the spatio-temporal time series dataset from spatio-temporal time series is in Appendix A [31]. Similarly, we define a synthetic spatio-temporal time series dataset (hereafter referred to as a synthetic dataset), which is obtained through dataset distillation, as follows: Definition 3 (Synthetic Dataset). A synthetic spatio-temporal time series dataset is defined as =(,)=((X~i)i=0M−1,(Y~i)i=0M−1),S=(X_S,Y_S)= (( X_i)_i=0^M_S-1,( Y_i)_i=0^M_S-1 ), where =(X~i)i=0M−1∈ℝM×Lin×F×NX_S=( X_i)_i=0^M_S-1 ^M_S× L_in× F× N_S denotes a list of input synthetic time series, =(Y~i)i=0M−1∈ℝM×Lout×Fout×NY_S=( Y_i)_i=0^M_S-1 ^M_S× L_out× F_out× N_S denotes a list of target synthetic time series, M_S denotes the number of input-target pairs, and N_S denotes the number of locations in the synthetic dataset. I-B Problem Definition Building on the basic concepts, we now define the problem of dataset distillation for spatio-temporal time series forecasting. Let gθg_θ denote a (predefined or arbitrary) machine learning model for spatio-temporal time series forecasting with parameters θ; and θ _S is the set of parameters obtained by training on S. Dataset distillation aims to construct a synthetic dataset S such that the model trained on it, gθg_ _S, achieves the best performance on ground-truth data distribution P, i.e., min⁡(X,Y)∼P[ℓ(gθ(X),Y)], _SE_(X,Y) P[ (g_ _S(X),Y)], (1) where ℓ is a loss function (e.g., sum of squared error). Since the ground-truth data distribution P is unknown in practice, we formulate the problem using a given dataset T as follows: Problem 2. (Dataset Distillation for Spatio-temporal Time Series Forecasting) • Given: a spatio-temporal time series dataset T, • Find: a synthetic dataset S, • to Minimize: the forecasting error of gθSg_ _S on T, i.e., min⁡ℒ(gθ,)subject toθ=arg⁡minθ⁡ℒ(gθ,), _SL(g_ _S,T) to _S= _θL(g_θ,S), where ℒ(gθS,)=1M∑t=0M−1ℓ(gθS(Xt),Yt)L(g_ _S,T)= 1M_T _t=0^M_T-1 (g_ _S(X_t),Y_t) is the average prediction error of gθSg_ _S on T; and ℒ(gθ,)=1M∑i=0M−1ℓ(gθ(X~i),Y~i)L(g_θ,S)= 1M_S _i=0^M_S-1 (g_θ( X_i), Y_i) is that of gθg_θ on S. I-C Gradient Matching In this section, we briefly review gradient matching [64], a dataset distillation method that our method builds upon. Gradient matching aims to learn synthetic data such that the model parameters θ _S trained on it converge to a solution close to parameters θ _T obtained from model training on original data. However, θ _S often converges to a local minimum different from θ _T, making direct alignment with θ _T ineffective. Gradient matching addresses this issue by aligning the gradients computed on T and S at every update of θ _S, even before convergence, without depending on the values of θ _T. To formalize the gradient matching objective, we denote the parameters of a surrogate machine learning model fθf_θ after t optimization steps as θt _t, initialized with θ0 _0. The objective is to minimize the distance between the gradients computed on synthetic data S and original data T along the optimization trajectory from t=0t=0 to Tmax−1T_max-1, i.e., min⁡θ0∼Pθ0[∑t=0Tmax−1Dist(∇θℒ(,θt),∇θℒ(,θt))], _SE_ _0 P_ _0 [ _t=0^T_max-1Dist( _θL(S, _t), _θL(T, _t)) ], subject toθt←θt−1−η∇θℒ(S,θt−1),subject to _t← _t-1-η _θL(S, _t-1), where Pθ0P_ _0 is the distribution of the initial parameter, η is the learning rate, DistDist is the sum of cosine distances between two gradients, and ℒL is the training loss function. IV Proposed Method In this section, we present STemDist (Spatio-Temperal Dataset Distillation), our proposed dataset distillation method for spatio-temporal time-series forecasting. We first outline the key challenges in solving Problem 2 and our corresponding solutions (Section IV-A). Then, we provide an overview of STemDist (Section IV-B) and describe the three key components of STemDist: location encoders (Section IV-C), clustering (Section IV-D), and granular distillation (Section IV-E). Lastly, we provide a complexity analysis (Section IV-F). IV-A Key Challenges and Our Corresponding Solutions There are three key challenges in solving Problem 2: • C1. Existing methods insufficiently reduce training cost. Existing dataset distillation methods focus on compressing only a single dimension, the temporal dimension in our context (i.e., M_S in Definition 3). However, the training cost of forecasting models is also affected by the number of locations (i.e., N_S in Definition 3), which contributes quadratically to the computational cost. Thus, training remains expensive without compressing the spatial dimension (i.e., the number of locations). • C2: Dataset distillation itself is time-consuming. The ultimate goal of dataset distillation is to reduce the overall computational cost associated with model training. Thus, if a distillation process itself is too costly, then regardless of its performance, a dataset distillation method fails to serve its intended purpose. Therefore, the distillation cost, which often grows substantially with the number of locations (i.e., N_T in Definition 2), must remain computationally efficient. • C3: Synthetic datasets may not fully capture original data. Real-world spatio-temporal time series are often collected from a large number of locations. The data from every location should be effectively distilled for effective model training on the synthetic data. However, when the number of locations is large, it becomes challenging to account for all of them during distillation, particularly given the limited capacity of synthetic datasets. To tackle these challenges, our proposed method, STemDist, introduces three corresponding solutions: • S1. Simultaneous compression of temporal and spatial dimensions with location encoders. To address C1, we jointly compress both temporal and spatial dimensions (i.e., M_S and N_S in Definition 3). To the best of our knowledge, we are the first to propose such a bi-dimensional dataset distillation strategy. A key to implementing this idea is the introduction of location encoders (see Section IV-C), which allow deep learning models trained on a smaller number of locations to be applied for inference on a larger number of locations. Jointly compressing both dimensions yields a synthetic dataset that balances both aspects, which not only lowers training cost but also improves model performance. • S2. Clustering of locations in original datasets. To tackle C2, we reduce the number of locations in the original dataset (i.e., N_T in Definition 2) by clustering them. The time series of individual locations are replaced with those of the cluster centroids, resulting in a reduced version of the original dataset, as detailed in Section IV-D. Distilling this clustered dataset speeds up the overall distillation process. Note that this idea of reducing the number of locations in the original dataset (N_T) is distinct from the earlier idea of reducing that in synthetic datasets (N_S). • S3. Subset-based granular distillation. To address C3, we propose subset-based granular distillation. Instead of using all locations at once, the distillation process leverages varying subsets of locations. This allows different parts (i.e., subsets) of the data to be more effectively reflected in the synthetic dataset, improving its overall quality. Note that this idea complements the cluster-based coarse-grained approach by mitigating potential quality loss caused by clustering. These solutions reflect the central idea of our method, and their details are provided in Sections IV-C, IV-D, and IV-E. IV-B Overview of STemDist In this subsection, we provide an overview of STemDist, with its overall procedure summarized in Algorithm 1. Lines 1 - 1 describe the preprocessing steps before distillation. First, locations in the original dataset T are clustered to obtain a spatially reduced version, =(,)C=(X_C,Y_C), and the cluster weights (wi)i=0N−1(w_i)_i=0^N_S-1, proportional to the number of locations in each cluster (line 1). The clustered dataset C replaces T in subsequent steps, accelerating the distillation process (see S2 in Section IV-A). Details for the clustering process are provided in Section IV-D (see Algorithm 3). Then, the input of the location encoders (see S1 in Section IV-A), I_C is computed from X_C in advance (line 1) and subsequently used in the following steps. The synthetic dataset =(,)S=(X_S,Y_S) is initialized by random sampling from C (line 1), naturally assigning each cluster to a synthetic location in S.111As distillation progresses, the semantic meaning of synthetic locations may be obscured, since distillation is designed to optimize training equivalence rather than interpretability. Thus, synthetic locations should be interpreted as training-efficient proxies rather than semantically meaningful locations. Lines 1 - 1 describe the distillation process. Given the preprocessed inputs (C, I_C, and initialized S), it outputs the distilled synthetic data S. Since the clustered dataset C can be large along the temporal dimension, we perform distillation using mini-batch pairs (B,B)(X_C^B,Y_C^B) sampled along the temporal dimension from =(,)C=(X_C,Y_C) (line 1). For subset-based granular distillation (see S3 in Section IV-A), we partition the locations into K disjoint subsets Dkk=0K−1\D_k\_k=0^K-1 (line 1). For each location subset DkD_k, we perform gradient matching (see Section I-C) to update the synthetic time series of the corresponding synthetic locations (lines 1 - 1), and employ an STGNN equipped with our location encoder module (see Section IV-C) as a surrogate model fθf_θ. A mini-batch BX_C^B of the input time series from the clustered dataset C and the input time series X_S of the synthetic dataset S are given to the model fθf_θ along with the inputs of the location encoders, I_C and I_S (lines 1 and 1). Then, the losses for C and S are computed considering the weights (wi)i=0N−1(w_i)_i=0^N_S-1 of clusters, assigned to each synthetic location (lines 1 and 1). After that, the discrepancy between the gradients is measured, yielding the gradient matching loss (line 1), which is backpropagated to update the synthetic dataset S (line 1). After updating the synthetic dataset for all location subsets, the parameters of fθf_θ are updated using the current synthetic dataset (line 1). The detailed training process of the surrogate model fθf_θ on the synthetic dataset =(,)S=(X_S,Y_S) is provided in Algorithm 2. Given S, we first compute the input I_S for the location encoders (line 2). For each training epoch t, the locations in the synthetic dataset are randomly partitioned into K disjoint subsets (line 2), to be consistent with the subset-level granularity used in the distillation stage. For each subset DkD_k, the model fθf_θ generates predictions for the corresponding inputs of the synthetic dataset (line 2). The weighted loss used also in the distillation process is computed (line 2), and finally, the model parameters are updated by gradient descent (line 2). Input: (1) the original dataset =(,)T=(X_T,Y_T), (2) a number TouterT_outer of outer iterations, (3) a number TdistillT_distill of iterations for distillation, (4) a number TmodelT_model of epochs for model training, (5) a number K of location sets, (6) a learning rate η for θt _t. Output: Synthetic dataset =(,)S=(X_S,Y_S). 1 2Cluster the locations in T to get the clustered dataset =(,)C=(X_C,Y_C) and cluster weights (wi)i=0N−1(w_i)_i=0^N_S-1 3 ⊳ Algo. 3 and Sect. IV-D 4Compute the input I_C of the location encoder from X_C 5 ⊳ Sect. IV-C 6Initialize =(,)S=(X_S,Y_S) by random sampling from (,)(X_C,Y_C) 7for touter←0,⋯,Touter−1t_outer← 0,·s,T_outer-1 do 8 9 Initialize the surrogate model fθf_θ with θ0∼Pθ0 _0 P_ _0 10 for t←0,⋯,Tdistill−1t← 0,·s,T_distill-1 do 11 Sample a mini-batch pair B∼X_C^B _C and B∼Y_C^B _C 12 Partition locations 0,⋯,N−1\0,·s,N_S-1\ into Dkk=0K−1\D_k\_k=0^K-1 13 ⊳ Sect. IV-E 14 for k←0,⋯,K−1k← 0,·s,K-1 do 15 16 Compute the input I_S of the location encoder from X_S ⊳ Sect. IV-C 17 ~←fθt(B(:,:,:,Dk),I(Dk,:)) Y_C← f_ _t(X_C^B(:,:,:,D_k),I_C(D_k,:)) 18 ℒ(,θt)←∑i∈Dkwi‖(~−B)(:,:,:,i)‖22L(C, _t)← _i∈ D_kw_i\|( Y_C-Y_C^B)(:,:,:,i)\|_2^2 19 ~←fθt((:,:,:,Dk),I(Dk,:)) Y_S← f_ _t(X_S(:,:,:,D_k),I_S(D_k,:)) 20 ℒ(,θt)←∑i∈Dkwi‖(~−)(:,:,:,i)‖22L(S, _t)← _i∈ D_kw_i\|( Y_S-Y_S)(:,:,:,i)\|_2^2 21 ℒgrad←Dist(∇θℒ(,θt),∇θℒ(,θt))L^grad← Dist( _θL(C, _t), _θL(S, _t)) 22 Backpropagate ℒgradL^grad and update (:,:,:,Dk)X_S(:,:,:,D_k) and (:,:,:,Dk)Y_S(:,:,:,D_k) ⊳ Sect. IV-E 23 θt+1← _t+1← Training the surrogate model fθf_θ using (,(X_S, ,Tmodel,K,η,θt,Y_S,T_model,K,η, _t, and (wi)i=0N−1)(w_i)_i=0^N_S-1) ⊳ Algo. 2 return ((X_S, )Y_S) Algorithm 1 Dataset Distillation in STemDist Input: (1) a synthetic dataset =(,)S=(X_S,Y_S), (2) a number T of training epochs, (3) a number K of locations sets, (4) a learning rate η, initial parameters θ0 _0, (5) cluster weights (wi)i=0N−1(w_i)_i=0^N_S-1. Output: trained parameters θT _T of the (surrogate) model fθf_θ. 1 Compute the input I_S of the location encoder from X_S 2 ⊳ Sect. IV-C 3for t←0,⋯,T−1t← 0,·s,T-1 do 4 Dkk=0K−1←\D_k\_k=0^K-1← Partition locations 0,⋯,N−1\0,·s,N_S-1\ 5 ⊳ Sect. IV-E 6 for k←0,⋯,K−1k← 0,·s,K-1 do 7 ~←fθt((:,:,:,Dk),I(Dk,:)) Y_S← f_ _t(X_S(:,:,:,D_k),I_S(D_k,:)) 8 ℒ(,θt)←∑i∈Dkwi‖(~−)(:,:,:,i)‖22L(S, _t)← _i∈ D_kw_i\|( Y_S-Y_S)(:,:,:,i)\|_2^2 9 θt←θt−η∇θℒ(,θt) _t← _t-η _θL(S, _t) 10 θt+1←θt _t+1← _t return θT _T Algorithm 2 (Surrogate) Model Training in STemDist IV-C Location Encoder In this subsection, we elaborate on our first solution, S1 introduced in Section IV-A. Specifically, we present the location encoder module that enhances STGNNs (described in Section I), which serve as the surrogate model fθf_θ in the distillation process and, more generally, as powerful models for spatio-temporal time series forecasting. As described in C1 in Section IV-A, while compressing the spatial dimension (in addition to the temporal dimension) is desirable for dataset distillation, it is not straightforward because STGNNs (and also most alternatives) trained on a (synthetic) dataset with fewer locations are inapplicable for inference on a larger number of locations. To address this, we introduce a location encoder that makes STGNNs inductive with respect to the number of locations, as described below. Transductivity of STGNNs: We first describe the components of STGNNs that make them inapplicable when the locations differ between training and inference. In the absence of a predefined graph (i.e., spatial dependencies between locations), STGNNs [4, 58, 56] learn the graph structure through transductive (i.e., directly learnable) location embeddings. For example, in AGCRN [4] and graph wavenet [58], an adjacency matrix A is derived as follows: A←Softmax(ReLU(E1TE2)),A (ReLU(E_1^TE_2)), (2) where E1∈ℝN×RE_1 ^N× R and E2∈ℝN×RE_2 ^N× R are transductive embeddings of dimension R for N locations, and SoftmaxSoftmax and ReLUReLU are activation functions. Note that the trained parameters are directly tied to the locations in the training dataset (i.e., the synthetic dataset in our dataset distillation context), making the STGNNs transductive. That is, when the semantics of locations change or when new locations are introduced during inference, the learned embeddings and the derived adjacency matrix become inapplicable. Overview of location encoders: Our approach to addressing this limitation is to use a location encoder that can replace the transductive location embeddings. Specifically, for spatio-temporal time series with an arbitrary number of locations, the encoder is designed to generate embeddings for each location in the dataset. Especially, even when trained with a smaller number of (synthetic) locations in the synthetic dataset S, the encoder should be capable of generalizing to a larger number of locations in the original dataset T during inference. To this end, we design the location encoder as a sequence-to-sequence architecture, since both the inputs (time series from multiple locations) and the outputs (corresponding location embeddings) are naturally represented as sequences indexed by locations. The encoder is further required to satisfy (1) Length-insensitivity: it should process sequences of arbitrary length, enabling it to handle both the synthetic and original datasets with different numbers of locations, and (2) Parameter sharing: it uses shared parameters to generate each location embedding, facilitating consistency and generalization across varying numbers of locations. Note that, while we focus on a sequence-to-sequence design, other architectures satisfying these properties may also be viable. Inputs and outputs of location encoders: To obtain the input to the location encoder, we preprocess ∈ℝM×Lin×F×NX_T ^M_T× L_in× F× N_T when using the original data T and ∈ℝM×Lin×F×NX_S ^M_S× L_in× F× N_S when using the synthetic data S (refer to Definitions 2 and 3 for X_T and X_S). Since they can be excessively long, we first average them along the temporal dimension. That is, ¯ X_T and ¯ X_S are computed as follows: ¯(,)←1M(,)∑m=0M(,)−1(,)(m,:,:,:). X_(T,S)← 1M_(T,S)Σ _m=0^M_(T,S)-1X_(T,S)(m,:,:,:). -2.84526pt (3) Then, we reshape ¯∈ℝLin×F×N X_T ^L_in× F× N_T and ¯∈ℝLin×F×N X_S ^L_in× F× N_S into matrices I∈ℝN×LinFI_T ^N_T× L_inF and I∈ℝN×LinFI_S ^N_S× L_inF, respectively, which are used as the input of the location encoder. Given the input I_T (or I_S), the location encoder outputs a matrix E∈ℝN×RE_T ^N_T× R (or E∈ℝN×RE_S ^N_S× R), where each row represents an R-dimensional embedding of the corresponding location. These output matrices are used (in place of the transductive location embeddings) to construct the adjacency matrix, following the structure-learning paradigm of STGNNs. Figure 2: An example illustrating location clustering, where eight locations in X_T are reduced into four locations (cluster centroids) in X_C. Entries with the same color correspond to the data for the same location. Refer to Example 1 for details. Detailed architecture of location encoders: The location encoder consists of a single-head self-attention layer [47] preceded and followed by linear layers. A residual connection is applied around the attention layer, followed by layer normalization [3]. For clarity, we present the equations using I_T, but they can also be expressed with I_S. H1=Linear1(I),H2=SelfAttn(H1),E=Linear2(LayerNorm(H1+H2)). split&H_1=Linear_1(I_T),~H_2=SelfAttn(H_1),\\ &E_T=Linear_2 (LayerNorm(H_1+H_2) ). split (4) The hidden vectors are denoted by H1∈ℝN×dH_1 ^N_T× d and H2∈ℝN×dH_2 ^N_T× d, with d representing the hidden dimension. By the self-attention layer (SelfAttnSelfAttn), each i-th row of H2H_2, the hidden vector for the i-th location, is computed as follows: H2(i,:)=∑j=1Nα(i,j)H1(j,:)WVWO, where H_2(i,:)=Σ _j=1^N_Tα(i,j)H_1(j,:)W_VW_O, where (5) α(i,j) α(i,j) =exp((H1(i,:)WQ)⋅(H1(j,:)WK)T/d)∑n=1Nexp((H1(i,:)WQ)⋅(H1(n,:)WK)T/d). = exp ((H_1(i,:)W_Q)·(H_1(j,:)W_K)^T/ d ) _n=1^N_Texp ((H_1(i,:)W_Q)·(H_1(n,:)W_K)^T/ d ). Note that WQ,WK,WVW_Q,W_K,W_V, and WO∈ℝd×dW_O ^d× d are learnable parameters. The self-attention layer computes the importance score α(i,j)α(i,j) of each j-th location with respect to the i-th location. Then, the information extracted from each j-th location, H1(j,:)WVH_1(j,:)W_V, is aggregated using α(i,j)α(i,j) as the weight to produce the hidden vector for the i-th location, H2(i,:)H_2(i,:). Thus, the resulting hidden vector H2(i,:)H_2(i,:) captures the information from all locations, weighted by their learned relevance to the i-th location. This architecture satisfies the two aforementioned properties required by location encoders. All components in Eq. (4) and Eq. (5) are length-insensitive, processing sequences of arbitrary length with parameter sizes independent of the sequence length. In addition, all parameters (i.e., those of the linear layers, layer normalization, and WQW_Q, WKW_K, WVW_V, and WOW_O) are shared across all locations. Thus, parameter sharing is achieved facilitating consistency and generalization across varying numbers of locations. Therefore, our location encoder enables STGNNs to be applicable and effective for datasets with varying numbers of locations, including the original dataset with more locations than the synthetic training dataset. Input: (1) the original dataset (,)(X_T,Y_T) (2) a number TkmeansT_kmeans of K-means iterations Output: (1) the clustered dataset (,)(X_C,Y_C) (2) the cluster weights (wi)i=0N−1(w_i)_i=0^N_S-1 1 Reshape X_T to matX^mat_T and Y_T to matY^mat_T 2Cii=0N−1←\C_i\_i=0^N_S-1← K-means(matX^mat_T, TkmeansT_kmeans) 3for i←0i← 0 to N−1N_S-1 do 4 5 (mat,mat)(i,:)←1|Ci|∑l∈Ci(mat,mat)(l,:)(X^mat_C,Y^mat_C)(i,:)← 1|C_i| _l∈ C_i(X^mat_T,Y^mat_T)(l,:) 6 wi←|Ci|Nw_i← |C_i|N_T 7Reshape matX^mat_C to X_C and matY^mat_C to Y_C return (,)(X_C,Y_C), (wi)i=0N−1(w_i)_i=0^N_S-1 Algorithm 3 Location Clustering in STemDist IV-D Clustering of Locations As discussed in C2 in Section IV-A, reducing the number of locations in the original dataset is necessary for efficient distillation. We address this by location clustering (A2) described in Algorithm 3, which consists of three steps: 1. We reshape the input-target time series pairs (,)(X_T,Y_T) ∈ (ℝM×Lin×F×N,ℝM×Lout×Fout×N)(R^M_T× L_in× F× N_T,R^M_T× L_out× F_out× N_T) of the original dataset T into matrices (mat,mat)∈(ℝN×(MLinF),ℝN×(MLoutFout))(X^mat_T,Y^mat_T)∈(R^N_T×(M_TL_inF),R^N_T×(M_TL_outF_out)) (line 3). Each row of matX^mat_T, matY^mat_T corresponds to a location in T. 2. Then, we treat each row of matX^mat_T as a feature vector for a location and apply K-means clustering on matX^mat_T to group N_T locations into N_S clusters (line 3). For each cluster CiC_i, we compute its representative feature vector by averaging over all the feature vectors of the locations assigned to CiC_i, yielding mat(i,:)X^mat_C(i,:) and mat(i,:)Y^mat_C(i,:) (lines 3). 3. Lastly, we reshape the obtained features of the clusters (mat,mat)∈(ℝN×(MLinF),(X^mat_C,Y^mat_C)∈(R^N_S×(M_TL_inF), ℝN×(MLoutFout))R^N_S×(M_TL_outF_out)) into original dimensions (,)∈(X_C,Y_C)∈ (ℝM×Lin×F×N,(R^M_T× L_in× F× N_S, ℝM×Lout×Fout×N)R^M_T× L_out× F_out× N_S), resulting in the clustered dataset C. We treat (,)(X_C,Y_C) as the input–target time series pairs from the reduced locations, each representing a cluster. Example 1. In Figure 2, we illustrate the clustering process for the input time series X_T. The same procedure is applied to the target time series Y_T to obtain Y_C. Entries with the same color correspond to the same location. First, ∈ℝ4×3×3×8X_T ^4× 3× 3× 8 is reshaped into mat∈ℝ8×36X^mat_T ^8× 36. We cluster the rows of matX^mat_T and use the result to build matX^mat_C. In this example, mat(0,:)X^mat_T(0,:) and mat(2,:)X^mat_T(2,:) are averaged to mat(0,:)X^mat_C(0,:), mat(3,:)X^mat_T(3,:) and mat(6,:)X^mat_T(6,:) are averaged to mat(1,:)X^mat_C(1,:), mat(4,:)X^mat_T(4,:) and mat(7,:)X^mat_T(7,:) are averaged to mat(2,:)X^mat_C(2,:), and mat(1,:)X^mat_T(1,:) and mat(5,:)X^mat_T(5,:) are averaged to mat(3,:)X^mat_C(3,:). Lastly, matX^mat_C of size 4×364× 36 is reshaped into X_C of size 4×3×3×44× 3× 3× 4. The clusters in C (i.e., reduced locations) may correspond to different numbers of locations in T. To account for this imbalance, we assign weights wi=0N−1\w\_i=0^N_S-1, where each weight is defined as the ratio between the number of locations contained in the corresponding cluster and the total number of locations in T (line 3). These weights are used for loss computations in the subsequent distillation process (lines 1 and 1 of Algorithm 1, and line 2 of Algorithm 2). As shown in the complexity analysis in Section IV-F, distillation cost grows quadratically with the number of locations. By replacing the original dataset with its clustered counterpart, this number is reduced, thereby improving both the speed and efficiency of distillation. Beyond its effectiveness, subset-based granular distillation also reduces the time and space complexity of STemDist by reducing the number of locations processed at once (see Theorem 3 in Section IV-F for details). Alternatives of K-means clustering: K-means can be replaced with alternative clustering methods, such as K-medoids [70].222Zorrilla et al. [70] use K-medoids to cluster locations and train a representative model (spec., ARIMA) for each cluster. During the online phase, a suitable representative model is selected for each query location to make predictions. This strategy avoids training a separate model for every location, thereby reducing overall training cost. Their strategy of training multiple cluster-specific models for selection stands in contrast to our approach that trains a single model (specifically, an STGNN) that jointly incorporates all locations and their spatial relationships using a reduced synthetic dataset. Since simple clustering methods may blur local topological structures by ignoring geographical relationships, when geographic information (e.g., longitude and latitude) of locations is given, spatially informed clustering methods [20, 29] can be used instead, as explored in Section V-E. Figure 3: An illustrative example showing how the clustered spatio-temporal dataset (which is averaged within each cluster) are partitioned into two location subsets and distilled at the subset level. Entries with the same color correspond to the results for the same location. Refer to Example 2 for details. IV-E Subset-based Granular Distillation In this subsection, we present subset-based granular distillation (S3 in Section IV-A). Our approach repeats distillation on fewer, randomly chosen subsets of locations. This enables weaker but important inter-location correlations, which are often overlooked by simultaneous distillation over all locations (C3 in Section IV-A), to be effectively reflected in the synthetic dataset. Thus, the quality of the synthetic dataset is improved, as we empirically show in Section V-E. Below, we describe the details of subset-based granular distillation. At each distillation iteration, we randomly partition the set of locations 0,…,N−1\0,...,N_S-1\ of the clustered dataset C (or the synthetic dataset S) into K non-overlapping subsets Dkk=0K−1\D_k\_k=0^K-1 of equal size NK N_SK (line 1 in Algorithm 1; see also Example 2). We re-partition the locations at every iteration to form a new set of different subsets to capture diverse spatial correlations among locations throughout the distillation process. Example 2. In Figure 3, we provide a partitioning example, where the locations are divided into two subsets (K=2K=2): D0=0,3D_0=\0,3\ and D1=1,2D_1=\1,2\. X_C and Y_C are split accordingly, and each subset pair (,)(:,:,:,Dk)(X_C,Y_C)(:,:,:,D_k) is used to distill the corresponding synthetic subset (,)(:,:,:,Dk)(X_S,Y_S)(:,:,:,D_k). The partitioned subsets are used in four ways. First, they are used to construct the subset-level inputs from the original inputs (i.e. CBX^B_C, X_S, ICI_C, and ISI_S) for the surrogate model. Second, the surrogate model fθf_θ, which receives these subset-level inputs, produces subset-level outputs, ~ Y_C and ~ Y_S (lines 1 and 1). Third, they are used to compute the subset-level losses. For the k-th subset DkD_k, the losses are computed within DkD_k by summing the reconstruction errors of its constituent locations i∈Dki∈ D_k (lines 1 and 1). Lastly, the gradients of the synthetic and clustered original datasets, computed from these losses, are applied to update the subset-specific synthetic time series (:,:,:,Dk)X_S(:,:,:,D_k) and (:,:,:,Dk)Y_S(:,:,:,D_k) (lines 1 and 1). IV-F Complexity Analysis In this subsection, we analyze the time and space complexities of the aforedescribed algorithms: (a) model training on synthetic datasets (Algorithm 2) and (b) dataset distillation (Algorithm 1). The complexity of location clustering is analyzed in Appendix B [31]. Key notations commonly used in the analysis are reviewed as follows. We use N_T and N_S to denote the numbers of locations in the original and synthetic datasets, respectively. The numbers of time series in the original and synthetic datasets are denoted by M_T and M_S, respectively. The input window size is denoted by LinL_in, the target window size by LoutL_out, the number of input features by F, and the number of location subsets by K. We assume that MTGNN [56] equipped with our location encoder is used for both distillation (as the surrogate model fθf_θ) and training; and for simplicity, we omit the hyperparameters that are treated as constants. Model training on synthetic datasets: We first analyze the complexity of model training using synthetic datasets generated by STemDist. Detailed proofs and analyses are provided in Appendix B [31]. Theorems 1 and 2 present the time and space complexity of model training on the synthetic dataset, where TmodelT_model denotes the number of training epochs in Algorithm 2. Theorem 1 (Time Complexity of Model Training). Algorithm 2 takes O(TmodelM(N2K+NLin))O(T_modelM_S( N_S^2K+N_SL_in)) time. Theorem 2 (Space Complexity of Model Training). Algorithm 2 requires O(NF(Lin+Lout)M+N2K2)O(N_SF(L_in+L_out)M_S+ N_S^2K^2) space. It is noteworthy that both the time and space complexities grow quadratically with N_S, the number of locations in the synthetic dataset, which demonstrates the importance of our key idea to reduce N_S (S1 in Section IV-A). Dataset distillation by STemDist: We analyze the overall data-distillation cost of STemDist in Theorem 3. Recall that, in Algorithm 1, TouterT_outer denotes the number of outer iterations, TdistillT_distill denotes the number of iterations for distillation, and TmodelT_model denotes the number of epochs for training the surrogate model. Batch size for the clustered dataset is denoted by B. Detailed proofs and analyses are provided in Appendix B [31]. Theorem 3 (Time Complexity of STemDist). The time complexity of Algorithm 1, the whole distillation process, is O(NN(MLinF)Tkmeans+TouterTdistill((TmodelM+B)(N2K)+(Lin(TmodelM+B)+(M+B)LoutFout)N))O(N_TN_S(M_TL_inF)T_kmeans+T_outerT_distill((T_modelM_S+B)( N_S^2K)+(L_in(T_modelM_S+B)+(M_S+B)L_outF_out)N_S)). As for space complexity, the location clustering step requires O((N+N)MLinF)O((N_T+N_S)M_TL_inF) space, and the remaining distillation steps require O(N2K2+NF(Lin+Lout)(M+M)O( N_S^2K^2+N_SF(L_in+L_out)(M_S+M_T)). Both the time and space complexities of STemDist show the importance of spatial dimension reduction (S1 in Section IV-A), which decreases N_S, and subset-based distillation (S3), which increases K. Our location clustering strategy (S2) allows the complexities to grow only linearly with N_T. V Experiments TABLE I: Dataset statistics. Data Dimensions # Total M N F Data Points Real-world Datasets GBA [14] 1,9771,977 2,3522,352 11 4,649,904 GLA [14] 1,9771,977 3,8343,834 11 7,579,818 ERA5 [23] 2,1372,137 6,5616,561 66 84,125,142 CAMS [24] 2,5562,556 7,0707,070 66 108,425,520 CA [14] 1,9771,977 8,6008,600 11 17,002,200 Generated Artificial Datasets Minimum 262,144262,144 2,3522,352 11 616,562,688 Maximum 8,388,6088,388,608 262,144262,144 8,1928,192 5,050,881,540,096 We perform experiments to answer the following questions: Q1. Distillation Performance: How effective and efficient is training with the synthetic dataset from STemDist, compared to competing methods? Q2. Cross-model Performance: How effective are synthetic datasets from STemDist for training various models? Q3. Scalability and Speed: How does STemDist scale w.r.t. the number of time series, locations, and features in the dataset? How fast is it, compared to competing methods? Q4. Ablation Study: How does each component of STemDist affect the distillation time and performance? Q5. Hyperparameter Analysis: How does STemDist perform under varying hyperparameter settings? V-A Experimental Setup Machines: All experiments were conducted on a workstation with four RTX A6000 GPUs and 512GB of RAM. Datasets: We use five real-world datasets, and their statistics are summarized in Table I. The GBA, GLA, and CA datasets are traffic measurements in several locations. We use their versions preprocessed and released in [14], where traffic volume is used as both the input and target variables. In addition, we use the ERA5 and CAMS datasets provided in [5]. ERA5 [23] provides global weather reanalysis data from ECMWF, while CAMS [24] offers atmospheric analysis and forecasts from the Copernicus Atmospheric Monitoring Service. For both datasets, the six most popular variables are used as inputs; the targets are 10U (10 m eastward wind) for ERA5 and CO (carbon monoxide concentration) for CAMS. We also generate artificial datasets of varying sizes for scalability evaluation. We split each dataset into a training set (60%), a validation set (20%), and a test set (20%) in chronological order. The length of the input and target sequences is fixed to 12. Evaluation metrics: To evaluate the performance of models trained on synthetic datasets, we use Relative Root Mean Squared Error (Relative RMSE) and Relative Mean Absolute Error (Relative MAE). They are defined as follows: RelativeRMSE=∑i=1N(yi−y^i)2∑i=1N(yi−y¯)2,RelativeMAE=∑i=1N|yi−y^i|∑i=1N|yi−y¯|, = _i=1^N(y_i- y_i)^2 _i=1^N(y_i- y)^2,$ subarraycRelative\\ MAE subarray$= _i=1^N y_i- y_i _i=1^N y_i- y , (6) where yiy_i is the ground truth value, y^i y_i is the predicted value, and y¯ y is the mean of the ground truth values. Relative metrics are used to normalize differences in scale across datasets, and for these metrics, smaller values indicate better accuracy. Baselines: We employ nine baselines, categorized as follows: • Coreset selection methods: Random sampling randomly draws samples (i.e., input-target time series pairs) from the original spatio-temporal time series dataset T. K-Center [17] selects representative samples from each cluster from K-means clustering. Herding [50] greedily selects samples to best approximate the mean of the original dataset. CRAIG [42] selects a subset of training data with weights to closely estimate the full gradient. • Distillation methods for general datasets: DC [64], DM [63], MTT [6], DATM [21], IDM [65], and Frepo [69] employ gradient matching, distribution matching, trajectory matching, and meta-learning, respectively, which are detailed in Section I. • Distillation methods for time series datasets: CondTSF [13] and TimeDC [40], which are described in Section I, are specialized dataset-distillation methods for univariate and multivariate time series datasets, respectively. Distillation details: For all baselines, we use or adapt the implementations provided by the authors. For STemDist and its variants, we use MTGNN [56] augmented with our proposed location encoder as the surrogate model fθf_θ. For DC, DM, MTT, and Frepo, we also use MTGNN but without the location encoder, as it is unnecessary when the number of locations is not reduced. For CondTSF and TimeDC, we adopt the same surrogate models as specified in their papers. We use the target feature as the sole input feature in TimeDC and CondTSF, as they support only a single input variable. For all methods, we set the number of outer iterations (TouterT_outer) to 200 for the GBA/GLA datasets and 100 for the CA/ERA5/CAMS datasets. We use 20 distillation iterations (TdistillT_distill), 10 model training epochs (TmodelT_model), and maximum of 300 iterations (TkmeansT_kmeans) for K-means clustering in STemDist. For distillation methods which reduce the number of locations, when the compression ratio is 0.5%, we set the number of time series (M_S) and locations (N_S) to 10% and 5% of the original dataset, respectively, and to 10% and 10% when the compression ratio is 1% (see Section V-F for a related anaylsis). For the baselines in Section V-B, to match the compression ratios, we compress only the number of time series (M_S) as they do not inherently support compression of the location count (N_S). We set the number of location subsets (K) to 44 and the embedding dimension for the location encoder model to 3232. In Sections V-B and V-C each dataset is distilled five times with different seeds per method, and each model is trained with five random initial parameters per distilled data, yielding 25 experiments in total. The learning rates used to train the surrogate model (η) and to update the synthetic data are tuned via grid search over 10−2,10−3,10−4\10^-2,10^-3,10^-4\. Refer to [31] for more details and hyperparameter settings of the baselines. Inference details: For inference after training, each batch consists of 8 time series. When the number of locations exceeds 2152^15, we further split the data into batches of 2152^15 locations to avoid out-of-memory errors in STGNNs. TABLE I: Effectiveness of dataset-distillation methods. The synthetic dataset from each method is used to train MTGNN, equipped with the location encoders when necessary, for evaluation. The performances of our proposed method are highlighted in bold; the best performances in each setting are highlighted in yellow. O.O.M.: out of memory. GBA GLA ERA5 CA CAMS Method Relative Relative Relative Relative Relative Relative Relative Relative Relative Relative MAE RMSE MAE RMSE MAE RMSE MAE RMSE MAE RMSE Random 0.380 ± 0.095 0.453 ± 0.086 0.313 ± 0.050 0.410 ± 0.051 0.482 ± 0.036 0.520 ± 0.028 0.323 ± 0.030 0.431 ± 0.065 0.660 ± 0.104 0.796 ± 0.109 K-Center [17] 0.319 ± 0.001 0.400 ± 0.042 0.269 ± 0.002 0.356 ± 0.028 0.461 ± 0.011 0.524 ± 0.012 0.289 ± 0.051 0.355 ± 0.034 0.883 ± 0.836 0.883 ± 0.073 Herding [50] 0.467 ± 0.008 0.506 ± 0.004 0.331 ± 0.013 0.428 ± 0.004 0.471 ± 0.000 0.522 ± 0.001 0.472 ± 0.000 0.521 ± 0.004 0.584 ± 0.000 0.730 ± 0.002 CRAIG [42] 0.291 ± 0.007 0.354 ± 0.007 0.258 ± 0.015 0.323 ± 0.017 0.430 ± 0.004 0.470 ± 0.003 0.287 ± 0.020 0.354 ± 0.021 0.563 ± 0.006 0.685 ± 0.008 DC [64] 0.281 ± 0.009 0.348 ± 0.006 0.243 ± 0.009 0.311 ± 0.009 0.392 ± 0.004 0.442 ± 0.004 0.254 ± 0.011 0.331 ± 0.014 0.630 ± 0.005 0.735 ± 0.006 DM [63] 0.338 ± 0.021 0.409 ± 0.051 0.298 ± 0.023 0.380 ± 0.025 0.420 ± 0.007 0.467 ± 0.008 0.321 ± 0.027 0.405 ± 0.028 0.528 ± 0.020 0.674 ± 0.011 MTT [6] 0.339 ± 0.051 0.414 ± 0.055 0.343 ± 0.058 0.439 ± 0.066 O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M DATM [21] 0.430 ± 0.012 0.517 ± 0.007 0.332 ± 0.016 0.402 ± 0.022 O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M IDM [65] 0.358 ± 0.062 0.420 ± 0.050 0.300 ± 0.017 0.381 ± 0.024 0.438 ± 0.018 0.481 ± 0.018 0.315 ± 0.037 0.394 ± 0.047 0.567 ± 0.054 0.709 ± 0.049 Frepo [69] 0.397 ± 0.008 0.465 ± 0.075 0.350 ± 0.087 0.439 ± 0.084 0.482 ± 0.036 0.510 ± 0.031 0.361 ± 0.085 0.436 ± 0.084 0.688 ± 0.116 0.802 ± 0.118 CondTSF [13] 0.337 ± 0.040 0.407 ± 0.041 0.311 ± 0.027 0.394 ± 0.031 0.449 ± 0.017 0.489 ± 0.017 0.322 ± 0.050 0.403 ± 0.051 0.555 ± 0.053 0.692 ± 0.037 TimeDC [40] 0.350 ± 0.031 0.422 ± 0.030 0.344 ± 0.061 0.445 ± 0.068 0.437 ± 0.015 0.480 ± 0.015 0.360 ± 0.073 0.448 ± 0.074 0.585 ± 0.105 0.729 ± 0.078 STemDist-K-Center 0.305 ± 0.014 0.367 ± 0.006 0.248 ± 0.007 0.313 ± 0.008 0.396 ± 0.003 0.440 ± 0.005 0.291 ± 0.012 0.348 ± 0.009 0.903 ± 0.093 0.884 ± 0.077 STemDist-DC 0.263 ± 0.007 0.325 ± 0.006 0.227 ± 0.002 0.292 ± 0.006 0.392 ± 0.002 0.431 ± 0.002 0.247 ± 0.006 0.314 ± 0.005 0.545 ± 0.007 0.668 ± 0.008 STemDist-MTT 0.263 ± 0.007 0.325 ± 0.004 0.227 ± 0.004 0.300 ± 0.004 0.399 ± 0.009 0.437 ± 0.007 0.250 ± 0.004 0.311 ± 0.006 0.570 ± 0.010 0.685 ± 0.010 STemDist-CondTSF 0.260 ± 0.006 0.335 ± 0.010 0.225 ± 0.010 0.298 ± 0.005 0.407 ± 0.002 0.452 ± 0.002 0.247 ± 0.008 0.319 ± 0.013 0.514 ± 0.003 0.648 ± 0.002 STemDist STemDist (1 feat.) 0.251 ± 0.005 0.317 ± 0.005 0.211 ± 0.008 0.277 ± 0.002 0.385 ± 0.007 0.400 ± 0.003 0.426 ± 0.004 0.447 ± 0.003 0.226 ± 0.001 0.292 ± 0.002 0.522 ± 0.005 0.520 ± 0.007 0.653 ± 0.005 0.649 ± 0.007 Original data 0.207 ± 0.008 0.264 ± 0.010 0.182 ± 0.001 0.250 ± 0.005 0.368 ± 0.004 0.424 ± 0.010 0.120 ± 0.002 0.253 ± 0.003 0.500 ± 0.005 0.661 ± 0.004 (a) Compression ratio: 0.5% GBA GLA ERA5 CA CAMS Method Relative Relative Relative Relative Relative Relative Relative Relative Relative Relative MAE RMSE MAE RMSE MAE RMSE MAE RMSE MAE RMSE Random 0.291 ± 0.016 0.364 ± 0.017 0.271 ± 0.023 0.355 ± 0.033 0.453 ± 0.012 0.494 ± 0.009 0.289 ± 0.031 0.374 ± 0.045 0.617 ± 0.039 0.737 ± 0.041 K-Center [17] 0.294 ± 0.001 0.366 ± 0.003 0.240 ± 0.001 0.309 ± 0.002 0.434 ± 0.000 0.480 ± 0.000 0.256 ± 0.001 0.326 ± 0.002 0.682 ± 0.010 0.841 ± 0.009 Herding [50] 0.312 ± 0.001 0.387 ± 0.001 0.273 ± 0.000 0.342 ± 0.001 0.441 ± 0.000 0.478 ± 0.001 0.243 ± 0.001 0.313 ± 0.001 0.567 ± 0.000 0.696 ± 0.000 CRAIG [42] 0.272 ± 0.007 0.339 ± 0.006 0.243 ± 0.005 0.311 ± 0.005 0.417 ± 0.011 0.457 ± 0.009 0.261 ± 0.005 0.323 ± 0.005 0.551 ± 0.010 0.679 ± 0.012 DC [64] 0.272 ± 0.005 0.340 ± 0.006 0.236 ± 0.003 0.304 ± 0.001 0.393 ± 0.002 0.434 ± 0.002 0.253 ± 0.003 0.318 ± 0.002 0.520 ± 0.009 0.652 ± 0.008 DM [63] 0.296 ± 0.013 0.361 ± 0.010 0.279 ± 0.028 0.356 ± 0.032 0.417 ± 0.017 0.461 ± 0.016 0.285 ± 0.022 0.361 ± 0.031 0.538 ± 0.018 0.678 ± 0.021 MTT [6] 0.289 ± 0.015 0.357 ± 0.013 O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M DATM [21] 0.376 ± 0.014 0.463 ± 0.014 0.267 ± 0.008 0.336 ± 0.003 O.O.M O.O.M O.O.M O.O.M O.O.M O.O.M IDM [65] 0.290 ± 0.018 0.356 ± 0.018 0.280 ± 0.009 0.358 ± 0.008 0.411 ± 0.013 0.457 ± 0.012 0.283 ± 0.021 0.349 ± 0.019 0.527 ± 0.008 0.666 ± 0.008 Frepo [69] 0.283 ± 0.002 0.356 ± 0.006 0.256 ± 0.012 0.330 ± 0.019 0.419 ± 0.012 0.460 ± 0.013 0.259 ± 0.011 0.329 ± 0.013 0.653 ± 0.007 0.748 ± 0.002 CondTSF [13] 0.294 ± 0.014 0.364 ± 0.012 0.274 ± 0.024 0.356 ± 0.033 0.413 ± 0.005 0.461 ± 0.005 0.279 ± 0.018 0.359 ± 0.029 0.534 ± 0.032 0.680 ± 0.028 TimeDC [40] 0.295 ± 0.017 0.367 ± 0.016 0.274 ± 0.028 0.358 ± 0.040 0.426 ± 0.013 0.472 ± 0.014 0.295 ± 0.036 0.377 ± 0.052 0.560 ± 0.054 0.689 ± 0.021 STemDist-K-Center 0.280 ± 0.003 0.340 ± 0.006 0.243 ± 0.007 0.302 ± 0.009 0.395 ± 0.004 0.436 ± 0.005 0.274 ± 0.008 0.332 ± 0.011 0.872 ± 0.106 0.866 ± 0.066 STemDist-DC 0.258 ± 0.010 0.328 ± 0.009 0.223 ± 0.004 0.295 ± 0.007 0.392 ± 0.003 0.432 ± 0.003 0.240 ± 0.004 0.305 ± 0.003 0.537 ± 0.008 0.663 ± 0.002 STemDist-MTT 0.261 ± 0.005 0.321 ± 0.004 0.224 ± 0.007 0.293 ± 0.003 0.403 ± 0.003 0.444 ± 0.004 0.248 ± 0.012 0.317 ± 0.014 0.550 ± 0.016 0.681 ± 0.005 STemDist-CondTSF 0.261 ± 0.010 0.321 ± 0.003 0.220 ± 0.005 0.289 ± 0.004 0.405 ± 0.002 0.449 ± 0.002 0.243 ± 0.006 0.313 ± 0.004 0.513 ± 0.007 0.645 ± 0.002 STemDist STemDist (1 feat.) 0.242 ± 0.001 0.307 ± 0.002 0.205 ± 0.001 0.272 ± 0.001 0.384 ± 0.005 0.399 ± 0.003 0.426 ± 0.004 0.445 ± 0.004 0.222 ± 0.002 0.287 ± 0.002 0.518 ± 0.008 0.501 ± 0.006 0.652 ± 0.006 0.644 ± 0.003 Original data 0.207 ± 0.008 0.264 ± 0.010 0.182 ± 0.001 0.250 ± 0.005 0.368 ± 0.004 0.424 ± 0.010 0.120 ± 0.002 0.253 ± 0.003 0.500 ± 0.005 0.661 ± 0.004 (b) Compression ratio: 1% Figure 4: Training on synthetic datasets distilled by STemDist is faster and more memory-efficient than training on those distilled by the baselines. TABLE IV: Cross-model generalization of dataset-distillation methods. The synthetic dataset produced by each method is used to train various machine learning models for evaluation. The performances of our methods are highlighted in bold; and the best performances in each setting are highlighted in yellow. Graph wavenet [58] GBA GLA ERA5 CA CAMS Method Relative Relative Relative Relative Relative Relative Relative Relative Relative Relative MAE RMSE MAE RMSE MAE RMSE MAE RMSE MAE RMSE K-Center [17] 0.336 ± 0.027 0.427 ± 0.032 0.279 ± 0.021 0.367 ± 0.018 0.466 ± 0.017 0.510 ± 0.012 0.318 ± 0.032 0.396 ± 0.030 0.763 ± 0.055 0.796 ± 0.028 DC [64] 0.321 ± 0.036 0.395 ± 0.033 0.288 ± 0.031 0.312 ± 0.009 0.424 ± 0.007 0.470 ± 0.004 0.292 ± 0.043 0.369 ± 0.038 0.550 ± 0.006 0.688 ± 0.004 CondTSF [13] 0.346 ± 0.065 0.416 ± 0.066 0.302 ± 0.030 0.387 ± 0.028 0.464 ± 0.008 0.514 ± 0.007 0.322 ± 0.038 0.413 ± 0.031 0.531 ± 0.026 0.695 ± 0.022 STemDist 0.241 ± 0.007 0.310 ± 0.007 0.212 ± 0.006 0.273 ± 0.003 0.385 ± 0.004 0.431 ± 0.003 0.222 ± 0.004 0.293 ± 0.006 0.528 ± 0.006 0.665 ± 0.007 Original data 0.181 ± 0.005 0.227 ± 0.009 0.166 ± 0.007 0.212 ± 0.011 0.376 ± 0.007 0.419 ± 0.005 0.169 ± 0.009 0.215 ± 0.007 0.488 ± 0.014 0.631 ± 0.007 STGCN [60] GBA GLA ERA5 CA CAMS Method Relative Relative Relative Relative Relative Relative Relative Relative Relative Relative MAE RMSE MAE RMSE MAE RMSE MAE RMSE MAE RMSE K-Center[17] 0.606 ± 0.091 0.714 ± 0.085 0.643 ± 0.094 0.746 ± 0.062 0.791 ± 0.081 0.800 ± 0.046 0.625 ± 0.046 0.731 ± 0.073 1.155 ± 0.045 1.075 ± 0.019 DC [64] 0.767 ± 0.077 0.812 ± 0.053 0.767 ± 0.078 0.866 ± 0.031 0.733 ± 0.044 0.753 ± 0.060 0.655 ± 0.088 0.736 ± 0.068 0.936 ± 0.035 0.986 ± 0.018 CondTSF [13] 0.628 ± 0.065 0.697 ± 0.038 0.622 ± 0.070 0.734 ± 0.087 0.762 ± 0.065 0.796 ± 0.052 0.616 ± 0.095 0.730 ± 0.094 0.896 ± 0.069 0.950 ± 0.037 STemDist 0.441 ± 0.038 0.501 ± 0.041 0.392 ± 0.032 0.459 ± 0.034 0.632 ± 0.032 0.667 ± 0.037 0.443 ± 0.050 0.504 ± 0.045 0.748 ± 0.022 0.844 ± 0.011 Original data 0.247 ± 0.008 0.294 ± 0.008 0.240 ± 0.005 0.295 ± 0.003 0.511 ± 0.006 0.546 ± 0.012 0.371 ± 0.348 0.268 ± 0.006 0.667 ± 0.020 0.743 ± 0.010 FourierGNN [59] GBA GLA ERA5 CA CAMS Method Relative Relative Relative Relative Relative Relative Relative Relative Relative Relative MAE RMSE MAE RMSE MAE RMSE MAE RMSE MAE RMSE K-Center[17] 0.353 ± 0.032 0.419 ± 0.027 0.310 ± 0.027 0.395 ± 0.025 0.432 ± 0.005 0.480 ± 0.005 0.360 ± 0.020 0.437 ± 0.021 0.651 ± 0.012 0.795 ± 0.021 DC [64] 0.335 ± 0.015 0.403 ± 0.015 0.301 ± 0.020 0.376 ± 0.021 0.426 ± 0.005 0.469 ± 0.003 0.337 ± 0.010 0.409 ± 0.011 0.561 ± 0.024 0.703 ± 0.028 CondTSF [13] 0.348 ± 0.021 0.419 ± 0.036 0.313 ± 0.018 0.421 ± 0.009 0.469 ± 0.022 0.511 ± 0.019 0.351 ± 0.028 0.441 ± 0.023 0.557 ± 0.059 0.715 ± 0.070 STemDist 0.309 ± 0.003 0.384 ± 0.002 0.263 ± 0.003 0.339 ± 0.004 0.412 ± 0.001 0.459 ± 0.001 0.293 ± 0.007 0.371 ± 0.006 0.534 ± 0.008 0.658 ± 0.001 Original data 0.248 ± 0.006 0.307 ± 0.012 0.216 ± 0.009 0.284 ± 0.006 0.396 ± 0.002 0.444 ± 0.002 0.219 ± 0.009 0.288 ± 0.006 0.496 ± 0.003 0.642 ± 0.002 V-B Distillation Performance We examine how effective and efficient training with the synthetic datasets from STemDist and its competitors are. To evaluate the synthetic datasets, we train MTGNN [56] and incorporate our location encoder when required. Regarding effectiveness, STemDist consistently outperforms all competitors across compression ratios, as shown in Table I, where we report the mean and standard deviation of the forecasting errors. Specifically, when the compression ratio is 0.5%, the model trained on the synthetic data distilled by STemDist achieves 12% lower Relative RMSE in the CA dataset compared to the best competitor. This demonstrates that STemDist effectively preserves both temporal and spatial information in a balanced manner within the synthetic dataset. Training the model with the synthetic datasets distilled by STemDist is also more efficient than training with those distilled by the baselines. As shown in Figure 4, at a compression ratio of 0.5%, training on the dataset distilled by STemDist is up to 6.3× faster with up to 7.7× less memory.333For clarity, this experiment differs from Preliminary Experiment 1 in Section I, as it employs different models (MTGNN with our location encoder) and training strategies (subset-based training in Algorithm 2). In contrast, Preliminary Experiment 1 follows those in [56]. These results are consistent with Theorems 1 and 2 in Section IV-F. In addition, we evaluate variants of STemDist that integrate our location encoder and spatial reduction into four baseline distillation methods, denoted as STemDist-[baseline name] in Table I. These variants outperform their corresponding baselines, showing the effectiveness of our ideas; however, the variants still underperform STemDist on most datasets. We further examine when the input is limited to a single feature, spec., the target feature for each dataset. For the GBA, GLA, and CA datasets, this corresponds to their original setup, as they contain only one feature. As shown in Table I, even under this constraint, STemDist outperforms most baselines. V-C Cross-model Performance We evaluate the cross-model generalization ability of the synthetic datasets, by training widely-used spatio-temporal time series forecasting models: Graph wavenet [58], STGCN [60], and FourierGNN [59], which are equipped with our location encoder when necessary.444We modify STGCN to learn a graph with Eq. (2) without relying on a predefined adjacency matrix. Note that some models (e.g., [59, 12]) do not require a location encoder when trained on synthetic datasets. K-Center [17], DC [64], and CondTSF [13], each of which is the best in its respective baseline category, are used as baselines; and the compression ratio is set to 0.5%. As shown in Table IV, STemDist leads to the best performance across all tested models and datasets, demonstrating strong cross-model generalizability. This indicates that the synthetic datasets from STemDist capture key information that generalizes beyond the surrogate model. Results for additional models (spec., AGCRN [4] and STNORM[12]) are in Appendix E [31]. Figure 5: Scalability of STemDist. Its empirical distillation time grows (sub-)linearly with (a) the number of original time series, (b) the location count in the original dataset, and (c) the feature count in the original dataset. V-D Scalability and Speed We evaluate the scalability of STemDist w.r.t. various input/output factors, using artificially generated large-scale datasets. We progressively double each factor while keeping the others fixed to their default values555262,144262,144 original time series, 6464 synthetic time series, 2,3522,352 locations, and 11 feature., as summarized in Table I, and measure the per-epoch distillation time required to make a pass over the dataset. While we control the number of locations in the original dataset, the synthetic dataset size is set accordingly as N=0.05×N_S=0.05× N_T. Each experiment is conducted three times, and the mean is reported. As shown in Figure 5, the distillation time of STemDist scales linearly (slope of 1 on log–log scale) with the numbers of original time series and locations, and sublinearly (slope less than 1 on log–log scale) with the numbers of features and the number of synthetic time series (see Appendix F [31] for results with respect to the number of synthetic time series). The results w.r.t. the number of original time series are consistent with Theorem 3 in Section IV-F. For other factors, STemDist shows better empirical scalability than the theoretical upper bounds in Theorem 3, due to the benefits of GPU parallelization within each batch. Note that parallelization does not take effect when the number of original time series grows, as it increases the number of batches rather than batch size. We also compare the distillation time of STemDist with the baselines, at a compression ratio of 0.5%, in Figure 6. The trajectory matching-based approaches (MTT [6], DATM [21], CondTSF [13], and TimeDC [40]) generally require less time for distillation. However, they require additional time to generate expert trajectories prior to the distillation process. Compared to DM [63] and IDM [65], STemDist shows clear time efficiency on datasets with a larger number of locations. Compared to DC [64], STemDist consistently shows superior time efficiency across all datasets. The location encoder incurs negligible inference latency, at most 0.0002 seconds for the real-world datasets (Table I) and 0.002 seconds for the artificially generated datasets, accounting for only 0.06% and 0.004% of the total inference time. Figure 6: Comparison of distillation time per outer iteration across different dataset distillation methods. V-E Ablation Study To confirm the effectiveness of each component of STemDist, we compare STemDist with three variants: • STemDist w/o G: STemDist without subset-based Granular distillation. • STemDist w/o G & C: STemDist without subset-based Granular distillation and Clustering of locations. • DC [64]: A naive gradient matching method. At a compression ratio of 0.5%, we perform the distillation three times for each method and evaluate each synthetic dataset three times. We report the mean. Figure 7 shows how the performance of each variant changes over distillation time for GLA, ERA5, and CAMS datasets (refer to Appendix H [31] for results on the other datasets). Notably, each component of STemDist helps achieve effective distillation (i.e., lower forecasting error) more rapidly. (a) GLA (b) ERA5 (c) CAMS Figure 7: Ablation study. Distillation performance of STemDist and its variants over distillation time on GLA, ERA5, CAMS datasets We evaluate variants of STemDist that replace K-means with spatially constrained clustering (RedCap [20]) or graph coarsening (METIS [29]) for location clustering. For ERA5 and CAMS, we construct grid-based spatial graphs from longitude and latitude coordinates, while traffic datasets are excluded due to missing spatial information. Compared to using K-means (with Relative RMSE of 0.426 on ERA5, 0.653 on CAMS), using METIS (0.421 on ERA5, 0.656 on CAMS) or RedCap (0.429 on ERA5, 0.672 on CAMS) results in comparable distillation performance without clear improvements. V-F Hyperparameter Analysis We examine the effect of hyperparameters on the distillation performance of STemDist under a fixed total compression ratio of 1% on the GBA, GLA, and ERA5 datasets. Except for the controlled hyperparameters, all others are fixed to the default values described in Section V-A. We evaluate four temporal–spatial ratio combinations and observe that STemDist tends to be more effective when the temporal and spatial ratios are balanced (Table V(a)). We further examine the impact of the number of subsets K in subset-based granular distillation. As shown in Table V(b), STemDist performs similarly well when the number of subsets is four or greater. Recall that, in Section V-E, we observe a substantial performance drop when the number of subsets is one (STemDist w/o G in Figure 7). Hyperparameter analysis results under a total compression ratio of 0.5% are provided in Appendix I [31]. TABLE V: Effect of hyperparameters on the distillation performance of STemDist, measured by Relative RMSE under a total compression ratio of 1%. Temporal–Spatial Compression Pairs (M,N)( M_SM_T, N_SN_T) Dataset (2%, 50%) (5%, 20%) (10%, 10%) (25%, 4%) GBA 0.319 ± 0.008 0.307 ± 0.004 0.307 ± 0.002 0.324 ± 0.007 GLA 0.291 ± 0.004 0.278 ± 0.002 0.272 ± 0.001 0.279 ± 0.003 ERA5 0.426 ± 0.004 0.424 ± 0.003 0.426 ± 0.004 0.424 ± 0.003 CA 0.302 ± 0.009 0.292 ± 0.002 0.287 ± 0.002 0.293 ± 0.003 CAMS 0.656 ± 0.007 0.652 ± 0.003 0.652 ± 0.006 0.653 ± 0.003 Avg. 0.399 ± 0.006 0.391 ± 0.003 0.389 ± 0.003 0.395 ± 0.004 (a) Effect of temporal and spatial compression ratios Number of Subsets K Dataset 2 4 8 16 GBA 0.352 ± 0.007 0.307 ± 0.002 0.313 ± 0.009 0.310 ± 0.011 GLA 0.274 ± 0.006 0.272 ± 0.001 0.275 ± 0.004 0.276 ± 0.004 ERA5 0.425 ± 0.005 0.426 ± 0.004 0.434 ± 0.005 0.429 ± 0.003 CA 0.297 ± 0.007 0.287 ± 0.002 0.281 ± 0.002 0.280 ± 0.006 CAMS 0.664 ± 0.003 0.652 ± 0.006 0.652 ± 0.003 0.654 ± 0.003 Avg. 0.402 ± 0.006 0.389 ± 0.003 0.391 ± 0.006 0.390 ± 0.005 (b) Effect of the number of subsets VI Conclusions and Future Directions In this work, we propose STemDist, a novel dataset distillation method specialized for spatio-temporal time series. The main idea of STemDist is bi-dimensional compression, which simultaneously reduces both the temporal and spatial dimensions in the synthetic dataset, enabling more effective and efficient model training. This is done by three core components (location encoders, location clustering, and subset-based granular distillation), which enhance both effectiveness and efficiency of the distillation process. In our experiments, the synthetic data generated by STemDist makes model training faster (up to 6×), more memory-efficient (up to 8×), and more effective (with up to 12% lower prediction error). A potential future direction is to extend STemDist in a cost-sensitive manner to prioritize performance preservation for high-stakes or rare events (e.g., extreme weather events in meteorological data). Acknowledgements: This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2024-00457882, AI Research Hub Project, 40%) (No. RS-2022-I220157, Robust, Fair, Extensible Data-Centric Continual Learning, 40%) (No. RS-2024-00438638, EntireDB2AI: Foundations and Software for Comprehensive Deep Representation Learning and Prediction on Entire Relational Databases, 10%) (RS-2019-I190075, Artificial Intelligence Graduate School Program (KAIST), 10%). AI-Generated Content Acknowledgement Portions of the manuscript were revised with assistance from ChatGPT (GPT-5, OpenAI) for language refinement. The authors reviewed and edited all AI-generated content and take full responsibility for the final version of the manuscript. References [1] R. A. Angryk, P. C. Martens, B. Aydin, D. Kempton, S. S. Mahajan, S. Basodi, A. Ahmadzadeh, X. Cai, S. Filali Boubrahimi, S. M. Hamdi, et al. (2020) Multivariate time series dataset for space weather data analytics. Scientific data. Cited by: §I. [2] G. Atluri, A. Karpatne, and V. Kumar (2018) Spatio-temporal data mining: a survey of problems and methods. ACM Computing Surveys 51 (4), p. 1–41. Cited by: §I. [3] J. L. Ba, J. R. Kiros, and G. E. Hinton (2016) Layer normalization. arXiv. Cited by: §IV-C. [4] L. Bai, L. Yao, C. Li, X. Wang, and C. Wang (2020) Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS, Cited by: §I, §I, §I-A, §IV-C, §V-C. [5] C. Bodnar, W. P. Bruinsma, A. Lucic, M. Stanley, A. Allen, J. Brandstetter, P. Garvan, M. Riechert, J. A. Weyn, H. Dong, et al. (2025) A foundation model for the earth system. Nature. Cited by: §V-A. [6] G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J. Zhu (2022) Dataset distillation by matching training trajectories. In CVPR, Cited by: §I, §I, 2nd item, §V-D, IIIa, IIIb. [7] C. Chai, J. Liu, N. Tang, J. Fan, D. Miao, J. Wang, Y. Luo, and G. Li (2023) Goodcore: data-effective and data-efficient machine learning through coreset selection over incomplete data. SIGMOD. Cited by: §I. [8] Y. Cheng, P. Chen, C. Guo, K. Zhao, Q. Wen, B. Yang, and C. S. Jensen (2023) Weakly guided adaptation for robust time series forecasting. PVLDB 17 (4), p. 766–779. Cited by: §I. [9] J. Choi, H. Choi, J. Hwang, and N. Park (2022) Graph neural controlled differential equations for traffic forecasting. In AAAI, Cited by: §I. [10] R. Cirstea, T. Kieu, C. Guo, B. Yang, and S. J. Pan (2021) EnhanceNet: plugin neural networks for enhancing correlated time series forecasting. In ICDE, Cited by: §I. [11] R. Cirstea, B. Yang, C. Guo, T. Kieu, and S. Pan (2022) Towards spatio-temporal aware traffic time series forecasting. In ICDE, Cited by: §I. [12] J. Deng, X. Chen, R. Jiang, X. Song, and I. W. Tsang (2021) St-norm: spatial and temporal normalization for multi-variate time series forecasting. In KDD, Cited by: §I, §V-C, footnote 4. [13] J. Ding, Z. Liu, G. Zheng, H. Jin, and L. Kong (2024) CondTSF: one-line plugin of dataset condensation for time series forecasting. NeurIPS. Cited by: §I, §I, 3rd item, §V-C, §V-D, IIIa, IIIb, TABLE IV, TABLE IV, TABLE IV. [14] Y. Fang, Y. Liang, B. Hui, Z. Shao, L. Deng, X. Liu, X. Jiang, and K. Zheng (2025) Efficient large-scale traffic forecasting with transformers: a spatial data management perspective. In KDD, Cited by: §I, §V-A, TABLE I, TABLE I, TABLE I. [15] Z. Fang, Q. Long, G. Song, and K. Xie (2021) Spatial-temporal graph ode networks for traffic flow forecasting. In KDD, Cited by: §I. [16] Z. Fang, L. Pan, L. Chen, Y. Du, and Y. Gao (2021) MDTP: a multi-source deep traffic prediction framework over spatio-temporal trajectory data. PVLDB 14 (8), p. 1289–1297. Cited by: §I. [17] R. Z. Farahani and M. Hekmatfar (2009) Facility location: concepts, models, algorithms and case studies. Springer Science & Business Media. Cited by: §I, 1st item, §V-C, IIIa, IIIb, TABLE IV, TABLE IV, TABLE IV. [18] J. Gao, J. Wu, and J. Ding (2024) Heterogeneous graph condensation. IEEE Transactions on Knowledge and Data Engineering 36 (7), p. 3126–3138. Cited by: §I. [19] X. Gao, T. Chen, Y. Zang, W. Zhang, Q. V. H. Nguyen, K. Zheng, and H. Yin (2024) Graph condensation for inductive node representation learning. In ICDE, Cited by: §I, §I. [20] D. Guo (2008) Regionalization with dynamically constrained agglomerative clustering and partitioning (redcap). International Journal of Geographical Information Science 22 (7), p. 801–823. Cited by: §IV-D, §V-E. [21] Z. Guo, K. Wang, G. Cazenavette, H. Li, K. Zhang, and Y. You (2024) Towards lossless dataset distillation via difficulty-aligned trajectory matching. ICLR. Cited by: §I, 2nd item, §V-D, IIIa, IIIb. [22] J. Han, W. Zhang, H. Liu, T. Tao, N. Tan, and H. Xiong (2024) Bigst: linear complexity spatio-temporal graph neural network for traffic forecasting on large-scale road networks. PVLDB 17 (5), p. 1081–1090. Cited by: §I. [23] H. Hersbach, B. Bell, P. Berrisford, G. Biavati, A. Horányi, J. Muñoz Sabater, J. Nicolas, C. Peubey, R. Radu, I. Rozum, D. Schepers, A. Simmons, C. Soci, D. Dee, and J.-N. Thépaut (2023) ERA5 hourly data on single levels from 1940 to present. Note: Copernicus Climate Change Service (C3S) Climate Data Store (CDS) External Links: Document Cited by: §V-A, TABLE I. [24] A. Inness, M. Ades, A. Agustí-Panareda, J. Barré, A. Benedictow, A. Blechschmidt, J. J. Dominguez, R. Engelen, H. Eskes, J. Flemming, et al. (2019) The cams reanalysis of atmospheric composition. Atmospheric Chemistry and Physics 19 (6). Cited by: §V-A, TABLE I. [25] Y. Jiang, X. Li, Y. Chen, S. Liu, W. Kong, A. F. Lentzakis, and G. Cong (2024) SAGDFN: a scalable adaptive graph diffusion forecasting network for multivariate time series forecasting. In ICDE, Cited by: §I. [26] G. Jin, Y. Liang, Y. Fang, Z. Shao, J. Huang, J. Zhang, and Y. Zheng (2023) Spatio-temporal graph neural networks for predictive learning in urban computing: a survey. IEEE Transactions on Knowledge and Data Engineering 36 (10), p. 5388–5408. Cited by: §I. [27] W. Jin, X. Tang, H. Jiang, Z. Li, D. Zhang, J. Tang, and B. Yin (2022) Condensing graphs via one-step gradient matching. In KDD, Cited by: §I, §I. [28] W. Jin, L. Zhao, S. Zhang, Y. Liu, J. Tang, and N. Shah (2021) Graph condensation for graph neural networks. ICLR. Cited by: §I, §I. [29] G. Karypis and V. Kumar (1998) Multilevelk-way partitioning scheme for irregular graphs. Journal of Parallel and Distributed computing 48 (1). Cited by: §IV-D, §V-E. [30] D. Kieu, T. Kieu, P. Han, B. Yang, C. S. Jensen, and B. Le (2024) TEAM: topological evolution-aware framework for traffic forecasting. PVLDB 18 (2). Cited by: §I, §I. [31] T. Kwon, Y. Choi, Y. Kim, and K. Shin (2025)Effective dataset distillation for spatio-temporal forecasting with bi-dimensional compression (code, datasets, and online appendix)(Website) External Links: Link Cited by: §I-A, §IV-F, §IV-F, §IV-F, §V-A, §V-C, §V-D, §V-E, §V-F. [32] Z. Lai, D. Zhang, H. Li, C. S. Jensen, H. Lu, and Y. Zhao (2023) Lightcts: a lightweight framework for correlated time series forecasting. SIGMOD. Cited by: §I. [33] S. Lan, Y. Ma, W. Huang, W. Wang, H. Yang, and P. Li (2022) Dstagnn: dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In ICML, Cited by: §I. [34] Y. Li, R. Yu, C. Shahabi, and Y. Liu (2018) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. ICLR. Cited by: §I. [35] Y. Liang, W. Zhang, X. Gao, L. Yang, C. Chen, H. Yin, Y. Tong, and B. Cui (2025) Training-free heterogeneous graph condensation via data selection. In ICDE, Cited by: §I, §I. [36] M. Liu, S. Li, X. Chen, and L. Song (2022) Graph condensation via receptive field distribution matching. arXiv. Cited by: §I. [37] Z. Liu, K. Hao, G. Zheng, and Y. Yu (2024) Dataset condensation for time series classification via dual domain matching. In KDD, Cited by: §I, §I, §I. [38] M. Ma, J. Hu, C. S. Jensen, F. Teng, P. Han, Z. Xu, and T. Li (2024) Learning time-aware graph structures for spatially correlated time series forecasting. In ICDE, Cited by: §I. [39] M. Ma, P. Xie, F. Teng, B. Wang, S. Ji, J. Zhang, and T. Li (2023) HiSTGNN: hierarchical spatio-temporal graph neural network for weather forecasting. Information Sciences 648, p. 119580. Cited by: §I. [40] H. Miao, Z. Liu, Y. Zhao, C. Guo, B. Yang, K. Zheng, and C. S. Jensen (2024) Less is more: efficient time series dataset condensation via two-fold modal matching. PVLDB 18 (2), p. 226–238. Cited by: §I, §I, 3rd item, §V-D, IIIa, IIIb. [41] H. Miao, Y. Zhao, C. Guo, B. Yang, K. Zheng, F. Huang, J. Xie, and C. S. Jensen (2024) A unified replay-based continuous learning framework for spatio-temporal prediction on streaming data. In ICDE, Cited by: §I. [42] B. Mirzasoleiman, J. Bilmes, and J. Leskovec (2020) Coresets for data-efficient training of machine learning models. In ICML, Cited by: §I, 1st item, IIIa, IIIb. [43] N. Sachdeva and J. McAuley (2023) Data distillation: a survey. arXiv. Cited by: §I. [44] Z. Shao, Z. Zhang, F. Wang, W. Wei, and Y. Xu (2022) Spatial-temporal identity: a simple yet effective baseline for multivariate time series forecasting. In CIKM, Cited by: §I. [45] Z. Shao, Z. Zhang, F. Wang, and Y. Xu (2022) Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In KDD, Cited by: §I. [46] Z. Shao, Z. Zhang, W. Wei, F. Wang, Y. Xu, X. Cao, and C. S. Jensen (2022) Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. PVLDB 15 (11), p. 2733–2746. Cited by: §I. [47] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. NeurIPS. Cited by: §IV-C. [48] L. Wang, W. Fan, J. Li, Y. Ma, and Q. Li (2024) Fast graph condensation with structure-based neural tangent kernel. In W, Cited by: §I. [49] T. Wang, J. Zhu, A. Torralba, and A. A. Efros (2018) Dataset distillation. arXiv. Cited by: §I. [50] M. Welling (2009) Herding dynamical weights to learn. In ICML, Cited by: §I, 1st item, IIIa, IIIb. [51] H. Wu, J. Xu, J. Wang, and M. Long (2021) Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. NeurIPS. Cited by: §I. [52] J. Wu, W. Fan, J. Chen, S. Liu, Q. Liu, R. He, Q. Li, and K. Tang (2025) Condensing pre-augmented recommendation data via lightweight policy gradient estimation. IEEE Transactions on Knowledge and Data Engineering 37 (1), p. 162–173. Cited by: §I. [53] X. Wu, X. Wu, B. Yang, L. Zhou, C. Guo, X. Qiu, J. Hu, Z. Sheng, and C. S. Jensen (2024) AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting. The VLDB Journal 33 (5). Cited by: §I. [54] X. Wu, X. Wu, D. Zhang, M. Zhang, C. Guo, B. Yang, and C. S. Jensen (2024) Fully automated correlated time series forecasting in minutes. PVLDB 18 (2), p. 144–157. Cited by: §I. [55] X. Wu, D. Zhang, C. Guo, C. He, B. Yang, and C. S. Jensen (2021) AutoCTS: automated correlated time series forecasting. PVLDB 15 (4), p. 971–983. Cited by: §I. [56] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang (2020) Connecting the dots: multivariate time series forecasting with graph neural networks. In KDD, Cited by: §I, §I-A, §IV-C, §IV-F, §V-A, §V-B, Preliminary Experiment 1, footnote 3. [57] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang (2020) Connecting the dots: multivariate time series forecasting with graph neural networks. In KDD, Cited by: §I. [58] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang (2019) Graph wavenet for deep spatial-temporal graph modeling. In IJCAI, Cited by: §I, §I, §I-A, §IV-C, §V-C, TABLE IV. [59] K. Yi, Q. Zhang, W. Fan, H. He, L. Hu, P. Wang, N. An, L. Cao, and Z. Niu (2023) FourierGNN: rethinking multivariate time series forecasting from a pure graph perspective. In NeurIPS, Cited by: §V-C, TABLE IV, footnote 4. [60] B. Yu, H. Yin, and Z. Zhu (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In IJCAI, Cited by: §I, §I, §I-A, §V-C, TABLE IV. [61] Y. Zhang, T. Zhang, K. Wang, Z. Guo, Y. Liang, X. Bresson, W. Jin, and Y. You (2024) Navigating complexity: toward lossless graph condensation via expanding window matching. In ICML, Cited by: §I. [62] B. Zhao and H. Bilen (2021) Dataset condensation with differentiable siamese augmentation. In ICML, Cited by: §I. [63] B. Zhao and H. Bilen (2023) Dataset condensation with distribution matching. In WACV, Cited by: §I, 2nd item, §V-D, IIIa, IIIb. [64] B. Zhao, K. R. Mopuri, and H. Bilen (2021) Dataset condensation with gradient matching. In ICLR, Cited by: §I, §I, §I-C, 2nd item, 3rd item, §V-C, §V-D, IIIa, IIIb, TABLE IV, TABLE IV, TABLE IV. [65] G. Zhao, G. Li, Y. Qin, and Y. Yu (2023) Improved distribution matching for dataset condensation. In CVPR, Cited by: §I, 2nd item, §V-D, IIIa, IIIb. [66] K. Zhao, C. Guo, Y. Cheng, P. Han, M. Zhang, and B. Yang (2023) Multiple time series forecasting with dynamic graph modeling. PVLDB 17 (4), p. 753–765. Cited by: §I. [67] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang (2021) Informer: beyond efficient transformer for long sequence time-series forecasting. In AAAI, Cited by: §I. [68] Q. Zhou, Y. Zhang, M. A. Makridis, A. Kouvelas, Y. Wang, and S. Hu (2025) MoGERNN: an inductive traffic predictor for unobserved locations. Transportation Research Part C: Emerging Technologies 174, p. 105080. Cited by: §I. [69] Y. Zhou, E. Nezhadarya, and J. Ba (2022) Dataset distillation using neural feature regression. NeurIPS. Cited by: §I, 2nd item, IIIa, IIIb. [70] R. Zorrilla, E. Ogasawara, P. Valduriez, and F. Porto (2024) A data-driven model selection approach to spatio-temporal prediction. In Transactions on Large-Scale Data-and Knowledge-Centered Systems, p. 98–118. Cited by: §IV-D, footnote 2.

Instant research discovery