Paper deep dive
Scale-Invariant Training for Time Series Foundation Models
Ignacy Stepka, Willa Potosnak, Kin G. Olivares, Artur Dubrawski
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/7/2026, 9:07:53 AM
Summary
This paper introduces Scale-Invariant Training (SCALEIN) for Time Series Foundation Models (TSFMs), addressing the issue of Scale-Contaminated Training (SCALECON). SCALECON, used by many existing models (e.g., those using ReVIN), computes loss on inverse-scaled predictions, causing gradients to be weighted by the series scale raised to the power of the loss degree (b^p). This leads to high-scale series dominating training. SCALEIN computes loss on scaled targets, ensuring gradients are invariant to series scale. Experiments show SCALEIN improves forecast accuracy (lower MASE) across various TSFM architectures and benchmarks compared to SCALECON.
Entities (10)
Relation Signals (6)
SCALECON → causes → high-scale series to dominate training
confidence 95% · SCALECON... causes high-scale series to dominate training
SCALEIN → improves → Forecast Accuracy
confidence 95% · SCALEIN... improves forecast accuracy in TSFM pretraining and supervised forecasting
SCALECON → uses → RevIN
confidence 95% · The default pipeline with ReVIN [Kim et al. 2022] first reverses the transform... We call this objective scale-contaminated (SCALECON)
SCALEIN → lowers → MASE
confidence 90% · SCALEIN lowers MASE in all 24 architecture-benchmark comparisons
SCALECON → uses → MSE
confidence 85% · Under a degree-p loss... including MSE... SCALECON... multiplies each series’ gradient by b^p
SCALEIN → uses → Quantile loss
confidence 85% · For any scale-equivariant scaler and residual loss that is homogeneous of degree p, including... Quantile Loss... yielding scale-invariant training (SCALEIN)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Time series foundation models (TSFMs) are trained on large collections of time series datasets that span various morphologies and domains. This setting exposes models to series whose scales -- typical magnitudes of their values -- can differ substantially. Affine scaling methods such as Reversible Instance Normalization (ReVIN) scale model inputs and reverse the transform before computing the loss. We show that this inversion multiplies each series' gradient by $b^p$ relative to loss on scaled targets, where $b$ is the scaling denominator (e.g., standard deviation) and $p$ is the loss degree. We call this scale-contaminated training (ScaleCon), because the scale of each series consequently becomes an importance weight, causing high-scale series to dominate training. For any scale-equivariant scaler and residual loss that is homogeneous of degree $p$, including MSE, MAE, and Quantile Loss, we prove that computing loss on scaled targets makes every mini-batch gradient and, consequently, the full optimization trajectory invariant to arbitrary independent rescaling of the training series, yielding scale-invariant training (ScaleIn). Notably, existing TSFMs use both objectives, with neither consistent reporting nor a common convention on how to compute training loss. We isolate the convergence disparity induced by ScaleCon and its correction under ScaleIn in controlled studies on synthetic and real data. In pretraining across four TSFM architectures, ScaleIn lowers MASE in all 24 architecture-benchmark comparisons, with average reductions across TSFMs of 18.8% on GIFT-Eval and 21.9% on the M-competitions. The gains extend to supervised neural forecasting, where it lowers MASE in 16 of 20 matched settings. Most existing time series forecasting pipelines can adopt ScaleIn with a one-line code change.
Tags
Links
- Source: https://arxiv.org/abs/2610.07324v1
- Canonical: https://arxiv.org/abs/2610.07324v1
Trouble viewing inline? Open PDF directly →
Full Text
116,794 characters extracted from source content.
Expand or collapse full text
Scale-Invariant Training for Time Series Foundation Models Ignacy Stepka 1 , Willa Potosnak 1,2 , Kin G. Olivares 2,3 , Artur Dubrawski 1 1 Carnegie Mellon University 2 Amazon ∗3 Nixtla Abstract Time series foundation models (TSFMs) are trained on large collections of time series datasets that span various morphologies and domains. This setting exposes models to series whose scales – typical magnitudes of their values – can differ substantially. Affine scaling methods such as Reversible Instance Normalization (ReVIN) scale model inputs and reverse the transform before computing the loss. We show that this inversion multiplies each series’ gradient byb p relative to loss on scaled targets, wherebis the scaling denominator (e.g., standard deviation) andpis the loss degree. We call this scale-contaminated training (SCALECON), because the scale of each series consequently becomes an importance weight, causing high-scale series to dominate training. For any scale-equivariant scaler and residual loss that is homogeneous of degreep, including MSE, MAE, and Quantile Loss, we prove that computing loss on scaled targets makes every mini-batch gradient and, consequently, the full optimization trajectory invariant to arbitrary independent rescaling of the training series, yielding scale-invariant training (SCALEIN). Notably, existing TSFMs use both objectives, with neither consistent reporting nor a common convention on how to compute training loss. We isolate the convergence disparity induced by SCALECON and its correction under SCALEIN in controlled studies on synthetic and real data. In pretraining across four TSFM architectures, SCALEIN lowers MASE in all 24 architecture-benchmark comparisons, with average reductions across TSFMs of 18.8% on GIFT-Eval and 21.9% on the M-competitions. The gains extend to supervised neural forecasting, where it lowers MASE in 16 of 20 matched settings. Most existing time series forecasting pipelines can adopt SCALEIN with a one-line code change. 1 Introduction Scale differences across time series pose a challenge for any forecasting model. Individual series can exhibit substantial scale variation across sliding windows due to nonstationarity, level shifts, or trends, and, in datasets containing multiple series, across series themselves. This challenge is particularly relevant in Time Series Foundation Models (TSFMs) [Garza et al., 2023, Das et al., 2024, Ansari et al., 2025, Woo et al., 2024, Goswami et al., 2024, Cohen et al., 2025, Ness et al., 2023], where pretraining combines series from diverse domains, so scale differences across series within a training batch arise by construction and can span many orders of magnitude. Prior empirical studies highlight related scale sensitivities. Wolff et al. [2025] finds that models underperform on low-magnitude series, Ahmed et al. [2026] find thatReVIN’s effectiveness depends on where the loss is computed, and Berthelier et al. [2026] finds empirically that computing loss on scaled targets improves model performance, Olivares et al. [2025] reveals accuracy struggles of TSFMs and traces them to scale heterogeneity. A large body of work has proposed scaling methods for time series focused on what statistics to use and how to compute them (ReVIN[Kim et al., 2022], DAIN [Passalis et al., 2020], GAS-Norm [Urettini et al., 2024], Non-stationary Transformers [Liu et al., 2022]). An orthogonal design choice is where the scaling boundary sits relative to the loss, that is, whether the loss is computed after inverse scaling of model predictions or after scaling the target. While recent work has empirically compared these loss spaces, we formalize the consequences in training and show that this boundary determines whether training is scale-invariant or scale-contaminated. ∗ All experiments were conducted in a controlled research setting and do not represent production deployments. Preprint. arXiv:2610.07324v1 [cs.LG] 5 Oct 2026 0246810 Time −40 −20 0 20 40 y (original scale) (a) Cross-series scale heterogeneity 0246810 Time 0 100 200 300 400 500 jr z Lj rL rL rL (b) Scale-contaminated r z L (gradients /b p ) 0246810 Time 0 100 200 300 400 500 jr z Lj rLrLrLrL === (c) Scale-invariant r z L (gradients independent of b) Series A (b=52) Series B (b=13) Series C (b=4) Series D (b=1) 10 −3 10 −2 10 −1 10 0 10 1 10 2 10 3 jrLj (d) Cross-series bias: contaminated r z (solid) vs invariant r z (hatched) Series A (b=52)Series B (b=13)Series C (b=4)Series D (b=1) −10 0 10 y t (e) Within-series bias (nonstationary scale) Raw y t Scaled z t =y t =b t −1 0 1 z t 0255075100125150175200 Time 0 10 20 30 40 50 jrLj (15-step avg.) ScaleCon (r z ) ScaleIn (r z ) Figure 1: SCALEIN removes scale-driven gradient bias (Theorem 3.2). (a) Synthetic series scales spanb ∈ 1, 4, 13, 52. (b) Under SCALECON, MSE gradients with respect toˆzscale asb 2 , givingb=52approximately 2,700×the gradient magnitude ofb=1for equal scaled residuals. (c) SCALEIN leaves gradients unchanged by rescaling, on the samey-axis. (d) Gradient distributions under SCALECON (solid) and SCALEIN (hatched). (e) Growing scale induces gradient drift under SCALECON but not SCALEIN. Lower-row curves use a 15-step moving average of|∇L|. When the loss is computed after rescaling predictions back to the original scale, as prescribed byReVIN[Kim et al., 2022], the full series scale is reintroduced into the loss function. We refer to this phenomenon as scale-contaminated training (SCALECON). Under a degree-ploss, each series’ gradient is weighted byb p , creating an implicit and potentially unintended importance weighting as shown in Fig. 1 and discussed in Section 3. Nonstationarity within a single series causes this weighting to vary across forecast creation dates even in traditional single-series training. In TSFMs, cross-dataset training compounds the effect across series within a batch. In this work, we show that computing the loss directly on scaled targets yields Scale-Invariant Training (SCALEIN) and improves forecast accuracy in TSFM pretraining and supervised forecasting. Yet despite its impact on TSFMs, loss space is rarely discussed or documented in published work. Tables 1 and 2 show inconsistent reporting across existing TSFMs and popular forecasting codebases. Of the 24 TSFMs surveyed (Table 1), only 9 explicitly document their loss space. For another 7, we infer the likely loss space by tracing how the paper describes normalization, model outputs, and loss computation, although the paper does not clearly establish it. Furthermore, inspecting the released code allows us to classify 5 more models, while 3 remain unresolved from either source. Among the 21 models assigned a loss space from these sources, 14 use SCALECON and 7 use SCALEIN, showing that no prevailing convention exists. Moreover, among popular forecasting codebases (Table 2), only 2 support scale-invariant loss in some capacity but none make loss space user-configurable or document it explicitly. The practical change for training pipelines is one line of code that computes the loss on scaled targets instead of inverse scaling predictions, using existing context statistics (see Appendix C). We advocate for explicit reporting of loss space as a training design choice that affects convergence and downstream accuracy. Our main contributions are as follows: (i)We formally characterize the implicitb p weighting induced by inverse scaling and prove that Scale- Invariant Training (SCALEIN) leaves every mini-batch gradient and the full optimization trajectory unchanged under arbitrary positive series-wise rescaling (Theorem 3.2). (i)We document inconsistent reporting of loss space across TSFM papers and forecasting codebases, establishing the need to report it explicitly as a design choice. The practical change is lightweight and computes the loss on scaled targets using existing context statistics. (i) We validate scale-invariance theory in four settings using controlled synthetic data, a real-world scale-swap intervention, TSFM pretraining, and supervised training. These experiments provide empirical evidence that SCALEIN removes scale-driven convergence disparities and improves forecast accuracy in both foundation model pretraining and supervised forecasting settings. 2 Table 1: A survey of loss spaces used in time series foundation models. Explicit: the paper states the loss space explicitly in text or in equations that define the normalized or denormalized quantities entering the loss. Inferred: it is possible to trace the target from the input through the normalization to the loss and infer with high likelihood what is the loss space. Code: the loss space is determined by the publicly released code. A dash means that we were not able to determine the loss space (or loss function) from that source. ModelLoss FunctionExplicitInferredCode TimeGPT-1Garza et al. [2023]– LagLLamaRasul et al. [2023]Student-t NLL–SCALECON ¶ SCALECON ¶ Time-LLMJin et al. [2024]MSE–SCALECONSCALECON MoiraiWoo et al. [2024]Mixture NLL–SCALECON ¶ SCALECON ¶ MOMENTGoswami et al. [2024]MSE–SCALECONSCALECON ‡ TimesFMDas et al. [2024]MSE–SCALECONSCALECON ∗‡ TimesFM-2.5Das et al. [2024]Quantile–SCALECON ∗ TimesFM-3Jain and Sen [2026]Quantile– Moirai-2.0Liu et al. [2025a]Quantile–SCALECONSCALEIN ∗ Chronos2Ansari et al. [2025]QuantileSCALEINSCALEINSCALEIN TotoCohen et al. [2025]NLL + Cauchy–SCALEIN ∗ Toto-2.0Khwaja et al. [2026]QuantileSCALEINSCALEINSCALEIN ∗ TiRexAuer et al. [2025]QuantileSCALECONSCALECON– TiRex-2Podest et al. [2026]QuantileSCALECONSCALECONSCALECON ∗ KairosFeng et al. [2025]Weighted quantile–SCALEIN SundialLiu et al. [2025b]TimeFlow– YingLongWang et al. [2025]Weighted quantileSCALECON † SCALECON † SCALECON † Falcon 2Xia et al. [2026]QuantileSCALEINSCALEIN– ZeusFu et al. [2026]Quantile–SCALEIN ∗ Timer-S1Liu et al. [2026]Weighted quantileSCALECON † SCALECON † – Time-MoEShi et al. [2025]Huber–SCALECON § TTMEkambaram et al. [2024]MSESCALECONSCALECONSCALECON FlowStateGraf et al. [2026]QuantileSCALECONSCALECON– PatchTST-FMWen et al. [2026]Quantile–SCALEINSCALEIN ∗ Inferred from released model, fine-tuning, or inference code, as the pretraining code is not released. ¶ If NLL is the loss function, the original-space loss differs from the normalized one only by a constantlogσwhich does not affect gradients. † Pinball loss divided by the mean absolute target. ‡ Additionally standardization globally per dataset and channel. § Series are globally standardized, with no per-window scaling. Table 2: Training properties of time series forecasting codebases. CodebaseReferenceSCALECONInvariant (SCALEIN)Fixed/User-specified GluonTSAlexandrov et al. [2020]NLL, CRPS, EnergyScore—Fixed PyTorch ForecastingBeitner [2020] MAE, RMSE, MAPE, SMAPE, QuantileLoss, NormalDist, LogNor- mal, NegBinomial, Beta, MQF2 —Fixed TSLibWu et al. [2023]—MSE, MAPE, MASE, SMAPEFixed NeuralForecastOlivares et al. [2022] NLL (Normal, StudentT, NegBino- mial, Poisson, Tweedie) MAE,MSE,RMSE,MAPE, SMAPE, MASE, QuantileLoss, MQLoss Fixed 2 Related Work Training time series models on heterogeneous data requires accounting for series whose scales differ by orders of magnitude. The most popular approach, and the one most frequently cited in TSFM papers, is Reversible Instance Normalization (ReVIN) [Kim et al., 2022], which scales each context window inside the model’s forward pass and restores predictions to the original scale before computing the loss. However, these TSFMs differ in where they compute the loss (see the summary in Table 1). One group [Rasul et al., 2023, Goswami et al., 2024, Jin et al., 2024, Woo et al., 2024, Auer et al., 2025, Das et al., 2024, Liu et al., 2026, Graf et al., 2026] reverses the scaling before 3 computing the loss, as prescribed byReVIN. Another group [Ansari et al., 2025, Cohen et al., 2025, Khwaja et al., 2026, Xia et al., 2026] scales the targets and computes a scale-invariant loss. DAIN [Passalis et al., 2020] learns adaptive input shifts, scales, and feature gates. Other approaches including Non-stationary Transformers [Liu et al., 2022], GAS-Norm [Urettini et al., 2024], and Vilhes et al. [2026], differ in how they estimate scaling statistics but, like ReVIN, restore predictions before computing the loss. Recent empirical studies examine some of these design choices. Ahmed et al. [2026] compare scaling schemes across TSFM architectures, including a comparison of loss computed on scaled targets with clipping and loss computed after returning predictions to the original scale. Berthelier et al. [2026] ablate the components ofReVIN on two supervised forecasters across four benchmark datasets. They observe that computing loss on scaled targets improves accuracy and hypothesize that the benefit comes from equal weighting of low- and high-scale series. We formally characterize this mechanism for homogeneous residual losses, deriving the scale-dependent gradient weights and establishing conditions for invariance of the full optimization trajectory. Moreover, our controlled experiments isolate the effect of loss space. On the empirical side, we evaluate TSFM pretraining and supervised forecasting on the M1, M3, M4, Tourism, Favorita, and GIFT-Eval datasets [Makridakis et al., 1982, Makridakis and Hibon, 2000, Makridakis et al., 2020, Athanasopoulos et al., 2011, Cook et al., 2021, Aksu et al., 2024]. Our synthetic studies isolate theb p mechanism, while pretraining and supervised experiments test its practical consequences (Section 4). These results establish why the loss space should be deliberately chosen and reported explicitly in forecasting papers as a design choice. 3 Forecasting Task and Scale-Invariant Training Losses We consider supervised univariate forecasting with real-valued targets. At forecast creation datet, the model receives a contexty t−L+1:t ∈ R L , whereLis the supported context length, and predicts the nextHvalues y t+1:t+H ∈ R H . Before the model sees the context, a scaling (or normalization) method computes an offsetaand a positive denominator b from observations available by time t and transforms each value as: z t = y t − a b .(1) After that, the model maps the scaled context z t−L+1:t to a scaled prediction ˆz t+1:t+H , ˆ z t+1:t+H = f θ (z t−L+1:t ).(2) In this work, we study residual losses of the formL(y, ˆy) = φ(y− ˆy) that satisfy φ(br) = b p φ(r) for b > 0. The three cases used throughout this manuscript and popular in related work are: MAE(y, ˆy) =|y− ˆy|,p = 1, MSE(y, ˆy) = (y− ˆy) 2 ,p = 2, QL q (y, ˆy) = q(y− ˆy) + + (1− q)(ˆy− y) + , p = 1. (3) The central choice is whether in training the loss is computed before or after returning the prediction to the original scale. The default pipeline with ReVIN [Kim et al., 2022] first reverses the transform, ˆy t+h = bˆz t+h + a,(4) and then compares ˆy t+h with y t+h . We call this objective scale-contaminated (SCALECON), L contam = 1 H H X h=1 φ (y t+h − (bˆz t+h + a)).(5) The alternative is to scale the observed target with the same a and b, z t+h = y t+h − a b , h = 1,...,H.(6) Note that the future observations do not contribute toaorb. As we show below, computing the loss before reversing the transform gives scale-invariant loss (SCALEIN), L inv = 1 H H X h=1 φ (z t+h − ˆz t+h ).(7) 4 Scale-contaminated (ScaleCon) Scale-invariant · ScaleIn Original context 0 500 1000 y t− L+ 1 : t Scaled context −2 0 z= (y− a) b Forecasting model ̂ z= f θ (z) Reverse scaling ̂ y= b ̂ z+ a Original target 0 500 1000 y t+ 1 : t+ H Prediction ̂ z Scaled target −2 −1 0 1 z ⋆ = (y t+ 1 : t+ H − a) b (y ⋆ , ̂ y) contam ∝ b p scale-weighted (z ⋆ , ̂ z) inv ⟂ b p scale-independent a, b a, b a, b Figure 2: Loss placement determines whether inverse scaling introducesb p gradient weighting. Both pipelines scale inputs using context statisticsa,b. Top: SCALECON reverses scaling before computing the loss, introducing theb p weight. Bottom: SCALEIN computes loss on scaled targets and reverses scaling only for inference, yielding scale-invariant training (Theorem 3.2). Lemma 3.1 (Inverse scaling creates importance weights). For every forecasting example, the scale-contaminated and scale-invariant objectives satisfy L contam = b p L inv ,(8) ∇ θ L contam = b p ∇ θ L inv .(9) The gradient relation also holds for subgradients at nondifferentiable residuals. Proof in Appendix B. For a mini-batchB, the resulting update is therefore ∇ θ L contam,B = 1 |B| X i∈B b p i ∇ θ L inv,i .(10) The denominatorb i is an importance weight inside every gradient descent update. For instance, the weight is b i for MAE and quantile loss andb 2 i for MSE. Becauseb i varies across series and forecast creation dates, scale- contaminated training changes the relative importance of forecasting examples even when their scaled forecasting problems are identical. Figure 1 visualizes the MSE gradient gap, and the left panel of Figure 9 confirms theb 2 pattern at initialization. In Figure 2 we visualize the difference between SCALECON and SCALEIN in a typical forecasting pipeline. Theorem 3.2 (Scale-Invariant Training). Assume that the scaling method is scale equivariant, meaning that multiplying a series by anyc > 0multiplies its offsetaand denominatorbby the samec. Under scale-invariant loss, independently multiplying each training series by any positive constant leaves the loss and parameter gradient of every forecasting example unchanged for allθ. Consequently, it leaves every mini-batch gradient and the full parameter trajectory unchanged when the initialization, mini-batch sequence, optimizer state, and optimizer randomness are fixed. Proof in Appendix B. The scale-equivariance assumption holds for commonly used scaling methods such as standardization ( x−μ σ ), min-max scaling ( x−min max−min ), and robust scaling ( x−m IQR ), all with positive denominators and without fixed numerical stabilizers. Thus, under SCALEIN, series’ scale cannot change which forecasting examples dominate training. Fixed additiveεterms or scale floors can break exact equivariance, so practical invariance may be approximate. Importantly, they do not remove the b p weighting in Lemma 3.1, which holds for the actual denominator used in the affine transform. Remark 3.3 (Practical intuition). Consider two series with identical scaled contexts and targets, but denominators b = 1andb = 10– for instance, assume standardization withbcorresponding to the standard deviation. If the model makes the same scaled error on both, for the second series, SCALECON yields a gradient that is10times larger under MAE or pinball loss and100times larger under MSE. The optimizer treats the second series as more 5 σ √ 10 σ 10σ global 0100200300400500 Training step 0.0 0.5 1.0 1.5 nMSE (a) Scale-invariant 0100200300400500 Training step (b) Scale-contaminated 0500100015002000 Training step (c) Equal-scale control 0500100015002000 Training step (d) Unit gradient Figure 3: Per-source validation nMSE over training steps. We report the mean±1standard deviation over five seeds. (a) The scale-invariant loss yields parallel convergence. (b) The scale-contaminated loss produces scale-ordered convergence. (c) The disparity disappears when all sources share the same scale. (d) The disparity persists after matching the global gradient norm. The dashed grey curve is global nMSE, aggregated rather than disaggregated by source. important solely because of its raw scale. Under SCALEIN, both series produce the same gradient. Note, however, that even under scale invariance, different forecasting problems can still produce different gradients and achieve nonuniform convergence due to inherent data difficulty. Some TSFMs (e.g.,Chronos2) additionally apply an arcsinh transform after standardization. We note that scale contamination persists when this transform is reversed, but the nonlinearity changes the residual geometry, so the two objectives are not related solely byb p . Appendix B.2 gives the corresponding identity and explains why SCALEIN remains scale-invariant under equivariance. Although stated for univariate point forecasting, our result extends componentwise to multivariate objectives that sum homogeneous residual losses across channels [Liu et al., 2024, Ekambaram et al., 2024]. It also covers quantile forecasting because pinball loss is homogeneous of degree one at every quantile level [Taylor, 2000, Ansari et al., 2025], as detailed in Appendix B.1. We include an algorithm implementation of SCALEIN in Section C. For clarity, we note that predictions ˆz return to the original scale for inference and evaluation. 4 Experiments We evaluate SCALEIN in 4 complementary settings, from controlled isolation of the mechanism (Sections 4.1 and 4.2) to TSFM pretraining and supervised forecasting (Sections 4.3 and 4.4). 4.1 Controlled Synthetic Data Experiment In this experiment, we expose the disparate convergence per-series resulting fromb p gradient bias in standard ReVIN and demonstrate how SCALEIN fixes this. To isolate the effect in a simple controlled setting, we generate a synthetic data set consisting of three sinusoidal sources with scaling denominators in an approximate ratio 1 : √ 10 : 10. As shown in Lemma 3.1,ReVIN’s denormalization under MSE loss introducesb 2 scaling of loss values and gradient multipliers, yielding the ratio1 : 10 : 100across the three sources. This scale-dependent gradient should cause the optimizer to disproportionately reduce error on high-scale series and lead to disparate convergence rates across sources. To test this, we train a 2-layerPatchTST-style Transformer withReVINfor 30K SGD steps, fixing the random seed, parameter initialization, and the examples seen during optimization. With all model and training aspects held fixed, the only experimental parameter that varies is whether the model learns on a scale-contaminated or a scale-invariant loss. In the synthetic and scale-swap experiments (Section 4.2), we report normalized mean squared error (nMSE), computed between predictions and targets normalized using each window’s context mean and standard deviation, then averaged over forecast timesteps and validation windows. Both loss conditions use this metric so errors are comparable across scales. The results show that under SCALECON, the high-scale source converges50×faster than the low-scale source (Figure 3b). In comparison, with SCALEIN in Figure 3a, all three scale sources converge in parallel at a similar pace. This confirms that theb p gradient bias characterized by Lemma 3.1 does indeed produce disparate per-series convergence, as hypothesized. Appendix E.1 provides the full experimental settings. 6 b=1b=10 Electricity Traffic Solar Taxi Wind Farms Pedestrian KDD 2018 FRED-MD 0 1 2 3 4 5 6 Time-averaged validation nMSE (a) Scale-invariant loss (SCALEIN) Electricity Traffic Solar Taxi Wind Farms Pedestrian KDD 2018 FRED-MD 0 1 2 3 4 5 6 Time-averaged validation nMSE (b) Scale-contaminated loss (SCALECON) Figure 4: Inflating scale leaves trajectory-level error nearly unchanged under SCALEIN (a), but lowers it on average under SCALECON (b). Bars showA (k) b,c averaged over five seeds and 15 complementary assignment pairs. Error bars show 95% Student-t intervals across assignment pairs. In addition, we controlled the experiments to better attribute the gap tob p and eliminate confounders. First, to isolate the effect of series scale, we give all sources unit amplitude and train again with the scale-contaminated loss (Figure 3c). The gap vanishes compared with Figure 3a, confirming that it stems from scale heterogeneity rather than intrinsic source difficulty. Second, to eliminate gradient-magnitude confounders, we rescale the global gradient to unit norm (Figure 3d). The convergence gap persists, matching the dynamics of SCALECON training in Figure 3b. Becauseb 2 scaling is applied item-wise within a batch, a single global scalar cannot fix the between- source ratio: it rescales all per-item gradients uniformly and preserves their relative magnitudes. These findings also hold under Adam [Kingma and Ba, 2015], where the gap narrows from approximately50×to10×, consistent with adapting gradient magnitude but not its per-source composition (Appendix E.2). 4.2 Scale-Swap Intervention Experiment In this experiment, we confirm the effect of loss-space gradient bias on disparate per-series convergence using real-world data. We selectively rescale series10×as a diagnostic: under SCALEIN, Theorem 3.2 predicts rescaling leaves convergence unchanged, while under SCALECON, theb p bias should cause the optimizer to favor the inflated series. We selectedK = 8datasets from different domains in GluonTS [Alexandrov et al., 2020] (Electricity, Traffic, Solar, Taxi, Wind Farms, Pedestrian, KDD Cup 2018, FRED-MD) and applied global standard scaling ( x−μ σ ). Then, for each datasetD (k) we create a matched pair(D (k) b=1 , D (k) b=10 ): the original series at scaleb=1and a copy multiplied byb=10. Because both copies of a dataset differ only in scale, any convergence difference between them is attributable to the b p -scaled gradient bias rather than to intrinsic difficulty. We construct 15 configurationsD c = D (k) b 8 k=1 , each assigning four datasets tob=1and four tob=10, plus a complementary run that reverses every assignment. Across the30resulting runs, every dataset appears 15 times at each scale, and we average over configurations to minimize the impact of batch composition. We use the PatchTST-style Transformer from Section 4.1 with four layers instead of two, trained under scale-invariant and scale-contaminated losses with SGD onS = 5seeds per condition. Because scale-contaminated MSE multiplies the gradient of ab=10window by100while leaving ab=1window unchanged, the two conditions cannot share a learning rate without confounding the diagnostic with raw update magnitude. We therefore divide the contaminated learning rate by the nominal average multiplier of a balanced batch,(1 + 100)/2 = 50.5, giving10 −4 for the scale-invariant condition and1.9802× 10 −6 for the scale-contaminated one. This approximately compensates for the overall gradient amplification while preserving the relative100 : 1weighting within each contaminated batch, which is the quantity our diagnostic tests. The comparison of interest is in either case within a condition, between its ownb=1andb=10copies. Figure 5a shows the paired convergence trajectories, and the weighted-average analysis below quantifies it. Appendix F.1 provides the full experimental settings. To validate our hypothesis from Section 4.1 on real-world data, we compare each dataset’s average validation nMSE over training when assignedb=1versusb=10. We compute this average as the area under the validation curve divided by training duration, then average over 5 seeds and 15 complementary assignment pairs. Figure 4 shows that inflating scale leaves this error essentially unchanged under SCALEIN, with a mean difference across datasets below0.002%. Under SCALECON, the mean error falls from2.6to1.9, a26.8%reduction, and a one-sided pairedt-test across the dataset-level differences is significant atα = 0.05. This shows that theb p gradient bias of 7 010k20k30k Training step 1.6 1.8 2.0 2.2 2.4 2.6 Median per-dataset nMSE Scale-invariant (ScaleIn) b= 1b= 10 Scale-contaminated (ScaleCon) b= 1b= 10 (a) Scale-swap corpus, K=8 datasets 0.7 1.8 2.9 Chronos-2 0.7 1.3 1.9 Moirai-2.0 020k40k60k80k Training step 1.4 2.4 3.4 TimesFM-2.5 020k40k60k80k Training step 1.1 1.9 2.7 PatchTST Median per-dataset MASE Scale-invariant (ScaleIn)Scale-contaminated (ScaleCon) (b) Full TSFM pretraining, four architectures Figure 5: SCALEIN removes scale-driven convergence disparities and improves TSFM convergence. (a) Median per-dataset nMSE over 8 scale-swap datasets, aggregated across 15 assignment pairs and five seeds. Theb=1and b=10curves overlap under SCALEIN and separate under SCALECON. (b) Median per-dataset MASE for four architectures over 80,000 updates. Plotted are averages across 4 seeds, with 95% Student-t intervals. Lemma 3.1 extends to real-world data. Full metric definitions, test statistics, and dataset-level trajectories are in Appendix F. 4.3 Time Series Foundation Model Pretraining In this experiment, we empirically analyze the difference between using SCALEIN and SCALECON in TSFM pretraining. We employ 5 forecasting benchmarks drawn from well-known forecasting competitions:M1[Makri- dakis et al., 1982],M3[Makridakis and Hibon, 2000],M4[Makridakis et al., 2020],Favorita[Cook et al., 2021], andTourism[Athanasopoulos et al., 2011]. We additionally evaluate on GIFT-Eval, a popular benchmark in the TSFM literature [Aksu et al., 2024]. We adopt the data handling and pre-processing practices established in prior work leveraging heterogeneous-dataset pretraining [Alexandrov et al., 2020, Olivares et al., 2025, Potosnak et al., 2026]. Benchmark training prefixes exclude reserved validation and test regions (Appendix G). We pretrain four architectures for80,000updates at batch size512,4seeds each, on GiftEvalPretrain [Aksu et al., 2024] with the training regions of M1, M3, M4, Tourism, and Favorita included. Two are recent foundation models in their small configurations,Chronos2(27.9M parameters, Ansari et al. [2025]) andMoirai-2.0(5.0M parameters, Liu et al. [2025a]). The third isTimesFM2.5(49.6M parameters, Das et al. [2024]), and the fourth is a supervisedPatchTST(1.5M parameters, Nie et al. [2023]). These architectures span a representative range of recent TSFMs in terms of scale (1.5M–49.6M parameters) and design (decoder-only and encoder-only).PatchTST was not originally proposed as a foundation model, but recent work has shown its effectiveness in multi-dataset pretraining [Wen et al., 2026]. Its inclusion highlights that this work is applicable to any general forecaster, particularly those trained across heterogeneous datasets. Each model keeps the temporal scaling method of its own published implementation. Within each model, SCALEIN and SCALECON runs see identical data in the same order and share initialization, optimization schedule, and learning rate, with no per-condition adjustment. AdamW normalizes each coordinate by the running root-mean-square of its own gradient, so a uniform rescaling of the objective leaves its updates unchanged (note, it does not wash out within batch item-wise b p scaling). We score the final checkpoint on one official test horizon per series for M1, M3, M4, and Tourism, 16 day holdout for Favorita, and GIFT-Eval’s non-overlapping short-term windows across 55 configurations. Models receive up to 512 historical observations, with shorter contexts padded and masked. Within each benchmark, we average scores between frequency groups (dataset configurations for GIFT-Eval), weighted by the number of forecasts evaluated, then average across 4 training seeds (Appendix G). We include Seasonal Naïve (S.Naive) as a reference, which repeats the last observed season using the benchmark’s seasonal period and repeats the last observation when fewer than two seasons are available [Hyndman et al., 2026]. We report two scale-independent metrics. The Mean Absolute Scaled Error (MASE) [Hyndman and Koehler, 2006] normalizes the forecast error by the in-sample seasonal naive error, MASE = 1 H H X h=1 |y t+h − ˆy t+h | 1 T−m P T j=m+1 |y j − y j−m | ,(11) 8 Table 3: Forecasting performance across benchmark datasets. The results were averaged across 4 seeds. SCALEIN uses scale-invariant loss and SCALECON uses scale-contaminated loss. These are the only differences between the two runs of each architecture.∆%reports SCALEIN relative to SCALECON. Green denotes an improvement and red a regression. The better loss is bold within each architecture. References are fitted on the series they forecast and appear behind the double rule. Chronos2 Moirai-2.0 TimesFM2.5 PatchTSTReference BenchmarkSC.IN SC.CON ∆%SC.IN SC.CON ∆%SC.IN SC.CON ∆%SC.IN SC.CON ∆%S.Naive MASE M1 1.8792.388-21.32.0132.123-5.22.7312.863-4.62.0913.859-45.82.117 M3 1.6081.963-18.11.6211.800-9.92.2332.573-13.22.0414.840-57.81.764 M41.8452.294-19.61.8791.986-5.42.6613.208-17.02.2354.555-50.92.057 Tourism2.3502.816-16.52.3162.564-9.73.8204.108-7.02.5093.123-19.72.412 Favorita0.8160.862-5.40.8250.864-4.51.0501.360-22.80.8890.968-8.21.079 GIFT-Eval1.4951.782-16.11.5471.676-7.72.1622.687-19.51.9022.813-32.41.824 WQL M10.1330.119+11.70.1740.164+6.60.2080.241-13.70.2270.284-20.20.150 M30.0840.088-4.70.1090.114-4.40.1580.176-10.60.1530.253-39.30.110 M4 0.0840.089-5.70.1100.107+2.90.1610.183-11.70.1470.229-36.00.115 Tourism0.0860.113-24.10.1160.132-12.00.2270.266-14.80.1490.200-25.50.123 Favorita 0.2970.326-9.10.3870.422-8.30.6380.848-24.80.5400.595-9.30.614 GIFT-Eval 0.2850.320-10.90.3710.397-6.50.5690.681-16.40.5180.597-13.30.705 wheremis the seasonal period. The Weighted Quantile Loss (WQL) averages twice the quantile loss over the quantile levelsQ supplied by each model, WQL = 2 |Q| X q∈Q P H h=1 QL q (y t+h , ˆy (q) t+h ) P H h=1 |y t+h | ,(12) withQL q defined in Section 3. For models that do not supply quantile forecasts, we treat the point prediction as the median and compute WQL withQ =0.5. Table 3 reports MASE and WQL across all benchmarks for all architectures at the final checkpoint. SCALEIN outperforms SCALECON in terms of lower MASE across all 24 architecture-benchmark comparisons, with error reductions (averaged across architectures) between 10.2% and 24.1%. On WQL it wins 21 of 24 comparisons, with average performance gains between 3.3% and 19.1%. The improvement is consistent regardless of the normalization method:TimesFM2.5computes statistics from an expanding causal window,Chronos2,Moirai-2.0, and PatchTST from the context window, with Chronos2 additionally applying an arcsinh transformation. In an ablation study withMOMENT[Goswami et al., 2024], we also find that the benefit extends beyond forecasting objectives: SCALEIN substantially lowers the reconstruction MASE ofMOMENT, trained on a masked reconstruc- tion task (Appendix I). The benchmark-frequency breakdowns, standard deviations, and cross-dataset variance trajectories can be found in Appendix G. We hypothesize that these accuracy improvements stem from more stable and scale-unbiased optimization. Fig- ure 5b follows the median per-dataset MASE throughout the80,000optimization steps. The SCALEIN develops an advantage during training for all four architectures and retains it at the final checkpoint. Additionally, we observe that SCALECON loss introduces excess difficulty andTimesFM2.5is the clearest case of that. Its scale- contaminated run degrades for the first25,000steps before recovering, while its scale-invariant run is flat from the start. 4.4 Supervised Neural Forecasting Models The experiments in Section 4.3 establish that the scale-contamination effect persists under TSFM-scale, multi- dataset pretraining, affecting forecast error on both in-distribution and zero-shot benchmarks. Beyond the TSFM pretraining setting, we show that forecast error improvements with SCALEIN extend to standard deep forecasters trained across heterogeneous datasets on a smaller scale, with supervised evaluation only. We train 5 established architectures,NHITS[Challu et al., 2023],NBEATS[Oreshkin et al., 2020],PatchTST[Nie et al., 2023],TCN[Bai et al., 2018], andLSTM[Hochreiter and Schmidhuber, 1997], on M1, M3, M4, and Tourism, fitting 6 separate models, one for each sampling frequency (i.e., yearly, monthly, quarterly, weekly, daily, and hourly) by aggregating training splits across datasets. All neural networks runs share data splits, Adam optimizer (lr = 10 −3 ), and a 9 Table 4: Forecasting performance by benchmark and architecture. For each seed, we select the checkpoint with the lowest validation MAE. Within each frequency, we average MASE across forecasts and compute WQL as a ratio of sums. We then macro-average both metrics across frequencies and report their means across 4 seeds. SCALEIN uses scale-invariant loss and SCALECON uses scale-contaminated loss. Lower is better, and bold marks the better loss within each architecture. Neural Forecast BaselinesReference Benchmark Metric N-HiTS N-BEATS PatchTST TCN LSTM S.Naive ETS ARIMA SC.IN SC.CONSC.IN SC.CONSC.IN SC.CONSC.IN SC.CONSC.IN SC.CON M1 MASE1.8261.8551.8221.8351.9171.9111.9411.9871.9982.0572.4781.904 1.977 WQL0.0990.0980.0990.0980.1060.1060.1000.1010.1050.1060.1670.108 0.132 M3 MASE1.6521.6401.6771.7672.8023.0831.8962.0602.1332.1342.0111.477 1.687 WQL0.0900.0910.0890.0920.1160.1230.0990.1030.1090.1070.1030.087 0.090 M4 MASE1.9912.0071.9872.0363.6393.5722.0902.1982.2642.2682.4462.030 2.351 WQL0.0720.0720.0720.0730.0930.0930.0750.0760.0800.0810.0830.071 0.079 Tourism MASE2.3462.4202.3072.3922.8842.9602.6682.6882.6432.6142.2322.259 2.503 WQL 0.1330.1370.1320.1350.1640.1670.1530.1530.1530.1520.1420.133 0.156 20,000-step budget across 4 seeds, selecting checkpoints via minimum validation MAE. See Appendix H for complete data, standard deviation, and aggregation details. For reference, we compare with classical statistical baselines S.Naive, ETS, and ARIMA [Hyndman et al., 2026]. Table 4 shows that the scale-invariant loss lowers MASE in 16 of 20 matched architecture-benchmark comparisons and WQL in 15 of 20. At the finer benchmark-frequency level, SCALEIN lowers MASE in 49 of 75 comparisons and WQL in 43 of 75, with the complete breakdown and standard deviations reported in Appendix H. Averaged over all architectures, benchmarks, and seeds, SCALEIN lowers MASE by1.9%and WQL by1.1%The effect is therefore broad, but rather modest, and it varies between architectures and benchmarks. The fitted statistical references remain competitive and lead the neural models on several benchmark-metric pairs. 5 Conclusion In this work, we show that reversing an affine scaling before computing a degree-phomogeneous loss injects ab p factor into the gradient that grows with series scale, imposing an implicit curriculum that favors high-scale series. The remedy is a one-line code change: compute the loss directly on scaled targets. We confirmed this across the full spectrum from controlled synthetic studies to real-world pretraining. Scale-invariant loss (SCALEIN) closes the convergence gap on synthetic data and is invariant to artificial scale assignments on real data (Sections 4.1 and 4.2). Employing SCALEIN in TSFM pre-training leads on average to substantial improvements in MASE and WQL performance metrics, reaching 18.8% on GIFT-Eval and 21.9% on M-competitions (Section 4.3). The effect on neural forecasting models is broadly positive, but expectedly modest (Section 4.4). Based on the results, we thus posit that loss space is a crucial training decision – especially for TSFMs – that should be made deliberately and reported explicitly in any forecasting paper to promote reproducibility. Our findings motivate a few directions for future work. While the same scale bias persists in multivariate settings with independent scaling of each series, its practical impact and the effects of more general normalization schemes that jointly transform correlated channels remain to be studied. Another direction is to characterize how loss-space choices interact with probabilistic objectives and their parameterizations, beyond the homogeneous losses studied here. References I. Ahmed, D. Krompaß, C. Feng, and V. Tresp. A comparative study on how data normalization affects zero-shot generalization in time series foundation models. In ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3436–3440, 2026. T. Aksu, G. Woo, J. Liu, X. Liu, C. Liu, S. Savarese, C. Xiong, and D. Sahoo. GIFT-eval: A benchmark for general time series forecasting model evaluation. In NeurIPS Workshop on Time Series in the Age of Large Models, 2024. URL https://openreview.net/forum?id=Z2cMOOANFX. 10 A. Alexandrov, K. Benidis, M. Bohlke-Schneider, V. Flunkert, J. Gasthaus, T. Januschowski, D. C. Maddix, S. Rangapuram, D. Salinas, J. Schulz, L. Stella, A. C. Türkmen, and Y. Wang. GluonTS: Probabilistic and neural time series modeling in Python. Journal of Machine Learning Research, 21(116):1–6, 2020. URL http://jmlr.org/papers/v21/19-820.html. A. F. Ansari, O. Shchur, J. Küken, A. Auer, B. Han, P. Mercado, S. S. Rangapuram, H. Shen, L. Stella, X. Zhang, M. Goswami, S. Kapoor, D. C. Maddix, P. Guerron, T. Hu, J. Yin, N. Erickson, P. M. Desai, H. Wang, H. Rangwala, G. Karypis, Y. Wang, and M. Bohlke-Schneider. Chronos-2: From univariate to universal forecasting, 2025. URL https://arxiv.org/abs/2510.15821. G. Athanasopoulos, R. J. Hyndman, H. Song, and D. C. Wu. The Tourism forecasting competition. Interna- tional Journal of Forecasting, 27(3):822–844, 2011. ISSN 0169-2070. doi: 10.1016/j.ijforecast.2010.04.009. URLhttps://w.sciencedirect.com/science/article/pii/S016920701000107X. Special Section 1: Forecasting with Artificial Neural Networks and Computational Intelligence. Special Section 2: Tourism Forecasting. A. Auer, P. Podest, D. Klotz, S. Böck, G. Klambauer, and S. Hochreiter. TiRex: Zero-shot forecasting across long and short horizons with enhanced in-context learning. In The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2025. S. Bai, J. Z. Kolter, and V. Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018. J. Beitner. PyTorchForecasting: Forecasting with neural networks made simple. GitHub Repository, 2020. URL https://pytorch-forecasting.readthedocs.io/en/stable/. G. Berthelier, T. Nabil, E. L. Naour, R. Niamke, S. Perlaza, and G. Neglia. On the role of reversible instance normalization, 2026. URL https://arxiv.org/abs/2603.11869. C. Challu, K. G. Olivares, B. N. Oreshkin, F. G. Ramirez, M. M. Canseco, and A. Dubrawski. NHITS: Neural Hierarchical Interpolation for Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):6989–6997, June 2023. ISSN 2374-3468. URLhttps://ojs.aaai.org/index.php/ AAAI/article/view/25854. B. Cohen, E. Khwaja, Y. Doubli, S. Lemaachi, C. Lettieri, C. Masson, H. Miccinilli, E. Ramé, Q. Ren, A. Rostamizadeh, J. O. du Terrail, A.-M. Toon, K. Wang, S. Xie, Z. Xu, V. Zhukova, D. Asker, A. Tal- walkar, and O. Abou-Amal. This time is different: An observability perspective on time series foundation models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=1jDAYXfcS2. A. Cook, DanB, inversion, and R. Holbrook. Store sales - time series forecasting.https://w.kaggle.com/ competitions/store-sales-time-series-forecasting, 2021. Kaggle. A. Das, W. Kong, R. Sen, and Y. Zhou. A decoder-only foundation model for time-series forecasting. In R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 10148–10167. PMLR, 21–27 Jul 2024. URLhttps://proceedings.mlr.press/v235/ das24c.html. V. Ekambaram, A. Jati, P. Dayama, S. Mukherjee, N. H. Nguyen, W. M. Gifford, C. Reddy, and J. Kalagnanam. Tiny time mixers (ttms): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 74147–74181. Curran Associates, Inc., 2024. doi: 10.52202/079017-2359. URLhttps://proceedings.neurips.c/paper_files/paper/ 2024/file/874a4d89f2d04b4bcf9a2c19545cf040-Paper-Conference.pdf. K. Feng, S. Lan, Y. Fang, W. He, S. Lu, S. Gu, L. Ma, X. Lu, and K. Ren. Kairos: Toward adaptive and parameter-efficient time series foundation models, 2025. URL https://arxiv.org/abs/2509.25826. Y. Fu, Z. Shao, C. Yu, Y. Li, Y. Xu, X. Cheng, and F. Wang. Zeus: Towards tuning-free foundation model for time series analysis. In Forty-third International Conference on Machine Learning, 2026. URLhttps: //openreview.net/forum?id=Dtue4WLYSk. 11 A. Garza, M. M. Canseco, C. Challú, and K. G. Olivares. StatsForecast: Lightning fast forecasting with statistical and econometric models. PyCon Salt Lake City, Utah, US 2022, 2022. URLhttps://github.com/Nixtla/ statsforecast. A. Garza, C. Challu, and M. Mergenthaler-Canseco. TimeGPT-1, 2023. URL https://arxiv.org/abs/2310. 03589. R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso. Monash time series forecasting archive. In Neural Information Processing Systems Track on Datasets and Benchmarks, 2021. M. Goswami, K. Szafer, A. Choudhry, Y. Cai, S. Li, and A. Dubrawski. MOMENT: A family of open time- series foundation models. In R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 16115–16152. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/goswami24a.html. L. Graf, T. Ortner, S. Wo ́zniak, and A. Pantazi. Flowstate: Sampling-rate-equivariant time-series forecasting. In Forty-third International Conference on Machine Learning, 2026. URLhttps://openreview.net/forum? id=9yNGBueKgZ. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. C. C. Holt. Forecasting seasonals and trends by exponentially weighted moving averages. Journal of Economic and Social Measurement, 29(1-3):123–125, 2004. URLhttps://journals.sagepub.com/doi/abs/10.3233/ JEM-2004-0211. A. Howard, H. Yui, M. McDonald, and W. Cukierski. Recruit restaurant visitor forecasting.https://w. kaggle.com/competitions/recruit-restaurant-visitor-forecasting, 2017. Kaggle Competition. R. J. Hyndman and B. Billah. Unmasking the theta method. International Journal of Forecasting, 19(2): 287–290, 2003. ISSN 0169-2070. URLhttps://w.sciencedirect.com/science/article/pii/ S0169207001001431. R. J. Hyndman and Y. Khandakar. Automatic time series forecasting: The forecast package for R. Journal of Statistical Software, 27(3):1–22, 2008. doi: 10.18637/jss.v027.i03. URLhttps://w.jstatsoft.org/ index.php/jss/article/view/v027i03. R. J. Hyndman and A. B. Koehler. Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4):679–688, 2006. ISSN 0169-2070. URLhttps://w.sciencedirect.com/science/ article/pii/S0169207006000239. R. J. Hyndman, G. Athanasopoulos, A. Garza, C. Challu, M. Mergenthaler, and K. G. Olivares. Forecasting: Princi- ples and Practice, the Pythonic Way. OTexts, Melbourne, Australia, 2026. available at https://otexts.com/fpppy/. A. Jain and R. Sen. TimesFM-3: A zero-shot foundation model for multivariate forecasting.https://research. google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting/, Aug. 2026. Google Research Blog. M. Jin, S. Wang, L. Ma, Z. Chu, J. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan, and Q. Wen. Time- llm: Time series forecasting by reprogramming large language models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, vol- ume 2024, pages 23857–23880, 2024. URLhttps://proceedings.iclr.c/paper_files/paper/2024/ file/680b2a8135b9c71278a09cafb605869e-Paper-Conference.pdf. E. Khwaja, C. Lettieri, G. Woo, E. Belouadah, M. Cenac, G. Jarry, E. Paquin, X. Zhao, V. Zhukov, O. Abou-Amal, C. Liu, A. Talwalkar, and D. Asker. Toto 2.0: Time series forecasting enters the scaling era, 2026. URL https://arxiv.org/abs/2605.20119. T. Kim, J. Kim, Y. Tae, C. Park, J.-H. Choi, and J. Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=cGDAkQo1C0p. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. URL https://arxiv.org/abs/1412.6980. 12 G. Lai, W.-C. Chang, Y. Yang, and H. Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 95–104, 2018. C. Liu, T. Aksu, J. Liu, X. Liu, H. Yan, Q. Pham, S. Savarese, D. Sahoo, C. Xiong, and J. Li. Moirai 2.0: When less is more for time series forecasting, 2025a. URL https://arxiv.org/abs/2511.11698. Y. Liu, H. Wu, J. Wang, and M. Long.Non-stationary transformers: Exploring the stationarity in time series forecasting.In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 9881–9893. Curran Associates, Inc., 2022.URLhttps://proceedings.neurips.c/paper_files/paper/2022/file/ 4054556fcaa934b0bf76da52cf4f92cb-Paper-Conference.pdf. Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long. iTransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=JePfAI8fah. Y. Liu, G. Qin, Z. Shi, Z. Chen, C. Yang, X. Huang, J. Wang, and M. Long. Sundial: A family of highly capable time series foundation models. In Forty-second International Conference on Machine Learning, 2025b. URL https://openreview.net/forum?id=LO7ciRpjI5. Y. Liu, X. Su, S. Wang, H. Zhang, H. Liu, Y. Wang, Z. Ye, Y. Xiang, J. Wang, and M. Long. Timer-s1: A billion- scale time series foundation model with serial scaling, 2026. URL https://arxiv.org/abs/2603.04791. S. Makridakis and M. Hibon. The M3-competition: results, conclusions and implications. International Journal of Forecasting, 16(4):451–476, 2000. ISSN 0169-2070. doi: 10.1016/S0169-2070(00)00057-1. URLhttps: //w.sciencedirect.com/science/article/pii/S0169207000000571. The M3- Competition. S. Makridakis, A. Andersen, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, E. Parzen, and R. Winkler. The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of Forecasting, 1(2):111–153, 1982. doi: 10.1002/for.3980010202. URLhttps://onlinelibrary.wiley. com/doi/abs/10.1002/for.3980010202. S. Makridakis, E. Spiliotis, and V. Assimakopoulos. The M4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54–74, 2020. ISSN 0169-2070. doi: 10.1016/j.ijforecast. 2019.04.014.URLhttps://w.sciencedirect.com/science/article/pii/S0169207019301128. M4 Competition. P. Mancuso, V. Piccialli, and A. M. Sudoso. A machine learning approach for forecasting hierarchical time series. Expert Systems with Applications, 182:115102, 2021. ISSN 0957-4174. URLhttps://w.sciencedirect. com/science/article/pii/S0957417421005431. M. V. Ness, H. Shen, H. Wang, X. Jin, D. C. Maddix, and K. Gopalswamy. Cross-frequency time series meta- forecasting, 2023. URL https://arxiv.org/abs/2302.02077. Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Jbdc0vTOcol. K. G. Olivares, C. Challú, A. Garza, M. M. Canseco, and A. Dubrawski. NeuralForecast: User friendly state-of- the-art neural forecasting models. PyCon Salt Lake City, Utah, US 2022, 2022. URL https://github.com/ Nixtla/neuralforecast. K. G. Olivares, M. Wolff, T. Konstantinova, S. Ramasubramanian, B. N. Oreshkin, A. G. Wilson, R. K. Vadlamani, A. Potapczynski, W. Potosnak, M. W. Mahoney, M. Cao, and D. Efimov. A more realistic evaluation of cross-frequency transfer learning and foundation forecasting models. In Recent Advances in Time Series Foundation Models Have We Reached the ’BERT Moment’?, 2025. URLhttps://openreview.net/forum? id=hPqpEzumiC. B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=r1ecqn4YwB. 13 S. Palaskar, V. Ekambaram, A. Jati, N. Gantayat, A. Saha, S. Nagar, N. H. Nguyen, P. Dayama, R. Sindhgatta, P. Mohapatra, et al. Automixer for improved multivariate time-series forecasting on business and it observability data. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 22962–22968, 2024. N. Passalis, A. Tefas, J. Kanniainen, M. Gabbouj, and A. Iosifidis. Deep adaptive input normalization for time series forecasting. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3760–3765, 2020. P. Podest, M. Pichler, E. Bürger, L. Zólyomi, B. Voggenberger, W. Berghammer, D. Klotz, S. Böck, G. Klambauer, and S. Hochreiter. Tirex-2: Generalizing tirex to multivariate data and streaming, 2026. URLhttps://arxiv. org/abs/2607.01204. W. Potosnak, M. Wolff, M. Cao, R. Ma, T. Konstantinova, D. Efimov, M. W. Mahoney, B. N. Oreshkin, and K. G. Olivares. Forking-sequences: Statistically and computationally efficient multi-horizon forecasting with reduced volatility. Transactions on Machine Learning Research, 2026. ISSN 2835-8856. URLhttps: //openreview.net/forum?id=dXdycy7WCX. K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, G. Adamopoulos, R. Bhagwatkar, M. Biloš, H. Ghonia, N. Hassen, A. Schneider, S. Garg, A. Drouin, N. Chapados, Y. Nevmyvaka, and I. Rish. Lag-Llama: Towards foundation models for time series forecasting. In R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models (NeurIPS 2023 Workshop), 2023. URLhttps://openreview.net/forum?id= jYluzCLFDM. S. Shen, V. Van Beek, and A. Iosup. Statistical characterization of business-critical workloads hosted in cloud datacenters. In 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pages 465–474, 2015. doi: 10.1109/CCGrid.2015.60. X. Shi, S. Wang, Y. Nie, D. Li, Z. Ye, Q. Wen, and M. Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 34635–34667, 2025. URLhttps://proceedings.iclr.c/ paper_files/paper/2025/file/558d48c1f08675daa636e09bfe94a89e-Paper-Conference.pdf. S. Smyl. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. Inter- national Journal of Forecasting, 36(1):75–85, 2020. ISSN 0169-2070. URLhttps://w.sciencedirect. com/science/article/pii/S0169207019301153. A. A. Syntetos, J. E. Boylan, and J. D. Croston. On the categorization of demand patterns. Journal of the Operational Research Society, 56(5):495–503, 2005. URL https://doi.org/10.1057/palgrave.jors.2601841. J. W. Taylor. A quantile regression neural network approach to estimating the conditional density of multiperiod returns. Journal of Forecasting, 19(4):299–311, 2000. URLhttps://onlinelibrary.wiley.com/doi/ abs/10.1002/1099-131X%28200007%2919%3A4%3C299%3A%3AAID-FOR775%3E3.0.CO%3B2-V. A. Trindade. ElectricityLoadDiagrams20112014 dataset. UCI Machine Learning Repository, 2015. URL https://archive.ics.uci.edu/ml/datasets/electricityloaddiagrams20112014. E. Urettini, D. Atzeni, R. J. Ramjattan, and A. Carta. Gas-norm: Score-driven adaptive normalization for non- stationary time series forecasting in deep learning. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM ’24, page 2282–2291. ACM, Oct. 2024. doi: 10.1145/ 3627673.3679822. URL http://dx.doi.org/10.1145/3627673.3679822. S.-M. Vilhes, G. Gasso, and M. Z. Alaya. Does normalization choice matter for causal large time-series models?, 2026. URLhttps://arxiv.org/abs/2606.09954. ICLR 2026 Workshop on Time Series in the Age of Large Models (TSALM). J. Wang, J. Jiang, W. Jiang, C. Li, and W. X. Zhao. Libcity: An open library for traffic prediction. In Proceedings of the 29th International Conference on Advances in Geographic Information Systems, SIGSPATIAL ’21, page 145–148, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450386647. doi: 10.1145/3474717.3483923. URL https://doi.org/10.1145/3474717.3483923. X. Wang, T. Zhou, J. Gao, B. Ding, and J. Zhou. Output scaling: Yinglong-delayed chain of thought in a large pretrained time series forecasting model, 2025. URL https://arxiv.org/abs/2506.11029. 14 Y. Wen, W. M. Gifford, C. Reddy, L. M. Nguyen, J. Kalagnanam, and A. A. Julius. Revisiting the generic transformer: Deconstructing a strong baseline for time series foundation models, 2026. URLhttps://arxiv. org/abs/2602.06909. M. Wolff, M. Li, R. K. Selvam, H. Zhu, K. G. Olivares, R. Ma, A. Katoch, S. Ramasubramanian, M. Cao, R. Bandarra, R. Gopalsamy, S. L. Vattiata, S. Yang, and M. W. Mahoney. SPADE-S: A sparsity-robust foundational forecaster. In KDD 2025 Workshop on AI for Supply Chain: Today and Future, 2025. URL https://openreview.net/forum?id=HQWXh2BQTv. G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo. Unified training of universal time series forecasting transformers. In R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 53140–53164. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/woo24a.html. H. Wu, J. Xu, J. Wang, and M. Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 22419–22430. Curran Associates, Inc., 2021.URLhttps://proceedings.neurips.c/paper_files/paper/2021/file/ bcc0d400288793e8bdcd7c19a8ac0c2b-Paper.pdf. H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long. TimesNet: Temporal 2D-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=ju_Uqw384Oq. H. Xia, Y. Liu, Y. Hu, P. Liu, and Z. Dong. Into the orbit for time series: Training regimes for foundation models, 2026. URL https://arxiv.org/abs/2608.13262. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021. 15 A Scale-Invariance and Contamination in the TSFM Open-Source Ecosystem Training loss space is a fundamental design choice that directly affects what a model learns, yet it is not consistently documented in papers or codebases. Tables 1 and 2 survey the loss space used by prominent TSFMs and open- source forecasting codebases respectively. Across the 24 models in the survey, we distinguish explicit reporting from conclusions drawn indirectly from the available sources. The papers of 9 models explicitly specify the loss space. For 7 others, the descriptions of normalization, outputs, and the objective suggest a likely loss space without establishing it explicitly. Inspecting released code provides a classification for a further 5 models, leaving 3 unresolved. Under these classifications, 7 models use SCALEIN. The survey thus identifies both scale-contaminated and scale-invariant training in current practice, with substantial gaps in how these choices are reported. We classify each model from its paper and its public code separately. A paper counts as explicit if it states the loss space in text, or if its equations define the normalized or denormalized quantities that enter the loss. A paper counts as inferred if it gives no such statement, but its description of the normalization, the model outputs, and the loss lets us trace the target from the input to the loss and infer the loss space with high likelihood. We note, that citing RevIN [Kim et al., 2022] or writing the loss on a generic targetyis not enough on its own to infer either SCALEIN or SCALECON. In code column, we mark TSFMs that released code which allows to determine where the loss is computed, and we flag cases where this follows from model, fine-tuning, or inference code rather than a released pretraining loop. Furthermore, prominent open-source codebases do not explicitly document loss space choice, nor support user specification of the loss space. The lack of documentation raises reproducibility concerns because practitioners who train, fine-tune or extend models may inadvertently change the loss space without realizing it, producing results that are not directly comparable to the original. More broadly, without awareness of the implications, practitioners may simply propagate the use of scale-contaminated losses, which we show has consequences for training convergence and downstream accuracy in cross-frequency training scenarios. We hypothesize that the lack of proper loss space reporting is a result of the widespread adoption of instance normalization schemes such as RevIN, which (by default) denormalize predictions before returning them. However, as we show in this work with both theoretical and empirical evidence, loss space selection is an important design choice. B Scale-Contamination and Scale-Invariance Proofs Proof of Lemma 3.1. For one series contribution, the observed and predicted targets on the original scale are y t+h = bz t+h + a and ˆy t+h = bˆz t+h + a. Their residual is therefore y t+h − ˆy t+h = b (z t+h − ˆz t+h ).(13) Substituting Equation 13 into the scale-contaminated objective and applyingφ(br) = b p φ(r)at every horizon step gives L contam = 1 H H X h=1 φ (b (z t+h − ˆz t+h ))(14) = b p H H X h=1 φ (z t+h − ˆz t+h )(15) = b p L inv .(16) The scaling offsetacancels, while the scaling denominator remains as the multiplicative factorb p . The statistics aandbdepend only on the observed history, so they are constant with respect to the model parametersθ. Differentiating the loss identity therefore yields ∇ θ L contam = b p ∇ θ L inv .(17) Whenφis nondifferentiable, multiplication by the positive constantb p scales every valid subgradient by the same factor. For a mini-batchB, linearity of summation then gives ∇ θ L contam,B = 1 |B| X i∈B b p i ∇ θ L inv,i ,(18) which is Equation 10.□ 16 Proof of Theorem 3.2. Consider multiplying every observation of one series by an arbitrary constantc > 0. Writey (c) s = cy s for the rescaled observation. Scale equivariance givesa (c) = caandb (c) = cb, so every scaled context and target value satisfies z (c) s = y (c) s − a (c) b (c) = cy s − ca cb = y s − a b = z s .(19) The model consequently receives the same scaled context and produces the same scaled prediction for everyθ, ˆ z (c) t+1:t+H = f θ z (c) t−L+1:t = f θ (z t−L+1:t ) = ˆ z t+1:t+H .(20) Equations 19 and 20 make the scale-invariant objective identical before and after the rescaling, L (c) inv (θ) = 1 H H X h=1 φ z (c) t+h − ˆz (c) t+h (21) = 1 H H X h=1 φ (z t+h − ˆz t+h )(22) =L inv (θ).(23) Because this equality holds as a function ofθ, the parameter gradients are also identical, ∇ θ L (c) inv (θ) =∇ θ L inv (θ).(24) The rescaling constants may differ across series because Equation 24 holds for everyc > 0separately. If seriesiis multiplied by c i > 0, the scale-invariant mini-batch gradient therefore satisfies ∇ θ L (c) inv,B = 1 |B| X i∈B ∇ θ L (c i ) inv,i = 1 |B| X i∈B ∇ θ L inv,i =∇ θ L inv,B .(25) For contrast, the same constant leaks directly into a scale-contaminated gradient. After rescaling, the scaling denominator in the inverse transform is b (c) = cb, so Lemma 3.1 gives ∇ θ L (c) contam = (b (c) ) p ∇ θ L (c) inv (26) = (cb) p ∇ θ L inv (27) = c p ∇ θ L contam .(28) Thus SCALEIN cancelscbefore the loss is evaluated, while a scale-contaminated loss multiplies the same series contribution by c p . For the same independent rescalings, the scale-contaminated mini-batch gradient becomes ∇ θ L (c) contam,B = 1 |B| X i∈B (c i b i ) p ∇ θ L inv,i ,(29) Comparing Equations 25 and 29 shows that only the scale-contaminated update changes with the chosenc i values. Fix the initialization, mini-batch sequence, optimizer state, and optimizer randomness as stated in the theorem. At the first step, Equation 25 gives both scale-invariant runs the same gradient, so the optimizer produces the same next parameters and state. Induction over training steps proves that the complete scale-invariant optimization trajectory is unchanged.□ B.1 Extensions to Multivariate and Quantile Forecasting Multivariate forecasting.Suppose the target hasDchannels and channeljuses scaling offseta j and denominator b j . For a loss φ homogeneous of degree p, the scale-invariant and scale-contaminated objectives are L multi inv = 1 HD H X h=1 D X j=1 φ (z t+h,j − ˆz t+h,j ),(30) L multi contam = 1 HD H X h=1 D X j=1 b p j φ (z t+h,j − ˆz t+h,j ).(31) The derivation applies Lemma 3.1 separately to each channel. Multiplying channeljby anyc j > 0leaves its scaled values unchanged under a channel-wise scale-equivariant scaling method, so the proof of Theorem 3.2 applies to the complete scaled input and target tensors. Cross-channel dependencies inside the forecasting model do not affect this conclusion because the model receives the same scaled tensor after rescaling. 17 Quantile forecasting.LetQbe a set of quantile levels and letˆz (q) t+h be the scaled prediction for levelq. Writing the pinball loss as QL q (r) = q(r) + + (1− q)(−r) + , the quantile objective is L quant inv = 1 H|Q| H X h=1 X q∈Q QL q z t+h − ˆz (q) t+h .(32) This objective averages pinball loss across allHhorizon steps and all requested quantile levels, so the model learns the quantile forecasts jointly. For everyb > 0, pinball loss satisfiesQL q (br) = bQL q (r). The samebapplies to every quantile level of a series, which gives L quant contam = bL quant inv , ∇ θ L quant contam = b∇ θ L quant inv .(33) Computing pinball loss on scaled targets removes this factor jointly across all quantile levels, so Theorem 3.2 applies without changing the model or the quantile set. B.2 Nonlinear Scaling in Chronos2 Writeu = arcsinh((y− a)/b)and letˆube the model output. Reversing the transform givesˆy = b sinh(ˆu) + a, so φ(y− ˆy) = b p φ sinh(u)− sinh(ˆu) .(34) Theb p factor still weights the loss and parameter gradient, but the remaining loss differs fromφ(u− ˆu)because of the nonlinear inverse. This comparison therefore changes residual geometry as well as scale weighting. Under scale-equivariant statistics, independently rescaling a series leavesuunchanged, so training on transformed targets retains scale invariance. Fixed numerical stabilizers can qualify this invariance as discussed in Section 3. C Algorithm Algorithm 1 contrasts the two training paradigms. Both begin identically: a scaling offsetaand denominatorbare computed from the context window, the context is normalized, and the model predicts in scaled space. The two approaches diverge only in how the loss is computed. In scale-contaminated training, predictions are denormalized back to the original scale before the loss is applied, coupling the gradient magnitude tob p and thereby making optimization sensitive to the absolute scale of each series. In scale-invariant training, the targets are instead normalized to match the prediction space, and the loss is computed entirely in scaled space, making the gradient independent ofb. The key observation is that the difference between the two paradigms reduces to a single step: whether the targets are denormalized predictions or normalized targets are passed to the loss function, a change that amounts to one line of code in practice. Algorithm 1 Scale-SCALECON and scale-invariant (SCALEIN) objectives Require: Context y t−L+1:t , targets y t+1:t+H , model f θ , loss φ, scaling method (a,b) 1: Compute offset a and denominator b from context 2: Scale context: z t−l ← (y t−l − a)/b for l = 0,...,L− 1 3: Predict scaled outputs: ˆ z t+1:t+H ← f θ (z t−L+1:t ) 4: // Scale-SCALECON objective 5: Reverse scaling: ˆy t+h ← bˆz t+h + a for h = 1,...,H 6: L contam ← 1 H P H h=1 φ (y t+h − ˆy t+h )▷ Loss in original scale; gradient∝ b p 7: // Scale-invariant (SCALEIN) objective 8: Scale targets: z t+h ← (y t+h − a)/b for h = 1,...,H 9: L inv ← 1 H P H h=1 φ (z t+h − ˆz t+h )▷ Loss in scaled space; gradient invariant to b 10: Backpropagate chosen loss and updateθ 18 D Dataset details D.1 M-series Benchmark Details 1975-06-241978-03-201980-12-141983-09-101986-06-06 Date 5.0 2.5 0.0 2.5 5.0 7.5 10.0 12.5 Cross Validation M1 Q id=T1 TestValidationTrain Figure 6: Example time series from the M1 competition dataset. The train and validation sets consist of all observations preceding the first dotted line. The cross validation test set is comprised of a set of forecast creation dates with lengths defined in Table 5. Table 5: Summary of forecasting datasets used in our empirical study. FrequencySeasonalityHorizonSeriesMin LengthMax Length% Erratic M1 Monthly1218617481500 Quarterly48203181140 Yearly1618115580 M3 Other48174711040 Monthly12181428661442 Quarterly4875624721 Yearly16645204710 M4 Hourly2448414748100817 Daily1144,22710799332 Weekly11335993261016 Monthly121848,0006028126 Quarterly4824,0002487411 Yearly1623,0001984118 Tourism Monthly12183669133351 Quarterly484273013039 Yearly16518114723 The earlyM1competition [Makridakis et al., 1982] competition features 1,001 demographic, industrial, and economic time series spanning varying frequencies with lengths from 9 to 132 observations. Its key finding, that simple methods like ETS [Holt, 2004] often outperform complex approaches, shaped a legacy emphasizing accuracy, automation, and overfitting caution while shifting focus from traditional forecasting. Held two decades afterM1, theM3competition [Makridakis and Hibon, 2000] featured 3,003 positive-valued time series across business, demographic, financial, and economic domains with lengths from 14 to 126 observations. Reinforcing the trend that simple methods often outperform complex alternatives, the Theta method [Hyndman and Billah, 2003] emerged as the top-performing approach. TheM4competition [Makridakis et al., 2020] massively scaled up size and diversity to 100,000 time series across six frequencies and multiple domains. Expanding evaluation beyond point forecasts to include prediction intervals, it featured 18% erratic series [Syntetos et al., 2005] and marked the first time a neural model, ESRNN [Smyl, 2020], outperformed traditional methods. TheTourismdataset [Athanasopoulos et al., 2011] comprises 1,311 monthly, quarterly, and yearly series designed to evaluate demand forecasting. It used Mean Absolute Scaled Error (MASE) for scaled point forecasts and evaluated forecast intervals, with 36% of series classified as erratic or intermittent, makingS.Naivedifficult to beat annually. 19 D.2 GIFT-eval Benchmark Details Table 6: Summary of GiftEval forecasting datasets used in our empirical study. DatasetSourceDomainFrequency# SeriesAvgMinMax# Obs Jena WeatherAutoformer [Wu et al., 2021]Nature10T152,70452,70452,70452,704 Jena WeatherAutoformer [Wu et al., 2021]NatureH18,7848,7848,7848,784 Jena WeatherAutoformer [Wu et al., 2021]NatureD1366366366366 BizITObs - ApplicationAutoMixer [Palaskar et al., 2024]Web/CloudOps10S18,8348,8348,8348,834 BizITObs - ServiceAutoMixer [Palaskar et al., 2024]Web/CloudOps10S218,8358,8358,835185,535 BizITObs - L2CAutoMixer [Palaskar et al., 2024]Web/CloudOps5T131,96831,96831,96831,968 BizITObs - L2CAutoMixer [Palaskar et al., 2024]Web/CloudOpsH12,6642,6642,6642,664 Bitbrains - Fast StorageGrid Workloads Archive [Shen et al., 2015]Web/CloudOps5T1,2508,6408,6408,64010,800,000 Bitbrains - Fast StorageGrid Workloads Archive [Shen et al., 2015]Web/CloudOpsH1,250721721721901,250 Bitbrains - rndGrid Workloads Archive [Shen et al., 2015]Web/CloudOps5T5008,6408,6408,6404,320,000 Bitbrains - rndGrid Workloads Archive [Shen et al., 2015]Web/CloudOpsH500720720720360,000 RestaurantRecruit Rest. Comp. [Howard et al., 2017]SalesD80735867478289,303 ETT1Informer [Zhou et al., 2021]Energy15T169,68069,68069,68069,680 ETT1Informer [Zhou et al., 2021]EnergyH117,42017,42017,42017,420 ETT1Informer [Zhou et al., 2021]EnergyD1725725725725 ETT1Informer [Zhou et al., 2021]EnergyW-THU1103103103103 ETT2Informer [Zhou et al., 2021]Energy15T169,68069,68069,68069,680 ETT2Informer [Zhou et al., 2021]EnergyH117,42017,42017,42017,420 ETT2Informer [Zhou et al., 2021]EnergyD1725725725725 ETT2Informer [Zhou et al., 2021]EnergyW-THU1103103103103 Loop SeattleLibCity [Wang et al., 2021]Transport5T323105,120105,120105,12033,953,760 Loop SeattleLibCity [Wang et al., 2021]TransportH3238,7608,7608,7602,829,480 Loop SeattleLibCity [Wang et al., 2021]TransportD323365365365117,895 SZ-TaxiLibCity [Wang et al., 2021]Transport15T1562,9762,9762,976464,256 SZ-TaxiLibCity [Wang et al., 2021]TransportH156744744744116,064 M_DENSELibCity [Wang et al., 2021]TransportH3017,52017,52017,520525,600 M_DENSELibCity [Wang et al., 2021]TransportD3073073073021,900 SolarLSTNet [Lai et al., 2018]Energy10T13752,56052,56052,5607,200,720 SolarLSTNet [Lai et al., 2018]EnergyH1378,7608,7608,7601,200,120 SolarLSTNet [Lai et al., 2018]EnergyD13736536536550,005 SolarLSTNet [Lai et al., 2018]EnergyW-FRI1375252527,124 Hierarchical SalesMancuso et al. [2021]SalesD1181,8251,8251,825215,350 Hierarchical SalesMancuso et al. [2021]SalesW-WED11826026026030,680 M4 YearlyMonash [Godahewa et al., 2021]Econ/FinA-DEC22,9743719284845,109 M4 QuarterlyMonash [Godahewa et al., 2021]Econ/FinQ-DEC24,000100248742,406,108 M4 MonthlyMonash [Godahewa et al., 2021]Econ/FinM48,000234602,81211,246,411 M4 WeeklyMonash [Godahewa et al., 2021]Econ/FinW-SUN3591,035932,610371,579 M4 DailyMonash [Godahewa et al., 2021]Econ/FinD4,2272,3711079,93310,023,836 M4 HourlyMonash [Godahewa et al., 2021]Econ/FinH4149027481,008373,372 HospitalMonash [Godahewa et al., 2021]HealthcareM76784848464,428 COVID DeathsMonash [Godahewa et al., 2021]HealthcareD26621221221256,392 US BirthsMonash [Godahewa et al., 2021]HealthcareD17,3057,3057,3057,305 US BirthsMonash [Godahewa et al., 2021]HealthcareW-TUE11,0431,0431,0431,043 US BirthsMonash [Godahewa et al., 2021]HealthcareM1240240240240 SaugeenMonash [Godahewa et al., 2021]NatureD123,74123,74123,74123,741 SaugeenMonash [Godahewa et al., 2021]NatureW-THU13,3913,3913,3913,391 SaugeenMonash [Godahewa et al., 2021]NatureM1780780780— Temperature RainMonash [Godahewa et al., 2021]NatureD32,072725725725780 KDD Cup 2018Monash [Godahewa et al., 2021]NatureH27010,8989,50410,9202,942,364 KDD Cup 2018Monash [Godahewa et al., 2021]NatureD270455396455122,791 Car PartsMonash [Godahewa et al., 2021]SalesM2,674515151136,374 ElectricityUCI ML Archive [Trindade, 2015]Energy15T370140,256140,256140,25651,894,720 ElectricityUCI ML Archive [Trindade, 2015]EnergyH37035,06435,06435,06412,973,680 ElectricityUCI ML Archive [Trindade, 2015]EnergyD3701,4611,4611,461540,570 ElectricityUCI ML Archive [Trindade, 2015]EnergyW-FRI37020820820876,960 The GiftEval [Aksu et al., 2024] benchmark leverages 10 distinct data sources to construct a comprehensive testing suite. This incorporates the Jena Weather collection from Autoformer [Wu et al., 2021], business and IT infrastructure observability data AutoMixer [Palaskar et al., 2024], cloud workloads sourced from the Grid Workloads Archive via Bitbrains [Shen et al., 2015], temperature profiles from Informer [Zhou et al., 2021], solar generation series from LSTNet [Lai et al., 2018], sales histories [Mancuso et al., 2021], recruit restaurant visitors [Howard et al., 2017], urban traffic logs from LibCity [Wang et al., 2021], selected non-overlapping subsets from the Monash repository [Godahewa et al., 2021], and smart-meter electricity tracking from the UCI Machine Learning Archive [Trindade, 2015]. Summary statistics for each source including frequencies, prediction lengths, variate counts, and scale, are detailed in Table 6. 20 E Controlled Synthetic Data Experiment Details E.1 Reproducibility Details Data. We generate 3 noiseless sinusoidal sources. For source k and training phase s, y (k,s) t = 1 + √ 2v k sin 2πf k t 32 + φ k + 2πs 8 ,(35) where(v k ,f k ,φ k )is(1, 1, 0),(10, 2,π/3), or(100, 3, 2π/3). Each 64-point context has population variancev k , giving scaling denominators in the approximate ratio1 : √ 10 : 10and MSE multipliersb 2 in the ratio1 : 10 : 100. Each source has8training series of length2,048. We use stride-1windows with context length64and horizon 16. Validation contains512fixed windows per source sampled from4series on a phase grid shifted byπ/8from training. Model and scaling. The univariate patch Transformer uses8nonoverlapping patches of length8, embedding dimension64,2encoder layers,4attention heads, feedforward width128, dropout0, and a direct linear head for the16-step horizon. Each context is scaled by its sample mean and sample standard deviation withε = 10 −5 . We compute MSE either on scaled targets for the scale-invariant condition or after reversing the affine transform for the scale-contaminated condition. Optimization. We train for30,000updates with batch size24(8windows per source) and seeds0, 1, 2, 3, 4. SGD uses learning rate10 −3 , momentum0, and weight decay0. Adam uses the same learning rate with (β 1 ,β 2 ) = (0.9, 0.999),ε = 10 −8 , and weight decay0. Learning rates are constant. Within each seed, all conditions share the initialization and sampled batches. Controls and evaluation.The equal-scale control sets all amplitudes to1. The gradient-matched control rescales the full parameter gradient to unit Euclidean norm after every backward pass. We report validation nMSE on scaled targets as the mean±1standard deviation over5seeds. Evaluation occurs every5updates through step200, every 50 through step 1,000, and every 1,000 thereafter. E.2 Scale-invariance Comparison Under Adam A natural objection to the controlled synthetic experiment of Section 4.1 is that an adaptive optimizer should absorb theb p factor. Adam [Kingma and Ba, 2015] is invariant to a global rescaling of the loss: scaling every gradient byc scales the first moment bycand the second byc 2 , leaving the update unchanged. If theb p factor applied uniformly to the objective, Adam would cancel it. It does not apply uniformly. Each source enters the batch gradient weighted by its ownb p , and the mean reduction fixes those relative weights before the optimizer acts, so Adam rescales the mixture without changing its composition. We repeat the synthetic loss-space comparison of Figures 3a and 3b, with Adam in place of SGD, keeping all other knobs identical. Adam normalizes each coordinate by the running root-mean-square of its own gradient. This cancels the global magnitude inflation of the scale-contaminated gradient, so scale-contaminated training no longer risks divergence and converges faster overall, but it does not change which source dominates training, and the step-0 per-source gradient ratio is identical to SGD. Under the scale-contaminated loss the highest-scale source reachesnMSE < 0.1within∼10steps (Figure 7), the next within∼20, and the lowest within∼105. The time-to-threshold gap is down to∼10×from the∼50×under SGD, but it remains clearly scale-ordered. Under the scale-invariant loss all three sources reach the threshold together in about10steps. The equal-scale and gradient-norm-matched controls (Figure 8), as under SGD, confirm the spread is not an artifact of a confounding factor. Together with the gradient-norm-matched control in the main text, this places the effect in the composition of the batch gradient rather than in its magnitude, which is what the learning rate, gradient clipping, and adaptive preconditioning all act on. 21 0255075100125150175200 step 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 nMSE Normalized-space loss var1 var10 var100 global 0255075100125150175200 step Original-space loss Figure 7: Adam, per-source nMSE over the first200steps (linear axis, mean over 5 seeds with±1std bands; dashed grey = global nMSE). Left: under the scale-invariant loss all three sources converge together. Right: under the scale-contaminated loss the curves spread out in scale order (high-scale first, low-scale last), the same disparity SGD shows but compressed into the first∼150 steps. 0255075100125150175200 step 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 nMSE Original + equal variance var1 var10 var100 global 0255075100125150175200 step Original + grad-norm matched Figure 8: Adam control experiments (first200steps, linear axis, mean±1std over 5 seeds). Left: on equal-scale data the scale-contaminated spread collapses to the scale-invariant baseline. Right: under gradient-norm-matched scale-contaminated training the scale-ordered spread persists. var1var10var100 category (increasing variance) 10 −2 10 −1 10 0 ‖∂ /∂ ̂ z ‖ (a) Per-source gradient magnitude at initialization. 0100200300400500 step 0.0 0.2 0.4 0.6 0.8 1.0 global nMSE Global convergence Normalized-space Original-space Equal variance Gradient-norm matched (b) Global nMSE (first 500 steps). Figure 9: Adam, gradient magnitude and global convergence (mean±1std over 5 seeds). The initialization gradient ratio is unchanged from SGD and scales asb p under the scale-contaminated loss. Adam rescales coordinate magnitude but not the scale-driven ordering of convergence. 22 var1 step 0step 10step 50step 100step 500step 30k var10 var100 (a) Scale-invariant loss (SCALEIN) var1 step 0step 10step 50step 100step 500step 30k var10 var100 (b) Scale-SCALECON loss Figure 10: Adam, qualitative per-source forecasts at training steps 0, 10, 50, 100, 500, and 30,000. 23 F Scale-Swap Intervention Experiment F.1 Reproducibility Details Data. We use the8datasets listed in Section 4.2, represented as univariate series of length128. We split across complete series, assigning a fixed random80%of the series to training and the remaining20%to validation. No series appears in both splits. We form stride-1windows only after this split, using context length64and horizon16 and removing windows with nonfinite or constant contexts. Each window is scaled to zero context mean and unit context standard deviation, then multiplied by its assignedb ∈ 1, 10. Validation uses512fixed windows per dataset. Assignments. Each configuration assigns4datasets to each scale. Its complement reverses all8assignments. We use15complementary pairs, giving30configurations in which every dataset appears15times at each scale. Each pair of data sets shares the larger scale in 6 or 7 configurations. Model and optimization. The patch Transformer uses8nonoverlapping patches of length8, embedding dimension128,4encoder layers,8attention heads, feedforward width256, dropout0, and a direct linear head for the16-step horizon. Scale each context using its sample mean and sample standard deviation withε = 10 −5 . We train for30,000SGD updates with batch size32(4windows per dataset), momentum0, weight decay0, and seeds 0, 1, 2, 3, 4. Within each seed and assignment pair, conditions share the initialization and sampled batches. Learning-rate control. Both conditions use constant learning rates. The scale-invariant condition uses10 −4 . Under scale-contaminated MSE,b = 10multiplies a window’s gradient by100, whileb = 1leaves it unchanged. Because every batch contains equal numbers from the2scales, the nominal average multiplier is(1+100)/2 = 50.5. We therefore use10 −4 /50.5 = 1.9802× 10 −6 for the scale-contaminated condition. This keeps the average update magnitude comparable without removing the relative 100 : 1 weighting between scales. Evaluation. We report validation nMSE on scaled targets. Evaluation occurs every5updates through step200, every50through step1,000, and every1,000thereafter. The trajectory summary and paired test are defined below. F.2 Convergence Statistic and Test Letτ 1 <· < τ J be the training steps at which we evaluate held-out nMSE. For datasetD (k) , scaleb, complemen- tary assignment pairc, seeds, and checkpointj, write this error ase (k) b,c,s,j . We summarize the complete validation trajectory by integrating the straight lines between consecutive checkpoints and dividing by the training duration, A (k) b,c = 1 S S X s=1 1 τ J − τ 1 J−1 X j=1 τ j+1 − τ j 2 e (k) b,c,s,j + e (k) b,c,s,j+1 . (36) This weighted average remains on the nMSE scale. It gives more weight to checkpoint intervals that cover more training steps. Because all runs share the same checkpoints, it compares their full trajectories on the same basis. Each complementary assignment pair places every dataset atb=1in one run and atb=10in the other. For each dataset, we average over the C = 15 paired assignments, ̄ A (k) b = 1 C C X c=1 A (k) b,c ,∆ k = ̄ A (k) 1 − ̄ A (k) 10 .(37) A positive∆ k means datasetD (k) has lower validation error over training when assigned the larger scale. The aggregate effect is ̄ ∆ = 1 K K X k=1 ∆ k .(38) The one-sided pairedt-test uses theK = 8dataset-level differencesd k = ∆ k as its statistical units. With sample mean ̄ d and sample standard deviation s d , the test statistic is t = ̄ d s d / √ K , s 2 d = 1 K− 1 K X k=1 (d k − ̄ d) 2 .(39) 24 The null hypothesis isE[d k ] ≤ 0and the one-sided alternative isE[d k ] > 0. Under the scale-invariant loss, the mean trajectory-level errors across datasets are1.964752atb=1and1.964783atb=10, a relative difference below0.002%. For the scale-contaminated loss, the test givest 7 =2.013andp=0.042. We compute the reported percentage from its means across datasets at each scale, ̄ A b = 1 K K X k=1 ̄ A (k) b , ̄ A 1 = 2.5925, ̄ A 10 = 1.8990,(40) which gives 100× ̄ A 1 − ̄ A 10 ̄ A 1 = 26.8%.(41) We repeat the trajectory and paired-effect calculations separately for the scale-invariant and scale-contaminated losses. Figure 11 disaggregates the aggregate scale-swap trajectory in Figure 5a. The2scales overlap under the scale- invariant loss for every dataset. Under the scale-contaminated loss, the inflated copy converges faster in7of8 datasets. Wind Farms is the only exception. These are the same dataset-level trajectories used for the paired analysis in Section 4.2. 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 nMSE electricity 0.0 0.5 1.0 1.5 2.0 2.5 traffic 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 solar 10 minutes 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 taxi 30min 05k10k15k20k25k30k step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 nMSE wind farms without missing 05k10k15k20k25k30k step 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 pedestrian counts 05k10k15k20k25k30k step 0 1 2 3 4 5 kdd cup 2018 without missing 05k10k15k20k25k30k step 0 1 2 3 4 5 6 7 8 fred md Scale-invariant loss, b= 1 Scale-invariant loss, b= 10 LR-adjusted scale-contaminated loss, b= 1 LR-adjusted scale-contaminated loss, b= 10 Figure 11: Dataset-level trajectories under scale-invariant loss and learning-rate-adjusted scale-contaminated loss. Under scale-invariant loss, theb = 1andb = 10trajectories overlap because the loss is computed on scaled targets. Lines and bands show the mean and 95% Student-tinterval across15assignment contexts after averaging5seeds within each context. 25 G TSFM Pretraining Details G.1 Reproducibility Details Corpus and windows. We pretrain on GiftEvalPretrain together with the training prefixes of M1, M3, M4, Tourism, and Favorita. All benchmark validation and test observations are excluded. The resulting corpus contains 73datasets with usable windows. Before constructing windows, a fixed series-level split assigns90%of the series in each dataset to training and10%to validation, so no series appears in both splits. The maximum context and prediction lengths are512and128, with minima of64and8. For a series of lengthn, we set H = min(128, max(8,⌊n/5⌋))andL = min(512,n− H). IfL < 64, we instead setL = 64andH = n− 64. Series shorter than72observations are excluded. We use stride512, retain windows with at least90%observed values, and pad shorter windows to the maximum shape with masks. Sampling. Each training example first samples a dataset uniformly, then a series uniformly within that dataset, and finally a window uniformly within that series. Thus, datasets and series with more available windows do not automatically receive more updates. The paired loss conditions use the same fixed sequence of sampled batches. Models and objectives.All four architectures start from random weights. The two foundation models use their Small configurations.Chronos2Small uses patch size16, hidden dimension512, feed-forward dimension2048, 6layers,8attention heads, and dropout0.1. It applies its native context standardization and arcsinh transform, then optimizes its native CRPS-style objective over13quantiles in0.01, 0.05, 0.1, 0.2,..., 0.9, 0.95, 0.99. Moirai-2.0Small uses patch size16, hidden dimension384, feed-forward dimension1024,6layers, and dropout 0.1. It applies its native context scaling and optimizes mean quantile loss over the9deciles from0.1to0.9on up to64target steps.TimesFM2.5is reduced to49.6M parameters from the released200M definition, keeping its head dimension of80and cutting width to800and depth to12layers. It keeps the input and output patch lengths of32and128, so a single forward pass covers the128-step horizon and no autoregressive decoding is needed. Its scaling is the per-patch expanding-window statistic thatTimesFM2.5introduced, where patchiis scaled by the masked running mean and standard deviation of every observed point up to and including itself. We train the point head with MSE, as the released model does, and drop the separate1024-step quantile head, which is unused at this horizon.PatchTSTfollows its original supervised configuration at1.5M parameters, with patch length16, stride8,3encoder layers,16heads, hidden dimension128, and dropout0.2. It scales by RevIN with the affine parameters disabled, which is the setting its own released code defaults to, and optimizes MSE. Within each model, scale-invariant training computes the native objective on scaled targets, while scale-contaminated training reverses the model’s scaling before computing the same objective. Everything else is held fixed. The four therefore differ in where their scaling statistics come from, which is what lets the loss-space comparison be read as independent of that choice. Optimization and validation. We train for80,000updates at batch size512with seeds0, 1, 2, 3. AdamW uses a constant learning rate of10 −4 , weight decay0.01, and global gradient clipping at norm1.0. Conditions within each model and seed share their initialization, batches, and optimizer settings. Every250updates, the training curves are evaluated on a fixed set of 64 validation windows per dataset. Test splits and inference. M1, M3, and Tourism reserve the final official horizon, while M4 uses the released train/test files. Favorita uses the final16days of series ending on August 15, 2017, with missing sales filled by zero. For each GIFT-Eval short-term configuration, we useW = min(20, max(1,⌈0.1L min /H⌉))non-overlapping windows of horizonHfrom each series’ tail, whereL min is the shortest series length in that configuration. Its M4 configurations useW = 1, and multivariate series are evaluated channel-wise. Each origin uses only preceding observations, including observations from earlier windows when available. Models receive the most recent512observations, with shorter histories left-padded and masked, and their outputs are truncated toHwithout autoregressive rollout or benchmark-specific fine-tuning. Metric aggregation. Within each subset, MASE is averaged over valid forecast cases using each case’s full preceding history for the seasonal-naive denominator. WQL pools its numerator and denominator across cases before division, including the factor of two and averaging over supplied quantile levels. Missing targets and cases with undefined metric denominators are excluded from the corresponding metric. Benchmark scores average subset scores weighted by their total forecast-case counts, then average over four seeds. Thus, benchmark WQL is a weighted average of subset ratios. The overlapping supervised test origins in Appendix H constitute a separate evaluation protocol. 26 Compute. All runs were performed with a single NVIDIA H200 GPU, where each 80k-step run took between 3.6 and 6.5 GPU-hours depending on the architecture. Generating Table 3 used 166 GPU-hours in total. G.2 Cross-Dataset Dispersion and Pooled Accuracy During Training Figure 12 complements the median MASE trajectory in Figure 5b for all four architectures. For each seed and checkpoint, it computes the variance across per-dataset MASE values before averaging across the4seeds. Scale-invariant training clearly reduces dispersion forChronos2and modestly reduces it forMoirai-2.0, while the two losses converge to similar dispersion forTimesFM2.5andPatchTST. Low dispersion alone does not imply accurate forecasts, so Figure 13 also reports MASE pooled over all usable validation forecast cases in their natural dataset proportions. Scale-invariant training achieves lower pooled MASE throughout training for every architecture, including the two architectures whose final cross-dataset variance is similar across losses. 020k40k60k80k Training step 0 50 100 150 200 Variance across per-dataset MASE Chronos-2 020k40k60k80k Training step Moirai-2.0 020k40k60k80k Training step TimesFM-2.5 020k40k60k80k Training step PatchTST Scale-invariant (ScaleIn)Scale-contaminated (ScaleCon) Figure 12: Cross-dataset variance of per-dataset MASE over80,000updates for all four architectures. Lines show the mean across4seeds and bands show 95% Student-tintervals. Lower variance indicates more uniform accuracy across datasets, but does not by itself imply lower forecast error. 020k40k60k80k Training step 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Pooled MASE Chronos-2 020k40k60k80k Training step Moirai-2.0 020k40k60k80k Training step TimesFM-2.5 020k40k60k80k Training step PatchTST Scale-invariant (ScaleIn)Scale-contaminated (ScaleCon) Figure 13: Pooled validation MASE over80,000updates for all four architectures. The pooled mean weights usable forecast cases in their natural dataset proportions, unlike the dataset-balanced median in Figure 5b. Lines show the mean across 4 seeds and bands show 95% Student-t intervals. Lower is better. 27 G.3 Benchmark-Frequency Results Tables 7 and 8 give the full breakdown behind Table 3. GIFT-Eval reports some frequencies with multipliers or anchors. We poolA-DECinto Y,Q-DECinto Q,10Sinto S,5T,10T, and15Tinto T, and the5anchored weekly codes into W. Stars mark these pooled rows. The6GIFT-Eval rows derived from M4 duplicate the official M4 evaluation, so the paper does not average the two benchmarks together. Table 7: In-domain MASE by benchmark and frequency after 80,000 updates, means with sample standard deviations (±, below) across 4 seeds. A star marks a base frequency that pools anchored or multiplied GIFT-Eval aliases. SCALEIN uses scale-invariant loss and SCALECON uses scale-contaminated loss. These are the only differences between the two runs of each architecture.∆%reports SCALEIN relative to SCALECON. Green denotes an improvement and red a regression. The better loss is bold within each architecture. References are fitted on the series they forecast and appear behind the double rule. Chronos2 Moirai-2.0 TimesFM2.5 PatchTSTReference BenchmarkFreq.SC.INSC.CON ∆%SC.INSC.CON ∆%SC.INSC.CON ∆%SC.INSC.CON∆%S.Naive M1Y 4.124 ±0.445 6.492 ±0.450-36.5 4.894 ±0.150 5.175 ±0.406-5.4 8.067 ±0.509 7.765 ±0.291+3.9 4.707 ±0.104 7.600 ±0.823-38.14.894 Q 1.827 ±0.056 1.997 ±0.035-8.5 1.814 ±0.056 1.885 ±0.053-3.8 1.943 ±0.042 2.394 ±0.088-18.8 1.963 ±0.078 3.625 ±0.917-45.92.078 M 1.238 ±0.019 1.312 ±0.013-5.6 1.234 ±0.011 1.306 ±0.024-5.5 1.425 ±0.018 1.579 ±0.015-9.8 1.366 ±0.018 2.838 ±1.050-51.91.314 M3Y 3.070 ±0.045 4.085 ±0.241-24.8 3.119 ±0.127 3.643 ±0.171-14.4 4.880 ±0.198 5.309 ±0.188-8.1 3.889 ±0.465 9.529 ±2.353-59.23.154 Q 1.258 ±0.060 1.461 ±0.072-13.9 1.255 ±0.039 1.295 ±0.077-3.1 1.476 ±0.016 1.846 ±0.110-20.1 1.590 ±0.142 3.867 ±1.043-58.9 1.425 M 0.955 ±0.018 1.012 ±0.009-5.6 0.956 ±0.014 1.011 ±0.020-5.4 1.116 ±0.014 1.389 ±0.024-19.6 1.220 ±0.007 2.666 ±0.799-54.2 1.146 M4Y 3.537 ±0.163 4.607 ±0.401-23.2 3.672 ±0.166 3.899 ±0.150-5.8 6.294 ±0.350 7.422 ±0.432-15.2 4.044 ±0.056 7.474 ±0.569-45.9 3.974 Q 1.327 ±0.040 1.553 ±0.038-14.6 1.333 ±0.024 1.360 ±0.027-2.0 1.559 ±0.029 1.956 ±0.015-20.3 1.603 ±0.047 3.039 ±0.697-47.21.602 M 1.123 ±0.035 1.248 ±0.042-10.0 1.131 ±0.028 1.160 ±0.008-2.5 1.294 ±0.024 1.545 ±0.016-16.3 1.378 ±0.065 3.299 ±0.835-58.21.260 W 2.793 ±0.210 4.823 ±0.694-42.1 2.615 ±0.073 3.516 ±0.149-25.6 3.906 ±0.385 5.805 ±0.083-32.7 5.135 ±1.455 8.625 ±0.881-40.52.777 D 3.765 ±0.114 5.502 ±0.406-31.6 3.656 ±0.106 4.354 ±0.243-16.0 4.451 ±0.236 5.743 ±0.029-22.5 5.375 ±0.745 11.379 ±1.590-52.8 3.278 H 1.166 ±0.043 3.175 ±0.698-63.3 1.784 ±0.076 2.302 ±0.098-22.5 3.894 ±1.002 6.283 ±0.832-38.0 3.066 ±0.253 2.786 ±0.179+10.0 1.193 TourismY 3.104 ±0.139 3.654 ±0.367-15.0 3.019 ±0.050 3.173 ±0.117-4.8 5.873 ±0.805 6.250 ±0.512-6.0 3.005 ±0.062 3.844 ±0.126-21.8 3.552 Q 1.868 ±0.015 2.269 ±0.067-17.7 1.875 ±0.022 2.067 ±0.042-9.3 2.363 ±0.065 2.403 ±0.032-1.7 2.182 ±0.084 2.653 ±0.161-17.81.699 M 1.845 ±0.030 2.269 ±0.015-18.7 1.834 ±0.038 2.284 ±0.039-19.7 2.616 ±0.149 3.068 ±0.138-14.7 2.189 ±0.056 2.650 ±0.084-17.41.631 FavoritaD 0.816 ±0.004 0.862 ±0.011-5.4 0.825 ±0.001 0.864 ±0.007-4.5 1.050 ±0.015 1.360 ±0.010-22.8 0.889 ±0.005 0.968 ±0.026-8.21.079 GIFT-EvalY ∗ 3.530 ±0.162 4.593 ±0.402-23.2 3.663 ±0.165 3.888 ±0.150-5.8 6.278 ±0.350 7.386 ±0.435-15.0 4.028 ±0.057 7.451 ±0.569-45.93.966 Q ∗ 1.327 ±0.040 1.553 ±0.038-14.6 1.333 ±0.024 1.360 ±0.027-2.0 1.559 ±0.029 1.956 ±0.015-20.3 1.603 ±0.047 3.039 ±0.697-47.21.602 M 1.124 ±0.034 1.233 ±0.038-8.9 1.119 ±0.025 1.141 ±0.008-1.9 1.288 ±0.021 1.531 ±0.015-15.9 1.357 ±0.060 3.150 ±0.782-56.91.249 W ∗ 1.861 ±0.056 2.332 ±0.168-20.2 1.884 ±0.057 2.032 ±0.034-7.3 2.217 ±0.051 2.752 ±0.070-19.5 2.449 ±0.313 3.920 ±0.384-37.5 1.916 D 1.572 ±0.012 1.798 ±0.021-12.6 1.598 ±0.013 1.763 ±0.016-9.3 2.299 ±0.021 2.870 ±0.023-19.9 2.051 ±0.017 2.497 ±0.108-17.8 2.153 H 1.393 ±0.025 1.650 ±0.038-15.6 1.456 ±0.014 1.677 ±0.008-13.2 1.895 ±0.022 2.571 ±0.084-26.3 1.640 ±0.015 1.857 ±0.057-11.71.572 T ∗ 1.129 ±0.022 1.402 ±0.015-19.4 1.239 ±0.059 1.355 ±0.009-8.5 1.650 ±0.039 2.103 ±0.023-21.6 1.610 ±0.032 1.952 ±0.071-17.5 1.320 S ∗ 2.241 ±0.337 3.568 ±0.443-37.2 2.090 ±0.199 2.595 ±0.135-19.5 2.466 ±0.139 3.807 ±0.121-35.2 3.808 ±0.223 4.733 ±0.262-19.51.272 28 Table 8: In-domain WQL by benchmark and frequency after 80,000 updates, means with sample standard deviations (±, below) across 4 seeds. A star marks a base frequency that pools anchored or multiplied GIFT-Eval aliases. SCALEIN uses scale-invariant loss and SCALECON uses scale-contaminated loss. These are the only differences between the two runs of each architecture.∆%reports SCALEIN relative to SCALECON. Green denotes an improvement and red a regression. The better loss is bold within each architecture. References are fitted on the series they forecast and appear behind the double rule. Chronos2 Moirai-2.0 TimesFM2.5 PatchTSTReference BenchmarkFreq.SC.INSC.CON∆%SC.INSC.CON∆%SC.INSC.CON ∆%SC.INSC.CON ∆%S.Naive M1Y 0.110 ±0.006 0.126 ±0.018-12.8 0.154 ±0.007 0.157 ±0.006-2.1 0.248 ±0.009 0.287 ±0.018-13.6 0.196 ±0.009 0.307 ±0.036-36.10.184 Q 0.065 ±0.008 0.076 ±0.007-13.7 0.094 ±0.004 0.095 ±0.008-1.2 0.115 ±0.013 0.142 ±0.008-18.9 0.118 ±0.009 0.148 ±0.024-20.2 0.117 M 0.162 ±0.016 0.131 ±0.007+23.5 0.207 ±0.024 0.188 ±0.021+10.0 0.227 ±0.006 0.261 ±0.021-12.9 0.272 ±0.015 0.322 ±0.066-15.7 0.150 M3Y 0.101 ±0.001 0.112 ±0.006-10.1 0.132 ±0.004 0.143 ±0.004-7.3 0.218 ±0.009 0.221 ±0.006-1.3 0.170 ±0.010 0.283 ±0.039-40.0 0.118 Q 0.063 ±0.003 0.066 ±0.002-4.8 0.079 ±0.003 0.080 ±0.003-0.5 0.104 ±0.001 0.126 ±0.007-17.7 0.110 ±0.006 0.202 ±0.034-45.4 0.082 M 0.085 ±0.001 0.086 ±0.002-0.5 0.110 ±0.002 0.115 ±0.002-3.8 0.151 ±0.002 0.177 ±0.003-14.6 0.167 ±0.005 0.262 ±0.055-36.40.121 M4Y 0.101 ±0.003 0.119 ±0.005-15.2 0.136 ±0.004 0.134 ±0.004+1.5 0.256 ±0.013 0.287 ±0.015-11.0 0.166 ±0.001 0.258 ±0.005-35.50.137 Q 0.067 ±0.002 0.074 ±0.002-8.3 0.089 ±0.002 0.088 ±0.001+1.1 0.119 ±0.001 0.141 ±0.001-15.2 0.119 ±0.003 0.175 ±0.021-32.00.098 M 0.090 ±0.004 0.088 ±0.001+2.4 0.117 ±0.004 0.111 ±0.002+5.0 0.149 ±0.001 0.167 ±0.001-10.3 0.162 ±0.009 0.258 ±0.039-37.4 0.122 W 0.041 ±0.000 0.055 ±0.004-25.1 0.054 ±0.001 0.061 ±0.003-11.7 0.078 ±0.004 0.100 ±0.001-21.9 0.089 ±0.013 0.125 ±0.007-29.0 0.061 D 0.020 ±0.001 0.029 ±0.002-32.4 0.025 ±0.001 0.030 ±0.002-17.5 0.037 ±0.002 0.048 ±0.000-22.5 0.044 ±0.006 0.079 ±0.007-43.9 0.024 H 0.019 ±0.001 0.023 ±0.001-17.5 0.026 ±0.001 0.029 ±0.004-11.0 0.057 ±0.010 0.076 ±0.009-24.4 0.045 ±0.007 0.048 ±0.003-6.70.038 TourismY 0.115 ±0.010 0.151 ±0.014-23.8 0.153 ±0.005 0.171 ±0.010-10.4 0.332 ±0.029 0.342 ±0.042-3.0 0.178 ±0.012 0.233 ±0.009-23.60.170 Q 0.056 ±0.002 0.077 ±0.006-26.3 0.083 ±0.018 0.094 ±0.006-11.4 0.115 ±0.002 0.120 ±0.003-4.0 0.110 ±0.008 0.150 ±0.017-27.10.098 M 0.077 ±0.004 0.100 ±0.003-22.7 0.102 ±0.003 0.121 ±0.003-15.8 0.208 ±0.018 0.330 ±0.041-36.9 0.153 ±0.007 0.211 ±0.009-27.30.086 FavoritaD 0.297 ±0.004 0.326 ±0.005-9.1 0.387 ±0.002 0.422 ±0.004-8.3 0.638 ±0.011 0.848 ±0.007-24.8 0.540 ±0.004 0.595 ±0.019-9.30.614 GIFT-EvalY ∗ 0.101 ±0.003 0.119 ±0.005-15.2 0.136 ±0.004 0.134 ±0.004+1.5 0.256 ±0.013 0.287 ±0.015-11.0 0.166 ±0.001 0.258 ±0.005-35.50.137 Q ∗ 0.067 ±0.002 0.074 ±0.002-8.3 0.089 ±0.002 0.088 ±0.001+1.1 0.119 ±0.001 0.141 ±0.001-15.2 0.119 ±0.003 0.175 ±0.021-32.00.098 M 0.146 ±0.005 0.132 ±0.002+11.0 0.175 ±0.002 0.160 ±0.001+9.5 0.227 ±0.001 0.251 ±0.002-9.7 0.229 ±0.007 0.317 ±0.037-27.50.194 W ∗ 0.125 ±0.006 0.141 ±0.004-11.8 0.163 ±0.004 0.172 ±0.003-5.0 0.253 ±0.004 0.290 ±0.009-12.9 0.257 ±0.007 0.331 ±0.022-22.20.268 D 0.410 ±0.000 0.468 ±0.007-12.5 0.533 ±0.002 0.592 ±0.003-10.0 0.888 ±0.006 1.088 ±0.008-18.4 0.771 ±0.003 0.847 ±0.028-9.01.182 H 0.247 ±0.014 0.275 ±0.006-10.1 0.339 ±0.012 0.360 ±0.011-5.8 0.496 ±0.017 0.657 ±0.029-24.5 0.426 ±0.011 0.461 ±0.012-7.70.425 T ∗ 0.346 ±0.018 0.398 ±0.009-13.0 0.454 ±0.010 0.481 ±0.005-5.5 0.633 ±0.014 0.718 ±0.012-11.8 0.632 ±0.015 0.732 ±0.026-13.70.868 S ∗ 0.028 ±0.005 0.039 ±0.006-28.9 0.034 ±0.005 0.034 ±0.004-1.0 0.035 ±0.002 0.069 ±0.004-49.9 0.064 ±0.005 0.076 ±0.004-16.4 0.040 29 H Supervised Forecasting Details Data and pooling.The experiment pools eligible M1, M3, M4, and Tourism series by frequency. One model is trained for each of the yearly, quarterly, monthly, weekly, daily, and hourly pools. M1, M3, and Tourism contribute only their available yearly, quarterly, and monthly series, while M4 contributes all six frequencies. The model horizon is the largest official horizon in its pool. The input context is twice that horizon. Table 9 reports the resulting dimensions and eligible population. Table 9: Frequency-specific dimensions and eligible training populations in the supervised experiment. The population is fixed before comparing loss spaces and is shared by the neural and statistical methods. FrequencyHorizonContextEligible series Yearly8167,841 Quarterly81623,239 Monthly244834,183 Weekly1326359 Daily14284,227 Hourly4896414 For a series with official horizonH, the final2H − 1observations form the rolling test region. The immediately preceding model-horizon block is used for validation. A series is eligible only when a complete input context and training horizon remain before validation. Evaluation uses allHrolling origins in the test region. At originj, only observations through the end of that origin’s input context are available to the model, scaler, and fitted references. Loss-space comparison.Both conditions receive inputs under standard scaling whose offsetaand denominator bare estimated from the input context. The scale-invariant loss (SCALEIN) computes MAE between predictions and targets after this scaling. The scale-contaminated loss (SCALECON) applies the same fixed inverse transform before computing MAE in original scale. The scaler has no learnable affine parameters in either condition, so matched runs differ only in the space where MAE is evaluated. Optimization and checkpointing. Every model trains from scratch for exactly20,000optimizer updates. Training does not stop early. Validation is evaluated every250updates, and the best checkpoint is the state with the lowest validation MAE in original scale. All results average seeds0,1,2, and3. Table 10 gives the shared settings. Table 10: Shared supervised training and evaluation settings. SettingValue OptimizerAdam Training updates20,000 Series batch size32 Sampled windows per batch512 Learning rate10 −3 Learning-rate decays3, evenly spaced Validation interval250 updates Early stoppingDisabled Training lossMAE Input scalerStandard scaling from the input context Start paddingDisabled Forecast typePoint forecast Random seeds0, 1, 2, 3 We use NeuralForecast 3.2.1 [Olivares et al., 2022] and retain its architecture defaults except for the shared settings above, the frequency-specific horizon and context. The architecture parameters that materially determine model capacity are listed in Table 11. Metrics and aggregation.MASE is computed for each series-origin forecast using the seasonal-naive in-sample MAE from all history available at that origin. WQL is the ratio of summed median quantile loss to summed absolute targets within each benchmark-frequency pool. For each neural model, metrics are first computed within a seed. The main table macro-averages eligible frequencies within each benchmark and then averages the four seed values. 30 Table 11: Architecture-specific parameters for the supervised experiment. Parameters not shown use the Neural- Forecast 3.2.1 defaults. ArchitectureParameters NHITSThree identity stacks, one block per stack, two hidden layers of 512 units per block, pooling kernels(2, 2, 1), frequency downsampling(4, 2, 1), ReLU activation, and linear interpola- tion. NBEATSIdentity, trend, and seasonality stacks, one block per stack, two hidden layers of 512 units per block, ReLU activation, and no shared block weights. PatchTSTThree encoder layers, 16 attention heads, hidden width 128, feed-forward width 256, patch length 16, stride 8, dropout 0.2, GELU activation, and internal RevIN disabled. TCNKernel size 2, dilations(1, 2, 4, 8, 16), encoder width 128, context width 10, and a two-layer decoder of width 128. LSTMTwo encoder layers of width 128, no encoder dropout, and a two-layer decoder of width 128. Forecasts use the direct decoder rather than recursive generation. Statistical references have no seed dimension.S.Naive,ETS[Holt, 2004], andARIMA[Hyndman and Khandakar, 2008, Hyndman et al., 2026] are refitted at every rolling origin using the same available history and exactly the same eligible series as the neural models. We use the baselines implementation available in StatsForecast 2.1.1 [Garza et al., 2022]. Compute.All runs were performed with a single NVIDIA GeForce RTX 2080 Ti GPU. Generating Table 4 used 51 GPU-hours in total. Benchmark-frequency results.Tables 12 and 13 give the full breakdown underlying the benchmark aggregates in Table 4. The frequency-level results support the same findings that are in the main paper and somewhat strengthen them. SCALEIN lowers MASE in 49 of 75 architecture-benchmark-frequency comparisons and WQL in 43 of 75. The effect is strongest at the lower frequencies. It wins 17 of 20 yearly and 15 of 20 quarterly comparisons on MASE, with corresponding WQL counts of 15 and 14. Monthly results split 8 to 12 for both metrics, while the weekly, daily, and hourly M4 pools are also mixed. The benchmark-level advantage is therefore broad across architectures but concentrated in the yearly and quarterly pools. The detailed tables also show why the statistical references remain important.ETSorARIMAoften leads on M3 and the higher-frequency M4 pools even when SCALEIN improves the matched neural model. 31 Table 12: Best-checkpoint MASE by benchmark and frequency in the supervised experiment. Neural values are mean±sample standard deviation across four seeds. SCALEIN uses scale-invariant MAE and SCALECON uses scale-contaminated MAE, both with the same input-window standard scaling. References are fitted at every rolling origin on the same eligible series. Lower is better, and bold marks the better loss within each matched pair. Architecture Benchmark FrequencySCALEINSCALECONS.Naive ETS ARIMA N-HiTS M1 Y2.368±0.051 2.461±0.1713.0462.4762.632 Q1.859±0.004 1.857±0.012 2.5271.8991.966 M1.251±0.012 1.248±0.009 1.8591.3381.333 M3 Y2.972±0.346 2.934±0.0923.3332.3843.030 Q1.121±0.006 1.127±0.0141.4571.1151.130 M0.863±0.012 0.860±0.009 1.2430.9320.901 M4 Y3.264±0.036 3.324±0.0834.5573.2504.754 Q1.192±0.002 1.199±0.0031.5901.1961.207 M0.917±0.005 0.919±0.0031.3010.9560.938 W2.477±0.051 2.466±0.0552.7992.4712.843 D3.160±0.006 3.167±0.007 3.2073.1783.234 H0.937±0.066 0.966±0.0741.2201.1271.133 Tourism Y3.689±0.112 3.878±0.3462.9563.4604.135 Q1.689±0.010 1.735±0.0471.9241.7011.750 M1.658±0.025 1.647±0.0051.8171.6151.623 N-BEATS M1 Y2.356±0.030 2.386±0.084 3.0462.4762.632 Q1.858±0.018 1.859±0.007 2.5271.8991.966 M1.252±0.005 1.259±0.0121.8591.3381.333 M3 Y3.059±0.146 3.327±0.449 3.3332.3843.030 Q1.120±0.016 1.118±0.0101.4571.1151.130 M0.853±0.006 0.856±0.0041.2430.9320.901 M4 Y3.246±0.034 3.448±0.3004.5573.2504.754 Q1.188±0.001 1.191±0.0021.5901.1961.207 M0.914±0.001 0.915±0.004 1.3010.9560.938 W2.479±0.021 2.497±0.045 2.7992.4712.843 D3.164±0.003 3.164±0.007 3.2073.1783.234 H0.933±0.015 1.001±0.0461.2201.1271.133 Tourism Y3.580±0.116 3.847±0.0732.9563.4604.135 Q1.704±0.007 1.695±0.0141.9241.7011.750 M1.637±0.023 1.635±0.0191.8171.6151.623 PatchTST M1 Y2.167±0.104 2.194±0.1153.0462.4762.632 Q2.056±0.079 2.018±0.0402.5271.8991.966 M1.527±0.025 1.520±0.0331.8591.3381.333 M3 Y6.081±0.790 6.911±1.3523.3332.3843.030 Q1.337±0.021 1.351±0.0301.4571.1151.130 M0.989±0.017 0.988±0.0181.2430.9320.901 M4 Y3.807±0.242 3.867±0.2824.5573.2504.754 Q1.352±0.015 1.341±0.025 1.5901.1961.207 M1.029±0.019 1.027±0.0221.3010.9560.938 W3.929±0.098 3.929±0.2142.7992.4712.843 D4.198±0.145 4.152±0.1893.2073.1783.234 H7.518±1.134 7.118±2.0431.2201.1271.133 Tourism Y3.283±0.500 3.487±0.4832.9563.4604.135 Q3.168±0.087 3.188±0.1421.9241.7011.750 M2.200±0.100 2.204±0.0831.8171.6151.623 TCN M1 Y2.467±0.127 2.606±0.1233.0462.4762.632 Q1.903±0.014 1.936±0.0522.5271.8991.966 M1.452±0.013 1.419±0.0111.8591.3381.333 M3 Y3.570±0.438 4.035±0.3723.3332.3843.030 Q1.160±0.029 1.202±0.0421.4571.1151.130 M0.957±0.008 0.943±0.0071.2430.9320.901 M4 Y3.636±0.147 3.751±0.151 4.5573.2504.754 Q1.229±0.011 1.268±0.0441.5901.1961.207 M1.003±0.008 0.991±0.0071.3010.9560.938 W2.468±0.024 2.545±0.1002.7992.4712.843 D3.171±0.006 3.169±0.014 3.2073.1783.234 H1.035±0.033 1.465±0.258 1.2201.1271.133 Tourism Y3.951±0.371 3.976±0.4942.9563.4604.135 Q1.963±0.127 2.062±0.2191.9241.7011.750 M2.089±0.073 2.026±0.0691.8171.6151.623 LSTM M1 Y2.477±0.044 2.607±0.1243.0462.4762.632 Q2.000±0.023 2.042±0.016 2.5271.8991.966 M1.516±0.007 1.521±0.0091.8591.3381.333 M3 Y4.113±0.157 4.104±0.717 3.3332.3843.030 Q1.235±0.046 1.246±0.031 1.4571.1151.130 M1.050±0.004 1.053±0.005 1.2430.9320.901 M4 Y3.877±0.149 3.909±0.168 4.5573.2504.754 Q1.293±0.002 1.303±0.0091.5901.1961.207 M1.056±0.008 1.059±0.0071.3010.9560.938 W2.780±0.041 2.813±0.107 2.7992.4712.843 D3.492±0.010 3.494±0.009 3.2073.1783.234 H1.087±0.074 1.030±0.0241.2201.1271.133 Tourism Y4.238±0.531 4.077±0.1372.9563.4604.135 Q1.864±0.009 1.939±0.0561.9241.7011.750 M1.827±0.009 1.826±0.0151.8171.6151.623 32 Table 13: Best-checkpoint WQL by benchmark and frequency in the supervised experiment. Neural values are mean±sample standard deviation across four seeds. SCALEIN uses scale-invariant MAE and SCALECON uses scale-contaminated MAE, both with the same input-window standard scaling. References are fitted at every rolling origin on the same eligible series. Lower is better, and bold marks the better loss within each matched pair. Architecture Benchmark FrequencySCALEINSCALECONS.Naive ETS ARIMA N-HiTS M1 Y0.119±0.003 0.117±0.0070.2190.1440.204 Q0.096±0.002 0.096±0.001 0.1390.0940.106 M0.083±0.001 0.081±0.003 0.1440.0880.086 M3 Y0.105±0.005 0.107±0.0060.1050.0930.100 Q0.081±0.001 0.082±0.0010.0920.0790.081 M0.083±0.001 0.083±0.001 0.1110.0890.088 M4 Y0.120±0.001 0.123±0.0040.1460.1180.152 Q0.099±0.000 0.099±0.0000.1180.0970.100 M0.077±0.000 0.077±0.0000.1020.0800.080 W0.061±0.002 0.061±0.0020.0660.0640.064 D0.026±0.000 0.026±0.000 0.0270.0260.027 H0.048±0.001 0.046±0.0010.0390.0420.050 Tourism Y0.146±0.004 0.156±0.0160.1250.1380.168 Q0.113±0.001 0.115±0.0010.1300.1200.152 M0.140±0.002 0.139±0.0030.1710.1410.146 N-BEATS M1 Y0.120±0.005 0.118±0.007 0.2190.1440.204 Q0.093±0.002 0.094±0.002 0.1390.0940.106 M0.082±0.001 0.083±0.0020.1440.0880.086 M3 Y0.104±0.003 0.113±0.009 0.1050.0930.100 Q0.081±0.001 0.081±0.0010.0920.0790.081 M0.082±0.000 0.082±0.0000.1110.0890.088 M4 Y0.119±0.001 0.126±0.0100.1460.1180.152 Q0.098±0.000 0.098±0.0000.1180.0970.100 M0.077±0.000 0.077±0.000 0.1020.0800.080 W0.062±0.002 0.062±0.001 0.0660.0640.064 D0.026±0.000 0.026±0.000 0.0270.0260.027 H0.047±0.001 0.046±0.0020.0390.0420.050 Tourism Y0.142±0.005 0.153±0.0030.1250.1380.168 Q0.115±0.002 0.116±0.0020.1300.1200.152 M0.138±0.002 0.138±0.0020.1710.1410.146 PatchTST M1 Y0.116±0.010 0.117±0.0100.2190.1440.204 Q0.100±0.005 0.097±0.0010.1390.0940.106 M0.103±0.002 0.104±0.0050.1440.0880.086 M3 Y0.165±0.015 0.185±0.0310.1050.0930.100 Q0.090±0.003 0.092±0.0020.0920.0790.081 M0.092±0.001 0.092±0.0010.1110.0890.088 M4 Y0.139±0.006 0.143±0.0120.1460.1180.152 Q0.110±0.003 0.110±0.003 0.1180.0970.100 M0.084±0.001 0.084±0.0010.1020.0800.080 W0.072±0.001 0.072±0.0020.0660.0640.064 D0.034±0.001 0.034±0.0010.0270.0260.027 H0.120±0.015 0.117±0.0290.0390.0420.050 Tourism Y0.133±0.019 0.142±0.0210.1250.1380.168 Q0.173±0.004 0.174±0.0050.1300.1200.152 M0.186±0.011 0.186±0.0090.1710.1410.146 TCN M1 Y0.111±0.015 0.117±0.0100.2190.1440.204 Q0.094±0.003 0.092±0.0020.1390.0940.106 M0.096±0.003 0.094±0.0010.1440.0880.086 M3 Y0.123±0.009 0.135±0.0100.1050.0930.100 Q0.083±0.002 0.085±0.0020.0920.0790.081 M0.091±0.000 0.089±0.0000.1110.0890.088 M4 Y0.133±0.005 0.137±0.005 0.1460.1180.152 Q0.101±0.001 0.104±0.0020.1180.0970.100 M0.084±0.000 0.083±0.0010.1020.0800.080 W0.061±0.000 0.062±0.0010.0660.0640.064 D0.026±0.000 0.026±0.000 0.0270.0260.027 H0.043±0.001 0.044±0.001 0.0390.0420.050 Tourism Y0.159±0.014 0.161±0.0230.1250.1380.168 Q0.123±0.007 0.126±0.0090.1300.1200.152 M0.176±0.009 0.172±0.0070.1710.1410.146 LSTM M1 Y0.121±0.020 0.125±0.0120.2190.1440.204 Q0.098±0.002 0.100±0.001 0.1390.0940.106 M0.096±0.002 0.095±0.0010.1440.0880.086 M3 Y0.142±0.007 0.136±0.014 0.1050.0930.100 Q0.089±0.003 0.089±0.003 0.0920.0790.081 M0.098±0.000 0.098±0.000 0.1110.0890.088 M4 Y0.140±0.005 0.139±0.005 0.1460.1180.152 Q0.106±0.000 0.106±0.0010.1180.0970.100 M0.088±0.000 0.088±0.0000.1020.0800.080 W0.064±0.001 0.064±0.001 0.0660.0640.064 D0.029±0.000 0.029±0.000 0.0270.0260.027 H0.056±0.005 0.057±0.0030.0390.0420.050 Tourism Y0.169±0.022 0.161±0.0060.1250.1380.168 Q0.123±0.004 0.126±0.0060.1300.1200.152 M0.168±0.002 0.170±0.0050.1710.1410.146 33 I Masked Reconstruction with MOMENT We also compare the two loss spaces for masked reconstruction task usingMOMENT[Goswami et al., 2024]. The matched runs use512-point input, a30%masking ratio,60,000steps, and a batch size of512. The results are averaged across 4 seeds. Evaluation covers held-out series from GiftEvalPretrain datasets. We measure MAE on masked positions divided by the input window’s seasonal-naive MAE (reconstruction MASE). After averaging each dataset across seeds, SCALEIN lowers error on all datasets and reduces median reconstruction MASE from 1.42 to 0.69 (51%). Figure 14 shows that the advantage is large and persists throughout training. 020k40k60k Training updates 0.6 0.8 1.0 1.2 1.4 1.6 1.8 Median reconstruction MASE ScaleIn ScaleCon Figure 14: MOMENT reconstruction on held-out GiftEvalPretrain series. Plotted are average per-seed median dataset MASE, with 95% Student-t intervals across 4 seeds. 34