The Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25)

Auto-Regressive Moving Diffusion Models for Time Series Forecasting

Jiaxin Gao12 * , Qinglong Cao12 * , Yuntian Chen2†

1
Shanghai Jiao Tong University, Shanghai, China
2
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang, China
jiaxingao@[Link]; caoql2022@[Link]; ychen@[Link]

Abstract
Time series forecasting (TSF) is essential in various domains,
and recent advancements in diffusion-based TSF models have
shown considerable promise. However, these models typi-
cally adopt traditional diffusion patterns, treating TSF as a
noise-based conditional generation task. This approach ne-
glects the inherent continuous sequential nature of time se-
ries, leading to a fundamental misalignment between diffu-
sion mechanisms and the TSF objective, thereby severely
impairing performance. To bridge this misalignment, and
inspired by the classic Auto-Regressive Moving Average
(ARMA) theory, which views time series as continuous se-
quential progressions evolving from previous data points, we
propose a novel Auto-Regressive Moving Diffusion (ARMD)
model to first achieve the continuous sequential diffusion-
based TSF. Unlike previous methods that start from white
Gaussian noise, our model employs chain-based diffusion
with priors, accurately modeling the evolution of time se-
ries and leveraging intermediate state information to improve
forecasting accuracy and stability. Specifically, our approach
reinterprets the diffusion process by considering future series
as the initial state and historical series as the final state, with
intermediate series generated using a sliding-based technique
during the forward process. This design aligns the diffusion
model’s sampling procedure with the forecasting objective,
resulting in an unconditional, continuous sequential diffu- Figure 1: The comparison between (a) existing diffusion-
sion TSF model. Extensive experiments conducted on seven based time series forecasting methods and (b) our methods.
widely used datasets demonstrate that our model achieves
state-of-the-art performance, significantly outperforming ex-
isting diffusion-based TSF models.
proposed. Notably, diffusion-based models have emerged as
Code — [Link] some of the most promising methods in the realm of TSF
tasks (Meijer and Chen 2024; Yang et al. 2024).
Diffusion models have garnered significant attention in
Introduction computer vision due to their capability to generate high-
Time series forecasting (TSF) plays a pivotal role in vari- quality images (Ho, Jain, and Abbeel 2020; Rombach et al.
ous real-world applications, including transportation plan- 2022; Cao et al. 2024). These methods typically consist of
ning (Fang et al. 2022), energy management (Gao et al. two main phases: the forward diffusion process, where noise
2023a, 2024), and financial market analysis (Lopez-Lira and is progressively added to an image until it transforms into
Tang 2023). With the development of deep-learning technol- white Gaussian noise, and the reverse denoising process,
ogy, numerous advanced deep-learning TSF models (Piao where a denoising network gradually removes noise to re-
et al. 2024; Wu et al. 2023; Zhou et al. 2021) have been construct the original image. While diffusion models are
* These authors contributed equally.
predominantly used in computer vision, recent research has
†
Corresponding author
begun exploring their applications in the TSF task.
Copyright © 2025, Association for the Advancement of Artificial Current diffusion-based TSF methods (Tashiro et al.
Intelligence ([Link]). All rights reserved. 2021; Alcaraz and Strodthoff 2022; Yuan and Qiao 2024)

16727
typically follow traditional diffusion patterns, treating TSF diction for the target series and its evolution trend.
as a noise-based conditional generation task. As illustrated During the final sampling/forecasting phase, the model
in Fig. 1 (a), the forward diffusion process gradually adds begins with the historical time series and gradually trans-
noise to transform the original time series into white Gaus- forms it into the future time series. This approach aligns the
sian noise. During forecasting, the historical series serves sampling process with the ultimate prediction objective, re-
as a condition, and the reverse denoising process converts moving the need for conditional generation. Consequently,
the white Gaussian noise into the predicted future time se- our method improves stability during both training and sam-
ries. While these noise-based conditional diffusion methods pling, resulting in enhanced forecasting performance.
have shown some success, they often overlook the continu- • Inspired by classical time series theory, specifically
ous sequential nature of time series, leading to a misalign- ARMA, and viewing the evolution of time series as an
ment between the diffusion mechanism and the TSF objec- entire diffusion process, we propose the Auto-Regressive
tive, which significantly suppresses performance. Addition- Moving Diffusion (ARMD) model as the first continuous
ally, by directly diffusing the true series into white Gaussian sequential diffusion-based TSF model.
noise, these methods fail to capture and utilize the valuable
intermediate information within the evolution of time series. • By employing a simple sliding operation, our method
To address these limitations, we propose a new con- leverages the valuable intermediate information of the
tinuous sequential diffusion-based model, Auto-Regressive time series as intermediate states within the diffusion
Moving Diffusion (ARMD), for TSF. Drawing inspira- process. This approach successfully aligns the diffusion
tion from the principles of the classic TSF theory, Auto- mechanism with the true evolution of time series, natu-
Regressive Moving Average (ARMA), the time series can rally leading to improved performance.
be viewed as a continuous progression of data points, where • The proposed ARMD is extensively evaluated on the
each new point is influenced by past data points and includes common TSF benchmarks, and the experimental results
some random noise. This relationship can be expressed as: demonstrate that our method achieves state-of-the-art
forecasting performance.
xt = ϕ1 xt−1 + ϕ2 xt−2 + · · · + ϕp xt−p
+ θ1 ϵt−1 + θ2 ϵt−2 + · · · + θq ϵt−q + ϵt , Related Work
where xt represents the value at time step t, and ϵt represents Traditional TSF models like ARMA and ARIMA (Box et al.
the random noise. Our model fully leverages the inherent 2015) rely on statistical methods that assume linear relation-
features of time series, which exhibit continuous sequential ships between past and present observations to identify pat-
evolution and form a chain of dependencies characterized terns. Recently, deep-learning models have gained promi-
by prior information. Unlike previous diffusion methods that nence in TSF. Temporal Convolution Networks (TCNs)
focus solely on denoising, ARMD is designed to simulate (Wu et al. 2023; Gao et al. 2023b) use convolution layers
and learn the underlying evolution of the series. to learn temporal dependencies. Transformer-based mod-
As illustrated in Fig. 1 (b), our proposed ARMD intro- els (Xu et al. 2024) are also widely used in TSF. Notably,
duces a novel approach where the future series is progres- PatchTST (Nie et al. 2022) segments the time series into
sively diffused into the historical series during the forward multiple tokens and uses attention to learn relationships be-
diffusion/evolution process. In contrast, the reverse denois- tween them. Client (Gao, Hu, and Chen 2023) and iTrans-
ing/devolution process leverages the historical series to fore- former (Liu et al. 2024) apply the attention mechanism on
cast the future series. This approach conceptualizes the evo- the inverted dimensions to capture multivariate correlations.
lution of a time series as a diffusion process, where each Additionally, DLinear (Zeng et al. 2023) demonstrates the
intermediate series at a given time step represents a specific efficacy of linear models in TSF tasks.
state within this process. Consequently, the forward diffu- In the realm of diffusion-based TSF models, TimeGrad
sion process can also be described as a forward evolution (Rasul et al. 2021) generates future values autoregressively,
process of the time series, and the traditional reverse denois- suitable for short-term forecasting but suffers from error ac-
ing process is reinterpreted as a reverse devolution process. cumulation and slow inference for long-term forecasting.
Unlike conventional methods that introduce noise to gen- CSDI (Tashiro et al. 2021) avoids autoregressive inference
erate intermediate states from the original series, ARMD by incorporating a self-supervised strategy with input mask-
derives intermediate states by sliding the series according ing. SSSD (Alcaraz and Strodthoff 2022) improves on CSDI
to the diffusion steps. This results in a deterministic pro- by using a structured state space model instead of Trans-
cess that eliminates the uncertainty typically associated with formers to address quadratic complexity issues, though both
noise. These deterministic intermediate series are then em- CSDI and SSSD face boundary disharmony problems (Lug-
ployed for model training. In the reverse devolution pro- mayr et al. 2022). TimeDiff (Shen and Kwok 2023) miti-
cess, a linear-based devolution (denoising) network is uti- gates these limitations with inductive biases tailored for time
lized to devolve the series through a distance-based method. series data. D3VAE (Li et al. 2022) uses a coupled diffusion
Specifically, the devolution network, equipped with a lin- probabilistic model for data augmentation, integrating multi-
ear module, estimates the distance between an intermediate scale denoising score matching and disentangling multivari-
state (series) and the target series. It then adaptively adjusts ate latent variables for enhanced accuracy and stability. TS-
the weighting of the intermediate state and the estimated dis- Diff (Kollovieh et al. 2024) employs a self-guidance mecha-
tance based on the time step, ultimately generating the pre- nism during inference without changing the training process,

16728
excelling in forecasting and synthetic data generation. mr- used in traditional diffusion models. Unlike conventional ap-
Diff (Shen, Chen, and Kwok 2024) applies seasonal-trend proaches that progressively add noise, the intermediate state
t 0
decomposition for trend extraction and non-autoregressive X1−t:T −t in ARMD corresponds to sliding the X1:T by t
denoising. Similarly, MG-TSD (Fan et al. 2024) leverages T
steps, approaching the historical series X−T +1:0 . This ap-
data granularity levels as intermediate diffusion targets for proach leverages the properties of time series, where each in-
better forecasting. Diffusion-TS (Yuan and Qiao 2024) intro- termediate state in the diffusion process reflects an interme-
duces an encoder-decoder Transformer framework with dis- diate series in the time series evolution. The process where
entangled temporal representations to generate high-quality t−1 t
X2−t:T −t+1 is diffused to X1−t:T −t represents a single-step
multivariate time series samples from noisy inputs. movement towards the historical series, as illustrated in Fig.
2, akin to the q process in denoising diffusion probabilistic
Preliminary model (DDPM) (Ho, Jain, and Abbeel 2020). The process
can be formally expressed as:
Time Series Forecasting. Given a historical time series
X−L+1:0 , where L denotes the series length. The objective t
X1−t:T t−1
−t = Slide(X2−t:T −t+1 , 1), (1)
of TSF is to forecast the future values of the same series, de-
noted as X1:T , where T represents the number of time steps where Slide(X, k) denotes the k-step movement of the se-
to forecast. X−L+1:T can be either a univariate time series ries window X towards the historical series. Furthermore,
t
or a multivariate time series. at any time step t, X1−t:T −t can be directly obtained us-
0
Diffusion Models. The traditional diffusion models first ing X1:T , similar to the process in DDPM. Building upon
progressively add noise (Ho, Jain, and Abbeel 2020) to the DDPM’s foundational assumption, we extend this principle
original data X 0 through a forward diffusion process, pro- to our sliding sampling process. This extension enables us
ducing noised data X T . Then the diffusion process is re- to calculate the evolution trends and train the devolution net-
versed to reconstruct the original data or generate the new work effectively. Specifically, the t-step forward process can
data. The intermediate state in each step of diffusion can be rewritten as:
be calculated based on the original data X 0 and the ran-
dom noise applied at each step. Trained to predict the added t 0 √ 0
√
X1−t:T −t = Slide(X1:T , t) = ᾱt X1:T + 1 − ᾱt z t ,
noise or the intermediate state at each step, diffusion mod-
els generate new instances by iterative sampling from white (2)
Gaussian noise. Diffusion models have been widely applied where z t represents the evolution trend from the series X1:T
0
in diverse fields and have demonstrated notable advantages t
to X1−t:T , functionally similar to the noise added in the
−t
in generation quality (Croitoru et al. 2023). t
original DDPM. Given that each time step of X1−t:T −t is
Diffusion Models for TSF in ARMD. In our proposed t
ARMD, TSF is aligned with diffusion models by associ- deterministic, z can be calculated as:
r r
ating the time series with states in the diffusion process. t 1 t 0 1
The future series X1:T would serve as the initial state of z =( X − X1:T )/ − 1. (3)
ᾱt 1−t:T −t ᾱt
the diffusion process. Following the notation convention in
(Shen and Kwok 2023), the initial state is further indicated Here, z t serves as the ground truth for the optimization ob-
0 jective at each time step. In this diffusion scheme, the max-
as X1:T . Conversely, the historical time series X−L+1:0 is
the final state of the diffusion process, which is further de- imum number of diffusion steps T equals the length of the
T series to be predicted.
noted as X−L+1:0 . Here, the upper index of X indicates its
state in the diffusion process, while the lower index repre-
sents the time steps it encompasses. In ARMD, the length of Reverse Denoising (Devolution) of ARMD
the historical series matches that of the future series, allow- The reverse process in ARMD utilizes the historical series
T T
ing the historical series to be defined as X−T +1:0 . The inter- X−T +1:0 to iteratively generate (forecast) the future series
mediate state corresponds to a series transitioning from the 0
X1:T . At each devolution step, the linear-based devolution
t
future series to the historical series, denoted as X1−t:T −t . network R(.) predicts the evolution trend z t , which is used
t t−1
to devolve X1−t:T −t to X2−t:T −t+1 . Given an intermediate
Proposed Model t
state (series) X1−t:T −t and the diffusion step t, R(.) pre-
In this section, we introduce Auto-Regressive Moving Dif- ˆ0 t
dicts X (X , t, θ) for the future series X1:T0
. More specif-
fusion (ARMD), a novel continuous sequential diffusion- ically, within the devolution network R(.), a linear module
based TSF model. ARMD’s overview is depicted in Fig. 2. first provides a prediction of the distance D from the input
t 0
X1−t:T −t to X1:T :
Forward Diffusion (Evolution) of ARMD t
D = Linear(X1−t:T −t ). (4)
In ARMD, the evolution of a time series is conceptualized
0 Then, the diffusion step t is used to adaptively balance the
as a diffusion process. Here, the future series X1:T serves
as the initial state of the forward diffusion (evolution) pro- interaction between the distance prediction D and the input
T t
cess. In contrast, the historical series X−T +1:0 represents the
X1−t:T −t , enabling the network to produce more accurate
t
final state, as opposed to white Gaussian noise commonly predictions. Particularly, as t decreases, the input X1−t:T −t

16729
Figure 2: Illustration of the diffusion process in ARMD. During the forward diffusion process, the future series is progressively
diffused into the historical series. Conversely, the reverse denoising process utilizes the historical series to iteratively generate/-
forecast the future series. For clarity, the series X is only annotated with an upper index to indicate its state.

0 0
becomes closer to the target X1:T , so the model’s output X1:T , which successfully aligns the sampling process with
should increasingly resemble the input, placing less empha- the TSF objective, making the model an unconditional diffu-
sis on D. Conversely, as t increases, the model places greater sion TSF model. The method follows the sampling approach
reliance on D. This adaptive balancing can be mathemati- from DDIM (Song, Meng, and Ermon 2020), replacing the
cally expressed as: predicted noise ϵθ (xt , t) with the predicted evolution trend
t
W (t) ∗ X1−t:T ẑ(t, θ). The sampling process from t to t − 1 (akin to the p
−t + (1 − bW (t)) ∗ D
Xˆ0 (X t , t, θ) = , process in DDPM/DDIM) can be expressed as:
(1 + cW (t))d
√
(5) √
 t
X1−t:T −t − 1 − ᾱt ẑ(t, θ)

t−1
where W (t) represents a weight coefficient that decreases X2−t:T −t+1 = ᾱt−1 √
ᾱt
as t increases, ranging from 0 to 1. We initialize W (t) with q
the predefined coefficients ᾱt of DDPM, and it is updated + 1 − ᾱt−1 − σt2 ẑ(t, θ) + σt ϵt ,
along with the linear module’s parameters during training.
(8)
The hyper-parameters b, c, and d are leveraged to balance
where ẑ(t, θ) is the predicted evolution trend at the time
the interaction between the distance prediction D and the
t step t, and ϵt ∼ N (0, I). Since the series evolution in
input X1−t:T −t . To increase sample diversity and prevent ARMD is deterministic, we remove the noise term σt ϵt . The
over-fitting, a small deviation is added to the input of R(.) content within the parentheses in the first term is actually
during the training process.
Xˆ0 (X t , t, θ). Thus, the simplified equation is:
After obtaining the prediction Xˆ0 (X t , t, θ), the predicted
√
evolution trend ẑ(t, θ) can be calculated as: t−1
X2−t:T −t+1 = ᾱt−1 Xˆ0 (X t , t, θ)
r r q (9)
1 t ˆ0 t 1 + 1 − ᾱt−1 − σt2 ẑ(t, θ).
ẑ(t, θ) = ( X − X (X , t, θ))/ − 1. (6)
ᾱt 1−t:T −t ᾱt
To accelerate sampling, k steps can be skipped at each
With the ground truth z t calculated in Equation (3) and iteration, and the process can be further inferred as:
the prediction ẑ(t, θ), the training objective is formulated as:
√
t−k
X1−t+k:T −t+k = ᾱt−k Xˆ0 (X t , t, θ)
Lθ = Et [|z t − ẑ(t, θ)|]. (7) q (10)
+ 1 − ᾱt−k − σt2 ẑ(t, θ).
Sampling/Forecasting of ARMD
In the sampling phase, starting from the historical series For better understanding, the training and sampling pro-
T
X−T +1:0 , ARMD iteratively generates the future series cedures are detailed in Algorithm 1 and Algorithm 2.

16730
Algorithm 1: Training. models, are summarized in Table 1, with each model’s
Require: Maximum number of diffusion steps T , which results averaged over 10 sampling runs. Notably, ARMD
also represents the length of the historical/future series; demonstrates superior forecasting performance, achieving
Predefined coefficients ᾱ0:T . optimal results in 12 out of the 14 experimental settings.
1: repeat Our extensive experiments reveal that while other diffusion-
2: Sample X1:T0
from the training set; based TSF models can achieve effective results on specific
3: Sample t ∼ Uniform({1, 2, . . . , T }); datasets, they often suffer from instability, performing well
Generate the diffused sample X1−t:Tt in certain scenarios but deviating significantly from the true
4: −t using Equa-
values in others, which indicates a lack of generalization
tion (2), and calculate the evolution trend z t using
across diverse data. In contrast, ARMD consistently exhibits
Equation (3);
robust performance across all datasets, suggesting a higher
5: Use the devolution network R(.) to generate the pre-
degree of reliability and adaptability to various multivariate
dicted sample Xˆ0 (X t , t, θ) using Equation (5), and forecasting challenges. Specifically, on the ETTm1 dataset,
obtain the predicted evolution trend ẑ(t, θ) using ARMD achieves a substantial 47.7% reduction in MSE and
Equation (6); a 30.1% reduction in MAE compared to the second-best
6: Calculate the loss Lθ using Equation (7); model, D3VAE. On the Stock dataset, ARMD surpasses TS-
7: Update the devolution network R(.) of ARMD by Diff, the second-best model in this setting, with a 28.8% re-
taking a gradient descent step on ∇θ L; duction in MSE and a 26.3% reduction in MAE. Further-
8: until converged. more, ARMD attains more than a 10% reduction in both
MSE and MAE on the Solar Energy, ETTh1, and Exchange
Algorithm 2: Sampling/Forecasting. datasets compared to the nearest competitor. These results
indicate that ARMD is the best diffusion-based TSF model.
T
Require: Historical series X−T +1:0 ; Trained devolution
network R(.); Sampling interval ∆t; Predefined coef- Comparison with Other TSF Models
ficients ᾱ0:T .
As demonstrated in Table 2, ARMD consistently outper-
1: for t = T to 0 by ∆t do
forms other advanced models in multivariate TSF tasks.
2: Obtain Xˆ0 (X t , t, θ) using X1−t:T
t
−t and t with the The comparative results for other TSF models are primar-
devolution network R(.), and calculate the corre- ily sourced from iTransformer (Liu et al. 2024), with the
sponding evolution trend ẑ(t, θ) using Equation (6); Stock dataset being an exception, where additional evalua-
t
3: Update X1−t:T −t using Equation (10); tions are conducted using official implementations to ensure
4: end for a thorough analysis. ARMD achieves the highest number of
0
5: Output the prediction of X1:T . best counts, indicating its consistent ability to surpass other
TSF methods across multiple benchmarks. These results not
only establish ARMD as the leading diffusion-based TSF
Experiments model but also underscore its superiority over existing mod-
Experimental Settings els. Overall, these findings highlight ARMD’s promise as a
robust and effective solution for multivariate TSF tasks.
ARMD is evaluated on seven widely used benchmark
datasets, including Solar Energy (Lai et al. 2018), Exchange Qualitative Analysis
(Lai et al. 2018), Stock (Yoon, Jarrett, and Van der Schaar
2019), and four ETT datasets (Zhou et al. 2021). We com- As illustrated in Fig. 3, we qualitatively analyze the per-
pare ARMD with five advanced diffusion-based TSF mod- formance of ARMD by comparing it with the advanced
els: Diffusion-TS (Yuan and Qiao 2024), MG-TSD (Fan diffusion-based model, Diffusion-TS. Given the same his-
et al. 2024), TSDiff (Kollovieh et al. 2024), D3VAE (Li torical series, both models make 10 independent predictions
et al. 2022), TimeGrad (Rasul et al. 2021). Additionally, for the future series. Our analysis reveals that ARMD con-
some other advanced TSF models, including iTransformer sistently produces more stable and accurate predictions, par-
(Liu et al. 2024), TimesNet (Wu et al. 2023), DLinear (Zeng ticularly in scenarios involving high peak values. This is
et al. 2023), PatchTST (Nie et al. 2022), and Client (Gao, in contrast to Diffusion-TS, which shows greater variability
Hu, and Chen 2023) are also compared with ARMD. and less precision in similar settings. We attribute ARMD’s
For all datasets, the historical length and prediction length superior performance to its deterministic training process
are both set to 96. Following the evaluation methodology and the reduced sampling steps required during inference.
employed in a previous study (Zhou et al. 2021), we calcu- These features of ARMD not only enhance the model’s abil-
late the mean squared error (MSE) and mean absolute error ity to generate predictions that closely align with actual se-
(MAE) on z-score normalized data, enabling a consistent as- ries but also result in narrower uncertainty distributions.
sessment of various variables.
Efficiency Comparison
Comparison with Diffusion-Based Models We compare the training and inference efficiency of ARMD
The multivariate forecasting results, comparing our pro- with the diffusion-based TSF models Diffusion-TS, MG-
posed ARMD with various advanced diffusion-based TSF TSD, and TimeGrad on ETTm1, as shown in Table 3. Due to

16731
 Methods Metric Solar Energy ETTh1 ETTh2 ETTm1 ETTm2 Exchange Stock Best Count
MSE 0.167 0.445 0.311 0.337 0.181 0.093 0.235
ARMD (Ours) 12
MAE 0.236 0.459 0.338 0.376 0.255 0.203 0.269
MSE 0.181 0.643 0.544 0.678 0.497 0.275 0.416
Diffusion-TS (Yuan and Qiao 2024) 0
MAE 0.252 0.586 0.494 0.613 0.459 0.382 0.533
MSE 0.443 1.096 0.295 0.690 0.202 0.396 0.365
MG-TSD (Fan et al. 2024) 1
MAE 0.529 0.765 0.345 0.631 0.278 0.460 0.453
MSE 0.352 0.614 0.470 0.686 0.242 0.125 0.330
TSDiff (Kollovieh et al. 2024) 0
MAE 0.432 0.521 0.418 0.603 0.311 0.240 0.365
MSE 0.416 1.123 0.389 0.644 0.394 0.240 0.345
D3VAE (Li et al. 2022) 0
MAE 0.492 0.728 0.373 0.538 0.410 0.371 0.390
MSE 0.359 0.884 0.297 0.661 0.182 0.508 0.333
TimeGrad (Rasul et al. 2021) 1
MAE 0.449 0.725 0.349 0.639 0.254 0.554 0.376

Table 1: Result comparisons of multivariate series forecasting with diffusion-based TSF models. The best results are highlighted
in bold. The “Best Count” column indicates the times of achieving the best result.

Methods Metric Solar Energy ETTh1 ETTh2 ETTm1 ETTm2 Exchange Stock Best Count
MSE 0.167 0.445 0.311 0.337 0.181 0.093 0.235
ARMD (Ours) 7
MAE 0.236 0.459 0.338 0.376 0.255 0.203 0.269
MSE 0.203 0.386 0.297 0.334 0.180 0.086 0.342
iTransformer (Liu et al. 2024) 2
MAE 0.237 0.405 0.349 0.368 0.264 0.206 0.413
MSE 0.250 0.384 0.340 0.338 0.187 0.107 0.427
TimesNet (Wu et al. 2023) 1
MAE 0.292 0.402 0.347 0.375 0.267 0.234 0.499
MSE 0.290 0.386 0.333 0.345 0.193 0.088 0.286
DLinear (Zeng et al. 2023) 1
MAE 0.378 0.400 0.387 0.372 0.292 0.218 0.325
MSE 0.234 0.414 0.302 0.329 0.175 0.088 0.516
PatchTST (Nie et al. 2022) 3
MAE 0.286 0.419 0.348 0.367 0.259 0.205 0.524
MSE 0.199 0.392 0.305 0.336 0.184 0.086 0.352
Client (Gao, Hu, and Chen 2023) 1
MAE 0.239 0.409 0.353 0.369 0.267 0.206 0.433

Table 2: Result comparisons of multivariate series forecasting with other TSF models. The best results are highlighted in bold.

Training time (s) Inference time (s) Ablation Studies

ARMD (Ours) 11.872 31.650 In this section, we set a range of ablation experiments to
testify diverse components of the proposed ARMD.
Diffusion-TS 149.974 1183.593
Intermediate State Generation Method. ARMD generates
MG-TSD 834.815 3299.918 intermediate states using a sliding-based method to maintain
TimeGrad 449.792 1749.896 series continuity. An alternative method to generate interme-
diate states involves interpolating between the initial series
0 T
X1:T and final series X−T +1:0 , as described by:
Table 3: Training/inference time (s) of ARMD, Diffusion-
TS, MG-TSD and TimeGrad on the ETTm1 dataset. t 0 T 0
X1−t:T −t = X1:T + (X−T +1:0 − X1:T ) ∗ t/T.

The results of using these interpolation-based intermediate

the optimized model structure and the reduction in the num- states are presented in the second row of Table 4. The results
ber of sampling steps, ARMD achieves more than a tenfold indicate that our sliding-based method consistently outper-
reduction in both training and inference times compared to forms the interpolation-based approach across all settings,
other diffusion models. underscoring its superiority in preserving series continuity.

16732
 Methods Metric Solar Energy ETTh1 ETTh2 ETTm1 ETTm2 Exchange Stock Best Count
MSE 0.167 0.445 0.311 0.337 0.181 0.093 0.235
ARMD (Ours) 11
MAE 0.236 0.459 0.338 0.376 0.255 0.203 0.269
MSE 0.184 0.509 0.407 0.359 0.210 0.131 0.249
Interpolation Method 0
MAE 0.250 0.481 0.373 0.385 0.265 0.249 0.278
MSE 0.308 0.707 0.376 0.598 0.235 0.233 0.350
T-embedding Method 0
MAE 0.380 0.589 0.380 0.527 0.331 0.330 0.423
MSE 1.343 0.631 0.285 0.578 0.168 0.137 0.239
Transformer Backbone 3
MAE 0.713 0.531 0.341 0.499 0.253 0.242 0.271
MSE 0.199 0.482 0.334 0.552 0.199 0.103 0.328
Adding Noise 0
MAE 0.315 0.479 0.353 0.526 0.295 0.228 0.347

Table 4: Ablation experimental results. Interpolation Method: replacing the intermediate state generation method with an
interpolation method. T-embedding Method: using a t-embedding approach for denoising. Transformer Backbone: utilizing
a Transformer-based backbone. Adding Noise: adding sampling noise. The best results are highlighted in bold.

ARMD sults of applying this t-embedding-based method within the

devolution network are shown in the third row of Table 4,
demonstrating that our distance-based method outperforms
the t-embedding-based method across all settings.
Backbone of the Devolution Network. Many existing
value

diffusion-based TSF models use a Transformer-based back-

bone in the denoising network (Yuan and Qiao 2024; Li
et al. 2024; Tashiro et al. 2021). Nevertheless, the devolu-
tion network in ARMD uses a linear-based backbone, which
accelerates training and sampling. The impact of replacing
Time the linear-based backbone with a Transformer-based back-
Diffusion-TS bone is presented in the fourth row of Table 4. The re-
sults show that the linear-based backbone outperforms the
Transformer-based backbone in most settings (11 out of 14).
Noise in the Sampling Process. Given that the series sam-
pling/forecasting process is deterministic, the noise term
value

σt ϵt in Equation (8) is set to 0 during sampling/forecasting.

The results of adding sampling noise are shown in the last
row of Table 4, showing that introducing randomness leads
to a decline in model performance.

Time Conclusion
Figure 3: The distributions of 10 different predictions made Inspired by the classic ARMA theory, we creatively intro-
by ARMD and Diffusion-TS given the same historical se- duce an Auto-Regressive Moving Diffusion (ARMD) model
ries. ARMD achieves more stable and accurate predictions. for TSF, which reinterprets the evolution of time series as
a diffusion process. Particularly, regarding the target future
series as the initial state, and the historical series as the fi-
nal state, the intermediate series are generated with the se-
Devolution/Denosing Learning Method. The devolution ries slide operations to complete the diffusion process. In
network of ARMD takes a distance-based method to pro- this manner, the sampling procedure of the diffusion model
vide the prediction Xˆ0 (X t , t, θ) and ẑ(t, θ), differing from becomes the series forecasting, which denotes that the dif-
traditional denoising (devolution) networks that utilize a t- fusion mechanism is successfully aligned with the TSF ob-
embedding-based method (Yuan and Qiao 2024; Shen and jective, and further resulting in an unconditional, continu-
Kwok 2023; Shen, Chen, and Kwok 2024). In this method, ous sequential diffusion TSF model. The proposed ARMD
the model initially generates a prediction based on the in- is validated across seven widely used datasets, and the ex-
put intermediate state, then embeds the time step t to gen- perimental results show that our method effectively suits the
erate a t-embedding, and finally combines the t-embedding unique characteristics of time series data, and achieves su-
with the initial prediction to produce the final output. The re- perior forecasting performance.

16733
 Acknowledgements Gao, J.; Chen, Y.; Hu, W.; and Zhang, D. 2023a. An adap-
This work is financially supported by the The Major Sci- tive deep-learning load forecasting framework by integrating
ence and Technology Projects of Ningbo (No. 2022Z236), Transformer and domain knowledge. Advances in Applied
Natural Science Foundation of Ningbo of China (No. Energy, 100142.
2023J027), China Meteorological Administration under Gao, J.; Hu, W.; and Chen, Y. 2023. Client: Cross-
Grant QBZ202316 as well as by the High Performance variable linear integrated enhanced transformer for multi-
Computing Centers at Eastern Institute of Technology, variate long-term time series forecasting. arXiv preprint
Ningbo, and Ningbo Institute of Digital Twin. We would like arXiv:2305.18838.
to express our gratitude to Associate Professor Wenbo Hu Gao, J.; Hu, Y.; Cao, Q.; Dai, S.; and Chen, Y. 2023b.
from Hefei University of Technology, Associate Professor CLeaRForecast: Contrastive Learning of High-Purity Rep-
Hao Sun from Renmin University of China, and Professor resentations for Time Series Forecasting. arXiv:2312.05758.
Cewu Lu from Shanghai Jiao Tong University for their as- Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion
sistance with this work. probabilistic models. In Neural Information Processing Sys-
tems.
CRediT Authorship Contribution Statement Kollovieh, M.; Ansari, A. F.; Bohlke-Schneider, M.;
Jiaxin Gao: Conceptualization, Investigation, Methodol- Zschiegner, J.; Wang, H.; and Wang, Y. B. 2024. Predict,
ogy, Software, Resources, Validation, Visualization, Writing refine, synthesize: Self-guiding diffusion models for proba-
– original draft, Writing – review & editing. bilistic time series forecasting. Advances in Neural Informa-
Qinglong Cao: Conceptualization, Investigation, Methodol- tion Processing Systems, 36.
ogy, Visualization, Writing – original draft, Writing – review Lai, G.; Chang, W.-C.; Yang, Y.; and Liu, H. 2018. Modeling
& editing. long-and short-term temporal patterns with deep neural net-
Yuntian Chen: Conceptualization, Funding acquisition, In- works. In The 41st international ACM SIGIR conference on
vestigation, Methodology, Resources, Supervision, Writing research & development in information retrieval, 95–104.
– review & editing. Li, Y.; Chen, W.; Hu, X.; Chen, B.; and Zhou, M. 2024.
Transformer-Modulated Diffusion Models for Probabilistic
Declaration of Competing Interest Multivariate Time Series Forecasting. In The Twelfth Inter-
national Conference on Learning Representations.
The authors declare that they have no known competing fi- Li, Y.; Lu, X.; Wang, Y.; and Dou, D. 2022. Generative time
nancial interests or personal relationships that could have ap- series forecasting with diffusion, denoise, and disentangle-
peared to influence the work reported in this paper. ment. Advances in Neural Information Processing Systems,
35: 23009–23022.
References Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; and
Alcaraz, J. M. L.; and Strodthoff, N. 2022. Diffusion-based Long, M. 2024. iTransformer: Inverted Transformers Are
time series imputation and forecasting with structured state Effective for Time Series Forecasting. In The Twelfth Inter-
space models. arXiv preprint arXiv:2208.09399. national Conference on Learning Representations.
Box, G. E.; Jenkins, G. M.; Reinsel, G. C.; and Ljung, G. M. Lopez-Lira, A.; and Tang, Y. 2023. Can ChatGPT Forecast
2015. Time series analysis: forecasting and control. John Stock Price Movements? Return Predictability and Large
Wiley & Sons. Language Models. arXiv preprint arXiv:2304.07619.
Cao, Q.; Wang, D.; Li, X.; Chen, Y.; Ma, C.; and Yang, X. Lugmayr, A.; Danelljan, M.; Romero, A.; Yu, F.; Timofte,
2024. Teaching Video Diffusion Model with Latent Physical R.; and Van Gool, L. 2022. Repaint: Inpainting using de-
Phenomenon Knowledge. arXiv:2411.11343. noising diffusion probabilistic models. In IEEE/CVF Con-
ference on Computer Vision and Pattern Recognition.
Croitoru, F.-A.; Hondru, V.; Ionescu, R. T.; and Shah, M.
Meijer, C.; and Chen, L. Y. 2024. The Rise of Diffu-
2023. Diffusion models in vision: A survey. IEEE Transac-
sion Models in Time-Series Forecasting. arXiv preprint
tions on Pattern Analysis and Machine Intelligence, 45(9):
arXiv:2401.03006.
10850–10869.
Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam,
Fan, X.; Wu, Y.; Xu, C.; Huang, Y.; Liu, W.; and Bian, J. 2022. A Time Series is Worth 64 Words: Long-
J. 2024. MG-TSD: Multi-granularity time series diffu- term Forecasting with Transformers. arXiv preprint
sion models with guided learning process. arXiv preprint arXiv:2211.14730.
arXiv:2403.05751.
Piao, X.; Chen, Z.; Murayama, T.; Matsubara, Y.; and
Fang, W.; Zhuo, W.; Yan, J.; Song, Y.; Jiang, D.; and Zhou, Sakurai, Y. 2024. Fredformer: Frequency Debiased Trans-
T. 2022. Attention meets long short-term memory: A deep former for Time Series Forecasting. arXiv preprint
learning network for traffic flow forecasting. Physica A: Sta- arXiv:2406.09009.
tistical Mechanics and its Applications, 587: 126485. Rasul, K.; Seward, C.; Schuster, I.; and Vollgraf, R. 2021.
Gao, J.; Cao, Q.; Chen, Y.; and Zhang, D. 2024. Cross- Autoregressive denoising diffusion models for multivariate
variable Linear Integrated ENhanced Transformer for Pho- probabilistic time series forecasting. In International Con-
tovoltaic power forecasting. arXiv:2406.03808. ference on Machine Learning.

16734
Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Om-
mer, B. 2022. High-resolution image synthesis with latent
diffusion models. In Proceedings of the IEEE/CVF confer-
ence on computer vision and pattern recognition, 10684–
10695.
Shen, L.; Chen, W.; and Kwok, J. 2024. Multi-Resolution
Diffusion Models for Time Series Forecasting. In The
Twelfth International Conference on Learning Representa-
tions.
Shen, L.; and Kwok, J. 2023. Non-autoregressive condi-
tional diffusion models for time series prediction. In Inter-
national Conference on Machine Learning, 31016–31029.
PMLR.
Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion
implicit models. arXiv preprint arXiv:2010.02502.
Tashiro, Y.; Song, J.; Song, Y.; and Ermon, S. 2021. CSDI:
Conditional score-based diffusion models for probabilistic
time series imputation. In Neural Information Processing
Systems.
Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M.
2023. TimesNet: Temporal 2D-Variation Modeling for Gen-
eral Time Series Analysis. In International Conference on
Learning Representations.
Xu, J.; Wu, C.; Li, Y.-F.; Danoy, G.; and Bouvry, P. 2024.
Survey and Taxonomy: The Role of Data-Centric AI in
Transformer-Based Time Series Forecasting. arXiv preprint
arXiv:2407.19784.
Yang, Y.; Jin, M.; Wen, H.; Zhang, C.; Liang, Y.; Ma, L.;
Wang, Y.; Liu, C.; Yang, B.; Xu, Z.; et al. 2024. A survey on
diffusion models for time series and spatio-temporal data.
arXiv preprint arXiv:2404.18886.
Yoon, J.; Jarrett, D.; and Van der Schaar, M. 2019. Time-
series generative adversarial networks. Advances in neural
information processing systems, 32.
Yuan, X.; and Qiao, Y. 2024. Diffusion-ts: Interpretable dif-
fusion for general time series generation. arXiv preprint
arXiv:2403.01742.
Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are trans-
formers effective for time series forecasting? In Proceedings
of the AAAI conference on artificial intelligence, volume 37,
11121–11128.
Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.;
and Zhang, W. 2021. Informer: Beyond efficient transformer
for long sequence time-series forecasting. In Proceedings of
the AAAI conference on artificial intelligence, volume 35,
11106–11115.

16735