Unit 2 Notes

 Moving average models MA(q).

Moving average (MA) models are a type of time series model where the current value of a series is expressed as a
linear combination of past error terms (also known as "shocks" or "white noise"). An MA(q) model, specifically,
includes q past error terms, where q is the order of the model. These models are useful for smoothing out
irregularities in data and identifying underlying trends by averaging out random fluctuations.
Here's a breakdown:
Key Concepts:
 Error Terms (εt):
These are random, uncorrelated disturbances or "shocks" that affect the time series. They are assumed to have a
mean of 0 and a constant variance.
 Order (q):
The order of the MA model determines how many past error terms are included in the model. For example, an
MA(2) model includes the current error term and the two previous error terms.
 Stationarity:
MA processes are inherently stationary, meaning their statistical properties (like mean and variance) don't change
over time.
MA(q) Model Equation:
The general form of an MA(q) model is:
yt = μ + εt + θ₁ε(t-1) + θ₂ε(t-2) + ... + θqε(t-q)
Where:
 yt is the value of the time series at time t.
 μ is the mean of the time series.
 εt is the error term at time t.
 θ₁, θ₂, ..., θq are the parameters (coefficients) associated with the lagged error terms.
Example (MA(2) Model):
yt = μ + εt + θ₁ε(t-1) + θ₂ε(t-2)
This equation indicates that the value of the time series at time t is influenced by the current error term, the error
term from the previous time step (t-1), and the error term from two time steps ago (t-2).
In essence, MA models capture how past random events (represented by the error terms) impact the current value of
the time series.
Practical Applications:
 Smoothing Time Series:
MA models can help remove short-term fluctuations (noise) and highlight underlying trends in data.
 Forecasting:
By understanding how past errors influence the present, MA models can be used to predict future values.
 Identifying Underlying Patterns:
MA models can reveal how recent shocks or events affect the series over time.
 Financial Modeling:
They are used in financial econometrics to model stock prices and other financial time series.
What is the Moving Average Model?
Moving Average Models are a type of time series analysis model usually used in econometrics to forecast trends
and understand patterns in time series data. In moving average models the present value of the time series
depends on the linear combination of the past white noise error terms of the time series. In time series
analysis moving average is denoted by the letter "q" which represents the order of the moving average model, or
in simple words we can say the current value of the time series will depend on the past q error terms. Therefore,
the moving average model of order q could be represented as:
Xt=c+ϵt+θ1.ϵt−1+θ2.ϵt−2+...+θq.ϵt−qXt=c+ϵt+θ1.ϵt−1+θ2.ϵt−2+...+θq.ϵt−q
Here,
 XtXt is the value of time series at time t
 c is a constant or the mean of the time series
 ϵt,ϵt−1,ϵt−2,...,ϵt−qϵt,ϵt−1,ϵt−2,...,ϵt−q are the white noise terms associated with the time series at time t, t-1,
t-2, ... , t-q.
 θ1,θ2,...,θqθ1,θ2,...,θq are the moving average constants.
For example, if we consider MA(1) model, in this model the present value of the time series will only depend on
a single past error term and the time series becomes:
Xt=c+ϵt+θ1.ϵt−1Xt=c+ϵt+θ1.ϵt−1
From this observation we can also conclude one of the most important aspects of moving average models that the
higher the value of the order of moving average model (q), the model will have longer memory and dependence
on the past values.
Interpretation of MA model:
There is a difference in the shock wave that is seen in the MA model and AR model that we can mention which
might help us get a better understanding how MA and AR model differ. For a better understanding let's look at
the AR model's general form as well:
Xt=c+ϕ[Link]−1+ϕ[Link]−2+...+ϕ[Link]−p+ϵtXt=c+ϕ[Link]−1+ϕ[Link]−2+...+ϕ[Link]−p+ϵt
 First that the past noise term ϵt−1ϵt−1 affects the MA model's present value XtXt directly as we can see in the
above equation of the MA model but in AR model the past noise term(ϵt−1ϵt−1) have an indirect influence
on the present AR model value(XtXt) since the AR model equation depends on the previous value of the
model(Xt−1Xt−1) and the previous model value depends on it's noise term(ϵt−1ϵt−1).
 The MA model works a a finite impulse model, which means that the current noise value affects the present
value of the model as well as "q" further values, as the moving average models only depend on q terms in the
past. Whereas AR models acts as infinite impulse model since the current noise affects infinite values of the
model in the future. In AutoRegressive model ϵtϵt value affects the XtXt term which affects
the Xt+1Xt+1 term and so on.
Concept Related to Moving Average:
Now let's discuss about some of the concepts that can help us in understanding the moving average model in a
better way:
 Stationarity: Stationarity is the principle of time series data that conveys that the statistical properties of the
data doesn't change with time, the mean of the data remains the same or we can also say that the data
fluctuates around a certain value, the standard deviation of the time series data nearly remains constant, and
there must not be any seasonality in the time series data or there is no periodic behavior in the data. We can
check for the stationarity of the dataset visually as well as through Augmented Dickey-Fuller(ADF) Test .
We consider stationarity to be one of the most important aspect that the time series data must possess in order
to be accepted by the models that are applied to time series data for accurate modelling.
 Differencing: Differencing is one of the most important steps to consider during time series analysis, after
taking a peek at the original time series data, if the data is not stationary and contains a lot of trends then
differencing must be considered since for accurate time series data analysis the data must be stationary. In
regular differencing the current time series data is subtracted by the previous data point. Δyt=yt−yt−1Δyt=yt
−yt−1, this method removes trends from the data, making it suitable for modelling.
 White Noise: White noise is the error term which has the mean of zero and a constant standard deviation
with no correlation of the data points with each other. White noise acts as a benchmark in the forecasting
process through time series modelling, if the forecast error is nor white noise further modifications could be
performed on the model, but if it reaches a state such that the forecast errors are white noise then the model
would need no further improvements. The value of white noise series are random and unpredictable therefore
if any time series data is a white noise then there is no method to model or forecast it.
White noise time series with mean = 0 and standard deviation = 1
 ACF Plot: Autocorrelation Function plot or the ACF plot is the plot of correlation between the time series
and its lagged version. It shows how similar the time series is with it's different lagged values. Here the lag
term is a fixed time displacement, in the ACF plot the x-axis is the lagged time series and the y-axis is the
correlation which ranges from -1 to 1.
 Moving-average (MA) models
 A moving-averge process of order q, or MA(q), is a weighted sum of the current random error plus
the q most recent errors, and can be written as
 xt=wt+θ1wt−1+θ2wt−2+⋯+θqwt−q,(4.22)
 where {wt} is a white noise sequence with zero mean and some variance σ2; for our purposes we
usually assume that wt∼N(0,q). Of particular note is that because MA processes are finite sums of
stationary errors, they themselves are stationary.
 Of interest to us are so-called “invertible” MA processes that can be expressed as an infinite AR
process with no error term. The term invertible comes from the inversion of the backshift operator ( B)
that we discussed in class (i.e., Bxt=xt−1). So, for example, an MA(1) process with θ<|1| is invertible
because it can be written using the backshift operator as
 xt=wt−θwt−1xt=wt−θBwtxt=(1−θB)wt,⇓wt=1(1−θB)xtwt=(1+θB+θ2B2+θ3B3+
…)xtwt=xt+θxt−1+θ2xt−2+θ3xt−3+…(4.23)
 4.8.1 Simulating an MA(q) process
 We can simulate MA(q) processes just as we did for AR(p) processes using [Link](). Here are 3
different ones with contrasting θ’s:
 [Link](123)
 ## list description for MA(1) model with small coef
 MA_sm <- list(order = c(0, 0, 1), ma = 0.2)
 ## list description for MA(1) model with large coef
 MA_lg <- list(order = c(0, 0, 1), ma = 0.8)
 ## list description for MA(1) model with large coef
 MA_neg <- list(order = c(0, 0, 1), ma = -0.5)
 ## simulate MA(1)
 MA1_sm <- [Link](n = 50, model = MA_sm, sd = 0.1)
 MA1_lg <- [Link](n = 50, model = MA_lg, sd = 0.1)
 MA1_neg <- [Link](n = 50, model = MA_neg, sd = 0.1)
 with their associated plots.
 ## setup plot region
 par(mfrow = c(1, 3))
 ## plot the ts
 [Link](MA1_sm, ylab = expression(italic(x)[italic(t)]), main = expression(paste(theta,
 " = 0.2")))
 [Link](MA1_lg, ylab = expression(italic(x)[italic(t)]), main = expression(paste(theta,
 " = 0.8")))
 [Link](MA1_neg, ylab = expression(italic(x)[italic(t)]), main = expression(paste(theta,
 " = -0.5")))


  Figure 4.25: Time series of simulated MA(1) processes with θ=0.2 (left), θ=0.8 (middle),
and θ=−0.5 (right).
 In contrast to AR(1) processes, MA(1) models do not exhibit radically different behavior with
changing θ. This should not be too surprising given that they are simply linear combinations of white
noise.
 4.8.2 Correlation structure of MA(q) processes
 We saw in lecture and above how the ACF and PACF have distinctive features for AR(p) models, and
they do for MA(q) models as well. Here are examples of four MA(q) processes. As before, we’ll use a
really big n so as to make them “pure,” which will provide a much better estimate of the correlation
structure.
 [Link](123)
 ## the 4 MA coefficients
 MA_q_coef <- c(0.7, 0.2, -0.1, -0.3)
 ## empty list for storing models
 MA_mods <- list()
 ## loop over orders of q
 for (q in 1:4) {
 ## assume sd = 1, so not specified
 MA_mods[[q]] <- [Link](n = 1000, list(ma = MA_q_coef[1:q]))
 }
 Now that we have our four MA(q) models, lets look at plots of the time series, ACF’s, and PACF’s.
 ## set up plot region
 par(mfrow = c(4, 3))
 ## loop over orders of q
 for (q in 1:4) {
 [Link](MA_mods[[q]][1:50], ylab = paste("MA(", q, ")", sep = ""))
 acf(MA_mods[[q]], [Link] = 12)
 pacf(MA_mods[[q]], [Link] = 12, ylab = "PACF")
 }
 
 Figure 4.26: Time series of simulated MA(q) processes (left column) of increasing orders from 1-4
(rows) with their associated ACF’s (center column) and PACF’s (right column). Note that only the first
50 values of xt are plotted.
 Note very little qualitative difference in the realizations of the four MA(q) processes (Figure 4.26). As
we saw in lecture and is evident from our examples here, however, the ACF for an MA( q) process goes
to zero for lags > q, but the PACF tails off toward zero very slowly. This is an important diagnostic tool
when trying to identify the order of q in ARMA(p,q) models.

 Condition of invertibility.

Invertibility in time series analysis, specifically for moving average (MA) models, refers to the ability to express a
time series using an infinite autoregressive (AR) representation. For an MA(q) process, this means that the roots of
the characteristic polynomial (related to the MA coefficients) must lie outside the unit circle. This condition ensures
that the infinite AR representation converges, making the model invertible and useful for forecasting and analysis.
 Here's a more detailed explanation:
MA Processes and Invertibility:
 MA(q) Process:
A moving average process of order q, denoted as MA(q), expresses the current value of a time series as a
weighted sum of past white noise (or error) terms. The general form is: Xt = θ(B)at = a_t + θ₁a_{t-1} + ... + θ_q
a_{t-q}, where a_t is white noise and θᵢ are the moving average coefficients.
 Invertibility Condition:
For an MA(q) process to be invertible, the roots of the characteristic polynomial, Θ(z) = 1 + θ₁z + θ₂z² + ... + θ_q
z^q, must lie outside the unit circle. Alternatively, the roots zᵢ must satisfy |zᵢ| > 1.
 Example:
For an MA(1) process (Xt = at + θat-1), invertibility requires |θ| < 1. For an MA(2) process, it requires |θ₂| < 1, θ₂
+ θ₁ < 1, and θ₂ - θ₁ < 1 according to Florida State University .
 Why Invertibility Matters:
 Unique Representation: Invertibility ensures a unique representation of the time series in terms of its past values.
 Forecasting: Invertible models are more suitable for forecasting because the recent past has a greater influence on
future values, making the model more stable and predictable according to Stack Exchange .
 Parsimonious Representation: Invertibility allows expressing the MA process as an infinite-order AR process, which
can be more parsimonious (require fewer parameters) than a high-order MA representation.
 Consequences of Non-Invertibility:
If the roots are not outside the unit circle, the infinite AR representation will not converge, and the model may not
be stable or useful for forecasting.
 Connection to AR Processes:
Invertibility of an MA process is analogous to stationarity of an AR process. For an AR(p) process, stationarity
requires the roots of its characteristic polynomial to lie outside the unit circle.
 Example with Roots:
According to Real Statistics Using Excel a MA(3) process with characteristic polynomial 1 - 0.5z - 0.2z² +
0.4z³ would be invertible if the roots of 1 - 0.5z - 0.2z² + 0.4z³ = 0 are all outside the unit circle.

In the context of stochastic processes, invertibility means that the process can be expressed as a linear
combination of its past observations, allowing the past observations to be recovered from the process's current and
future values. For an Autoregressive Moving Average (ARMA) process, invertibility is determined by the roots of
the Moving Average (MA) polynomial. Specifically, an MA(q) process is invertible if the absolute values of all the
roots of its characteristic polynomial are greater than 1.
Invertibility of MA(q) processes:
 An MA(q) process is represented by: Xt = μ + εt + θ1εt-1 + ... + θqεt-q, where εt is a white noise process.
 To determine invertibility, consider the characteristic polynomial: 1 + θ1L + θ2L^2 + ... + θqL^q = 0, where L is the
lag operator.
 The process is invertible if all the roots of this polynomial are outside the unit circle (i.e., their magnitudes are
greater than 1).
 If an MA process is invertible, it can be expressed as an infinite-order AR process: εt = Xt - θ1Xt-1 - θ2Xt-2 - ....
 Invertibility implies that recent observations have a higher weight than past observations when reconstructing the
noise term from the observed data.
Invertibility of AR(p) processes:
 Autoregressive (AR) processes are always invertible.
 However, for an AR(p) process to be stationary, its characteristic polynomial (obtained by setting the AR parameters
to the lag operator) must have roots outside the unit circle.
Invertibility of ARMA(p,q) processes:
 An ARMA(p,q) process is invertible if its MA part is invertible.
 Therefore, the invertibility condition for ARMA(p,q) processes is the same as for MA(q) processes: the roots of the
MA characteristic polynomial must be outside the unit circle.

 Autoregressive models AR(p).

 An autoregressive (AR) model of order p, denoted as AR(p), is a statistical model that uses past values of
a time series to predict its future values. It essentially models the current value of a variable as a linear
combination of its past p values, plus a random error term.
Here's a breakdown:
 Autoregressive:
The model is "auto-regressive" because it regresses the variable on its own past values.
 Order p:
The "p" indicates how many past time steps are used in the model. For example, an AR(1) model uses
only the most recent past value, while an AR(2) model uses the two most recent past values, and so on.
 Linear Combination:
The model assumes a linear relationship between the current value and its past values. This means the
current value is calculated by multiplying each past value by a coefficient and summing them up, along
with a constant and a random error term.
Mathematical Representation:
The general formula for an AR(p) model is:
Xt = c + φ1Xt-1 + φ2Xt-2 + ... + φpXt-p + εt
Where:
 Xt is the value of the time series at time t.
 c is a constant term.
 φ1, φ2, ..., φp are the model parameters (coefficients) that determine the influence of each past value.
 Xt-1, Xt-2, ..., Xt-p are the lagged values of the time series (past values).
 εt is a random error term (also called white noise), assumed to be independent and identically distributed
with a mean of zero and constant variance.
Example:
An AR(1) model would look like this:
Xt = c + φ1Xt-1 + εt
In this case, the current value Xt is predicted based on the previous value Xt-1, a constant, and a random
error.
Applications:
AR models are widely used in various fields, including:
 Time series analysis: Modeling and forecasting stock prices, economic indicators, weather patterns, etc.
 Signal processing: Analyzing and processing signals like audio and image data.
 Generating training data: Augmenting datasets for machine learning models.
Autoregressive models (AR models) are a concept in time series analysis and forecasting that
captures the relationship between an observation and several lagged observations i.e previous time
steps. Its idea is that the current value of a time series data can be expressed as a linear combination of
its past values with some random noise.
Mathematical Explanation
Mathematically, autoregressive model of order p, denoted as AR(p) and can be expressed as:
Xt=c+ϕ1Xt−1+ϕ2Xt−2+…+ϕpXt−p+εtXt=c+ϕ1Xt−1+ϕ2Xt−2+…+ϕpXt−p+εt
Where:
 XtXt is the value at time t.
 c is a constant.
 ϕ1,ϕ2,…,ϕpϕ1,ϕ2,…,ϕp are the model parameters.
 Xt−1,Xt−2,…,Xt−pXt−1,Xt−2,…,Xt−p are the lagged values.
 εtεt represents white noise or random error at time tt.
Working of Autoregressive (AR) Model
Autocorrelation Function (ACF) in Autoregressive Models is the one that measures the correlation
between a time series and its past lagged values. Its working is as follows:
1. Understanding Lag and Temporal Dependence
A lag represents the number of time steps by which the series is shifted. For example:
  Lag 1 compares each value with the one immediately preceding it.
 Lag 2 compares values with those two time steps earlier, and so on.
The autocorrelation coefficient at a specific lag quantifies this relationship:
 A high autocorrelation indicates a strong connection between the present and the past value at that
lag.
 A low or near-zero autocorrelation suggests weak or no temporal dependence.
2. Visualizing Autocorrelation using ACF Plot
To analyze autocorrelation patterns, we use an ACF plot which displays autocorrelation values across
various lags:
1. The x-axis shows the lag values.
2. The y-axis shows the autocorrelation coefficients.
3. Peaks that lie outside the confidence bounds (usually shaded) indicate statistically significant
correlations.
Such patterns help reveal the underlying temporal structure of the data and guide the selection of an
appropriate lag order in AR models.
3. Role of ACF in AR Model Selection
The ACF helps determine how many past time steps (lags) should be included in the model. Typically
you will look at the ACF plot along with a Partial Autocorrelation Function (PACF) plot to choose a
suitable lag order.
It also helps assess whether a time series is stationary which means its statistical properties like mean
and variance stay consistent over time. In a stationary series autocorrelations typically decrease
gradually as lag increases. If autocorrelations persist or decay slowly it may indicate non-
stationarity suggesting that the series needs transformation before modeling.
Types of Autoregressive Models
Autoregressive models vary based on the number of past values (lags) they use. The two most common
types are:
1. AR(1) Model: This is a autoregressive model of order 1 which is the simplest form of an
autoregressive model. In this model the current value of the time series depends only on its immediate
past value along with a constant and some random noise. This model is particularly useful when the
data shows strong autocorrelation at lag 1 i.e the current value depends only on the previous value. It is
expressed as:
Xt=c+ϕ1Xt−1+εtXt=c+ϕ1Xt−1+εt
2. AR(p) Model: It is the generalized form of the autoregressive model where the current value
depends on the past pp values. Choosing the correct order pp is a crucial step and typically involves
analyzing the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots.

 Yull-Worker equations.

The Yule-Walker equations are a set of linear equations that relate the parameters of an autoregressive
(AR) model to the autocorrelations of the time series. They are used to estimate the parameters of an AR
model from observed data. The equations are particularly useful for AR models because they provide a
direct way to calculate the model parameters based on the autocorrelation function.
In more detail:
 Autoregressive (AR) Model:
An AR model represents a time series as a linear combination of its past values, plus a random error
term.
 Autocorrelation Function (ACF):
The ACF measures the correlation between values of a time series at different points in time, separated
by a lag.
 Yule-Walker Equations for AR(p) Model:
 For an AR(p) model (a model where the current value depends on the previous p values), the Yule-
Walker equations are a set of p linear equations that relate the AR coefficients (φ₁, φ₂, ..., φ ₚ) to the
first p autocorrelations (ρ₁, ρ₂, ..., ρ ₚ):

 ρ(k) = φ₁ρ(k-1) + φ₂ρ(k-2) + ... + φ ₚρ(k-p) for k = 1, 2, ..., p.

 These equations can also be written in matrix form as Rφ = r, where R is the autocorrelation matrix, φ is the
vector of AR coefficients, and r is the vector of autocorrelations.
 Application:
The Yule-Walker equations are used to estimate the parameters of an AR model by solving the system
of equations. This involves estimating the autocorrelations from the observed data and then using them
to calculate the AR coefficients.
In essence, the Yule-Walker equations provide a bridge between the observed autocorrelation structure of
a time series and the parameters of an autoregressive model that can represent that structure.
Understanding Yule-Walker Equations
The Yule-Walker equations are a set of linear equations used in the field of statistics and signal
processing to estimate the coefficients of an autoregressive (AR) model. An AR model is a type of time
series model that expresses a variable of interest as a linear combination of its own past values. The Yule-
Walker equations provide a method for estimating the parameters of the AR model from the
autocorrelation function of the time series data.
Background of Autoregressive Models
Before diving into the Yule-Walker equations, it is important to understand autoregressive models. An
autoregressive model of order p, denoted as AR(p), can be written as:
Xt = φ1Xt-1 + φ2Xt-2 + ... + φpXt-p + εt,
where Xt is the value of the time series at time t, φ1, φ2, ..., φp are the parameters of the model, and
εt is white noise with zero mean and constant variance.
Derivation of Yule-Walker Equations
The Yule-Walker equations are derived from the autocorrelation function of the time series. The
autocorrelation function, ρ(k), measures the correlation between values of the time series at different
times, separated by a lag k. For an AR(p) model, the Yule-Walker equations are given by:
ρ(1) = φ1ρ(0) + φ2ρ(1) + ... + φpρ(p-1),
ρ(2) = φ1ρ(1) + φ2ρ(0) + ... + φpρ(p-2),
...
ρ(p) = φ1ρ(p-1) + φ2ρ(p-2) + ... + φpρ(0).
These equations can be written in matrix form as:
Rφ = r,
where R is the autocorrelation matrix, φ is the vector of AR coefficients, and r is the autocorrelation
vector. The autocorrelation matrix R is Toeplitz, meaning that its diagonals from top left to bottom right
are constant.
Solving Yule-Walker Equations
To estimate the AR coefficients, one can solve the Yule-Walker equations using methods for solving
linear equations, such as the Levinson-Durbin recursion, which is computationally efficient for Toeplitz
matrices. The solution provides estimates of the AR coefficients that are consistent and asymptotically
efficient, meaning that as the size of the time series data grows, the estimates converge to the true values
with minimal variance.
Applications of Yule-Walker Equations
The Yule-Walker equations are widely used for parameter estimation in AR models, which have
applications in various domains such as economics, finance, engineering, and environmental science. AR
models are used to forecast future values of a time series based on its own past behavior, making them
valuable tools for prediction and analysis.
Limitations and Considerations
 While the Yule-Walker equations are a powerful tool for estimating AR model parameters, there are some
limitations to consider. The accuracy of the estimates depends on the assumption that the time series is
stationary, meaning that its statistical properties do not change over time. If the time series is non-
stationary, differencing or other transformations may be required before applying the Yule-Walker
method. Additionally, the Yule-Walker estimates are based on the sample autocorrelation function, which
can introduce sampling error, especially for small sample sizes.
Conclusion
The Yule-Walker equations are a fundamental component of time series analysis, providing a method to
estimate the parameters of autoregressive models. They leverage the autocorrelation structure of the data
to produce parameter estimates, enabling the modeling and forecasting of time series. Understanding and
applying the Yule-Walker equations is essential for statisticians, economists, and engineers working with
time series data.
 Stationarity conditions.

A time series is considered stationary if its statistical properties, such as mean, variance, and
autocorrelation, remain constant over time. This means that a stationary series does not exhibit trends or
seasonality, and its behavior in the future should be similar to its behavior in the past.
Key conditions for stationarity:
 Constant Mean:
The average value of the time series remains the same over time. This implies no upward or downward
trend.
 Constant Variance:
The spread or dispersion of the data around the mean remains consistent over time. This means no
increasing or decreasing volatility.
 Constant Autocorrelation:
The relationship between a data point and its past values (lagged values) is consistent over time. This
means the correlation at different lags does not change with time.
Why is stationarity important?
Stationarity is a crucial assumption for many time series analysis techniques, such as forecasting and
modeling. If a time series is not stationary, these techniques may produce inaccurate or misleading
results. For example, if a time series has an upward trend, a simple model might predict higher values in
the future based on that trend, without accounting for the fact that the trend might change or stop.
Different types of stationarity:
While the above conditions define weak or wide-sense stationarity, there's also strict stationarity, which
requires that the joint probability distribution of the series remains the same at any set of time points. In
practice, weak stationarity is often sufficient for many applications.
A stationary time series is one whose properties do not depend on the time at which the series is
observed.17 Thus, time series with trends, or with seasonality, are not stationary — the trend and
seasonality will affect the value of the time series at different times. On the other hand, a white noise
series is stationary — it does not matter when you observe it, it should look much the same at any
point in time.
Some cases can be confusing — a time series with cyclic behaviour (but with no trend or seasonality)
is stationary. This is because the cycles are not of a fixed length, so before we observe the series we
cannot be sure where the peaks and troughs of the cycles will be.
In general, a stationary time series will have no predictable patterns in the long-term. Time plots will
show the series to be roughly horizontal (although some cyclic behaviour is possible), with constant
variance.
 Autoregressive-moving average models ARMA (p,q).

An Autoregressive Moving Average (ARMA) model is a statistical tool used to analyze and forecast time
series data. It combines two approaches: autoregression (AR) and moving average (MA). An ARMA
 model of order (p, q), denoted as ARMA(p, q), uses 'p' lagged values of the time series and 'q' lagged
forecast errors to predict future values.
Here's a breakdown:
1. Autoregressive (AR) Component:
 An AR model predicts the current value of a time series based on its past values.
 The 'p' in ARMA(p, q) represents the number of lagged observations (past values) included in the model.
 For example, in an AR(1) model, the current value depends on the previous value.
2. Moving Average (MA) Component:
 An MA model uses past forecast errors (also known as white noise or residuals) to predict the current
value.
 The 'q' in ARMA(p, q) represents the number of lagged forecast errors considered.
 In an MA(1) model, the current value is influenced by the previous period's error.
3. ARMA(p, q) Model:
 An ARMA(p, q) model combines both AR and MA components, using 'p' lagged observations and 'q'
lagged forecast errors.
 This allows the model to capture more complex patterns in the time series than either AR or MA models
alone.
4. Key Characteristics:
 Stationarity:
ARMA models are typically used for stationary time series, meaning their statistical properties (like
mean and variance) don't change over time.
 Linearity:
ARMA models are based on linear relationships between past values, past errors, and the current
value.
 Parsimony:
ARMA models can be more parsimonious (require fewer parameters) than using very high-order AR or
MA models to represent the same data.
In essence, ARMA models provide a flexible framework for modeling and forecasting time series by
leveraging both the past behavior of the series itself and Understanding ARMA Model
The ARMA model is a combination of two simpler models: the Autoregressive (AR) model and the
Moving Average (MA) model. The ARMA model is used to describe time series data that is stationary,
meaning its statistical properties do not change over time.
 Autoregressive (AR) Model : This model uses the dependency between an observation and a
number of lagged observations (previous time points). It is denoted as AR(p), where p is the
number of lagged observations included.
 Moving Average (MA) Model: This model uses the dependency between an observation and a
residual error from a moving average model applied to lagged observations. It is denoted as MA(q),
where ?q is the number of lagged forecast errors included.
The ARMA model combines these two approaches and is denoted as ARMA(p, q), where p is the order
of the autoregressive part and q is the order of the moving average part.
1. ARMA Components: Autoregressive (AR)
The Autoregressive (AR) part of the ARMA model uses the relationship between an observation and a
number of lagged (previous) observations to predict future values. Imagine, that you are attempting to
forecast the temperature for tomorrow by using the data from the last several days. The AR portion
makes the assumption that the current temperature and the temperatures from earlier days are
connected. For instance suppose we write the temperature of today as TtTt and the temperatures of the
last two days as Tt−1Tt−1 and Tt−2Tt−2, an AR(2) model (since it uses two lagged values) can be
written as:
Tt=c+ϕ1Tt−1+ϕ2Tt−2+etTt=c+ϕ1Tt−1+ϕ2Tt−2+et
Where:
 c is a constant.
 ϕ1ϕ1 and ϕ2ϕ2 are coefficients that determine the influence of the past temperatures.
 etet is the error term (random noise).
2. ARMA Components: Moving Average (MA)
The Moving Average (MA) part of the ARMA model uses the dependency between an observation and
a residual error from a moving average model applied to lagged observations. Continuing with our
temperature example, the MA part assumes that today's temperature is also influenced by the errors
made in predicting previous days' temperatures. If we denote today's error as etet and the errors of the
last two days as et−1et−1 and et−2et−2 an MA(2) model can be written as:
Tt=c+et+θ1et−1+θ2et−2Tt=c+et+θ1et−1+θ2et−2
Where:
 c is a constant.
 ϕ1ϕ1and ϕ2ϕ2 are coefficients that determine the influence of the past temperatures.
Mathematical Representation of ARMA Model
The ARMA model is a combination of both AR and MA components. An ARMA(p, q) model,
where pp is the number of lagged observations (AR part) and qq is the number of lagged forecast errors
(MA part), is represented as:
Tt=c+Σi=1pϕiTt−i+Σj=1qθjet−j+etTt=c+Σi=1pϕiTt−i+Σj=1qθjet−j+et
How to Determine the Orders p and q in ARMA Model?
Determining the appropriate values for p and q is crucial for building an effective ARMA model. This
can be done using the following methods:
1. Partial Autocorrelation Function (PACF) :
 PACF is used to determine the order p of the AR model. It measures the correlation between
observations at different lags, excluding the influence of intermediate lags.
 The order p is determined by the lag at which the PACF plot cuts off.
2. Autocorrelation Function (ACF) :
 ACF is used to determine the order q of the MA model. It measures the correlation between
observations at different lags.
 The order q is determined by the lag at which the ACF plot cuts off.
the past errors in predictions.
Application and Use Cases of ARMA Model
For predicting and evaluating time series data the ARMA model is extensively utilized in many
different domains. A few typical uses are as follows:
 Economics: Predicting stock prices, exchange rates, and economic indicators.
 Weather Forecasting: Analyzing temperature, rainfall, and other meteorological data.
 Sales Forecasting: Predicting future sales based on past sales data.
 Engineering: Monitoring and controlling industrial processes.
 Inventory management: Forecasting future demand for products.
 Epidemiology: Predicting the spread of diseases.
Advantages and Disadvantages of ARMA Model
Advantages Limitations

Stationarity Requirement: The ARMA model assumes

that the time series data is stationary, meaning its
Simplicity: The ARMA model is statistical properties do not change over time. Non-
relatively simple to understand and stationary data needs to be transformed before applying
implement. the ARMA model.

Effectiveness: It works well for many Complexity with High Parameters: For large values
 Advantages Limitations

types of time series data, especially when of ? and ?, the model can become complex and difficult
there are clear patterns or trends. to interpret.

Combination of AR and MA: By

combining both autoregressive and
Choosing the right order for the AR and MA components
moving average components, the ARMA
can be challenging.
model can capture more complex
patterns in the data.

Conclusion
The ARMA model is a powerful tool for time series analysis, helping us predict future values based on
past trends. It offers a thorough method for deciphering patterns and generating forecasts by merging
the moving average and autoregressive components. Even though it has drawbacks, its ease of use and
potency make it a useful technique in a variety of sectors.
We have deconstructed the ARMA model in this easy-to-read introduction for beginners. Always keep
in mind that improving forecasts requires balancing historical values and mistakes