DAV CSC601

Module 3. Time Series

Time Series

A time series is a collection of observations of well-defined data

items obtained through repeated measurements over time.
For example, measuring the value of retail sales each month of
the year would comprise a time series.
Time series data can be found in economics, social sciences,
finance, epidemiology, and the physical sciences.
Time Series
Example of time series data
Healthcare industry – Blood pressure monitoring, Heart rate monitoring.
Environment – Global temperature and air pollution levels.
Society – Birth rates over a period of time, Population, etc
Time Series Data
Time Series Data

Time series data is data that is recorded over consistent intervals of time.
Time series analysis

Analyzing this time series data with certain tools and techniques is called Time
series analysis.
Forecasting
Forecasting is the process of making predictions from the historical data so
that they can predict the future from the past and present data.
Types of forecasting:
1) Quantitative forecasting
2) Qualitative forecasting
Regression vs Time Series
Regression
It is the relationship between dependent and independent variables.
The target variable is continuous.
This involves finding patterns in the data and predict the target with this
pattern.
Time Series
It is the series of data points associated with time.
The target variable is continuous.
This involves finding trends in the data and forecast the future with this
trend.
Regression vs Time Series

[Link]
How to Analyze Time Series

Collecting the data and cleaning it

Preparing Visualization with respect to time vs key feature
Observing the stationarity of the series
Developing charts to understand its nature.
Model building – AR, M, ARMA and ARIMA
Extracting insights from prediction
What Are the Limitations of Time Series Analysis?

we have to take care of those during our data analysis.

Similar to other models, the missing values are not supported by TSA
The data points must be linear in their relationship.
Data transformations are mandatory, so they are a little expensive.
Models mostly work on Uni-variate data.
Components of Time Series Analysis
Data Types of Time Series

stationary Non-stationary.

The mean value of them should be If either the mean-variance or covariance

completely constant in the data is changing with respect to time, the
during the analysis. dataset is called non-stationary.
The variance should be constant with
respect to the time-frame
Covariance measures the relationship
between two variables.
Data Types of Time Series

stationary Non-stationary.

The mean value of them should be If either the mean-variance or covariance

completely constant in the data is changing with respect to time, the
during the analysis. dataset is called non-stationary.
The variance should be constant with
respect to the time-frame
Covariance measures the relationship
between two variables.
Methods to Check Stationarity

Augmented Dickey-Fuller (ADF) Test or Unit Root Test

Null Hypothesis (H0): Series is non-stationary
Alternate Hypothesis (HA): Series is stationary
p-value >0.05 Fail to reject (H0)
p-value <= 0.05 Accept (H1)
Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test
Null Hypothesis (H0): Series is stationary
Alternate Hypothesis (HA): Series is non-stationary
ADF
The statsmodels package provides a reliable implementation of the ADF test
via the adfuller() function in [Link].
It returns the following outputs:
The p-value
The value of the ADF statistic(Test statistic)
Number of lags considered for the test
The critical value cut-offs.
When the ADF statistic is lower than the critical value shown,
Reject the null hypothesis and
infer that the time series is stationary.
ADF
KPSS
The statsmodels package provides a reliable implementation of the KPSS test
via the KPSS() function in [Link].
It returns the following outputs:
The p-value
The value of the KPSS statistic
Number of lags considered for the test
The critical value cut-offs.
When the KPSS statistic is greater than the critical value shown,
KPSS
Methods to Check Stationarity

ADF KPSS Result

Case 1 Stationary Stationary Stationary

Case 2 Non - Stationary Non - Stationary Non - Stationary

strictly Stationary
Case 3 Non - Stationary Stationary

Difference
Case 4 Stationary Non - Stationary
Stationary
Autoregressive Models

Autoregressive models belong to the family of time series models.

These models capture the relationship between an observation and
several lagged observations (previous time steps).
The core idea is that the current value of a time series can be expressed as
a linear combination of its past values, with some random noise.
Autoregressive Models

AR(1) Model:
In the AR(1) model, the current value depends only on the previous value.
It is expressed as:

AR(p) Model:
The general autoregressive model of order p includes p lagged values.
Autocorrelation (ACF) in AR Model

“ACF” (Autocorrelation Function), is a fundamental concept in time series

analysis and autoregressive models.
It refers to the correlation between a time series and a lagged version of
itself.
Autocorrelation measures how closely the current value of a time series is
related to its past values, specifically those at different time lags.
Concept of autocorrelation
1. Autocorrelation involves calculating the correlation between a time series
and a lagged version of itself.
The “lag” represents the number of time units by which the series is
shifted.
For example, a lag of 1 corresponds to comparing the series with its
previous time step,
while a lag of 2 compares it with the time step before that, and so on.
Lag values help you calculate autocorrelation, which measures how each
observation in a time series is related to previous observations.
Concept of autocorrelation
2. The autocorrelation at a particular lag provides insights into the temporal
dependence of the data.
If the autocorrelation is high at a certain lag, it indicates a strong
relationship between the current value and the value at that lag.
If the autocorrelation is low or close to zero, it suggests a weak or no
relationship.
Concept of autocorrelation
3. To visualize autocorrelation, a common approach is to create an ACF plot.
This plot displays the autocorrelation coefficients at different lags. The
horizontal axis represents the lag, and the vertical axis represents the
autocorrelation values.
Significant peaks or patterns in the ACF plot can reveal the underlying
temporal structure of the data.
Concept of autocorrelation
4. In an Autoregressive model of order p, the current value of the time series
is expressed as a linear combination of its past p values,

5. Autocorrelation can also be used to assess whether a time series is

stationary. In a stationary time series, autocorrelation should gradually
decrease as the lag increases.
Autocorrelation
The autocorrelation function (ACF) at lag k for a time series.
Cov() is the covariance function.
Var() is the variance function.
k is the lag.
​X t is the value of the time series at time t.
X t-k is the value of the time series at time t-k
Autocorrelation Interpretation

Positive ACF: A positive ACF at lag k indicates a positive correlation

between the current observation and the observation at lag k.
Negative ACF: A negative ACF at lag k indicates a negative correlation
between the current observation and the observation at lag k.
Decay in ACF: The decay in autocorrelation as lag increases often signifies
the presence of a trend or seasonality in the time series.
Significance: Significant ACF values at certain lags may suggest potential
patterns or relationships in the time series
Partial autocorrelation

Partial autocorrelation removes the influence of intermediate lags,

providing a clearer picture of the direct relationship between a variable
and its past values.
Partial autocorrelation focuses on the direct correlation at each lag.
The partial autocorrelation function (PACF) at lag k for a time series.
Partial autocorrelation

is the value of the time series at time.

​is the value of the time series at time (t-k)
is the conditional covariance between
and and given the values of the intermediate lags.
is the conditional variance of given the
values of the intermediate lags.
is the conditional variance of
given the values of the intermediate lags.
Partial autocorrelation Interpretation
Direct Relationship: PACF isolates the direct correlation between the
current observation and the observation at lag k, controlling for the
influence of lags in between.
AR Process Identification: Peaks or significant values in PACF at specific
lags can indicate potential orders for autoregressive (AR) terms in time
series models.
Modeling Considerations: Analysts often examine PACF to guide the
selection of lag orders in autoregressive integrated moving average
(ARIMA) models.
 Autocorrelation Partial Autocorrelation

Used for identifying the order of a Used for identifying the order of an
moving average (MA) process. autoregressive (AR) process.

Represents the overall correlation Highlights the direct relationships

structure of the time series. between observations at specific lags.

Partial Autocorrelation measures the

Autocorrelation measures the linear direct linear relationship between an
relationship between an observation observation and its previous
and its previous observations at observations at a specific lag,
different lags. excluding the contributions from
intermediate lags.
Benefits of Autoregressive Models:

Simplicity
Interpretability
Useful for Stationary Data
Efficiency
Modeling Temporal Patterns
Drawbacks of Autoregressive Models:

Stationarity Assumption
Limited to Short-Term Dependencies
Lag Selection
Sensitivity to Noise
Limited Forecast Horizon
Data Quality Dependence
Autocorrelation plot using Matplotlib

Autocorrelation plots are a commonly used tool for checking randomness in

a data set.
Characteristics Of Autocorrelation Plot :
It varies from +1 to -1.
An autocorrelation of +1 indicates that if time series one increases in value
the time series 2 also increases in proportion to the change in time series 1.
An autocorrelation of -1 indicates that if time series one increases in value
the time series 2 decreases in proportion to the change in time series 1.
Autocorrelation plot using Matplotlib

Application of Autocorrelation:
Pattern recognition.
Signal detection.
Signal processing.
Estimating pitch.
Technical analysis of stocks.
Moving Average

Moving Average Models are a type of time series analysis model

usually used in to forecast trends and understand patterns in time
series data.
In moving average models the present value of the time series
depends on the linear combination of the past white noise error
terms of the time series.
In time series analysis moving average is denoted by the letter “q”
which represents the order of the moving average model,
Moving Average

Therefore, the moving average model of order q could be

represented as:
Moving Average

Therefore, the moving average model of order q could be

represented as:

i MA(1) model,
in this model the present value of the time series will only depend on a
single past error term and the time series becomes:
ARMA
The name ARMA is short for Autoregressive Moving Average. It
comes from merging two simpler models - the Autoregressive, or
AR, and the Moving Average, or MA.
In analysis, we tend to put the residuals at the end of the model
equation, so that’s why the “MA” part comes second.
Let’s suppose that “Y” is some random time-series variable. Then, a
simple Autoregressive Moving Average model would look
something like this:
yt = c + ϕ1 yt-1 + θ1 ϵ t-1 + ϵ t
ARIMA

Autoregression (AR): A model that uses the correlation

between the current observation and lagged observations. The
number of lagged observations is referred to as the lag order or
p.
Integrated (I): The use of differencing of raw observations to

ARIMA(p,d,q) make the time series stationary. The number of differencing

operations is referred to as d.
Moving Average (MA): A model takes into account the relationship between the
current observation and the residual errors from a moving average model applied to
past observations. The size of the moving average window is the order or q.
ARIMA

The parameters of the ARIMA model are defined as follows:

p: The number of lag observations included in the
model, also called the lag order.
d: The number of times that the raw observations are
differenced, also called the degree of differencing.
ARIMA(p,d,q)
q: The size of the moving average window, also called
the order of moving average.
What Is Box Jenkins Methodology?

It comprises of different steps including

Identification,
Estimation,
Diagnostic checking,
Model refinement and forecasting.
Box-Jenkins method is an iterative process, and steps 1 to 4 from identification to
model refinement are often repeated until a suitable and well-diagnosed model
is obtained.
Example

Example #1
Consider a retail business employing the Box Jenkins methodology to
forecast monthly sales data. The business identifies seasonal patterns by
analyzing past sales trends and employs autoregression and moving
averages within the ARIMA model to make short-term sales predictions.
This enables the business to adjust inventory, marketing strategies, and
staffing levels, optimizing its operations based on anticipated customer
demand.