TIME SERIES

ANALYSIS
Stochastic Processes · Moving Average · Regression · ARIMA

A comprehensive guide to forecasting and modeling temporal data

C O U R S E AG E N D A

01 02
Stochastic Process Moving Average Method

Random variables, stationarity, ergodicity & white noise Simple, weighted & exponential moving averages for smoothing

03 04
Regression in Time Series ARIMA Model

Trend analysis, OLS, autocorrelation & Durbin-Watson test AR, MA, differencing, ACF/PACF & model identification
STOCHASTIC
PROCESS
Understanding randomness and temporal dependencies in data
STOCHASTIC PROCESS 01

A stochastic process is a collection of random variables {X(t)} indexed by time t, where each observation is a realization of a
probability distribution.

Stationarity Ergodicity
Mean, variance & autocovariance remain constant over time. Weak Time averages converge to ensemble averages. Allows statistical
stationarity requires constant mean & finite variance. Key inference from a single long realization instead of many realizations.
assumption for many TS models.

White Noise Autocorrelation

A sequence of i.i.d. random variables with zero mean and constant Correlation of a series with its own past values. ACF and PACF plots
variance σ². Acts as the building block for ARMA models. reveal the dependence structure and guide model selection.
TYPES OF STO CHASTIC PROCESSES

Random Walk Markov Process Gaussian Process

Xₜ = Xₜ₋₁ + εₜ P(Xₜ | Xₜ₋₁, ...) = P(Xₜ | Xₜ₋₁) X ~ GP(μ(t), k(t,t'))

• Non-stationary process • Memory-less property • Fully defined by mean & covariance

• Variance grows with time • Future depends only on present • Any finite subset is jointly normal
• Cannot be predicted accurately • Basis for many financial models • Foundation for Bayesian methods
• First difference is white noise • Discrete or continuous state • Used in kriging & ML
MOVING
AVERAGE METHOD
Smoothing techniques to reveal underlying trends in time series
MOVING AVERAGE: TY PE S & F ORMULAS

Simple Moving Average (SMA) SMA(t) = (1/n) × Σ Xₜ₋ᵢ for i = 0 to n-1

Equal weight to all n past observations. Simple to compute but slow to react to
recent changes. Window size n controls smoothing — larger n = smoother
output.

Weighted Moving Average (WMA) WMA(t) = Σ wᵢ × Xₜ₋ᵢ where Σwᵢ = 1

Assigns greater weight to recent observations. More responsive than SMA.

Weights must sum to 1. Linearly decreasing weights are common: n, n-1, ..., 1.

Exponential Moving Average (EMA) EMAₜ = α·Xₜ + (1-α)·EMAₜ₋₁ (0 < α < 1)

Exponentially decreasing weights. Smoothing factor α controls responsiveness.

No fixed window size. Most popular for financial time series.
R EG RES SI O N I N
T IM E SER I ES DAT A
Modeling trends, seasonality and causal relationships over time
 REG RESS IO N A NALY SI S IN TIM E SER IES

Regression Models for Time Series Key Issues & Diagnostics

Autocorrelation
Simple Linear Trend Yₜ = β₀ + β₁t + εₜ
OLS residuals in TS are often correlated, violating
independence assumption. Use Durbin-Watson test.
Fits a straight-line trend through time. OLS estimates minimize residual sum of squares.

Spurious Regression

Non-stationary series may appear related but are not.

Polynomial Trend Yₜ = β₀ + β₁t + β₂t² + εₜ Check for cointegration before modeling.

Captures non-linear trends. Higher-order polynomials risk overfitting.

Heteroscedasticity

Variance of residuals changes over time. Use

ARCH/GARCH or robust standard errors.

Seasonal Regression Yₜ = Trend + Σ γₛ·Dₛₜ + εₜ Durbin-Watson Test

DW ≈ 2: no autocorrelation. DW < 2: positive AC. DW

Includes dummy variables Dₛ for each season. Captures periodic patterns. > 2: negative autocorrelation.
ARIMA ACF

MODEL
AutoRegressive Integrated Moving Average — the gold standard for univariate forecasting
ARIMA(p, d, q) — COMPONENTS

ARIMA(p, d, q): φ(B)·ΔᵈXₜ = θ(B)·εₜ where B is the backshift operator

AR I MA
AutoRegressive (p) Integrated (d) Moving Average (q)

Xₜ = φ₁Xₜ₋₁ + ... + φₚXₜ₋ₚ + εₜ ΔᵈXₜ = Xₜ - Xₜ₋₁ (d times) Xₜ = εₜ + θ₁εₜ₋₁ + ... + θqεₜ₋q

• Regresses series on its own past p • d = number of non-seasonal • Regresses series on past q error terms
values differences • q = order of moving average
• p = order of autoregression • Makes non-stationary series stationary • Identified via ACF plot
• Identified via PACF plot • Determined via ADF / KPSS tests • ACF cuts off after lag q
• PACF cuts off after lag p • d = 0 for stationary series (ARMA)
BO X-JENK I NS M ETHO DOLO GY : MO DEL B UI LDI NG

1 Identification 2 Estimation

Estimate model parameters using Maximum Likelihood or

Plot data. Check stationarity with ADF or KPSS test. Difference if
Conditional Least Squares. Compare AIC / BIC across candidate
needed. Use ACF & PACF to tentatively identify p and q.
models.

3 Diagnostic Checking 4 Forecasting

Examine residuals: check ACF/PACF of residuals, Ljung-Box test for Generate point forecasts and prediction intervals. Monitor forecast
white noise, QQ-plot for normality. Revise if needed. errors. Update model as new data arrives (rolling window).

↺ Iterative process — revisit previous steps if diagnostics fail

A CF & PA CF: M ODEL IDENTI FI CATI ON GU ID E

Model ACF Pattern PACF Pattern

AR(p) Tails off (decaying/oscillating exponentially) Cuts off after lag p

MA(q) Cuts off after lag q Tails off (decaying/oscillating exponentially)

ARMA(p,q) Tails off after lag q Tails off after lag p

White Noise No significant spikes at any lag No significant spikes at any lag

Random Walk Very slow decay (near unit root) Large spike at lag 1 only

ARIMA(p,1,q) Slowly decaying until differencing Large spike at lag 1

Spikes outside the 95% confidence bands (±1.96/√n) are considered statistically significant
C O M P AR A TI V E S U M MA R Y

Method Strength Best Use Case Limitation

Stochastic Process Theoretical foundation Probabilistic modeling & simulation Abstract; needs large data

Moving Average Simple & interpretable Smoothing & noise reduction Lag; doesn't capture cycles

Regression Causal interpretation Trend & seasonal modeling Assumes independence of errors

ARIMA Flexible & accurate Short-term univariate forecasts Stationary data required
 KEY TAKEAWAYS

Stochastic Processes provide the mathematical framework — stationarity and autocorrelation are fundamental.

Moving Averages (SMA, WMA, EMA) are powerful smoothing tools; α and window size control responsiveness.

Regression models capture trends and seasonality but require careful handling of autocorrelated errors.

ARIMA is the comprehensive Box-Jenkins framework: identify p, d, q using ACF/PACF → estimate → diagnose →
forecast.

Thank You