This document rigorously organizes the foundational concepts, modeling objectives, and

decomposition techniques of time series analysis, drawing exclusively from the provided
sources.

I. Foundations and Objectives of Time Series Analysis

A. Definition and Context
A time series is a sequence of observations recorded in time order, typically at equal intervals,
where the ordering matters because nearby values are often related. A time series can also be
defined as a collection of random variables indexed according to the order they are obtained in
time (X t).
Key Objectives of Analysis:
1. Description: Look for trend, seasonality, cycles, changes in level, turning points, missing
values, and outliers, often starting with a time plot. Simple summaries (like mean,
variance, ACF, PACF) help determine if a simple trend/seasonal description is sufficient
or if a richer stochastic model is required.
2. Explanation: When multiple series are observed, determine if variation in one series
helps account for variation in another (e.g., sales vs. price).
3. Prediction (Forecasting): Use the historical pattern to forecast future values and
quantify uncertainty, reporting both point forecasts and prediction intervals.
4. Control: Monitor a process to keep it on target, often using control charts or model-
based approaches to proactively adjust inputs and prevent drift.
B. Time Index and Frequency
A time series is observed at times t ∈ T (the time index).

Concept Definition Common Examples

Sampling Interval ( Δ t ) The gap between 1 day, 1 month.
consecutive, equally
spaced times.
Frequency ( s) The number of Monthly: s=12;
observations per chosen Quarterly: s=4 ; Daily
time unit. A seasonal temperature (annual
pattern repeats roughly cycle): s ≈ 365.
every s steps, meaning
Y t and Y t − s are expected
to be related.
C. Stochastic Processes and Realization
The primary objective of time series analysis is to develop mathematical models that provide
plausible descriptions for sample data that fluctuate over time. These models treat data as
realizations of a stochastic process, providing a probabilistic framework.

Concept Definition/Role
Stochastic Process ({Xt :t ∈ T }) A collection of random variables
indexed by time. It is the underlying
probabilistic mechanism that specifies
the probability laws (mean, variance,
autocovariance) and can generate
many possible paths.
Random Variable (X t) Each individual point in the process is
random.
Realization (Sample Path) The single, specific sequence
(outcome) we actually observe (e.g.,
the actual weather record from 2000–
2025). We often use lowercase y t or x t
for the observed value.

Inference: We only observe one realization, but we model the underlying process to infer
structure (trend, seasonality, dependence), estimate parameters (mean, variance, AR
coefficients), and forecast future values.

D. Capabilities of Time Series Models

Time series models serve five primary functions:
1. Capture Patterns in Data: Identify and model structural components like Trend (long-
term direction), Seasonality (regular cycles), and Autocorrelation (dependence across
time).
2. Provide a Probabilistic Framework: Treat data as realizations of a stochastic process
and specify the mean, variance, and autocovariance.
3. Support Forecasting: Predict future values based on past behavior and quantify
uncertainty with confidence intervals.
4. Extract Signals & Reduce Noise: Distinguish systematic structure from random
variation using methods like filtering and smoothing (e.g., Kalman filter).
5. Enable Statistical Inference: Test hypotheses about interventions, policies, cycles, or
hidden structures in data (e.g., impact of new regulation on unemployment rates).
II. Measures of Dependence and Stationarity
A. Measures of Dependence
These measures provide summary statistics for dependence and guide model identification (e.g.,
AR vs. MA).

Measure Definition Key Property

Mean Function ( μt ) The expected value at Represents the "average
time t : μt =E[X t ]. level" of the process. If
μt is constant, the series
has a constant mean.
Autocovariance Measures linear Reduces to the variance
Function (γ (s , t)) dependence between two when s=t :
points (X s and X t) in the γ (t ,t )=V a r (Xt ). It is
same series: large when X s and X t
γ (s , t)=C o v (Xs , Xt )=E[(X s )(X t − μt )]
s − μtogether.
move
.
Autocorrelation Measures the linear Bounded between −1
Function (ACF) ( ρ(s , t) predictability of the and 1: −1 ≤ ρ(s , t)≤ 1.
) series at time t from the ρ(h)=C o r r (Xt , Xt −h )
value at time s. Defined gives the correlation at
as lag h.
γ (s , t)
ρ(s , t)= .
√ γ (s , s )γ (t ,t )
B. Stationary Time Series
Stationarity is crucial because many modeling and inference techniques rely on it. It captures
regularity in the behavior of a time series.
1. Strict Stationarity (Strong Condition)
A series is strictly stationary if the probabilistic behavior of every collection of values
{Xt 1 , Xt 2 , … , Xt k } is identical to that of the time-shifted set {Xt 1 +h , Xt 2+h , … , Xt k+h } for all time
points, constants, and time shifts h.
Implications:
 The entire joint distribution of the process does not change over time.
 The marginal distribution at time s is the same as at time t .
2. Weak Stationarity (Second-Order Stationarity)
A weakly stationary time series {Xt } is a finite variance process where only the first and second
moments are stable.
Conditions:
 1. Constant Mean Function: The mean function μt =E[X t ]=μ is constant and does not
depend on time t .
2. Lag-Dependent Autocovariance: The autocovariance function γ (s , t) depends only on
the time difference (lag) ¿ s −t∨¿, not on the specific times s and t : γ (s , t)=γ ¿.
Interpretation:
 The mean level of the process is stable across time.
 The variance γ (t ,t ) is constant and finite.
 The dependence structure is time-invariant: correlations depend only on the lag between
points.
3. Relationship
 Every strictly stationary process with finite variance is also weakly stationary.
 Not every weakly stationary process is strictly stationary.
 Special Case: If the process is Gaussian, then weak stationarity implies strict
stationarity.

III. Time Series Models (AR and White Noise)

A. White Noise
A white noise process ({wt }) is a sequence that represents pure randomness in time series. It
serves as the error term (innovations) and the building block for AR, MA, ARMA, and ARIMA
models.
Properties:
 Zero Mean: E [wt ]=0.
 Constant Variance: V a r (wt )=σ 2 (constant over time).
 No Autocorrelation: C o v (wt , w s )=0 for t ≠ s.
 If w t ∼ N (0 , σ 2), it is called Gaussian white noise.

Modeling Criterion: A good model’s residuals should resemble white noise.

B. Autoregressive (AR) Models

An AR model expresses the current value of a time series as a linear function of its own past
values plus a random shock.
General Form: AR( p) X t =ϕ1 X t − 1+ ϕ 2 X t −2 +…+ ϕ p X t − p+ wt where w t is the innovation (white
noise, w t ∼ W N (0 , σ 2 )) and p is the order of the AR model.

Understanding AR Coefficients (ϕ )
 Role: ϕ coefficients measure the strength and direction of dependence on past values.
 Interpretation:
 o If ϕ >0 : Past values push the series in the same direction.
o If ϕ <0 : Past values push the series in the opposite direction (can create
oscillations).
o If ϕ=0: That lag has no effect.
 Memory: Together, the ϕ 's describe the memory of the series.
C. Random Walk (RW)
The Random Walk is a fundamental time series model, viewed as an ARIMA (0 , 1, 0) process.

Model (without drift): x t =xt −1 +w t where w t ∼ W N (0 , σ 2 ) and x 0=0 .

Random Walk with Drift (RWD): x t =δ+ x t − 1+ wt The parameter δ adds a constant trend.
Key Properties:
 Nonstationary: The variance grows with time t (the series "wanders").
 Stationarity via Differencing: The first difference ( x t − x t − 1=δ+ wt ) is stationary (white
noise with mean δ ).

IV. Time Series Decomposition Models

A. Purpose and Components
Time Series Decomposition separates a series into several components, each representing a
particular type of behavior over time, to better understand structure, isolate meaningful patterns,
and improve forecasting accuracy.
Three Main Components:
1. Trend-Cycle (T t ): The long-term movement or direction in the data.
2. Seasonal Component (St ): Regular and predictable fluctuations occurring within
specific periods.
3. Remainder (Irregular Component, R t ): Random noise or unexplained variation left
after removing trend and seasonality.
B. Decomposition Model Types
The choice depends on how seasonal fluctuations relate to the mean level.

Model Type Equation When to Use

Additive y t =T t +S t + R t Use when seasonal
swings are about the
same size across the
series (magnitude of
seasonal variation
remains roughly
Model Type Equation When to Use
constant).
Multiplicative y t =T t × S t × R t Use when seasonal
swings grow/shrink with
the level of the series
(seasonal amplitude
increases with the mean
level).

C. Moving Averages (MA)

Moving averages are used to estimate the trend-cycle component by smoothing data. The
average eliminates some randomness, leaving a smooth trend-cycle component.
 Definition: A moving average of order m (m -MA) averages values of the time series
within k periods of t , where m=2 k +1 .
 Smoothing Effect: Replacing an observation with an average of its neighbors (e.g., a 3-
1
point centered MA: v t= (w t −1 + wt + wt +1) ) reduces variability, highlights slower
3
changes, and makes neighboring points dependent (short-range autocorrelation).
Estimating Trend-Cycle from Seasonal Data: If the seasonal period is m :
 If m is odd, use an m -point moving average.
 If m is even, use a 2 ×m moving average (which is equivalent to a weighted MA of order
m+1).
Using the appropriate m -MA (or 2 ×m MA) ensures that seasonal variation is averaged out,
leaving little or no seasonality in the resulting trend-cycle estimate (T^ t).

D. Classical Decomposition Procedure

Classical decomposition originated in the 1920s and assumes that the seasonal component (St ) is
constant from year to year.

Multiplicative
Additive Decomposition Decomposition (
Step ( y t =T t +S t + R t) y t =T t × S t × R t)
1. Estimate Trend- Use an m -MA (if m is Use an m -MA (if m is
Cycle (T^ t ) odd) or 2 ×m -MA (if m odd) or 2 ×m -MA (if m
is even). is even).
2. Detrend the Series Remove the trend by Remove the trend by
subtraction: ^y t − T^ t yt
division: ^ (isolates
(isolates St + R t ). Tt
St × R t ).
3. Estimate Seasonal Average the detrended Average the detrended
 Multiplicative
Additive Decomposition Decomposition (
Step ( y t =T t +S t + R t) y t =T t × S t × R t)
Component ( ^St ) values for each season values for each season to
(e.g., all March values). obtain seasonal indices.
Adjust these averages so Adjust these indexes so
they sum to zero. that they sum to m (i.e.,
their mean is 1).
4. Calculate Remainder Remove both trend and Remove both trend and
(^
Rt ) seasonal estimates by seasonal estimates by
subtraction: yt
Rt = y t − T^ t − S^ t.
^ division: ^Rt = .
T^ t S^ t

E. STL Decomposition
STL (Seasonal–Trend decomposition using LOESS) decomposes a time series into trend,
seasonal, and remainder components using locally weighted regression (LOESS).
Advantages:
 It is flexible (seasonality may change over time).
 It handles any seasonal period (m ).
 It has a robust option (robust =TRUE) to limit the influence of outliers.
 It models the series additively: y t =T t +S t + R t.
Key Controls:
 Seasonal Window: Controls how quickly the seasonal pattern can evolve.
[Link]= periodic forces a fixed, repeating pattern. [Link]=k (odd integer) allows
the seasonal shape to change slowly.
 Trend Window: Controls the smoothness of the trend–cycle. Larger trend window k
results in a smoother trend.
 Robustness: Downweights outliers, improving the stability of St and T t .

V. Interpretation and Quality Checking

A. Interpreting Decomposition Output
Decomposition output typically shows four panels: Original Series, Trend Component (T t ),
Seasonal Component (St ), and Remainder (R t ).

Component Interpretation
Original Series The raw data showing both trend and
seasonal movement.
Trend (T t ) Shows the long-term direction of the
Component Interpretation
data (e.g., consistent exponential
growth or gradual decline). Represents
the underlying baseline free from
seasonal fluctuations.
Seasonal (St ) Captures the recurring pattern every
cycle (e.g., months). Seasonal highs
and lows repeat regularly. In
multiplicative decomposition, indices
(e.g., 1.2 or 0.8) indicate the
percentage above or below the trend.
Remainder (R t ) Random noise and short-term
irregularities unexplained by trend or
seasonality. Should appear patternless.

B. Checking Decomposition Quality

A good decomposition should show:
1. A smooth trend line that reflects long-term changes.
2. A consistent seasonal pattern (stable peaks/troughs per period).
3. A random remainder (no leftover pattern).
The presence of autocorrelation in the remainder implies that the "random" component is not
entirely random yet, and further temporal dependence remains.
Verification using the ACF Plot of the Remainder:
 The ACF (Autocorrelation Function) plot shows the correlation of the remainder series
with itself at specific lags.
 Significance Bounds: The blue dashed lines represent the significance bounds
(approximately ± 1.96/ √ N , where N is the sample size).
 Interpretation:
o If bars exceed the blue dashed lines, there is significant autocorrelation at that lag.
o If most bars are within the blue lines, the series behaves like white noise (no
autocorrelation), indicating the systematic structure has been successfully
removed.
If autocorrelation persists in the residuals (e.g., at lag 1 or lag 12), the next step is often to use a
time series model like ARIMA, which can handle residual correlations more effectively.
Understanding the components helps in deciding whether to difference the data for stationarity
and guides forecasting models.