0542-5101: Advanced Multivariate Analysis

Dr. Md. Rezaul Karim

PhD(KULeuven), MS(Biostatistics, UHasselt), MS(Statistics, JU)
Professor, Department of Statistics and Data Science
Jahangirnagar University (JU), Savar, Dhaka - 1342, Bangladesh
Prof. dr. M. R. Karim
Email: rezaul@[Link]

August 14, 2025

1
These course slides should not be reproduced nor used by others (without
permission).

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 1 / 52
First...

Textbook

1 Verbeke, G., & Molenberghs, G. (Eds.). (2012). Linear mixed models

in practice: a SAS-oriented approach (Vol. 126). Springer Science &
Business Media.
Prof. dr. M. R. Karim
2 Michael Kutner, Christopher Nachtsheim, John Neter, William Li
(2004): Applied Linear Statistical Models. 5th edition. New York:
McGraw-Hill/Irwin.

3 W. Holmes Finch, Jocelyn E. Bolin, and Ken Kelley (2019):

Multilevel Modeling Using R, CRC Press, Taylor & Francis Group.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 2 / 52
Lecture Outline I

1 Chapter 5: Generalized Estimating Equations (GEE)

1.1 Generalized Estimating Equations (GEE)

1.2 Likelihood Function in GEE

1.3 GEE Estimation Prof. dr. M. R. Karim

1.4 Selecting the Best Correlation Matrix for GEE

1.5 Practical Problem - 8

1.6 Practical Problem - 9

1.7 Practical Problem - 10

1.8 Practical Problem - 11

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 3 / 52
 Chapter 5: Generalized Estimating Equations (GEE)

Chapter 5: Generalized Estimating Equations (GEE)

Prof. dr. M. R. Karim

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 4 / 52
 Chapter 5: Generalized Estimating Equations (GEE)

1 Chapter 5: Generalized Estimating Equations (GEE)

1.1 Generalized Estimating Equations (GEE)

1.2 Likelihood Function in GEE

1.3 GEE Estimation

1.4 Selecting the Best Correlation

Prof. dr. M. [Link]
Karim for GEE

1.5 Practical Problem - 8

1.6 Practical Problem - 9

1.7 Practical Problem - 10

1.8 Practical Problem - 11

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 5 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Generalized Estimating Equations (GEE)

Overview of GEE
Generalized Estimating Equations (GEE) is a statistical method used to
estimate the parameters of a generalized linear model with correlated data,
particularly in longitudinal or clustered settings. The key purpose of GEE
is to model the population-averaged response, focusing on the average
eect of a predictor across the entire population, while accounting for the
correlation structure within the data. It accounts for the correlation within
subjects or clusters but doesn't
Prof. explicitly
dr. M. [Link]
Karimthe subject-specic eects.
Instead, it uses a "working" correlation structure (e.g., independence,
exchangeable, or autoregressive) as an approximation. GEE can be used
for non-normal data, such as binary (logistic regression) or count data
(Poisson regression).

Analyzes correlated data, particularly in longitudinal studies.

Extends generalized linear models (GLMs) to accommodate

correlations.

Provides robust standard errors.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 6 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Generalized Estimating Equations (GEE)

Model Specication

For a response variable Yi :

µi = E (Yi ∣Xi ) = g −1 (XT

i β)

Prof. dr. M. R. Karim

where:

µi is the expected value of Yi .

g is the link function.

β is the vector of parameters.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 7 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Likelihood Function in GEE

The likelihood function for Generalized Estimating Equations (GEE) is

given by:
n
L(β) = ∏ f (Yi ; µi , φ)
i=1

where,

▸ Yi : Observed [Link] theM.i -th

dr. [Link].
Karim
▸ µi = g −1 (ZT i β): Mean response related to the linear predictor.
▸ f (Yi ; µi , φ): Probability density (or mass) function of the response
variable.
The log-likelihood function is expressed as:

n
L(β) = ∑ log f (Yi ; µi , φ)
i=1

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 8 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Examples

For a binomial response, the likelihood function is:

n
L(β) = ∑ [Yi log(µi ) + (ni − Yi ) log(1 − µi )]
i=1

For a Poisson response, the likelihood function is:

Prof. dr. M. R. Karim
n
L(β) = ∑ [Yi log(µi ) − µi − log(Yi !)]
i=1

For a Gaussian response, the likelihood function is:

n
n 1
L(β) = − log(2πσ 2 ) − ∑(Yi − µi )
2
2 2σ 2 i=1

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 9 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Dierentiating the Log-Likelihood

To nd the rst derivative of the log-likelihood:

∂L n ∂ log f (Yi ; µi , φ) ∂µi

=∑ ⋅
∂β i=1 ∂µi ∂β
Prof. dr. M. R. Karim
Using the Chain Rule:
∂ log f (Yi ; µi , φ) 1 ∂f (Yi ; µi , φ)
= ⋅
∂µi f (Yi ; µi , φ) ∂µi

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 10 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Expectation and Simplication

In many cases:
∂ log f (Yi ; µi , φ) Yi − µi
≈
∂µi φ
Thus:
∂L n Yi − µi ∂µi
= ∑( ⋅ )
∂β i=1 φ ∂β
This leads us to the commonly
Prof. used expression:
dr. M. R. Karim

n
∂L ∂µi
∝∑ (Yi − µi )
∂β i=1 ∂β
Interpretation:
The derivative captures how the likelihood changes with respect to
the parameters β.
(Yi − µi ): Deviation of observed values from expected values.
∂µi
∂β : Change in expected values as parameters vary.
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 11 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Score Function
The estimating equations arise from setting the score function to zero:

n
∂L
U(β) = ∑ =0
i=1 ∂β

1 Working Independence: Assume conditional independence of

observations. Prof. dr. M. R. Karim
2 First Derivative of the Likelihood:

n
∂L ∂µi
∝∑ (Yi − µi )
∂β i=1 ∂β

3 Normalization by Variance:

ZTi Vi−1 (Yi − µi )

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 12 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

where, the design matrix Zi is dened as:

∂µi 1 −1 T ′
Zi = = g (Xi β) Xi
∂β φ

where:
Prof. dr. M. R. Karim
g −1 is the inverse link function.

g′ is the derivative of the link function.

φ is the scale parameter (for certain models).

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 13 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Likelihood Function in GEE

Final Form of the Estimating Equations

Summing over all clusters yields:

n
U(β) = ∑ ZTi Vi−1 (Yi − µi ) = 0
i=1

where Prof. dr. M. R. Karim

Zi : Design matrix for the i -th observation.

µi : Mean response for the i -th cluster, related to β by the link

function.

Yi : Observed response for the i -th cluster.

Vi : Working variance-covariance matrix for the i -th cluster.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 14 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

GEE Estimation

The GEE estimator ̂

β is obtained by solving the estimating equations:

n
U(β) = ∑ ZTi Vi−1 (Yi − µi ) = 0
i=1
Prof. dr. M. R. Karim
where:

n is the total number of observations.

Zi is the vector of derivatives of the link function.

Vi is the working variance-covariance matrix of Yi .

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 15 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Functional Form of Vi

The working variance-covariance matrix Vi depends on the correlation

structure chosen. Common forms include:

Independent Structure: For an independent correlation structure,

the variance-covariance matrix is:

2
⎛σ 0 0 ⋯ 0⎞
Prof. dr. R.2 Karim
⎜ 0M. σ 0 ⋯ 0⎟
⎜ ⎟
Vi =σ I=⎜0 0 σ ⋯ 0⎟
2 ⎜ 2
⎟
⎜ ⋮ ⋮ ⋮ ⋱ ⋮ ⎟
⎜ ⎟
⎝ 0 0 0 ⋯ σ2⎠

Working Independence
Initial parameter estimates are obtained by tting a GLM.

Assumes observations are independent for initial estimates.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 16 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Exchangeable Structure: For an exchangeable correlation structure,

the variance-covariance matrix is:

⎛1 ρ ρ ⋯ ρ⎞
⎜ρ 1 ρ ⋯ ρ⎟
⎜ ⎟
Vi = σdr.⎜ρM. ρR.
Prof.
2⎜
Karim
1 ⋯ ρ⎟⎟
⎜⋮ ⋮ ⋮ ⋱ ⋮⎟
⎜ ⎟
⎝ρ ρ ρ ⋯ 1⎠

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 17 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Autoregressive (AR(1)) Structure For an autoregressive correlation

structure, the variance-covariance matrix is:

⎛ 1 ρ ρ2 ⋯ ρn−1 ⎞
⎜ ρ
⎜ 1 ρ ⋯ ρn−2 ⎟
⎟
Vi 2⎜
= σProf. 2
⎜ ρ dr. M.ρ R. Karim
1 ⋯ ρn−3 ⎟
⎟
⎜ ⋮ ⋮ ⋮ ⋱ ⋮ ⎟
⎜ ⎟
⎝ρn−1 ρn−2 ρn−3 ⋯ 1 ⎠

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 18 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

AUnstructured Structure For an unstructured variance-covariance

matrix, it can be represented as:

2
⎛ σ1 σ12 σ13 ⋯ σ1n ⎞
⎜σ21
⎜ σ22 σ23 ⋯ σ2n ⎟ ⎟
Vi Prof.
=⎜⎜σ31dr. σM.
32
2
R.σKarim
3 ⋯ σ3n ⎟
⎟
⎜ ⋮ ⋮ ⋮ ⋱ ⋮ ⎟
⎜ ⎟
⎝σn1 σn2 σn3 ⋯ σn2 ⎠

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 19 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Toeplitz Structure For a Toeplitz correlation structure, the

variance-covariance matrix is:

2
⎛σ ρ1 ρ2 ⋯ ρn−1 ⎞
2
⎜ ρ1 σ ρ1 ⋯ ρn−2 ⎟
⎜ ⎟
Vi =Prof.
⎜ ρ2 dr. ρM.
⎜ 1 R. σ
2
Karim⋯ ρn−3 ⎟
⎟
⎜ ⋮ ⋮ ⋮ ⋱ ⋮ ⎟
⎜ ⎟
⎝ρn−1 ρn−2 ρn−3 ⋯ σ 2 ⎠

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 20 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Correlation Structure

GEE uses a working correlation matrix R to model correlations among

observations. Common structures include:

Independent: Assumes no correlation between observations.

Exchangeable: Assumes constant correlation among all pairs of
observations within a cluster.
Prof. dr. M. R. Karim
Autoregressive (AR(1)): Correlation decreases as the time lag
increases.

Unstructured: Allows for dierent correlations for each pair of

observations.

Toeplitz: A structured form where correlations depend only on the

distance between observations.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 21 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Asymptotic Results of GEE Estimator

The GEE estimator ̂
β has several important asymptotic properties as the
sample size n increases:

Consistency The GEE estimator is consistent:

̂ÐP
β → β0

This means that as ̂

n → ∞, β converges in probability to the true
Prof. dr. M. R. Karim
parameter value β0 .
Asymptotic Normality The estimator β̂ is asymptotically normally
distributed:
→ N(0, C)
√ d
̂−β )Ð
n(β 0

where C is the asymptotic covariance matrix given by:

n −1 n n −1
C = (∑ ZTi Vi−1 Zi ) ∑ Zi
T
Vi−1 Var(Yi )Vi−1 Zi (∑ ZTi Vi−1 Zi )
i=1 i=1 i=1

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 22 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Eciency The GEE estimator is asymptotically ecient:

̂
Var(β) ̂
≥ Var(β MLE )

This means it achieves the lowest possible asymptotic variance among

estimators that do not use the correct correlation structure.

Robustness Prof. dr. M. R. Karim

GEE provides robust parameter estimates:

▸ Estimates remain valid even when the specied working correlation

structure is incorrect.
▸ Consistency and asymptotic normality hold under mild conditions.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 23 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Robust Variance Estimator

The robust variance estimator for GEE is given by:

n −1 n n −1
̂ = (∑ ZT V−1 Zi )
S(β) i i (∑ Z T
i V Ui U V Z
−1
i
T
i
−1
i i ) (∑ Z V Z
T
i
−1
i i)
i=1 i=1 i=1

where
Ui = Yi − µi
Prof. dr. M. R. Karim
Role in Robust Variance Estimation
The term Ui captures the dierence between the observed and expected
values, reecting how the data deviate from the estimated model.
Importance:
It adjusts the variance estimation to account for potential correlation
in the data.

The term Ui UTi contributes to estimating the variability of the score

function.
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 24 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Sandwich Estimator
The sandwich estimator for the variance of an estimator ̂
β is expressed as:

̂
Var(β) ≈ H−1 DH−1
Where:
H: Hessian matrix (information matrix):
n
H = ∑ ZTi Vi−1 Zi
Prof. dr. i=
M. 1 R. Karim

D: Empirical estimate of the variance of the score function:

n
D = ∑ ZTi Vi−1 Ui UTi Vi−1 Zi
i=1
Advantages of the Sandwich Estimator
Robustness: Provides valid variance estimates even when model
assumptions (e.g., independence) are violated.
Flexibility: Applicable in various contexts, including GEE, logistic
regression, and generalized linear models.
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 25 / 52
 Chapter 5: Generalized Estimating Equations (GEE) GEE Estimation

Running GEE in R

To run a Generalized Estimating Equations (GEE) analysis in R, you can

use the geepack package. Below is an example:

# Install and load the geepack package

[Link]("geepack")
library(geepack)
Prof. dr. M. R. Karim
# Fit a GEE model
gee_model <- geeglm(y ~ x1 + x2, data = data, id = id,
family = binomial(link = "logit"), corstr =
"exchangeable")

# Summarize the results

summary(gee_model)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 26 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Selecting the Best Correlation Matrix for GEE

Selecting the Best Correlation Matrix for GEE

1. Understanding Correlation Structures

Independent: No correlation between observations.
Exchangeable (Compound Symmetry): Equal correlation among
all observations within a cluster.

Autoregressive (AR-1): Correlation decreases with increasing time

lag. Prof. dr. M. R. Karim

Unstructured: Unique correlation for each pair of observations.

2. Model Fitting
Fit GEE models using dierent correlation structures.

Estimate regression coecients while accounting for specied

correlation.

Use quasi-likelihood methods for estimation.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 27 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Selecting the Best Correlation Matrix for GEE

3. Model Comparison Using Information Criteria

QIC (Quasi-likelihood under Independence model Criterion):
Generalization of AIC for GEE.

QICu: Adjusted version of QIC for small sample sizes.

Selection: Choose model with the lowest QIC or QICu.
Prof. dr. M. R. Karim
4. Model Diagnostics
Check residuals for random distribution.

Assess robustness of the chosen structure.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 28 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Selecting the Best Correlation Matrix for GEE

5. Consideration of Data Characteristics

Use AR structure for time-ordered data.

Consider exchangeable or independent structures for clustered data.

6. Practical Considerations
Prof. dr. M. R. Karim
Computational feasibility: Unstructured models can be intensive.

Interpretability: Simpler structures may be preferred for ease of

understanding.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 29 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Selecting the Best Correlation Matrix for GEE

Identifying Inuential Observations and Outliers

Identifying inuential observations is crucial for valid model

conclusions:

▸ Cook's Distance : A measure to identify inuential data points that

disproportionately aect the model.
▸ : Standardized
Residual AnalysisProf. dr. M. [Link]
Karim can help detect outliers;
values beyond ±3 are often considered inuential.
▸ Leverage Points: Monitor observations with extreme predictor values;
these can greatly aect the estimated coecients.
Addressing outliers may involve model renement or data
transformations.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 30 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

Practical Problem - 8

he study focuses on worldwide Salmonellosis, a common zoonotic infection

primarily transmitted through animals, with 26% of cases linked to pork.
The analysis involves a dataset, [Link], which includes
variables such as weight, SP (sample
Prof. to R.
dr. M. positive
Karimratio), and days. The goal
is to examine how SP-ratios, which range from 0 to 4 (with values above
0.5 indicating Salmonella-positive animals), depend on the covariates of
weight and time, creating a binary response variable for infected and
susceptible animals.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 31 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

That is,
1 if SP-ratio > 0.4
yi = {
0 if SP-ratio ≤ 0. 4.
The time variable days is categorized in seasons as follows (1 referring to
winter, etc):
⎧
⎪ 1 0≤ days ≤ 90
⎪
⎪
⎪
⎪ 2 91≤ days ≤ 181
xi = ⎨
⎪
⎪ 3 182 ≤ days ≤ 273
Prof.
⎪
⎪ dr. M. R. Karim
⎪
⎩ 4 274 ≤ days ≤ 365.

We treat all variables (SP-ratio, sampling time, and weight) as categorical

variables.

(a) Fit a logistic regression model for yi with season and weight as
explanatory variables. Select a best tting model and interpret.

(b) Repeat exercise (a), but now taking the clustered nature of the data
(herds as clusters) into account. (hints: you can use GEE).

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 32 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

R Code
# Load necessary libraries
library(geepack)

# Import the dataset from [Link]

data <- [Link]("[Link]", header = TRUE)

# Check the structure of the data

str(data)

# Create binary response variable

Prof. based
dr. [Link] R.
SP Karim
data$y <- ifelse(data$SP > 0.4, 1, 0)

# Fit logistic regression model

logistic_model <- glm(y ~ factor(weight) + factor(days), data
= data, family = binomial)
summary(logistic_model)

# Fit GEE model accounting for clustering by herd_id

gee_model <- geeglm(y ~ factor(weight) + factor(days), data =
data, id = herd_id, family = binomial, corstr =
"exchangeable")
summary(gee_model)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 33 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

Selecting the Best Correlation Structure for GEE

1. Fit GEE Models with Dierent Structures
library(geepack)

# Fit models with different correlation structures

gee_independent <- geeglm(y ~ factor(weight) + factor(days),
data = data, id = herd_id, family = binomial, corstr =
"independence") Prof. dr. M. R. Karim
gee_exchangeable <- geeglm(y ~ factor(weight) +
factor(days), data = data, id = herd_id, family =
binomial, corstr = "exchangeable")
gee_ar1 <- geeglm(y ~ factor(weight) + factor(days), data =
data, id = herd_id, family = binomial, corstr = "AR-M")
gee_unstructured <- geeglm(y ~ factor(weight) +
factor(days), data = data, id = herd_id, family =
binomial, corstr = "unstructured")

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 34 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

2. Compare Models Using Quasi-Likelihood Criteria

# Calculate QIC for each model
qic_ind <- QIC(gee_independent)
qic_exch <- QIC(gee_exchangeable)
qic_ar1 <- QIC(gee_ar1)
qic_unstr <- QIC(gee_unstructured)
Prof. dr. M. R. Karim
# Compare QIC values
qic_values <- c(independent = qic_ind, exchangeable =
qic_exch, AR1 = qic_ar1, unstructured = qic_unstr)
print(qic_values)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 35 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 8

3. Select the Structure with the Lowest QIC

The model with the lowest QIC value is preferred.

4. Check Model Diagnostics

Prof. dr. M. R. Karim
Perform residual diagnostics to ensure assumptions are met.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 36 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 9

Practical Problem - 9
You are provided with a dataset ( [Link]) for a Generalized
Estimating Equations (GEE) analysis. The dataset consists of repeated
measures from 100 patients, with data collected at 5 dierent time points.
The key variables in the dataset are:

patient_id: A unique identier for each patient (1 to 100).

time: The time pointProf.
of the
[Link]
M. R. Karim(1 to 5).
age: The age of the patient.
gender: The gender of the patient (binary variable: 0 = Female, 1 =
Male).

response: A binary outcome variable indicating whether the patient

experienced a health event (success = 1) or did not experience the
event (failure = 0) at the given time point, based on factors such as
treatment eectiveness or disease progression.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 37 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 9

1 Using the dataset, specify a GEE model to analyze the relationship

between the binary response variable and the covariates (time, age,
and gender). Indicate the link function and correlation structure you
would use.

2 Write the R code to t the specied GEE model to the dataset.

Include the necessary libraries and explain each step of your code.

3 [Link].
Interpret your coecients theM. R. Karim
context of the dataset. What do
they imply about the relationship between the predictors and the
binary response?

4 Conduct a hypothesis test for the eect of time on the binary

response. Describe the null and alternative hypotheses, and explain
how you would interpret the p -value obtained from the GEE output.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 38 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 9

5 Discuss the importance of checking the correlation structure in GEE

models. How would you assess whether your chosen correlation
structure is appropriate for this dataset?

6 Suppose the average age of the patients increases by 5 years. How

would you expect thisProf. dr. M.
to impact [Link]
the Karim probabilities of the
binary response? Explain your reasoning using the coecients from
your GEE model.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 39 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

Practical Problem - 10
The dataset ([Link]) used in this analysis consists of information
collected from patients visiting multiple clinics. The variables in the
dataset are as follows:

Patient_ID: Unique identier for each patient.

Clinic_ID: Identier for the clinic where the patient was seen (1 to
5).

Number_of_Visits:Prof.
[Link]. R. Karim
doctor visits made by each patient
over a year (count data).

Age: Age of the patient (in years), ranging from 30 to 70.

Gender: Gender of the patient, either Male (M) or Female (F).
Treatment_Group: Indicates whether the patient belongs to the
Control group or the Treatment group.

The study aims to assess the eect of the treatment on the number of
doctor visits while controlling for age and gender. The data exhibits a
hierarchical structure, with patients nested within clinics.
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 40 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

Questions on GEE Analysis

1 Descriptive Statistics
1 Calculate and report the mean and standard deviation of the number of
visits for each treatment group (Control and Treatment).
2 What is the distribution of ages in the dataset? Create a histogram to
visualize it.
2 Statistical Modeling
1 Fit a Generalized Estimating Equations
Prof. dr. M. (GEE) model to examine the
R. Karim
eect of treatment group on the number of visits, controlling for age
and gender. Present the R code you used and summarize the results.
2 Interpret the coecient for the Treatment_GroupTreatment variable
in the context of the study.
3 Hypothesis Testing
1 State the null and alternative hypotheses for the eect of the
treatment group on the number of visits.
2 Based on the GEE model results, do you reject or fail to reject the null
hypothesis? Justify your answer using the p-value.
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 41 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

4 Model Diagnostics
1 Discuss the implications of using an exchangeable correlation structure
in the GEE model. What does this assumption imply about the data?
2 If you suspected that the correlation structure might not be
exchangeable, suggest an alternative structure and explain your
reasoning.
5 Visualization
1 Create a boxplot to compare the number of visits between the Control
and Treatment groups.
Prof. Interpret
dr. M. [Link]
results.
2 Generate a scatter plot to visualize the relationship between age and
the number of visits. Discuss any patterns you observe.
6 Advanced Analysis
1 If you wanted to explore the eect of an additional variable (e.g.,
socioeconomic status) on the number of visits, how would you modify
the existing model? Describe the steps you would take.
2 Discuss potential confounding variables in this study and how you
would control for them in your analysis.

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 42 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

R Code for the Solution

# [Link]("ggplot2")
library(ggplot2)

# 1. Descriptive Statistics
# a. Mean and standard deviation of number of visits for each
treatment group
summary_stats <- aggregate(Number_of_Visits ~ Treatment_Group,
data = GEECount, Prof. dr. M. R. Karim

FUN = function(x) c(mean = mean(x), sd = sd(x)))

print(summary_stats)

# b. Histogram of ages
ggplot(GEECount, aes(x = Age)) +
geom_histogram(binwidth = 2, fill = "blue", color = "black",
alpha = 0.7) +
labs(title = "Distribution of Ages", x = "Age", y = "Frequency")
+ theme_minimal()
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 43 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

# 2. Statistical Modeling
# a. Fit the GEE model
gee_model <- geeglm(Number_of_Visits ~ Treatment_Group + Age +
Gender, data = GEECount, id = Clinic_ID, family =
poisson(link = "log"), corstr = "exchangeable")
Prof. dr. M. R. Karim
summary(gee_model)

# b. Interpret the coefficient for Treatment_GroupTreatment

# (This would be done in the interpretation part of the exam)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 44 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

# 3. Hypothesis Testing
# a. Null and alternative hypotheses
# H0: Treatment has no effect on the number of visits.
# H1: Treatment has a significant effect on the number of visits.

# b. Decision based on p-value

# Check p-value for Treatment_GroupTreatment
Prof. dr. M. R. Karim
from the model
summary
# If p < 0.05, reject H0; otherwise, fail to reject.

# 4. Model Diagnostics
# Discuss implications (this would be a written answer)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 45 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 10

# 5. Visualization
# a. Boxplot for number of visits by treatment group
ggplot(GEECount, aes(x = Treatment_Group, y = Number_of_Visits))
+ geom_boxplot(fill = "lightgreen") +
labs(title = "Number of Visits by Treatment Group", x =
"Treatment Group", y = "Number of Visits") +
theme_minimal()

# b. Scatter plot of Age vs. Number of Visits

ggplot(GEECount, aes(x Prof.
= Age,dr.y M.
= Number_of_Visits))
R. Karim +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", color = "red") +
labs(title = "Scatter Plot of Age vs. Number of Visits", x =
"Age", y = "Number of Visits") +
theme_minimal()

# 6. Advanced Analysis
# Discuss potential additional variable (written answer)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 46 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

Practical Problem - 11
Worldwide Salmonellosis, the illness from Salmonella infection, is the most
frequently occurring zoonoses, which is an infectious disease directly or
indirectly transmitted between animals and humans. Of the reported
Salmonella infections 26% of these are likely to be due to transmission of
the pathogen via pork. MAP is determined by the sample to positive (SP)
ratio for each sample, where S = optical density of the sample  optical
density of the negative. SP-ratios normally cover a range between 0 and 4,
Prof. dr. M. R. Karim
but even higher values can be observed. The dataset
[Link] contains a random subset of a (very) large
sample.

weight SP days herd_id

1 3 0.06 3 688
2 3 0.10 3 688
3 3 0.14 3 688
4 3 0.34 3 688
5 3 0.03 3 688
Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 47 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

The variables are dened as: weight, SP, days. he main objective is to
examine the dependency of the SP-ratios on the covariates weight and
time. The time variable days is categorized in seasons as follows (1
referring to winter, etc):

⎧
⎪ 1 0≤ days ≤ 90
⎪
⎪
⎪
⎪ 2 91≤ days ≤ 181
xi = ⎨
⎪
⎪ 3 182 ≤ days ≤ 273
⎪
⎪
⎪
⎩ 4 274 ≤ days ≤ 365.
Prof. dr. M. R. Karim
We treat all variables (SP-ratio, sampling time, and weight) as categorical
variables. As the infection status of an animal with SP-ratio between 0.5
and 1 is doubtful, we include a middle category inconclusive leading to
the trinomial response

⎧
⎪ 1 SP-ratio ≤ 0.4
⎪
⎪
yi = ⎨ 2 0 .4 < SP-ratio ≤ 0.9
⎪
⎪
⎪
⎩ 3 SP-ratio > 0.9

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 48 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

(i). Fit multicategory logit models for yi with season and weight as
explanatory variables. Which model(s) do you prefer and why?

(ii). Repeat exercise (a), but now taking the clustered nature of the data
Prof. dr. M. R. Karim
(herds as clusters) into account. (hints: use GLMM model).

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 49 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

Solution of Problem 11
# Load necessary libraries
library(tidyverse)
library(nnet) # For the multinom function
library(lme4) # For fitting GLMM

# Load the dataset

data <- [Link]("[Link]")
Prof. dr. M. R. Karim

# Create categorical variables for days (seasons)

data$season <- cut(data$days,
breaks = c(0, 90, 181, 273, 365),
labels = c("1", "2", "3", "4"))

# Create categorical variable for SP-ratio

data$SP_category <- cut(data$SP,
breaks = c(-Inf, 0.4, 0.9, Inf),
labels = c("1", "2", "3"))

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 50 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

# Check the data

head(data)
# Convert categorical variables to factors
data$season <- [Link](data$season)
data$weight <- [Link](data$weight)
data$y <- [Link](data$y)

# (i) Fit multicategory logit models for y with season and

weight as explanatory variables
# Load the required Prof.
library [Link]
dr. M. Karim logistic
regression
library(nnet)

# Fit the multinomial logistic regression model

model1 <- multinom(SP_category ~ season + factor(weight),
data = data)

# Summary of the model

summary(model1)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 51 / 52
 Chapter 5: Generalized Estimating Equations (GEE) Practical Problem - 11

# (ii) Fit a GLMM model taking into account the clustered

nature of the data
# Load the ordinal package
library(ordinal)
# Convert categorical variables to factors
data$season <- factor(data$season)
data$weight <- factor(data$weight)
data$SP_category <- factor(data$SP_category, levels = c("1",
"2", "3"))

# Check the structure Prof.

of the
[Link]
M. R. Karim
str(data)

# Fit the cumulative link mixed model

model_clmm <- clmm(SP_category ~ season + weight + (1 |
herd_id),
data = data,
link = "logit") # or "probit", depending on your preference

# Summary of the model

summary(model_clmm)

Prof. Dr. Md. Rezaul Karim, Department of Statistics and Data Science, JU. August 14, 2025 52 / 52