Introduction to Statistical Methods
in Meta-Analysis
Jian Wang
October 31, 2013
What is meta-analysis?
Meta-analysis: the statistical synthesis of information
from multiple independent studies.
Increase powers and reduce false-positive findings
Advantages:
Results can be generalized to a larger population
Can use summary data (no sharing individual-level data)
The precision and accuracy of estimates can be improved
......
Pitfalls:
Sources of bias are not controlled by the method: a good
meta-analysis of badly designed studies will still result in bad
statistics
Publication bias: studies show negative or insignificant results
are less likely to be published
……
What is meta-analysis?
General steps in meta-analysis:
Formulation of the problem
Literature search
Selection of studies: e.g. selection of specific studies on a well-
specified subject
Decide summary measures or dependent variables: differences,
means, OR, or relative risk
Statistical analysis
History:
A historical instance of Meta-analysis dates back to the twelfth
century in China, a famous philosopher, Chu Hsi (朱熹,
1130~1200), built up his philosophical theory by summarizing a
series of related literatures. He called this research methodology
'Theory of Systematic Rule'(道統論).
Karl Pearson analyzed the data from five studies on the
correlation between the vaccination for enteristic fever and its
mortality.
Fisher’s method
Combine p values from independent tests bearing upon the
same overall hypotheses:
When the p values tend to be small, the test statistic will be
large suggesting that the null hypotheses are not true for
every test
Under null (all null hypotheses are true) and when all p
values are independent, it is a chi-squared distribution with
2m degrees of freedom.
Extend to dependent tests
Scaled chi-squared distribution random variable
Brown’s method: known covariance
Kost’s method: unknown covariance
Z score method
Limitations of combining p values
Combining p values may be spurious when the direction of
effects in the combined studies is not consistent
Not straight forward to include weights
Combine Z scores:
wi is the square root of sample size of the ith study.
Zi=Φ-1(1-pi)
Limitations
Can not provide an overall estimate of effect size
Can not address between-studies heterogeneity
Fixed Effect Model
Most popular
Weighted average of effect sizes from a series of
studies
Yi is effect size of study i, such as logarithm of ORs,
beta-coefficients, mean difference or standardized
mean difference for a continuous phenotype
The inverse of the studies’ variance is commonly used
as study weight, such that larger studies tend to
contribute more.
wi=1/vi, vi is the variance of study i.
Z=M/sqrt(VM) is used to test the null hypothesis
Heterogeneity
Fixed effect model:
Assumes all studies in the analysis share a common
underlying true effect
All observed variance reflects sampling error within study
Weights are assigned with the goal of minimizing this within-
study error.
Heterogeneity:
Sources of heterogeneity:
Some phenotypes are difficult to define and standardize,
e.g., behavioral traits
Effect size might be higher in studies when individuals are
older, or more educated or healthier
Genetic studies: different ethnicity groups, different
genotyping platform or imputation software
In this case, there may be different underlying true effect
sizes for different studies
Test for heterogeneity
Cochran’s Q test:
Under null, it is approximately distributed as a chi-square
with k-1 degrees of freedom.
Not powerful when number of studies is small or within-
study variance is large
It can not be used to estimate the magnitude of true
variance
Quantifying heterogeneity
I2:
Expect value of Q on the assumption that all studies share a
common effect size is df
Q-df is the excess variation. The part that will be attributed to
differences in the true effects from study to study
Describes the percentage of total variation across studies that
is due to heterogeneity rather than chance.
Not directly affect by the number of studies
A measure of inconsistency across the findings of the studies
and not as a measure of the real variation across the
underlying true effects
A value of 0% indicates no observed heterogeneity
Low, moderate, large and very large for 0-25%, 25-50%, 50-
75% and >75%
Random effect model
Incorporate the between-study variance
DerSimonian-Laird method for between-study
variance
Total variance
wi*=1/vi*, vi* is the total variance of study i.
Random effect model
Fix or random?
Difference opinions:
Random effect model: robust
Conservative
Between-studies variance have poor precision
Heterogeneity test is under powered
Fixed effect model: powerful
Not realistic
False positive increases
Bayesian approach for estimating between-
studies variance from outside of the current
set of studies.
Depend on the priors
Fix or random?
Start with a fixed
effect model and then
switch to a random
effect model based
on heterogeneity
test?
Heterogeneity tests
often suffers from
low powers
Decision should be
based on our
understanding of
whether or not all
studies share a
common effect size,
and not on the
outcome of a
statistical test
Meta-analysis in GWAS
Genetic effects due to common alleles are small, and detection of
signals requires large sample sizes
Single GWAS are underpowered due to the sample size
Meta-analysis has become a popular approach for the discovery of new
genetic loci for common diseases and phenotypes
Several hundred GWAS meta-analyses have already been published.
Most studies so far used fixed effect model: p values are
important for the publication
Start with a fixed effects model but to report the random effects
model when heterogeneity is found.