Meta-analysis using R

(tailored in early 2020)

Adi Wijaya, MKom*)

*) Researcher and Lecturer at STIKES Indonesia Maju, Jakarta. Currently, a Candidate Doctor
in Information Engineering at DTETI-UGM

C Adi Wijaya | adiwjj@[Link]

Why R?
• Free software
• Many ready to use library or package
• Community support
• Able to interact with other programming or
tools, such as: python, Rapidminer, etc.
• Less and neat line of code
• …and many more…

c Adi Wijaya | adiwjj@[Link]

Outline
We will learn about:
1. What is Meta-analysis (MA)
2. What is Systematic Literature Review (SLR)
3. Relation between MA and SLR
4. PRISMA Protocol Guidelines
5. MA Reporting Component
6. Case Study:
“Association between Health Literacy and
Information Seek Behavior among Adults: A
Systematic Literature Review and Meta-
analysis study”
c Adi Wijaya | adiwjj@[Link]
 Let’s begin…

c Adi Wijaya | adiwjj@[Link]

What is Meta-analysis

c Adi Wijaya | adiwjj@[Link]

What is Meta-analysis (MA)
• Glass (1976) first introduce MA: “The analysis of analyses”
• MA is the use of statistical techniques to combine and
summarize the results of multiple studies (Moher et. al., 2015)
• MA is a statistical integration of evidence from multiple
studies that address a common research question (Quintana,
2015)

• By combining data from several studies, MAs can provide

more precise estimates of the effects of health care than
those derived from the individual studies (Moher et. al., 2015)
• MA outcomes are often used to summarize a research
area in an effort to better direct future work (Quintana, 2015)

c Adi Wijaya | adiwjj@[Link]

Strength and Limitation of MA
Strength:
Summarizes a body of research
Objective and transparent
Robust and replicable
Research consolidation
Publication bias estimation

Limitation:
Apples and oranges argument
Garbage in, garbage out.
Source: (Del Re, 2015)

c Adi Wijaya | adiwjj@[Link]

What is Systematic Literature Review (SLR)

c Adi Wijaya | adiwjj@[Link]

What is SLR
• A SLR attempts to collate all relevant evidences that fits
pre-specified eligibility criteria to answer a specific
research question (Moher et. al., 2015)

• A SLR is a means of evaluating and interpreting all

available research relevant to a particular research
question, topic area, or phenomenon of interest.
(Kitchenham & Charters, 2007)

• SLR uses explicit, systematic methods to minimize bias

in the identification, selection, synthesis, and summary
of studies (Moher et. al., 2015)

c Adi Wijaya | adiwjj@[Link]

What is SLR
The key characteristics of a SLR:
o a clearly stated set of objectives with an explicit,
reproducible methodology;
o a systematic search that attempts to identify all studies that
would meet the eligibility criteria;
o an assessment of the validity of the findings of the included
studies (e.g., assessment of risk of bias and confidence in
cumulative estimates);
o a systematic presentation, and synthesis, of the
characteristics and findings of the included studies

c Adi Wijaya | adiwjj@[Link]

Relation between SLR and MA

c Adi Wijaya | adiwjj@[Link]

Relation between SLR and MA
• SLR provide MA eligible papers
• 3 first MA activities (MA Protocol) are related to SLR:
o Study rationale
o Study eligibility criteria
o Search strategy

c Adi Wijaya | adiwjj@[Link]

PRISMA Protocol Guideline

c Adi Wijaya | adiwjj@[Link]

PRISMA Protocol Guideline
• PRISMA: Preferred Reporting Items for Systematic review
and Meta-Analysis

• PRISMA Protocol Guideline:

o Study rationale
o Study eligibility criteria
o Search strategy
o Moderator variables
o Risk of bias
o Statistical approach.

Source: (Shamseer et al., 2015)

c Adi Wijaya | adiwjj@[Link]

Meta-analysis Report Components

c Adi Wijaya | adiwjj@[Link]

Meta-analysis Report Component
1. Study Heterogeneity
→ Q-statistic, I-square, Tau-square
→ Target: No Heterogeneity of Studies

2. Visual Effect size

→ Forest Plot
→ Target: Significant and no bias

3. Publication Bias
→ Funnel Plot, Egger’s Regression test
→ Target: No bias publication

c Adi Wijaya | adiwjj@[Link]

Meta-analysis Report Component
Study Heterogeneity:
• Q-square
o reveal how much of the overall heterogeneity can be attributed to
true between-studies variation
o prone to underestimating heterogeneity in small samples and
overestimating for large samples (Higgins et al., 2003)
• I-square
o not sensitive to the number of studies
o I-square thresholds have been proposed with 25, 50, and
75% representing low, moderate, and high variance, respectively
(Higgins et al., 2003)
• Tau-square
oCan be used to estimate the total amount of study heterogeneity in
random-effects models
oTau-square is zero this is indicative of no heterogeneity

Source: (Quintana, 2015)

c Adi Wijaya | adiwjj@[Link]
Meta-analysis Report Component
Visual Effect Size: → Forest Plot
• Forest plots visualize the effect sizes and CIs from the
included studies, along with the computed summary effect
size
• The summary effect size is represented by the polygon at
the bottom of the plot, with the width of the polygon
representing the 95% CI

Source: (Quintana, 2015)

c Adi Wijaya | adiwjj@[Link]
Meta-analysis Report Component
Publication Bias
• A funnel plot is a visual tool used to examine potential
publication bias in meta-analyses
• if the funnel is uneven with more positively associated
studies to the right of the line this provides evidence for
publication bias
• asymmetry in funnel may reflect other types of bias, such
as study quality, location bias, and study size (Egger et al., 1997)
• An alternative test that is better suited to smaller meta-
analyses (<25 studies) is Egger’s regression test (Egger et al., 1997)

Source: (Quintana, 2015)

c Adi Wijaya | adiwjj@[Link]
Meta-analysis Paper Example

c Adi Wijaya | adiwjj@[Link]

MA Paper Example

c Adi Wijaya | adiwjj@[Link]

MA Paper Example

c Adi Wijaya | adiwjj@[Link]

MA Paper Example

c Adi Wijaya | adiwjj@[Link]

Don’t Worry

c Adi Wijaya | adiwjj@[Link]

 CASE STUDY

c Adi Wijaya | adiwjj@[Link]

Association between Health Literacy
and Information Seek Behavior
among Adults: A Systematic Literature
Review and Meta-analysis study
By: Sobar* & Adi Wijaya^

Contributions:
^ Conceptualization, Data Curation, Formal Analysis, Methodology,
Visualization, Writing-review & editing
* Conceptualization, Data Curation, Funding acquisition, Writing-original draft

 Contribution based on CRediT ([Link]

MA Protocol
• Search strategy
o Database : Science Direct & Pubmed
o Search string : health literacy [title] AND information [title]
o Publication year : 2010 – 2020 (early May, 2020)
o Paper type : Article paper (not survey paper)

• Inclusion-Exclusion criteria
o Inclusion criteria:
▪ Study design : cross-sectional
▪ Variable involved :
✓ Health literacy
✓ Health information (seek or access or behavior)

o Exclusion criteria:
▪ Statistical results : correlation not reported

c Adi Wijaya | adiwjj@[Link]

 PRISMA flow diagram
PubMed ScienceDirect
identification

(143 papers) (32 papers)

Remove Duplicate paper

(23 papers were removed)
screening

108 papers were excluded

Title and Abstract review
for not fulfilling the
(152 papers)
inclusion criteria
eligibility

40 papers were excluded

Full-text paper review
since they were not
(44 papers)
report the correlation test
included

Included in the meta-

analysis
(4 papers)

c Adi Wijaya | adiwjj@[Link]

Data Extraction
1. Authors
2. Publication Year
3. Number of Sample
4. Correlation coefficient (r)
5. Study design
6. Health Literacy (HL) Measurement
7. Health Information Seeking Behavior (HISB) Measurement
8. Average age from sample

c Adi Wijaya | adiwjj@[Link]

Data Extraction, here I come…

c Adi Wijaya | adiwjj@[Link]

Data Extraction – Paper#1

c Adi Wijaya | adiwjj@[Link]

 Paper#1 – Title, Authors, Journal

Kim, S. H., & Utz, S. (2018). Association of health literacy with health information-seeking
preference in older people : A correlational , descriptive study. Nurse Health Science,
20(December 2017), 355–360. [Link]

c Adi Wijaya | adiwjj@[Link]

Paper#1 – Study Design, Number of Sample, Average Age

Study design:
Cross-sectional

Nr. of Sample:
129

Avg. Age:
73.59

c Adi Wijaya | adiwjj@[Link]

Paper#1 – HL and HISB measurement

Information seeking → the Autonomy Preference Index

Health Literacy → S-KHLT

c Adi Wijaya | adiwjj@[Link]

Paper#1 – HL-HISB Correlation Coefficient

Correlation coeff. (r) = 0.24 (p-value = 0.007)

c Adi Wijaya | adiwjj@[Link]

Data Extraction – Paper#2

c Adi Wijaya | adiwjj@[Link]

 Paper#2 – Title, Authors, Journal

Song, S., Lee, S., Jang, S., Lee, Y. J., Kim, N., & Sohn, H. (2017). Mediation effects of
medication information processing and adherence on association between health literacy
and quality of life. BMC Health Service Research, 17(661), 1–11.
[Link]

c Adi Wijaya | adiwjj@[Link]

Paper#2 – Study Design, Number of Sample, Average Age

Study design: Avg. Age: Nr. of Sample:

Cross-sectional 41.76 305

c Adi Wijaya | adiwjj@[Link]

Paper#2 – HL and HISB measurement

HL → REALM

Information
Seeking
Behavior
→ Questionnaire

c Adi Wijaya | adiwjj@[Link]

Paper#2 – HL-HISB Correlation Coefficient

Correlation coeff. (r) = 0.16 (p-value < 0.01)

c Adi Wijaya | adiwjj@[Link]

Data Extraction – Paper#3

c Adi Wijaya | adiwjj@[Link]

Paper#3 – Title, Authors, Journal

Crook, B., Stephens, K. K., Pastorek, A. E., Mackert, M., &

Donovan, E. E. (2015). Sharing Health Information and
Influencing Behavioral Intentions : The Role of Health
Literacy , Information Overload , and the Internet in the
Diffusion of Healthy Heart Information Sharing Health
Information and Influencing Behavioral Intentions : The R.
Health Communication, 0(August 2015), 1–12.
c Adi Wijaya | adiwjj@[Link]
Paper#3 – Study Design, Number of Sample, Average Age

Study design:
Cross-sectional

Nr. of Sample:
180

Avg. Age:
38.68

c Adi Wijaya | adiwjj@[Link]

Paper#3 – HL and HISB measurement

Health
Information
→ Questionnaire

HL → NVS

c Adi Wijaya | adiwjj@[Link]

Paper#3 – HL-HISB Correlation Coefficient

Correlation coeff. (r) = 0.32 (p-value <= 0.001)

c Adi Wijaya | adiwjj@[Link]

Data Extraction – Paper#4

c Adi Wijaya | adiwjj@[Link]

 Paper#4 – Title, Authors, Journal

Taylor, P., Kim, Y., Lim, J. Y., Park, K., Kim, Y., Lim, J. I. Y., & Park, K. (2015). Effects of
Health Literacy and Social Capital on Health Information Behavior Effects of
Health Literacy and Social Capital on Health Information Behavior. Journal of
Health Communication, 1–11. [Link]

c Adi Wijaya | adiwjj@[Link]

Paper#4 – Study Design, Number of Sample, Average Age

Study design:
Cross-sectional

Nr. of Sample:
950

Avg. Age:
NA

c Adi Wijaya | adiwjj@[Link]

Paper#4 – HL and HISB measurement

Health Information Seeking → Questionnaire

HL→ KAHLS
c Adi Wijaya | adiwjj@[Link]
Paper#4 – HL-HISB Correlation Coefficient

Correlation coeff. (r) = 0.211 (p-value < 0.001)

c Adi Wijaya | adiwjj@[Link]

 Tabulasi Included Papers
Study HL Infoseek Mean
Authors Year ni ri
Design Measure Measure Age
Autonomy
cross-
Kim et al. 2018 129 0.24 S-KHLT Preference 73.59
sectional
Index
cross-
Song et al. 2017 305 0.16 REALM Questionnaire 41.76
sectional
cross- Internet-Use
Crook et al. 2015 180 0.32 NVS 38.68
sectional Questionnaire
Health
Information
cross- NA
YC Kim et al. 2015 950 0.21 KAHLS Seeking
sectional (20-79)
Intention
Questionnaire

c Adi Wijaya | adiwjj@[Link]

Olah Data – R Script
Persiapan:
• Siapkan file CSV dari Table hasil Tabulasi
dari included papers
• Install R Base dan RStudio (jika belum)
• Install package: “Metafor” (jika belum)

c Adi Wijaya | adiwjj@[Link]

R Base dan RStudio

c Adi Wijaya | adiwjj@[Link]

Olah Data – transform r to Z value

yi dan vi akan digunaan untuk kalkulasi Meta-analysis berikutnya

c Adi Wijaya | adiwjj@[Link]

Olah Data – random effect model

tau^2 = 0.0000 → no heterogeneity

I^2 = 0.08% → low heterogeneity
Q p-value = 0.3388 → no heterogeneity

c Adi Wijaya | adiwjj@[Link]

Olah Data – average correlation

c Adi Wijaya | adiwjj@[Link]

Olah Data – Forest Plot

Correlation = 0.22 (CI 95%, 0.17 – 0.26)

c Adi Wijaya | adiwjj@[Link]

Olah Data – Funnel Plot

Funnel plot is symmetric → no publication bias

c Adi Wijaya | adiwjj@[Link]
Olah Data – Egger’s Regression Test

Egger’s regression test, p-value = 0.58 → no publication bias

c Adi Wijaya | adiwjj@[Link]

Olah Data – Summary
• Correlation Model → moderate relation
o r = 0.22 (CI 95%, 0.17 – 0.26)

• Heterogeneity test → no heterogeneity

o tau^2 = 0.0000 → no heterogeneity
o I^2 = 0.08% → low heterogeneity
o Q p-value = 0.3388 → no heterogeneity
• Publication Bias → no publication bias
o Funnel plot is symmetric → no publication bias
o Egger’s regression test, p-value = 0.58 → no publication bias

All results are support the meta-analysis hypothesis:

“There is association between health literacy and
health information seeking behavior”
[moderate association]
c Adi Wijaya | adiwjj@[Link]
Olah Data- Script R Lengkap

c Adi Wijaya | adiwjj@[Link]

Reference
• Gene V. Glass (1976) Primary, secondary and meta-analysis of research,
Educational Researcher
• Moher et. al. (2015) Preferred reporting items for systematic review and meta-analysis
protocols (PRISMA-P) 2015 statement, Systematic Reviews, 4:1
• Quintana DS (2015) From pre-registration to publication: a non-technical primer for
conducting a meta-analysis to synthesize correlational data. Front. Psychol. 6:1549
• Del Re, A. C. (2015) A Practical Tutorial on Conducting Meta-Analysis in R. The Quantitative
Methods for Psychology, 11 (1), 37-50.
• Kitchenham & Charters (2007) Guidelines for performing Systematic Literature Reviews in
Software Engineering. EBSE Technical Report
• Higgins, J. P. T., Thompson, S. G., Deeks, J. J., and Altman, D. G.
(2003). Measuring inconsistency in meta-analyses. BMJ 327, 557–560
• Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al.
(2015). Preferred reporting items for systematic review and meta-analysis
protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 349, g7647–
g7647
• Egger, M., Davey Smith, G., Schneider, M., and Minder, C. (1997). Bias in
meta-analysis detected by a simple, graphical test. BMJ 315, 629–634

c Adi Wijaya | adiwjj@[Link]

 That’s all for now…
thank you for your attention…

any question?
adiwjj@[Link]

See you next episode!