Quantitative Research Design
Study design
Provides a plan for answering the research question
Includes plan for study setting, participant selection, study procedures, variable measurement, data
collection, data management, data analysis
Considerations
Intervention
Comparisons
Potential confounding variables (other variables that could be the explanation for what you are
finding; if so, can you control them?)
o Controlling study context
o Controlling participant factors (randomization, homogeneity, matching, statistical control)
Blinding
Data collection times –
o Cross-sectional (all at one time)
o Longitudinal (longer, change over time)
Relative timing – prospective vs retrospective
Location
Classifications
Experimental
Quasi-experimental
Nonexperimental
Experimental designs
Requirements:
o Intervention
o Control
o Randomization
Experimental group
Control group
Examples:
Advantages
o Testing cause-and-effect relationships
Disadvantages
o Control vs practical significance
o Generalizability
o Need to randomize participants
Quasi-experimental designs
Requirements:
o Intervention
o Control
Examples:
Advantages
o Practical
o More acceptable to participants
o Can use when it’s unethical to randomize participants
Disadvantages
o More difficult to make causal inferences
Nonexperimental designs
Requirements:
o None
Types:
o Correlation studies - Study relationships between variables that are not manipulated;
correlation does not prove causation
Cohort design
Prognosis or etiology questions
Start with exposures and follow over time to see what outcome is
Case-control design
Look at past to see what exposures were
o Descriptive studies – Observe, describe, and document a phenomenon
Advantages
o Very practical
o Efficient way to collect large amounts of data
Disadvantages
o Cannot make causal inferences
o Self-selection
Critiquing a quantitative study design
Does the design match the research question/hypothesis?
What are the strengths of the design?
What are the limitations of the design?
Validity:
o Statistical conclusion validity
Low power
High attrition
o External validity
Can the study be practiced in real life?
Plan, scope, etc.
o Construct validity
Intervention and context – are these appropriate for study?
o Internal validity
Did IV really cause DV?
Threats to internal validity
Temporal ambiguity: Can we say one variable really happened first?
Selection
History: Did something happen in the world that count impact your
outcome?
Maturation: Things that happen organically over time
Mortality/attrition: Loss, more people in one group than the next, etc.
Sample and setting
Sampling plan
Population
o Target (entire group)
o Accessible (portion that you have access to study)
Sample (Smaller subset of accessible)
Nonprobability (Not random)
Convenience sampling
Easy, might not be fully representative
Quota sampling
Enroll a # of people, and ensure you enroll enough and
picking number of people for each group
Consecutive sampling
Enroll based from order
Purposive sampling
Hand selection of appropriate patients
Probability (Random sampling)
Simple probability sampling
Have a list, randomly pick numbers
Stratified probability sampling
Subgroups, pick randomly
Systematic sampling
List of everyone, pick every 5th person on the list, etc.
Sample size
Power – How many subjects are needed for research
o Sample size
o Attrition – Loss of people
Setting
Laboratory vs real setting
Critiquing a quantitative sample & setting
Is there a clear description of the sampling strategy provided?
Is this a representative sample?
Is there an appropriate number of subjects?
Is there a clear description of the setting?
Is the setting representative of where care would be provided?
Measurement & Data Collection
Measurement of variables
Levels of measurement –
o Nominal: NO ranking or ordering
Ex: Race, yes/no, gender, religion, political party, hair color
o Ordinal: Orders/range *the order is more important than the number
Cannot say the things are equal size between each other
Ex:
Grades: A, B, C, D, F
Pain scale: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Good, fair, poor
Strongly agree, agree, disagree
o Interval/ratio: Can take on any number within a range
Ex:
Length, weight
Reliability – Consistency that an instrument measures
o Stability: Similar results on separate occasions
o Internal consistency: Crohn Bacs Alpha
o Equivalence: Interrater reliability
Validity – Degree to which instrument measures what it is supposed to be measuring
o Face validity: Does the instrument make sense?
o Content validity: Is tool fulling encompassing
o Criterion-related validity: Does score on tool relate to external criteria
o Construct validity: Can tool differentiate between groups?
Data collection
Existing records vs original data
o Pros: Easier, can look back in charts
o Cons: Can’t add additional information (documentation, variable, etc.)
Original data
o Self-report
Interviews (get more data most likely) vs questionnaires (good for sensitive topics)
Open-ended vs close-ended
Investigator developed vs validated tools
o Observation
Categories
Rating scales
Observational sampling
o Biophysiological measures
In vivo: In a living person (BP, SpO2)
In vitro: Removed from person and tested later (blood work)
Critiquing a quantitative measurement & data collection plan
Can I trust the data?
Does the data accurately and validly reflect the variables under study?
Is the reliability and validity of measures reported?
Is there a clear description of the data collection procedures provided?
Was the best method of data collection used?
Were efforts made to enhance the data quality?
Quantitative Research – Data Analysis
Descriptive statistics
Xample: Please check the box representing your income:
High (>$100,000/year)
Medium ($30,000 - $100,000/year)
Low (<$30,000/year)
Frequency distributions:
Measures of central tendency:
Variable data:
Inferential statistics
Hypothesis testing
o Develop null and research hypotheses
o Choose a level of significance
o Determine which statistical test is appropriate
What is the research question/hypothesis?
Identify the independent and dependent variables
Identify what levels of measurement the variables were measured
Identify whether a parametric or non-parametric analysis is appropriate
o Run analysis to obtain test static and p value
o Make decision about whether to reject or fail to reject the null hypothesis
o Make a conclusion
Critiquing quantitative data analysis
Can I trust the results?
Does the analysis make sense considering the variables measured?
Are there multiple comparisons?
What is the statistical significance of the findings?
What is the clinical significance of the findings?
Literature Synthesis Project Example
Picot question
Among children with central lines (p), does receiving a daily bath with chlorhexidine (I), as compared to
receiving a daily bath with soap and water (c), affect rates of clabsis (o) during an inpatient admission (t)?
Step #3 of EBP Critically appraise the evidence
Cincinnati Children’[Link] -> evidence
James M Anderson Center of HS excellence: The Legend’s System
1. What time of question is my picot?
2. Pick what each article is
3. Review the sheet and rate the article