Statistics for Health Economics (STAT0039)

Chapter 5
Study design
Reserach process

There are three stages in a research process: Planning,

implementation and analysis. One of the key step in the planning
phase is to design an appropriate study, that may involve a
multidisciplinary team. The choice of which study design to use
depends on the research question.
For example, we may want to describe a population characteristic,
evaluate new treatments for diseases, find relations between factors
and outcomes, or to predict or forecast certain events or risks.
Reserach process

素⻝与⼼脏病⻛险

Lets look at the following example to see how we can handle the
following situation. Suppose we want to know:
Does a vegetarian diet reduce the risk of heart attack?
How should we collect the data to help us decide the answer?
Some ideas

Idea 1: We could conduct an experimental type design.

We find two groups of people, ask half of them to have a
vegetarian diet and ask the other half to eat non-vegetarian food
regularly. After 20 years, we collect the data from them to see how
many heart attack they had over the period.
Problems:
▶ not ethical: it is unethical to force or prevent someone from a
certain diet for a long period;
▶ not practical: very difficult to ensure people stick with the
diet as requested.
Some ideas

Idea 2: We could conduct an observational type, follow up design.

We monitor a group of vegetarian for 20 years and record their
diet. We then see how many heart attack they had over the period.
Not problem:
▶ ethically fine if they agreed for us to collect the data.

Problems:
▶ not practical: We need to wait for a long time to get results;
▶ the diet is difficult to measure;
▶ may need a large sample size because the number of people
who had heart attacks may be low;
▶ may have other explanations for any association.
Some ideas
Idea 3: We could conduct an observational type, retrospective
single group design.
We identify a group of people with heart attacks before, and ask
them to recall their diet in the past 20 years.
Not problems:
▶ ethically fine if they agreed for us to collect the data;
▶ we can have a relatively small sample size;
▶ more practical with quicker results.

Problems:
▶ no comparison group: we can only describe the diet in those
with heart attacks;
▶ rely on the accurate recall of diet from people;
▶ may have other explanations for any association.
Some ideas
Idea 4: We could conduct an observational type, retrospective
design with control.
Same as idea 3, but we get another similar group without heart
attacks and ask them to recall their diet. We then compare the
diet bewteen those with an without heart attacks.
Not problems:
▶ ethically fine if they agreed for us to collect the data;
▶ we can have a relatively small sample size;
▶ more practical with quicker results;
▶ have comparison group.
Problems:
▶ difficult to define who the comparison group should be;
▶ rely on the accurate recall of diet from people;
▶ may have other explanations for any association.

Can we do better?
Design considerations

In setting up a study design we should consider:

▶ Ethical aspects
▶ Practical aspects
▶ Reliability of results

We will look at a few standard designs to understand the

advantages and disadvantages of them.
Estimation with binary outcomes: risk and odds

For a binary variable the risk is defined as the probability of the

event or outcome occurred over a specified time period.

number of events
Risk = p = .
total number of subjects at risk(n)

For example, suppose in a certain study we have 200 smokers and

there are 5 of them who have oral cancer. What is the risk of oral
cancer among smokers?

5
Risk = p = = 2.5%.
200
Risk difference

The (absolute) risk difference is defined to be risk in exposed

minus risk in unexposed.
Continue with the example above, now on top of the previous
information, suppose we have 300 nonsmokers and there are 3 of
them who have oral cancer. We want to know if there a difference
in the risk of oral cancer between smokers and non smokers.

5 3
Risk difference = − = 1.5%.
200 300
Relative risk or Risk ratio

The relative risk or risk ratio (RR) is defined to be risk in exposed

divided by the risk in unexposed.
Continue with the example above we want to know the strength of
the associated risk of oral cancer between smokers and non
smokers.

5/200
Relative risk = = 2.5.
3/300

How do we know if this risk is genuinely higher or just because of

sample variability? We need some statistical tests.
Notice that: Absolute risk and relative risk provide different
interpretations and it can be misleading if we only have one of
them.
Odds

The odds is defined as the chance of an event or outcome happens

over a specified period of time.

p Risk probability that an event happens

odds = = = .
1−p 1 − Risk probability that an event does not happen

Continue with the example above, the odds of smokers to have

0.025
oral cancer is 1−0.025 = 2.56% and the odds of non smokers to
0.01
have oral cancer is 1−0.01 = 1.01%.
Odds ratio

The Odds (OR) is defined to be the ratio of the odds in the two
groups.
Continue with the example above the odds ratio of smokers to
non-smokers is

2.56%
Odds ratio = = 2.54.
1.01%
Study design

Study Design is a set of methods and procedures used to collect

and analyse data to address a research objective.
There are two categories of study designs in research:
Observational studies and Experimental (Interventional) studies.
Each have advantages and disadvantages and the choice depending
of the research question and availability of resources.
Observational studies

In a observational study, we only observes individuals without

manipulation or intervention. The event of interest or exposure has
been decided naturally or by other factor.
There are three common types of properties to distinguish different
observational studies:
▶ prospective vs. retrospective;
▶ cross-sectional vs. longitudinal;
▶ descriptive vs. analytic.
Why observational studies?

In medical statistics and health economics, experimental methods

are often considered the ‘gold standard’ for the measurement of
treatment or intervention effects. However, there are some reasons
why we want to perform an observational study:
▶ an experiment may be unethical;
▶ certain populations may be ineligible;
▶ an experiment may be not feasible, impractical or too costly,
e.g. study of rare events;
▶ interest may not necessarily lie in an assessment of a
particular intervention.
Some examples to use observational studies

Here are some examples that we may want to conduct a

observation study.
▶ study disease prevalence;
▶ investigate disease progression and trends over time;
▶ quantify the relationship between exposure to a risk factor and
the risk of disease or other health outcome.
Prospective vs. Retrospective
前瞻性和回顾性是研究的时间视⻆，决定了数据的收集顺序

Prospective study:
▶ data are collected forwards in time from the start of study
(i.e. at the planning stage data do not exist).
▶ there is always a follow up.

Retrospective study:
▶ data are collected about past events (i.e. at the planning
stage, data exist but may (or may not) have been collated).
Cross-sectional vs. Longitudinal
横断⾯和纵向研究是数据收集⽅式的分类

Cross-sectional study:
▶ subjects are observed only once we get a ‘snapshot’ of data at
one particular time point.

Longitudinal study:
▶ subjects are observed at more than one point in time we get a
set of longitudinal data.
Three types of observational studies

There are three common types of observation studies, with the

classification of the above properties:
Cross-sectional studies: 横断⾯研究

▶ descriptive or analytic;

Cohort studies: 队列研究

▶ descriptive or analytic;
▶ longitudinal;
▶ prospective or retrospective;

Case-Control Studies: 病例对照研究

▶ analytic;
▶ retrospective.
Bias

Bias is defined as the systematic errors in study design that result

in an incorrect estimate of the studied association.
Bias can be introduced into a study:
▶ as a result of the procedures for identifying and recruiting
subjects;
▶ from the procedures for collecting or analyzing information.

For experimental studies, since there is a direct intervention, we

can control follow up, data collection (and the control group) to
minimize possible sources of bias.
For observational studies, since we passively observe events that
occur, so they are more prone to biases than experimental.
Selection bias

Selection bias occurs in the recruitment stage of subjects and/or in

the process of retaining them in the study. There is a selection bias
when the sample does not represent the population either overall
or the group they were meant to.
Some examples are the following:
▶ inappropriate selection of subjects (cohort studies)
e.g. subjects who decline to participate in study on alcohol
consumption and cardiovascular disease may be heavy
drinkers;
▶ inappropriate selection of controls (case control studies);
▶ large dropout or missing observations that reduce the available
sample and make in unrepresentative of the population.
How to reduce selection bias

▶ Select an appropriate study sample;

▶ have in place mechanisms to ensure completeness of data,
e.g. record causes of loss to follow up, e.g. Death;
▶ carefully deal with missing data at the analysis stage,
e.g. Pre-plan to ensure that variables that can be predictors of
missing data have been recorded.
Information bias

Information bias occurs when measurements are made. Even if the

selection of subjects into a study is a fair representation of the
population, bias may arise if subjects are incorrectly categorized
with respect their exposure or outcome status (misclassification).
Types of information bias include:
▶ recall bias;
▶ observer bias;
▶ response bias.
Information bias
Recall Bias is the systematic differences in the way subjects
remember or report exposures or outcomes.
▶ for example, patients with the disease investigated may
remember more accurately past events or exposures.

Response Bias is the systematic differences of knowledge of the

treatment/study objective by the responders which affect
subjective measures such as pain relief.
▶ The subject may respond differently to questions relating to
their levels of exposure if they have been classified as a
subject with or without the disease under investigation.

Observer Bias is the systematic differences of knowledge of the

subject area that affect the ability of observers to collect or analyse
the data in a fair manner.
How to reduce information bias

▶ Carefully design data abstraction forms or questionnaires

(avoid leading questions);
▶ Blind respondents, assessors, interviewers etc;
▶ train interviewers/data handlers;
▶ standardize processes.
Confounding bias

Confounding bias occurs at the analysis stage. It can cause

estimates of association between an exposure and outcome to be
either larger or smaller than the true effects. A confounding
variable is a factor that is related to both the exposure and the
outcome of interest.
We need to consider confounding variables at the design stage or
else we may fail to collect information on these variables.
Confounding variables can be dealt with at the design or analysis
stages of a study, e.g. using matching, randomization,
stratification, etc.
Cross sectional studies
横断⾯研究：在⼀个时间点收集数据，描述群体特征或探讨暴露与结果的关系。

All information is collected at only one time point. There is no

follow up and the exposures and outcomes are measured
simultaneously.
Most cross sectional studies are descriptive, e.g. questionnaire,
survey, but can also be analytic.
The type of outcome can be continuous or binary. The target is
usually to describe characteristics of a group or explore
associations between exposures and outcomes.
Some effect measures includes:
▶ descriptive: mean (continuous), prevalence (binary);
▶ analytic: relative risk, odds ratio to explore associations.
Cross Sectional studies

Advantages:
▶ easy and reasonably cheap to perform;
▶ no predefined formats the sampling and analysis approaches
can be flexible to meet the exact needs of the research;
▶ useful to inform and guide larger/more sophisticated future
studies.

Disadvantages:
▶ response rates, particularly to questionnaires and surveys, can
be low resulting in missing data and selection bias;
▶ cause effect relationships are difficult to infer and model
because of a lack of time ordered data;
▶ need to ensure that a sample is representative of the study
population (selection bias).
Cohort Study
队列研究：可以是前瞻性或回顾性，研究暴露与疾病结果之间的关系。

A cohort study can be prospective or retrospective. Procedures to

carry out a cohort study:

1. Identify a suitable group of individuals who may share

common characteristics.
2. Fix an appropriate follow up period.
3. Follow up subjects over time.

At the end of the study period we can:

▶ Compare exposed and unexposed groups;
▶ Investigate associations between risk factors and the outcome
Cohort Study: Advantages

▶ Suitable for time varying exposure;

▶ multiple outcomes can be handled;
▶ temporal relationships can be assessed.
▶ suitable to study rare exposures, recruit people with
uncommon exposures (e.g. ionising radiation or chemicals in
workplace)
Cohort Study: Disadvantages

▶ can be time consuming and expensive;

▶ not always suitable for an outcome is rare or takes a long time
to develop;
▶ selection bias: The sample may not be representative of the
population. For example: in a study for the association
between drinking and cardiovascular disease, heavy drinkers
tend to reduce to participate; Loss to follow up/drop out can
complicate the analysis;
▶ information bias: Different quality of recorded information for
exposed and unexposed groups; the reviewer may interpret
differently medical images from a scan depending on whether
the subject had the exposure or not.
Case-control Study
病例对照研究：研究过去的暴露与当前疾病之间的关系。

▶ Retrospective study design;

▶ use to assess the effect of past exposure/risk factors on
current health/disease status.

Procedures to carry out a case control study:

1. Identify a group of subjects with a particular condition under

study.(cases)
2. Identify a group of subjects without the condition under study.
(control)
3. Look back in time to see which subjects in each group had the
exposure(s) of interest.
4. Compare the levels of exposure in controls and cases.
Case-Control Study: Advantages

▶ Cheap to perform and quick: cases already exist; no follow up

required;
▶ most efficient design for rare diseases or diseases with long
period between exposure and disease occurrence;
▶ can examine many exposures/risk factors;
▶ useful when studying dynamic populations where follow up is
difficult.
Case-Control Study: Disadvantages

▶ Prone to selection bias: controls should be representative of

the population ‘at risk’ of becoming cases and be sampled
independent of the exposure of interest;
▶ prone to information bias: Recall bias: recall of past events
differs between controls and cases; Observer bias: assessment
of the exposure status of a subject affected by knowledge of
whether that subject is a case or a control;
▶ can be inefficient when the exposure is rare;
▶ not suitable for multiple outcomes (cohort studies can be);
▶ do not allow for calculation of absolute risk and relative risk of
the disease.
Case-Control Study: Confounding

Case control studies can be problematic, if cases differ from

controls for reasons other than exposure or risk factor because of
another variable that is also related to the exposure. This variable
is a confounder and may distort the relationship between exposure
and outcome.
Matching is one way to control for confounding in case-control
studies at the design stage. We can include sampling the groups of
cases and controls in away that they are balanced with respect to a
set of predefined characteristics. We will discuss more on this later.
Experimental studies

The researcher actively performs a planned intervention, an

experiment in some or all members of a group. Procedure is as
follows:

1. Define the study population and the planned intervention(s).

2. Randomly assign subjects to one of two or more treatment
groups. There is often a ‘control’ group’ and an ‘intervention’
group.
3. Subjects are followed up over time. The outcomes of the
groups are compared.

Randomized trials is the “gold standard” for evaluating

interventions.
Basic Trial Design (Parallel group design)
We first identify some eligible and consenting subjects, and divide
them (how?) into two groups which will receive different
treatments. We then follow up all subjects to assess the outcomes
of interest. For example, lets think about if a Covid-19 vaccine is
beneficial or not.
We can’t be sure because:
▶ The vaccinated group may be people who chose to have the
vaccine
▶ The doctor recommended it because of a perceived benefit (or
increased risk)
▶ Those who didn’t have the vaccine might have declined it for
some reason

Vaccinated people may have different characteristics and lifestyles

than those unvaccinated. It may be one or more of these
differences that has led to the observed lower risk of Covid (not
necessarily an effect of the vaccine).
Randomized trials 随机化试验

随机化试验能够通过随机分配受试者，确保各组间
除了⼲预措施外的其他所有特征相似，从⽽使得任
何观察到的结果差异可以归因于⼲预措施本身。这
样能⽐其他类型的研究提供更强的因果推断。

The key idea is to compare groups of subjects who are similar in all
aspects, except in terms of the intervention they receive, so that
any difference in outcome can be attributed to the intervention.
This can allow stronger inferences of causality than other studies.
▶ Randomization:

We want to have a random allocation of subjects to ensure patient

groups are similar with regard to (known and unknown) factors
that may affect outcome to remove selection bias in allocation of
treatment.
Wrong ways to do it

Allowing the investigator, clinicians or participants to

choose/influence allocation?
▶ This will result in subjects with different characteristics in the
two groups.

Using methods that based on a pattern? For example: alternating,

day of enrollment, hospital record number?
▶ These are predictable and the person recruiting and
randomizing subjects could influence group allocation.
Right ways to do it 随机化⽅法

随机化的⽬的是确保患者组在已知和未知的影
响结果的因素上尽可能相似，从⽽消除选择偏
倚。随机化有多种⽅法，正确的做法是使⽤随
机数表或计算机程序来进⾏随机分配。

Allocation lists are produced using random number tables or

computerized randomization
There are several methods of randomization
▶ Simple
▶ Blocked
▶ Stratified
▶ Minimization
Simple randomization

Guideline: simple randomization is only appropriate for studies

recruiting a minimum of 200 subjects.
With small studies this method can produce (by chance) groups:
▶ of different sizes.

A way to fix this is to use blocked randomization.

▶ imbalanced for some characteristics.

Ways to fix includes stratified randomization or minimization.

Blocked randomization

Blocked randomization is also called restricted randomization.

▶ Ensures similar numbers allocated to each group;
▶ randomization list composed of blocks of patients within
which an equal number of subjects receive each treatment.
Stratified randomization

Stratified randomization reduces chance of differences in important

characteristics between groups. The procedure is as follows:

1. Identify key subject characteristics that are strongly related to

the outcome
2. Represent each characteristic as categories (strata)
3. Produce separate blocked randomization list for each stratum

Notice that the number of strata is limited by sample size.

▶ Guideline: maximum n
4B strata (n =no. patients, B =block
size).
Minimization

Minimization provides groups very similar for several factors (more

than by stratification).
▶ No predefined allocation list;
▶ dynamic process, implemented using a computer algorithm.

Basic idea:
▶ the first subject is allocated to a group at random;
▶ when allocating subsequent subjects: their characteristics and
those of subjects already randomized are considered;
▶ allocation is weighted towards the group that will minimize
the overall imbalance in characteristics between the groups.
Potential bias

Advanced knowledge of next allocation could result in deliberate or

subconscious allocation of particular types of patient to particular
groups (selection bias).
One way to solve this is to use concealed allocation, e.g. Web
based randomization, telephone randomization at a central office,
opaque sealed envelopes, numbered or coded identical bottles from
pharmacy.
Concealed allocation should be implemented in all trials.
Potential bias

Outcomes could systematically be affected by the knowledge of

whether intervention or control is being given
▶ patient’s response may be affected by knowing which
treatment they have;
▶ clinician’s behavior may be influenced by knowing which
treatment the patient is receiving.
Potential bias
One way to solve this is to use masking (also called blinding). This
prevents those involved in the trial from knowing whether a patient
is receiving intervention or control.
▶ Sometimes masking is simply not possible (open label),
e.g. surgery vs no surgery.

Placebos:
▶ masked trials will often require control groups to be given a
placebo.
▶ placebo identical to active treatment but biologically inactive.
▶ removes the placebo effect.

There are also other bias, e.g.

▶ bias due to withdrawals or loss to follow up;
▶ bias due to non compliance.
References

▶ Kirkwood BR,Sterne JA. 2003. MedicalStatistics(2nd ed).

Blackwell Science.
▶ Altman DG. 1991. Practical Statistics for Medical Research.
Chapman and Hall.
▶ Bland JM. 2000. An introduction to Medical Statistics, 3rd
edition. Oxford University Press.
▶ Bland JM, Altman DG. 1999. Treatment allocation in
controlled trials: why randomize? BMJ; 318:1209.
▶ Altman DG, Bland JM. 1999. How to randomize
BMJ;319:703–4.
References

▶ Altman DG, Schulz KF. 2001. Concealing treatment

allocation in randomized trials BMJ;323:446–7.
▶ Day SJ, Altman DG. 2000. Blinding in clinical trials and other
studies BMJ; 321:504.
▶ Schulz KF, Grimes DA. 2002. Generation of allocation
sequences in randomized trials: chance, not choice.
Lancet;359:525–9.
▶ Schulz KF, Grimes DA. 2002. Allocation concealment in
randomized trials: defending against deciphering.
Lancet;359:614–8.