Study Design Fundamentals in Research
Study Design Fundamentals in Research
5
Introduction to Study Design
5.1 Introduction to Etiologic Research
• Hypothesis Statement
• Variables
• Data
5.2 Ethical Conduct of Studies Involving Human Subjects
5.3 Selected Elements of Study Designs
• Necessity of a Referent (“Control”) Group
• Experimental vs. Nonexperimental (Observational) Designs
• Unit of observation
• Cross-sectional vs. Longitudinal Measurements
• Cohort vs. Case-control Studies
5.4 Common Types of Epidemiologic Studies
The advancement of knowledge about the causes of ill-health is an ongoing process that
progresses in overlapping stages. An issue may be brought to light by an intriguing case
report. This may be followed by the analysis of readily available vital statistics,
surveillance information, and routinely collected health survey data. Various types of
refined studies may follow, perhaps culminating in case-control studies, cohort studies,
and epidemiologic experiments. In all instances, the investigator pursues techniques that
are most advantageous to revealing causal relations.
Not to be overlooked in this process is the need to have sharply focused research
questions, clearly defined hypotheses, and study designs that connect research questions
to data that will be collected. Foreseeable facets of study hypotheses are specified within
the confines of pragmatism before the study is begun. As new questions arise, additional
hypotheses are defined and articulated to guide further study.
Study hypotheses should identify those factors that influence disease occurrence and
should, in general, address these aspects of the study’s design:
4. The expected change in incidence associated with given levels of exposure (dose-
response relation)
5. The time period that is thought to elapse between the exposure and its putative
effect (time-response relation)
6. Cofactors that contribute to the health-related outcome being studied, and how
these will be addressed by the study’s design
7. Plausible causal mechanism by which the exposure induces its effects
8. The sample size required to demonstrate the expected change in incidence
Illustrative Example 5.1 (Example of a Hypothesis Statement). (1) Let us consider the
relationship between specific oral contraceptive formulations and the risk of venous
thromboembolic disease in a population of women of childbearing age in the United
States between 2002 and 2006. (2) The explanatory factor of this study will be different
types of combination oral contraceptives, varying in their estrogen dose and progesterone
type. (3) The case definition will be based on diagnostically confirmed and treated cases
using specific clinical criteria for deep venous thrombosis and pulmonary embolism. (4)
We expect that the effects of different oral contraceptives would increase according to the
estrogen dose and progesterone type of the formulation and that (5) effects would
manifest only during current use of the contraceptive product. (6) Information about
cofactors associated with venous thromboembolism such as age, surgery, trauma, the
post-partum period, obesity, life-threatening illness (and so on) will be collected and
evaluated during the data analysis phase of the study. (7) The causal mechanism relates to
laboratory findings suggesting increases in thrombogenesis and decreases in fibrinolysis
with different formulations. (8) A sample size of approximately 60,000 women in each of
the groups studied for a year is needed to detect a risk differential of 8 per 10,000 in some
formulations compared to 4 per 10,000 in others with 80% power at an alpha level of .05.
Variables
The typical generic epidemiologic research question is: “Is there a causal relation
between the explanatory factor X and health outcome Y?”
The explanatory factor may represent any personal attribute, environmental factor,
social factor, personal attribute or behavior thought to influence health. This is the
independent variable of the analysis. In the epidemiologic literature, this has
traditionally been referred to as the “exposure.”
The health-related outcome may represent any disease, illness, or injury, or any
surrogate or predictor of ill- or good-health. This is the dependent variable of the
analysis, and has traditionally been referred to as merely the “disease” in epidemiologic
research.
In addition to the explanatory factor X and health-outcome Y, the research question must
also address other factors associated with factors X and Y. These additional factors are
referred to as potential confounders, control variables, extraneous variables, or
©B. Gerstman Page 2 Last printed 7/2/2011 4:33:00 PM
Chapter 5: Introduction to Study Design DRAFT
X →Y
C1 ──┘
C2 ──┘
⁞ ⁞
Ck ──┘
For the hypothesis suggested by Illustrative Example 5.1, the variables corresponds to:
Data
The data that constitute variables X, Y, C 1, C2, … Ck may be derived from a variety of
sources including personal interviews, interviews of surrogates (e.g., relatives,
roommates), self-administered questionnaires, employment records, environmental
records, medical and pharmacy records, administrative databases, government data
systems, and direct examination. The type of data used, of course, will depend on the
research question being addressed, the cost of obtaining data, the need for confidentiality,
the type population studied, and available technology.
Collection of information from medical and health service records requires cooperation
from study subjects, hospitals, medical practitioners, and the health care systems that
“own” the data. Obtaining cooperation from these parties can be difficult and requires
special care be paid to principals of confidentiality and ethical guidelines for research
involving human subjects, as discussed in the following section.
The information in medical records must be abstracted for analysis. Training medical
record abstractors is essential to obtain accurate and uniform information. It is often
advisable to blind the medical abstractor to the study hypothesis and selected information
in the medical records to prevent conscious and unconscious biases from entering into the
abstraction process. When more than one medical record abstractor is involved in a study,
it is wise to produce separate analyses for their data to check for consistency of results.
Lack of consistent results, such as a positive association derived by data from one
reviewer but not the other, is clearly cause for concern.
Questionnaires and abstraction forms should be brief and simple to complete. The art of
creating clearly worded, non-ambiguous, non-presumptuous, and non-leading questions
takes extra thought and planning (Payne, 1951). Data collection forms must be piloted
and tested repeatedly to remove ambiguities and redundancies before being used in the
study proper. Completed interviews are reviewed by the study coordinator before being
entered and validated. After data from the various sources (i.e., questionnaire, interview,
examination, medical record abstraction) are collected, information are entered into a
data base, documented, and audited for validity before being merged to form the study
database.
Table 5.1 lists three fundamental ethical principals for conducting research using human
subjects. These include respect for persons, beneficence, and justice.
Respect for persons Informed consent given freely. Implies ability to comprehend
consequences; confidentiality is maintained for private
information.
Beneficence Risk and benefits are assessed. Benefits can be direct, indirect,
collateral, or aspirational. Harms may involve physiological,
psychological, or socioeconomic consequences.
In addition, for studies involving participation, subjects must freely give their informed
consent before being allowed to participate in a study. Informed consent implies the
subject is given a chance to ask questions, is not coerced, is under no obligation to
participate in the study, and may withdraw from the study at any time. A signed
statement of consent is required.
Separate from the IRB, studies involving interventions require a Data and Safety
Monitoring Board (DSMB). The DSMB is an independent group of outside experts that
periodically reviews and evaluates accumulated evidence from the study to monitor its
safety and progress. The job of the DSMB is to make recommendations concerning the
continuation, modification, or termination of the study.
Studies that require treatments and interventions present additional special concerns in
their design. To ethically assign treatments, none of the treatments can be known to be
superior to any other. Treatments that present special hazards cannot be ethically
assigned and, just as importantly, treatments that are held to be beneficial cannot be
ethically withheld. Therefore, a true state of uncertainty about the pros and cons of the
treatment must exist before it can be submitted to a trial. This balanced void of
knowledge is sometimes referred to as equipoise.
The effect of a factor can be judged only in relation to what would have occurred in its
absence. This is why etiologic research requires at least two groups: one that is exposed
to the study factor (the index group), and another that not exposed (a referent or control
group).
The effects of treatments, for example, are not always obvious. If a patient recovers
following a treatment, how do we know whether recovery was due to the treatment or
might not have occurred spontaneously? In fact, how do we know that the treatment did
not delay recovery? Now consider administering the treatment to one group of patients
while leaving an identical group of patients untreated. If the treated patients experienced
a higher incidence of recovery than the control group, we may safely conclude that either:
(a) the treatment had a positive effect, (b) the difference was due to chance, or (c) the
groups differed in some important way other than the treatment. It is our job to weight the
evidence concerning these three possible explanations.
The same situation applies to non-experimental studies. If a child that lives near a high
tension electric power lines develops leukemia, the public will be tempted to attribute the
leukemia to the electromagnetic fields that emanated from that power line. In a similar
vein, when a brain tumor develops in an individual who uses their cell phone heavily,
they will be tempted to attribute their condition to cell phone use. These outcomes,
however unfortunate, cannot arbitrarily attributed to these particular environmental
exposures. The question of course is not whether there is an association within these
particular individuals. The question is whether these specific environmental exposures
caused these outcomes. On a population basis, this translates to these exposures increase
risks of occurrence. Thus, nonexposed referent group are needed to compare leukemia
rates in children who are and who are not exposed to electromagnetic fields; brain tumor
rates must be compared in heavy- and light-users of cell phones. If differences are then
observed, the same three explanations listed above must be entertained. Either: (a) the
exposure had an effect, (b) the observed difference are due to chance, or (c) the exposed
and nonexposed groups differ is some other important way
In both experimental and non-experimental studies, the purpose of the referent group is to
provide a baseline rate that reflects its incidence in the absence of exposure. Without this
referent rate, it is nearly impossible to determine whether an exposure has a positive or
negative effect, or makes no difference at all. There are many ways to incorporate a
nonexposed referent group into a study. Experimentation provides one archetype.
Randomized Experiment
Observational Cohort
One of the leading concerns in both experimental and observational research is the comparability
of the study groups. For comparisons to be valid, the groups being compared should be similar in
all relevant ways except for the study exposure. Experimentation is helpful in achieving “like-to-
like” comparisons.
The WHI program included both experimental studies and observational studies. The
experimental part of the WHI was designed to test the effects of postmenopausal
hormone therapy, diet modification, and calcium and vitamin D supplements on heart
disease, fractures, and breast and colorectal cancer. In the hormone trial, participants
were randomly assigned to groups that received a pill containing estrogen, estrogen plus
progesterone, or an identical-looking pill that contained no active ingredients. Incidents
of various health outcomes (e.g., coronary disease, stroke, pulmonary embolism, breast
cancer) were tracked in the study subjects.
The observational part of the WHI study complemented the experimental study by
providing estimates of the extent to which various risk factors predicted heart disease,
cancers, fractures, and other adverse health outcomes in women. The observational study
tracked the experience of 93,676 postmenopausal women between the ages of 50 to 79.
Women who joined this part of the study filled out health forms and visited study clinics
periodically. The observation study participants were not required to take any medication
or change their health habits while being followed.
Unit of Observation
This topic was introduced in §4.3 (Ecological Studies) and will not be covered here
except to note that person-level studies are generally preferable when making specific
judgments about pathogenesis. However, multi-level studies that combine various units
of observation, with some units of observation at the individual-level and some at the
environmental-level well-adapted for delineating how aggregate-level and person-level
variables interact as health determinants. The remainder of this chapter restricts itself to
studies that rely on individual-units of observation.
Longitudinal data are preferable when conducting etiologic research because, for a factor
to be causal, it must precede the event it caused by a reasonable amount of time.
However, for characteristics that do not change over time (e.g., blood type, eye color,
ethnicity), it matters little whether the measurement is longitudinal or cross-sectional: if
we know the status of an individual now, we also know their status in the past. In
addition, many human habits are potentially changeable, such as dietary choices, but
display some degree of long-term permanence—dietary habits for instance. For stable
characteristics such as these, the current status of the attributed might serve as a suitable
proxy for its longitudinal measurement.
Illustrative Example 5.3 (Cross-Sectional Study, Mental Health). Table 5.2 contains
data from a historically important mental health survey from the 1950s. Data were
compiled by a team of psychiatrists and sociologists using a census of psychiatric
patients and a five percent sample of the general population from the New Haven urban
community. Social economic status (the explanatory factor or “independent variable”)
was based on a combination of information about neighborhood of residence, occupation,
and education. Psychosis and neurosis (the health related outcomes or “dependent
variables”) were based on the diagnosis and treatment of these conditions in the
community (Hollingshead & Redlich, 1953). Table 5.2 shows the results.
TABLE 5.2. Prevalence per 100,000 of Psychosis and Neurosis by Social Economic
Status (SES), Connecticut, 1950a
SES Psychosis Neurosis
1 and 2 (high) 188 349
3 291 250
4 518 114
5 (low) 1505 97
Source: Hollingshead and Redlich, (1964, pp. 230, 235).
a
Prevalences have been adjusted for sex and age.
These data suggest a positive association between low socioeconomic status and
psychosis. Just the opposite is seen with low socioeconomic status and neurosis.
Detection bias. Some persons with mental disorders may not come to the attention
of the health care system used to identify cases. This creates a problem of
detection bias. Psychosis, for example, may come to the attention of psychiatrists
through legal intervention. Since social class is strongly correlated with legal
interventions, the selective forces that bring patients to care are also strongly
correlated with the study exposure and study outcome. This could create or at
least exaggerate the positive association between low SES and psychosis.
Diagnostic bias. There may be a diagnostic bias associated with both of the study
outcomes. For example, state hospitals, where the lower SES patients are more
likely to be seen, may have been more likely to diagnosis psychosis than neurosis,
whereas private providers may have been more likely to diagnose neurosis. Thus,
the diagnosed rates may be a spurious function of the provider type, as opposed to
SES. This is referred to as a diagnostic bias.
Reverse-causality bias. Another source of bias that we must consider is called
reverse-causality bias or cart-before-the-horse bias. Because data are cross-
sectional, it is difficult to establish the correct temporal sequence of events. That
is, the order of the exposure–disease may be turned around. An essential property
of a causal factor is for the exposure to precede the onset of disease. With reverse
causality bias, the temporal sequencing of events is reversed or inadequately
documented. Although one may be tempted to say that low social status causes
psychosis, another plausible explanation is that psychosis causes downward social
mobility; psychotics cannot often maintain the normal social relations required to
maintain a reasonable socioeconomic status. With cross-sectional data, the proper
temporal sequence can only be assumed.
Incidence-prevalence bias. Varying duration of illness may confuse the
interpretation of results in a type of bias called prevalence-incidence bias
(Neyman, 1955). Prevalence is related to both incidence and average duration of
illness (see Chapter 3.4). In studying the prevalence of a condition, therefore,
cases of long duration are more heavily weighted than those of short duration. If
the socioeconomic groups had similar incidences of neurosis, for example, but
SES groups had more persistent diagnoses, an apparent gradient in prevalence
would exist with no difference in incidence. Indeed, this is what the researchers
found in a subsequent analysis (Table 5.3). The incidence of neurosis was not
linked to socio-economic status. However, rates of reentry to treatment and the
continuous status were. In contrast, the positive association between low
socioeconomic status and psychosis persisted when the analysis was restricted to
incident cases.
Historical efforts to correct some of the weaknesses of cross-sectional field survey when
used for causal inferences have lead to greater rigor in epidemiologic methods and the
development of longitudinal methods in the form of modern cohort and case-control
studies.
Not losing sight of individual experiences from the beginning to the end underlies the
basis of longitudinal analyses. The two primary types of observational studies that are
longitudinal are cohort studies and case-control studies.
Cohort
Case-control studies can be viewed as a sampling variant of cohort studies. The case-
control approach starts by identifying disease events in the source population: these study
subjects form the case series. It then selects a sample of non-cases from the source
population: this is the referent or control series. Prior exposures to possible risk factors
are then ascertained from historical records or recall. We will see in Chapter 8 that the
exposure odds ratio from a case-control study is stochastically equivalent to the rate ratio
in the source population.
Case-Control
┌ cases → prior ┐
exposure
└ non-cases → prior ┘
exposure
Notice the large numbers of study subjects in the above cohort study. This was necessary
because the outcome was relatively rare (less than one in thousand per year),and sizable
©B. Gerstman Page 11 Last printed 7/2/2011 4:33:00 PM
Chapter 5: Introduction to Study Design DRAFT
cohorts were needed to derive a suitable number of cases to derive statistically reliable
estimates of risk. Case-control studies overcome the need to study large numbers of
individuals through an efficient sampling of “the study base.”
Illustrative Example 5.5 (Case-Control Study, Toxic Shock Syndrome). Toxic shock
syndrome is an illness characterized by high fever, vomiting, diarrhea, confusion, and an
exfoliating skin rash. It is fatal in 3% to 15% of cases, and is caused by the exotoxin of a
particular strain of Staphylococcus species.. In late 1979 and early 1980, the Centers for
Disease Control received an unusual number of report of this syndrome from state health
departments in Wisconsin, Minnesota, Illinois, Utah, and Idaho (CDC, 1980). The cases
occurred almost exclusively in women of childbearing age. Several case-control studies
followed. One of the case-control studies evaluated prior exposures to risk factors in 52
cases and 52 age- and sex-matched controls (Shands et al., 1980). All 52 cases had used
tampons during the menstrual periods coincident with the onset of illness. In contrast, 44
of the controls had used tampons during the period before the interview. Among the case-
control pairs in which both women had used tampons, 42 of the 44 cases used tampons
throughout menstruation, while 34 of 44 controls did similarly. Thus, cases were more
likely than controls to use tampons and, when they did, were more likely to use them
continuously. This was one of the clues that led eventually to the discovery of a highly
absorbent brand of tampon as a risk factor for the proliferation of toxogenic
Staphylococci.
Notice the relatively small number of individuals in the above case-control study (104
total, 52 cases and 52 controls). Despite the small number of study subjects, this study
was able to provide evidence about the relationship between antecedent tampon use and
subsequent disease. Also note that case-control sampling does not permit an estimate of
the sizes of the populations at risk from which cases arose. They are therefore unable to
determine rates of events: the incidence of toxic shock syndrome in those exposed and
not exposed to tampons cannot be determined from the above study. Nevertheless, case-
control studies can derive a measure of association between the exposure and disease
with a statistic called the exposure odds ratio. Exposure odds ratios will be studied in
Chapter 8.
Figure 5.1 presents this book’s scheme for discussing the most common types of
epidemiologic studies that are in use today. Studies are initially classified according to
whether they are observational or experimental. As discussed earlier, experimental
studies allows the investigator to assign an experimental treatment to some study subjects
while leaving other study subjects unexposed. Observational studies study individuals as
they are, without intervention.
Figure 5.1. Common types of epidemiologic studies and where to find them covered in
this text. [[Link]]
Observational studies are divided according to whether observations are primarily made
at the aggregate-level (ecological) or at the person-level. Person-level observational
studies are divided according to whether the analysis is cross-sectional or longitudinal.
Longitudinal studies, which permit the tracking of individual experience over time, are
classified as cohort or case-control.
Experimental studies are divided into community trials, field trials, or clinical trials.
Community trials address the efficacy of preventive interventions applied to groups
(e.g., marketing campaigns), field trials address preventive interventions applied to
individuals (e.g., a vaccine trials), and clinical trials address the efficacy of therapeutic
interventions (e.g., a treatments for illnesses). Since experimental studies provide the
archetype studies of epidemiologic research, we will commence with their consideration
in the next chapter.
EXERCISES
5.1 For each of the brief descriptions below, decide whether the study is (a)
an ecological study, (b) a cross-sectional study, (c) an observational cohort
study, (d) a case-control study, or (d) an experimental study. In addition,
explain each response by considering whether the unit of observation is at
the aggregate- or individual level, whether observations are longitudinal or
cross-sectional, and whether the design is experimental or observational.
5.4 Both cohort studies and case-control studies are longitudinal observational study
designs. What distinguishes the two?
5.6 You have developed the hypothesis that automobile drivers that talk on their cell
phones have a higher rate of fatal automobile accidents than those who do not.
How would you design a cohort study to test this hypothesis?
(A) How would you define the exposure groups in this study?
(B) How would you identify the cases in this study?
(C) What additional characteristics must you strive to measure similarly in the
cohort members?
(D) What difficulties would be encountered in measuring these characteristics?
(E) Since fatal automobile accidents are, fortunately, a relatively rare
occurrence, you have reconsidered pursuing this hypothesis with a cohort
study and are now considering a case-control approach. How would you
design a case-control study to test the hypothesis?
5.7 A cohort study evaluated risk factors for agriculture-related injuries among
African-American and Caucasian farmers and African-American farm workers
(McGwin et al., 2000). A total of 1,246 subjects (685 Caucasian owners, 321
African-American owners, and 240 African-American workers) were enrolled
between January 1994 and June 1996. Demographic, farming, and behavioral
information was collected at baseline. Subjects were contacted biannually to
monitor the occurrence of an agriculture-related injury (McGwin et al, 2000).
Some of the data from this study are presented in this Table:
(A) List the two exposure variables addressed by the data in this table.
(C) Explain why experimentation was not possible when addressing this issue.
REFERENCES
Belmont Report. (1979). Ethical principles and guidelines for the protection of human
subjects of research. The National Commission for the Protection of Human Subjects
CDC. (1980). Toxic shock syndrome -- United States. MMWR, 29(May 23, 1980), 229-
230.
CDC. (1980). Toxic shock syndrome -- United States. MMWR, 29(June, 27, 1980), 297-
299.
Gerstman, B. B., Piper, J. M., Tomita, D. K., Ferguson, W. J., Stadel, B. V., & Lundin, F.
E. (1991). Oral contraceptive estrogen dose and the risk of deep venous
thromboembolic disease. American Journal of Epidemiology, 133(1), 32-37.
Hollingshead, A. B., & Redlich, F. C. (1964). Social class and mental illness: A
community study. New York: Wiley.
Kannel, W. B., Dawber, T. R., Kagan, A., Revotskie, N., & Stokes III, J. (1961). Factors
of risk in the development of coronary heart disease—six-year follow-up experience.
Annals of Internal Medicine, 55, 33–50.
Lesko, S. M., Rosenberg, L., & Shapiro, S. (1993). A case-control study of baldness in
relation to myocardial infarction in men. JAMA, 269(8), 998-1003.
Lilienfeld, A. M. (1982). Ceteris paribus: The evolution of the clinical trial. Bulletin of
History of Medicine, 56, 1–18.
Mayo, E. (1933). The human problems of an industrial civilization (1960 reprint). New
York: Viking Press.
McGwin, G., Jr., Enochs, R., & Roseman, J. M. (2000). Increased risk of agricultural
injury among African-American farm workers from Alabama and Mississippi.
American Journal of Epidemiology, 152(7), 640-650.
Shands, K. N., Schmid, G. P., Dan, B. B., Blum, D., Guidotti, R. J., Hargrett, N. T., et al.
(1980). Toxic-shock syndrome in menstruating women: association with tampon
use and Staphylococcus aureus and clinical features in 52 cases. New England
Journal of Medicine, 303(25), 1436-1442.
Women's Health Initiative. (2010, 9/21/10). Retrieved June 24, 2011, from
[Link]