We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
Experimental Design:
An Introduction
WHAT IS EXPERIMENTAL DESIGN? 2
EXPERIMENTAL DESIGN AS VARIANCE CONTROL 3
Systematic Variance 3
Extraneous Variance 3
Randomization 4
Elimination 5
Matching 5
Additional Independent Variable 5
Statistical Control 6
Error Variance 6
Validity 7
‘TYPES OF EXPERIMENTAL DESIGNS 8
Single Case Experimental Design 8
Quasi-Experimental Design &
Experimental Design 9
BASIC TERMINOLOGY IN EXPERIMENTAL DESIGN 10
Factor 10
Levels 10
Dimensions 17
Treatment Combinations 11
Replication 11
Main Effects 12
Simple Effects 12
Interaction Effects 12
BASIC TERMINOLOGY OF STATISTICAL ANALYSIS 13
Null Hypothesis (H,) 14
Level of Significance 14
Type | and Type Il Error 15
Power of the Test 15
Region of Rejection 15
— This book is about basic principles of experimental design and analysis, and is addressed tothe students
and researchers in behavioural sciences. More particularly, itis an attempt to set forth the principles of
designing experiments, methods of data collection, analysis, and interpretation of results. Before we
focus on the principles of designing experiments and methods of analysis, it would be putting the matter
in proper perspective if we first defined experimental design,
“The term experimental design has been used differently by different authors. A look at the literature
on the subject reveals that the term experimental design has been used to convey mainly two different,
though interrelated, meanings. In the first category are those who have used the term in general sense to
include wide range of basic activities for carying out experiments, that is, everything from formulation
of hypotheses to drawing of conclusions. The second definition ofthe term is comparatively restricted
The term is used in the “Fisher tradition” that is, to state statistical principles underlying experimental
designs and their analysis, wherein an experimenter can schedule treatments and measurements for
optimal statistical efficiency. It contains activities like procedure for selection of factors and their levels
for manipulation, identification of extraneous variables that need to be controlled, procedures for handling
experimental units, selection of criterion measure, selection of specific design (e.g,, factorial design,
Latin square design) and analysis of data
In this book, we shall be dealing with the designs that conform primarily to the later definition of
the term, although, ther aspects of designing, contained in the former definition, cannot be ignored in
entirety. The reader will appreciate that research i an integrated activity, where one step out ofa sequence
of steps cannot be effectively isolated from the rest. Thus, knowledge of the basic principles of
experimental design covered by both the definitions isa prerequisite for achieving the objectives of
research. However, in this book, we shall concentrate primarily on the second definition ofthe term,
that is, principles of experimental design and analysis, in the “Fisher tradition”.
WHAT IS EXPERIMENTAL DESIGN?
eee
Winer (1971) has compared the design of an experiment to an architect’s plan for the structure of a
building, The designer of experiments performs a role similar to that ofthe architect. The prospective
owner of a building gives his basic requirements tothe architect, who then exercising his ingenuity
prepares a plan ora blue-print outlining the final shape of the structure. Similarly, the designer ofthe
experiment has to do the planning of the experiment so that the experiment on completion fulfils the
objectives of research. According to Myers (1980), the design is the general structure ofthe experiment
not its specific content. ‘ExpeRIMENTAL DESIGN: AN INTRODUCTION
Though, there are different objectives of designing of an experiment, it may not be out of proportion
to state that the most important function of experimental design is to control variance. According to
Lindquist (1956), “Research design is the plan, structure, and strategy of investigation conceived so as
to obtain answer to research question and to control variance”. The first part of the statement emphasizes
only upon the objective of research, that is, to obtain answer to research question. The most important
function of the design is the strategy to control variance. This point will be elaborated in the discussion
that follows,
EXPERIMENTAL DESIGN AS VARIANCE CONTROL
‘Variance control, as we shall notice throughout this book, is the central theme of experimental design.
Variance is a measure of the dispersion or spread of a set of scoras. It describes the extent to which the
scores differ ftom each other. Variance and variation, though used synonymously, are not identical
terms. Variation is a more general term which includes variance as one of the statistical methods of
representing variation. A lot more is discussed about variance in chapter 2. Here we shall confine the
discussion and only emphasize its importance and methods of its control.
‘The problem of variance control has three aspects that deserve full attention. The three aspects of
\ variance are: systematic variance, extraneous variance and error variance. Main functions of experimental
\. design are to maximize the effect of systematic variance, control extraneous source of variance!, and
al minimize error variance. The major function of experimental design is to take care of the second function,
xx \__ that is, control of extraneous source of variance. Here we shall consider this aspect in comparatively
ts ‘greater detail. It will be seen later on that various designs are available for controlling the extraneous
re) —_-source of variance in different situations, and with the help of these designs, an experimenter can draw
n, valid inference.
SYSTEMATIC VARIANCE
of Systematic variance is the variability in the dependent measure due to the manipulation of the experimental
in 3 variable by the experimenter. An important task of the experimenter is to maximize this variance. This
we | objective is achieved by making the levels of the experimental vatiable/s as unlike as possible, Suppose,
of an experimenter is interested in studying the effect of intensity of light on visual acuity. The experimenter
of decides to study the effect by manipulating three levels of light intensity, ie., 10 mL, 15 mL, and
mn, 20 mL. As the difference between any two levels of the experimental variable is not substantial, there is
little chance of separating its effect from the total variance. Thus, in order to maximize systematic
variance, it is desirable to make the experimental conditions (levels) as different as possible. In this
experiment, it would be appropriate, then, to modify the levels of light intensity to 10 mL, 20 mL, and
30 mL, so that the difference between any two levels is substantial.
ofa t EXTRANEOUS VARIANCE
tive | In addition to the independent variable and the dependent variable, which are main concerns in any
uity experiment, extraneous variables are encountered in all experimental situations that can influence the
rhe \ dependent variable
3 the a 'Extraneous source of variance is contributed by all the variables other than the independent variable whose effect is
yent, 3 being studied in the experiment. These variables have often been called extraneous variables, irrelevant variables,
secondary variables, nuisance variables etc. In this book, all variables in the experimental situation other than the
independent variable have been termed as extrancous variables ot secondary variables. Ie. oer Deen wi Bewanounns Reiko
—
There are five basic procedures for controlling the extraneous source of variance. These procedyng,
are:
( Randomization
(iif) Matching
(») Statistical Control
(i) Elimination
(iv) Additional Independent Variable
Randomization
An important methed of controlling extraneous variable/s is randomization. It is considered to be the
most effective way to control the ¥:
atiability due to all possible extraneous sources. If thorough
randomization has been achieved, then the treatment groups in the experiment Gould be considered
Statistically equal in all possible ways, Randomization is a powerful method of controlling seconlaty
variables, In other words, it is a procedure for equating groups with respect to secondly vatiabl
According to Cochran and Cox (1957), "Randomization is somewhat analozous to insurance in that its
de eattion against disturbances that may or may not occur and that may or may not be
lo occur”.
ious if they
Randomization in the experiment could mean random selection of th
ser Population of interest to the experimenter, and/or random assignme
Subjects to the treatment conditions. Random assignment means that o
equal chance of being placed in any of the treatment conditions of rou
equal in the experiment, we may have random assignment w’ ignment is random,
fe ooP oe os Unalentlons ontaumiber of subjects per group or equal wuniber of mules aia eerclcs and
ss ss ————,—L_L
random sample from a population, but then assignment of experimental units to srcape may
Random assignment of subjects is critical to intemal validity. If subjects are aot
confounding? may occur. :
we experimental units from the
larg
‘nt of the experimental units or
very experimental unit has an
ips. However, in making g
ith constraints. The as
‘oups
et biased.
igned randomly,
An experimental design that employs randomization as a meth
is called randomized group design. For example, in tie randomiz
source of variance due to individual differences is controll
treatment conditions in the experiment. According to MeCall (19s
secured by the method of chance, equivalence may be secured by ¢
of subjects to be used is sufficiently numerous”. This refers to achieving compa
principle of chance. It may, however, be noted that randomization is employ
‘od of controlling extraneous variable
{ group design (chapt
3), extraneous,
ndomly to, say, k
by assigning subjects 1:
Just as represeniativeness ean be
ance, provided the number
zroups through the
J even when subjects are
matched. In repeated measures design (within subject design), where each subject undergoes all the
treatment conditions, the order in which treatments are administered to the subjects is randomized
independently for each subject (see chapters 6, 11 and 12),
Fisher's most fundamental contribution has been the concept of achieving pre-experimental equation
of groups through randomization, Equating of the effects through random assignment of subjects to
groups in the experiment is considered to be the overall best tool for controlling various sources of
* escril jon of variables in an experiment that confuses the interpretation of data. If the
2Term is used to describe an operation of confines he in ik
independent variable is confounded with a secondary variable, the experimenter cannot separate the effects of the two
variables on the dependent measure. Experimentat Desict
mitonocucnes 6 ERE
extraneous variation at the same time, Perhaps, the most important discriminating feature of the
experimental design, as compared to the quasi-experimental design’, is the principle of randomization.
Elimination
“Another procedure for controlling the unwanted extraneous variance is elimination of the variable by so
choosing the experimental units that they become homogeneous, as far as possible, on the variable to be
controlled. Suppose, the sex of a subject, an unwanted secondary variable, is found to influence the
dependent measure in an experiment. Therefore, the variable of sex (secondary source of variance) has
to be controlled. The experimenter may decide to take either all males or all females in the experiment,
and thus, control through elimination the variability due to sex variable. Procedure explained in this
particular example is also referred to as the method of constancy. Let us take another example to illustrate
the control of unwanted extraneous variance by elimination. Suppose, intelligence of the subjects in the
group is found to influence the scores of the subjects on achievement test. Its potential effect on the
dependent variable can be controlled by selecting subjects of nearly uniform intelligence. Thus, we can
control the extraneous variable by eliminating the variable itself. However, with this procedure we lose
the power of generalization of results. If we select subjects from a restricted range, then we can discuss
the outcome of experiment within this restricted range, and not outside it. Elimination procedure for
controlling the extraneous source of variance is primarily a non-experimental design control procedure.
Elimination as a procedure has the effect of accentuating the between group variance through decrease
in the within group or error variance.
Matching
Another procedure, which is also a non-experimental design procedure, is control of extraneous source
of variance through matching, The procedure is to match subjects on that variable which is substantially
related to the dependent variable. That is, if the investigator finds that the variable of intelligence is
highly correlated with the dependent variable, it is better to control the variance through matching on
the variable of intelligence. Suppose, an investigator is interested in studying the efficacy of method of
instruction on the achievement scores of the 10% grade children. The methods to be evaluated are:
lecture, seminar, and discussion. Here the method of instruction is the experimental variable of interest
to the investigator. The investigator discovers that the achieveinent scores (DV) are positively correlated
with the intelligence of the subjects, that is, subjects with high intelligence tend to score high on the
achievement test and those who are low on intelligence score are low on the achievement test. Thus, the
variable of intelligence (not of direct interest to the investigator) needs to be controlled because it is a
source of variance that will influence the achievement scores. In this experiment, the extraneous variable
(intelligence) can be controlled by matching the subjects in the three groups on intelligence (concomitant
variable).
However, matching as a method of control limits the availability of subjects for the experiment. If
the experimenter decides to match subjects on two or three variables, he may not find enough subjects
for the experiment. Besides this, the method of matching biases the principle of randomization. Further,
matching the subjects on one variable may result in their mismatching on other variables.
Additional Independent Variable
Sometimes the experimenter may consider elimination inexpedient or unpractical. He may not eliminate
the extraneous variable (of not direct interest to the experimenter) from the experiment and, thus, build
Min this book, as stated earlier, the subject of experimental design is treated in the “Fisher tradition”. The reader is
advised to go through the other aspect of designing, referred to in the introductory paragraph with the first meaning of
the term, experimental design. Campbell and Stanley (1963) have presented an excellent treatment of the subject
‘experimental and quasi-experimental designs in the non-statistical tradition, it right into the
studying the
the variable
on the basis of inte}
With the help of anal
(main effet of intelli;
design as a second independent variab
eHicacy of methods of instruction on achie’
oFintelligence. He introduces intelligence
Supp interes.
ent scores. He does not want tg gtd
im
an ateibute! variable, He creates yee ta
intelligence and low intelligence as levels of Seis
vss of varies, the experimenter con tke ot the varia oe
gence) from the total variance. The experimenter may sn a Fe th
of intelligence on achievement, and also the interaction between intelligence and method of
Tim's. the secondary souree of varianve is controlled by introducing the secondary vay
independent variable in the experiment, and the experimenter gets the advantage of isolatin,
oFintelligence on achievement and the interaction efeet as additional information
Due outeome of such
3 factorial design. Th
second
classifi
:
lligen,
elligence
re influence
instruction
iable as aq
i the effect
‘control procedure is a factorial design. In the above example, it
on re, the first variable or factor is intelligence (having three levels) and the
Variable is the method of instruction (three levels). The first factor or independent variable is a
“ation variable or a control variable and the second one is the experimental variable, wh
'Y manipulated by the experimenter.
Statistical Control
In this approach, no attem
‘one or more con it
Will be a
ich was
direct
the total variance. Thus,
is statistically held constant.
this point. Suppose, an investigator finds that anxiety level of
the subjects, an extraneous variable of no direct. Consequence to the purposes of the experiment, influences
the dependent variable in the experiment. The experimenter can control. this secondary source of variation
through elimination, that is,
differences between the blocks are high. In such a design, the variability due to the block differences is
taken out from the total variation. Thus, the statistical co1
ntrol technique can be utilized by the experimenter
to control the variance contributed by an extraneous vatiable.
ERROR VARIANCE
The results of experiments are affected by extraneous variables which tend to mask the a sf
a a ct er suel
experimental variable. The term experimental error or error variance is used to refer to al
*A characteristic that can be identified and measured,
si Broun, \
Higence scores of the subjects. The thiee groups consist of subjects of «, EES '“SS98“
Erewen Des: trsooucron 5 ERB
trolled sources of variation in experiments. Error variance results from random fluctuations in the
experiment. Experimental errors can be controlled either through experimental procedures or some
Statscal procedure If we arena able to effectively control the extraneous source of variation, then it
will form the part of error variance. By controlling secondary source of variation, one can reduce the
experimental error.
Two main sources of error variance may be distinguished. First is inherent variability in the
experimental unit to which treatments are applied. Second source of eror variance slack of uniformity
in physical conduct of experiment, or in other words lack o
refers to the errors of measurement.
f standardized experimental technique. This
aptitude, interests, anxiety, etc. Al these person-
her source of error variance is associated
easuring instrument, fatigue on the part of
1, inattention by subjects at some point of
with errors of measurement and could be due to unreliable m
experimental units, transient emotional states of the subjec
time, and so on,
Statistical controls can be applied to minimize such error variance.
design can be used to minimize the experimental err. By this technique the variability due to individual
Aifferences is taken out from the total variability, and tha, the ene Variance is reduced. Analysis of