Experimental design
IC011017 Science and Technology Applications for Life
Asst. Prof. Dr. Jesper Knijnenburg (Aj. Jesper)
jespth@[Link]
Experimental design
The Scientific Method
IC011017 Science and Technology Applications for Life
Hypothesis
proven
Observations Questions Hypothesis Experiment Results
Hypothesis not proven 2
Experimental design
Experimental design
Independent variables Dependent variables
Manipulated variable Responding variable
IC011017 Science and Technology Applications for Life
Experimental design based on hypothesis
Extraneous variables
3
Experimental design
Types of experimental design
1) True experimental design
IC011017 Science and Technology Applications for Life
2) Quasi-experimental design
3) Non-experimental design
4
Experimental design
True experimental design
Independent variables Dependent variables
Manipulated variable Responding variable
IC011017 Science and Technology Applications for Life
Experimental design based on hypothesis
Extraneous variables
In this case, researchers have complete control over EV
5
Experimental design
True experimental design
Characteristics of true experimental design:
➢ Manipulation
IC011017 Science and Technology Applications for Life
The researcher manipulates the MV
➢ Control
All other variables are kept constant or carefully controlled (no influence of EV!)
There is a control group
➢ Randomization
Subjects are randomly assigned into groups
Before applying any treatment, each group is assumed to be the same (on average)
6
Experimental design
True experimental design
Example:
Subjects are divided into 2 (or more) groups:
IC011017 Science and Technology Applications for Life
• 1 control group (no treatment)
• 1 or more experimental group(s) (treatment(s))
How are the subjects divided??
7
Experimental design
True experimental design
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
Measurement Measurement
(NO treatment)
Experimental COMPARE
RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2 Measurement Measurement
(treatment)
With random assignment into groups, it can generally be assumed that the groups are
the same before applying the treatment
8
Experimental design
True experimental design
Research question:
Does eating a ‘high protein diet’ increase the weight of a person?
IC011017 Science and Technology Applications for Life
• Hypothesis?
• How are you going to design the experiments?
➢ Control group = ?
➢ Treatment group(s) = ?
• How do you separate the subjects into different groups?
9
Experimental design
True experimental design
Does eating a ‘high protein diet’ increase the weight of a person?
Manipulated variable:
IC011017 Science and Technology Applications for Life
Responding variable:
Control group:
Treatment group(s):
Extraneous variables:
Hypothesis:
10
Experimental design
True experimental design
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
( )
Experimental COMPARE
RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2
( )
Any difference in body weight between groups after the treatment is caused by diet 11
Experimental design
True experimental design
Change in body weight is due to EV
Before treatment After treatment
60 kg 64 kg
Group 1 Control group
IC011017 Science and Technology Applications for Life
Body weight Body weight
(normal diet)
Experimental COMPARE
Difference after
RANDOM assignment Example:
subjects into groups Duration of experiment is 3 months Differences?
treatment is due
(sample) Cause-effect?
to TREATMENT
Experimental group
Group 2 Body weight Body weight
(high protein diet)
60 kg 67 kg
Change in body weight is due to EV + TREATMENT
Any difference in body weight between groups after the treatment is caused by diet 12
Experimental design
True experimental design
Advantages:
Effect on the RV comes directly from the MV
Easy for the researcher to analyze
IC011017 Science and Technology Applications for Life
Less variability → easier to prove cause-effect with statistics
Considered the most powerful type of research design
Disadvantages:
Very difficult to control all EV
Subjects do not always want to cooperate
May sometimes endanger health of subjects (ethics!)
13
Experimental design
ACTIVITY: True experimental design
You want to find out if drinking coffee before an exam helps students to
perform better
IC011017 Science and Technology Applications for Life
➢ Manipulated variable?
➢ Responding variable?
➢ Hypothesis?
➢ How will you design the experiments?
➢ How do you separate the subjects into different groups?
14
SOLUTION
IC011017 Science and Technology Applications for Life Experimental design
15
Experimental design
True experimental design
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
( )
Experimental COMPARE
RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2
( )
Any difference in body weight between groups after the treatment is caused by diet 11
Experimental design
Quasi-experimental design
True experimental design: Quasi-experimental design:
❖ Manipulation ❖ Manipulation
IC011017 Science and Technology Applications for Life
❖ Control ❖ (Control)
❖ Randomization ❖ NO Randomization
A quasi-experimental design is an experimental design
without random assignment of samples into groups
16
Experimental design
Quasi-experimental design
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
Measurement Measurement
(NO treatment)
Experimental COMPARE
NON-RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2 Measurement Measurement
(treatment)
With non-random assignment into groups, it CANNOT be assumed that the groups are
the same before applying the treatment
17
Experimental design
Quasi-experimental design
Example:
Test a new fertilizer to increase the yield of two groups of farmers
IC011017 Science and Technology Applications for Life
How to design the experiments according to quasi-experimental design?
➢ Control group = ?
➢ Treatment group(s) = ?
18
Experimental design
Quasi-experimental design
Manipulated variable:
Responding variable:
IC011017 Science and Technology Applications for Life
Control group:
Treatment group:
Measure:
1.
2.
3.
19
Experimental design
Quasi-experimental design
Manipulated variable: Fertilizer / Amount of new fertilizer
Responding variable: Yield (how much product per area of land)
IC011017 Science and Technology Applications for Life
Control group: No fertilizer (or traditional fertilizer)
Treatment group: New fertilizer
Measure:
1. Yield of both groups before new treatment
2. Yield of both groups after new treatment
3. Compare the yield of the two groups before and after the treatment
20
Experimental design
Quasi-experimental design
Hypothesis:
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
( )
Experimental COMPARE
NON-RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2
( )
What is the difference with true experimental design?
21
Experimental design
Quasi-experimental design
Hypothesis: Using the new fertilizer increases the yield (over no/traditional fertilizer)
Before treatment After treatment
Group 1 Control group
IC011017 Science and Technology Applications for Life
Yield Yield
(No/traditional fertilizer)
Experimental COMPARE
NON-RANDOM assignment
subjects into groups Differences?
(sample) Cause-effect?
Experimental group
Group 2 Yield Yield
(New fertilizer)
What is the difference with true experimental design?
22
Experimental design
Quasi-experimental design
Quasi-experimental design: NO random assignment of samples
Yield of both groups before new treatment was already different
IC011017 Science and Technology Applications for Life
So is the difference in yield after the treatment caused by the new treatment?
This depends on how different the groups are before and after treatment
What information can we get from quasi-experimental design?
For each group, compare before and after treatment
23
Experimental design
Quasi-experimental design
Advantages:
More practical to use than true experimental design
More realistic, closer to ‘real world’
IC011017 Science and Technology Applications for Life
Can give causal relationship (cause → effect)
Disadvantages:
No complete control over EV
No ‘real’ control group
Less reliable
Statistics are less useful
24
Experimental design
ACTIVITY: Quasi-experimental design
You want to find out if drinking coffee before an exam helps
students to perform better
IC011017 Science and Technology Applications for Life
In the previous activity you have designed the
experiment according to true experimental design
➢ How can you make it a quasi-experimental design?
➢ What changes in your experimental design?
25
Experimental design
Non-experimental design
True experimental: Quasi-experimental: Non-experimental:
❖ Manipulation ❖ Manipulation ❖ NO Manipulation
IC011017 Science and Technology Applications for Life
❖ Control ❖ (Control) ❖ NO Control
❖ Randomization ❖ NO Randomization ❖ NO Randomization
26
Experimental design
Non-experimental design
❖ No manipulation of a variable
❖ No control group
IC011017 Science and Technology Applications for Life
❖ No random assignment of subjects into groups
→ When would we use such experiments?
27
Experimental design
Non-experimental design
We use non-experimental design in the case of:
❖ Descriptive research (observe and write down)
IC011017 Science and Technology Applications for Life
❖ Surveys (using questionnaires) your research project
❖ Study relations between variables without manipulation
❖ Not possible to manipulate the independent variable
❖ Case studies (study a single subject)
28
Experimental design
Non-experimental design
Advantages:
Low effort
Inexpensive
IC011017 Science and Technology Applications for Life
Non-invasive (does not affect the subjects)
Can find relations between variables
Disadvantages:
Very limited information
Cannot prove cause-effect
29
Experimental design
Non-experimental design
Characteristics of non-experimental design:
❖ NO Manipulation
IC011017 Science and Technology Applications for Life
❖ NO Control → Cannot prove cause-effect!
❖ NO Randomization
➢ So how can we write a hypothesis for such experiments?
30
Experimental design
Non-experimental design – hypothesis
A non-experimental hypothesis can generally be written in two ways:
IC011017 Science and Technology Applications for Life
❖ The hypothesis can describe a relation between two variables
❖ The hypothesis can be written about single variable
Ex:. Movie preferences are (not) evenly distributed among subjects
➢ We will see later how we can test such a hypothesis
31
Experimental design
Non-experimental design – hypothesis
The hypothesis can describe a relation between two variables:
▪ Null hypothesis (H0): the variables are not related
IC011017 Science and Technology Applications for Life
▪ Alternative hypothesis (Ha): the variables are related
Examples:
Gender is not related to level of satisfaction
Female students have a higher level of satisfaction than males
Happiness is not related to income
People with a higher income have a higher level of happiness
Remember: a relation between variables does NOT prove cause-effect!
32
Experimental design
Correlation and causation
Hot weather
IC011017 Science and Technology Applications for Life
Ice cream sales Shark attacks
Correlation
Do shark attacks increase ice cream sales? Correlation is NOT the same as causation!
Or do ice cream sales cause shark attacks?
33
Experimental design
Another example
Germs
IC011017 Science and Technology Applications for Life
Bad smell Diseases
Correlation
Causation (?)
(miasma theory)
34