RESEARCH

METHODOLOGY
Module 4 - Data Collection Methods
Reference: Research Methodology: Methods and Techniques – C.R. Kothari

[Link]/great-physics-experiments
 ▪ Enables testing of hypotheses and validation
of theories

▪ Supports accurate analysis, conclusions, and

WHY IS DATA
decision-making
COLLECTION
▪ Enhances the credibility and scientific
REQUIRED?
validity of the research study
 Introduction to Data Collection
▪ Systematic process of gathering relevant information

▪ Follows research design and objectives

▪ Poor data leads to invalid conclusions

▪ Depends on accuracy, cost, and time constraints

 Types of Data
▪ Primary Data – first-hand collected

➢ Mainly in experimental research

➢ Could be used in descriptive research (surveys)

▪ Secondary Data – already available

▪ Choice depends on research purpose

▪ Often both are used together

 Primary Data Overview
▪ Collected directly from source

▪ Highly specific to research problem

▪ Methods: Observation, Interview, Questionnaire, Schedules

▪ Accurate but costly and time-consuming

 Observation Method
▪ We all observe things around us, but this observation isn’t scientific

➢ It becomes scientific when serving a formulated research purpose

➢ With systemic planning and recording

➢ And subject to scrutiny (validity and reliability)

▪ Here, information is sought without asking the respondent

➢ Eliminates respondent bias

➢ Based on present behaviour

➢ Doesn’t need the respondent’s attention

 Observation Method
▪ Directly observing behavior/events

▪ Structured vs Unstructured

▪ Participant vs Non-participant

▪ Controlled vs Uncontrolled

▪ Example: customer behavior tracking

Pros & Cons

▪ Advantages: real-time, unbiased recall

▪ Limitations: observer bias, time-consuming

▪ Cannot capture emotions/thoughts

 Interview Method
▪ Verbal interaction with respondents

▪ Structured, semi-structured, unstructured

▪ Face-to-face, phone, online

▪ Useful for deep insights

Pros & Cons

▪ Pros: detailed data, flexibility

▪ Cons: costly, requires skill, bias risk

 Interview Method
▪ Employs verbal questioning as its principal technique of data collection

▪ Interviews are employed by people in everyday life

▪ But as a method of data collection, interviewing is different with regard to

its preparation, construction, and execution in that it is prepared and
executed in a systematic way

▪ Interviews basically consist of asking questions, listening to individuals and

recording their responses

▪ Interviews allow participants to provide rich, contextual descriptions of

events

9
 Why
Interview?
The most obvious way of finding the information is to ask someone who may
be able to help. Interviews also have a large number of potential advantages
for a qualitative researcher. Specifically, in an information setting some of the
advantages are especially significant.

10
 Interview Types
▪ Personal Interviews: Personal interview method requires a person
known as the interviewer asking questions generally in a face-to-face
contact to the other person or persons
➢ This sort of interview may be in the form of direct personal investigation or it
may be indirect oral investigation

▪ Telephone Interviews: This method of collecting information consists in

contacting respondents on telephone itself
➢ It is not a very widely used method, but plays important part in industrial
surveys, particularly in developed regions

11
 Pre-requisites & Tenets
▪ Interviewers should be carefully selected, trained and briefed

▪ They should be honest, sincere, hardworking and impartial

▪ Must possess the technical competence and necessary practical experience

▪ Occasional field checks should be made to ensure that interviewers are

neither cheating, nor deviating from instructions given to them for
performing their job efficiently

12
 Pre-requisites & Tenets
▪ Every effort should be made to create friendly atmosphere of trust and
confidence, so that respondents may feel at ease while talking to and
discussing with the interviewer

▪ Interviewer must ask questions properly and intelligently and must record
the responses accurately and completely

▪ Interviewer must answer legitimate question(s), if any, asked by the

respondent and must clear any doubt that the latter has

▪ Interviewers approach must be friendly, courteous, conversational and

unbiased

13
 Personal Interviews
▪ Structured Interview: Has predetermined questions with fixed wording,
usually in a pre-set order
➢ The use of mainly open-response questions is the only essential difference
from an interview-based survey questionnaire

▪ Semi-structured Interview: Has predetermined questions, but the order

can be modified based upon the interviewer's perception of what seems
most appropriate
➢ Question wording can be changed and explanations given; particular
questions which seem inappropriate with a particular interviewee can be
omitted, or additional ones included 14
 Personal Interviews
▪ Unstructured Interview: The interviewer has a general area of interest
and concern, but lets the conversation develop within this area
➢ It can be completely informal

▪ Focused Interview: Meant to focus attention on the given experience of

the respondent and its effects
➢ Under it the interviewer has the freedom to decide the manner and sequence
in which the questions would be asked and has also the freedom to explore
reasons and motives

15
 Personal Interviews
▪ Clinical Interview: Concerned with broad underlying feelings or
motivations or with the course of individual’s life experience
➢ The method of eliciting information under it is generally left to the
interviewer’s discretion

▪ Non-directive Interview: The interviewer often acts as a catalyst to a

comprehensive expression of the respondents’ feelings and beliefs and of
the frame of reference within which such feelings and beliefs take on
personal significance

16
 Pros
▪ More information and that too in greater depth can be obtained

▪ Interviewer by his own skill can overcome the resistance, if any, of the
respondents; the interview method can be made to yield an almost perfect
sample of the general population

▪ There is greater flexibility under this method as the opportunity to restructure

questions is always there, specially in case of unstructured interviews

▪ Personal information can as well be obtained easily under this method

17
 Cons
▪ It is a very expensive method, specially when large and widely spread
geographical sample is taken

▪ There remains the possibility of the bias of interviewer as well as that of the
respondent

▪ This method is relatively more-time-consuming, specially when the sample is

large and recalls upon the respondents are necessary

▪ Interviewing at times may also introduce systematic errors

18
DATA COLLECTION
THROUGH
QUESTIONNAIRES
 Questionnaire
Overview
▪ Set of questions arranged systematically

▪ Sent to respondents for self-response

▪ Questions printed or types in a definite order

▪ Used in large-scale surveys

▪ Used by private individuals, research workers and

organizations

20
 Questionnaire

Pros Cons
▪ Low cost ▪ Low response rate

▪ No interviewer bias ▪ Only for educated respondents

▪ Wide reach ▪ Inflexible

▪ Time for respondents ▪ Ambiguous responses possible

▪ Creation of large samples ▪ Time-consuming

21
 Questionnaire
Pilot Survey
▪ Trial run before main survey

▪ Helps to:

➢ Identify confusing questions

➢ Improve structure

➢ Test reliability

▪ Improves questionnaire design

▪ Example: Testing a student feedback form on 10 students before full rollout

22
 Questionnaire
Aspects: General Form
1. Structured Questionnaire 2. Unstructured Questionnaire

▪ Fixed questions + fixed order ▪ Flexible questions

▪ Definite, concrete and pre- ▪ Exact question formulation is

determined largely their own responsibility

▪ Comments in the respondent’s own ▪ Replies are to be taken down in the

words are held to the minimum respondent’s own words

▪ Easy to analyze ▪ Used for exploratory research

▪ Ex: MCQ survey ▪ Ex: Open interview-style form

23
 Questionnaire
Aspects: Question Types
1. Closed-ended 2. Open-ended

▪ Yes/No or multiple choice ▪ Free response

▪ Easy to analyze ▪ Provides deeper insight

▪ Ex: “Do you exercise daily? Yes/No” ▪ Ex: “Why do you prefer this brand?”

24
 Questionnaire
Aspects: Question Sequence
▪ Start with easy and interesting questions

▪ Avoid:

➢ Personal questions

➢ Questions that put too great a strain on the memory or intellect of the
respondent

➢ questions related to personal wealth

▪ Maintain logical flow (vital successive research questions with connection)

▪ Move from: General → Specific

▪ Ex: Habits → Preferences → Personal opinions

25
 Questionnaire
Aspects: Question Formulation
▪ Questions must be:

➢ Clear

➢ Simple

➢ Unbiased

▪ Avoid:

➢ Ambiguous words

➢ Emotional or leading language

▪ Ex: “Don’t you think this product is bad?”

“How would you rate this product?”

26
 Questionnaire
Aspects: Question Formulation
▪ Questions must be:

➢ Clear

➢ Simple

➢ Unbiased

▪ Avoid:

➢ Ambiguous words

➢ Emotional or leading language

▪ Ex: “How many razor blades do you use annually?”

“How many razor blades did you use last week?”

27
 Questionnaire
Aspects: Question Formulation

▪ Proper attention to the wording of questions

➢ reliable and meaningful returns depend on it to a large

extent

28
 Questionnaire
What is a Good Questionnaire?
▪ Short and simple

▪ Logical sequence
▪ Avoid technical jargon (depends)

▪ Questions may be
▪ dichotomous (yes or no answers),

▪ multiple choice (alternative answers listed) or

▪ open-ended
▪ Include control questions (for accuracy check)
▪ Provide adequate space for answers
▪ Attractive layout increases response rate
29
 Questionnaire

Designing a Questionnaire
Questions
Content Sequencing Formatting Translation
formulation

[Link]
 Questionnaire
Step 1: Content

▪ Take your objectives and variables as a starting point

▪ Decide what questions will be needed to measure your variables and reach
your objectives

▪ When developing the questionnaire, you should reconsider the variables you
have chosen and, if necessary, add, drop or change some

▪ You may even change some of your objectives at this stage

[Link]
 Questionnaire
Step 2: Formulating questions
▪ Formulate one or more questions that will provide the information needed for
each variable

▪ The question, as a rule, has to be broken up into different parts and made so
specific that all informants focus on the same thing

▪ Check whether each question measures one thing at a time

▪ Avoid leading questions

▪ Avoid words with double, complex or vaguely defined meanings or that are
emotionally laden

▪ Ask sensitive questions in a socially acceptable way

[Link]
 Questionnaire
Step 3: Sequencing the questions
▪ Design your interview schedule or questionnaire to be ‘informant friendly’

▪ The sequence of questions must be logical for the informant and allow, as
much as possible, for a ‘natural’ conversation, even in more structured
questionnaires

▪ At the beginning of the questionnaire a limited number of questions

concerning ‘background variables’ (e.g., age, education, marital status) may
be asked

[Link]
 Questionnaire
Step 3: Sequencing the questions

▪ Start with an interesting but non-controversial question that is directly

related to the subject of the study, this type of beginning should help to raise
the informants’ interest and lessen suspicions concerning the purpose of the
questionnaire

▪ Pose more sensitive questions as late as possible in the questionnaire (e.g.,

questions pertaining to income, political matters, sexual behaviour, or stigma
experienced in case of stigmatising diseases)

▪ Use simple, everyday language

[Link]
 Questionnaire
Step 4: Formatting the questionnaire
When you finalise your questionnaire, be sure that:

▪ A separate, introductory page is attached to each questionnaire, explaining

the purpose of the study, requesting the informant’s consent to be
interviewed and assuring confidentiality of the data obtained

▪ Each questionnaire has a heading and space to insert the number, date and
location of the interview

[Link]
 Questionnaire
Step 4: Formatting the questionnaire
▪ Make sure that questions belonging together appear together visually. If the
questionnaire is long, you may use subheadings for groups of questions

▪ Sufficient space is provided for answers to open-ended questions, categories

such as ‘other’ and for comments on pre-categorised questions

▪ Boxes for pre-categorised answers are placed in a consistent manner

[Link]
 Questionnaire
Step 5: Translation

▪ If interviews will be conducted in one or more local languages, the

questionnaire should be translated in order to standardise the way questions
will be asked

▪ After having it translated you should have it retranslated into the original
language by a different person

▪ You can then compare the two versions for differences and make decisions
concerning the final phrasing of difficult concepts

[Link]
DATA COLLECTION
THROUGH
SCHEDULES
 Schedule
Overview
▪ Similar to questionnaire

➢ BUT: Filled by enumerators (research

assistants)

▪ Enumerator:

➢ Asks questions

➢ Records answers

▪ Enumerators should be very carefully selected

▪ Scope of the investigation should be explained

▪ Must possess the capacity of cross-examination

▪ Ex: Census workers collecting household data 39

 Questionnaire vs Schedule

▪ Self-filled ▪ Enumerator-filled

▪ Economical ▪ Expensive

▪ Low response ▪ Higher response

▪ Not always clear as to who replies ▪ Identity of respondent is known

▪ Slower ▪ Faster

▪ Needs literate respondents ▪ Doesn’t need literate respondents

40
Questionnaire/ Schedule
 Task

Scenario: You are engineers working on a self-driving car system. Despite

high accuracy, users are reluctant to trust it.

Your goal: Find out WHY people don’t trust autonomous vehicles
CASE STUDY
METHOD
 Case Study
Overview
▪ Popular form of qualitative analysis

▪ Complete observation of a social unit

▪ Full analysis of a limited no. of events/

conditions and their interrelations

▪ Deals with the processes that take

place and their interrelationship

▪ Objective is to locate the factors that

account for the behaviour-patterns of
the given unit as an integrated totality

44
 Case Study
Overview
▪ “The case study method is a technique by which
individual factor whether it be an institution or
just an episode in the life of an individual or a
group is analysed in its relationship to any other
in the group.” – H. Odum

▪ The case study method is a form of qualitative

analysis where in careful and complete
observation of an individual or a situation or an
institution is done

45
 Case Study
Characteristics
▪ The researcher can take one single social unit or more of such units for their
study purpose; they may even take a situation to study the same
comprehensively

▪ Selected unit is studied intensively i.e. it is studied in minute details. Generally,

the study extends over a long period of time to ascertain the natural history of
the unit so as to obtain enough information for drawing correct inferences

▪ We make complete study of the social unit covering all facets

▪ Through this method we try to understand the complex of factors that are
operative within a social unit as an integrated totality

46
 Case Study
Characteristics
▪ Under this method the approach happens to be qualitative and not quantitative.
Mere quantitative information is not collected

▪ Every possible effort is made to collect information concerning all aspects of life

➢ For instance, under this method we not only study how many crimes a man has
done but shall peep into the factors that forced him to commit crimes when we
are making a case study of a man as a criminal. The objective of the study may
be to suggest ways to reform the criminal.

▪ An effort is made to know the mutual inter-relationship of causal factors

▪ The behavior pattern of the concerning unit is studied directly and not by an
indirect and abstract approach
47
 Case Study
Assumptions
▪ Uniformity in the basic human nature in spite of the fact that human behavior
may vary according to situations

▪ Studying the natural history of the unit concerned

▪ Comprehensive study of the unit concerned

48
 Case Study
Phases
1. Recognition and determination of the status of the phenomenon to be
investigated or the unit of attention

2. Collection of data, examination and history of the given phenomenon

3. Diagnosis and identification of causal factors as a basis for remedial or

developmental treatment

4. Application of remedial measures i.e., treatment and therapy (this phase is

often characterized as case work)

5. Follow-up programme to determine effectiveness of the treatment applied

49
 Case Study
Pros Cons
▪ Provides deep and detailed ▪ Findings are not easily generalizable
understanding of a social unit to larger populations

▪ Captures real-life experiences, ▪ High subjectivity and researcher bias

motivations, and behavior patterns may affect results

▪ Helps in tracing historical ▪ Data may lack scientific

development and social context standardization and comparability

▪ Useful for hypothesis generation and ▪ Risk of false generalizations due to

theory development small sample size

▪ Allows use of multiple data collection ▪ Time-consuming and expensive

methods 50
 Case Study
Conclusion
▪ Widely used, especially in sociology

▪ Valued for strong research advantages

▪ Needs modern data & analysis methods

▪ Data can be quantified and analyzed

▪ Growing popularity in research

51
OTHER METHODS
OF DATA
COLLECTION
 Other Methods
Warranty Cards
▪ Cards included with products to collect customer info

▪ Filled by consumers and returned to company

▪ Used for product feedback and user details

Distributor / Store Audits

▪ Data collected by observing retail stores

▪ Tracks inventory, sales trends, market share

▪ Useful for evaluating promotion effectiveness

53
 Other Methods
Consumer Panels
▪ Group of consumers providing regular data over time

▪ Types:

➢ Transitory (before-after studies)

➢ Continuous (long-term tracking)

➢ Used for behavior and trend analysis

54
 Other Methods
Mechanical Devices
▪ Uses tools like eye cameras, audiometers, etc.

▪ Measures attention, interest, and reactions

▪ Common in advertising and market research

Depth Interviews
▪ In-depth interviews to uncover motives and emotions

▪ Focus on hidden attitudes and personality traits

▪ Requires high skill and time

55
 Other Methods
Projective Techniques
▪ Indirect methods to reveal hidden motives and attitudes

▪ Uses stimuli like words, pictures, stories

▪ Requires specialized training

Types:

➢ Word association → first response to words

➢ Sentence/story completion → complete given prompts

➢ Verbal projection → explain others’ behavior

➢ Pictorial tests (TAT, Rorschach, etc.) → interpret images

➢ Play techniques → role-play or doll-based behavior

56
 Task
Match the Description
• Observation | A. Uses indirect stimuli (words, images) to uncover hidden motives and
attitudes
• Interview | B. Involves detailed, in-depth study of a single individual or unit over time
• Questionnaire | C. Data collected through direct watching of behavior without interaction
• Schedules | D. Written set of structured questions filled by respondents themselves
• Case Study | E. Enumerator records responses after interacting with participants
• Projective Techniques | F. Verbal interaction allowing probing and clarification of
responses
• Pantry Audit | G. Examines household consumption patterns by inspecting stored items
• Consumer Panel | H. Same group of respondents provides data repeatedly over time
• Mechanical Devices | I. Uses instruments like eye cameras or audiometers to track
reactions
• Sociometry | J. Studies relationships, choices, and group dynamics among individuals
57
 Task

Identify the method

▪ A researcher watches customer movement in a supermarket without interacting

▪ A survey is sent via Google Forms to 500 students

▪ A trained investigator fills responses for rural participants

▪ A psychologist uses inkblot images to study personality

▪ A company tracks TV viewing habits using a device

58
 EXPERIMENTAL
METHODS

[Link]/blog/experimental-research
 EXPERIMENTAL METHODS
Overview
▪ Most familiar type of research design
for individuals in the physical sciences
and a host of other fields

▪ A type of research that may cost more

than the potential benefits or profits,
even when a favorable outcome is
achieved

▪ Conducted under strict control of the

researcher

60
[Link]/science/article/100303-lhc-large-hardron-collider-restart-big-bang
 EXPERIMENTAL METHODS
Overview
▪ Commonly used in:

➢ Scientific experiments

➢ Social sciences

➢ Medical science

▪ Generally, field-based rather than

theoretical in nature

61
[Link]/science/article/100303-lhc-large-hardron-collider-restart-big-bang
 EXPERIMENTAL METHODS
What?
▪ Research is a methodical and organized process of collecting, arranging, and
analyzing data to support decision-making

▪ It is a systematic approach used to solve problems, find solutions, or

generate new knowledge

▪ There are different types of research depending on the context or problem,

with examples ranging from military applications to social issues, which can
be entirely different in nature

▪ Research design involves critical decisions, including selecting research

questions, theoretical frameworks, measurement methods, sampling
strategies, and data analysis techniques
62
 EXPERIMENTAL METHODS
What?
"Experimental design is the process of carrying out research in an objective and
controlled fashion so that precision is maximized and specific conclusions can
be drawn regarding a hypothesis statement." (Bell, 2009)

▪ One or more independent variables are manipulated to observe their effect

on dependent variables

▪ Involves the researcher maintaining control over all influencing factors to

determine or predict experimental outcomes; described in statistics as a
controlled information-gathering process
▪ Aims to minimize the effects of extraneous variables to enhance the
reliability of results; experimental units may include people, plants,
animals, etc
63
 EXPERIMENTAL METHODS
What?
"Experimental design is the process of carrying out research in an objective and
controlled fashion so that precision is maximized and specific conclusions can
be drawn regarding a hypothesis statement." (Bell, 2009)

▪ Consists of two groups:

➢ Experimental group – receives the treatment or intervention

➢ Control group – subjected to predefined conditions

(independent/experimental/treatment variables)

➢ Ex: Experimental Group: Patients receive a new drug being tested

Control Group: Patients receive a placebo (no active treatment)
under standard conditions
64
 EXPERIMENTAL METHODS
Task
A researcher studies whether a new training program improves employee
performance. The study is conducted under controlled conditions, and data
is collected after implementing the program.

Why is this an example of experimental research?

Is this more field-based or theoretical?

Justify what role does control play in this study?

65
 EXPERIMENTAL METHODS
Type of Experimental Design
Pre-Experimental Design
▪ The most basic form of experimental design
▪ Involves observing one or more groups after identifying cause and
effect
▪ Used to assess whether further research is needed
▪ Examines the effect of an independent variable on dependent groups
▪ Does not include a control group
▪ Example: A teacher introduces a new teaching method to one class and
checks student performance afterward, without comparing it to another
class
66
 EXPERIMENTAL METHODS
Type of Experimental Design
True Experimental Design
▪ Most accurate and widely used design, especially in physical sciences
▪ Establishes clear cause-and-effect relationships
▪ Factors needed for this method:
➢ Random variable
➢ Variable can be manipulated by the researcher
➢ Presence of control and experimental groups
▪ May include pretesting and requires proper random distribution
▪ Example: Participants are randomly assigned to two groups—one
receives a new drug (experimental group) and the other receives a
placebo (control group)—to test its effectiveness 67
 EXPERIMENTAL METHODS
Type of Experimental Design
Quasi-Experimental Design

▪ Similar to true experiments but lacks random assignment

▪ Does not allow definite cause-and-effect conclusions

▪ Used when randomization is impractical (e.g., educational settings)

▪ Includes designs such as time series and counterbalanced studies

▪ Example: Two existing school classes are compared—one uses a new

curriculum and the other follows the old one, but students are not
randomly assigned

68
 EXPERIMENTAL METHODS
Type of Experimental Design
Between-Group Design

▪ Compares two or more groups exposed to different treatments

▪ Enables statistical comparison and examination of causal relationships

▪ Example: One group of participants follows a high-protein diet, while

another follows a low-protein diet, and their outcomes are compared

69
 EXPERIMENTAL METHODS
Type of Experimental Design
Within-Group Design

▪ Same participants are tested under all conditions

▪ Comparisons are made within subjects to assess variable effects

▪ Example: The same participants are tested on memory tasks under

different conditions (e.g., with music vs. without music)

70
 EXPERIMENTAL METHODS
Type of Experimental Design
Completely Randomized Design

▪ Simplest design with random assignment to treatment groups

▪ Reduces bias and controls chance variation

▪ Provides a strong basis for statistical analysis using probability theory

▪ Example: Participants are randomly assigned to different fertilizer

treatments to study plant growth, ensuring no selection bias

71
 EXPERIMENTAL METHODS
Task
Identify the type of design and justify:

1. A single group is observed after applying a treatment

2. Participants are randomly assigned to control and experimental groups

3. Existing groups are compared without random assignment

72
 EXPERIMENTAL METHODS
Characteristics
Experimental research is characterized by the active manipulation of
independent variables and the objective observation of outcomes within a
highly controlled environment. The researcher deliberately creates conditions in
which one or more variables are systematically modified, allowing the study of
cause-and-effect relationships with precision.

73
 EXPERIMENTAL METHODS
Characteristics
Variables

▪ Includes independent, dependent, and extraneous variables

▪ Dependent variable: the outcome being studied or affected

▪ Independent variable: the treatment or factor being applied

▪ Extraneous variables: external influences that may affect results

▪ May involve multiple independent variables (e.g., time, skills, test

scores)

74
 EXPERIMENTAL METHODS
Characteristics
Setting

▪ Refers to the location of the experiment

▪ Can be conducted in a laboratory (high control) or real-world

environment (less controlled)

▪ Choice of setting depends on the nature of the experiment

75
 EXPERIMENTAL METHODS
Characteristics
Control

▪ Researcher must control all relevant variables except the independent

variable

▪ Essential for accurately determining the effect of the independent

variable

▪ Without control, valid conclusions cannot be drawn

76
 EXPERIMENTAL METHODS
Characteristics
Manipulation

▪ Involves applying a set of predetermined conditions to participants

▪ These conditions represent the independent (experimental/treatment)

variable

77
 EXPERIMENTAL METHODS
Characteristics
Observation

▪ Researcher observes changes in the dependent variable

▪ These changes occur as a result of manipulating the independent

variable

78
 EXPERIMENTAL METHODS
Task
A researcher tests whether caffeine intake affects reaction time.

1. Identify the independent variable

2. Identify the dependent variable

3. List any possible extraneous variables

4. Why is controlling extraneous variables important?

79
 EXPERIMENTAL METHODS
Process of Experimental Design
Experimental design is a systematic process of developing methods to test
a hypothesis. It involves a sequence of well-defined steps that guide the
researcher from identifying a problem to collecting accurate data, ensuring
that the experiment is structured, controlled, and capable of producing
meaningful conclusions

80
 EXPERIMENTAL METHODS
Process of Experimental Design
1. Defining the Research Problem

▪ First step in the research process

▪ Focuses on identifying a practical problem to be solved rather than

theoretical gaps

2. Determining Variables

▪ Involves identifying key variables

▪ Converts the research question into an experimental hypothesis by

predicting relationships between variables

81
 EXPERIMENTAL METHODS
Process of Experimental Design
3. Formulating Hypothesis

▪ Development of a clear and testable hypothesis

▪ Must align closely with the research problem

4. Designing a Controlled Experiment

▪ Involves planning all elements, conditions, and relationships in the

experiment

➢ Systematic and precise manipulation of independent variables

➢ Accurate measurement of dependent variables

➢ Proper control of confounding factors 82

 EXPERIMENTAL METHODS
Process of Experimental Design
5. Designing Experimental Treatments

▪ Focuses on how the independent variable is manipulated

▪ Affects the external validity of the study

▪ Requires decisions on: Range of variation of the independent variable |

Degree of precision in altering it

▪ If multiple treatments are used, results apply only to similar conditions

and order

83
 EXPERIMENTAL METHODS
Process of Experimental Design
6. Measuring Dependent Variables

▪ Involves selecting appropriate data collection methods

▪ Aims for accuracy and validity in measurement

▪ May require use of scientific tools or operational definitions to reduce

bias

84
 EXPERIMENTAL METHODS
Terminology
▪ A confounding variable occurs when an experimenter cannot tell the
difference between the effects of different factors on a variable
➢ Example: If a study tests a new diet but participants also exercise differently, the effect of
diet vs. exercise cannot be separated

▪ The placebo effect occurs when a subject (or experimental unit) reacts
favorably to a placebo when no medicated treatment has been given
➢ Example: A patient reports reduced pain after taking a sugar pill believing it is real medicine

▪ Blinding is a technique used to make the subjects blind to which treatment

(or placebo) they are being given
➢ Example: Participants in a drug trial are not told whether they receive the actual drug or a
placebo

85
 EXPERIMENTAL METHODS
Terminology
▪ A double-blind experiment is one in which neither the experimenter nor
the subjects know which treatment is being given
➢ Example: In a clinical trial, neither the doctor nor the patient knows who receives
the drug or placebo

▪ Randomization is a process of randomly assigning subjects to

treatment groups
➢ Example: Participants are assigned to treatment groups using a random number
generator

➢ A completely randomized design assigns subjects to different treatment groups

through random assignment

86
 EXPERIMENTAL METHODS
Terminology
➢ A randomized block design is sometimes used to make sure that subjects with
certain characteristics are assigned to each treatment. For example, when
testing a certain medication, you might first want to split subjects in groups
according to either gender or age (or both), then randomly assign each of these
groups to the different treatments

▪ A matched pairs design pairs up subjects according to similarities. One

subject in the pair receives one treatment, while the other receives a
different treatment
➢ Example: wins are paired, with one receiving a new drug and the other receiving a placebo

87
 EXPERIMENTAL METHODS
Terminology
▪ Sample size is the number of participants in the experiment. The larger
the sample, the more representative of the population the results will
be, but the costs of the experiment will also be higher
➢ Example: A study with 500 participants has a larger sample size than one with 50 participants

▪ Replication is the ability to reproduce the experiment (and results)

under similar conditions
➢ Example: Another researcher repeats the same experiment and obtains similar results

88
 EXPERIMENTAL METHODS
Task
For the following experiment, determine the experimental units, treatments,
and sample size. Indicate whether this experiment is blind, double-blind, and/or
randomized. Also identify any potential problems with the design.

▪ A study with 233 low-income adult smokers evaluated the effectiveness of

usual care (physician advice and follow-up) for smokers wishing to quit to
the usual care enhanced by computer-assisted telephone counseling
sessions. Each subject was assigned randomly either to the usual care or to
the usual care plus counseling, and their smoking status (still smoking or
quit smoking) was observed after 3 months. The percentage who had quit
smoking was higher for the group receiving counseling. (from Journal of
Family Practice 2000;50:138-144)
89
 EXPERIMENTAL METHODS
Task
How would you design a placebo-controlled double-blind experiment with a
randomized block design for the following situation.

▪ A veterinarian wants to test a strain of antibiotic on calves to determine

their resistance to a common infection, and if their gender plays a role. In a
pasture, there are 22 newborn calves (11 males and 11 females). There is
enough vaccine for 10 calves, but blood tests to determine their resistance
to infection can be done on all calves.

90