Research Design &

Instrumentations
Presented at the

Research Methodology Course for PhD Candidates

30th MAY 2014

By:
Saadiah Binti Yahya, PhD
Prof. of Computer Sciences
Malaysia Institue of Transport (MITRANS)
Shah Alam
Research Design & Instrumentations
1. Purpose of the study
2. Types of investigation
3. Extent of researcher interference
4. Study setting
5. Unit of analysis (population to be
studied)
6. Time horizon
7. Instrumentation - Validity
2
 Purpose of the study

Exploratory study:
 When not much is known about the situation at hand, or
no information is available on how similar problems or
research issues have been solved in the past
 To bettercomprehend the nature of the problem since
very few studies might have been conducted in that area
 When some facts are known, but more information is
needed for developing a viable theoretical framework
 For obtaining a good graps of the phenomena of interest
and advancing knowledge through subsequent theory
building and hypotheses testing
3
 Purpose of the study
 Descriptive study:
 To ascertain and be able to describe the characteristics
of the variables of interest in a situation
 The goal: to offer to the researcher a profile or to
describe relevant aspects of the phenomena of interest
from an individual, organization, industry-oriented, or
other perspective
 Present data in meaningful form, help to:
 Understand the characteristics of a group in a given
situation
 Think systematically about aspects in a given situation
 Offer ideas for further probe and research
Help make certain simple decisions
4

 Purpose of the study

 Hypotheses testing:

 Explain the nature of certain relationships, or

establish the differences among groups or the
independence of two or more factors in a situation
 To explain the variance in the dependent variable or
to predict organizational outcome

5
 Purpose of the study

 Case study analysis:

 Involve in-depth, contextual analyses of matters

relating to similar situations in other organizations
 Problem-solving technique
 Qualitative in nature, useful in applying solutions to
current problems based on past problem-solving
experiences
 Useful in understanding certain phenomena, and
generating further theories for empirical testing
6
 TYPE OF INVESTIGATION

 Causal study: the study in which the

researcher wants to delineate the cause of
one or more problems

 Correlational study: when the
researcher is interested in delineating the
important variables associated with the
problem
7
 STUDY SETTING
 Field studies:correlational studies done in organizations

 Field experiments: studies conducted to establish
cause-and-effect relationship using the same natural
environment in which employees normally function

 Lab experiments: experiments done to establish cause
and effect relationship beyond the possibility of the least
doubt require the creation of an artificial, contrived
environment in which all the extraneous factors are
strictly controlled. Similar subjects are chosen carefully
to respond to certain manipulated stimuli
8
 UNIT ANALYSIS

 Unit of analysis: level of aggregation of the data

collected during the subsequent data analysis stage.
Depend on problem statement focuses.

 Individual: data gathered from each individual and

treating each employee’s response as an individual data
source

9
 UNIT ANALYSIS - continue

 Dyads: interested in studying two-persons interactions,

then several two-persons groups

 Groups: even though we may gather relevant data from

all individuals comprising, we would aggregate the
individual data into group data so as to see the
differences among some groups (missal jadi 6 group)

 Our research question determines the unit of

analysis.

10
 TIME HORIZON

 Cross-sectional/one-shot studies:
data are gathered just once, perhaps over
a period of days or weeks or moths, in
order to answer a research question

 Longitudinal studies: data on the

dependent variable are gathered at two or
more points in time to answer the
research question
11
 Validity considerations
 When we say that a knowledge claim (or
proposition) is valid, we make a JUDGEMENT
about the extent to which relevant evidence
supports that claim to be true
 Is the interpretation of the evidence given the
only possible one, or are there other plausible
ones?

12
 The logic of causal social research in the
controlled experiment
 Explanatory rather than descriptive
 Different from correlational research - one variable is
manipulated (IV) and the effect of that manipulation
observed on a second variable (DV)
 If … then ….
 E.g.
 "Animals respond aggressively to crowding" (causal)
 "People with premarital sexual experience have more stable
marriages" (noncausal)

13
 Three pairs of components:
 Independent and dependent variables
 Pre-testing and post-testing
 Experimental and control groups

14
 Components
 Variables
 Dependent (DV)
 Independent (IV)
 Pre-testing and post-testing
 Experimental and control groups
 To off-set the effects of the experiment itself; to
detect effects of the experiment itself

15
 Sampling
 1. Selecting subjects to participate in the research
 Careful sampling to ensure that results can be
generalized from sample to population
 The relationship found might only exist in the sample;
need to ensure that it exists in the population
 Probability sampling techniques

16
 Evaluating research (experiments)
 We know the structure of research
 We understand designs
 We know the requirements of "good"
research
 Then we can evaluate a study
 Is it good? Can we believe its conclusions?
 Back to plausible rival hypotheses

17
 Validity in designs
 If the design is not valid, then the
conclusions drawn are not supported; it is
like not doing research at all
 Validity of designs come in two parts:
 Internal validity
 can the design sustain the conclusions?
 External validity
 can
the conclusions be generalized to the
population?
18
 Internal validity
 Each design is only capable of supporting certain
types of conclusions
 e.g. only experiments can support conclusions about causality
 Says nothing about if the results can be applied
to the real world (generalization)
 Generally, the more controlled the situation, the
higher the internal validity
 The conclusions drawn from experimental
results may not accurately reflect hat has gone
on in the experiment itself
19
 Instrumentation
 Instruments with low reliability lead to
inaccurate findings/missing phenomena

 e.g. human observers become more

skilled over time (from pretest to posttest)
and so report more accurate scores at
later time points

20
 External validity
 Can the findings of the study be
generalized?
 Do they speak only of our sample, or of a
wider group?
 To what populations, settings, treatment
variables (IV's), and measurement
variables can the finding be generalized?

21
 External validity
 Mainly questions about three aspects:
 Research participants
 Independent variables, or manipulations
 Dependent variables, or outcomes
 Says nothing about the truth of the result that
we are generalizing
 External validity only has meaning once the
internal validity of a study has been established
 Internal validity is the basic minimum without
which an experiment is uninterpretable
22
 External validity
 Our interest in answering research questions is rarely
restricted to the specific situation studied - our interest is
in the variables, not the specific details of a piece of
research
 But studies differ in many ways, even if they study the
same variables:
 operational definitions of the variables
 subject population studied
 procedural details
 observers
 settings
 Generally bigger samples with valid measures lead to
better external validity 23
 Sources of external invalidity
 Subject selection - Selecting a sample which
does not represent the population well, will
prevent generalization
 Interaction between the testing situation and
the experimental stimulus
 When people have been sensitized to the issues
by the pre-test
 Respond differently to the questionnaires the
second time (post-test)
 Operationalization
24
 IN CONCLUSION
 Donald Campbell often cited Neurath's
metaphor:
 "in science we are like sailors who must repair
a rotting ship while it is afloat at sea. We
depend on the relative soundness of all other
planks while we replace a particularly weak
one. Each of the planks we now depend on
we will in turn have to replace. No one of
them is a foundation, nor point of certainty,
no one of them is incorrigible"
25
 Main Reference

I. Research Methods for Business – A Skill Building Approach

(UmaSakaran

II. Babbie& Mouton (2001)

26