SSE5210
CHAPTER 1
OVERVIEW OF EMPIRICAL
METHODS
WHAT IS EMPIRICAL RESEARCH?
Empirical research is defined as research based on
observed and measured phenomena.
It reports research based on actual observations or
experiments using quantitative research methods and it
may generate numerical data between two or more
variables.
2
DEPLOYING EMPIRICAL RESEARCH METHODS
Observation Idea Prior Knowledge
Research hypothesis or question
Research Method
Research Design
What to observe and Subject Population/ How the data will be
appropriate measures Sampling technique analyzed
Step-by-step research plan 3
Steps of Empirical Research
1. Select a topic or phenomenon to study
2. Perform background investigation/literature review
3. Identify specific research questions or hypotheses
4. Select a research method
5. Develop a detailed research plan/proposal
a. For identifying or recruiting subjects
b. For collecting data
• Define variables or phenomena to be observed and
a coding scheme
• Determine how the data will be obtained
c. For analyzing data
6. Execute the plan
7. Document the research you performed (ongoing) 4
BASIC CONCEPTS OF MEASUREMENT-1
“Measurement is a mapping from the empirical world to the formal,
relational world. Thus, a measure is the number or symbol assigned to
an entity by this mapping in order to characterize an attribute”
[Fenton96]
A measure is a mapping from the attribute of an entity to a
measurement value, usually a numerical value.
Measure is used to characterize an attribute in a formal way.
The measurement must preserve the empirical observations of the
attribute.
E.g.:
Object A is longer than object B, the measure of A must be greater
than the measure of B.
BASIC CONCEPTS OF MEASUREMENT-2
When we use a measure in empirical studies, we must certain
that the measure is valid.
The measure must not violate any necessary properties of the
attribute it measures – must be a proper mathematical
characterization of the attribute.
The mapping from an attribute to a measurement value can be
made in many different ways-each different mapping –a scale
Admissible transformation (rescaling)- able to transfer one
measure to another and preserve the relationships-
Meaningful vs. meaningless relate with the scale types.
SCALE TYPES
The most common scale types:
1. Nominal
2. Ordinal
3. Interval
4. Ratio
7
NOMINAL SCALE
Least powerful
Only maps the attribute of entity into a name or
symbol.
Classification of entities according to the attribute.
Possible transformation: one-to-one
Example: classification, labeling and defect typing
8
ORDINAL SCALE
Consists of classes that are ordered with respect
to the attribute – more powerful than nominal
scale.
Example of ordering criteria: “greater than”,
“better than” , “more complex”
The numbers represent ranking only, so addition,
subtraction, other arithmetic operations have no
meaning.
Any mapping that preserves the ordering (any
monotonic function) is acceptable.
9
Example: grades, software complexity
INTERVAL SCALE
Preserve order same as the ordinal scale but
there is notion of “relative distance” between two
entities.
Possible transformation M’ = aM + b, where M’
and M are different measures on the same
attribute.
Example: temperature measure in Celsius and
Fahrenheit.
Fahrenheit = 9/5 Celsius + 32 10
RATIO SCALE
Preserves ordering, size of intervals between entities, and
ratio between entities.
Start at zero and increase at equal intervals, known as units.
Possible transformation, M’ = aM where M’ and M
are different measures on the same attribute.
Example: length; convert feet to inches: I = 12 F
Other example: Length of software code,
11
duration of the development phase.
MEANINGFUL VS. MEANINGLESS
Meaningful- if the statements are true even if the
measures are rescaled, otherwise they are meaningless.
1. Fred is twice as tall as Jane - at least on ratio scale and meaningful no
matter what unit is used
2. The temperature in Tokyo today is twice than in London.
Twice-> implies ratio scale but not meaningful because we
measure temperature on two scales, Fahrenheit and Celsius.
If Tokyo 40° C and London is 20° C then the statement is true,
however on Fahrenheit, Tokyo is 104° F while London is 68 ° F.
OBJECTIVE MEASURE
There is no judgment in the measurement value-only
dependent on the object that is being measured.
Ensure consistency in measurement.
Different people produce the same measures- can be
measured several times and the same value can be
obtained within the measurement error.
E.g. Lines of code, delivery date
13
SUBJECTIVE MEASURE
The measure depends on both the object and the
viewpoint from which they are taken.
Subjective measures depend on the environment, vary
with the person measuring.
Can be different if the object is measured again.
E.g. personal skill, usability
- mostly nominal or ordinal scale type
14
DIRECT AND INDIRECT MEASUREMENT
Direct measurement of an attribute of an
entity involves no other attribute or entity.
E.g. lines of code, number of defects found in
test
Indirect measurement – involves the
measurement of other attributes – derived
from the other measures
E.g. defect density (number of defect /LOC),
programmers productivity (LOC/effort)
MEASUREMENT IN SE
Can be classified into 3 classes:
1. Processes (collections of software-related activities)
2. Products (any artifacts, deliverables or documents that result
from a process activity)
3. Resources (entities required by a process activity)
Within each class, can distinguish between
1. Internal attributes (can be measured purely in terms of the
objects itself)
2. External attributes (can only be measured with respect to how
16
the product, process or resource relates to other objects)
EXAMPLE OF MEASURES IN SE
Class Examples of Type of attribute Example of
objects measures
Product Code Internal Size
External Reliability
Process Testing Internal Effort
External Cost
Resource Personnel Internal Age
External Productivity
17
SUMMARY OF MEASUREMENT SCALES AND STATISTICS RELEVANT
TO EACH
Scale Defining Example of appropriate Appropriate
Type relations statistics statistical tests
Nominal Equivalence Mode Non-Parametric
Frequency
Ordinal Equivalence Median Non-Parametric
Greater than Percentile
Spearman r
Kendall ζ
Kendall W
Interval Equivalence Mean Non-Parametric and
Greater than Standard deviation parametric
Known ratio of any intervals Pearson product-moment
correlation
Multiple product-moment
correlation
Ratio Equivalence Geometric mean Non-Parametric and
Greater than Coefficient of variation parametric
Known ratio of any intervals
Known ratio of any two scale values 18