Module 2
Measurement and Scaling techniques
(chapter 1)
1
Measurement in Research
• Measurement is relatively complex and demanding task,
especially when it comes to qualitative or abstract
phenomena
• Assigning numbers to objects and observations; usually in
respect of the properties
• The properties of “perception on motivation” and “ability to
be successful” is different from “age” and “financial position”
• Measurement process maps aspects of a domain into other
aspects of a range according to some rule of correspondence
• The rule of correspondence is that, for eg.; if the object in the
domain appears to be “male”, assign(code) “0” OR “1”, and if it
is “female”, assign “1” OR “2”.
2
Measurement Scales
▪ A tool or mechanism by which individuals and entities are
distinguished as to how they differ from one another on the
variables of interest to our study
▪ Four scales; depend on the degree of sophistication and
precision
▪ Attributes of Scale:
❖ Difference,
❖ Order,
❖ Distance
❖ & Origin
3
1. Nominal Scale
• Categorise individuals or objects into
mutually exclusive and exhaustive groups
• Gives some basic, categorical or gross
information
• Mainly qualitative
• Possesses “difference” property
From the following which one will
describe your nationality ?
4
2. Ordinal Scale:
• Categories are ordered based on preference/priority
• Ranking of various interventions, nutrition choices
• Possesses “difference & ordering” properties
5
3. Interval Scale :
▪ A quantitative measurement scale
▪ Measures the distance between any two points on the scale
▪ Order and the difference between two values are meaningful
▪ Attitudes, perceptions etc are measured using Int scale
▪ Possesses “difference, ordering, & distance” properties
Physicians’ attitudes towards nutrition counselling
6
4. Ratio scales
Used when exact numbers on objective factors are called for
Any variable whose base can be ascribed to “zero”
Possesses “difference, ordering, distance, & origin” properties
Example:
Years of taking medicine
Volume of vitamin intake
Income, Height, weight, BMI, Pulse rate………..
7
Refer: Sekaran, U: Research Methods for Business, 4 th edn, pp. 185 – 192
Zikmund et al. BRM, p.315 8
Chawla & Sondhi.(2011). Research Methods, pp.146 - 149
Males/females Levels of Education
Average
Income group
Average BMI
Bread
Years of education
Energy intake (Kcal/day) Cheese or cream cake
Protein (g/day) Fruit drink
Protein percent Mixed dishes
Carbohydrates Sunflower seeds
Carbohydrates percent
Sugar cookies
Total fat (g/day)
Fat per cent French fries
Vitamin E (mg/day) Cooked potatoes
Iron (mg/d) Soy sauce
Zinc (mg/d)
9
Summary of outcome Variables of risky food safety behaviours
Cooking:
I eat chicken or turkey when the meat is pink or has pink or red juices
I cook food until it is steaming hot and cooked all the way through
Cleaning:
I wash my hands before starting to prepare or cook food
Chilling:
I check the fridge temperature
I will consume leftovers stored in the fridge after more than two days
Use-by dates:
I check use-by date before eating or preparing food
I would eat meat/smoked fish/bagged salad/yoghurt after the use- by
date
Cross-contamination:
I wash raw chicken at least occasionally
10
Attitudinal/behavioural Scale construction Techniques:
Likert (Summated) scale:
▪ A particular item is evaluated based on how well it discriminates
between those persons whose total score is high and those whose
score is low
▪ Consists of a number of statements which express either a
favourable or unfavourable attitude towards the given object to
which the respondent is asked to react
▪ 3 to 11 degrees, usually 5 points
Fixed or Constant sum Scale
A given number of points are given to each item in such
a way its total does not exceed a given value
11
Semantic Differential scale
▪ Bipolar attributes are placed at the extremes to indicate the
response to attitudes
Numerical Scale
▪ Similar to SDS with the difference that numbers will be provided
▪ Also an interval scale
12
Goodness of measurement
▪ Attempt to measure whether a particular concept accurately
measures the variable
▪ The use of better instruments will ensure more accuracy of
results
▪ The researcher is to be reasonably sure that the instruments
used in the research do indeed measure the variables they are
supposed to do so
▪ Reliability, Validity & Sensitivity
13
Reliability
Reliability refers to the extent to which a scale produces
consistent results if repeated measurements are made
▪To what extent the measure is error-free
▪Indicates stability and consistency to help assess the
goodness of a measure
a. Stability of measure
▪ Ability of a measure to remain the same over time
Test-Retest Reliability
• The same reliability co-efficient obtained with a repetition
• The responses of a particular respondent remain the same
after sometime
• if the correlation between the scores obtained at different
times is higher, the better test-retest reliability
14
b. Consistency of Measure
▪ Indicative of the homogeneity of the items in the measure that tap
a construct
▪ The items and subsets of the items in the measuring instruments
are to be highly correlated
▪ Test the consistency of respondents’ answers
▪ Most popular test of inter-item consistency is Cronbanch’s Alpha;
ranges between 0 and 1; above 0.7 preferrable
▪ The higher the co-efficient, the better the measuring instruments@
15
Validity
▪Measures the reasonability of instruments to measure a
concept
▪Content validity, criterion validity & construct validity
1. Content Validity
▪ Measures the set of items are adequate and
representative
▪ Shows how well dimensions and elements are
delineated @ #
2. Criterion validity
▪Differentiates individuals on a criterion it is expected to
predict
▪Two types: concurrent validity and predictive validity
16
• Concurrent validity is established when the scale
discriminates individuals who are known to be different
❖Eg: the perception of food practices by ‘more
health-conscious’ and ‘less health-conscious’ must
be different
• Predictive validity indicates the ability of the measuring
instrument to differentiate among individuals with
reference to a future criterion
❖Eg: the aptitude test conducted at the beginning
should be able to discriminate the candidate as high
and low performers
17
3. Construct Validity
▪ How well the results obtained from the use of the
measure (instrument) fit the theoretical framework of
the study
▪ Assessed through convergent validity & discriminant
validity
• Convergent validity is established when the scores
obtained with two different instruments measuring
the same concept are highly correlated (Numeracy
skill and score in physics)
• Discriminant validity is established when two
variables are predicted to be uncorrelated
(Mathematical skill and linguistic skills)
18
Sensitivity:
▪ Ability to measure accurately the variability in responses
▪ Wider scale and more items, enhance scale sensitivity
19
20
Validity of an Experiment: Internal & External
• The extent to which a study accurately measures what it intends
to measure
• The conclusions drawn are legitimate and reflective of the actual
relationship between the variables
Internal Validity:
• The extent to which changes in the dependent variable can be
attributed directly to manipulations of the independent variable
• High internal validity suggests that the experiment has effectively
ruled out alternative explanations for the observed effects,
thereby affirming the causal relationships established in the study
External validity:
• Refers to the generalizability of the experimental findings beyond
the specific conditions of the study
• Whether the results can be applied to different populations,
settings, or time periods 21
Factors Influencing Internal Validity
1. History
▪ Events that occur outside the experiment but may affect the
dependent variable
▪ Any significant event occurs between pre-tests and post-tests
2. Maturation
▪ Natural changes that occur within study participants over time,
such as ageing or learning
▪ Make it difficult to ascertain whether observed effects are due to
the treatment or these natural developments
3. Testing
▪ Participants’ familiarity with test results in improved performance
or altered responses on subsequent assessments
▪ Repeated exposure to the same tests can lead to practice effects22
4. Instrumentation/administration
▪ Changes in measurement tools or procedures that may occur over time
▪ Changes in the method of data collection or the metrics used during the
course of the study
▪ Create inconsistencies in results, complicating the attribution of changes to
the treatment alone
5. Selection Bias
▪ When participants are not randomly assigned to treatment groups
▪ Lead to systematic differences between groups
6. Experimental Mortality (attrition)
▪ Participants may drop out of the study over time
7. Regression to the Mean
▪ The tendency for extreme scores to move closer to the average upon
retesting
▪ Lead to misleading results of treatment
23
Strategies to Deal with Threats to Internal Validity
❖ Critical to ensuring the validity of experimental research findings
1. Randomization
▪ Increase the likelihood that any differences in outcomes are due
to the interventions themselves rather than other extraneous
2. Control Groups
▪ Control groups serve as a benchmark against which the
treatment group's results can be
▪ Can help isolate the effect of the independent variable, thereby
providing clearer evidence of causality
24
3. Eliminating Extraneous Variables
▪ Controlling extraneous variables is essential
▪ Eliminate or control these variables, either through design
strategies such as matched pairs or by holding variables constant
across conditions
4. Blinding
▪ Significantly reduce biases in treatment administration and
outcome assessment
▪ Single or double-blind designs help prevent expectations from
influencing results
5. Statistical Controls
▪ ANCOVA or regression analyses allow researchers to account for
potential confounders in their analyses, isolating the effect of
the primary independent variable
25
Threat to Internal Description Mitigation Strategies
Validity
Events outside the experiment Use a control group; ensure timing
History
that affect results consistency; random assignment
Natural changes in Use a control group; shorten study
Maturation
participants over time duration; random assignment
Use alternative forms of
Influence of taking a test more
Testing (Test Effects) assessment; include control group;
than once
minimize pre-testing
Standardize instruments and
Changes in measurement
Instrumentation procedures; calibrate tools
instruments or procedures
consistently; train researchers
Extreme scores tend to move Use random assignment; include
Regression to the
toward the average on control group; avoid selecting
Mean
retesting groups based on extreme scores
Systematic differences Random assignment; matching
Selection Bias between groups prior to groups on key variables; use
treatment random sampling
Keep participants engaged; analyze
Loss of participants from the
Attrition (Mortality) dropout patterns; use intention-to-
study, causing unequal groups 26
treat analysis
Exercise 1:
In an online nutrition counselling trial, many participants
drop out before the follow-up assessment.
How might this affect your results and conclusions?
Develop strategies to minimize or account for attrition.
27
Exercise 2:
Patients take a food frequency questionnaire before
and after a diet intervention.
How might repeated testing affect their responses?
Discuss the strategies to avoid/testing for this effect.
28
Exercise 3:
You use two different brands of scales to measure
body weight before and after an intervention.
What problems might this pose for internal
validity?
How can you maintain instrument consistency?
29