Validity & Reliability
Slides prepared by Tuhpawana [Link]
Main reference: Grad Coach/ Derek Jansen
Validity
• Validity and reliability are about ensuring the quality and accurately of
a study’s measurements.
• Validity (construct validity) assesses whether a research instrument
accurately measures what it tends to measure.
• Survey questions can easily misunderstood by respondents.
• Make sure that your measures accurately the key theory of your
study.
• Clearly define each of your key constructs and/or variables.
• Theoretical framework >survey instruments-----
Reliability
• Reliability assesses a measurements instrument’s level of consistency and
stability.
• Instruments i,e. survey, weight scale, or blood pressure monitor.
• To test reliability/ internal consistency is used Cronbach’s Alpha:
• Cronbach’s Alpha indicates how closely related the items to one another and
whether they consistently capture the same concept’
• Does it measure the thing that is supposed to be measuring? (Validity)
• Does it measure in a consistent and stable fashion (Reliability)
•
Cronbach’s Alpha
• Cronbach’s Alpha Internal consistency
• 0.91-1.00 Excellent
• 0.81-0.90 Good
• 0.71-0.80 Acceptable
• 0.61-0.70 Questionable
• 0.51-0.60 Poor
• <0.50 Unacceptable
Questions & Answers
• 3.. If you developed a survey to measure agility, but the survey actually measure
flexibility, what problem do you have…………?
• 4. If you weigh yourself 5 times in a row and you get identical results each time, your
bathroom scale has ………?
• 5. A clock that adds a few minutes each week has what issue…..?
• 6. A researcher develops a new test to measure depression that strongly and
negatively correlates with a well establish measure of depression. This is ……………….?
• 7. A researcher randomly divide 1000 tests into two groups. If the mean scores of
each group are identical, What likely exists……..?
• 8. A clock that is set to the correct time zone has …….?
•
Questions & Answers
• The alternatives of answer : High reliability
• Low reliability
• High validity
• Low validity
• 1. If the measurement instrument has poor consistency, this
indicates……………..?
• 2. When an instrument measures what it intends to measure, it
has………..?
Reference : Fiona Middleton (Jan
2025)
• A measurement tool can be reliable (consistent) but not valid
( not measuring what it’s intended to), and vice versa.
• Reliability and validity are concepts used to evaluate the quality
of research. They indicate how well a method, technique or test
measures something. Reliability is about consistency of a
measure, and validity is about the accuracy of a measure.
• Reliability and validity are closely related, but they mean
different things. A measurement can be reliable without being
valid. However, if a measurement is valid it is usually reliable.
Fiona…..(2)
• Reliability refers to how consistency a method measures something. If
the same result can be consistently achieved by using the same methods
under the circumstances, the measurement is considered reliable.
• Example: You measure the temperature of a liquid sample several times
under identical conditions, the thermometer displays the same
temperature every time, so the results are reliable.
• Example: A doctor uses a symptom questionnaire to diagnose a patient
with a long- term medical condition. Several different doctors use the
same questionnaire with the same patient but give different diagnoses.
This indicate s that the questuionaire has low reliability as ameasure of
the conditions.
Fiona……(3)
• Validity refers to how accurately a method measures what it is , intended to
measure. If research has high validity, that means it produces results that
correspond to real properties, characteristics, and variations in the physical or
social world.
• High reliability is one indicator that the measurement is valid. If a method is not
reliable, it probably isn’t valid.
• Example: If the thermometer shows different temperatures, each time, eventhough
you have carefully controlled conditions to ensure the sample’s temperature stay
the same, the thermometer is probably malfunctions, and therefore its
measurement are not valid.
• Example: If a symptom questionnaire’s results in a reliable diagnosis when
answered the different times and different doctors, this indicates that it has high
validity as a measurement of the medical condition.
Fiona…….(4)
• However, reliability on its own is not enough to ensure validity. Even if a test is reliable,
it may not accurately reflect the real situation.
• Example : The thermometer that you used to test the sample gives reliable results.
However the thermometer has not been calibrated properly, so the result is 2 degrees
lower than the true value. Therefore , the measurement is not valid.
• Example: A group of participants take a test designed to measure working memory. The
results are reliable, but participants scores correlate strongly with their level of reading
comprehension. This indicates that the method might have low validity, the test may be
measuring participants’ reading comprehension instead of their working memory.
• Validity is harder to assess than reliability, but it is even more important. To obtain
useful results, the methods you use to collect data must be valid : the research must be
measuring what it claims to measure. This ensures that your discussion of the data and
the conclusions you draw are also valid