Scribd
Upload
17 views3 pages

Understanding Test Reliability in Psychology

The document discusses different types of reliability in psychological testing: test-retest reliability measures consistency over time by administering the same test twice; alternate-form reliability uses two equivalent forms of a test; split-half reliability measures internal consistency by splitting a test into two halves; inter-item consistency examines consistency between all items on a single test; and scorer reliability correlates scores from two independent raters. Reliability estimates the proportion of score variance due to true differences versus chance errors, with higher values between 0.7-0.9 generally desired. Reliability can be increased by adding more test items.

Uploaded by

Steffi Perilla
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
17 views3 pages

Understanding Test Reliability in Psychology

The document discusses different types of reliability in psychological testing: test-retest reliability measures consistency over time by administering the same test twice; alternate-form reliability uses two equivalent forms of a test; split-half reliability measures internal consistency by splitting a test into two halves; inter-item consistency examines consistency between all items on a single test; and scorer reliability correlates scores from two independent raters. Reliability estimates the proportion of score variance due to true differences versus chance errors, with higher values between 0.7-0.9 generally desired. Reliability can be increased by adding more test items.

Uploaded by

Steffi Perilla
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Tuesday, 21 February 2017

Psych 162: Reliability

Classical Test Theory


- True score + error = text score
- Sources of error
• Time
• Items/Content
• Consistency
- Standard error of measurement to create confidence intervals (95% confident that the
true score will fall between these two values)

Reliability
- Consistency of scores obtained by the same person across time, items, or other test
conditions
- Extent to which individual differences in test scores represent “true” differences or
chance errors
- Estimate what proportion of test score variance is error variance
• Error variance: difference n scores resulting from conditions that are irrelevant to the
purpose of the test

• Reliability of 0.8= 80% of the variance is a variance in true scores, and remaining
20% is error variance

• Desired: Between 0.7 and 0.9


- No test is a perfectly reliable instrument

Test-Retest Reliability
- Repeat same test on the same person on another occasion
- Test for correlation between scores on the two separate testing occasions
- Source of error variance: fluctuations in performance between the two testing
occasions
- Shows how test can be generalized across situations

1
Tuesday, 21 February 2017
- Higher reliability, lower susceptibility to random changes
• smaller error attributable to time factors
- Need to specify the length of interval (between two measurement situations)
• max. 6 months
- Disadvantage: practice effect
- Can only be applied to test in which performance is not affected by repetition (e.g
sensorimotor, motor)

Alternate-Form Reliability
- 1 person, 2 equivalent forms
• Form A: original
• Form B: restate
- Test for correlation of scores on the two forms
- Measure of:
• item factors
• stability over time
• consistency of responses to two different item samples
- Sources of error variance: time + content sampling
- Disadvantage: reduce but does not completely eliminate practice effect
- Parallel forms must:
• Be independently constructed
• Items should be expressed in the same form
• Same type of content
• Equivalent range and level of difficulty
• Instructions, time limits and sample items must be equivalent

Split-Hald Reliability
- 1 form, 1 person, split into two sections

2
Tuesday, 21 February 2017
- Source of error variance: content sampling
- Test for coefficient of internal consistency
- Longer test = more reliable (usually, not always)
- Spearman-Brown formula: for estimating the effect of shortening or lengthening the
test

• Used because this type or reliability only technically computes for the reliability of
half the test

Inter-Item Consistency
- a.k.a. Kuder-Richardson reliability and coefficient alpha
- Single administration of a single form
- Consistency of all items in the test
- Source of error variance:
• content sampling
• Heterogeneity of behavior
- More homogenous items, more consistency

Scorer Reliability
- Correlate results obtained by two separate scores
- Factors excluded from error variance:
• True variance (remains in scores)
• Irrelevant factors that can be controlled experimentally

Increasing Reliability
- Acceptable reliability: between 0.7 to 0.9
- Adding number of items

You might also like