Module 3.
Item Analysis and Validation 30
Item Analysis and
Validation
Module 3. Item Analysis and Validation 31
In the previous module, we had examined the different types of tests and
the different rules in test constructions. We had discussed the steps in
administering paper-pencil test as well as the sample format in creating a Table
of Specifications. In this section, we will try to analyse the characteristics of the
whole test through Item Analysis and Validation.
SELF-CHECK!!!
After finishing this module, you are expected to achieve the following
intended learning outcomes
1. Determine the characteristics of the entire test by analysing each item
trough item analysis and index of discrimination.
2. Determine the validity of the instrument as to appropriateness,
correctness, meaningfulness and usefulness.
3. Illustrate how item analysis and index of discrimination are prepared
by solving problems and showing examples.
LEARNING RESOURCES
For further readings, you may refer to the following books;
Student Workbook and Portfolio in Assessment of Student Learning 1 by;
Stimson T. Camano
Or you may visit.
[Link]
432d-a598-16471733fd87&v=&b=&from_search=2
Module 3. Item Analysis and Validation 32
EXPLORE!!!!
We will now continue on studying assessment in student’s
learning. In this section, we examine the manner and conduct of
the so-called Item Analysis and Validation.
ITEM ANALYSIS
In deciding whether a particular item in a test should be revised, retained
or discarded, Item Analysis is the main tool. It is a must for you, as future
teachers, to learn the concepts and principles of Item Analysis. Item Analysis is
the study of each item in a test to determine the effectiveness of the whole test. It
is done after an examination has been administered and scored. There are many
procedures that may be applied. The most common is by analysing the index of
the difficulty of the item and find the discriminating power of the item from the
number of students who answered the item correctly, considering the upper
group and the lower group of the distribution. It is either by using 1/3 of the upper
group and 1/3 of the lower group or 1/4 of the upper group and 1/4 of the lower
group.
PURPOSE OF ITEM ANALYSIS
As educators, we need to understand the underlying reason why we need
to study item Analysis. Below are some of the purposes of Item Analysis.
1. Item Analysis provides information about,
a. The difficulty of the item
b. The discriminating power of the item; and
Module 3. Item Analysis and Validation 33
c. The value of each alternative.
2. This will allow the teacher (you) to decide whether to revise or replace an
item in a test. Each item in a test is analysed in terms of difficulty and its
ability to discriminate.
The following are some of the benefits that you can derive from item analysis.
a. Information for the difficulty of the test.
b. Provide data which help students improve their learning.
c. Provide insights and skills that lead to the preparation of better tests in the
future.
TWO TYPICAL METHODS USED IN THE ANALYSIS OF A TEST
There are two methods used in the analysis of a test. It includes Item
Difficulty and Discrimination Index.
1. Item Difficulty
Item Difficulty is the percentage of students who got the answered right. This
gives us information of the difficulty of the item as to easy, with right difficulty or
difficult to have an action whether to discard, retain or revise the item. Item
difficulty is also called Difficulty Index, Facility Index or –value
The following are important conditions that you should remember:
The higher the value the easier is the item.
A -value of 0.76 and above indicates very easy item that should not be
included in subsequent test.
Module 3. Item Analysis and Validation 34
A -value below 0.25 indicates very difficult item, it might be confusing and
consider revising. This can be removed from subsequent tests.
Finding the Index of Difficulty
Item difficulty tells us whether the item is difficult, with right difficulty or easy to
have an action whether to discard, retain or revise the item.
Formula:
Where:
= Index of Difficulty (Percentage who answered the item correctly)
= The 25% number of respondents in the upper group who answered the
item correctly.
= The 25% number of respondents in the lower group who answered the item
correctly.
= The total number who tried the item.
Table of Interpretation
Range of Difficulty Interpretation Action
Index
Extremely Difficult Revise or Discard
Difficult Revise or Discard
Right Difficulty Retain
Easy Revise or Discard
Extremely Easy Revise or Discard
Module 3. Item Analysis and Validation 35
2. Discrimination Index
Discrimination index tells whether it can be discriminated between those who
know and those who do not know the answer.
The following are important conditions that you should remember.
If more from the upper group got the item correctly than the lower
group, the test item has Positive Discrimination Index.
If more from the lower group got the item correctly than those from the
upper group, the test item has Negative Discrimination Index. This is
not normal. It indicates that students with less knowledge and skills are
getting it right more than those who are better students. Examine for
ambiguousity of the item or off the topic or it may be mis-keyed or
unkeyed.
If the same number of examinees from the upper and lower groups got
the item correctly, the test item has No or Zero Discrimination Index.
Finding the Index of Discrimination
Index of Discrimination is the ability to answer the problems by those who know
and those who are merely guessing especially to the difficult items because easy
items cannot be discriminated by these two groups of learner. This can be
inferred from the difference between the number of proportion of respondents of
the upper group who got the item right and the proportion of the respondents of
the lower group who got the item right. The index is dependent upon the difficulty
of the item.
Formula:
Module 3. Item Analysis and Validation 36
Where:
= Discrimination Index
= Number of students in the upper 25% with Correct Answer
= Total number of students in the upper 25%
= Number of students in the lower 25% with Correct Answer
= Total number of students in the lower 25%
Table for Interpretation
Index Range Interpretation Action
Can discriminate but item Discard
is questionable
Non-discriminating Revise
Discriminating item Include
Item discrimination indices are interpreted in the context of the type of test which
is being analysed. Items with low discrimination indices should be examined.
Negative indices should be scrutinized to determine why a negative value was
obtained. (It may be mess-keyed or unkeyed)
Values of discrimination indices are classified as a. “Good” if the index is
above 0.30, b. “Fair” if it is between 0.10 and 0.30; and c. “Poor” if it is below
0.10. The smaller is the index the more difficult is the item. Maximum
discriminating power is indicated by an index of 1.0.
Module 3. Item Analysis and Validation 37
Example: In the table below summarizes the answers of 36 students who took
the 100-item multiple choice test in English 1 subjecting item number 20 for Item
Analysis with letter B as correct answer. Shall we retain, revise or discard item
number 20?
Students Score Answer Students Score Answer
1 89 B 19 90 B
2 85 A 20 87 E
3 85 E 21 83 C
4 89 A 22 82 D
5 99 B 23 82 D
6 73 A 24 82 D
7 85 E 25 84 A
8 87 A 26 87 B
9 89 B 27 80 B
10 95 B 28 97 A
11 86 A 29 97 B
12 98 B 30 87 A
13 74 D 31 79 C
14 76 D 32 77 B
15 85 C 33 84 C
16 90 B 34 78 A
17 74 A 35 79 D
18 95 B 36 95 E
Module 3. Item Analysis and Validation 38
Solution: Using the upper 25% right responses and lower 25% right responses
Step1. Arrange the test scores from highest to lowest value.
Suggestion: Use a master sheet of scores to arrange the scores fast without
missing any value.
Procedure:
99 – Highest Score
73 – Lowest Score
2 – The difference of their tens digit
2 + 4 (constant) = 6 – this represents the number of horizontal lines in the table
counting from the range of lowest score to highest score. So there will be 5 rows
in the table.
13 – is the constant which represents the number of vertical lines in the table. So
we have 12 columns in the table.
Therefore, we have a 5 (rows) by 12 (columns) table.
Step 2. Tally the scores in a table
Master sheet of scores
0 1 2 3 4 5 6 7 8 9 T
9 II III II I I 9
8 I III I II IIII I IIII III 19
7 I II I I I II 8
T 3 0 3 2 4 7 2 7 2 6 36
Step 3. Determine the scores to be found on the upper and lower 25% of the
distribution.
Module 3. Item Analysis and Validation 39
25% = 36 x 0.25 = 9
Above 25% Below 25%
1. 99 – B 80 – B
2. 98 – B 79 – C
3. 97 – B 79 – D
4. 97 – A 78 – A
5. 95 – E 77 – B
6. 95 – B 76 – D
7. 95 – B 74 – A
8. 90 – B 74 – D
9. 90 – B 73 – A
Step 4. Write the frequency of occurrence
Option A B C D E Total
Upper 1 7 0 0 1 9
25%
Lower 3 2 1 3 0 9
25%
Step 5. Solve for the Item Difficulty and the Discrimination Index of the item by
using the formula and interpret the results.
a. Item Difficulty
Right Difficulty, Retain.
b. Discrimination Index
Discriminating Item, include.
Based from the value of ID and DI, item 20 should be included or retained.
Module 3. Item Analysis and Validation 40
VALIDATION
Validation is the process of collecting and analysing the evidences to
support the meaningfulness and usefulness of the test. Validation of instrument
comes after performing the item analysis and revision. The purpose of validation
is to determine the characteristics of the whole test itself, namely the validity and
reliability of the test.
Process of Test Validation
Test validation involves collecting the three main types of evidences
namely; Content – Related Evidence of Validity, Criterion – Related Evidence of
Validity and, Construct – Related Evidence of Validity.
Content – Related Evidence of Validity
Content and format of the instrument.
How appropriate is the content?
How comprehensive?
How adequately does the sample of items or questions represent the content to
be assessed?
Criterion – Related Evidence of Validity
Relationship between scores obtained using the instrument and scores obtained
using one or more other tests. Scores are compared with scores on some other
test which presumes has already high quality. How strong is the relationship?
How well such scores estimate present or predict future performance of a certain
type?
Construct – Related Evidence of Validity
Module 3. Item Analysis and Validation 41
The nature of the psychological construct or characteristic being measured by
the test. How well does a measure of the construct explain differences in the
behaviour of the individuals or their performance on a certain task?
DISCUSSION BOARD
We will now try to interact with each other in a two way
process at a least possible way! I will post a question/s and try to answer it on
your own.
1. When do we say that a test item needs to be revised?
2. How can we say that a particular item is discriminating?
3. How can we say that an item?
POST COMPETENCY CHECKLISTS
Let us try to determine if you learned something from our topic. Answer the
following Self-Assessment Question.
SELF – ASSESSMENT QUESTION 3
1. The table below reflects the scores in a 100-item multiple choice test taken
by 30 students in Mathematics I and their answers in item number 7 with
letter E as the correct response, subjecting this item (number 7) for analysis,
compute the Item Difficulty and the Discrimination Index of the item and
interpret your findings.
Students Score Answer Students Score Answer
1 86 D 16 60 D
2 81 A 17 80 A
Module 3. Item Analysis and Validation 42
3 73 E 18 50 C
4 82 E 19 80 B
5 85 D 20 89 C
6 74 E 21 90 E
7 94 A 22 77 E
8 74 E 23 63 A
9 75 C 24 57 B
10 76 D 25 70 B
11 75 E 26 95 A
12 79 A 27 72 C
13 65 D 28 79 E
14 87 E 29 83 B
15 98 E 30 97 E
2. Create a rubric reflecting the three main types of evidences that needs to be
collected, that could be used in validating a test.
3. Write your reflection about the topic covered in this module by completing
the following statement.
After the activity, I realized that;
a. Item Analysis is _________________________________
b. Index of Discrimination is _________________________
c. Validation is _____________________________________