Test C o ns t ruc t i on
Guidelines for Item Writing
1. Define clearly what you want to measure. To
do this, use substantive theory as a guide and
try to make items as specific as possible.
2. Generate an item pool. Theoretically, all items
are randomly chosen from a universe of item
content. Avoid redundant items.
Guidelines for Item Writing
3. Avoid exceptionally long items, which are rarely
good.
4. Keep the level of reading difficulty appropriate
for those who will complete the scale.
5. Avoid “double-barreled” items which convey
two or more ideas at the same time.
Double-barreled questions
Examples of double-barreled questions:
"Please agree or disagree with the following statement: Cars should be faster
and safer."[3]
"How satisfied are you with your pay and job conditions?"[4]
"How often and how much time do you spend on each visit to a hospital?"[5]
"Does your department have a special recruitment policy for men and
women?"[5]
"Do you think that there is a good market for the product and that it will sell
well?"
"Should the government spend less money on the military and more on
education?"
"Is this tool interesting and useful?"
Guidelines for Item Writing
6. Consider mixing positively and negatively worded
items. Sometimes, respondents develop the
acquiescence response set. This means that the
respondents will tend to agree with most items
without fully understanding them.
Example:
Adamson College Depression Scale
7. I feel sad most of the time. (-)
8. I feel hopeful about the future. (+)
Item Formats
Dichotomous Format
Offers two alternatives for each item. Usually a point is
given for the selection of one of the alternatives.
Advantages: simple, easy to administer, can be scored
quickly, requires absolute judgment.
Disadvantages: if used in ability tests, it encourages
memorization, test of luck and response style; forced
choice
Item Formats
Polytomous/Polychotomous Format
Offers more than two alternatives. Typically, a point is given for
the selection of one of the alternatives and no point is given
for selecting any other choice.
Advantages: easy to administer and score, uses distractors or
incorrect choices, requires absolute judgment, reduces
guessing
Disadvantages: if used in ability tests, it can encourage test of
luck and response style, forced choice
Item Formats
Likert Format
Requires the respondent indicate the degree of agreement with
a particular attitudinal question.
Instead of asking for a yes/no reply, five alternatives are
offered: strongly agree, agree, neutral, disagree and strongly
disagree.
Positively worded items are normally scored while negatively
worded items are reversely scored.
Item Formats
Category Format
Similar to the Likert format but uses a scale which holds greater
number of choices.
Example: People can rate items on a 10-point scale
Variation: visual analogue scale – the respondent is given a
100-centimeter line and asked to place a mark between two
well-defined end points.
Item Formats
Checklist
The subject receives a long list of descriptive or adjectival statements
and indicates whether each one is characteristic of him/herself or
someone else.
Q-sorts
Combination of the checklist format and the category format; the
subject is given statements and asked to sort them into 9 piles.
Statements that are least descriptive of the person are placed on Pile 1
while those that are most descriptive are placed on Pile 9.
Item Formats
Spiral omnibus format
A format used in ability tests wherein the items are arranged
from easy to difficult.
Examples:
VOCABULARY TEST ARITHMETIC TEST
Define the following words Get the results of the ff. equations
1. shoes 1. 12 + 7
2. jump 2. 50 – 9
3. computer 3. 12 x 15
4. rotate 4. 473 / 11
5. parasol 5. (261 + 90) – 46
6. mimic 6. (837 – 41) x (63 + 37)
Item Analysis
It is a general term for a set of methods used to
evaluate test items in order to come up with a
cluster of valid and reliable test items.
The basic methods involve assessment of item
difficulty and item discriminability.
Item Analysis
Item-difficulty index (p)
It is defined by the number of people who get a
particular item correct.
The higher the proportion of people who get the
item correct, the easier the item.
Np Np = number of test takers who
got the item correct
Formula: p =
N N = total number of test takers
Item Analysis
Interpretation of Item-difficulty index (p)
Item-difficulty index (p) Interpretation
0.81 and above Very easy
0.61 – 0.80 Easy
0.41 – 0.60 Optimum
0.21 – 0.40 Difficult
0.20 and below Very difficult
Item Analysis
Item-discriminability index (d)
It determines whether the people who have done well on
particular test items have also done well on the whole test.
An item can have a positive or negative discriminating power.
The higher the d value, the better the test item.
Up & Lp = number of test takers in the
upper and lower groups who got the item
Formula: d = Up - Lp correct
U U = total number of test takers in the
upper group
Item Analysis
Item-discriminability index (d)
For this calculation, we divide the test takers into three
groups according to their scores on the test as a
whole:
an upper group consisting of the 27% who make the highest
scores (U)
a lower group consisting of the 27% who make the lowest
scores (L)
a middle group consisting of the remaining 46%. (M)
Item Analysis
Interpretation of Item-discriminability index (d)
Item-discriminability index Interpretation
(d)
0.40 and above Very good item
0.30 – 0.39 Good item
0.20 – 0.29 Fair item
0.09 – 0.19 Poor item
0.08 and below Very poor item
Preparing for Item Analysis
Arrange test scores from highest to lowest.
Get 27% of the papers from the highest scores and
another 27% of the papers from the lowest
scores.
Record separately the number of times the correct
answer was chosen by the test takers in each
group.
What To Do After Item Analysis
When should a test item be rejected? Retained? Modified
or revised?
A test item can be retained if it’s level of difficulty is easy, optimum,
or difficult and discriminating power is fair to very good.
It has to be rejected if it is either very easy or very difficult and its
discriminating power is poor, negative, or zero.
An item can be modified if its difficulty level is optimum and its
discriminating power is negative.
Creating Norms
Once you have completed getting the raw scores
of your sample, create a table showing the
equivalent Z score, percentile and stanine for
each score. Include a qualitative interpretation
of the scores.
Raw Score Z score Percentile Stanine Qualitative
Interpretation
73 2.2 98.61 9 Very High
61 1.5 93.32 8 High
54 1.1 86.43 7 Above Average
42 0.5 69.15 6 Average
Any Questions?
“In deep waters, you
encounter only the wise
and the brave; in
shallow waters, the
ignorant and the
coward!”
- Mehmet Murat Ildan
Thank You!