Unit- 3
Constructing the items
Definition
•Items are specific questions or problems that make up a test (Kaplan &Saccuzo, 2009).
An item is a specific stimulus to which a person responds overtly (i.e can be observed) or can be scored. This
response can be scored or evaluated . For example on a scale or grade e.g. 75% meaning out of a 100-item test, the
individual has scored 75 items correct.
A test is a measurement device or technique used to quantify behavior or help in understanding and prediction of
behavior. It is also termed as a collection of items. It requires the proper selection of item types, like multiple
choices, true/false questions, and open-ended questions in respect of content to be assessed. In conclusion, high-
quality item writing promotes the overall effectiveness of an assessment tool bringing about meaningful evaluation
and interpretation of results. Item writing is a basic process of evaluation whereby questions or tasks designed to
assess particular knowledge, skills, or attitude in the learners are produced. It should strike an acceptable balance
between clarity, relevance, and fairness so that each item represents what is intended by the constructs that are being
measured. Test items may range from different categories, including multiple choices, true or false, and even open-
ended questions. This type of variety would better evaluate and assess the diverse understanding and cognitive
processes. However, it is only to a limited extent that test-item qualities influence the validity and reliability of the
measure itself. Hence, careful planning, scrutiny, and editing are also essential for item writers. Ultimately, good
item-writing skills improve the quality of an assessment and will therefore provide useful information on a learner’s
performance, thus enabling an informed educator or professional decision. Item writing best practices help in
developing good evaluation, which allows valid and reliable measurements of the knowledge, skills, or
psychological constructs that the learners are being tested on.
Item writing involves 8 key steps:
1. Defining what is to be measured: Clearly specifying the concept, trait, or behavior you want to assess or
measure through your test or survey items.
2. Generating an item pool: Creating a large set of potential questions or items that could measure the
concept or trait of interest. It’s like making a big list of possible questions for your test or survey. You write
down lots of ideas, and then pick the best ones later!.
3. Avoiding long items: Keeping questions concise (Creating a list of potential questions for a test or survey)
and to the point, avoiding complex or lengthy sentences that may confuse respondents.
4. Considering reading level: Ensuring that the language and vocabulary used in the items are
understandable by the target audience.
5. Avoiding double-barrelled items:
Asking about only one concept or issue per item, avoiding questions that combine two or more separate making
it unclear which part the test-taker is responding to. This can lead to confusion, inaccurate responses, and
unreliable results.
Example: “Do you like reading books and watching movies?”
6. Mixing positively and negatively worded items: Balancing the wording of items to avoid response bias,
using both positive and negative phrasing to ensure respondents are paying attention.
7. Considering cultural sensitivity: Ensuring that items are free from cultural bias and are relevant and
appropriate for the target audience, avoiding offense or misunderstandings.
8. Realizing items become obsolete: (“Obsolete” refers to the test items are no longer useful, relevant, or
effective for measuring what they’re supposed to measure).
Recognizing that test items can become outdated over time and need to be reviewed and updated regularly
to remain relevant and effective.
Type of test items
A written test composed of a number of test items. Items differ on the basis of the type of responses they involve.
There are items in which responses are free. They are called free response type test items. Here respondents have
freedom to respond in his own way. In fixed response type test items he does not have this freedom. There are three
different types of test items coming under these two categories that are objective type items, short answer type items
and essay type items.
1. Objective Type items
An objective type test item is one in which the response and hence the scoring will be objective. In order to ensure
such objectivity, the responses are made fixed whereby the freedom of the respondent to deviate subjectively is
restricted. Objective type test items can be broadly classified into
i. Supply type (Recall type)
ii. Selection type (Recognition type)
For supply type test items the respondents have to supply the response where as for selection type they have to select
the responses among the given responses. The generally used objective type items are true-false items, multiple
choice items, matching type and completion type. Of these completion type items are supply type while the
remaining belongs to selection type.
A)True-false items: In this form, the respondent is asked to read a statement and indicate in some specific manner
suggested, whether it is true or false. When a number of such items are included in a test, some of them may be true
and the remaining ones are false, arranged in random order.
It is advisable that avoid ambiguity in the statement as much as possible. For this avoid words like all, never, always,
frequently, in most cases, etc. Another point should be kept in mind that uses positive statements as far as possible. It
is also desirable to make the given statements in a set of true-false items almost equal in length. Likewise, the
number of true items and false items should be balanced.
The greatest drawback of true-false items is that the examinee guesses and even when one does not possess the
knowledge about the subject matter. He gets ½ possibility to get it right.
B)Multiple Choice items: Multiple choices item has been designed t reduce the limitation of true-false item to the
minimum. In this type, a question had to be answered by the respondent by selecting the correct response out of the
alternative responses provided for. If carefully prepared, such items can be used to appraise even the most complex
behavioral changes associated with instructional objectives.
A multiple choice item has two parts- the stem and the options. The stem presented in the form of an incomplete
sentence or an interrogative sentence. The number of options may be generally four or five, out of which one is the
correct answer and hence it is called keyed response and the all other options are called distracters.
The point should be kept In mind that the distracters should be meaningfully connected in the sense that they really
distract the thoughts and mode of reasoning of the respondent. Another point is that the stem should be short and at
the same time, reflect the real problem in precise manner. Avoid unnecessary adjectives and complications that
might confuse the examinee. It is advisable to avoid negative statement, as far as possible, as stems.
C)Matching type items: This is a modified form of multiple choice items. In fact matching type comprises an
economical from of combining a number of multiple choice items in the same question. This is more appropriate
where the options for a number of stems are mutually related and sometimes repeating. In this type the student has to
associate a word or a phrase given in another column containing options.
D)Completion type items: A test item which requires the examinee to fill-in the blank is called a completion test
item. He has to supply the word or phrase omitted. Care should be taken to avoid ambiguous statements. The answer
should be only a key word rather than unimportant details.
Advantages :
•It will ensure the objectivity of the test.
•It will ensure the coverage of the content area.
•It will reduce the language expertise, speed in writing good handwriting, neatness, etc. of the learner.
•It promotes economy of time, and ensures the practicability of the test.
•It will ensure the reliability of the test.
•It will ensure the validity of the test.
•A well prepared objective type item will be able to evaluate the higher order objectives also.
Limitations of Objective Type Items:
•It is very much difficult to prepare a good objective type test item. It demands very much time, talent and
imagination of the examiner.
•It creates a problem of guessing.
•It creates an opportunity to cheating (copy answers from other scripts).
•This type of test item is not able to check the depth of the knowledge of the learner.
•It is very much difficult to test complicated skills of the examiner with objective type items.
•It demands high printing cost.
2. Short Answer type items
A question having two or three value points at the most may be regarded as a short answer type test item. The
term value point indicate that a point to be given credit in the expected answer.
A short answer type Item is a question that requires a brief answer, typically covering one or two key points, to
assess a student’s understanding of a specific concept or topic. It’s a type of question that demands a concise
and to-the-point response, usually a sentence or two, or a short phrase.
Advantages:
•A relatively large proportion can be covered in a test.
•It is easy to construct.
•It provides little opportunity for guessing because the examinee demands to supply specific information.
•If proper care gave at the time of preparing the items, objectivity can be ensured to a certain extend.
Limitations:
•It is more subjective than objective type test item.
•Excessive use of this type of test item may encourage the rote memorization of the learners.
•Mechanical scoring is not possible.
3. Essay type items
This is the conventional type of question calling for a rather long answer covering a number of pints and a variety of
objectives. This type test items will help in evaluating complex skills and other behavioral pattern. An essay type
item is a question that requires a student to write a detailed answer, typically covering multiple points and
perspectives, to assess their understanding, critical thinking, and writing skills. It’s a conventional type of question
that allows students to express their thoughts and ideas in a more comprehensive and elaborate way.
Advantages:
•It is easy to prepare.
•The method of administration of test item is rather simple.
•It is useful to measure complex skills.
•It is very much useful to evaluate the depth of knowledge of the examinee.
•The possibility of guess work and spot copying can be reduced to the maximum.
•Comprehensiveness of the subject matter is ensured.
Limitations:
•They are very much subjective in nature. Mark allotment will vary widely.
•It is very difficult to ensure the comprehensiveness of the content area. It is very difficult to prepare essay type
items from every content area.
•Linguistic ability, speed in writing, handwriting ability, neatness, etc. of the examinee will influence very much in
this type of test items.
•It is very difficult to administer every time.
•Proper evaluation of the specific abilities is very difficult.
•It is very difficult to ensure the reliability and validity of these types of test items.
GENERAL GUIDELINES FOR ITEM WRITING
The different types of items used in psychological, sociological and educational researches, and some specific
guidelines for writing effective items have already been discussed. Besides these specific guidelines, the item writer
must also keep in view some general guidelines, which are essential for writing good items. These are listed as
follows.
1. Clarity in writing test items is one of the main requirements for an item to be considered good. Test items
should be written clearly and simply, so they can accurately assess the examinee's knowledge, not their
ability to understand complex language. Vague items can lead to misunderstandings and affect the test's
validity, often due to poor thinking or lack of skill on the item writer's part.
2. Nonfunctional words must not be included in the item as they tend to lower the validity of the item.
Nonfunctional words are extra words in a test item that don't help the examinee pick the right answer. Item
writers might add them to hide the correct answer or create a tricky distractor.
3. The item writer should avoid including irrelevant details that might mislead examinees or make them think
a statement is true when it’s not. This shows poor thinking on the part of the item writer.
4. The items must not be too easy or too difficult for the examinees. The level of difficulty of the item should
be adaptable to the level of understanding of the examinees. Experienced writers can adjust item difficulty.
Similar answer choices make it harder, different choices make it easier. They should balance difficulty for
all examinees.
5. Use of stereotyped words either in the stem or in the alternative responses must be avoided because these
facilitate rote learners in guessing the correct answer. Moreover, such stereotyped words fail to discriminate
between those who really know and understand the subject, and those who do not. Thus, stereotyped words
do not provide an adequate and discriminatory measure of index. The most obvious way of getting rid of
such words is to paraphrase the words in a different manner so that those who really know the answer can
pick up the meaning.
6. Irrelevant clues must be avoided. Item writers should avoid giving hints that lead examinees to the correct
answer without actually knowing the content. Irrelevant clues can be given through:
•Clang association (similar sounding words).
•Verbal association (word connections).
•Length of the answer (longer or shorter options).
• Keeping a different foil (odd option that stands out).
Homogeneous foils, giving the same order of the correct answer, etc. In general, such clues tend to decrease
the difficulty level of the item because they provide an easy route to the correct answer. The common
observation is that examinees who do not know the correct answer, choose any of these irrelevant clues and
answer on that basis. The item writer must, therefore, take special care to avoid such irrelevant clues.
Specific determiners like never, always, all, none must also be avoided because they are also irrelevant
clues to The correct answer, especially in the two-alternative items.
7. Interlocking items must be avoided. Interlocking items, also known as interdependent items, are items that
can be answered only by referring to other items. In other words, Interlocking items are questions that can
only be answered correctly if you know the answer to another question. They depend on each other, making
it unfair for examinees who don't know the first question. For example:
1. Item 1: The capital of France is Paris. True/False
2. Item 2: The city mentioned in Item 1 is in Europe. True/False
(Answering Item 2 depends on knowing the answer to Item 1)
Recently, Devellis (1991) has provided some encouraging and stimulating guidelines which are worth considering.
Six of these guidelines may be presented as under:
1. Define what you’re testing: Use theory to guide you and make items specific.
2. Keep it short: Avoid long items, they’re not effective.
3. Match the reading level: Make sure it’s right for the test-takers.
4. Create a pool of items: Write multiple items for each concept and avoid repetition.
5. One idea per item: Avoid “double-barreled” items with multiple ideas.
6. Mix positive and negative items: Prevents people from just agreeing with everything.
Example: For depression, use “I feel sad” (positive) and “I feel hopeful” (negative) to balance it out.