English Language Testing Overview
English Language Testing Overview
Bicol University
OPEN UNIVERSITY
Daraga, Albay
Email: bu-ou@[Link]
[Link]
A language test which seeks to find out what candidates can do with language provides a
focus for purposeful, everyday communication activities. Such a test will have a more useful
effect on the learning of a particular language than a mechanical test of structure. In the past
event good tests of grammar, translation or language manipulation had a negative and even
harmful effect on teaching. A good communicative test of language, however, should have a
much more positive effect on learning and teaching and should generally result in improved
learning habits.
Next, a test has the purpose of measuring. Some measurements are rather broad and
inexact, while others are quantified in mathematically
mathematically precise terms. The difference between
formal and informal assessment exists to a great degree in the nature of the quantification of
data.
A test measures a person’s ability or knowledge. Care must be taken in any test to
understand who the test-takers
test are. What is their previous experience and background? Is the
test appropriate for them? How are scores to be interpreted for individuals?
Finally, a test measures a given domain.. In the case of a proficiency test, even though the
actual performance on the test involves only a sampling of skills of a language. Other tests may
have more specific criteria. A test of pronunciation might well be a test only of a particular
phonemic minimal pair in a language. One of the biggest obstacles to overcome in construc
constructing
ting
adequate tests is to measure the desired criterion and not inadvertently include other factors.
The function indicated in the preceding paragraph provides one of the answers to question;
why test? But it must be emphasized that the evaluation
evaluation of student performance for purposes of
comparison or selection is only one of the functions of a test. Furthermore, a good classroom
test will also help to locate the precise areas of difficulty encountered by the class or by the
1
individual student. Just as it is necessary for doctor first to diagnose the patient’s illness, so it is
equally necessary for the teacher to diagnose the student’s weakness and difficulties. Unless
the teacher is able to identify and analyze the errors a student makes in handling the target
language.
The test should also enable the teacher to ascertain which parts of the language program
have been found difficulty by the class. In this way the teacher can evaluate the effectiveness of
the syllabus as well as the methods and materials he or she is using. The test result may
indicate, for example, certain areas or the language syllabus which have not taken sufficient
account of foreign learner difficulties or which, for some reason, have been glossed over. A test
which sets out to measure students’ performance as fairly as possible without in any way setting
traps for them can be effectively used to motivate them.
Before a test is constructed, it is important to question the standards which are being set.
What standards should be demanded of learners of a foreign language? For example, should
foreign language learners after a certain number of months or years be expected to
communicate with the same ease and fluency as native speakers.
Examinations in the written language have in the past artificial standards even for native
speakers have often demanded skill similar to those acquired by the great English essayists and
critics. In imitating first language examinations have proved far more unrealistic in their
expectations of the performances of foreign learners, who have been required to write some of
the greatest literary masterpieces in their own words or to write original essays in language
beyond their capacity.
Four major skills in communicating through language are often broadly defined as listening,
listening and speaking, reading and writing. In many situation, where English is taught to
perform as many genuinely communicative tasks as possible. Where it is important for the test
writer to concentrate on those types of test items which appear directly relevant to the ability to
use language for real-life communication, especially in oral interaction. Thus, questions which
test the ability to understand and respond appropriately to polite requests, advice, instructions,
etc. would be preferred to test of reading aloud or telling stories. In the written section of a test,
questions requiring students to write letters, memos, reports and messages would be used in
place many of the more traditional composition used in the past. In listening and reading tests,
questions in which students show their ability to extract specific information of a practical
nature would be preferred to questions testing the comprehension of un important and irrelevant
details.
Ways of assessing performance in the four major skills may take the form of tests of:
it is the test constructor’s task to assess the relative importance of these skills at the various
levels and to devise an accurate means of measuring the student’s success in developing these
skills.
In an attempt to isolate the language areas learnt, a considerable number or tests include
section on:
2
phonology (concerned with phonemes, stress and intonation)
These tests measure students’ ability to recognize appropriate grammatical forms and to
manipulate structures.
Although it (1) ……. quite warm now. (2) …… will change later today. By tomorrow
morning, it (3) ……… much colder and there may even be little snow …. (etc.)
Note that this particular type of question is called a multiple-choice item. The term multiple-
choice item is used because the students are required to select the correct answer from a
choice of several answers. The word item is used in preference to the word question because
the latter word suggests the interrogative form; many test items are, in fact, written in the form of
statements.
A test of vocabulary measures students’ knowledge of the meaning of certain words as well
as the patterns and collocations in which they occur. Such a test may test their active
vocabulary (the words they should be able to use in speaking and in writing) or their passive
vocabulary (the words should be able to recognize and understand when they are listening to
someone or when they are reading) obviously, in this kind of test the method used to select the
vocabulary items(=sampling) is of the outmost importance.
In the following item students are instructed to circle the letter at the side of the word which
best completes the sentence.
In another common type of vocabulary test students are given a passage to read and
required to replace certain words at the end of the passage with their equivalents in the
passage.
X. TEST OF PHONOLOGY
Test items designed to test phonology might attempt to assess the following such skills;
ability to recognize and pronounce the significant sound contrasts of a language, ability to
recognize and use the stress patterns of a language, and ability to hear and produce the melody
or patterns of the tunes of a language (i.e. the rise and fall of the voice).
In the following item, students are required to indicate which of the three sentences they hear
are the same;
Spoken:
Although this item, which used to be popular in certain tests, is now very rarely included as a
separate item in public examinations, it is sometimes appropriate for inclusion in a class
progress or achievement test at an elementary level. Successful performance in this field,
however, should not be regarded as necessarily indicating an ability to speak.
3
XI. RECOGNITION AND PRODUCTION
Methods of testing the recognition of correct words and forms of language often take the
following form in tests:
This multiple-choice test item tests students’ ability to recognize the correct form: this ability is
obviously not quite the same as the ability to produce and use the correct form in real-life
situations. However, this type of item has the advantage of being easy to examine statistically.
If the four choices were omitted, the item would come closer to being a test of production:
Students would then be required to produce the correct answer (=for). In many cases, there
would only be one possible correct answer, but production items do not always guarantee that
students will deal with the specific matter the examiner had in mind (as most recognition items
do).
A good language test may contain either recognition-type items or production-type items, or
a combination of both. Each type has its unique functions, and these will be treated in detail
later.
A good test should never have constructed in such a way as to trap the students into giving
an incorrect answer. When techniques of error analysis are used, the setting of deliberate trap
or pitfalls for unwary students should be avoided many testers, themselves, are caught out by
constructing test items which succeed only in trapping the more able students. Care should be
taken to avoid trapping students by including grammatical and vocabulary items which have
never been taught.
In the following example, students have to select the correct answer (C), but the whole item
is constructed so as to trap them into making choice B or D. When this item actually appeared
in a test, it was found that the more proficient students, in fact chose B and D, as they had
developed the correct habit of associating the tense forms have seen and have been
seeing with since and for.
When I met Tim yesterday, it was the first time I ………. him since Christmas.
To summarize, all tests should be constructed primarily with the intention of finding out what
students know – not of trapping them. By attempting to construct affective language tests, the
teacher can gain a deeper insight into the language he or she is testing and the language
learning process involved.
4
PART II: OBJECTIVE AND SUBJECTIVE TESTING
Subjective and objective are terms used to refer to the scoring tests. All test items, no matter
how they are devised, require candidates to exercise a subjective judgment. In an essay test, for
example, candidates must think of what to say and then express their ideas as well as possible; in a
multiple choice test they have to weigh up carefully all the alternatives and select the best one.
Furthermore, all tests are constructed subjectively by the tester, who decides which areas of
language to test, how to test those particular areas, and what kind of items to use for this purpose.
Thus, it is only the scoring of a test that can be described as objective. This means that a testee will
score the same mark no matter which examiner marks the test.
Since objective tests usually have only one correct answer (or, at least, a limited number of
correct answer), they can be scored mechanically. The fact that objective tests can be marked by
computer is one important reason for their evident popularity among examining bodies responsible
for testing large numbers of candidates.
Clearly, the ability to write can only be satisfactorily tested by a subjective examination requiring
the students to perform a writing task similar to that required in real life. On the other hand, reliability
will not be difficult to achieve in the marking of the following objective item. The question of how valid
such an item is, however, may now be of considerable concerned. How far do items like this reflect
the real use of language in everyday life?
A. for B. on C. in D. at E. since
On the whole objective tests require far more careful preparation than subject tests. Examiners
tend to spend a relatively short time on setting the questions but considerable time on marking. In
objective test the tester spends a great deal of time constructing each test item as carefully as
possible, attempting to anticipate the various reactions of the testees at each stage. The effort is
rewarded, however, in the case of the marking.
I. OBJECTIVE TESTS
Objective tests are frequently criticized on the grounds that they are simpler to answer than
subjective tests. Items in an objective test, however, can be made just as easy or as difficult as the
test constructor wishes. The fact that objective tests may generally look easier is no indication at all
that they are easier. Another criticism is that objective tests of the multiple-choice type encourage
guessing. However, four or five alternatives for each item are sufficient to reduce the possibility of
guessing. Furthermore, experience shows that candidates rarely make wild guesses on partial
knowledge.
- irrelevant areas and skills are emphasized in the test simply because they are ‘testable’; and
The distinction here is between methods of scoring, and nothing else. If no judgment is required
on the part of the scorer, then the scoring is objective. A multiple-choice test, with the correct
responses unambiguously identified, would be a case in point. If judgment is called for, the scoring is
said to subjective. There are different degrees of subjectivity in testing. The impressionistic scoring
5
of composition may be considered more subjective than the scoring of short answers in response to
questions on a reading passage.
Objectivity in scoring is sought after by many testers, not for itself, but for the greater
reliability it brings. In general, the less subjective the scoring, the greater agreement there will
be between two different scorers (and between the scores of one person scoring the same test
paper on different occasion). However, these are ways of obtaining reliable subjective scoring,
even of compositions.
It should always be remembered that such items as the above test the ability to recognize or
produce correct form of language rather than the ability to use language to express meaning,
attitude, emotions, etc. nevertheless, it is essential that students master the grammatical system
of the language they are learning. Thus, classroom tests of grammar and usage can play a
useful part in a language program.
The type of multiple-choice item favored by many constructors of grammar tests is the
incomplete statement type, with a choice of four or five options. These items may be written in
any of the following ways:
Type 1 Tom ought not to ……… (A. tell B. having told C. be telling D. have told) me your
secret, but he did.
A. tell
B. having told
C. be telling
D. have told
A. tell
Type 3 Tom ought not to B. having told me your secret, but he did.
C. be telling
D. have told
Type 4 Tom ought not to have told me your secret, but he did.
A. No change
B. Tell
C. Having told
D. Be telling
Item types 2 and 3 are preferable to 1 because the option do not interrupt the flow of
meaning in the sentences: these items present the entire sentence so that it can be read at a
glance. Unfortunately, type 1 confuses the reader because of the long parenthesis (i.e. the four
options occurring between ought not to and me). Item type 4 shows the correct (or an incorrect)
form as part of the sentence in such a way that it cannot be compared on equal terms with the
other options: a correct option, for instance, is generally easier to recognize when it appears in
the framework of the sentence than as part of a list of distractors.
Another item type appears below, but it is not recommended since it requires the testees to
spend time on unnecessary reading. Not only is it uneconomical but it does not present the
‘problem’ (i.e. the options) as clearly as item type 2 does.
6
B. Tom ought not to having told me your secret, but he did.
C. Tom ought not to be telling me your secret, but he did.
D. Tom ought not to have told me your secret, but he did.
The following method is useful for testing short answers and responses:
In type 7 requires the students to select the alternative which is true according to the
information conveyed in each sentence. Such an item may be included either in a test of
reading comprehension or in a test of understanding of the sentence.
It may be argued that an understanding of syntactical patterning is just as necessary for the
following item:
‘Three times.’
Items which appear in a test of grammar and structure should be made to sound as natural
as possible. The following mechanical test item:
Note that distractors should generally be correct both in written and in speech. The following
item proved unsuccessful when it was included in a test because many of the more able
students selected option D. the reason being that they pronounced used to quite correctly as
use to/ju:stә/:
Note that sample items discussed in this section have so far taken the form of short
decontextualized items. In practice, however, such items would all form part of a paragraph or
series of paragraphs of descriptive, narrative or expository prose. The provision of a detailed
context in this way, however, often limits the range of grammatical features being tested. It is
usually impossible, for example, to test the past continuous tense in a narrative set in the past
(unless direct speech is used). Similarly, a paragraph describing a simple manufacturing
process may not provide the test writer with the opportunity to test all the verb forms and tenses
7
he or she may wish to test. This is the price to be paid for including more natural, contextualized
items. This is real language used for a particular purpose. Furthermore, the provision of context
helps to ensure tat there is only one correct option in each case. Short decontextualised
sentences can lead to ambiguity as they are usually open to several interpretations when used
as stems for multiple choice items. For example, option D in the following decontextualised item
might be correct (as well as option B) if the student happens to assist with research and
deliberately catch a cold so as to be able to test various cures.
Much better for testing purposes is the following item. The passage is taken straight from a
newspaper article and thus the language is authentic background knowledge and details to
avoid ambiguity and alternative interpretations, and the newspaper report itself is very
interesting. Does it really matter if it will not allow us the opportunity to test every point of
grammar which we may want to test. Students taking this test are being given a real feel for the
language they are learning.
A 72-year-old Samoan who (1) ……… no English at all spent thirteen days (2) ……. on buses
in the San Frasisco area he had become separated (3) ….. his family, police said. (4) ….. said
that Faitua Logo, (5) …….. moved to the United States two-years ago, left his son and
daughter-in-law, etc………
Although it is not always possible to use samples of students’ own written work to provide
the basis for the test items, it should not be too difficult for constructors of classroom test and
school achievement tests to take advantage of the types of errors made by students in their free
compositions and open-ended answers to questions.
Item 1
Let us ignore the error in the first sentence for the time being and concentrate on the error of
tense after hope.
Step 1: The first step is to reduce the length of the sentence and to correct the error (and any
other errors in the original sentence). Thus,
I hope that you wouldn’t mind on such a long period between my last letter and this one.
Step 2: Next we write out the sentence, substituting a blank for the area being tested. We write
in the correct option and the distractor which the student has provided for us. However, we have
to add a sentence because in certain (rare) contexts, wouldn’t may be correct.
I hope you ……. mind waiting for so long. I promise to reply sooner in the future.
A. won’t B. wouldn’t
Step 3: We now add another two distractors. Again, we go to the written work of our students to
provide these distractors. But if we cannot locate any suitable errors without too much difficulty,
we use our own experience and knowledge of the target and native languages. Thus, two useful
distractors which would also balance the existing two options might be shouldn’t and shan’t.
8
Step 4: One suggestion may be that we replace shan’t with can’t. if students from a particular
language background make such mistakes as can’t mind, can’t should be used as a distractor,
and possibly shouldn’t changed to couldn’t. as can be seen at this early stage, the actual
process of item writing is extremely subjective.
I hope you ……. mind waiting for so long. I promise to reply sooner in future.
Unfortunately, both don’t and didn’t are correct. However, in the following context, didn’t is not
acceptable.
‘About half an hour. I hope you ……. mind waiting for so long.
It may be argued that didn’t stands out too much. If so-and if it is equally useful to test the use
of don’t (instead of won’t) after hope – the item could be rewritten as:
Item 2
A. looking C. looking on
B. looking at D. looking to
Some test constructors might be tempted to use for as a distractor. It can be argued,
however, that looking for is correct: old Mr. Jones might enjoy looking for the children playing
(i.e. he might enjoy walking through the park, chatting to his friends, etc. while he is in the
process of looking for his grandchildren, who are playing).
Note that the correct option is now in the third position. C. it is important to vary its position.
Note also that the word looking appears in each option: in some tests the item might appear as
follows:
A. - B. on C. at D. to
However, when this format includes a dash (-), it is unnatural and not recommended since the
insertion of a dash in the stem would not be normal practice in real life.
Item 3
9
Note that at is also incorrect and may be used as a possible distractor. On the other hand, it
may be felt that a number of native English speakers do say angry at a person. The decision
whether or not to include at in the list of incorrect option is again a very subjective one.
The sentence ’Sun is shining, trees become green and ….. The error caused by the omission of
the article may be tested as follows, using a multiple-choice item:
It may be argued, however, that the choice here is strictly between option A and B at certain
levels where students have learned to avoid using ‘a’ and ‘some’ with ‘sun’. in such instances,
one useful device (still using the multiple-choice format) is the error-recognition type of item.
Type 1
Each sentence contains four words or phrases underlined, marked A, B, C and D. Select the
underlined word or phrase which is incorrect or unacceptable.
A B C D
A B C D
A B C D
A B C D
Type 2
There is a mistake in grammar in each of the following sentences. Write the letter of that part of
the sentence in which it occurs.
A B C D
A B C D
A B C D
A B C D
Item type 2 allows the test writer to test errors caused by omission: e.g. Sun is
shining and looking the children. This type of error cannot be tested by the first item of the error-
recognition type. However, there are different ways of correcting many sentences. For example,
students may write B or C to denote the incorrect part of the third sentence above, according to
which of these correct versions is in their mind:
Tony’s father would not permit him to stay out late. (=B)
Tony’s father would not let him stay out late. (=C)
For this reason, the test writer is strongly advised to avoid items of the second type.
10
IV. CONSTRUCTING REARRANGEMENT ITEMS
Rearrangement items can take several forms, the first of which to consider here will be the
multiple-choice type.
If we attempt to test the first error by means of an ordinary multiple-choice item, we are faced
with the problem of being restricted to only two options: the correct option and the distractor (i.e
the error).
A. it is B. is it
As the item stands here, we cannot possibly construct other options. It becomes necessary,
therefore, is lengthen the original statement to: You know how warm it is today. The item would
then read:
A. warm is it today
B. today is it warm
C. is it warm today
D. warm it is today
E. today is it warm
There seems to be a danger here of confusing the testees by presenting them with the
problem in such a way that a certain amount of mental juggling becomes necessary on their
part. A preferable item type is the following word-order item:
Complete each sentence by putting the words below it in the right order.
A. it B. today C. warm E. is
Word order items are useful for testing other structures and features involving inversion:
A. me B. he C. did D. meet
11
The order of adjectives and the position of adverbs can be tested in this way, as indeed can
several other grammatical areas
Carefully constructed completion items are a useful means of testing a student’s ability to
produce acceptable and appropriate forms of language. They are frequently preferable to
multiple-choice items since they measure production rather than recognition, testing the ability
to insert the most appropriate words in selected blanks in sentences. The words selected for
omission are grammatical or functional words (e.g. to, it, in, is, she): content words may be
selected in a vocabulary or reading test.
The error Sun is shining, although only one answer is possible here, this completion item
would have to appear as:
The former item suggests to the testees that no determiner is necessary (since Sun is
written with capital letter) while the latter item suggests that a determiner is necessary
(because sun is written without a capital).
The main item can be simply be rewritten as a question to overcome its problem:
Here are two more examples of completion items based on the student’s letter:
Put a, the, or some in each blank only where necessary. If you think that no word should be
placed in the blank, put a cross (x) there.
Completion items cannot, of course, be machine-marked but they are very useful for
inclusion in classroom tests and for exercise purposes. The following example indicates the
wide range of possibilities for one completion item:
12
The answer obviously required by the tester is haven’t been; however, possible answers
are :
If the aim of this particular item is to force the use of the present perfect tense, there are three
ways of restricting the choice available to the testees (although the first two ways depend
heavily on reading comprehension):
Kim usually goes to the cinema about once a week but she …….four films already this month
and it’s only the 20th today. (possible answers: has seen/will have seen/must have seen)
I go to the cinema regularly, but it’s ages since I last saw a play.
(Possible answers: haven’t been/haven’t gone/haven’t been going/haven’t been able to go)
A. go C. had gone
B. have been D. went
There are two major advantages in using a passage of continuous prose rather than separate
sentences when giving a completion type test. Firstly, the use of context often avoids the kinds
of ambiguity referred to in the previous paragraphs. Secondly, the students experience the use
of grammar in context, being required to use all the context clues available in order to guess
many of the missing words.
The transformation type of item is extremely useful for testing ability to produce structures in
the target language and helps to provide a balance when included in tests containing multiple-
choice items. It is the one objective item type which comes closest to measuring some of the
skills tested in composition writing, although transforming sentences is different from producing
sentences.
The following transformation items have been based on errors which occurred in the student’s
letter.
Rewrite each of the following sentences in another way, beginning each new sentence with the
words given. Make any changes that are necessary but do not change the general meaning of
the sentence.
13
Other transformation items giving some idea of the range of areas that can be covered in this
way are:
This type of item is useful for testing the student’s ability to use correct tenses and verb
forms. It is a traditional type of test but the layout is improved in this particular case by providing
blanks on the right of the text for completion. The continuity of the text is not impaired more than
necessary by having both blanks and underlined words inserted in the sentences. Thus the risk
of obscuring the meaning of the text is reduced.
14
VIII. CONSTRUCTING ‘BROKEN SENTENCE’ ITEMS
This type of item tests the student’s ability to write full sentences from a series of words and
phrases, and thus not allow the test writer to concentrate exclusively on testing those particular
grammatical features which may have just been practiced in class. It is nevertheless a useful
device for testing grammar provided that the tester is aware that several other areas of the
language are being tested in addition to those on which he or she wishes to focus attention.
In this type of test item, students should be instructed to make whatever changes are
necessary to form good sentences, adding articles, prepositions, etc. where required and
putting verbs in their correct tense.
Take / drug and stimulants / keep awake / while revise examination / often be very harmful / it
be far better / lead / balanced life / and get enough sleep / every night. / There / be / limit /
degree and span / concentration / which you be capable/ exert. / Brain / need rest / as much
body. / Indeed, / it be quality / than quantity work / that be important.
This type of item usually consists of a short conversation: e.g. a stimulus in the form of a
statement or question followed by a response often in the form of a statement. It is used to test
the ability to select appropriate responses to stimuli which would be presented orally in normal
everyday situations. The item is more useful for testing students’ sensitivity to appropriacy and
their awareness of the functions of language rather than their knowledge of grammar (although
grammatical clues may prove important in completing this item satisfactorily). To perform the
task required, students are simply required to write the letter of the correct response in the
space provided.
It should be remembered, of course, that this is not authentic discourse. However, although
the language and situation here are inevitably artificial, the item does serve to help students to
associate the language they are learning with real-life situations, albeit to a limited extent.
These objective-type items have long been used in past tests. They should be used
sparingly, however, as they involve largely mechanical responses on the part of the student.
Note that although the separate sentences are linked to one another by theme, the items can
hardly be described as being contextualized in any real way.
1. Combination items
(Students are instructed to join each pair of sentences using the word in brackets)
(a) You finish the paper then check your answers carefully. (AFTER)
(b) Some questions may be very difficult. They should be left until later. (WHICH)
15
(c) You should usually write answers in complete sentences for all the questions on
your paper. However, write notes for those questions which you do not have time to
answer. (ALTHOUGH)
2. Addition items
(Students are instructed to insert the word in capitals in the most appropriate place in each
sentence.)
(b) STILL Some students had not mastered the correct techniques for answering
examination questions.
(c) OCCASIONALLY There may be little choice of questions.
16