Chapter 1: Preliminaries of Language Testing
Today's Learning Outcomes By the end of this session, you will be able to:
Explain why testing is crucial in language education
Distinguish between test, measurement, and evaluation
Analyze the different purposes of various test types
Evaluate responses based on communicative appropriateness
Apply key concepts to real-world language testing scenarios
1.1 The Importance of Testing Why Does Testing Matter?
Not a new concept: Measurement and evaluation have always been part of education
Why it is more critical NOW:
Educational goals are more complex
Student populations have exploded
Programs are constantly being revised
Education is a GIANT enterprise:
Second largest economic sector (after defense)
Over 25% of the population attends school
Every citizen is involved at some point
"It is crucial that [education's] processes and products be evaluated."
Who Needs Testing Information? Stakeholders in Educational Testing
Teachers
Want to know: How much have my students learned?
Government and Employers
Want to know: What can these graduates actually do?
Students
Want to know: Am I making progress toward my goals?
Parents and Administrators
Want to know: Are our educational goals being achieved?
All stakeholders work together toward educational goals. Evaluation tells us if we are
getting there.
1.2 Testing and Decision Making
Every Educational Decision Requires Information
Decisions Great or Small ↓ Should be based on ACCURATE INFORMATION Tests provide
this
Examples of Decisions:
Should Reza be in an advanced reading group? (Placement)
Should we keep this textbook? (Curriculum)
Is grammar overemphasized? (Instructional balance)
Am I doing well in English? (Self-assessment)
Should I go to college? (Life planning)
"A good decision is one that is based on all relevant information."
1.3 Key Terminology Test vs. Measurement vs. Evaluation
TEST
The Narrowest Term
A specific set of questions or tasks
Example: A 50-item grammar quiz
MEASUREMENT
Broader Term
The process of assigning a number to a characteristic
Example: Scoring the quiz = 42/50
Can use tests, observations, rating scales
EVALUATION
The Broadcast Term
Making a JUDGEMENT about value or desirability
Example: "42/50 shows mastery of grammar"
Requires a GOAL or OBJECTIVE to judge against
CRITICAL POINT: We NEVER measure or evaluate PEOPLE. We measure or evaluate
CHARACTERISTICS of people:
Knowledge of English
Scholastic potential
Ability to teach
The Same Score, Different Evaluation Why Context Matters in Evaluation
Scenario: End of 3rd Grade Junior High - Both students score at 3rd grade level
Student A
Started the year: 1st grade level
Gained: 2 years of knowledge
EVALUATION: Above Average Progress
Student B
Started the year: 2nd grade level
Gained: 1 year of knowledge
EVALUATION: Below Average Progress
Same MEASUREMENT → Different EVALUATION (because of different starting points and
goals)
1.4 Language Testing - Two Key Benefits How Testing Helps Students
Benefit 1: Motivation and Positive Experience
Tests provide a sense of ACCOMPLISHMENT
Good evaluation REDUCES frustration and complaints
Key insight:
"Good instruction cannot do much if it is not accompanied with appropriate evaluation."
Benefit 2: Learning and Mastery
Tests encourage PREPARATION
Repeated practice leads to MASTERY
Test analysis reveals WEAKNESSES
Students become more aware of COURSE OBJECTIVES
Students can ADJUST their personal activities
1.5 Why Test? - Guiding Questions A Good Test Should Answer These Questions
About INSTRUCTION
Has instruction been successful?
Were materials at the right level?
Have all skills been emphasized equally?
What points need reviewing?
Should we use the same materials next year?
About the TEST ITSELF
Were the test instructions clear?
Was the allotted time sufficient?
How did students FEEL when responding?
Do results reflect PERFORMANCE during the course?
"Tests can help us improve the quality of instruction in ANY educational setting."
1.6 Teacher-Made vs. Standardized Tests Two Approaches to Achievement Testing
Teacher-Made Tests
Created by classroom teacher
Purpose: Measure specific classroom objectives, diagnose weaknesses, motivate
Content Coverage: Single unit or term
Construction: Can be hurried or haphazard
Administration: Usually no uniform directions
Norms: Local classroom only
Standardized Tests
Created by skilled test-makers and measurement experts
Purpose: Measure broad curriculum objectives, compare across schools or districts
Content Coverage: Wider range, often year or more
Construction: Meticulous with blueprints, tryouts, analysis, revision
Administration: Specific, uniform procedures
Norms: National, district, school building norms
Value of Teacher-Made Tests Why Teacher-Made Tests Are Valuable
Schwartz and Tiedeman, 1957
Measure student progress based on classroom activities
Motivate students
Diagnose student weaknesses
Help plan remedial instruction
Warning: Teacher-made tests MUST be based on predetermined content to avoid
student complaints about irrelevance or ambiguity.
Common student complaints:
"This was a useless exercise."
"I didn't know what the teacher was looking for."
"I was only tested on unimportant points."
1.7 Language Teaching and Language Testing
Teaching and testing are so closely interrelated that it is virtually impossible to work in
either field without taking the other into account. Testing is viewed as a constructive and
practical teaching strategy giving learners useful opportunities for discussion of
language choices.
As Madsen (1983) states:
"Language testing today reflects current interest in teaching genuine communication,
but it also reflects earlier concerns for scientifically sound tests."
1.7.1 Traditional Tests
These tests are closely related to the grammar-translation method in language teaching.
This relationship was much stronger in the early stages. This stage of language testing
could be called the intuitive stage. Many teachers with no training in teaching and
testing would strongly emphasize knowing about the language as well as using the
language. Consequently, students had to memorize many language rules and lists of
words.
Examples of Traditional Test Items:
Example 1 Convert the following statement into past tense: "He drives his car fast."
Answer: He drove his car fast.
Example 2 Write the main parts of these verbs: go, buy, ring, lay Answers: go-went-gone;
buy-bought-bought; ring-rang-rung; lay-laid-laid
Example 3 Make sentences using each of these words: intelligent, bashful, diligent
Example 4 Translate the following passage into Farsi: A few sentences or a short
passage in English
Traditional tests also include a great deal of writing (composition) and reading
comprehension. These item types are very similar to what is used in first-language
testing.
Example 5 Write a composition of about 150–200 words on "the importance of
education in third world countries."
Example 6 A passage for dictation
Example 7 A reading comprehension passage with questions for definitions,
information, or inferences.
1.7.2 Multiple-Choice Items
Multiple-choice items are the most popular type of objective tests. The students are
presented with three, four or five alternatives or options including a correct response
and distractors. The student is expected to choose the correct alternative from a range
of answers provided. The student's task is quite simple and scoring is easy as well.
Example 13
He left home ...... ten o'clock. a) in b) at c) by d) until
This is a typical form of a grammar item. It has been suggested that such an item
measures only a single, or discrete, feature of the grammar.
Characteristics of Multiple-Choice Items:
Multiple-choice tests provide the learner with restricted contexts, usually no wider than
the item context. The testee is given no more information about the speaker or the
situation, and consequently, choosing the appropriate option is difficult. Many
grammatical rules are not observed in informal situations, and it is possible to find
possible contexts for many of the distractors.
Challenges in Constructing Multiple-Choice Items:
Constructing good test items with reasonable distractors is very difficult, and
unfortunately many teacher-made tests are bad ones.
Advice from Elizabeth Ingram:
"The inexperienced test constructors should first prepare open-ended items. These
items should be administered to some students of the sort he wishes to test ultimately.
The wrong answers provided by the students could be used as reasonable distractors
later on."
Uncommon and implausible distractors are dangerous instruments to be used in
language testing. Frequently, many types of multiple-choice tests expose students to a
lot of unlikely errors.
Additional Multiple-Choice Examples Example 14
The teacher ...... me what to do. a) told b) suggested to c) explained to d) said
Example 15
How is everything? a) Not so good. b) All right. c) Thank you so much. d) Everything is all
right.
The examples given have all provided unrelated sentences. It is also possible to provide
much longer contexts in which many test items can be embedded.
Discrete Item Testing The behaviorist view considers the process of language learning
as learning a set of habits. This complex network of habits can be built up step by step.
Many language tests prepared by teachers intend to examine these linguistic
components separately. These linguistic components are thought to constitute
language skills.
Thus, tests can measure the four traditional language skills and their related
components. Many of these discrete item tests are of the multiple-choice type.
1.7.3 Global Testing
In the later stages of language testing development, a number of tests have begun to
move toward global testing. These types of tests make more comprehensive demands
on the language learner.
Two very popular types of global tests are:
Dictation
Its use has always been controversial
Tests listening, writing, and grammar in context
Cloze Test (the term taken from Gestalt psychology)
Based on a passage with some deleted words
Providing these deleted words requires perceptive and productive skills
Requires sound knowledge of lexical and grammatical systems
The student should rely on contextual clues as well
1.7.3 Testing Communication
Language teaching is becoming increasingly concerned with communication and its
objectives are being reassessed. Language testing, however, has to some extent failed
to develop techniques for measuring effectively the use which is made of language in a
communicative situation.
Many of the procedures used prefer to measure students' ability to manipulate the
grammatical and phonological systems of a language. Nevertheless, tests can be
constructed to enable learners to manipulate language functions, and to identify
utterances as belonging to a certain function of language on account of their
appropriateness. This movement is referred to as "the functional approach."
Core Concepts:
There is more to language than strings of words. Authentic language consists of more
than either words or the grammatical rules for arranging these words into sentences.
One of the misconceptions about language is that successful language usage would
lead to successful language use. This is by no means true, because linguistic aspects of
language are only one part of the communication process. When language is embedded
in a social context, it conveys meanings other than those present in the sentence in
isolation.
Example of Miscommunication:
Imagine a scene where you, with your arms full of books, intend to leave the room. The
door is closed and you need some help. So you address one of your students and tell
him:
"Do you think you could open that door?"
Student (without moving to open the door): "Yes, I do."
Analysis of the Example:
This student has not responded in the way you had hoped he would. This example
demonstrates that the problem does not lie in the misuse of words or in something
ungrammatical in the exchange. In the grammar of individual sentences, a question is a
request for information, and that is exactly how your student has treated it. Yet
questions are in fact one of the most common ways of making a polite request.
Conclusion:
The present literature in TEFL indicates that the teaching of English for communication,
and often communication for specific purposes, differs from more traditional
approaches. Some scholars in the field feel that testing is lagging behind teaching in
this area.
Summary - What We Have Learned Today
Importance of Testing
Education is a giant enterprise
Testing informs decisions
Test vs. Measurement vs. Evaluation
Test = tool
Measurement = number
Evaluation = judgment
Teacher-Made vs. Standardized
Different purposes
Different construction
Different norms
3 Stages of Testing
Traditional: Grammar-translation focus, intuitive stage
Scientific: Structural linguistics and behaviorism, discrete point testing
Communicative: Functional approach, language in social context
Communicative Competence
Grammatical correctness does not equal appropriate use
Questions can function as polite requests
Context determines meaning