WRITING ENGLISH LANGUAGE TESTS

Testing and Teaching (relationship)

-to reinforce learning and to motivate students (testing based on teaching : testing what
we teach)

-to assess students' performances (teaching based on testing ; teaching students for
the purpose of get a high score in exam)

Test : a tool or technique, method of measuring students skills, knowledge or

performance in language teaching learning process

Measurement : numerical value

Assessment : the systematic process of gathering information or evidence of students

development and achievement over a period

Evaluation : making overall decisions and judgements about the instruction, course,
curriculum... making selection

Test- measurement-assessment-evaluation (özelden genele sıralama)

A language test provides a focus for purposeful, everyday communication activities. A

good communicative test of language, however, should have a much more positive
effect on learning and teaching and should generally result in improved learning habits.

Communicative task- real life tasks , there is a context, provides for producing and
communicating in target language

Mechanical test- there is one single answer, students deal with the trivial details

External test- select and compare students, large scale public exam

Classroom test- teacher design and implements it according to their own course
objectives, enables teacher to increase her own effectiveness by making adjustments
in her teaching, helps locate the precise areas of difficulty encountered by the class or
by the individual

Aim of classroom test;

-to diagnose the difficulties faced by students (remedial teaching)

-to evaluate effectiveness of the syllabus

-to motivate students

Now well -known public exams measure the candidates' success in performing
purposeful and relevant tasks and their actual ability to communicate in the language.

WHY WE TEST?

-to diagnose what students can do and need to learn

-to inform parents about students' performance

-to inform learners about their expectations and learning objectives

-to provide feedback on learners’ learning process

-to prepare effective teaching planning

What should be tested and do what standard do?

-prescriptive grammarians : try to set standard rules

-descriptive grammarians : examine the whole system of language skills and patterns of
linguistic behavior.

Standards change depending on the teacher, objectives or syllabuses.

Testing the language skills

Skills are integrated in the exams. We don’t separate them we only change our focus.

Aim to test ability

Skills

-speaking : to communicate fluently or oral interaction (interview, role-play, problem

solving, discussion, dialogue)

-writing : to communicate in written mode (e -mail, summary, memories). Letters)

-reading : to comprehend the text details and overall (main idea, details, authentic
materials)

-listening : to comprehend the gist and details (short utterances, talks , lectures)

Language areas

-grammar : to recognize appropriate grammatical forms and structures (form, meaning,

use)
-vocabulary : to recognize and use the meaning of words and its collocations (word
meaning, formation, collocations)

-phonology : to recognize and pronounce the significant sound contrasts (phonemes)

Both the level and the purpose of the test determine the area of language that we
should test. If the students have been learning English for only a brief period it is high
likely that we will be concerned with their ability to handle the language elements
correctly. However at all levels, teachers generally measure the ability to communicate
in the target language. Successful communication in situations that stimulate real life
is the best test of mastery of a language.

Test item : statement

Test question : requires a question? Mark

Recognition

-identify the correct answer

-easy to examine details

-easy to score

-it is not appropriate to test the ability to produce

Production

-produce language to test the grammatical structure

-easy to construct

-scoring takes time

-possibility for more than one answer

We use both of them in testing. Depending on the students' age, proficiency level, and
purpose of the test, the percentage of use changes.

Problems of sampling
 - To include sufficient number of items for each language area and skill covered in
the lesson
- Make use of the table of specifications.
- The test must cover an adequate and representative section of those areas and
skills it desires to test.
- The longer the test, the more reliable a measuring instrument it will be.
- Test has to reflect the actual teaching and the course being followed
- If most of the students don't share the same language, the test must be
universal by nature.

AVOIDING TRAPS

- Unfamiliar words, topics, vocabulary and structures

APPROACHES TO LANGUAGE TESTING

1-The Essay Translation Approach (GTM)

-considered as the pre-scientific stage of language testing

-no special skill or expertise in testing is required

-subjective judgement of the teacher

-tests usually consist of essay writing, translation, and grammatical analysis (comment
on the rules)

-tests also have a heavy literary and cultural bias

2-The Structuralist Approach (audio lingual)

-concerned with the systematic acquisition of a set of habits

-work of structural linguistics, especially the importance of contrastive analysis

-measure the learner's mastery of the separate elements of the target language:
phonology, vocab and grammar

-test one thing at one time

-language points are separately tested (to measure discrete units of language)
-tests include words and sentences given out of context and language forms

-phychometrists have been able to show clearly that such traditional examinations as
essay writing are highly subjective and unreliable

-multiple choice

3-The Integrative Approach

-concerned primarily with meaning and the total communicative effect of discourse

-contrary to the separation of language items

-tests are often designed to assess the learner's ability to use two or more skills
simultaneously (global view of proficiency)

-language competence or grammar of expectancy

-involves functional language but not the use of functional language

-cloze testing and dictation (oral interviews, translation and essay writing)

-supports the idea of the unison of language elements in context

Cloze testing

-based on the Gestalt theory of closure

-measure the reader's ability to decode interrupted messages by making the most
acceptable substitutions

- The text should be long enough to allow a reasonable number of deletions (ideally 40-
50)

-the more blanks contained in the text the more reliable

-two types of scoring; awarded for each acceptable answer, awarded for each exact
answer

-no students should be penalized for misspellings unless a Word is so badly spelt that it
cannot be understood

-grammatical errors should be penalized because linguistic competence is important

-it is essential in cloze testing to draw upon a subject which is neutral in both content
and language variety used

-should provide lead in (no deletions in the first few sentences)

-linguistic knowledge + textual knowledge + knowledge of the World

Types of cloze tests

1-standart cloze test: based on the integrative approach, overall language ability

2-modified cloze test: deletion is decided by the examiner according to the purpose of
the test (discrete point)

3-multiple choice cloze test: alternative items (discrete point)

4-c-test: mechanical deletion every second of the word / half of the word (h..f, s..e)
(discrete point)

Dictation

-auditory discrimination

-auditory memory span

-spelling

-the recognition of sound elements

-overall textual comprehension

-a familiarity with the grammatical and lexical patterning of the language

-no reliable way of assessing

-each error penalized

-a good predictor of global language ability

-longer pieces of discourse rather than short ones

Applying

-read through the whole dictation passage at approaching normal conversational speed

-teacher should begin to dictate in meaningful units of sufficient length to challenge the
students short term memory span

-after the dictation the whole passage is read once more at slightly slower than normal
speed

-students revise and check

The Communicative Approach (CLT, CLL)

-concerned primarily with how language is used in communication

-the meaning of utterances within a context

-success judged in terms of the effectiveness of the communication which takes place
rather than formal linguistics accuracy

-communicative performance= language use + language usage

-language use: how people actually use the language language usage: formal patterns
of language

-language skills both integratively and separately

-use of language in various contexts

-authentic materials and real life tasks

-based on a divisibility hypothesis

-obtain different profiles of a learner's performance in the language

-several measures of proficiency rather than simply and overall measure

Critical Views of Assessment in Communicative Tests

-it is possible for a native speaker to score less than a non-native speaker on a test of
English for Specific Purposes

-sometimes the assessment of language skills in isolation may have only a very limited
relevance to real life

-communicative tests must of necessity reflect the culture of a particular country

because of their emphasis on context and the use of authentic materials

-comm. tests should be based on precise and detailed specifications of the needs of
the learners

-comm. testing has introduced the concept of qualitative modes of assessment in

preference to quantitative ones, each performance is evaluated according to her
degree of success in performing language tasks rather than solely in relation to the
performances of other students

WE SHOULD ADOPT ECLECTIC APPROACH BASED ON THE PURPOSE AND THE LEVEL
TYPES OF TESTS

1-Achievement tests (attainment)

-intend the measure what extent student have already learned

A- Class Progress Test

-the progress test is designed to measure the extent to which the students have
mastered the material taught in the classroom

-it is based on the language programme which the class has been following and is just
as important as an assessment of the teacher’s own work as the students’ own learning

-results obtained from the progress test enable teacher to become more familiar with
the work of each student and with the progress of the class in general

-its backwash effect on teaching and motivation being important features

-a good progress test should encourage students to perform well in the target language
and to gain additional confidence

-its aim is to stimulate learning and reinforce what has been taught

-the progress test is chiefly concerned with allowing the students to show what they
have mastered

High Stakes Testing

-external standardized tests

-gatekeeping exam

-require testing experts

Low Stakes Testing

-teacher made test

-small group of learners

B- Achievement Tests

-far more formal than progress tests and are intended to measure achievement on a
larger scale
-frequently take the form of secondary school entrance tests and school certificate
examinations; many are based on a published syllabus and exert a strong influence on
the teaching in schools

-a good achievement test should reflect the particular approach to learning and
teaching that has previously learned

-several achievement tests are standardized: they are pre-tested, each item is analyzed
and revised where necessary, norms are established and comparisons made between
performances of different students and different schools

B-1 – Proficiency Test

-is concerned simply with measuring students' control of the language in the light of
what she will be expected to do with it in the future performance of a particular task

-concerned with measuring not general attainment but specific skills in the light of the
language demands made letter on the student by a future course of study or job

B-2- Aptitude test

-is designed to measure the students' probable performance in a foreign language

which she has not started to learn

-they use artificial language

-consisting of such factors: intelligence, age, motivation, memory, phonological

sensitivity and grammatical patterning

-sometimes divided into various aspects according to the specific tasks

-generally, seek to predict the students’ probable strengths and weaknesses in learning
a foreign language by measuring performance in an artificial language.

B-3- Diagnostic Tests

-areas of difficulty are diagnosed in different tests so that remedial action can be taken
later

-frequently carried out for groups

Admission Test
-meet the requirements to be accepted to any university

Selection Test

-deciding to accept it or not

Placement Test

-to determine whether the students are ready to take the course

SUBJECTIVE AND OBJECTIVE TESTING

-all tests are subjective in construction stage

-all test items (no matter how they are devised) require subjective judgement of test
makers

-so only scoring of tests can be objective: a testee will score the same mark no matter
which examiner marks the test

-objective tests are the ones that give the same score, subjective tests may reflect the
personal scoring

The Nature of Objective Tests

-difficulty: depends on the content of the course

-skill/area: grammar and usage, vocabulary, phonology, reading, listening and writing
tasks

-advantages: it is easy to score, mostly one correct answer

-disadvantages: test recognition, difficult to construct, real-life representation is limited

-test types: multiple choice, true-false, matching

Criticism on Objective Tests

1-they are simpler to answer than subjective tests

-difficult to design

-select and construct items carefully

-analyze performance on each item

-set pass mark or cut off points

2-objective tests of the multiple-choice type encourage guessing

-four or five alternatives for each item are sufficient to reduce the possibility of guessing

An objective test will be a very poor test if:

-the test items are poorly written

-irrelevant areas and skills are emphasized in the test simply because they are testable

-it is confined to language-based usage and neglects the communicative skills involved

Objective tests never test the ability to communicate in the target language, nor can
they evaluate actual performance.

The Nature of Subjective Tests

-difficulty: depends on the content of the course

-skills/ area: writing and speaking

-advantages: easy to construct, allows testing many things at one time

-disadvantages: difficult to score, difficult to establish inter-rates reliability (many

degrees of acceptability and ways of scoring)

-test types: essay writing

A GOOD CLASSROOM TEST WILL USUALLY CONTAIN BOTH OF THEM

MULTIPLE CHOICE ITEMS: GENERAL

-helps both students and teachers to identify areas of difficulty

-helps to assess large population

-do not tend to test language as a means of communication

-one of the most widely used types of items in objective tests; however, the usefulness
of this item is limited

- multiple-choice item is one of the most difficult and time-consuming types of items to
construct
-the test must be long enough to allow for a reliable assessment of a testee's
performance and short enough to be practicable

-the number of items will vary according to; levels of difficulty, purpose of the test,
nature of the skill/area

-present the items within both linguistic and situational context

-isolated sentences in a multiple -choice test simply add to the artificiality and give rise
to the ambiguity

Components

1-the initial part of the item is known as the stem

2-the choices from which the students select their answers are referred to as
options/responses/alternatives

3-one option is the answer, correct option or key while the other options are distractors

Principles when mp items are constructed

1-each multiple choice item should have only one answer

2-only one feature at a time should be tested

-sometimes word order and sequence of tenses are tested simultaneously, such items
called impure items

3-each option should be grammatically correct when placed in the stem, except of
course in the case of specific grammar test items

4- All multiple-choice items should be at a level appropriate to the proficiency level of

the testees

5-multiple choice items should be as brief and as clear as possible

6-In many tests, items are arranged in rough order of increasing difficulty
Multiple Choice Items : The Stem/Correct Option/Distractors

A-The Stem

-the primary purpose of the stem is to present the problem clearly and concisely

-the stem should not contain extraneous information or irrelevant clues, thereby
confusing the problem being tested

-the stem may take following forms

1-an incomplete statement

2-a complete statement

3-a question

-the stem should usually contain those words or phrases which would otherwise have
to be repeated in each option

-the stem should allow for the number of choices which have been decided upon

B-The Correct Option

-it is very important to avoid confusing the students by having a different number of
correct options for each item.
-the correct option should be approximately the same length as the distractors.

C- The Distractors

-each distractor or incorrect option, should be reasonably attractive and plausible

- Items should be constructed in such a way that students obtain the correct option by
direct selection rather than by the elimination of obviously incorrect options

-each distractor should be grammatically correct when it stands by itself

-distractors should not contain absurd items that can be easily omitted

-distractors should not be too difficult

Plausible Distractors are Best Based on

-mistakes in the students’ own written work

-their answers in previous tests

-the teacher’s experience

-a contrastive analysis between the native and target languages

Other Characteristics

-the stem should be meaningful without having to read all the options first

-state the stem in positive rather than negative, the use of negatives can confuse test-
takers, highlight the negative and avoid the double negative

-avoid using the alternative ‘’all the above’’ or ‘’none of the above’’ because they make
it possible to answer the test item correctly on the basis of partial information; if
needed, make such options not the correct one

-avoid giving clues to the answer to other test items in the alternatives provided or in the
stem

- test items need to be dependent to one another

- test items need to stand alone

-avoid trick questions

Writing The Test

The testees may be required to perform any of the following tasks

1-write out the correct option in full in the blank

2-write only the letter of the correct option in the blank or in a box

3-put a tick or a cross at the side of the correct option or in a separate box

4-underline the correct option

5-put a circle round the letter at the side of the correct option

6-bubble answer sheet, provide a separate answer sheet

!be consistent about the ways of answering throughout the test

!award equal marks for each question

!mix up the order of the answers

CRITERIA

1-VALIDITY (ACCURACY)

-The validity of a test is the extent to which it measures what it is supposed to measure
and nothing else.

-It must be clear, and students must use their background knowledge.

How to ensure?

a)Face Validity

-Should look as if it valid on the face of it.

- Unlike other more technical forms of validity, face validity is about appearances and
the perspective of the people taking or viewing the test.

-A test which looks right and appears to measure the knowledge or abilities it claims to
measure.

-Clear items, clear directions, well organized tasks, related task to coursebook,
appropriate difficulty level, doable within time.

-Can’t be empirically tested but a factor on the eye of the beholder.

-The test appears to have little of relevance in the eyes of the student, it will clearly lack
face validity.

-Language tests which have been designed primarily for one country and are adopted
by another country may lack face validity.

b)Content Validity

- Content validity is about how well a test covers the entire range of the subject matter
it is intended to measure.

-Depends on a careful analysis of the language being tested and of the particular
course objectives.

-The test should be so constructed as to contain a representative sample of the course.

-To achieve high content validity, teachers often use a Table of Specifications. This is a
grid that maps out which objectives are being tested and how many questions are
assigned to each.

Comparison: Face vs. Content Validity

Face Validity: "Does this look like a good test to the students?" (Subjective/Surface).

Content Validity: "Does this test actually cover everything I taught?"

(Objective/Structural).

c)Construct Validity

-It is capable of measuring certain specific characteristics in accordance with a theory

of language behaviour and learning.

- Construct validity is often considered the "mother of all validities." While face validity
is about appearances and content validity is about coverage, construct validity asks, "Is
the test actually measuring the invisible, underlying trait or skill it claims to measure?"

- In language teaching, a "construct" is an abstract idea like communicative

competence, reading fluency, or listening comprehension. You cannot touch these
things; you can only observe them through a test.
d)Empirical Validity

-Statistic, criterion related validity

-Extent to which the criterion of the test has been reached or not

-Obtained as the result of comparing the result of the test with the result of some
criterion measure such as;

-an existing test, known or believed to be valid and given at the same time

-the teacher’s ratings or any other such form of independent assessment given at the
same time

-the subsequent performance of the testees on a certain task measured by some valid
test

-the teacher’s ratings or any such form of independent assessment given later

1-Concurrent Validity

- This measures how well a new test compares to an already established, highly
respected test taken at the same time.

2-Predictive Validity

- This measures how well a test predicts a person’s performance in a future situation.

Validity
Evidence Source Best For...
Type

Face Student/Public Perception Maintaining motivation and "buy-in."

Ensuring you tested what you actually

Content The Syllabus/Textbook
taught.

Making sure you're measuring the right

Construct Theoretical Definition
"skill."
 External Scores/Future Proving your test works as well as
Empirical
Success "official" ones.

FACTORS THAT AFFECT VALIDITY

-inadequate sample of items

-items that don’t function as intended (not representative)

-improper arrangement

-unclear directions

-too few items for interpretation (length of a test)

-improper test administration

-scoring that is subjective

-a systematic pattern of correct answer

-difficult structures and words (incomprehensible and ambiguity)

2-RELIABILITY(CONSISTENCY)

-Good test should give consistent results depending on certain factors such as time,
scoring, etc.

FACTORS AFFECTING THE RELIABILITY OF A TEST

-the extent of the sample of material selected for testing (the larger the sample the
greater the probability that the test as a whole is reliable)

-the administration of the test (is the test administered to different groups under
different conditions or at different times)
-test instructions

-personal factors such as motivation and illness

-scoring the test (objective tests overcome this with the marker reliability)

Profile reporting: In order to obtain a full profile of student’s ability in the target
language, it is necessary to assess his or her performance separately for each of the
different areas of communication: listening comprehension, speaking and listening,
reading and writing. Profile reporting is very valuable for placement purposes, and
indeed it is an essential feature of one of the most widely used proficiency tests set.

Measuring the reliability of a test

1-Test-Retest Reliability (Stability)

This is the simplest method. You give the same test to the same group of students
twice, with a short time gap (e.g., two weeks) in between.
The Logic: If the students' scores are nearly identical both times, the test is stable.
The Problem: The "Practice Effect." Students might remember the questions from the
first time, which artificially inflates their second score.

2. Parallel-Forms Reliability (Equivalence)

You create two different versions of the same test (Form A and Form B) that cover the
same content but use different questions.
The Logic: You give both versions to the students. If their scores correlate, the tests are
equivalent.
The Problem: It is very difficult and time-consuming for a teacher to write two "identical"
tests.

3. Internal Consistency (Split-Half Reliability)

Instead of giving two tests, you take one test and split it in half (usually even-numbered
questions vs. odd-numbered questions).
The Logic: You compare the scores of the two halves. If a student gets the odds right,
they should also get the evens right.

4. Inter-Rater and Intra-Rater Reliability (Scorer Consistency)

This is crucial for subjective tests like speaking or writing (e.g., your students'
roleplays).
Inter-Rater: Two different teachers grade the same student. Do they give the same
grade? (High inter-rater reliability means your rubric is clear).
Intra-Rater: The same teacher grades a paper today, and then grades it again in a week
without looking at the first score. Does the teacher stay consistent?

3-PRACTICALITY

- Practicality focuses on the logistics, time, and money required to create and
administer it.

-A test can be perfectly valid and highly reliable, but if it is too expensive or takes 5
hours to grade, it has low practicality and cannot be used in a real school.

-time, cost, administration, scoring, availability of resources

-Effective test should be ;

-user friendly, practical to apply, easy to score, easy to interpret and grade

Other Related Principles

-bias

-ethics

fairness

-transparency

- interactiveness

-authenticity

-security

4- DISCRIMINATION
-Capacity to discriminate among the different candidates and to reflect the differences
in the performances of the individuals in the group.

5-ADMINISTRATION

-A test must be fairly straight to administer.

Feature Administration Discrimination

Focus The process of testing. The quality of the questions.

Goal Fairness and consistency. Accuracy in identifying student levels.

Making sure all 30 students Making sure the "Mystery Worker" clues
Example have a desk and a sharp aren't so easy that everyone guesses it in 1
pencil. second.

The Big Picture (The "Big Five" of Testing)

You have now covered the primary pillars of language testing. In your reports, you can
evaluate any activity (like your Weather Tour) using these:

1. Validity: Does it measure what it should?

2. Reliability: Is it consistent?
3. Practicality: Is it easy to do with 30 kids?
4. Administration: Was it handled fairly?
5. Discrimination: Does it show me who knows the material?

6- BACKWASH EFFECT

-Backwash effect (also known as washback) refers to the influence that a test or
examination has on the teaching and learning process.
-In simple terms, it is the "ripple effect" where the format and content of an assessment
dictate how a teacher teaches and how students study.