Testing
Writing
The Testing Problem
VALIDLY
REPRESE
REPRESE VALIDLY AND
NTATIVE
NTATIVE SAMPLED RELIABLY
SCORED
Testing problem
1. Representative tasks
(i) Specify all
possible
content
(ii) Include a
representative
sample of the
specified content
Cambridge Certificates in Communicative Skills in English
(CCSE)
Certificate in Writing (Level 1)
Specification of ALL the POSSIBLE
content
Expressing, Form,
Operatio Types Boss,
directing, letter, Adressees
ns
describing offax,
text friends,
… family…
note…
1 page, Not
Length Topics
specialist, Dialect
Standard
150
of texts
words… relevant…
Formal,
Style
informal…
REPRESENTATIVE SAMPLE OF THE
SPECIFIED CONTENT
Try to include a representative sample
of tasks
MORE
MORE
VALIDIT
TASKS
Y
EXAMPLES
CCSE (CAMBRIDGE CERTIFICATES
IN COMMUNICATIVE SKILLS IN
ENGLISH)
FIRST STAGE:
READING
TOPIC: Working
in a Summer
Camp for
Children in
America
TASK 1
Write a letter to American
Summer Camps.
Find out about
Dates
Hours of work
Type of accommodation
Ask for the application form
TASK 2
Fill in the
application
form.
TASK 3
You are now working
in the ASCFC. You
write a postcard to
an English-speaking
friend
On your postcard tell
your friend:
Where you are
Why you are there
Two things you like
about the summer
TASK 4
You have arranged to
go out tonight with
two other Helpers at
the ASCFC in Florida.
You have to change
your plans suddenly,
and cannot meet
them. Leave them a
note.
Apologize and
explain why you
cannot meet them
Testing problem
2. Elicit a valid sample of writing
ability
(i) Set as many separate tasks as is
feasible
(ii) Test only writing ability, nothing
else
SET AS MANY
SEPARATE TASKS AS
FEASIBLE
People’s performance
Consistent
Offer them more opportunities
to start
De Pra
sira ctic
ble al
TEST ONLY WRITING,
NOTHING ELSE
NO YES
Imaginat
ion
Intellige
n ce
Creativit
y Writ
in g
Testing problem
2. Elicit a valid sample of writing
ability
(iii) Restrict candidates
Full
sentences
avoided
Testing problem
3. Ensure valid and reliable scoring
(i) Set tasks which can be reliably
scored
Set as Restrict
many tasks candidates
as possible
Sugge
stions
Ensure Give no
long choice of
enough tasks
samples
Testing problem
(ii) Create appropiate scales for
scoring
Holistic • A single scoring
to a piece of
scoring writing
Two approaches
• A method that
Analyti requires a
c separate score
for each aspect
scoring of a task
HOLISTIC SCORING
“Impressionist
ic”
Overall
impression
Purpose of the
test
PRO
Very rapid
[Link]
CON
ANALYTIC SCORING
Separate score – separate aspects
PROS
Uneven development of subskills
Scorers consider details of performance
More reliable
CONS
Time consuming
Too specific (The whole is greater than
the sum of its parts)
http://
[Link]/wma/teachlearn/
Testing problem
(iii) Calibrate the scale to be
used
Collect samples and members of the
testing team cover the full range of
the scales
(iv) Select and train
scorers
Trainee scores should be native speakers,
be sensitive to the languages, have had
experience of teaching writing.
Testing problem
(v) Follow acceptable scoring procedures
Multiple
scoring
should
If the differences ensure
A senior are small the scorer
member of two scores can reliability
Two o more the tem be averaged but
scorers to identify if if they are big
give a score there are senior members
independent discrepancie will decide the
ly s score.
Testing problem
4. Feedback
In some The
situations content
feedback that
is very appears in
useful to the
the feedback
students can be
decided
during
calibratio
n
CONCLUSION
S
CONCLUSIONS
Pay attention to validity
and reliability
Give importance to
writing as an independent
skill
Always be meaningful
and enhance real
WRITING TESTING
TECHNIQUES
1. Multiple Choice
Items
2. Error-recognition
Items
3. Re-arrangement
4. Changing words
CONTROLLED
WRITING
1. Transformation
2. Broken sentences
3. Sentence and
paragraph completion
4. Form-filling
5. Notes and diaries
FREE WRITING
1. Choosing subjects
2. Realistic writing tasks
3. writing for a purpose
4. Writing for an
audience
5. Specific instructions