====~~===~~~~=====~ ITEMS • Instructional Topics in Educational Measurement ======~~==============

NCME Instructional Module on

Design and 'Development of

Performance Assessments
Richard J. Stiggins
Northwest Regional Educational Laboratory

Achievement can be, and often is, measured by means of observation by the examinee. In fact, however, recent studies of school
and professional judgment. This form of measurement is called per- assessment suggest that teachers rely at least as much on
formance assessment. Developers of large-scale assessments of com- observation and judgment in evaluating student achievement
munication skills often rely on performance assessments in which as they do on paper-and-pencil assessment strategies.
carefully devised exercises elicit performance that is observed and
Teachers observe and evaluate such behaviors as oral reading
judged by trained raters. Teachers also rely heavily on day-to-day
observation and judgment. Like other tests, quality performance
fluency or speaking skills and such products as writing
assessment must be carefully planned and developed to con/orm to samples or art and craft products as they track student
specific rules of test design. This module presents and illustrates those development. At some grade levels (especially primary
rules in the form of a step-by-step strategy for designing such assess- grades) and in some subject-matter areas (e.g., language arts),
ments, through the specification of(a) reason(s) for assessment, (b) type teachers rely very heavily on measurement that is based on
of performance to be evaluated, (c) exercises that will elicit perfor- observation and professional judgment. In this module,
mance, and (d) systematic rating procedures. General guidelines are measurement based on observation and judgment is labeled
presented for maximizing the reliability, validity, and economy of performance assessment.
performance assessments.
Large-scale standardized tests often rely on selection-type
An Overview of Performance Assessment paper-and-pencil test items because such test items sample
content effectively and can be scored quickly. Efficient assess-
When we think of measuring achievement, the natural
ment is crucial when there are many thousands of tests to
tendency is to think of published standardized, multiple-
be scored. However, even in this context we see increasing
choice, true/false, or fill-in tests, all of which rely on written
use of performance assessment, particularly in instances
test items that are read and answered correctly or incorrectly
where observation and judgment represent the most valid
way to assess. For example, nearly three quarters of the
states are conducting statewide writing assessments based
Richard J. Stiggins is Director of the Center for Performance
Assessment, Northwest Regional Educational Laboratory, 101 SW
on a teacher's subjective evaluation of student writing sam-
Main St., Suite 500, Portland, OR 97204. His specializations are ples. Trained raters evaluate overall quality, organization,
performance assessment methodology and classroom assessment. style, content, and other key factors by applying clearly artic-
ulated performance standards in the process of evaluation.
Funds for development of this unit were provided in part by When used carefully, performance assessment can produce
the Office of Educational Research and Improvement (OERI), dependable results.
U.S. Department of Education. The opinions expressed in this Whether that use is in the classroom on a daily basis, in
module do not necessarily reflect the position of OERI, and no
an annual statewide testing program, or in evaluation for pro-
official endorsement by OERI should be inferred. The module
represents an adaptation of a guide entitled' 'Evaluating Students
fessionallicensing and certification, performance assessments
by Classroom Observation" published in 1982 by the National rely on the judgmental rating of achievement. Such rating
Education Association Professional Library, Washington, DC. processes are subject to a variety of measurement errors and
Published by permission ofthe National Education Association. must be conducted very carefully. The keys to success-to
obtaining valid and reliable results-are (a) to make the
Series Information judgment-based evaluation process as systematic and objec-
ITEMS is a series of units designed to facilitate instruction in tive as it can be while (b) focusing on the most important at-
educational measurement. These units are published by the Na- tributes of performance. Systematic assessments have a clear
tional Council on Measurement in Education. This module may purpose, are based on explicit criteria, rely on appropriate
be photocopied without permission if reproduced in its entirety exercises, and include precise performance rating procedures.
and used for instructional purposes. Barbara S. Plake, Univer-
Our goal is to be sure that performance ratings reflect the
sity of Nebraska-Lincoln, has served as the editor for this module.
examinee's true capabilities and are not a function of the
perceptions and biases of the person evaluating performance.
Fall 1987 33
 ==~~~~~~~~~~ ITEMS • Instructional Topics in Educational Measurement ~~~~~~~~~~~;:;

This module is written to help the reader reach this goal most formal of these assessments are structured, preplanned
through the use of practical, economically feasible, efficient events designed in advance to provide a decisionmaker with
performance assessments. a specific piece of performance information. For instance,
a remedial reading teacher might listen to each student read
The Components of a Performance Test orally early in the year to help make placement decisions.
Performance assessments call upon the examinee to dem- Informal assessments, on the other hand, are those casual,
onstrate specific skills and competencies, that is, to apply the spontaneous insights that often come to the teacher as a
skills and knowledge they have mastered. The demonstra- result of an incident noticed during an instructional activ-
tion can take place during the normal course of everyday ity. For example, a third grade teacher might notice a par-
'; events (e.g., during normal classroom life) or in response to ticular student stumbling repeatedly over a particular begin-
specific structured exercises provided by the examiners. ning sound in a reading group. The teacher might make a
Regardless, the examinee's task is to construct an original mental note for later action. The rules of good evidence and
response, which the examiner observes and evaluates. sound assessment need to be applied to both formal and in-
Performance assessments are valuable tools for measuring formal performance assessment, if each is to produce useful
communication skills such as reading, writing, speaking, and information. However, the evaluator often has little control
listening. They serve well in business and industrial educa- over the circumstances surrounding informal events. For this
tion, science lab, visual and performing arts, and physical reason, our purpose is to concentrate on performance assess-
education. They represent valuable tools in special education ment in the formal case through the remainder of this module.
contexts where examinees lack paper-and-pencil test-taking Extrapolations from the formal to the informal case will be
skills and/or where paper-and-pencil tests fail to test impor- highlighted as we go.
tant skills. They also play key roles in assessment of profes-
sional skills. Structure of This Module
All performance assessments are composed of four basic The presentation that follows is designed to take you
components: a reason for assessment, a particular perfor- through the step-by-step process of designing a blueprint for
mance to be evaluated, exercises to elicit that performance, a performance assessmeht. The process is broken down into
and systematic rating procedures. The reason for assessment four steps, one for each of the major components specified
is defined in terms of decisions to be made, decisionmakers, earlier. You will specify a reason for assessment, describe
and examinees to be evaluated. Performance to be evaluated the performance to be evaluated, plan exercises, and outline
is defined in terms of content and/or skills, type of behavior rating procedures. In all, you will make 13 different design
or product to be observed, and performance criteria. Exer- decisions. In each case, you will learn the design alternatives
cises are structured or natural, preannounced or unannounced, available to you and the factors to consider in making your
and variable in number. Finally, rating procedures are defined choices among alternatives. When you have completed all
in terms of the type of data needed, identity of the rater, and choices, you will have a very detailed picture of a useful
nature of the recording method. Those who design and develop assessment tool.
performance assessments must consider all of these factors. This test design simulation is intended to inform you about
The goal of this training module is to spell out these per- each of the active ingredients in a sound performance assess-
formance assessment design alternatives and teach the user ment. We do not advocate the systematic completion of ea"l:l
how to select from among available options to develop per- step every time a new professional judgment is made. How-
formance tests that fit particular purposes. ever, the more important the purpose for the assessment the
more crucial it is that we obtain valid and reliable data from
Performance Assessment in the Larger Scheme the assessment and, therefore, the more crucial it is that all
Educators and assessment specialists use a wide variety active ingredients be carefully considered.
of tools to evaluate achievement. They use formal and infor-
mal performance assessments, objective tests and quizzes, Assessment Design
essay tests, and oral questions in class, to mention a few . How As we begin the process of performance assessment design,
do performance assessments compare with the other alter- please reflect on the educational environment around you and
natives in terms of active ingredients and keys to successful identify a particular situation in which a performance assess-
use? Consider the differences spelled out in Table 1. Each ment might be (or has been) useful to you. For teachers, this
form of assessment is different. Each is a valuable tool in might be a reading, writing, science, or arts context. For
the hands of a skilled user. Each can contribute to an effec- administrators, it might be a program evaluation or a per-
tive assessment. sonnel evaluation situation. For those involved in professional
assessment, it might be a work sample-based performance
Purpose of This Module review. Select a situation requiring a systematic, evaluation-
Many measurement textbooks deal effectively with objec- based professional judgment. If possible, make it a real, cur-
tive tests (whether standardized or teacher-developed) and rent need. This will make the simulation most meaningful
essay tests. Since oral questions are often like short-answer and useful to you.
test items in form and evaluation, some measurement texts When you have identified such a situation, follow the Per-
may help with these also. However, fewer treat performance formance Assessment Blueprint, which appears on page 40,
assessment in a systematic manner. This module is designed as you proceed through this instructional guide, beginning
to help fill that void. the design process with Step 1, "Clarify reason(s) for assess-
Performance assessments can vary in their formality. The ment." To illustrate the test design process, we will accom-
34 Educational Measurement: Issues and Practice
~~~=~~~~~:::;;:::;;== IT EMS • Ins tru ct ion aI Top i cs in Ed u ca ti on a I Meas u rem ent ;::=:::;;:::;;======:::;;:::;;===~========

pany each step with the detailed development of an example results. Some of the alternatives are listed below with brief
of a performance assessment of writing skills. descriptions. Choose from among these or specify another
that fits your context and enter it in the space provided on
Step 1: Clar~fy Reason(s) for Assessment the-Blueprint for Item lA. You may identify more than one
No assessment should ever be conducted unless and until decision if appropriate.
the evaluator knows exactly how the results are to be used. • Individual diagnosis-identifying the strengths and weak-
As you will see, the manner in which results are to be used nesses in the performance of individuals.
influences many of the subsequent design decisions. • Group needs assessment-diagnosing the strengths and
Therefore, your first step is to state why you are designing weaknesses of a group of examinees, such as a particular
this assessment. class.
A. Specify decision(s) to be made on the basis of assessment • Grading-assigning a grade (A, B, C, etc.) to individual

TABLE 1
Comparison of Various Types of Assessment

Performan ce
Objectiv e test Essay test Oral question assessment
Purpose Sample knowl edge with Assess thinking skills Ass ess knowledge Assess ability to
maximum eff ic iency and/or mastery of a during instruction transla te knowledge
and reliability stru cture of and ' u nderstanding
knowledge into action
Typi ca l exe rcise , Test items: Writing task Open-ended question Written prompt or
Multiple-choice natural event framing
True/false the kind of
Fill-in performan ce required
Matching
Student's response Read , evaluate, select Organize, compose Oral answer Plan, construct, and
deliver original
respon se
Scoring Count correct answers Judg e understanding Determin e correctness Check attributes
of answ er present, rate
profici ency
demon strated , or
describe p erformance
via an ecdot e
Major ad vantage Eff iciency-can Can measure complex Joins asse ssment and Provides ri c h evidence
administer many cog nitive outcom es instru ction of perform ance skills
items per unit of
testing time
Poten tia l so urces Poorly written item s. Poorly written exercises, Poor questions, Poor exe rc ises, too few
of ina cc urate overemphasis on wri t ing skill students' lack of sample s of
assessment recall of fa ct s, poor co nfounded with willingn es s to perform ance, vague
test-tak i ng sk ills, knowledge of content, respond, too few criteria, poor rating
failure to sample poor scoring que stions procedures, poor test
co ntent procedures conditions
representatively
Influ ence o n Overemphasis on Encourages thinking and Stimulates participation Emphasi zes us e of
l ear ning recall encourages d evelopment of in instruction, available sk ill and
memorization; can writing skills provid es teacher knowledge in relevant
encourage thinking imm ed iate feedback probl em co ntexts
skills if properly on eff ecti ve ness of
constructed teaching
Keys to success Clear test blu ep rint or Carefully prepared Clear questions, Carefully prepared
specifications that writing exercis es, represe ntative sample performan ce
match instru ction, preparation of model of questions to each exercises; c lea r
skil! in item writing, answe rs, time to read stud ent, adequate perform ance
time to write items and score tim e provided for expectati o ns; careful,
student response thoughtfu I rating;
time to rate
perform ance

Fall 1987 35
~~~~~~~~~~~~ ITEMS • Instructional Topics in Educational Measurement =~~~~~~~~~~~

students as a reflection of the amount of material learned. of examinees and time required for assessment. Fewer exami-
• Grouping-assigning examinees to specific instructional nees allow more time per assessment; more permit less time
groupings within a class. per assessment. Our goal is to design assessments that ar.e
• Selection-identifying those who are far ahead of or behind as economical as they can be and to make them realistic.
the rest for placement into special advanced or remedial
programs. Example: We will assess the skills of 100 11th grade lan-
• Certification-verifying the mastery of specified skills by guage arts students.
individuals.
• Evaluation-determining if a particular program of instruc- Describe your performers under Blueprint entry 1D.
tion has worked effectively or needs revision.
Reflect on your context and decide which decision(s) best Step 2: Clarify Performance to Be Evaluated
describe(s) your intended use of the results. IT you have a pur-
An obvious key to quality assessment is the clear defini-
pose in mind that does not appear above, write it in the space
tion of the performance to be evaluated. We need to specify
provided.
the general content area, the type of performance to be
Example: We will conduct a writing assessment to diagnose observed, and the' specific dimensions of performance to be
strengths and weaknesses in individual students' writing skills. considered in evaluation.
A. Specify the content or skill focus of the assessment, in
B. Specify decisionmaker(s) by identifying the person(s) who a very general way. What subject matter area and class of
will use the results to make the decision(s) listed under 1A. proficiencies are to be demonstrated?
This may be a teacher, administrator, student, parent, board
member, employer, certification board, or some combination Example: We will assess the English composition skills of
of these. It mayor may not be the person who actually rates high school students.
performance. It is the person who acts upon performance
Place your answer in Item 2A of your Blueprint.
ratings to make decisjons that influence how examinees and
B. Select the type of performance to be evaluated. You can
programs interact.
observe and evaluate a process of behavior as it occurs, as
Example: The purpose for our writing assessment is to help when the teacher listens to and evaluates oral reading flu-
the teacher and student identify weaknesses in the student's ency. Or you can observe and evaluate a product developed
writing skills. by the performer, as when he or she creates a woodworking
project. You also can use both, as when a typing teacher
In the space provided for 1B, specify the decisionmaker(s) evaluates both typing technique and the finished product.
who will use your assessment results. The· key question is: Where can you find the best evidence
C. Specify use to be made of results. You have two choices. of proficiency? Is the process important, as in speaking assess-
All educational decisions require that we either ment? Must steps be carried out in a specified order, as in
carrying out a science experiment? Or does a tangible product
• rank examinees-place in order of achievement from result which is to have certain characteristics?
lowest to highest; or
• determine mastery-verify that each performer has mas- Example: Our writing assessment will be based on a
tered material regardless of how others perform. product-actual samples of student writing.
You may also use these in combination as in ranking students
and exploring the skills not mastered by those who rank low. On what will you base your judgment? Check the appropriate
The key factor to consider in making your selection is the indicator on Blueprint Item 2B.
nature of the decision to be made. Some decisions, such as C. List performance criteria. No other single specifica-
grouping and selection, require rank order data, whereas tion will contribute more to the quality of your perfor-
others, like diagnosing strengths and weaknesses, require in- mance assessment than this one. Before the assessment is
formation about specific skills mastered and not mastered. . conducted, you must state the performance criteria, in other
words, the dimensions of examinee performance (observable
Example: In our writing assessment, we will determine behaviors or attributes of products) you will consider in
mastery of specific writing skills. rating.
Each dimension of performance is to be specified in two
Which do you need? Make your choice in space 1C of the parts: a definition and a performance continuum. Perfor-
Blueprint. mance criteria should reflect those important skills that are
D. Describe students to be assessed-Describe the examinees the focus of instruction. Definitions spell out what we, as the
whose performance is to be evaluated by specifying how many evaluators, mean by each particular criterion. The continuum
will participate, at what grade level, and any salient charac- specifies the range of possible ratings from high to low for
teristics of those examinees. Salient characteristics include each criterion-in observable terms (see the example below).
those factors that might need special attention in assessment The key to identifying sound performance criteria is to
design. Are the students advanced or remedial? Participants place yourself in the hypothetical situation of having to give
in a special new program? If they are unique in some special feedback to someone who has performed poorly on the task.
way, note that fact here. This information will be useful later What factors are you likely to mention? The answer to this
as we try to design assessments that are feasible in terms question requires and deserves a great deal of thought on
36 Educational Measurement: Issues and Practice
;:;;::::==~~~~==========~ ITEMS • Instructional Topics in Educational Measurement ~====~==""'"'==========~

the part of the evaluator. If you do not have a clear sense certification. In these cases, it is crucial that every examinee
of the key dimensions of sound performance-a vision of have a fair and equal opportunity to demonstrate his or her
poor and outstanding performance-you can neither teach skill. Under such circumstances, a carefully planned, stan·
students to perform nor evaluate their performance. dardized, and tightly controlled assessment is essential.

Example: Key dimensions of writing performanc~ to be evaluated Example: Sample writing exercise: Assume you have a
in our writing assessment friend who is moving to another city to find a job. You know
Factor Meaning Continuum someone there who may be able to hire your friend. Write
Organization Arrangements fol- Beginning, middle, and end a letter of introduction for your friend to take to the new
low in an effective are clear and effective, to city. Describe your friend in a way that will make the reader
sequence very haphazard arrange-
feel he or she would like to meet and interview this person.
ment, hard to follow
Voice Extent to which the Writer [Link] to
writer speaks to the reader and seems sin- Make your choice under Blueprint i3A now, providing a sam-
the reader cere, to writing is flat, ple exercise, if you select that option, and/or describing the
lifeless, with no sense of relevant natural events to be observed. In your sample exer-
the individual writer cise, provide a complete set of instructions to the examinees
Writing Grammar, usage, Good grasp, no glaring er- letting them know what they are to do and under what condi-
mechanics punctuation, etc. rors, not a factor in read- tions. If you choose naturally occurring events, describe what
ing paper, to too many the performer will be doing and under what circumstances.
errors make paper hard B. Determine the obtrusiveness of assessment by specify-
to read and understand ing if it will be preannounced, open, and public assessment
or if you plan to conduct your assessment without advising
In the table provided in the Blueprint under 2C, space is pro- performers that an assessment is taking place. Again, you
vided for up to three performance criteria. In fact, you may
may wish to use a combination, gathering .evidence of profi-
specify more or fewer, depending on your assessment. But
ciency under both obtrusive and unobtrusive circumstances.
be sure to specify all relevant criteria and expand your initial
The key difference between the two is seen in examinee
sketch as needed to ehsure clear understanding of perfor-
motivation and level of test anxiety. Under standard testing
mance expectations by both raters and performers (see the
circumstances, performers will operate at levels of maximum
Appendix for more detail on criteria listed above).
motivation-demonstrating their best possible work. How-
ever, in a few situations (such as with safety rules in a science
Step 3: Design Exercises lab) it may be more helpful to know how they perform under
Performance assessment exercises present those instruc- conditions of typical everyday motivation. Unobtrusive-but
tions or describe those events that give rise to examinee per- system!1tic-observations can provide that information.
formance. Once the performance to be evaluated is defined, Under normal testing conditions, examinees often experi-
we must decide how to elicit or sample performance so as ence varying degrees of test anxiety. In moderation, this
to observe and evaluate it. We need to specify the form, obtru- anxiety can create a keen performance edge that maximizes
siveness, and number of exercises to be used. performance. On rare occasions, however, some people expe-
A. Select form of exercises. Specify if you plan to (a) observe rience debilitating test anxiety. At test time, capable but
everyday events as they occur, such as by watching exam- extremely anxious performers fail to demonstrate skills you
inees during everyday discussion to evaluate speaking skills, know they have. When this occurs, unobtrusive observations
or (b) design specific exercises to cause them to speak on a can disarm the test situation, eliminate the anxiety, and pro-
standard topic, for example, under standard, controlled cir- vide needed evidence of proficiency.
cumstances so you can evaluate performance. Please note
that you can use both in gathering evidence of proficiency. Example: Our writing assessment will be announced in
Consider two factors in making this selection: the natural advance.
availability of dependable evidence and the seriousness of the
decision to be made. Evidence available through natural Specify your choice under Blueprint 3B now.
observation can be relatively inexpensive, as these events will C. Determine the amount of evidence you plan to gather.
unfold whether or not you use them as a source of perfor- You have three choices: one sample of performance gathered
mance ratings. So, in effect, the design of the exercise has at one time, multiple samples at one time, or multiple samples
no cost. When natural circumstances provide sufficient collected over several occasions.
samples of the kind of behavior you wish to observe, those In evaluating these design options, consider three factors:
circumstances provide the basis for a very efficient, low-cost • The importance of the decision-the more important the
performance assessment. Just remember, however, that fre- decision, the surer you have to be and therefore the more
quency of occurrence of desired behavior is a factor in select- evidence you need to gather.
ing this option. If the desired performance is likely to be infre- • Representativeness of samples-any assessment reflects
quent, you can waste a great deal of time in unproductive a sample of all of the possible examples of performance
observation. In this case, structured exercises may be needed that could have been gathered. Gather enough samples of
to elicit evidence of proficiency. performance to be sure you have sampled the range of
Structured exercises also may be needed when very impor- possible applications of the skills to be evaluated.
tant decisions hang in the balance, such as grade promotions, • Time-consider the amount of time you have available to
high school graduation, college scholarships, or professional rate each sample, the number of performers to be evalu-
Fall 1987 37
 ~~~==:=;============= ITEMS • Instructional Topics in Educational Measurement :=;========~===========
ated, and the time until the decision must be made. This bination, as when you score all performance samples holistic-
will indicate how many samples you can realistically gather ally and then analyze the specific weaknesses of those scoring
and evaluate. low in an overall sense.
There are no simple rules as to how many samples are B. Determine who is to rate performance. In this case, as
enough. The number varies greatly from context to context. in others, you have several choices. You can rely on the expert
Rely on your own judgment. How many will you need to be professional judgment of a teacher or other qualified special-
relatively sure you can accurately judge an examinee's true ist. Or you can have the performers evaluate their own and/or
capabilities? each other's performance. Which you choose depends on your
assessment context.
Example: Students in our writing assessment will provide Use teacher or other professional judgment if
,!
three 30-minute samples of writing. They will produce one • rating requires the specialized, technical knowledge and
per day for 3 days. experience that only a professional possesses;
• important competitive decisions require standard, uniform
Specify your sampling plan under Blueprint 3C. test conditions and unbiased ratings for all examinees;
and/or
Step 4: Design Performance Rating Plan • examinees have a vested interest in results and may be
To complete the performance assessment blueprint, you perceived as having a chance to benefit unfairly from self-
must plan how performance will be "scored" and how results evaluation (e.g., a grade hangs in the balance).
can be put in a communicable form. The performance rating
Consider examinee peer or self-rating if
plan must be designed to give the decisionmaker(s) the assess-
ment results they need in a timely fashion and in a form they • examinees are capable of learning and applying the per-
can use . Therefore, the key factor to consider in designing formance criteria;
this component of the assessment is the nature of the deci- • slight variation from rater to rater is acceptable in your
sion to be made. To complete the performance rating plan, context;
answer these questions: What type of score is needed? Who • examinees have nothing to gain from artificially inflating
is to rate performance? How are the data to be recorded? or deflating their [Link]; and/or
A. Determine the type of score needed by reflecting on the • the evaluator's scoring task exceeds time available to com-
information needs of the decisionmaker. Some decisions, such plete it-examinees sometimes represent a low-cost, effi-
as grouping and selection, can be based on a ranking of exam- cient scoring resource.
inees (see Blueprint IC) and therefore require only a general U sing more than one rater is often an excellent idea. By
overall index of performance-a holistic rating, if you will. using two evaluators, we can gather evidence on the extent
Other decisions, such as diagnosing individual or group needs to which independent ratings of the same examinee's per-
and certifying minimum competencies, require a more de- formance agree. Such agreement argues that criteria have
tailed breakdown of dimensions present or absent in student been dependably applied and rater bias has been controlled.
performance. These require analytic scoring. On the other hand, if independent raters disagree, we may
Please note that both holistic and analytic scores are based need to reconsider criteria and/or train raters more
on the performance criteria spelled out under Blueprint 2C. thoroughly.
But the criteria are used differently in each case. In analytic Remember that all raters must be trained to rate examinee
scoring, each criterion is evaluated individually and is as- performance-whether raters are experts or students. Just
signed a score. In holistic scoring, all criteria are considered because experts are selected from the same field, there is
simultaneously when assigning an overall score. Some criteria no guarantee that they view key dimensions of achievement
may be given more weight than others in this process, alike. They must be trained to agree by practicing with the
depending on the contribution of each to the quality of per- criteria on samples of performance.
formance in the evaluator's opinion. Remember also that training students to be raters repre-
The advantage of holistic scoring is that it is less time- sents an excellent instructional (and assessment) strategy.
consuming and therefore less expensive than analytic scoring. Once they internalize performance criteria and see how
It takes less time to make one overall judgment than it does those criteria come into play in their own and each other's
to make several individual judgments. When holistic will suf- performance, students often become better performers.
fice in terms of information needed, it is definitely the score
of choice. Analytic, on the other hand, offers more focused E xample: In our writing assessment, students and the
information. It offers a profile of performance characteristics. teacher will rate performance. Each student will rate his or
The price we pay, of course, is the greater time required to her own paper plus the writing samples of one other student.
make several evaluations-one for each criterion. But when The teacher will independently rate all papers.
we need the diagnostic information, analytic scoring is worth
the price. Specify the raters you will use in your assessment under
Blueprint entry 4B now .
Example: In our writing assessment, we will score all C. Clarify score recording method to be used. This is the
papers analytically, because we wish to determine the specific final design decision and represents the place where we
weaknesses of the students. merge the performance criteria into the scoring process.
Recording alternatives include:
Specify the type of score you need for your assessment in • a checklist of specific attributes present or absent in the
Blueprint Item 4A now. Remember, you can use them in com- performance, such as characteristics of an art object;

38 Educational Measurement: Issues and Practice

=======================: ITEMS • Instructional Topics in Educational Measurement =======================:;~

• a rating scale in which dimensions of performance are back to performers. See examples of a rating scale for writing
evaluated along a continuum from adequate to inadequate, assessment in the Appendix.
such as rating key aspects of a speech;
• an anecdotal record in which important behaviors are Ensuring High Quality
described in a verbal form that communicates evaluative Professional observations and carefully considered judg-
judgments, as in anecdotal records of social/emotional prob- ments represent a viable means of measuring achievement.
lems; and, They can and do serve us very well in the day-to-day manage-
• a portfolio of examples of examinee performance selected ment of instruction. They also playa key role in large-scale
to illustrate the level of skill and/or development over time. assessment programs-especially in writing assessment. If
Please note that these can be used in combination to create they are to fulfill their potential in these contexts, the per-
a very thorough record of performance, such as by develop- formance rating results must be dependable. They can be de-
ing a portfolio of work samples, each of which has a rating pendable only if performance assessment developers and
scale attached. users adhere to basic rules of test design. They must
1. be clear on the purpose of the assessment. Without
Rating scales and checklists have distinct advantages. For
knowledge of the decision(s) to be made and decisionmaker(s),
instance, they
we cannot plan useful, economical assessments.
• combine the observation and judgment into an easy-to- 2. communicate effectively, with (a) performance criteria
interpret record; conveyed in an understandable way to students prior to the
• apply equally well to process and product evaluations; assessment and (b) challenging performance exercises.
• can be tailored to many types of performance criteria; 3. maximize the validity of the assessment by (a) being sure
• provide a convenient frame of reference for summarizing about the purpose; (b) defining the student characteristics
and comparing assessment results; and to be evaluated; (c) specifying levels of performance along
• permit a quick, efficient means of recording data. appropriate continuums; (d) using exerci~es that sample the
Rating scales and checklists are most efficient, and therefore range of performance contexts; and (e) comparing ratings
most useful, when man,y examinees and/or many performance with other achievement data when possible.
criteria are involved. 4. maximize the reliability of assessment by (a) using clear
criteria; (b) training raters thoroughly; (c) planning and im-
Anecdotal records and portfolios are more cumbersome but plementing appropriate scoring procedures; (d) gathering
provide a level of detail and richness of information not enough samples of performance; (e) minimizing rater bias
achievable with rating scales. They also allow the quality of through cultural awareness, clear criteria, and thorough
behavior or products to be described with examples, for in-
training; and (f) providing for a standard, uniform assessment
stance, as performance changes over time. These are most
condition when needed.
useful when the assessment involves a few examinees and
5. attend to the economy of assessment by (a) adapting the
a limited range of traits. Please note that video and audio
form of the assessment to the pur pose; (b) gathering only
tape can also serve as a basis for recording examples of as many samples of performance as the decision requires;
achievement for inclusion in an anecdotal record or portfolio. (c) reusing good exercises when possible; (d) rating in terms
One final recordkeeping system that is often used, and
of the relevant criteria only; (e) training raters to be efficient
almost always is ill-advised, is mental recordkeeping. When
and to reduce the need to reevaluate performances; (f) using
we attempt to remember records of achievement, five dif- rating scales, checklists, and work samples (portfolios) when
ferent things can happen, and four of them are bad. The one
appropriate; and (g) relying on students as raters when possi-
good (but unlikely) outcome can be that we maintain a vivid,
ble and appropriate.
accurate recollection of the performance. On the downside, Adherence to these guidelines will result in high-quality per-
we can formance assessment results.
• forget and lose valuable information;
• ascribe the recollection to the wrong examinee;
• change the mental record as a result of observing subse-
quent performance; and,
• most dangerous of all, filter all subsequent observations
through the recollection of that initial event, biasing
results. Teaching Aids Are Available

Written, audio tape, or even video tape recorded record- A set of teaching aids, designed by Richard J. Stiggins to
keeping is advised. complement his ITEMS module of "Design and Development
Select your recording strategy on Blueprint entry 4C, then of Performance Assessment," is available at cost from
develop your checklist or rating scale, if selected. The check- NCME. These teaching aids consist of brief additional text
list should include all key attributes of performance. The and 19 overhead masters. As long as they are available, they
rating scale should include all performance criteria and a can be obtained by sending $1.00 to: Teaching Aids, ITEMS
descriptive scale or continuum with each point described in Module #1, NCME, 1230 17th St., NW, Washington, DC
specific observable terms. Scales relying on general labels 20036.
such as "poor" to "excellent"will not suffice. Poor perfor-
mance needs to be defined in concrete performance terms,
if we are to provide valid, dependable, and meaningful feed-
Fall 1987 39
 ~=======:==============~ ITEMS • Instructional Topics in Educational Measurement ~====~====~~==~====

PERFORMANCE ASSESSMENT BLUEPRINT

Step 1: Clarify reason(s) for assessment

A . Spec ify dec ision(s) to be made

'i B. Specify decisionmaker(s)

C. Specify use to be made of results

_ _ Mas te ry _ _ Rank order _ _ Combination
D . Students to be assessed
How many? _ _ What grade level?
Unique characteristics?

Step 2: Clarify performan ce to be evaluated

A. Specify the content or skill focus of the assessment _ _ _ _ _ _ _ _ __ _ _ _ __ _ __ _ _ _ _ _ _ _ __ __ __

B. Select type of performance to be evaluated

Process; id ent ify proc ess _ _ _ _ _ _ _ _ __ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _.........,..:.._ _ _ _ _ __

Product; :dentify product _ _ _ _ _ _ _ _ _ __ _ _ _ __ _ _ _ _ _ _ _ _ _ __ _ __ _ _ _ _ _ _ _ __

_ _ Combination; identify each in spaces provided above

C. List performan ce criteria

Factor Meaning Cont inuum

Step 3: Design exerc ises

A . Select form of exe rcises

_ _ Structured exerc ises; devise sample exercise _ _ __ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

_ _ Natural events; describe events to be observed _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __

_ _ Combination; devise sample exercise and describe events in spaces provided above
B. Determine obtrusiveness of assessment
_ _ Examin ees aware of assessment _ _ Examinees un aware of assessm ent _ _ Combination
C. Determine amount of eviden ce needed
_ _ 1 sample, 1 time _ _ Multiple samples, 1 time _ _ Multipl e samples over time

Step 4: Design performance rating plan

A Determine type of score need ed

_ _ Holistic _ _ Analytic _ _ Combination
B. Determine who is to rate performance
_ _ Teach er _ _ Self-rating _ _ Combination
_ _ Other expert _ _ Peer-rating
C Clarify score reco rding method
_ _ Checkli st _ _ Anecdotal record _ _ Combination (specify)
_ _ Rating scale _ _ Portfolio

40 Educational Measurement: Issues and Practice

================ ITEMS. Instructional Topics in Educational Measurement ~~~~~~~~~~~~

of attention to the concept of simulation, the application of high tech-

Self-Test nology in performance assessment, the basic concepts of fidelity, cost,
and the essentials of good simulations. In addition, they emphasize
Instructions: Use these exercises for additional practice the use of situation tests such as in-basket tests, games, contests,
or to assess your proficiency in performance assessment and diagnostic problem-solving tests in which examinees engage in
design. As you do so, please realize that there is more than some real-life tasks. Finally, the authors provide guidelines for
one appropriate design. Any of a variety of assessments developing performance tests, covering test specifications, stimulus
might serve the purpose. In addition, as you design the assess- conditions, response modes, conditions for appropriate test admin-
ments, address issues of performance criteria within the istration and scoring, and guidelines for test use.
limits of your background knowledge.
Haynes, S. N., & Wilson, C. C. (1979). Behavioral assessment. San
1. For the sake of this exercise, assume you have been
Francisco, CA: Jossey-Bass, 526 pp.
selected to design and carry out a science fair in your school.
Behavioral assessment-based on the observation of actual
Students are to develop and pres~nt science projects. Devel-
behavior-is as important to the clinical psychologist as performance
opers of the best projects are to be awarded scholarships. assessment is to the classroom teacher. Haynes and Wilson illustrate
Complete a performance assessment blueprint, filling in . this by referencing more than 70 journal articles addressing
details where needed, to reflect how you would conduct the behavioral assessment in educational contexts. Educators will find
assessment. the description of developments in behavioral assessment and their
educational applications both interesting and useful. The publication
2. If you were to design a performance assessment to eval- also includes reference to current research literature on such recent
uate whether someone could drive an automobile on public methodological advances as behavioral coding systems, strategies
streets and highways, what would the blueprint look like? for assessing interobserver agreement, and advances in participant
3. Primary level teachers (grades 1, 2, and 3) often need observation; observation in natural environments, including schools;
to identify students who may require special help in the devel- observations of child behavior in structured le(Lrning environments;
and use of self-monitoring, behavioral questionnaires and behavioral
opment of their reading skills. Since these students have not
interviews.
yet developed traditional test-taking skills, paper-and-pencil
tests cannot be used. Using performance assessment, how
might primary teachers identify these students? Priestley, M. (1982). Performance assessment in education and train-
4. Assume that you are a high school chemistry teacher ing: Alternative techniques. Englewood Cliffs, NJ: Educational
whose objective is to determine if safety rules are being Technology Publications, 263 pp.
adhered to in the lab. Because the school has had a history This is a basic introductory textbook on performance assessment.
of accidents, your supervisor has asked for proof that your In it, Priestley describes 25 different types of performance assess-
classes are being conducted safely. Design a performance ments in terms of their form, uses, advantages, disadvantages, and,
most important, steps in test development. He includes many con-
assessment to provide needed data.
crete illustrations of assessments and addresses keys to successful
5. Teacher evaluation is often based, at least in part, on administration and coding. Among the illustrations are actual per-
the observation and judgment of teacher behavior in the class- formance tests including work samples, identification tests, and
room. Assume you are a principal whose assignment is to supervisor, peer, and self-ratings. Simulations including job simula-
evaluate a first-year teacher to be sure that that teacher is tions, written simulations, and management simulations are de-
competent in terms of state-specified minimum competen- scribed. Observational assessments, such as checklists, rating scales,
cies. Your job will be to decide to retain or release this teacher and anecdotal records are summarized as are oral assessments, paper-
and-pencil assessments, and personnel records.
at the end of the year. How would you design and conduct
this assessment?

APPENDIX
Additional Readings
EXAMPLES OF ANALYTICAL RATING GUIDES
Berk, R. A. (Ed.). (1986). Performance assessment: Methods and
applications. Baltimore, MD: The Johns Hopkins University Press. These scoring guides are adapted from scales developed by Carol
This is the most comprehensive treatment of performance assess- Meyer and Vicki Spandel, Beaverton Public Schools, Beaverton, OR.
ment methodology completed to date. Nineteen chapters address
methods and methodological issues, as well as applications in business
and education. Of particular interest to educators are chapters on Organization
teacher evaluation, student evaluation, writing assessment, and
listening and speaking assessment. 5 The writer organizes material in a way that enhances the reader's
understanding or that helps to develop a central idea or theme. The
Fitzpatrick, R., & Morrison, E. J. (1971). Performance and prod- order may be conventional or not, but the sequence is effective and
uct evaluation. In R. L. Thorndike (Ed.), Educational measure- moves the reader through the paper.
ment (pp. 237-270). Washington, DC: American Council on • Details seem to fit where they're placed, and the reader is not left
Education. with the sense that "something is missing."
This discussion of performance assessment addresses the topic from • The writer provides a clear sense of beginning and ending, with
some unique perspectives. For example, the authors give a great deal an inviting introduction and a satisfying conclusion ("satisfying"

Fall 1987 41
=~ ....."""'= ====..........===== ITEMS • Instructional Topics in Educational Measurement ;;:::::===="""'....."""'======.....=
in the sense that the reader feels the paper has ended at the right without a strong sense of personal commitment or involvement. The
spot). result is often pleasant or acceptable, yet not striking or compelling
• Transitions work well; the writing shows unity and cohesion, both in a way that draws the reader in.
within paragraphs and as a whole. • WI'iter may seem self-conscious or unwilling to take a risk-may
• Organization flows so smoothly that the reader doesn't have to seem to be writing what he/she thinks the reader wants.
think about it. • Paper lacks individuality or the ring of conviction.
• The writing communicates, but only in a routine, predictable fash-
3 The writer attempts to organize ideas and details cohesively, but ion that tends to make it blend in with the efforts of others.
the resulting pattern may be somewhat unclear, ineffective, or • Voice may be inconsistent; it may emerge strongly on occasion,
awkward. Although the reader can generally follow what's being said, only to shift or even disappear altogether.
the organizational structure may seem at times to be forced, obvious, • The reader has only an occasional or limited sense of interaction
incomplete, or ineffective. with the writer.
• The writer seems to have a sense of beginning and ending, but
the introduction and/or conclusion tend to be less effective than 1 The writer may not have understood the assignment or may simply
desired. have felt indifferent toward the topic. As a result, no clear voice
• The order may not be a graceful fit with the topic (e.g., a forced emerges. The result is flat, lifeless, very mechanical, and stilted, or
conventional pattern, or lack of structure). possibly inappropriate.
• The writer may miss some opportunities for transitions, requir- • The writing has virtually no individual personality or character;
ing the reader to make assumptions or inferences. there is no identifiable voice behind the words.
• Placement or relevance of some details may be questionable (e.g., • There is little or no evidence of the writer's involvement in the
interruptive information; writer gets to the point in roundabout topic.
fashion). • The reader has no sense that this writer was "writing to be read"
• while some portion of the paper may seem unified (e.g., organiza- and experiences virtually no writer-reader interaction.
tion within a given paragraph may be acceptable), cohesion of the
whole may be weak.
Writing Conventions
1 Organization is haphazard and disjointed. The writing shows little 5 The writer has a good grasp of standard ~riting conventions
or no sense of progression or direction. Examples, details, or events (gTammar, capitalization, punctuation, usage, spelling, paragraphing).
seem unrelated to any central idea, or may be strung together helter- There are no glaring errors. In fact, errors tend to be so minor that
skelter with no apparent pattern. reader can easily overlook them unless searching for them
• There is no clear sense of a beginning or ending. specifically.
• Transitions are very weak or absent altogether. • Sentence structure and paragraphing tend to be sound.
• Arrangement of details is confusing or illogical. • Agreement of subject and verb is correct.
• There are noticeable information "gaps"; the reader is left dan- • Punctuation is smooth and enhances meaning. (Informalities-
gling or cannot readily see how the writer got from one point to use of dashes, contractions- are allowed.)
another. • Spelling is generally correct.
• The paper lacks unity and solidarity.
3 Errors in writing conventions begin to impair readability. Sentence
Voice structure is generally correct on simple sentences, though more com-
plicated patterns may contain such problems as faulty parallelism,
5 The paper bears the unmistakable stamp of the individual writer. inconsistent tense, voice shift (e.g., first to second person), dangling
The writer speaks directly to the reader, and seems sincere, can- modifiers, or vague pronoun reference.
did, and committed to the topic. The overall effect is individualistic, • Errors may reflect hasty writing or lack of careful attention to
expressive, and engaging. detail in editing.
• The paper is honest. There is a real effort to communicate, even • The reader can follow what's being said overall, but errors in con-
when it means taking a risk (e.g., an unexpected approach or ventions may require the reader to pause or reread on occasion.
revealing of self).
• The writing is natural and compelling. 1 There are so many errors in usage, sentence structure, spelling,
• Tone is appropriate and consistently controlled. and/or punctuation that the paper is hard to understand.
• The writer's own enthusiasm or interest comes through and brings • The student shows very limited understanding of, or ability to
the topic to life. apply, conventions.
• The reader feels an interaction with the writer and, through the • Basic punctuation tends to be omitted, haphazard, or just plain
writing, gains a sense of what the writer is like. wrong.
• Spelling errors are typically frequent, even in common words.
3 The writer makes an honest effort to deal with the topic, but • Fragments, run-ons, and awkward constructions abound.

42 Educational Measurement: Issues and Practice

 Instructor's Guide

for

DESIGN AND DEVELOPMENT OF PERFORMANCE ASSESSMENTS

Richard J. Stiggins
Center for Perfonnance Assessment
Northwest Regional Educational Laboratory
300 s.w. Sixth Avenue
Portland, Oregon 97204

January 1987

Funds for development of this unit provided in part by the Office of

Educational Research and Improvement (OERI), u.s. Department of Education.
The opinions expressed in this pUblication do not necessarily reflect the
position of OERI, and no official endorsement by OERI should be inferred. The
material presented herein represents an adaptation of a guide entitled
ItEvaluating Students by Classroom Observatiop" published in 1982 by the
National Education Association Professional Library, Washington, D.C.
Published by permission of the National Education Association.

11
 INSTRUCTOR I S GU IDE

Rationale for the Module

Th1s instructional module deals with a form of Dleasurement that 1S central

to our evaluation of achievement--measurement based on observation and

protessional jud~ment, or performance assessment. One of the important

paradoxes in the field of educational measurement, in my opinion. is the fact

that performance assessment 1S so crucial to the effective management of

instruction, and yet is so completely disregarded in the measuremen·t training

offered to teachers, administrators and assessment specialists. Until

recently, professional judgments regarding student achievement have been all

but ignored outside the classroom and are often ignored in [Link]

textbooks used in educator training.

The one skill arena where this pattern has been broken is writing

assessment. Over the past decade, multiple-choice tests of language usage,

grammar and mechanics have been replaced by writing samples as the primary

means of measuring writing skill. This change took place because English

teachers across the land felt writing samples represented the most valid way

to measure [Link] skills and they no longer would sett Ie for language usage

tests just because someone felt they were less expensive to administer 'and

score. Speaking in a unified voice, the National Council of Teachers of

English demanded the most valid assessment and now they have it. Virtually

every state-level writing assessmegt program now bases at least part of Its

score on a teacher-rated writing sample, and a majority of [Link]

achievement battery published 1n the last decade includes a writing sample as

a local testing option.

12
 How did this occur? Why is it that teacher judgments ga ined such

credibility in the writing domain? What lessons can we learn from th is

example about the ingredients that have to be present for a performance

assessment to be of high quality and to be perceived as sound? This module

explores answers to these questions.

In my opinion, professional judgments are regarded as second-class

measurement citizens because those judgm,ents appear to be "intuitions" or

unsubstant iated guesses. In many cases. they may be [Link] more. They may be

high quality professional judgments based on sound underlying [Link]

methodology. In other cases, however, they may be based on unsound

measurement. In cases where judgments are based on sound methodology J we need

to provide evaluators with an assessment framework and means of communication

that will make the underlying rigor apparent. In cases where assessments are

based on unsound methodology, we need to teach users a framework and means of

cOlIlIUunication that .... ill raise the quality of the measurement. This training

module is designed to serve both of these purposes.

Performance assessments 1n the area of writing have gained credibility

because many feel that writing samples offer the highest fide lity and most

valid means of measuring the important skills 1.n that domain. 1 n acid i t ion ,

measurement specialists and teachers have .... orked hand-i.n-hand to (1) develop

clearly articulated performance criteria and strategies for training teachers

to apply them in a rigorous manner, (2) devise writing exercises capable of

eliciting useful samples of [Link], and {) plan and conduct scor1ng

procedures that ensure reliable ratings of student performance. This approacn

car. be followed 1.0 any subject matter area to devise performance assessments

yielding valid, ['eliable and credible data. This [Link] module sho .... s how

this goal can be reached.

13
 Training Options

These training material~ are equally effective presented as a group

\lorkshop or as an individual study guide. The study guide format takes the

user through the step-by-step process of designing a performance assessment.

The four major steps are (1) clarifying the reason for assessment. (2)

defining t~e performance to be evaluated, (3) designing exercises. and (4)

planning scoring procedures. Within these four steps are 13 test design

decisions. Each design decision is spelled out in terms of design

alternatives and factors to consider in selecting from among those options.

The user who makes all 13 decisions and fills in detail around each has

completed a detailed blueprint of a performance test.

The same material also can be presented in a 90 to 120 minute \lorkshop.

Transparencies are included here to allo\l the instructor to take a student

group through the same step-by-step design process. Workshop participants can

work individually or in small groups to design a performance assessment

relevant to them. The instructor begins by establishing the importance of

observation and professional judgment as a means of measurlng achievement (via

discussion or lecture), then turns to the 13 design decisions, spelling each

out and allowing participants time to make their choices. After revlewing key

quality control [Link], the instructor can have participants s-hare their

blueprints with the class. As an assignment, the instructor can have

developers polish and refine their blueprints to hand in for evaluatlon.

Before presenting such a workshop, carefully revie\l the instructional module

along \lith the transparencles to ensure coordinatlon.

Please bear in mind the fact that [Link] design of a performance assessment

lS not a simple process. There are some complex decisions to be made, -such as

14
that. of spelling out clear and relevant performance criteria. The entire

design process LS likely to look long and complicated to workshop participants

the first time through. Be encourag1ng and let them know it becomes easier

wi th pract ice.

In addition, be careful not to advocate too much. The idea of this unit

1S not to teach a rigid framework that must be used 1n its entirety every time

a professional judgment is to be made. Rather. the idea is to make

participants aware of the key dimensions of sound performance assessment.

Advise them to account for as many as possible each time they devise a

performance assessment and to follow the entire blueprint process step by step

for the most important assessment.

Intended Audience

This training 1S relevant for a wide variety of educators and personnel

assessment [Link]. For example, teachers at all levels and in all

subjects use performance assessments. Therefore, this training module fits

well in both preservLce and inservice teacher tr3ining programs 1n

assessment. Principals and district administrators are urged to be

instructional leaders. Assessment is a key ingredient of effective

ins truct ion. If school administrators are to provide leadership in

assessment, they must do so with a clear understanding of performance

assessment methodology. Therefore. this training module is relevant for them.

Those who a~e concerned with professional certification often rely on

performance assessment methodology. Fer instance, ~!rsonnel evaluation is a

performance assessment task. Therefore. the design decisions outlined in this

module must be <:arefully considered in that context. Those who des 19n and
 '\
i
develop professional licensing examinations or who supervue trainees i.n field

internship experiences are also concerned with: the valid, reliable and

efficient measurement of work-related perform~lOce. Training along the lines

spelled out in this module will help maximize the quality of thes'e assessments

also.

However, for all of these audiences, this module represents only the basic

course in performance assessment methodology. Depending on the context and

the seriousness of the decisions to be made on the basis of assessment

results, the test developer may need to know a great deal more than we can

present in this one module (see re ferences below). But regardless of the

context, the module represen ts an appropriate starting point.

Additional Resources

Those who are interested Ln developing a stronger background in

performance assessment, so they are better equipped to teach performance

assessmen t methodology andlor to deve lop sound assessments, are urged to

select readings from the bibliography provided below.

Those particularly interested in historical perspective and the evolution

of thought on performance assessment might focus particularly on references by

Ryans and Fredericksen (1951), Glaser and Klaus (1962), Fitzpatrick and

Morrison (1971) and Priestly (1982). These present detailed frameworks as

conceptualized at the beginning of each decade.

Others may be interested in information on educational applications of

performance assessment. In this regard, Gronl~nd (1981), Priestly (1982) and

Stiggins (1984) are likely to be helpful. In addition, Stiggins and

BridlSetord (1986) explore the role of performance assessments in day-co-day

classroom assessment.

16
 Advanced topics of interest may include the use of as-sessment center-.

methodology (Moses and Byham, 1977), [Link] of dependability of

performance ratings via genera1izability analysis (Brennan and Kane, 1979) and

application of performance assessment mechodology in the context of· beh·av ior

therapy. (Haynes and Wilson, 1979; and Citilillero, Calhoun and Adams, 1986).

For a general desk reference on the topic of performance assessment as

developed and used across educational and work-related contexts, instructors

and scholars are urged to refer to the volume edited by Berk (1986).

17
 REFERENCES

. Ber'k, R.A. (Ed.) (986) Performance Assessment. Baltimore, H!): The Johns
Hopkins University Pre~s, 530 pp.

Brennan, R.L. & Kane, M.T. (1979) Generaliz~bility theory: A [Link]. New
D=-,-,i..:;.r..;e,-:c..;t..;i..;o..;n;.:..s=-,-,_f-:-o-:r_T:-:e-es,-t..;i:.;n~goL-a_n_d_M_e_a_s_u_r-ee_m_e_n_t • Sa n Fr anc is co, CA: Jos sey-Bass
Publishers, 33-51.

Cimmero, A.R., Calhoun, K.S. & Adams, H.E. (Eds.) (1986) Handbook of
Behavioral Assessment C2nd ed.). New York, NY: John Wiley & Sons,
789 pp.

Fitzpatrick, R. & Morrison, EeJ. (1971) "Performance and production

evaluat ion." In R .L. Thornd ike (Ed.) Educat ional Measurement.
Washington, DC: American Council on Education, 237-270.

Gronlund, N.E. (1981) Evaluating learning and development: Observational

teChniques. Chapter 16 in N.E. Gronlund Measurement and Evaluation in
Teaching (4th ed.). New York, NY: MacMillan, 433-455.

Haynes, N. & Wilson, C. (1979) Behavioral Assessment. San Francisco, CA:

Jossey-Bass Publishers, 526 pp.

Moses, J.L. & Byham, W.C. (977) Applying the Assessment Center Method.
New York, NY: Pergamon Press, 310 pp.

Pries tly, M. (1982) Performance Assessment in Education and

Train ing: Alternative Techniques. Englewood Cliffs, N.J: Educational
Technology Publications, 263 pp.

Ryans, D.G. & Fredericksen, N. (L951) Performance tests of educational

achievement. In E.F. Lindquist (Ed.) Educational Measureme~~.
'Washington, DC: American Council on Education, 455-493.

Stiggins, R.J. (984) Evaluating students by classroom observation: Watching

students grow. Washington, DC: National Education Association
Professional Library, 32 pp.

Stiggins, R.J. & Bridgeford, N.J. (1986) The ecology of :::lassroom assessment.
Journal of Educational Measurement, 22(4) 271-286.

18
WORKSHOP VI SUALS

19
I\)
o

PERFO CE
ASSESSMENT
Au A · bOlt Ute gO~
 PERFORMANCE
ASSESSMENT
* Apply [Link] r ski lis
* Naturql/y occurring or
structured exercises
* Origina1re~ponses
* Observe + rate

21
~EXAMPLES<'
Ora I read j n9
WritiYl9
Speakins
Foretgn JaY19 uaSe dialOBue
Product evaluation .

22
 SUBJECT MATTER
AREAS
•Communication Ski I/s - Reading
Wrifins
Speakin9
Listenin9
•Business and Industria I Education
•Psychomotor Development
· Visual and Perform in9 Arts
•Social- Emotional Development
~ Affective Assessment
 Why Use Performance Tests?
.:. Demonstrate Ski/Is
\\ Req/-life SitUqtiOJ1S and
Competencies
.:. Effective Diagnosis
~. observe NJturCJ/ Events
.:. Continuous Feedbqck

24
 PERFORMANCE
ASSESSMENTS

Formal: PreplCll1ned, ~tructured

Informal: Sponf4neous observqtions

25
 PERFORMANCE ASSESSMENT
DEVELOPMENT
-------
* Clarify Reason(s) for
Assessment

* Clarify Performance

* Design Exercises

* Design Rating Plan

26
1. Clarif ying Reason( s)

A. Spec-ify Decisions to be Made

_Individual Diagnosis
_Group Needs Assessment
_Grading
_Grouping
_Selection
_Certification
_Evaluation
_Other ________- -
 8. Specify Decision MCJker( s)

_.Teacher
_Student

- Adm i n.i s t rat 0 r

__ Parent
_Other __________

c. Use of Results

_Mastery
_Rank
_ Combination

D. Describe Students

_Number
_Grade

28
 For steps lC and
all of 4

FACTOR TO. CONSIDER

* NA TURE OF THE DECISiON

TO BE M)\DE

29
 2. Specify Peroformance

A. Specify Focus of Assessment

- Content Domain
_Skill Focus

B. Select Type of Performance

- PrDcess
_Product
- Combination

C. Identify Performance Criteria

List All Key Ingredients

30
 For step 2C

FACTORS TO CONSIDER

* IN OBSERV ABLE TERMS

* FACTOR DEFINITIONS

* CONTINUUM (POOR-GOOD)

* FEEDBACK TO PERFORMER

31
 3. Design Exercises

A. Select Form

_Structured Exercises
Natural Events
_Combination

B. Obtrusiveness

_Aware of ,Assessment
Unaware of Assessment
Combination

C. Amount of E:vidence

_1 Sample, 1 Time
Mult. Samples, 1 Time
Mult. Samples Over Time
32
 For step JA

FACTORS TO CONSIDER

* SERIOUSNESS OF DECISION

* EQUITY ISSUE

* NA TURAL AV AIL ABILITY

- OF EVIDENCE
 For step 3A

FACTORS TO CONSIDER

* ALL NECESSARY INSTRUCTIONS

* CLARITY OF COMMUNICA TION

34
 For step 38

fACTORS TO CONSIDER

* IMPORT ANCE OF DECISION

* NA TURAL AV AILA81LITY

OF EVIDENCE

• MOTIV ATION ISSUE

* STUDENT PRIV ACY RIGHTS

35
 For step 3c

FACTORS TO CONSIDER

* REPRESENT A TIVENESS

* IMPORT ANCE OF DECISION

* TIME PER SAMPLE

* TIME UNTIL DECISION

36
4. Design Rating Pion

A. Type of Score Needed

_Holistic
_Analytical
_Combination

B. Specify Rater

_Teacher

- Peer
Self

- Combination

37
 c. Design Recording [Link]

_Checklis·t
_Rating Scale

- Anecdotal Record
_Portfolio
_Combination

38