0% found this document useful (0 votes)
22 views20 pages

Enhancing Assessment Literacy in Education

This document discusses the need for assessment literacy to improve the quality of assessment in higher education. It notes that assessment faces many pressures today and is often high-stakes. While attempts at reform have been made, systemic change is difficult due to resistance from tradition and social expectations. The document contrasts conservative versus adaptive approaches to assessment and argues that assessment practice should embrace constructivist and collaborative approaches. It states that developing assessment literacy among institutional leaders and academics is the missing factor needed for successful assessment reform. Assessment literacy involves understanding the complex landscape of factors that influence assessment.

Uploaded by

Timothy Allen
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
22 views20 pages

Enhancing Assessment Literacy in Education

This document discusses the need for assessment literacy to improve the quality of assessment in higher education. It notes that assessment faces many pressures today and is often high-stakes. While attempts at reform have been made, systemic change is difficult due to resistance from tradition and social expectations. The document contrasts conservative versus adaptive approaches to assessment and argues that assessment practice should embrace constructivist and collaborative approaches. It states that developing assessment literacy among institutional leaders and academics is the missing factor needed for successful assessment reform. Assessment literacy involves understanding the complex landscape of factors that influence assessment.

Uploaded by

Timothy Allen
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Making a Difference

Assessment Literacy:
A Precursor to Improving the Quality of Assessment

Janice Orrell

Higher Education today….

Puts pressures on assessment:


! Public accountability
! Collaborative relationships with industries and
professions
! Burgeoning new possibilities for place, space
and mode of education
! More students
! Diversity of students’ backgrounds
Assessment is High Stakes

! Defines educational goals and outcomes


! Shapes student learning
! For many years, students are defined by
the grades they receive
! Academics are defined by the grades they
give
! Grades used as currency

Assessment is regarded as vehicle for


reform but … systemic change is
difficult

! Attempts for reform are atomistic,


! Attends largely to the point of practice
! Ignores
– the complexity
– the integral relationship with T&L
! Resistance to reform is reinforced by
tradition, common practice, social
expectations
Responses to the Challenges

Varied and recognisable on a continuum

Conservative to Innovative

Resistant to Adaptive

Merely useful to Really useful

Conservatism is evident in much of


respected higher education
assessment practice

Longing for past glories


Efforts focus on inventing better
mouse traps
Assessment measures knowledge
known & is merely useful
Conservative responses to the
challenges

Seek assessment processes that are:


Fixed, non-negotiable
Within known boundaries
Predictable
Safe

Prevailing source of assessment


“expertise” is its own practice.

Practice making practice is


problematic (Britzman, 2003)
Many of the major problems academic
organisations experience today is
due to a habit of clinging to past
practices and organisational
structures
(Tierney:1999,172).

Adaptive approaches to assessing


learning are…

Constructivist
Independence & challenge seeking
Interdependent & collaborative
Authentic & relevant
Reflective and creative
Engage in self review & peer review
Adaptive

The context in which information is used is


unpredictable, so…
Learning to learn independently and
interdependently is really useful
Risky
Seeks to identify the unknown and ways to find out.
‘Ignorance’ curricula(Ann Kerwin)

The desire for such innovation,


exploration of the unknown and
risk taking is highly evident in the
posted theories and ideals of the
everyday academic but…..
Assessment practice is surounded by
uncertainty (Allen 1998)

Academics award grades that,


! They feel comfortable with,
! They believe the student will feel is fair,
! Can withstand robust scrutiny by colleagues.

Decisions that lead to preferred and expected


outcomes using their experience to predict
the best and worst outcomes!

New Possibilities with ICTs

Dialogue between learners


! Dialogue using appropriate evidence well
! Dialogue with experts in the field
! Focus on what is to be learnt and what is not
already known or ‘coverage’.
! Self assessment & immediate feedback for
low level learning
Goal: Critical consuming of available
information
Assessment Practice Embodies
Choices

Formative or summative Grade or no grade How much feedback?


Teach to the test or encourage creativity
Essays or authentic projects
Student or peer assessment or
Weekly work or end of semester exam
teacher graded
Individual or group work Who will grade? University or clinicians
Set or negotiated tasks? Assess participation in class or not?

How many pieces of Seen or unseen exams


assessment is enough? Open or closed book exams

Should all students have to do the same tasks?


Should presentation impact on
grades?
How important are “hand-in” dates?

Assessment is driven by the best of


intentions

! Motivate and reward student learning


! Fairness and equity for students
! Maintenance of intellectual rigour in
disciplines & standards of professional
practice (Fit for purpose, Biggs)
! Efficiencies in practice
! Conforming to government & social
expectations (Value for money, Biggs)
“Good practice” in assessment
underpins fairness

" Appropriate tasks that encourage learning


" Clear descriptions of grading criteria &
grade related performance standards
" Decision-making guided by posted criteria
and standards when grading
" Timely feedback that linked to the
assessment criteria (adapted from Carroll 2004)

Assessment cannot be fair unless


students know the ‘rules of the game’
How is this possible if those managing
‘the game’ don’t understand the rule
or their basis or make the up as they
go along???
Assessment in higher education is
often a triple blindfolded high jump!

Many academics are unclear about


exactly what they are expecting
students to produce.

! Many rules governing assessment


are tacit & are learnt by trial and
error.

Assessment is Complex but…

Few are adequately prepared for their


assessment responsibilities
– as assessors (graders),
– designers,
– policy drafters,
– examination boards,
– curriculum leaders,
– accrediting bodies.
Assessment is Complex but Over
Simplified

! Identified as marking or grading


! Is a postscript in curriculum design and
development

Assessment is Complex but…

It is rarely guided by scholarship


Available information is:
! Theoretical treatise
! ‘Gut piece’ writing
! Atomistic research
! ‘How to’ guides
Oils ain’t just oils!

Assessment is treated differently


depending on the orientation or
paradigm used!

– Vocational/Traditional
– Liberal progressive
– Socially critical

Common Strategies

! Stafftraining (curriculum development


& grading)
! Student education and remediation
! Curriculum review & innovation
! Policy revision and expansion
! Funding (learning & quality)
! Govt audits, regulation & monitoring
These are NOT NEW Solutions!

But problems with assessment


persist !

So what are we not addressing?

Assessment reform

Has failed to systematically engage


institutional and system wide
leadership and management of
assessment.
Has not recognised the need for
assessment literacy of its formal
and academic leaders
Leadership grounded in assessment
literacy is the missing factor

! Providing vision and reform


! Resource allocation
! Moderation processes
! Confronting and managing the
academic culture

Academic leadership is crucial in


assessment reform!

! Focus on assessment literacy


! Engaging with discipline cultures
! Explicit confronting of the academic
culture
! Multi-level institutional & academic
assessment leadership
! Holistic, systems approach
What is assessment literacy?

Knowledge of the vast & complex factors


that constitute assessment
Knowing the landscape of the field and
explicit and implicit structures and
strictures([Link] accessed18/11/2005)
Awareness of their inter-relatedness
How individual decisions about one aspect
impacts on all the processes involved

Assessment literacy is needed to


address..

Quality assurance policies and process


Alignment between intended learning
outcomes and assessment tasks
Institution & society’s understanding of
assessment and use of grades
Use of valid interpretation processes
Students’ awareness of the standards of
performance expected of them
Interpretation& grading is influenced
by non-student performance

! Quality of other student’s work,


! Assessors’ personality
! Assessors’ experiences and beleifs
! Knowledge of the student and classroom events
Surface features,
! Introductory paragraph,
! Presentation factors

Some other issues (Baume, Yorke, & Coffey (no date)

Asking assessors to explain and justify grades


may reduce the grade!
Even where criteria exist assessors
themselves may not understand them or
agree with them!
Assessment policy?

What is a good assessment policy?

Analysis of 10 randomly selected universities


Shows the following

Common Policy Concerns & Principles


! Fairness & equity
! Transparency and clarity of expectation
! Fitness for purpose
! Alignment
! Timely feedback and opportunity for practice
! Student centred/learning centred
! Attend to and support generic graduate attributes
! Validity, reliability, consistency, reproducibility
! Realistic submission deadlines
! Catering for students’ special needs
Uncommon Policy concerns

! Exploration of potential unintended negative


consequences (Sydney)
! Efficiency for students and staff (Sydney)
! Access to remediation (Sydney)
! Ethical relationship between assessor and
student (QUT)
! Engage reflective capabilities (Curtin)
! Considered assessment of First Years (QUT)

Assessment literacy is essential at


every level

! Institutional policy makers


! Educational leaders & managers
! Curriculum designers
! Academic teachers & tutors
! Students and student associations
! Ministers of Education

! Community stakeholders
SACWG in the UK multi-institutional
studies

Comparison of coursework & examination marks


Mark distributions and marking practices
Benchmarking and threshold standards
Discipline related marking behaviour
Honours classification standards
Impact of modular mark distributions
Impact of assessment in first year on Widened
participation agenda

Our Challenge

Collaborative research and development


agenda that engages multiple
institutions and all stakeholders in
establishing multi-level literacy.
! Impact of assessment on widened
participation agenda
! Impact of particular forms of
assessment on graduate capabilities
Achieving Assessment Literacy
(Holroyd 2000)

! Activities informed by different paradigms


! Assessment scholarship*
! A range of types decision making
! Collegiality, negotiation and interaction
between all stakeholders
! Code of practice for assessment grounded
in values
! A commitment to critical reflection

Common questions

Powered by AI

Assessment literacy involves understanding the complex and interrelated factors that constitute assessment, including policy alignment, interpretation of grades, and ensuring fairness and equity in grading practices. It allows institutional leaders and educators to align learning outcomes with assessment tasks, thus contributing to more informed and systematic reform efforts. Additionally, fostering assessment literacy promotes critical reflection and adaptation within institutions, leading to assessments that are valid, reliable, and aligned with educational goals .

Academic organizations struggle with assessment reform due to systemic challenges such as entrenched traditional practices, resistance to change, and a lack of engagement with institutional leadership in driving reform. Attempts at innovation are often atomistic and focus narrowly on practice rather than addressing the broader, integral relationship between assessment, teaching, and learning. Many academics are also inadequately prepared for their roles as assessors, leading to policies and grading practices that are not consistently reliable or valid .

Conservatism in higher education assessment often results in adherence to traditional practices that prioritize predictability, safety, and adherence to known boundaries. This approach limits innovation and adaptation in assessment techniques and largely focuses on processes that measure known knowledge, which can hinder meaningful educational reform. These conservative practices are resistant to change due to their long-standing acceptance and the comfort they provide within academic culture .

Adaptive assessment approaches enhance learning by promoting constructivist methods that encourage independence, collaboration, and creativity. These approaches focus on authentic and relevant tasks that require students to engage in self-review and peer review, fostering critical thinking and lifelong learning skills. By seeking to identify unknowns and encouraging exploration, adaptive assessments prepare students for real-world challenges and enable them to learn both independently and interdependently .

Considering unintended negative consequences is important because assessment policies can inadvertently create barriers to student success or equity. For instance, rigid deadlines or standardized tasks may disadvantage certain student groups and fail to address diverse learning needs. By exploring these potential outcomes, institutions can create more inclusive and equitable assessment practices that accommodate various student backgrounds and abilities, and thereby uphold fairness within higher education .

Feedback is integral to effective assessment practices as it provides students with insights into their performance relative to the set criteria and standards. Timely and specific feedback helps students understand their strengths and areas for improvement, facilitating ongoing learning and adjustment. Moreover, feedback loops align with educational standards, maintaining rigor and informing both teachers and students for better learning outcomes .

Integrating ICTs in assessment facilitates dialogue between learners and experts and provides immediate feedback mechanisms, which can enhance engagement and understanding. ICTs allow for dynamic and interactive assessment formats that focus on what is to be learned rather than merely testing existing knowledge. This approach encourages critical consumption of information and supports self-assessment practices, fostering deeper learning and reflection on the part of the student .

Transparency in assessment policies is crucial because it ensures that expectations are clear to both students and educators, fostering a fair and equitable grading process. Clear guidelines help students understand the criteria they are assessed against, which in turn motivates learning and improves academic outcomes. Transparent policies also align assessment practices with institutional goals and societal expectations, thereby maintaining intellectual rigor and fairness .

Assessment literacy among academic leaders is crucial for driving reform as it equips them with the knowledge to understand the complexity of assessment practices and their impact on education quality. Literate leaders can better align assessment practices with learning outcomes, address institutional and societal expectations, and engage effectively with stakeholder cultures. This comprehensive understanding supports policy moderation and resource allocation, fostering a climate of continuous improvement in assessment methods .

Different assessment paradigms, such as vocational/traditional, liberal progressive, and socially critical, influence how assessments are conceived and implemented in higher education. Vocational paradigms may emphasize objective, standardized testing, while liberal progressive paradigms might focus on student-centered and inquiry-based assessments. Socially critical paradigms could encourage assessments that critique societal norms. These differences affect how assessment is integrated into curriculum design, the type of assessment tasks favored, and the overall educational objectives of the institution .

You might also like