2 PISA 2022 Mathematics Framework

Mathematics is the major domain of assessment of the 2022 cycle of the

Programme for International Student Assessment (PISA). The mathematics
framework presented in this chapter was considerably updated for the PISA
2022 assessment. The chapter outlines the “organisation of the domain”, in
four aspects: a) mathematical reasoning and its three processes; b) the way
mathematical content knowledge is organised in the PISA 2022 framework;
the relationship between mathematical literacy and 21st Century skills; and
contexts in which students face mathematical challenges. Moreover, the
chapter outlines structural aspects of the assessment, including a test
blueprint and other technical information. Various sample items from the
mathematics assessment are included at the end of this chapter.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 19

Introduction

The assessment of mathematics has particular significance for PISA 2022, as it is the major domain
assessed. Although it was assessed by PISA in 2000, 2003, 2006, 2009, 2012, 2015 and 2018, the
domain was the main area of focus only in 2003 and 2012.
The return of mathematics as the major domain in PISA 2022 provides both the opportunity to continue
to make comparisons in student performance over time, and to re-examine what should be assessed in
light of changes that have occurred in the world, the field and in instructional policies and practices.
Each country has a vision of mathematical competence and organises their schooling to achieve it as an
expected outcome. Mathematical competence historically encompassed performing basic arithmetic
skills or operations, including adding, subtracting, multiplying, and dividing whole numbers, decimals, and
fractions; computing percentages; and computing the area and volume of simple geometric shapes. In
recent times, the digitisation of many aspects of life, the ubiquity of data for making personal decisions
involving initially education and career planning, and, later in life, health and investments, as well as
major societal challenges to address areas such as climate change, governmental debt, population
growth, spread of pandemic diseases and the globalising economy, have reshaped what it means to be
mathematically competent and to be well equipped to participate as a thoughtful, engaged, and reflective
citizen in the 21st Century.
The critical issues listed above as well as others that are facing societies throughout the world all have a
quantitative component to them. Understanding them, as well as addressing them, at least in part,
requires being mathematically literate and thinking mathematically. Such mathematical thinking in more
and more complex contexts is not driven by the reproduction of the basic computational procedures
1
mentioned earlier, but rather by reasoning (both deductive and inductive). The important role of
reasoning needs greater emphasis in our understanding of what it means for students to be
mathematically literate. In addition to problem solving, this framework argues that mathematical literacy
in the 21st Century includes mathematical reasoning and some aspects of computational thinking.
Countries today face new opportunities and challenges in all areas of life, many of which stem from the
rapid deployment of computers and devices like robots, smartphones and networked machines. For
example, the vast majority of young adults and students who started university post 2015 have always
considered phones to be mobile hand-held devices capable of sharing voice, texts, and images and
accessing the internet – capabilities seen as science fiction by many of their parents and certainly by all
of their grandparents (Beloit College, 2017[1]). The recognition of the growing contextual discontinuity
between the last century and the future has prompted a discussion around the development of 21 st
Century skills in students (Ananiadou and Claro, 2009 [2]; Fadel, Bialik and Trilling, 2015[3]; National
Research Council, 2012[4]; Reimers and Chung, 2016[5]).
It is this discontinuity that also drives the need for education reform and the challenge of achieving it.
Periodically, educators, policy makers, and other stakeholders revisit public education standards and
policies. In the course of these deliberations new or revised responses to two general questions are
generated: 1) what do students need to learn?, and 2) which students need to learn what?. The most
used argument in defence of mathematics education for all students is its usefulness in various practical
situations. However, this argument alone gets weaker with time – a lot of simple activities have been
automated. Not so long ago, waiters in restaurants would multiply and add on paper to calculate the price
to be paid. Today they just press buttons on hand-held devices. Not so long ago, people used printed
timetables to plan travel – it required a good understanding of the time axis and inequalities as well as
interpreting complex two-way tables. Today we can just make a direct internet inquiry.
As to the question of “what to teach”, many restrictive understandings arise from the way mathematics is
conceived. Many people see mathematics as no more than a useful toolbox. A clear trace of this approach can
be found in the school curricula of many countries. These are sometimes confined to a list of

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

mathematics topics or procedures, with students asked to practice a selected few, in predictable (often
test) situations. This perspective on mathematics is far too narrow for today’s world. It overlooks key
features of mathematics that are growing in importance. Notwithstanding the above remark, there are an
increasing number of countries that emphasise reasoning and the importance of relevant contexts in their
curricula. Perhaps these countries cab serve as helpful models to others.
Ultimately the answer to these questions is that every student should learn (and be given the opportunity
to learn) to think mathematically, using mathematical reasoning (both deductive and inductive) in
conjunction with a small set of fundamental mathematical concepts that support this reasoning and which
themselves are not necessarily taught explicitly but are made manifest and reinforced throughout a
student’s learning experiences. This equips students with a conceptual framework through which to
address the quantitative dimensions of life in the 21st Century.
The PISA 2022 framework is designed to make the relevance of mathematics to 15-year-old students
clearer and more explicit, while ensuring that the items developed remain set in meaningful and authentic
contexts. The mathematical modelling cycle, used in earlier frameworks (e.g. OECD (2004[6]; 2013[7])) to
describe the stages individuals go through in solving contextualised problems, remains a key feature of
the PISA 2022 framework. It is used to help define the mathematical processes in which students
engage as they solve problems – processes that together with mathematical reasoning (both deductive
and inductive) will provide the primary reporting dimensions.
For PISA 2022, computer-based assessment of mathematics (CBAM) will be the primary mode of
delivery for assessing mathematical literacy. However, paper-based assessment instruments will be
provided for countries choosing not to test their students by computer. The framework has been updated
to also reflect the change in delivery mode introduced in 2015, including a discussion of the
considerations that should inform the development of the CBAM items as this will be the first major
update to the mathematics framework since computer-based assessment was introduced in PISA.
The development of the PISA 2022 framework takes into account the expectation of OECD that there will
be an increase in the participation in PISA of low- and middle -income countries. In particular the PISA
2022 framework recognises the need to increase the resolution of the PISA assessments at the lower
end of the student performance distribution by drawing from the PISA for Development (OECD, 2017 [8])
framework when developing the assessment; the need to expand the performance scale at the lower
end; the importance of capturing a wider range of social and economic contexts; and the anticipation of
incorporating an assessment of out-of-school 14- to 16-year-olds.
The increasing and evolving role of computers and computing tools in both day-to-day life and in
mathematical literacy problem solving contexts is reflected in the recognition in the PISA 2022 framework
that students should possess and be able to demonstrate computational thinking skills as they apply to
mathematics as part of their problem-solving practice. Computational thinking skills include pattern
recognition, designing and using abstraction, pattern decomposition, determining which (if any)
computing tools could be employed in analysing or solving a problem, and defining algorithms as part of
a detailed solution. By foregrounding the importance of computational thinking as it applies to
mathematics, the framework anticipates a reflection by participating countries on the role of
computational thinking in mathematics curricula and pedagogy.
The PISA 2022 mathematics framework is organised into three major sections. The first section, “Definition of
Mathematical Literacy”, explains the theoretical underpinnings of the PISA mathematics assessment, including
the formal definition of the mathematical literacy construct. The second section, ‘Organisation of the Domain’,
describes four aspects: a) mathematical reasoning and the three mathematical processes (of the
modelling/problem solving cycle); b) the way mathematical content knowledge is organised in the PISA 2022
framework, and the content knowledge that is relevant to an assessment of 15-year-old students; c) the
relationship between mathematical literacy and the so-called 21st Century skills; and d) the contexts in which
students will face mathematical challenges. The third section, “Assessing Mathematical

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 21

Literacy”, outlines structural issues about the assessment, including a test blueprint and other technical
information.
For the sake of ensuring the preservation of trend, the majority of the items in the PISA 2022 will be
items that have been used in previous PISA assessments. A large collection of release items based on
the previous framework can be found at [Link] Annex A provides seven
illustrative items that attempt to illustrate the most important new elements of the 2022 framework.
The 2022 framework was written under the guidance of the mathematics expert group (MEG), a body
appointed by the PISA contractor for the mathematics framework (RTI International), in consultation with
the PISA Governing Board (PGB). The eight MEG members included mathematicians, statisticians,
mathematics educators, and experts in assessment, technology, and education research from a range of
countries. The MEG were further supported by an extended MEG (eMEG) group, made up of ten experts
acting as peer reviewers of the framework version created by the MEG. The eMEG included experts with
a range of mathematics expertise from differing countries. Additional reviews were undertaken by experts
on behalf of the over 80 countries constituting the PISA Governing Board. RTI International, as
contracted by the Organisation for Economic Co-operation and Development (OECD), conducted two
further research efforts: a face validity validation survey amongst educators, universities and employers;
and a cognitive laboratory with 15-year-olds in different countries to obtain student feedback on the
sample items presented in the framework. The work of the MEG builds on previous versions of the PISA
Mathematics Framework and incorporates the recommendations of the Mathematics Strategic Advisory
Group convened by OECD in 2017.

Definition of mathematical literacy

An understanding of mathematics is central to a young person’s preparedness for participation in and

contribution to modern society. A growing proportion of problems and situations encountered in daily life,
including in professional contexts, require some level of understanding of mathematics before they can
be properly understood and addressed. Mathematics is a critical tool for young people as they confront a
wide range of issues and challenges in the various aspects of their lives.
It is therefore important to have an understanding of the degree to which young people emerging from
school are adequately prepared to use mathematics to think about their lives, plan their futures, and
reason about and solve meaningful problems related to a range of important issues in their lives. An
assessment at age 15 provides countries with an early indication of how individuals may respond in later
life to the diverse array of situations they will encounter that both involve mathematics and rely on
mathematical reasoning (both deductive and inductive) and problem solving to make sense of.
As the basis for an international assessment of 15-year-old students, it is reasonable to ask: “What is
important for citizens to know and be able to do in situations that involve mathematics?” More specifically,
what does being mathematically competent mean for a 15 -year-old, who may be emerging from school or
preparing to pursue more specialised training for a career or university admission? It is important that the
construct of mathematical literacy, which is used in this framework to denote the capacity of individuals to
reason mathematically and solve problems in a variety of 21 st Century contexts, not be perceived as
synonymous with minimal, or low-level, knowledge and skills. Rather, it is intended to describe the capacities
of individuals to reason mathematically and use mathematical concepts, procedures, facts and tools to
describe, explain and predict phenomena. This conception of mathematical literacy recognises the importance
of students developing a sound understanding of a range of mathematical concepts and processes and
realising the benefits of being engaged in real-world explorations that are supported by that mathematics. The
construct of mathematical literacy, as defined for PISA, strongly emphasises the need to develop students’
capacity to use mathematics in context, and it is important that they have rich experiences in their
mathematics classrooms to accomplish this. This is as true for those 15-year-old

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

students who are close to the end of their formal mathematics training, students who will continue with
the formal study of mathematics, as well as out-of-school 15-year-olds.
Mathematical literacy transcends age boundaries. For example, OECD's Programme for the International
Assessment of Adult Competencies (PIAAC) defines numeracy as the ability to access, use, interpret, and
communicate mathematical information and ideas, in order to engage in and manage the mathematical
demands of a range of situations in adult life. The parallels between this definition for adults and the PISA
2022 definition of mathematical literacy for 15-year-olds are both marked and unsurprising.
The assessment of mathematical literacy for 15-year-olds must take into account relevant characteristics
of these students; hence, there is a need to identify age-appropriate content, language and contexts.
This framework distinguishes between broad categories of content that are important to mathematical
literacy for individuals generally, and the specific content topics that are appropriate for 15-year -old
students. Mathematical literacy is not an attribute that an individual either has or does not have. Rather,
mathematical literacy is an attribute that is on a continuum, with some individuals being more
mathematically literate than others – and with the potential for growth always present.
For the purposes of PISA 2022, mathematical literacy is defined as follows:

Mathematical literacy is an individual’s capacity to reason mathematically and to formulate, employ, and
interpret mathematics to solve problems in a variety of real-world contexts. It includes concepts, procedures,
facts and tools to describe, explain and predict phenomena. It assists individuals to know the role that
mathematics plays in the world and to make the well-founded judgements and decisions needed by
constructive, engaged and reflective 21st Century citizens.

The PISA 2022 framework, when compared with the PISA 2003 and PISA 2012 frameworks, while
appreciating and preserving the basic ideas of mathematical literacy developed there, acknowledges a
number of shifts in the world of the student which in turn signal a shift on how to assess mathematical
literacy in comparison to the approach used in previous frameworks. The trend is to move away from the
need to perform basic calculations to a rapidly changing world driven by new technologies and trends in
which citizens are creative and engaged, making judgements for themselves and the society in which
they live.
As technology will play a growing role in the lives of students, the long- term trajectory of mathematical
literacy should also encompass the synergistic and reciprocal relationship between mathematical
thinking and computational thinking, introduced in (Wing, 2006[9]) as “the way computer scientists think”
and regarded as a thought process entailed in formulating problems and designing their solutions in a
form that can be executed by a computer, a human, or a combination of both (Wing, 2011[10]) (Cuny,
Snyder and Wing, 2010[11]). The roles computational thinking play in mathematics include how specific
mathematical topics interact with specific computing topics, and how mathematical reasoning
complements computational thinking (Gadanidis, 2015[12]; Rambally, 2017[13]). For example, Pratt and
Noss (2002[14]) discuss the use of a computational microworld for developing mathematical knowledge in
the case of randomness and probability; Gadanidis et al. (2018[15]) propose an approach to engage
young children with ideas of group theory, using a combination of hands-on and computational thinking
tools. Hence, while mathematics education evolves in terms of the tools available and the potential ways
to support students in exploring the powerful ideas of the discipline (Pei, Weintrop and Wilensky,
2018[16]), the thoughtful use of computational thinking tools and skill sets can deepen the learning of
mathematics contents by creating effective learning conditions (Weintrop et al., 2016 [17]). Moreover,
computational thinking tools offer students a context in which they can reify abstract constructs (by
exploring and engaging with maths concepts in a dynamic way) (Wing, 2008[18]), as well as express ideas
in new ways and interact with concepts through media and new representational tools (Grover, 2018 [19];
Niemelä et al., 2017[20]; Pei, Weintrop and Wilensky, 2018[16]; Resnick et al., 2009[21]).

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 23

A view of mathematically literate individuals in PISA 2022

The focus of the language in the definition of mathematical literacy is on active engagement with
mathematics to solve real-world problems in a variety of contexts, and is intended to encompass
mathematical reasoning (both deductive and inductive) and problem solving using mathematical
concepts, procedures, facts and tools to describe, explain and predict phenomena.
It is important to note that the definition of mathematical literacy not only focuses on the use of mathematics to
solve real-world problems, but also identifies mathematical reasoning as a core aspect of being
mathematically literate. The contribution that the PISA 2022 framework makes is to highlight the centrality of
mathematical reasoning both to the problem-solving cycle and to mathematical literacy in general.
Figure 2.1 depicts the relationship between mathematical reasoning (both deductive and inductive) and
problem solving as reflected in the mathematical modelling cycle of both the PISA 2003 and PISA 2012
framework.

Figure 2.1. Mathematical literacy: the relationship between mathematical reasoning and the
problem solving (modelling) cycle

In order for students to be mathematically literate they must be able, first to use their mathematics content
knowledge to recognise the mathematical nature of a situation (problem) especially those situations
encountered in the real world and then to formulate it in mathematical terms. This transformation – from an
ambiguous, messy, real-world situation to a well -defined mathematics problem – requires mathematical
reasoning. Once the transformation is successfully made, the resulting mathematical problem needs to be
solved using the mathematics concepts, algorithms and procedures taught in schools. However, it may require
the making of strategic decisions about the selection of those tools and the order of their application
– this is also a manifestation of mathematical reasoning. Finally, the PISA definition reminds us of the
need for the student to evaluate the mathematical solution by interpreting the results within the original
real-world situation. Additionally, students should also possess and be able to demonstrate
computational thinking skills as part of their problem-solving practice. These computational thinking skills
which are applied in formulating, employing, evaluating and reasoning include pattern recognition,
decomposition, determining which (if any) computing tools could be employed in the analysing or solving
the problem, and defining algorithms as part of a detailed solution.
Although mathematical reasoning and solving real-world problems overlap, there is an aspect to
mathematical reasoning which goes beyond solving practical problems. Mathematical reasoning is also a
way of evaluating and making arguments, evaluating interpretations and inferences related to statements

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

(e.g. in public policy debates etc.) and problem solutions that are, by their quantitative nature, best
understood mathematically.
Mathematical literacy therefore comprises two related aspects: mathematical reasoning and problem
solving. Mathematical literacy plays an important role in being able to use mathematics to solve real-
world problems. In addition, mathematical reasoning (both deductive and inductive) also goes beyond
solving real-world problems to include the making of informed judgements about that important family of
societal issues which can be addressed mathematically. It also includes making judgements about the
validity of information that bombards individuals by means of considering their quantitative and logical,
implications. It is here where mathematical reasoning also contributes to the development of a select set
of 21st Century skills (discussed elsewhere in the framework).
The outer circle of Figure 2.2 shows that mathematical literacy takes place in the context of a challenge
or problem that arises in the real world.

Figure 2.2. PISA 2022: The relationship between mathematical reasoning, the problem solving
(modelling) cycle, mathematical contents, context and selected 21st Century skills

Figure 2.2 also depicts the relationship between mathematical literacy as depicted in Figure 2.1 and: the
mathematical contents domains in which mathematical literacy is applied; the problem contexts and the
selected 21st Century skills that are both supportive of and developed through mathematical literacy.
These categories of mathematics content include: quantity, uncertainty and data, change and
relationships, and space and shape. It is these categories of mathematics content knowledge which
students must draw on to reason, to formulate the problem (by transforming the real world situation into a

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 25

mathematical problem situation), to solve the mathematical problem once formulated, and to interpret
and evaluate the solution determined.
As in the previous frameworks, the four context areas that PISA continues to use to define real-world
situations are personal, occupational, societal and scientific. The context may be of a personal nature,
involving problems or challenges that might confront an individual or one’s family or peer group. The
problem might instead be set in a societal context (focusing on one’s community – whether it be local,
national or global), an occupational context (centred on the world of work), or a scientific context (relating
to the application of mathematics to the natural and technological world).
Included for the first time in the PISA 2022 framework (and depicted in Figure 2.2) are selected 21 st
Century skills that mathematical literacy both relies on and develops. 21 st Century skills are discussed in
greater detail in the next section of this framework. For now, it should be stressed that while contexts
(personal, societal, occupational and scientific) influence the development of test items, there is no
expectation that items will be deliberately developed to incorporate or address 21 st Century skills.
Instead, the expectation is that by responding to the spirit of the framework and in line with the definition
of mathematical literacy, the 21st Century skills that have been identified will be incorporated in the items.
The language of the definition and the representation in Figure 2.1 and Figure 2.2 retain and integrate the
notion of mathematical modelling, which has historically been a cornerstone of the PISA framework for
mathematics e.g. (OECD, 2004[6]; OECD, 2013[7]). The modelling cycle (formulate, employ, interpret and
evaluate) is a central aspect of the PISA conception of mathematically literate students; however, it is often not
necessary to engage in every stage of the modelling cycle, especially in the context of an assessment
(Galbraith, Henn and Niss, 2007[22]). It is often the case that significant parts of the mathematical modelling
cycle have been undertaken by others, and the end user carries out some of the steps of the modelling cycle,
but not all of them. For example, in some cases, mathematical representations, such as graphs or equations,
are given that can be directly manipulated in order to answer some question or to draw some conclusion. In
other cases, students may be using a computer simulation to explore the impact of variable change in a
system or environment. For this reason, many PISA items involve only parts of the modelling cycle. In reality,
the problem solver may also sometimes oscillate between the processes, returning to revisit earlier decisions
and assumptions. Each of the processes may present considerable challenges, and several iterations around
the whole cycle may be required.
In particular, the verbs ‘formulate’, ‘employ’ and ‘interpret’ point to the three processes in which students
as active problem solvers will engage. Formulating situations mathematically involves applying
mathematical reasoning (both deductive and inductive) in identifying opportunities to apply and use
mathematics – seeing that mathematics can be applied to understand or resolve a particular problem or
challenge presented. It includes being able to take a situation as presented and transform it into a form
amenable to mathematical treatment, providing mathematical structure and representations, identifying
variables and making simplifying assumptions to help solve the problem or meet the challenge.
Employing mathematics involves applying mathematical reasoning while using mathematical concepts,
procedures, facts and tools to derive a mathematical solution. It includes performing calculations,
manipulating algebraic expressions and equations or other mathematical models, analysing information
in a mathematical manner from mathematical diagrams and graphs, developing mathematical
descriptions and explanations and using mathematical tools to solve problems. Interpreting mathematics
involves reflecting upon mathematical solutions or results and interpreting them in the context of a
problem or challenge. It involves applying mathematical reasoning to evaluate mathematical solutions in
relation to the context of the problem and determining whether the results are reasonable and make
sense in the situation; determining also what to highlight when explaining the solution.
Included for the first time in the PISA 2022 framework is an appreciation of the intersection between
mathematical and computational thinking engendering a similar set of perspectives, thought processes
and mental models that learners need to succeed in an increasingly technological world. A set of

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

constituent practices positioned under the computational thinking umbrella (namely abstraction,
algorithmic thinking, automation, decomposition and generalisation) are also central to both
mathematical reasoning and problem-solving processes. The nature of computational thinking within
mathematics is conceptualised as defining and elaborating mathematical knowledge that can be
expressed by programming, allowing students to dynamically model mathematical concepts and
relationships. A taxonomy of computational thinking practices geared specifically towards mathematics
and science learning entails data practices, modelling and simulation practices, computational problem-
solving practices, and systems thinking practices (Weintrop et al., 2016[17]) . The combination of
mathematical and computational thinking not only becomes essential to effectively support the
development of students’ conceptual understanding of the mathematical domain, but also to develop
their computational thinking concepts and skills, giving learners a more realistic view of how mathematics
is practiced in the professional world and used in the real-world and, in turn, better prepares them for
pursuing careers in related fields (Basu et al., 2016[23]; Benton et al., 2017[24]; Pei, Weintrop and
Wilensky, 2018[16]; Beheshti et al., 2017[25]).

An explicit link to a variety of contexts for problems in PISA 2022

The reference to ‘a variety of real-world contexts’ in the definition of mathematical literacy recognises that the
21st Century citizen is a consumer of quantitative, sometimes statistical, arguments. The reference is intended
as a way to link to the specific contexts that are described and exemplified more fully later in this framework.
The specific contexts themselves are not so important, but the four categories selected for use here (personal,
occupational, societal and scientific) reflect a wide range of situations in which individuals may meet
mathematical opportunities. The definition also acknowledges that mathematical literacy helps individuals to
recognise the role that mathematics plays in the world and to make the kinds of well-founded judgements and
decisions required of constructive, engaged and reflective citizens faced with messages and arguments of the
form: "a study found that on average...", "a survey shows a big drop in....", “certain
scientists claim that population growth will outpace food production in x years …” etc.

A visible role for mathematical tools, including technology, in PISA 2022

The definition of mathematical literacy explicitly includes the use of mathematical tools. These tools
include a variety of physical and digital equipment, software and calculation devices. Computer-based
mathematical tools are in common use in workplaces of the 21 st century, and will be increasingly more
prevalent as the century progresses both in the workplace and in society generally. The nature of day-to-
day and work-related problems and the demands on individuals to be able to employ mathematical
reasoning (both deductive and inductive) in situations where computational tools are present has
expanded with these new opportunities – creating enhanced expectations for mathematical literacy.
Since the 2015 cycle, computer-based assessment (CBA) has been the primary mode of testing,
although an equivalent paper-based instrument is available for those countries who chose not to test
their students by computer. The 2015 and 2018 mathematical literacy assessments did not exploit the
opportunities that the computer provides.
Computer-Based Assessment of Mathematics (CBAM) will be the format of the mathematical literacy
from 2022. Although the option of a paper-based assessment will remain for countries who want to
continue in that way, the CBAM will exploit the opportunities of the CBAM. The opportunities that this
transition creates are discussed in greater detail later in the framework.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 27

Organisation of the domain

The PISA mathematics framework defines the domain of mathematics for the PISA survey and describes
an approach to the assessment of the mathematical literacy of 15-year-olds. That is, PISA assesses the
extent to which 15-year-old students can reason mathematically and handle mathematics adeptly when
confronted with situations and problems – the majority of which are presented in real-world contexts.
For purposes of the assessment, the PISA 2022 definition of mathematical literacy can be analysed in
terms of three interrelated aspects (see Figure 2.2):
mathematical reasoning (both deductive and inductive) and problem solving (which includes the
mathematical processes that describe what individuals do to connect the context of the problem
with mathematics and thus solve the problem);
the mathematical content that is targeted for use in the assessment items;
2
the contexts in which the assessment items are located coupled with selected 21st Century skills
that support and are developed by mathematical literacy.
The following sections elaborate these aspects to support understanding and to provide guidance to the
test developers. In highlighting these aspects of the domain, the PISA 2022 Mathematics Framework
helps to ensure that assessment items developed for the survey reflect a range of mathematical
reasoning and problem solving, content, and contexts and 21 st Century skills, so that, considered as a
whole, the set of assessment items effectively operationalises what this framework defines as
mathematical literacy. Several questions, based on the PISA 2022 definition of mathematical literacy lie
behind the organisation of this section of the framework. They are:
What do individuals engage in when reasoning mathematically and solving contextual mathematical
problems?
What mathematical content knowledge can we expect of individuals – and of 15-year-old students in
particular?
In what context is mathematical literacy able to be both observed and assessed and how do these
interact with the identified 21st Century skills?

Mathematical reasoning and problem-solving processes

Mathematical reasoning

Mathematical reasoning (both deductive and inductive) involves evaluating situations, selecting
strategies, drawing logical conclusions, developing and describing solutions, and recognising how those
solutions can be applied. Students reason mathematically when they:
identify, recognise, organise, connect, and represent;
construct, abstract, evaluate, deduce, justify, explain, and defend;
interpret, make judgements, critique, refute, and qualify.
The ability to reason logically and to present arguments in honest and convincing ways is a skill that is
becoming increasingly important in today’s world. Mathematics is a science about well -defined objects
and notions which can be analysed and transformed in different ways using ‘mathematical reasoning’ to
obtain conclusions about which we are certain. Through mathematics, students learn that using
appropriate reasoning they can reach results and conclusions which they can trust to be true. Further,
those conclusions are logical and objective, and hence impartial, without any need for validation by an
external authority. This kind of reasoning which is useful far beyond mathematics, can be learned and
practiced most effectively within mathematics.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

Two aspects of mathematical reasoning are especially important in today’s world and in defining the PISA
items. One is deduction from clear assumptions (deductive reasoning), which is a characteristic feature of
mathematical process. The usefulness of this ability has already been stressed. The second important
dimension is statistical and probabilistic (inductive) reasoning. At the logical level, there is these days frequent
confusion in the minds of individuals between the possible and the probable, leading many to fall prey to
conspiracy theories or fake news. From a technical perspective, today’s world is increasingly complex and its
multiple dimensions are represented by terabytes of data. Making sense of these data is one of the biggest
challenges that humanity will face in the future. Our students should be familiarised with the nature of such
data and making informed decisions in the context of variation and uncertainty.
Mathematical reasoning (both deductive and inductive), enabled by some key understandings that
undergird school mathematics, is the core of mathematical literacy. Included among these key
understandings are:
understanding quantity, number systems and their algebraic properties;
appreciating the power of abstraction and symbolic representation;
seeing mathematical structures and their regularities;
recognising functional relationships between quantities;
using mathematical modelling as a lens onto the real world (e.g. those arising in the physical,
biological, social, economic, and behavioural sciences);
understanding variation as the heart of statistics.
The description of each of these that follows provides an overview of the understanding and how it
supports reasoning. While the descriptions may appear abstract, the intention is not for them to be
treated in an abstract way in the PISA assessment. The message that the descriptions should convey is
how these ideas surface throughout school mathematics and how, by reinforcing their occurrence in
teaching we support students to realise how they can be applied in new and different contexts.

Understanding quantity, number systems and their algebraic properties

The basic notion of quantity may be the most pervasive and essential mathematical aspect of engaging with,
and functioning in, the world (OECD, 2017, p. 18 [26]). At the most basic level it deals with the useful ability to
compare cardinalities of sets of objects. The ability to count usually involves rather small sets – in most
languages, only a small subset of numbers have names. When we assess larger sets, we engage in more
complex operations of estimating, rounding and applying orders of magnitude. Counting is very closely related
to another fundamental operation of classifying things, where the ordinal aspect of numbers emerges.
Quantification of attributes of objects (measurement), relationships, situations and entities in the world is one
of the most basic ways of conceptualising the surrounding world (OECD, 2017[26]).
Understanding quantity, number systems and their algebraic properties includes the basic concept of
number, nested number systems (e.g., whole numbers to integers to rationals to reals), the arithmetic of
numbers, and the algebraic properties that the systems enjoy. In particular, it is useful to understand how
progressively more expansive systems of numbers enable the solution of progressively more
complicated equations. This lays the foundation for enabling students to see more evidence of
mathematics in the real world in as they learn more mathematics.
To use quantification efficiently, one has to be able to apply not just numbers, but the number systems.
Numbers themselves are of limited relevance; what makes them into a powerful tool are the operations
that we can perform with them. As such, a good understanding of the operations of numbers is the
foundation of mathematical reasoning.
It is also important to understand matters of representation (as symbols involving numerals, as points on
a number line, as geometric quantities, and by special symbols such as π) and how to move between

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 29

them; the ways in which these representations are affected by number systems; the ways in which
algebraic properties of these systems are relevant and matter for operating within the systems; and the
significance of the additive and multiplicative identities, associativity, commutativity, and the distributive
property of multiplication over addition. Algebraic principles undergird the place value system, allowing
for economical expression of numbers and efficient approaches to operations on them. They are also
central to number-line based operations with numbers, including work with additive inverses that are
central to addition and subtraction of first integers, then rationals and finally reals.
The centrality of number as a key concept in all the other mathematical areas under consideration here
and to mathematical reasoning itself, is undeniable. Students’ grasp of the algebraic principles and
properties first experienced through work with numbers is fundamental to their understanding of the
concepts of secondary school algebra, along with their ability to become fluent in the manipulations of
algebraic expressions necessary for solving equations, setting up models, graphing functions, and
programming and making spreadsheet formulas. And in today’s data-intensive world, facility with
interpretation of patterns of numbers, comparison of patterns, and other numerical skills are evolving in
importance.
A broad understanding of quantity and number systems supports reasoning in the real-world applications
of mathematics envisaged by this framework.

Appreciating the power of abstraction and symbolic representation

The fundamental ideas of mathematics have arisen from human experience in the world and the need to
provide coherence, order, and predictability to that experience. Many mathematical objects model reality,
or at least reflect aspects of reality in some way. However, the essence of abstraction in mathematics is
that it is a self-contained system, and mathematical objects derive their meaning from within that system.
Abstraction involves deliberately and selectively attending to structural similarities between mathematical
objects, and constructing relationships between those objects based on these similarities. In school
mathematics, abstraction forms relationships between concrete objects, symbolic representations and
operations including algorithms and mental models. This ability also plays a role in working with
computational devices. The ability to create, manipulate, and draw meaning in working with abstractions
in technological contexts in an important computational thinking skill.
For example, children begin to develop the concept of “circle” by experiencing specific objects that lead
them to an informal understanding of circles as being “roundish”. They might draw circles to represent
these objects, noticing similarities between the drawings to generalise about “roundness” even though
the circles are of different sizes. “Circle” becomes an abstract mathematical object when students start to
“use” circles as objects in their work and more formally when it is defined as the locus of points
equidistant from a fixed point in a two-dimensional plane.
Students use representations – whether text-based, symbolic, graphical, numerical, geometric or in
programming code – to organise and communicate their mathematical thinking. Representations enable
us to present mathematical ideas in a succinct way which, in turn, lead to efficient algorithms.
Representations are also a core element of mathematical modelling, allowing students to abstract a
simplified or idealised formulation of a real-world problem. Such structures are also important for
interpreting and defining the behaviour of computational devices.
Having an appreciation of abstraction and symbolic representation supports reasoning in the real-world
applications of mathematics envisaged by this framework by allowing students to move from the specific
details of a situation to the more general features and to describe these in an efficient way.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 Seeing mathematical structures and their regularities

When elementary students see: 5 + (3 + 8) some see a string of symbols indicating a computation to be
performed in a certain order according to the rules of order of operations; others see a number added to
the sum of two other numbers. The latter group are seeing structure; and because of that they don’t need
to be told about the order of operation, because if you want to add a number to a sum you first have to
compute the sum.
Seeing structure continues to be important as students move to higher grades. A student who sees f( ) =
5 + ( − 3)2 as saying that f(x) is the sum of 5 and a square which is zero when x = 3 understands that the
minimum of f is 5. This lays the foundation for functional thinking discussed in the next section.
Structure is intimately related to symbolic representation. The use of symbols is powerful, but only if they
retain meaning for the symboliser, rather than becoming meaningless objects to be rearranged on a
page. Seeing structure is a way of finding and remembering the meaning of an abstract representation.
Such structures are also important for interpreting and defining the behaviour of computational devices.
Being able to see structure is an important conceptual aid to procedural knowledge.
The examples above illustrate how seeing structure in abstract mathematical objects is a way of
replacing parsing rules, which can be performed by a computer, with conceptual images of those objects
that make their properties clear. An object held in the mind in such a way is subject to reasoning at a
level that is higher than simple symbolic manipulation.
A robust sense of mathematical structure also supports modelling. When the objects under study are not
abstract mathematical objects, but rather objects from the real world to be modelled by mathematics,
then mathematical structure can guide the modelling. Students can also impose structure on non -
mathematical objects in order to make them subject to mathematical analysis. An irregular shape can be
approximated by simpler shapes whose area is known. A geometric pattern can be understood by
hypothesising translational, rotational, or reflectional transformations and symmetry and abstractly
extending the pattern into all of space. Statistical analysis is often a matter of imposing a structure on a
set of data, for example by assuming it comes from a normal distribution or supposing that one variable
is a linear function of another, but measured with normally distributed error.
Being able to see mathematical structures supports reasoning in the real-world applications of
mathematics envisaged by this framework by allowing students to apply knowledge about situations or
problems in one context to problems in another context that share a similar structure.

Recognising functional relationships between quantities

Students in elementary school encounter problems where they must find specific quantities. For
example, how fast do you have to drive to get from Tucson to Phoenix, a distance of 180 km, in 1 hour
and 40 minutes? Such problems have a specific answer: to drive 180 km in 1 hour and 40 minutes you
must drive at 108 km per hour.
At some point students start to consider situations where quantities are variable, that is, where they can
take on a range of values. For example, what is the relation between the distance driven, d, in
kilometres, and time spent driving, t, in hours, if you drive at a constant speed of 108 km per hour? Such
questions introduce functional relationships. In this case the relationship, expressed by the equation d =
108t, is a proportional relationship, the fundamental example and perhaps the most important for general
knowledge.
Relationships between quantities can be expressed with equations, graphs, tables, or verbal descriptions. An
important step in learning is to extract from these the notion of a function itself, as an abstract object of which
these are representations. The essential elements of the concept are a domain, from which inputs are
selected, a codomain, in which outputs lie, and a process for producing outputs from inputs.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 31

Recognising the functional relationships between the variables in the real-world applications of
mathematics envisaged by this framework supports reasoning by allowing students to focus on how the
interdependence of and interaction between the variables impacts on the situation.

Using mathematical modelling as a lens onto the real world

Models represent a conceptualisation of phenomena. Models are simplifications of reality that fore-
ground certain features of a phenomenon while approximating or ignoring other features. As such, ‘‘all
models are wrong, but some are useful’’ (Box and Draper, 1987, p. 424 [27]). The usefulness of a model
comes from its explanatory and/or predictive power (Weintrop et al., 2016 [17]). Models are, in that sense,
abstractions of reality. A model may present a conceptualisation that is understood to be an
approximation or working hypothesis concerning the object phenomenon or it may be an intentional
simplification. Mathematical models are formulated in mathematical language and use a wide variety of
mathematical tools and results (e.g., from arithmetic, algebra, geometry, etc.). As such, they are used as
ways of precisely defining the conceptualisation or theory of a phenomenon, for analysing and evaluating
data (does the model fit the data?), and for making predictions. Models can be operated – that is, made
to run over time or with varying inputs, thus producing a simulation. When this is done, it is possible to
make predictions, study consequences, and evaluate the adequacy and accuracy of the models.
Throughout the modelling process cognisance needs to be taken of the real-world parameters that
impact on the model and the solutions developed using the model.
Computer-based (or computational) models provide the ability to test hypothesis, generate data,
introduce randomness and so on. Mathematical literacy includes the ability to understand, evaluate and
draw meaning from computational models.
Using models in general and mathematical models in particular supports reasoning about the real- world
applications of mathematics envisaged in this framework by encouraging students to focus on the most
significant elements of the situations and in so doing to reduce the problem to its essence.

Understanding variation as the heart of statistics

In statistics accounting for variability is one, if not the central, defining element around which the
discipline is based. In today’s world people often deal with these types of situations by merely ignoring
the variation and as a result suggesting sweeping generalisations which are often misleading, if not
wrong, and as a result very dangerous. Bias in the social science sense is usually created by not
accounting for the sources and magnitudes of the variability in the trait under discussion.
Statistics is essentially about accounting for or modelling variation as measured by the variance or in the
case of multiple variables the covariance matrix. This provides a probabilistic environment in which to
understand various phenomena as well as to make critical decisions. Statistics is in many ways a search
for patterns in a highly variable context: trying to find the single defining “truth” in the midst of a great deal
of random noise. “Truth” is set in quotes as it is not the nature of truth that mathematics can deliver but
an estimate of truth set in a probabilistic context, accompanied by an estimate of the error contained in
the process. Ultimately, the decision maker is left with the dilemma of never knowing for certain what the
truth is. The estimate that has been developed is, at best a range of possible values – the better the
process, for example, the larger the sample of data, the narrower the range of possible values, although
a range cannot be avoided. Some aspects of this have been present in previous PISA cycles, the
growing significance contributes to the increased stress in this framework.
Understanding variation as a central feature of statistics supports reasoning about the real-world
applications of mathematics envisaged in this framework in that students are encouraged to engage with
data-based arguments with awareness of the limitations of the conclusions that can be drawn.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 Problem solving

The definition of mathematical literacy refers to an individual’s capacity to formulate, employ, and
interpret (and evaluate) mathematics. These three words, formulate, employ and interpret, provide a
useful and meaningful structure for organising the mathematical processes that describe what individuals
do to connect the context of a problem with the mathematics and solve the problem. Items in the 2022
PISA mathematics test will be assigned to either mathematical reasoning or one of three mathematical
processes:
formulating situations mathematically;
employing mathematical concepts, facts and procedures;
interpreting, applying and evaluating mathematical outcomes.
It is important for both policy makers and those engaged more closely in the day-to-day education of
students to know how effectively students are able to engage in each of these elements of the problem
solving model/cycle. Formulating indicates how effectively students are able to recognise and identify
opportunities to use mathematics in problem situations and then provide the necessary mathematical
structure needed to formulate that contextualised problem in a mathematical form. Employing refers to
how well students are able to perform computations and manipulations and apply the concepts and facts
that they know to arrive at a mathematical solution to a problem formulated mathematically. Interpreting
(and evaluating) relates to how effectively students are able to reflect upon mathematical solutions or
conclusions, interpret them in the context of the real-world problem and determine whether the result(s)
or conclusion(s) are reasonable and/or useful. Students’ facility at applying mathematics to problems and
situations is dependent on skills inherent in all three of these stages, and an understanding of students’
effectiveness in each category can help inform both policy-level discussions and decisions being made
closer to the classroom level.
Moreover, encouraging students to experience mathematical problem-solving processes through
computational thinking tools and practices encourage students to practice prediction, reflection and
debugging skills (Brennan and Resnick, 2012[28]).

Formulating situations mathematically

The word formulate in the mathematical literacy definition refers to individuals being able to recognise
and identify opportunities to use mathematics and then provide mathematical structure to a problem
presented in some contextualised form. In the process of formulating situations mathematically,
individuals determine where they can extract the essential mathematics to analyse, set up and solve the
problem. They translate from a real-world setting to the domain of mathematics and provide the real-
world problem with mathematical structure, representations and specificity. They reason about and make
sense of constraints and assumptions in the problem. Specifically, this process of formulating situations
mathematically includes activities such as the following:
3
selecting an appropriate model from a list;
identifying the mathematical aspects of a problem situated in a real-world context and identifying the
significant variables;
recognising mathematical structure (including regularities, relationships, and patterns) in problems or
situations;
simplifying a situation or problem in order to make it amenable to mathematical analysis (for example
by decomposing);
identifying constraints and assumptions behind any mathematical modelling and simplifications
gleaned from the context;

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 33

representing a situation mathematically, using appropriate variables, symbols, diagrams, and

standard models;
representing a problem in a different way, including organising it according to mathematical concepts
and making appropriate assumptions;
understanding and explaining the relationships between the context-specific language of a problem
and the symbolic and formal language needed to represent it mathematically;
translating a problem into mathematical language or a representation;
recognising aspects of a problem that correspond with known problems or mathematical concepts,
facts or procedures;
choosing among an array of and employing the most effective computing tool to portray a
mathematical relationship inherent in a contextualised problem;
creating an ordered series of (step-by-step) instructions for solving problems.

Employing mathematical concepts, facts and procedures

The word employ in the mathematical literacy definition refers to individuals being able to apply
mathematical concepts, facts, procedures, and reasoning to solve mathematically formulated problems
to obtain mathematical conclusions. In the process of employing mathematical concepts, facts and
procedures to solve problems, individuals perform the mathematical procedures needed to derive results
and find a mathematical solution (e.g. performing arithmetic computations, solving equations, making
logical deductions from mathematical assumptions, performing symbolic manipulations, extracting
mathematical information from tables and graphs, representing and manipulating shapes in space, and
analysing data). They work on a model of the problem situation, establish regularities, identify
connections between mathematical entities, and create mathematical arguments. Specifically, this
process of employing mathematical concepts, facts and procedures includes activities such as:
4
performing a simple calculation; **
drawing a simple conclusion; **
selecting an appropriate strategy from a list; **
devising and implementing strategies for finding mathematical solutions;
using mathematical tools, including technology, to help find exact or approximate solutions;
applying mathematical facts, rules, algorithms, and structures when finding solutions;
manipulating numbers, graphical and statistical data and information, algebraic expressions and
equations, and geometric representations;
making mathematical diagrams, graphs, simulations, and constructions and extracting mathematical
information from them;
using and switching between different representations in the process of finding solutions;
making generalisations and conjectures based on the results of applying mathematical procedures to
find solutions;
reflecting on mathematical arguments and explaining and justifying mathematical results;
evaluating the significance of observed (or proposed) patterns and regularities in data.

Interpreting, applying and evaluating mathematical outcomes

The word interpret (and evaluate) used in the mathematical literacy definition focuses on the ability of
individuals to reflect upon mathematical solutions, results or conclusions and interpret them in the context of
the real-life problem that initiated the process. This involves translating mathematical solutions or reasoning
back into the context of the problem and determining whether the results are reasonable and

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

make sense in the context of the problem. Interpreting, applying and evaluating mathematical outcomes
encompasses both the ‘interpret’ and ‘evaluate’ elements of the mathematical modelling cycle. Individuals
engaged in this process may be called upon to construct and communicate explanations and arguments in the
context of the problem, reflecting on both the modelling process and its results. Specifically, this process of
interpreting, applying and evaluating mathematical outcomes includes activities such as:
5
interpreting information presented in graphical form and/or diagrams;**
evaluating a mathematical outcome in terms of the context;**
interpreting a mathematical result back into the real-world context;
evaluating the reasonableness of a mathematical solution in the context of a real-world problem;
understanding how the real world impacts the outcomes and calculations of a mathematical
procedure or model in order to make contextual judgements about how the results should be
adjusted or applied;
explaining why a mathematical result or conclusion does, or does not, make sense given the context
of a problem;
understanding the extent and limits of mathematical concepts and mathematical solutions;
critiquing and identifying the limits of the model used to solve a problem;
using mathematical thinking and computational thinking to make predictions, to provide evidence for
arguments, to test and compare proposed solutions.

Mathematical content knowledge

An understanding of mathematical content – and the ability to apply that knowledge to solving meaningful
contextualised problems – is important for citizens in the modern world. That is, to reason mathematically
and to solve problems and interpret situations in personal, occupational, societal and scientific contexts,
there is a need to draw upon certain mathematical knowledge and understanding.
Since the goal of PISA is to assess mathematical literacy, an organisational structure for mathematical
content knowledge is proposed that is based on mathematical phenomena that underlie broad classes of
problems. Such an organisation for content is not new, as exemplified by two well-known publications:
On the Shoulders of Giants: New Approaches to Numeracy (Steen, 1990[29]) and Mathematics: The
Science of Patterns (Devlin, 1994[30]).
The following content categories (previously used in 2012) are again used in PISA 2022 to reflect both
the mathematical phenomena that underlie broad classes of problems, the general structure of
mathematics, and the major strands of typical school curricula. These four categories characterise the
range of mathematical content that is central to the discipline and illustrate the broad areas of content
used in the test items for PISA 2022 (which will include PISA-D items to increase opportunities at the
lower end of the performance spectrum):
change and relationships;
space and shape;
quantity;
uncertainty and data.
With these four categories, the mathematical domain can be organised in a way that ensures a spread of
items across the domain and focuses on important mathematical phenomena, while at the same time,
avoiding too granular a classification that would prevent the analysis of rich and challenging
mathematical problems based on real situations.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 35

While categorisation by content category is important for item development, selection and reporting of
the assessment results, it is important to note that some items could potentially be classified in more than
one content category.
National school mathematics curricula are typically organised around content strands (most commonly:
numbers, algebra, functions, geometry, and data handling) and detailed topic lists help to define clear
expectations. These curricula are designed to equip students with knowledge and skills that address
these same underlying mathematical phenomena that organise the PISA content. The outcome is that
the range of content arising from organising it in the way that PISA does is closely aligned with the
content that is typically found in national mathematics curricula. This framework lists a range of content
topics appropriate for assessing the mathematical literacy of 15-year-old students, based on analyses of
national standards from eleven countries.
The broad mathematical content categories and the more specific content topics appropriate for 15-year-
old students described in this section reflect the level and breadth of content that is eligible for inclusion
in the PISA 2022 assessment. Descriptions of each content category and the relevance of each to
reasoning and solving meaningful problems are provided, followed by more specific definitions of the
kinds of content that are appropriate for inclusion in an assessment of mathematical literacy of 15-year-
old students and out-of-school youth.
Four topics have been identified for special emphasis in the PISA 2022 assessment. These topics are
not new to the mathematics content categories. Instead, these are topics within the existing content
categories that deserve special emphasis. In the work of Mahajan et al. (“PISA Mathematics 2022”,
(2016[31])) the four topics are presented not only as commonly encountered situations in adult life in
general, but as the types of mathematics needed in the emerging new areas of the economy such as
high-tech manufacturing etc. The four are: growth phenomena; geometric approximations; computer
simulations; and conditional decision making. These topics should be approached in the test items in a
way that is consistent with the experiences of 15-year-olds. Each topic is discussed with the discussion
of the corresponding content category as follows:
growth phenomena (change and relationships);
geometric approximation (space and shape);
computer simulations (quantity);
conditional decision making (uncertainty and data).

Change and relationships

The natural and designed worlds display a multitude of temporary and permanent relationships among
objects and circumstances, where changes occur within systems of interrelated objects or in
circumstances where the elements influence one another. In many cases these changes occur over time,
and in other cases changes in one object or quantity are related to changes in another. Some of these
situations involve discrete change; others change continuously. Some relationships are of a permanent,
or invariant, nature. Being more literate about change and relationships involves understanding
fundamental types of change and recognising when they occur in order to use suitable mathematical
models to describe and predict change. Mathematically this means modelling the change and the
relationships with appropriate functions and equations, as well as creating, interpreting and translating
among symbolic and graphical representations of relationships.
Change and relationships is evident in such diverse settings as growth of organisms, music, seasonal
change and cycles, weather patterns, employment levels and economic conditions. Aspects of the
traditional mathematical content of functions and algebra, including algebraic expressions, equations and
inequalities, tabular and graphical representations, are central in describing, modelling and interpreting
change phenomena. Computational tools provide a means to visualise and interact with change and

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

relationships. Recognising how and when a computational device can augment and complement
mathematical concepts is an important computational thinking skill.
Representations of data and relationships described using statistics are also used to portray and
interpret change and relationships, and a firm grounding in the basics of number and units is also
essential to defining and interpreting change and relationships. Some interesting relationships arise from
geometric measurement, such as the way that changes in perimeter of a family of shapes might relate to
changes in area, or the relationships among lengths of the sides of triangles.
Growth phenomena: Understanding the dangers of flu pandemics and bacterial outbreaks, as well as the
threat of climate change, demand that people think not only in terms of linear relationships but recognise
that such phenomena need non-linear (often exponential but also other) models. Linear relationships are
common and are easy to recognise and understand but to assume linearity can be dangerous. A good
example of linearity and one probably used by everyone is estimating the distance travelled in various
amounts of time while travelling at a given speed. Such an application provides a reasonable estimate as
long as the speed stays relatively constant. But with flu epidemics, for example, such a linear approach
would grossly underestimate the number of people sick in 5 days after the initial outbreak. Here is where
a basic understanding of non-linear (including quadratic and exponential) growth and how rapidly
infections can spread given that the rate of change increases from day to day is critical. The spread of
the Zika infection is an important example of exponential growth; recognising it as such helped medical
personnel to understand the inherent threat and the need for fast action.
Identifying growth phenomena as a focal point of the change and relationships content category is not to
signal that there is an expectation that participating students should have studied the exponential
function and certainly the items will not require knowledge of the exponential function. Instead, the
expectation is that there will be items that expect students to (a) recognise that not all growth is linear, (b)
that non-linear growth has particular and profound implications on how we understand certain situations,
and (c) appreciate the intuitive meaning of “exponential growth” as an extremely rapid rate of growth, for
example in the earthquake scale, every increase by 1 unit on the Richter scale does not mean a
proportional increase in its effect, but rather by 10, 100, and 1000 times etc.

Space and shape

The topic of space and shape encompasses a wide range of phenomena that are encountered
everywhere in our visual and physical world: patterns, properties of objects, positions and orientations,
representations of objects, decoding and encoding of visual information, navigation and dynamic
interaction with real shapes as well as with representations, movement, displacement, and the ability to
anticipate actions in space. Geometry serves as an essential foundation for space and shape, but the
category extends beyond traditional geometry in content, meaning and method, drawing on elements of
other mathematical areas such as spatial visualisation, measurement and algebra. For instance, shapes
can change and a point can move along a locus, thus requiring function concepts. Measurement
formulas are central in this area. The recognition, manipulation and interpretation of shapes in settings
that call for tools ranging from dynamic geometry software to Global Positioning Systems (GPS), and to
machine learning software are included in this content category.
PISA assumes that the understanding of a set of core concepts and skills is important to mathematical
literacy relative to space and shape. Mathematical literacy in the area of space and shape involves a
range of activities such as understanding perspective (for example in paintings), creating and reading
maps, transforming shapes with and without technology, interpreting views of three-dimensional scenes
from various perspectives and constructing representations of shapes.
Geometric approximations: Today’s world is full of shapes that do not follow typical patterns of evenness
or symmetry. Because simple formulas do not deal with irregularity, it has become more difficult to
understand what we see and find the area or volume of the resulting structures. For example, finding the

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 37

needed amount of carpeting in a building in which the apartments have acute angles together with
narrow curves demands a different approach than would be the case with a typically rectangular room.
Identifying geometric approximations as a focal point of the space and shape content category signals
the need for students to be able use their understanding of traditional space and shape phenomena in a
range of typical situations.

Quantity

The notion of quantity may be the most pervasive and essential mathematical aspect of engaging with,
and functioning in, our world. It incorporates the quantification of attributes of objects, relationships,
situations and entities in the world, understanding various representations of those quantifications and
judging interpretations and arguments based on quantity. To engage with the quantification of the world
involves understanding measurements, counts, magnitudes, units, indicators, relative size and numerical
trends and patterns. Aspects of quantitative reasoning – such as number sense, multiple representations
of numbers, elegance in computation, mental calculation, estimation and assessment of reasonableness
of results – are the essence of mathematical literacy relative to quantity.
Quantification is a primary method for describing and measuring a vast set of attributes of aspects of the
world. It allows for the modelling of situations, for the examination of change and relationships, for the
description and manipulation of space and shape, for organising and interpreting data and for the
measurement and assessment of uncertainty. Thus, mathematical literacy in the area of quantity applies
knowledge of number and number operations in a wide variety of settings.
Computer simulations: Both in mathematics and statistics there are problems that are not so easily addressed
because the required mathematics are complex or involve a large number of factors all operating in the same
system or because of ethical issues relating to the impact on living beings or their environment. Increasingly in
today’s world such problems are being approached using computer simulations driven by algorithms. In the
illustrative example savings simulation the student uses a computer simulation as a tool in decision making.
The computer simulation does the calculations for the student, leaving the student to plan, predict and solve
problems based on the variables that they can control.
Identifying computer simulations as a focal point of the quantity content category signals that in the
context the Computer-Based Assessment of Mathematics (CBAM) of PISA being used from 2022, there
are a broad category of complex problems including budgeting and planning that students can analyse in
terms of the variables of the problem using computer simulations provided as part of the test item.

Uncertainty and data

In science, technology and everyday life, variation and its associated uncertainty is a given. It is a
phenomenon at the heart of the theory of probability and statistics. The uncertainty and data content
category includes recognising the place of variation in the real world including, having a sense of the
quantification of that variation, and acknowledging its uncertainty and error in related inferences. It also
includes forming, interpreting and evaluating conclusions drawn in situations where uncertainty is
present. The presentation and interpretation of data are key concepts in this category (Moore, 1997[32]).
Economic predictions, poll results, and weather forecasts all include measures of variation and
uncertainty. There is variation in manufacturing processes, test scores and survey findings, and chance
is fundamental to many recreational activities enjoyed by individuals. The traditional curricular areas of
probability and statistics provide formal means of describing, modelling and interpreting a certain class of
phenomena in which variation plays a central role, and for making corresponding stochastic inferences.
In addition, knowledge of number and of aspects of algebra such as graphs and symbolic representation
contribute to engaging in problem solving in this content category.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

Conditional decision making: statistics provides a measure of the variation characteristic of much of what
people encounter in their daily lives. That measure is the variance. When there is more than one
variable, there is variation in each of the variables as well as co-variation characterising the relationships
among the variables. These inter-relationships can often be represented in two-way tables that provide
the basis for making conditional decisions (inferences). In a two-way table for two dichotomous variables
(i.e. two variables with two possibilities each), there are four combinations. The two-way table (analysis
of the situation) provides three types of percentages which, in turn, provide estimates of the
corresponding probabilities. These include the probabilities of the four joint events, the two marginal, and
the conditional probabilities which play the central role in what we have termed conditional decision
making. The expectation for the PISA test items is that students will be able to read the relevant data
from the table with a deep understanding for the meaning of the data that they are extracting.
In the illustrative example Purchasing Decision the student is presented with a summary of customer
ratings for a product in an online store. Additionally, the student is provided with more a more detailed
analysis of the reviews by the customers who provided 1- and 2-star ratings. This effect sets up a two-
way table and the student is asked to demonstrate an understanding of the different probability estimates
that the two-way table provides
Identifying conditional decisions making as a focal point of the uncertainty and data content category
signals that students should be expected to appreciate how the formulation of the analysis in a model
impacts the conclusions that can be dawn and that different assumptions/relationships may well result in
different conclusions.

Content topics for guiding the assessment of mathematical literacy of 15-year-old

students
To effectively understand and solve contextualised problems involving change and relationships; space
and shape; quantity; and uncertainty and data requires drawing upon a variety of mathematical concepts,
procedures, facts, and tools at an appropriate level of depth and sophistication. As an assessment of
mathematical literacy, PISA strives to assess the levels and types of mathematics that are appropriate
for 15-year-old students on a trajectory to become constructive, engaged and reflective 21st Century
citizens able to make well-founded judgements and decisions. It is also the case that PISA, while not
designed or intended to be a curriculum- driven assessment, strives to reflect the mathematics that
students have likely had the opportunity to learn by the time they are 15 years old.
In the development of the PISA 2012 mathematical literacy framework, with an eye toward developing an
assessment that is both forward-thinking yet reflective of the mathematics that 15-year-old students have
likely had the opportunity to learn, analyses were conducted of a sample of desired learning outcomes
from eleven countries to determine both what is being taught to students in classrooms around the world
and what countries deem realistic and important preparation for students as they approach entry into the
workplace or admission into a higher education institution. Based on commonalities identified in these
analyses, coupled with the judgement of mathematics experts, content deemed appropriate for inclusion
in the assessment of mathematical literacy of 15-year-old students on PISA 2012, and continued for
PISA 2022, is described below.
For PISA 2022, four additional focus topics have been added to the list. The resulting lists is intended to
be illustrative of the content topics included in PISA 2022 and not an exhaustive listing:
Growth phenomena: Different types of linear and non-linear growth.
Geometric approximation: Approximating the attributes and properties of irregular or unfamiliar
shapes and objects by breaking these shapes and objects up into more familiar shapes and
objects for which there are formulae and tools.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 39

Computer simulations: Exploring situations (that may include budgeting, planning, population
distribution, disease spread, experimental probability, reaction time modelling etc.) in terms of the
variables and the impact that these have on the outcome.
Conditional decision making: Using basic principles of combinatorics and an understanding of inter-
relationships between variables to interpret situations and make predictions.
Functions: The concept of function, emphasising but not limited to linear functions, their properties,
and a variety of descriptions and representations of them. Commonly used representations are
verbal, symbolic, tabular and graphical.
Algebraic expressions: Verbal interpretation of and manipulation with algebraic expressions,
involving numbers, symbols, arithmetic operations, powers and simple roots.
Equations and inequalities: Linear and related equations and inequalities, simple second-degree
equations, and analytic and non-analytic solution methods.
Co-ordinate systems: Representation and description of data, position and relationships.
Relationships within and among geometrical objects in two and three dimensions: Static relationships
such as algebraic connections among elements of figures (e.g. the Pythagorean theorem as
defining the relationship between the lengths of the sides of a right triangle), relative position,
similarity and congruence, and dynamic relationships involving transformation and motion of
objects, as well as correspondences between two- and three-dimensional objects.
Measurement: Quantification of features of and among shapes and objects, such as angle
measures, distance, length, perimeter, circumference, area and volume.
Numbers and units: Concepts, representations of numbers and number systems (including
converting between number systems), including properties of integer and rational numbers, as
well as quantities and units referring to phenomena such as time, money, weight, temperature,
distance, area and volume, and derived quantities and their numerical description.
Arithmetic operations: The nature and properties of these operations and related notational
conventions.
Percents, ratios and proportions: Numerical description of relative magnitude and the application of
proportions and proportional reasoning to solve problems.
Counting principles: Simple combinations.
Estimation: Purpose-driven approximation of quantities and numerical expressions, including
significant digits and rounding.
Data collection, representation and interpretation: Nature, genesis and collection of various types of
data, and the different ways to analyse, represent and interpret them.
Data variability and its description: Concepts such as variability, distribution and central tendency of
data sets, and ways to describe and interpret these in quantitative and graphical terms.
Samples and sampling: Concepts of sampling and sampling from data populations, including simple
inferences based on properties of samples including accuracy and precision.
Chance and probability: Notion of random events, random variation and its representation, chance and
frequency of events, and basic aspects of the concept of probability and conditional probability.

Contexts for the assessment items and selected 21st Century skills
The definition of mathematical literacy introduces two important considerations for the PISA assessment
items. First, the definition makes it clear that mathematical literacy takes place in real-world contexts.
Second, mathematical literacy assists individuals to know the role that mathematics plays in the world
and to make the well-founded judgements and decisions needed by constructive, engaged and reflective

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

21st Century citizens. In this section we discuss how both real-world contexts and 21 st Century skills
impact on item development.
The real-world context nature of mathematical literacy is not unproblematic for PISA. Real-world contexts
involve information which is communicated using text. The quantitative and statistical information that
flows in the world and reaches citizens is communicated through printed or spoken text, e.g. media
articles, press releases, blogs, social networks, advertisements etc. This printed and spoken text is used
to present messages or arguments that may or may not involve numbers and/or graphs. Text is the main
tool for communicating context, and it follows that text comprehension is a fundamental and pre-requisite
skill for success in mathematical literacy. The challenge this creates for PISA and item development is
not insignificant. On the one hand the assessment must present socially meaningful quantitative
messages using rich text, on the other hand the comparative nature of the assessment, the many
languages it is translated into and the wide range of text comprehension levels among participating 15-
year-olds places limits on the richness of the text that can realistically be used. This challenge is
discussed further in the section on item development.

Contexts

An important aspect of mathematical literacy is that mathematics is used to solve a problem set in a
context. The context is the aspect of an individual’s world in which the problems are placed. The choice
of appropriate mathematical strategies and representations is often dependent on the context in which a
problem arises, and by implication there is the need to utilise knowledge of the real -world context in
developing the model. Being able to work within a context is widely appreciated to place additional
demands on the problem solver (see Watson and Callingham, (2003 [33]), for findings about statistics) .
For PISA, it is important that a wide variety of contexts are used. This offers the possibility of connecting
with the broadest possible range of individual interests and with the range of situations in which
individuals operate in the 21st Century.
In light of the number of countries participating in PISA 2022 and with that an increasing range of
participants from low- and middle-income countries as well as the possibility of out -of- school 15 -year-
olds, it is important that item developers take great care to ensure that the contexts used for items are
accessible to a very broad range of participants. In this regard it is also important that the reading load of
the items remains modest so that the items continue to assess mathematical literacy.
For purposes of the PISA 2022 Mathematics Framework, the four context categories of the PISA 2012
framework have been retained and are used to inform assessment item development. It should be noted
that while these contexts are intended to inform item development, there is no expectation that there will
be reporting against these contexts.
Personal – Problems classified in the personal context category focus on activities of one’s self, one’s
family or one’s peer group. The kinds of contexts that may be considered personal include (but are not
limited to) those involving food preparation, shopping, games, personal health, personal transportation,
recreation, sports, travel, personal scheduling and personal finance.
Occupational – Problems classified in the occupational context category are centred on the world of
work. Items categorised as occupational may involve (but are not limited to) such things as measuring,
costing and ordering materials for building, payroll/accounting, quality control, scheduling/inventory,
design/architecture and job-related decision making either with or without appropriate technology.
Occupational contexts may relate to any level of the workforce, from unskilled work to the highest levels
of professional work, although items in the PISA survey must be accessible to 15-year-old students.
Societal – Problems classified in the societal context category focus on one’s community (whether local,
national or global). They may involve (but are not limited to) such things as voting systems, public transport,
government, public policies, demographics, advertising, health, entertainment, national statistics and

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 41

economics. Although individuals are involved in all of these things in a personal way, in the societal
context category, the focus of problems is on the community perspective.
Scientific – Problems classified in the scientific category relate to the application of mathematics to the
natural world and issues and topics related to science and technology. Particular contexts might include
(but are not limited to) such areas as weather or climate, ecology, medicine, space science, genetics,
measurement and the world of mathematics itself. Items that are intra-mathematical, where all the
elements involved belong in the world of mathematics, fall within the scientific context.
PISA assessment items are arranged in units that share stimulus material. It is therefore usually the case
that all items in the same unit belong to the same context category. Exceptions do arise; for example,
stimulus material may be examined from a personal point of view in one item and a societal point of view
in another. When an item involves only mathematical constructs without reference to the contextual
elements of the unit within which it is located, it is allocated to the context category of the unit. In the
unusual case of a unit involving only mathematical constructs and being without reference to any context
outside of mathematics, the unit is assigned to the scientific context category.
Using these context categories provides the basis for selecting a mix of item contexts and ensures that
the assessment reflects a broad range of uses of mathematics, ranging from everyday personal uses to
the scientific demands of global problems. Moreover, it is important that each context category be
populated with assessment items having a broad range of item difficulties. Given that the major purpose
of these context categories is to challenge students in a broad range of problem contexts, each category
should contribute substantially to the measurement of mathematical literacy. It should not be the case
that the difficulty level of assessment items representing one context category is systematically higher or
lower than the difficulty level of assessment items in another category.
In identifying contexts that may be relevant, it is critical to keep in mind that a purpose of the assessment
is to gauge the use of mathematical content knowledge and skills that students have acquired by age 15.
Contexts for assessment items, therefore, are selected in light of relevance to students’ interests and
lives and the demands that will be placed upon them as they enter society as constructive, engaged and
reflective citizens. National Project Managers from countries participating in the PISA survey are involved
in judging the degree of such relevance.

21st Century skills

There is increased interest worldwide in what are called 21 st Century skills and their possible inclusion in
educational systems. The OECD has put out a publication focusing on such skills and has sponsored a
research project entitled The Future of Education and Skills: An OECD 2030 Framework in which some
25 countries are involved in a cross-national study of curriculum including the incorporation of such skills.
The project has as its central focus what the curriculum might look like in the future, focusing initially on
mathematics and physical education.
Over the past 15 years or so a number of publications have sought to bring clarity to the discussion and
consideration of 21st Century skills. A summary of key reports and their conceptualisation of 21 st Century skills
is provided in PISA 2021 Mathematics: A Broadened Perspective [EDU/PISA/GB(2017)17]. After careful
analysis of these publications the authors recommended that a strong case can be made for the infusion of
specific 21st Century skills into specific disciplines. For example, it will become increasingly important to teach
students at school how to make reasonable arguments with appropriate justification. The arguments they
make should be mathematically rigorous, based on sound theory and strong enough to withstand criticism,
and yet, whenever possible, avoid referring to authorities (e.g. ‘it says so on the internet’). This is part of the
fundamental competence to make independent judgements and take responsibility for them (OECD, 2005 [34]).
In the social context it is not enough to be right; one must be able and ready to present arguments and to
defend them. Learning mathematics, with its clarity of contexts and

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

strong emphasis on logical reasoning and rigour at the appropriate level, is a perfect opportunity to
practice and develop the ability for this kind of argumentation.
Similarly, in the modern era, it is critical to equip students with tools that they can use to defend themselves
from lies and inferences that purport to be based on mathematical reasoning. Quite often some fluency in
logical reasoning is sufficient; a lie usually hides some hidden contradiction. The alertness of young minds
towards possible contradictions can be developed most easily in good classes of mathematics.
Using the logic of finding the intersection between generic 21 st Century skills and related but subject-matter
specific skills that are a natural part of the instruction related to that subject matter results in the following
identified eight 21st Century skills for inclusion in the PISA 2022 assessment framework. They are:
critical thinking;
creativity;
research and inquiry;
self-direction, initiative, and persistence;
information use;
systems thinking;
communication;
reflection.

Assessing mathematical literacy

This section outlines the approach taken to implement the elements of the framework described in previous
sections into the PISA survey for 2022. This includes the structure of the mathematics component of the PISA
survey, the desired distribution of score points for mathematical reasoning and the processes of problem
solving; the distribution of score points by content area; a discussion on the range of item difficulties; the
structure of the survey instrument; the role of the computer-based assessment of mathematics; the design of
the assessment items; and the reporting of levels of mathematical proficiency.

Structure of the PISA 2022 mathematics assessment

In accordance with the definition of mathematical literacy, assessment items used in any instruments that are
developed as part of the PISA survey are set within a context. Items involve the application of important
mathematical concepts, knowledge, understandings and skills (mathematical content knowledge) at the
appropriate level for 15-year-old students, as described earlier. The framework is used to guide the structure
and content of the assessment, and it is important that the survey instrument include an appropriate balance of
items reflecting the components of the mathematical literacy framework.

Desired distribution of score points by mathematical reasoning and problem-

solving process
Assessment items in the PISA 2022 mathematics survey can be assigned to either mathematical
reasoning or one of three mathematical processes associated with mathematical problem solving. The
goal in constructing the assessment is to achieve a balance that provides approximately equal weighting
between the two processes that involve making a connection between the real world and the
mathematical world (formulating and interpreting/evaluating) and mathematical reasoning and employing
which call for students to be able to work on a mathematically formulated problem. While it is true that
mathematical reasoning can be observed within the process of formulating, interpreting and employing
items will only contribute to one domain.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 43

Table 2.1. Approximate distribution of score points by domain for PISA 2022
Percentage of score points in PISA 2022
Mathematical reasoning Approximately 25
Mathematical problem Formulating situations mathematically Approximately 25
solving Employing mathematical concepts, facts and Approximately 25
procedures
Interpreting, applying and evaluating mathematical Approximately 25
outcomes
TOTAL 100

It is important to note that items in each process category should have a range of difficulty and
mathematical demand. This is further addressed in the table of demands for mathematical reasoning and
each of the problem solving processes.

Desired distribution of score points by content category

PISA mathematics items are selected to reflect the mathematical content knowledge described earlier in
this framework. The trend items selected for PISA 2022 will be distributed across the four content
categories, as shown in Table 2.2. The goal in constructing the survey is a distribution of items with
respect to content category that provides as balanced a distribution of score points as possible, since all
of these domains are important for constructive, engaged and reflective citizens.

Table 2.2. Approximate distribution of score points by content category for PISA 2022
Content category Percentage of score points in PISA 2022
Change and relationships Approximately 25
Space and shape Approximately 25
Quantity Approximately 25
Uncertainty and data Approximately 25
TOTAL 100

It is important to note that items in each content category should have a range of difficulty and
mathematical demand.

A range of item difficulties

The PISA 2022 mathematical literacy survey includes items with a wide range of difficulties, paralleling
the range of abilities of 15-year-old students. It includes items that are challenging for the most able
students and items that are suitable for the least able students assessed on mathematical literacy. From
a psychometric perspective, a survey that is designed to measure a particular cohort of individuals is
most effective and efficient when the difficulty of assessment items matches the ability of the measured
subjects. Furthermore, the described proficiency scales that are used as a central part of the reporting of
PISA outcomes can only include useful details for all students if the items from which the proficiency
descriptions are drawn span the range of abilities described.
Table 2.3 describes the range of actions that are expected of students for mathematical reasoning and
each of the problem-solving processes. These lists describe the actions that the items will demand of
students. For each category there are a number of items marked with “**” to denote the actions that are
expected of the students that will perform at levels 1a, 1b and 1c as well as level 2 of the proficiency
scale. Item developers will need to ensure that there are sufficient items at the lower end of the
performance scale to allow students at these levels to be able to show what they are capable of.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

In order to gain useful information for the new lower levels, 1b and 1c, it is vital that context and language
do not interfere with the mathematics being assessed. To this end, the context and language must be
carefully considered. That said, the items must still be interesting to avoid the possibility that students will
simply not attempt the items because it holds no interest.
The context for both 1b and 1c level items should be situations that students encounter on a daily basis.
Examples of these contexts may include money, temperature, food, time, date, weight, size and
distance. All items should be concrete and not abstract. The focus of the item should be mathematical
only. The understanding of the context should not interfere with the performance of the item.
Equally important, it is to have all items formulated in the simplest possible terms. Sentences should be
short and direct. Compound sentences, compound nouns and conditional sentences should be avoided.
Vocabulary used in the items must be carefully examined to ensure that students will have a clear
understanding of what is being required. In addition, special care will be given to ensure that no extra
difficulty is added due to a heavy text load or by a context that is unfamiliar to students based on their
cultural background.
Items designed for Level 1c should only ask for a single step or operation. However, it is important to
note that a single step or operation is not limited to an arithmetical step. This step might be demonstrated
by making a selection or identifying some information. Both mathematical reasoning and all of the
problem solving processes should be used to measure the mathematical literacy capabilities of students
at Levels 1b and 1c.

Table 2.3. Expected student actions for mathematical reasoning and each of the problem-solving
processes
Reasoning
Draw a simple conclusion
Select an appropriate justification
Explain why a mathematical result or conclusion does, or does not, make sense given the context of a problem
Represent a problem in a different way, including organising it according to mathematical concepts and making appropriate assumptions

Utilise definitions, rules and formal systems as well as employing algorithms and computational thinking
Explain and defend a justification for the identified or devised representation of a real-world situation
Explain or defend a justification for the processes and procedures or simulations used to determine a mathematical result or solution

Identify the limits of the model used to solve a problem

Understand definitions, rules and formal systems as well as employing algorithms and computational reasoning
Provide a justification for the identified or devised representation of a real-world situation
Provide a justification for the processes and procedures used to determine a mathematical result or solution
Reflect on mathematical arguments, explaining and justifying the mathematical result
Critique the limits of the model used to solve a problem
Interpret a mathematical result back into the real-world context in order to explain the meaning of the results
Explain the relationships between the context-specific language of a problem and the symbolic and formal language needed to represent
it mathematically
Reflect on mathematical arguments, explaining and justifying the mathematical result
Reflect on mathematical solutions and create explanations and arguments that support, refute or qualify a mathematical solution to
a contextualised problem
Analyse similarities and differences between a computational model and the mathematical problem that it is
modelling Explain how a simple algorithm works and to detect and correct errors in algorithms and programmes

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 45

Formulating Employing Interpreting

** Select a mathematical description or ** Perform a simple calculation **Interpret a mathematical result back into the real world
a representation that describes a context
problem
** Identify the key variables in a model ** Select an appropriate strategy from a ** Identify whether a mathematical result or conclusion
list does, or does not, make sense given the context of a
problem
** Select a representation appropriate ** Implement a given strategy to ** Identify the limits of the model used to solve a problem
to the problem context determine a mathematical solution

Read, decode and make sense of ** Make mathematical diagrams, Use mathematical tools or computer simulations to
statements, questions, tasks, objects or graphs, constructions or computing ascertain the reasonableness of a mathematical solution
images to create a model of the artifacts and any limits and constraints on that solution, given the
situation context of the problem
Recognise mathematical structure Understand and utilise constructs based Interpret mathematical outcomes in a variety of formats in
(including regularities, relationships, on definitions, rules and formal systems relation to a situation or use; compare or evaluate two or
and patterns) in problems or situations including employing familiar algorithms more representations in relation to a situation

Identify and describe the mathematical Develop mathematical diagrams, Use knowledge of how the real world impacts the
aspects of a real-world problem graphs, constructions or computing outcomes and calculations of a mathematical procedure or
situation including identifying the artifacts and extracting mathematical model in order to make contextual judgements about how
significant variables information from them the results should be adjusted or applied
Simplify or decompose a situation or Manipulate numbers, graphical and Construct and communicate explanations and arguments
problem in order to make it amenable statistical data and information, in the context of the problem
to mathematical analysis algebraic expressions and equations,
and geometric representations
Recognise aspects of a problem that Articulate a solution, showing and/or Recognise [demonstrate, interpret, explain] the extent and
correspond with known problems or summarising and presenting limits of mathematical concepts and mathematical
mathematical concepts, facts or intermediate mathematical results solutions
procedures
Translate a problem into a standard Use mathematical tools, including Understand the relationship between the context of the
mathematical representation or technology, simulations and problem and representation of the mathematical solution.
algorithm computational thinking, to help find Use this understanding to help interpret the solution in
exact or approximate solutions context and gauge the feasibility and possible limitations
of the solution

Use mathematical tools (using Make sense of, relate and use a variety
appropriate variables, symbols, of representations when interacting with
diagrams) to describe the mathematical a problem
structures and/or relationships in a
problem
Apply mathematical tools and Switch between different
computing tool to portray mathematical representations in the process of finding
relationships solutions
Identify the constraints, assumptions Use a multi-step procedure leading to a
simplifications in a mathematical model mathematical solution, conclusion or
generalisation
Use an understanding of the context to
guide or expedite the mathematical
solving process, e.g. working to a
context-appropriate level of accuracy
Make generalisations based on the
results of applying mathematical
procedures to find solutions

Note: Table 2.3 is a reformulation of the figure used in previous frameworks to link mathematical processes with mathematical capabilities. All
of the examples and illustrations from that figure are included in this reformulation.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

Computer-based assessment of mathematics

The main mode of delivery for PISA 2022 will be the computer-based assessment of mathematics
(CBAM). The transition has been anticipated with both the 2015 and 2018 studies moving to computer-
based delivery. In order to maintain trends across the studies, both the 2015 and 2018 assessments
were computer neutral despite using a computer- based delivery mode. The transition to a full CBAM in
2022 provides a range of opportunities to develop the assessment of mathematical literacy to be better
aligned with the evolving nature of mathematics in the modern world, while ensuring backward trends to
previous cycles. These opportunities include new item formats (e.g. drag and drop); presenting students
with real-world data (such as large, sortable datasets); creating mathematical models or simulations that
students can explore by changing the variable values; curve fitting and using the best fit curve to make
predictions. In addition to a wider range of question types and mathematical opportunities that the CBAM
provides, it also allows for adaptive assessment.
The adaptive assessment capability of the CBAM, which was previously implemented in the PISA
reading assessment, provides the opportunity of better describing what it is that students at both ends of
the performance spectrum are able to do. By providing students with increasing individualised
combinations of test units according to their responses and scores to the early units that they respond to,
increasingly detailed information on the performance characteristics of students at both ends of the
performance scale is generated.
Making use of enhancements offered by computer technology results in assessment items that are more
engaging to students, more visually appealing, and easier to understand. For example, students may be
presented with a moving stimulus, representations of three-dimensional objects that can be rotated or
more flexible access to relevant information. New item formats, such as those calling for students to
‘drag and drop’ information or use ‘hot spots’ on an image, are designed to engage students, permit a
wider range of response types and give a more rounded picture of mathematical literacy. A key challenge
is to ensure that these items continue to assess mathematical literacy and that interference from domain
irrelevant dimensions is kept to a minimum.
Investigations show that the mathematical demands of work increasingly occur in the presence of
electronic technology so that mathematical literacy and computer use are melded together (Hoyles et al.,
2002[35]). For employees at all levels of the workplace, there is now an interdependency between
mathematical literacy and the use of computer technology. A key challenge is to distinguish the
mathematical demands of a PISA computer-based item from demands unrelated to mathematical
competence, such as the information and communications technology (ICT) demands of the item, and
the presentation format. Solving PISA items on a computer rather than on paper moves PISA into the
reality and the demands of the 21st Century.
Questions that seem well suited to the CBAM and the evolving nature of mathematical literacy include:
Simulation in which a mathematical model has been established and students can change the
variable values to explore the impact of the variables to create “an optimal solution”.
Fitting a curve (by selecting a curve from a limited set of curves provided) to a data set or a
geometric image to determine the “best fit” and using the resulting best fit curve to determine the
answer to a question about the situation.
Budgeting situations (e.g. online store) in which the student must select combinations of products to
meet achieve a range of objectives within a given budget.
Purchase simulation in which the student selects from different loan and associates repayment
options to purchase an item using a loan and meeting a budget. The challenge in the problem is
to understand how the variables interact.
Problems that include visual coding to achieve a given sequence of actions.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 47

Notwithstanding the opportunities that the CBAM presents (described above), it is important that the
CBAM remains focused on assessing mathematical literacy and does not shift to assessing ICT skills.
Similarly, it is important that the simulations and other questions hinted at above do not become so
“noisy” that the mathematical reasoning and problem-solving processes are lost.
The CBAM must also retain some of the paper version features for example the ability to revisit items
already attempted – although in the context of adaptive testing this will of necessity be limited to the unit
on which the student is working.

Design of the PISA 2022 mathematics items

Three item format types are used to assess mathematical literacy in PISA 2022: open constructed-
response, closed constructed-response and selected-response (multiple-choice) items.
Open constructed-response items require a somewhat extended written response from a student. Such
items also may ask the student to show the steps taken or to explain how the answer was reached.
These items require trained experts to manually code student responses. To facilitate the adaptive
assessment feature of the CBAM, it will be necessary to minimise the number of items that rely on
trained experts to code the student responses.
Closed constructed- response items provide a more structured setting for presenting problem solutions,
and they produce a student response that can be easily judged to be either correct or incorrect. Often
student responses to questions of this type can be coded automatically. The most frequently used closed
constructed-responses are single numbers.
Selected-response items require the choice of one or more responses from a number of response
options. Responses to these questions can usually be automatically processed. About equal numbers of
each of these item format types are being used to construct the survey instruments.
The PISA mathematics survey is composed of assessment units comprising written stimulus material
and other information such as tables, charts, graphs or diagrams, plus one or more items that are linked
to this common stimulus material. This format gives students the opportunity to become involved with a
context or problem by responding to a series of related items.
Items selected for inclusion in the PISA survey represent a broad range of difficulties, to match the wide
ability range of students participating in the assessment. In addition, all the major categories of the
assessment (the content categories; mathematical reasoning and problem-solving process categories
and the different context categories and 21st Century skills) are represented, to the degree possible, with
items of a wide range of difficulties. Item difficulties are established as one of a number of measurement
properties in an extensive field trial prior to item selection for the main PISA survey. Items are selected
for inclusion in the PISA survey instruments based on their fit with framework categories and their
measurement properties.
In addition, the level of reading required to successfully engage with an item is considered very carefully
in item development and selection. A goal in item development is to make the wording of items as simple
and direct as possible. Care is also taken to avoid item contexts that would create a cultural bias, and all
choices are checked with national teams. Translation of the items into many languages is conducted very
carefully, with extensive back-translation and other protocols.
PISA 2022 will include a tool that will allow students to provide typed constructed response answers and
show their work as required for mathematical literacy. The tool allows students to enter both text and
numbers. By clicking the appropriate button, students can enter a fraction, square root, or exponent.
Additional symbols such as π and greater/less than signs are available, as are operators such as
multiplication and division signs. An example is shown in Figure 2.3 below.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

Figure 2.3. Example of the PISA 2022 editor tool

The suite of tools available to students is also expected to include a basic scientific calculator. Operators
to be included are addition, subtraction, multiplication and division, as well as square root, pi,
parentheses, exponent, square, fraction (y/x), inverse (1/x) and the calculator will be programmed to
respect the standard order of operations.
Students taking the assessment on paper can have access to a hand-held calculator, as approved for
use by 15-year-old students in their respective school systems.

Item scoring

Although the majority of the items are dichotomously scored (that is, responses are awarded either credit
or no credit), the open constructed-response items can sometimes involve partial credit scoring, which
allows responses to be assigned credit according to differing degrees of “correctness” of responses and
or to the extent to which an item has been engaged with or not. It is anticipated that the need for partial
credit scoring will be particularly significant for the mathematical reasoning items which will seldom
involve the production of single number response but rather responses with one or more elements.

Reporting proficiency in mathematics

The outcomes of the PISA mathematics survey are reported in a number of ways. Estimates of overall
mathematical proficiency are obtained for sampled students in each participating country, and a number
of proficiency levels are defined. Descriptions of the degree of mathematical literacy typical of students in
each level are also developed. For PISA 2022, the six proficiency levels reported for the overall PISA
mathematics in previous cycles will be expanded as follows: Level 1 will be renamed Level 1a, and the
table describing the proficiencies will be extended to include Levels 1b and 1c. These additional levels
have been added to provide greater granularity of reporting in students performing at the lower end of the
proficiency scale. In Table 2.4, summary descriptions of the eight proficiency levels on the mathematical
literacy for the overall PISA mathematics scale in 2022 are presented.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 49

Table 2.4. Summary descriptions of the eight proficiency levels on the mathematical literacy scale
Level What students can typically do
At Level 6, students can work through abstract problems and demonstrate creativity and flexible thinking to develop solutions. For
example, they can recognise when a procedure that is not specified in a task can be applied in a non-standard context or when
demonstrating a deeper understanding of a mathematical concept is necessary as part of a justification. They can link different
information sources and representations, including effectively using simulations or spreadsheets as part of their solution. Students at
this level are capable of critical thinking and have a mastery of symbolic and formal mathematical operations and relationships that
they use to clearly communicate their reasoning. They can reflect on the appropriateness of their actions with respect to their solution
and the original situation.
At Level 5, students can develop and work with models for complex situations, identifying or imposing constraints, and specifying
assumptions. They can apply systematic, well-planned problem-solving strategies for dealing with more challenging tasks, such as
deciding how to develop an experiment, designing an optimal procedure, or working with more complex visualisations that are not
given in the task. Students demonstrate an increased ability to solve problems whose solutions often require incorporating
mathematical knowledge that is not explicitly stated in the task. Students at this level reflect on their work and consider
mathematical results with respect to the real-world context.
At Level 4, students can work effectively with explicit models for complex concrete situations, sometimes involving two variables, as well as
demonstrate an ability to work with undefined models that they derive using a more sophisticated computational-thinking approach.
Students at this level begin to engage with aspects of critical thinking, such as evaluating the reasonableness of a result by making
qualitative judgements when computations are not possible from the given information. They can select and integrate different
representations of information, including symbolic or graphical, linking them directly to aspects of real-world situations. At this level,
students can also construct and communicate explanations and arguments based on their interpretations, reasoning, and
methodology.
At Level 3, students can devise solution strategies, including strategies that require sequential decision making or flexibility in
understanding of familiar concepts. At this level, students begin using computational-thinking skills to develop their solution strategy.
They are able to solve tasks that require performing several different but routine calculations that are not all clearly defined in the
problem statement. They can use spatial visualisation as part of a solution strategy or determine how to use a simulation to gather
data appropriate for the task. Students at this level can interpret and use representations based on different information sources and
reason directly from them, including conditional decision making using a two-way table. They typically show some ability to handle
percentages, fractions and decimal numbers, and to work with proportional relationships.
At Level 2, students can recognise situations where they need to design simple strategies to solve problems, including running
straightforward simulations involving one variable as part of their solution strategy. They can extract relevant information from one or
more sources that use slightly more complex modes of representation, such as two-way tables, charts, or two-dimensional
representations of three-dimensional objects. Students at this level demonstrate a basic understanding of functional relationships
and can solve problems involving simple ratios. They are capable of making literal interpretations of results.
1a At Level 1a, students can answer questions involving simple contexts where all information needed is present, and the questions are
clearly defined. Information may be presented in a variety of simple formats and students may need to work with two sources
simultaneously to extract relevant information. They are able to carry out simple, routine procedures according to direct instructions in
explicit situations, which may sometimes require multiple iterations of a routine procedure to solve a problem. They can perform
actions that are obvious or that require very minimal synthesis of information, but in all instances the actions follow clearly from the
given stimuli. Students at this level can employ basic algorithms, formulae, procedures, or conventions to solve problems that most
often involve whole numbers.
1b At Level 1b, students can respond to questions involving easy to understand contexts where all information needed is clearly given in
a simple representation (i.e., tabular or graphic) and, as necessary, recognise when some information is extraneous and can be
ignored with respect to the specific question being asked. They are able to perform simple calculations with whole numbers, which
follow from clearly prescribed instructions, defined in short, syntactically simple text.
1c At Level 1c, students can respond to questions involving easy to understand contexts where all relevant information is clearly given in
a simple, familiar format (for example, a small table or picture) and defined in a very short, syntactically simple text. They are able to
follow a clear instruction describing a single step or operation.

As well as the overall mathematics scale, additional described proficiency scales are developed after the
field trial and are then reported. These additional scales are for mathematical reasoning and for the three
processes of mathematical problem solving: formulating situations mathematically; employing
mathematical concepts, facts and procedures; and interpreting, applying and evaluating mathematical
outcomes.

Mathematical literacy and the background questionnaires

Since the first cycle of PISA, student and school context questionnaires have served two interrelated purposes
in service of the broader goal of evaluating educational systems: first, the questionnaires provide a context
through which to interpret the PISA results both within and between education systems. Second,

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

the questionnaires aim to provide reliable and valid measurement of additional educational indicators,
which can inform policy and research in their own right.
Since mathematical literacy is the major domain in the 2022 survey, the background questionnaires are
expected to provide not only trend data for the constructs that continue to be assessed, but additionally
to provide rich information on the innovations that are evident in the PISA 2022 mathematical literacy
framework. In particular it is expected that mathematical literacy will feature prominently in the analysis of
the domain-specific contextual constructs as well in a number of the different categories of policy focus
that range from individual level variables such a demographics and social and emotional characteristics
to school practices, policies and infrastructure (OECD, 2018[36]).
Two broad areas of students’ attitudes towards mathematics that dispose them to productive
engagement in mathematics were identified as being of potential interest as an adjunct to the PISA 2012
mathematics assessment. These are students’ interest in mathematics and their willingness to engage in
it. It is expected that these will continue to be a focus of the questionnaires in 2022.
Interest in mathematics has components related to present and future activity. Relevant questions focus
on students’ interest in mathematics at school, whether they see it as useful in real life as well as their
intentions to undertake further study in mathematics and to participate in mathematics -oriented careers.
There is international concern about this area, because in many participating countries there is a decline
in the percentage of students who are choosing mathematics related future studies, whereas at the same
time there is a growing need for graduates from these areas.
Students’ willingness to do mathematics is concerned with the attitudes, emotions and self-related beliefs
that dispose students to benefit, or prevent them from benefitting, from the mathematical literacy that
they have achieved. Students who enjoy mathematical activity and feel confident to undertake it are
more likely to use mathematics to think about the situations that they encounter in the various facets of
their lives, inside and outside school. The constructs from the PISA survey that are relevant to this area
include the emotions of enjoyment, confidence and (lack of) mathematics anxiety, and the self-related
beliefs of self-concept and self-efficacy. An analysis of the subsequent progress of young Australians
who scored poorly on PISA at age 15 found that those who “recognise the value of mathematics for their
future success are more likely to achieve this success, and that includes being happy with many aspects
of their personal lives as well as their futures and careers” (Hillman and Thomson, 2010, p. 31[37]). The
study recommends that a focus on the practical applications of mathematics in everyday life may help
improve the outlook for these low-achieving students.
The innovations evident in the PISA 2022 Mathematics Framework point to at least four areas in which
the background questionnaires can provide rich data. These areas are: mathematical reasoning;
computational thinking and the role of technology in both doing and teaching mathematics; the four
focal content areas; and 21st Century skills in the context of mathematics.
Mathematical reasoning

The PISA 2022 Mathematics Framework foregrounds mathematical reasoning enabled by some key
understandings that undergird school mathematics (understanding quantity, number systems and their
algebraic properties; appreciating the power of abstraction and symbolic representation; seeing
mathematical structures and their regularities; recognising functional relationships between quantities;
using mathematical modelling as a lens onto the real world; and understanding variation as the heart of
statistics).
The focus on reasoning has implications for the background questionnaires which should provide
measures to understand students’ opportunities to learn to reason mathematically and employ the key
understandings that undergird school mathematics. In particular the questionnaires should establish the
frequency with which students, for example:

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 51

identify, recognise, organise, connect, and represent;

construct, abstract, evaluate, deduce, justify, explain, and defend;
interpret, make judgements, critique, refute, and qualify.
In addition to establishing the frequency of the opportunities (to learn) to reason, the questionnaires
should get at what forms these opportunities take (verbal or written).
Finally, with respect to reasoning, the questionnaires should get a sense of the willingness of students to
persist with tasks that involve reasoning.
In the case of teachers and teaching there is the need to better understand how they see the role of
reasoning in mathematics in general and in their teaching and assessment practices in particular.

Computational thinking

Aspects of computational thinking form a rapidly evolving and growing dimension of both mathematics
and mathematical literacy. The PISA 2022 mathematical literacy framework illustrates how computational
thinking is both part of doing mathematics and impacting on doing mathematics. The values and beliefs
about learning and open-mindedness modules of the background questionnaires can explore student’s
experience of the role of computational thinking in doing mathematics.
The PISA 2022 mathematical literacy framework draws attention to the different ways in which technology is
both changing the world in which we live and changing what it means to engage in mathematics. Key
questions for the background questionnaires include developing a deep understanding of first, how students’
experiences of mathematics and doing mathematics are changing (if at all) and second, how classroom
pedagogy is evolving due to the impact that technology is having on how students engages with mathematics
and mathematical artifacts and on what it means to do mathematics. In the case of students, it is of interest to
better understand how technology is impacting student performance which could be explored in the task
performance module of the questionnaire framework. The pedagogical issues could be explored in both the
learning time and curriculum and teaching practices modules.
The focus on computational thinking and the role of technology in both doing and teaching mathematics
has implications for the background questionnaires which should provide measures to better understand
students’ opportunities to learn in this regard. In particular the questionnaires should establish the
frequency with which students, for example:
design or work with computer simulations and or computer models;
code or program both inside the mathematics classroom and outside it;
are exposed to Computer Mathematics Systems (CSM) (including dynamic geometry software;
spreadsheets; programming software (e.g. Logo and Scratch); graphing calculators; games etc.).

Four focal content areas

In recognition of the changing world the PISA 2022 Mathematics Framework has suggested that four
content areas within the existing content framework receive special focus. These content areas are:
growth phenomena (within change and relationships); geometric approximation (within space and
shape); computer simulations (within quantity); and conditional decision making (within uncertainty and
data). The focus on these content areas has implications for the background questionnaires which
should provide measures to better understand students’ opportunities to learn in this regard. In particular
the questionnaires should establish the frequency with which students are exposed to these contents
and the different forms that the opportunities take.

PISA 2022 ASSESSMENT AND ANALYTICAL FRAMEWORK © OECD 2023

 21st Century skills in the context of mathematics
The PISA 2022 mathematical literacy framework introduces a particular set of 21 st Century skills both as
an outcome of and focus for mathematics. The background questionnaires could productively examine
both whether or not mathematics is contributing to the development of these skills and if teaching
practices are focusing on them. In particular, the learning time and curriculum module could explore
whether or not these skills appear in the enacted curriculum.
The results of the PISA 2022 survey will provide important information for educational policy makers in
the participating countries about both the achievement -related and attitude-related outcomes of
schooling. By combining information from the PISA assessment of mathematical literacy and the survey
information on attitudes, emotions and beliefs that predispose students to use their mathematical literacy
as well as the impact of the four developments described above, a more complete picture will emerge.

Conclusion

The PISA 2022 mathematical literacy framework while maintaining coherence with the previous
mathematical literacy frameworks acknowledges that the world is ever changing and with it the demand
for mathematically literate citizens to reason mathematically rather than reproducing mathematical
techniques as routines,
The aim of PISA with regard to mathematical literacy is to develop indicators that show how effectively
countries are preparing students to use mathematics in the everyday aspect of their personal, civic and
professional lives, as constructive, engaged and reflective 21st Century citizens. To achieve this, PISA
has developed a definition of mathematical literacy and an assessment framework that reflects the
important components of this definition.
The mathematics assessment items selected for inclusion in PISA 2022, based on this definition and
framework, are intended to reflect a balance between mathematical reasoning, problem solving
processes, mathematical content and contexts.
The assessment design will assure valid measurement of ability across the range of achievement
extending to two levels below the previous PISA scale, while preserving the quality and content of the
assessment.
The CBAM to be used from 2022 provides problems in a variety of item formats with varying degrees of
built-in guidance and structure and a range of formats retaining throughout an emphasis on authentic
problems that require students to reason and demonstrate their thinking.