Slide 1

Introduction to cognitive ergonomics& Notes on Perception (CH1) and affordances

Slide 2
I am Dr Simone Borsci. Assistant professor in Human factors and cognitive Ergonomics at the
Department of Learning, Data analysis, and Technology.
After 8 years in the UK working on VR and human Factors for Medical devices I moved at
the University of Twente roughly 2 years ago.

I will be your teacher for this course

Slide 3
Course structure

Slide 4
In the next 7 weeks, we will discuss several key concepts of Cognitive Ergonomics. Some of
these concepts are included in the handbook of the course, some concepts are not (therefore I
strongly recommend you to attend the course).
Lessons will be live.

+ The lesson is the place to ask questions and clarify doubts i.e., before the exam
+ Moreover, roughly every week we are going to do an activity in class, to help you to better
understand some contents and to also practice for the exam.

In the slides you have a summary of the contents for each lesson:
L1 – Introduction to Ergonomics, Perception, Affordances
L2 – Introduction to Usability and UX methods
L3 – Usability and UX applied: Case Study on usability – Design principles
L4 – Gestalt principles, Attention, Memory
L5 – Decision making and Persuasion and Errors
L6 – Introduction to eye tracking
L7 – Q&A session & recap

Slide 5
This course aims to prepare you:
• To Describe basic principles from cognitive psychological theory related to:
perception, action, attention, memory and decision making
• To name design principles related to human cognitive strengths and weaknesses.
• And, To Apply cognitive design principles to simple design problems.
Slide 6
- You may find the handbook free online by the University library:Johnson, J.
(2013). Designing with the mind in mind: simple guide to understanding user interface
design guidelines. Elsevier.

You also have online a list of MANDATORY READINGS in PREPARATION FOR EACH
LESSON

The lessons will not be VIDEO RECORDED

Slide 7
The exam will be composed of open questions about the contents of the handbook and the
contents of the SLIDES. I will post details about the exam modalities on Canvas

Slide 8
This course will be focused on the cognitive elements involved during the interaction between
people and artifacts

We will discuss principles and methods that can be extended to multiple interactive products.
objects of everyday use
Automotive
Healthcare
virtual reality
etc.

Slide 9
During this course, we are going to look at: How we perceive, elaborate and process
information, how we memorize the reality and how we react to certain events.

This figure represents “The model of Human Information processing” proposed by Wickens
and colleagues.
We are going to use this figure as a map during the course: for instance today in the second
part of the lecture we are going to talk about perception, which is part of the perceptual
encoding on the left of this figure

Slide 10
Cognitive Ergonomics
Slide 11
History of CE and task analysis
History of CE and task analysis
Our story starts with Wojciech Jastrzębowski, a polish scientists professor of physics and
zoology. He coined the term “ergonomics” in 1857. We are before the establishment of the
first psychology lab of Wundt in 1870.
At that time the focus was on how we can measure people behaviour in terms of reaction
time, how people think in terms of introspection and retrospection.
The main question was: How we can alter or control the behaviour of individuals?
The work of Jastrzebowski in Poland was mainly focused on: how we can design technologies
to better support human performance? Which is a particularly advanced and modern research
question.
In all this pioneering work the concept of “Tasks” ( sequential and cognitive) was important.
It is only around 1911 that the notion of task analysis, as a methodology for time an motion
studies, was widely used and applied. Suddenly researchers started to reflect on the fact that
certain tasks are not behavioural but cognitive (mental operations).
It is during that first and the second world war that the idea of Man-Machine interaction
exploded especially related to the idea of reducing the fatigue of the human operators.

Weston advocate for the importance of considering human during the design phase in order to
avoid injury and accidents. He specifically mentions << the need to determine the
―physiological and psychological facts, so that they can be considered during the design
process>> (Ref. Weston, H. G. (1923). A note on machine design concerning the operative. In
the "Third Annual Report of the Industrial Fatigue Research Board to 31 December, 1922"
(pp. 71-75). London: His Majesty's Stationery Office)

This idea was dominant until the 60, and today cognitive task analysis and Cognitive
engineering and ergonomics as a discipline is a fundamental part of a good design process and
evaluation.

Slide 12
Ergonomics can be defined as the natural law of people at work from the Greek words ergon
(work) nomos (natural Law).
Nowadays, we have an international definition of Ergonomics and Human Factors. Let’s read
this definition together:

<<is the scientific discipline concerned with the: understanding of interactions among humans
and other elements of a system, and the profession that applies theory, principles, data and
methods to design in order to optimize human well-being and overall system performance.
Practitioners of ergonomics and ergonomists contribute to the design and evaluation of tasks,
jobs, products, environments and systems in order to make them compatible with the needs,
abilities and limitations of people.
Ergonomics helps harmonize things that interact with people in terms of people’s needs,
abilities and limitations.>>
Ref . International Ergonomics Association ([Link]
Maybe you have noticed that The word system appears 3 times in this definition. So, Maybe
we need to clarify what a system is…but let’s have a look before the definition of cognitive
ergonomics

Slide 13
Cognitive Ergonomics is concerned with mental processes, such as perception, memory,
reasoning, and motor response, as they affect interactions among humans and other elements
of a system.

Key topics: Workload, work stress, Training Decision-making, Performance and reliability
Interaction with tools, processes and people

Cognitive Ergonomics focuses on PEOPLE (processes, action and interaction) in a system i.e.
People in a SYSTEM

Slide 14
So if we need to answer the question: Why cognitive ergonomics is important?
• To know the general rules behind cognition e.g., how people: perceive and construct the
world, how people remember, act, think, feel, find their way, make decisions and
communicate
• To learn how to use this knowledge to design and evaluate (complex systems) and to
simplify people life, work, well-being etc.

Slide 15
Cognition and Heuristics

Slide 16
We apply heuristics and rules of thumb to simplify our life and to inform our decision-making
and we tend to distribute our cognitive processes

Every day we need to perform fast decisions, and often we use shortcuts (heuristics) that we
inherited from evolution or that we acquired socially.
We know from cognitive psychology that people apply heuristics and rules of thumb every
day to simplify their life and to inform their decision-making.
This enables us to deal with, and minimize, the complexity around us.

To inform our decision making we often interact with other people and we use tools. So that
some researchers say that we have distributed our cognition

Slide 17
We have a biased mind but we are surprisingly good at handling uncertainty.
As exemplified by Gerd Gigerenzer and Henry Brighton << Homo heuristicus has a biased
mind and ignores part of the available information, yet a biased mind can handle uncertainty
more efficiently and robustly than an unbiased mind relying on more resource-intensive and
general-purpose processing strategies>>

Slide 18
Recognition Heuristics
Gerd Gigerenzer and Golstein1 suggest that under uncertainty people may use heuristics to
infer information in a quite accurate way.
For instance, in an experiment, they asked American participants to infer how many
inhabitants were in pairs of American and German cities: answering which was the larger and
the smaller.
Results showed that people were actually good at answering the number of inhabitants in over
70% of the cases.

The process/strategy performed by participants could be described as follows:

People ask themselves whether they know the city or not. And if they don’t; they can also ask
themselves have I have heard of this city in the news or other places (e.g., Mediator)? And if
the answer was no, it could be concluded the city was (probably) a small one.

Slide 19
Heuristics are powerful, but simple models of the world, and may lead to systematic errors
(“Biases”), for instance: We tend often to perform decisions based on what is immediately
in front of us overestimating what we are perceiving in a specific moment and our
expectations/memories. This could prevent us to see the overall picture and to consider
alternative ways to perform certain actions or to achieve our goals

For example
We may be exposed to the phenomenon of cognitive tunnelling: when we just consider our
own hypothesis, or we are so fixated on a specific task or specific information in front of us
that we miss to pay attention or notice other relevant information.
OR
We may be affected by a confirmation bias: We look at and search for evidence or information
that tend to confirm our hypotheses or beliefs. In doing that we tend to only look and interpret
evidence that confirms existing beliefs while rejecting or ignoring any conflicting data.

Slide 20
Fitness and balance
To understand the relationship between human and artefacts, services and systems we need to
understand the limitations and the strengths of humans and technology and find a balance.
When people design, they aim (or should aim) at creating an equilibrium among the elements
(human and technology etc.) by a process of fitness. The ergonomists aim at assessing this
fitness.

1
Gigerenzer, G., & Brighton, H. (2009). Homo heuristicus: why biased minds make better inferences. Topics in
Cognitive Science,1(1), 107–43. Link UT Library: [Link] (Links to an
external site.)
Slide 21
During the interaction exchange with humans, a product creates an experience! Only when
this experience is in line or even exceed the expectations of the end-users we talk about a
positive user experience that creates the value for people.

Slide 22
Cognition, artefacts and Systems

Slide 23
In terms of distributed cognition: Around us, there are natural objects, cultural and cognitive
artifacts.
- Natural objects, such as trees, are not modified by people to convey information or for
any practical purpose.

We can think about the ways in which we may use natural objects. For instance, we may
climb a tree for fun, or we may use a tree as a shelter when it is raining etc.

Slide 24
However, we may manipulate natural objects and transform these in artefacts
• Cole defines an Artifact as a material object that has been modified by human beings
as a means of regulating their interactions with the world and each other”
• For instance, when a tree is manipulated (like in the picture) this communicates
information such as:
• Someone owns this tree
• Someone cuts the tree in a particular way to produce a sense of harmony in a
garden, etc…

So a natural object was transformed to change the aesthetics of the GARDEN. The tree
becomes a functional element of the system (garden)

Slide 25
Artefacts may be divided in:

 Cultural artefacts: These are objects that transpose and transport know-how,
tradition and culture: Such as the tree in the garden…
 Cognitive artefacts: These are different from the cultural ones. These are devices
and tools designed to maintain, display, or operate upon the information to improve
cognition and performances of people
Slide 26
Ergonomists
In our complex world, the role of Cognitive Ergonomists is to support the designer during the
design of artefacts (cultural and cognitive ones) and to measure and represent the needs of the
users

An interactive product is a “medium” (a tool) that enables an indirect communication between

the designer and end-user by symbols (i.e., cultural artefacts e.g., words, graphic elements
etc.) and interactive functions (e.g., cognitive artefacts).

• Cognitive Engineers mediate the relationship between the Designers and the Users
and support the design of usable technology by user research, experiments, and
data analysis

Slide 27
Gulfs of execution and evaluation2
When we are interacting with someone or with a product, we need to know the rules to
understand how to act and which action we need to perform to achieve a goal.
Then we interpret the answer.
For instance, when we are talking with someone we can rapidly assess if we obtained the
information we are looking for: If you ask the direction for a certain place, You can
immediately understand if the person knows that or not.

Because language is something that we learned and we master, it is usually easy to understand
the rules of language exchanges for us.
However, this process is a bit harder with interfaces. To interact with products we need to
learn how to operate these systems, but also how to interpret the answer of the system.
The distance between the gulf of execution and the gulf of evaluation is what we are trying to
bridge when, for instance, we are designing usable products.

Slide 28
Ergonomists can focus on different aspects, and adopt at least three different
perspectives to look at systems:

- Macro: Macro ergonomics deals with structural dimensions of the work system. Today
this perspective is also called Organizational Design (OD) and is used in the design of
organizational structures and relationships among components of such structure. This
is focused on the organizational aspects that affect people behaviour and actions

- Micro: Activities analysing work or worker posture, estimating productivity, designing

work tools, work physiology, work biomechanics, physical environment,

2
Norman, D. A., & Draper, S. W. (1986). User centered system design; new perspectives on
human-computer interaction. L. Erlbaum Associates Inc
 anthropometry, percentiles, standard time and others, these are scientific within the
scope of micro ergonomics. Thus, micro ergonomics is focused on people
performances with artefacts.

- Meso: Meso ergonomics encompasses relationships between individuals and

organizations and sociotechnical systems. In designing the work system, the main
factor to be considered as the centre of attention is the human factor. The approach in
industrial design is Human Integrated Design, that utilises all information about
human beings covering its advantages and disadvantages and being integrated as the
basis for a system design. This is focused on people interacting with products and
constraints due to the organizational aspects

These perspectives are focusing on different aspects or portion of a system.

Slide 29
We think in terms of Systems
We learn to explore our world as humans and to connect things and concepts. We tend to see
the whole instead of the parts and to make a synthesis of information

Slide 30
Cabrera and colleagues attempted in 20083 to identify 4 key rules to explain why systems
thinking is a natural way of think for humans. They identified 4 elements:
Distinction (D); Relationship (R); System (S); Perspective (P)

In terms of DISTINCTION, we need to develop an identity to identify the others. In terms of

Development, to have an “EGO”, we need to develop a theory of mind; A baby should learn
that what s/he thinks is not the same of what his/her mother or father thinks. We know that a
baby is aware of that distinction when s/he starts to lie, for instance, when a baby starts to
pretend to be at sleep.
But this works also at the conceptual level. We need to know what is a “dog”, to identify what
is not a “dog”.
Once that we are able to distinguish elements than we need to be able to connect them. The
interconnection between elements (RELATIONSHIP) is what composes a SYSTEM. For
instance, we know that dogs are animals, and also some “not dogs” are animals. So we can
draw a connection and say that dogs and frogs are animals.

However, from an objective PERSPECTIVE frogs are Amphibians and dogs are Mammals
with some very different characteristics.

3
Cabrera, D., Colosi, L., & Lobdell, C. (2008). Systems thinking. Evaluation and program
planning, 31(3), 299-310
Slide 31
To design, understand or explore the basic components of a system at least the following
questions should be answered:
• Distinctions (identity and other): What is __? What is not __?
• Systems (part and whole): Does __ have parts? Can you think of __ as a part?
• Relationships (inter and action): Is __ related to __? Can you think of __ as a
relationship?
• Perspectives (point and view): From the perspective of __, [insert question]? Can you
think about __ from a different perspective?

Slide 32
Definition of a system and sociotechnical systems
We can define in general a System as a complex whole: a set of things working together as
parts of an interconnecting network;
Experts of Human Factors focus on Sociotechnical systems.

Slide 33
A Sociotechnical system consists of sub-systems: people, technologies, structures, and
processes.

For instance, this lesson is a sociotechnical system (we have different people, with different
roles, we are following processes and rules, we have communication exchanges mediated by a
set of technology). Moreover, this course, is a subsystem of Module 6 and your bachelor
curriculum. And this Virtual room is a sub-system of this Virtual platform (canvas).
To enable these components to interact without conflicts is the main goal of Human factors
How to evaluate the cognitive effect of the systems is the key goal of Cognitive ergonomics

Slide 34
To think in terms of systems means to model the reality to inform how to redesign systems.

Here you have an example of a sociotechnical model of Healthcare systems. The Model of
Systems Engineering for Patient Safety4. This model shows that a system in healthcare is
usually composed of:
People, with their tasks, and goals
Processes (designed)
Tools (designed)
Context (natural or designed)
Rules, constraints

4
Holden, R. J., Carayon, P., Gurses, A. P., Hoonakker, P., Hundt, A. S., Ozok, A. A., & Rivera-Rodriguez, A. J.
(2013). SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals
and patients. Ergonomics, 56(11), 1669-1686
And outcomes. Usually, it is only when we look at outcomes that we may assess how good is
the system to produce the expected results. To improve the outcomes we need to change how
the System Works and its processes.

Now if you are an HCI expert you focus on how you may change the interaction of these
elements of the system: people, technology and tasks.
If you are a Human factor expert you focus on how you may change the interaction with all
the elements of a system.

Slide 35
A sociotechnical system
• Can be designed and modified. So, it is an artefact
• And it is the context in which we distributed our cognition

Slide 36
System thinking means to help the designers to look at how a product should be design to
maximise the benefit by taking in to account different perspectives and needs. And by
removing potential issues. This is particularly important before the product reach the market.

The designer (in this case the person who applied the brand name on the van) did not consider
all the elements of the system (the Van itself). In particular that the van has a sliding door

Slide 37
Let see an example of good design but bad system perspective.
This is a good design solution:
• Blind people can play by perceiving the colours in Braille
• It is as difficult as the Rubik’s cube with colours

Why this is bad from a systems perspective?

Only people who know how to interpret Braille can play with it, therefore, this design could
promote social exclusion with Braille readers using it and non-Braille readers playing with the
classic Rubik’s cube

Slide 38
Sometimes there are simple solutions to solve issues. Moreover, this solution also solves
issues related to colour blindness. Therefore, from the technical point of view, we can say that
this system is more accessible: Accessibility is something that we are not going to deal with in
this course! But it is an important topic in System Design!

Slide 39
Responsibility and Responsible design
To adopt a system perspective also means to be aware that to design a product, a service, a
system or an intervention means to think ahead about the consequences.
Sometimes a ‘successful design’ is harmful to people, damaging to the environment, or bad
for our society.

Slide 40
The Design of Systems affects people life!
The opposite of responsible design is:
- a violation of the trust of end-user e.g., Dark patterns and Persuasive Evil Design
- lack of user research and testing that brings to unseen consequences of use “Opss…I
did not think of that?”

Slide 41
Example of design against crime ([Link]
These are products that aim (with simple solutions) at reducing crime or discourage criminal
behaviour.

Slide 42
We can originally summarise irresponsible Design as follows
- Intentional
o Lack of design ethics and Manipulative Design
o Lack of care regarding the impact of technology (effects on people, waste etc.)

- Unintentional
o Lack of research and understanding toward the user needs in the real world

Slide 43
What’s the problem here? (Dark Patterns Manipulative Design)
Dark Patterns are techniques to persuade you to do an activity that brings an advantage to the
company behind the design.

Slide 44
Unsustainable design
Agbogbloshie is a dump, outside Accra in Ghana, where thousands of computers, phones and
tablets fail to decompose, and poison the ground for decades to come

Slide 45
(Lazy design) = Politically correct: design affected by lack of research and
understanding toward the user needs in the real world.
This is an example of a service of shareable e-scooters in Atlanta (USA) that was not designed
appropriately. Because of the bad diffusion of the technology and service associated with that
in 2019 the city council was forced to heavily regulate the use of these artefacts
Slides 46
Agency and Responsibility is a big topic in the context of artificial intelligence. Which are the
potential effects on people of AI?

Slide 47
BREAK

Slide 48
Perception (CH 1)

Slide 49
Sensation and Perception
A sensation is the stimulation of our sensory receptors (vision, auditory, tactile etc.)
Sensory perception is the sensation information transferred to the brain
Perception is the brain interpretation of the sensation

IT IS IMPORTANT TO UNDERSTAND THAT WE:

 Perceive only a part of what we sense (we can not attend everything)
 Sense: only a part of what we perceive (our brain is limited) like in optical illusions

Slide 50
Your eyes contain rods & cones.
 Rods detect light intensity (No colours)
 Cones detect light colour.
Rods and cones are not distributed equally

Slide 51
Where the optical nerve intrudes in the eye, there is no retina and therefore no cones and rods.
This point of our visual system is called the blind spot because we cannot capture information
that is located in that spot

Slide 52
Blind Spot test

Slide 53
Something to think about: Most predators including humans have a stereoscopic vision to
gauge the distance to their prey as they hunt.
Most herbivores' eyes are on the sides of their heads so that they can see as much of their
environment as possible. They don't see the depth the way we do.

Slide 54
The binocular disparity, intended as the difference in image location of an object seen by the
left and right eyes, resulting from the eyes’ horizontal separation (parallax).
This binocular disparity is one of the most important pieces of information the visual centers
of the brain use to reconstruct the depth of a scene, and to estimates in milliseconds distance,
speed and dimension of objects

Slide 55
Today's VR headsets and AR glasses present two 2D images or superimposed the images in a
way that cues the viewer's brain to combine the images into the impression of a 3D scene.
This type of stereoscopic display causes what is known as a vergence-accommodation
conflict, which over time makes it harder for the viewer to fuse the images and causes
discomfort and eye fatigue

Slide 56
Can we always rely on what we see?
Our perception is a representation of the real world?
The answer is no!

Slide 57
In this example of Illusory motion
Our tiny eye movements and blinking cause the dots to look like they're moving…it is our
brain trying to make sense of what we are perceiving that creates the movement

Slide 58
We are discussing this part of the information theory model of Wickens and collegues

Slide 59
Our perception of the world around is not a true depiction of what is actually there.
Our perception is heavily biased by at least three factors:
• The past: our experience
• The present: the current context
• The future: our goals

We interpret the world around us in two concurrent ways: bottom up – We perceive what we
see; and top down we project expectations on what we are perceiving.

Slide 60
You see this Cube, and you recognise this as a cube because you can perceive the three faces
organised in a specific way which suggests you that this is a three-dimensional object that you
know with the name of cube…
Bottom-up processes are driven by stimuli/data from the world that trigger our senses.
Form the Bottom-up perspective: our perception is data driven, we collect data throughout our
sense and we used these data to process cognitively the information and to make decisions
regarding our behaviour

Slide 61
Top-down processes (Knowledge influences our perception).
Our conceptual data (knowledge, previous experience, etc.) drive our behavior, however, our
knowledge is also coming bottom-up (from the senses to our mind) creating a connetion
between Bottom-Up and Top-Down processes.
Slide 62
The Necker Cube illusion is a bistable visual illusion. The same sensory data are able to
support two perceptual inferences (one in which corner 1 is closest to the observer, the other
in which corner 2 is closest).

Slide 63
As suggested by Nour & Nour (2017)5 <<Perception is therefore the product of two sources
of information: the sensory data (bottom-up) and prior expectations (top-down) about the
sort of information that the sensory data contain. What happens, however, when prior
expectations are given too much weight?>>>

Slide 64 - 67
Our perception can be used to design and create multiple levels of perception of the same
objects (Example based on Gestalt Principles)

Slide 68
How something is presented (content, wording etc.) affect our perception! In this cases, we
may talk about the so-called “priming effect”.

In the example: if I am saying “Below you can see a map of 5 black shapes representing
different buildings.”

Slide 69
You will probably look at these elements in a separated way (as five figures of buildings)

Slide 70
However, if I will tell you that this is the advert of the association LIFE, it should be easier for
you to perceive the overall picture

Slide 71
Perceptual patterns (or mental frames): we have expectations about objects or events that
are usually encountered in a situation. We tend to see these also when they are not there.
Perceptual Priming
The context (how something is presented) affects how we perceived a stimulus – e.g., a text, a
message, a picture and object.

Slide 72
Why our perception could be easily tricked?

5
Nour, M. M., & Nour, J. M. (2017). Perceptual distortions and deceptions: what computers can teach
us. BJPsych bulletin, 41(1), 37–40. [Link]
Slide 73
We heavily rely on our visual ability for a large part of our activities.
It is important to notice that on top of those phenomena that may bias our perception (such
as priming and frames) other factors affect our visual perception:
 We have seen the importance of Cones and Rods in our eyes (and the blind spot) and how
these affect how we see and what we see.
 Moreover, reflectance and brightness of the light affect how we perceive an object, and
finally, the perception of objects (size and depth) is also affected by the surrounded
environment of the objects.

Slide 74Reflectance and brightness

Here you see an example of the Checker shadow illusion (Adelson, 2005) that shows how
reflectance and brightness of the light may affect our perception of an object

Slide 75
Moreover, as showed by the Ebbinghaus and Titchener illusion (Ebbinghaus, 1902, Titchener,
1901) we can perceived objects with different sizes because of their surrounding elements

Slide 76
The real question is not why we can fall into a trick, but if our brain is lying to us?
The answer is: we can often see things that are not there, so yes the brain is lying to us, but
maybe there is a reason!
Maybe what your retina gets is not what the brain perceives but sometimes it is very useful…

Slide 77
Pattern recognition and the ability to see the whole (even when it is not there) it is an
extraordinary capacity. Yes, this brings us to fall into illusions, but it also enables us to see
beyond and sometimes to anticipate issues, like in this case.

Slide 77
Recap about perception

Slide 78
Affordances and constraints and systems

Slide 79
We (as humans) have mental and physical constraints. Concurrently designed systems have
constraints (physical and digital)
The combination of our limitations with the limitations of the products determine:
• how we perceive and recognise a product and its functions
• how to use it, and how to not use it
• how we understand and learn how to use it correctly
Slide 80
Because of years of evolution, we have learned quite well to use objects around us in all the
possible ways, sometimes in very creative ways, some others in a very dangerous way.
To design constraints means to make clear at the perceptual level what can be done and what
can not be done in and with a system. Therefore, like in this case we are introducing physical
limitations to prevent a certain potential use (an affordance).

Slide 81
Gibson defined affordances as:
<<Perceived and actual properties of a thing… that determine just how the thing could
possibly be used>>6
These are “implicit” indications of the potential use of a system. When a designer takes
advantage of affordances, the user knows what to do with an object just by looking at it
without labels or instructions.

Limitations and affordances are strongly related.

Someone may oversimplify by saying that affordances are the opposite of limitations and
constraints. But affordances are actually built upon limitations and constraints.

If we look at the rock in the picture. In terms of Affordances: We can seat on the rock, eat on
it, enjoy the sun and the view…
In terms of Limitations: We have to pay attention because we may fall off the rock

Slide 82
Designed objects have limitations and constraints.
As Donald Norman suggests these limitations are actually very useful to suggest to people
what they can do, and what they cannot do with an object.
There are four main types of limitations and constraints:
• Physical limitations: The physical aspect of an object usually suggests how to use
the object. Door keys can be inserted in keyholes. And usually, Keys can only be
inserted into keyholes vertically, but you can still insert the key upside down (so you
may still make a mistake).
• Semantic constraints. Objects have textual indications, symbols, labelling, colours
and icons to convey messages to users.
We know, for instance, that red lights mean stop and green lights mean go, so we
infer that a red light means a device is off or inoperative, and a green light means it’s
on or ready to function.

Slide 83
3. Cultural constraints (conventions). We have rules/ traditions/habits that are based on our
culture. Somehow we all know without being told that we’re supposed to face forward in the
elevator. These “conventions” can be used by designers to design systems for a specific
context of use.

6
Gibson (1979): The Ecological Approach to Visual Perception. New Jersey, USA, Lawrence
Erlbaum Associates
4. Logical constraints. Logic and relationship among elements of a system can be used to
communicate (without instructions) how to use a system. When we read something on a
digital tool we know that we have to scroll down when there is a scroll bar on the right of the
screen, to see objects located below the visible set.

Slide 84
Signifiers
A signifier is an indicator of some sort (semantic constraints).
In our tap example, this might be red/blue dots signifying which way to turn the tap to
get hot or cold water.

Physical constraint + Logical constraints. Move up to open and to the right or to the left to
decide the temperature of the water.

The affordance: the tap/faucet can run hot or cold water (opportunities)

Slide 85
To interact WITH and IN a System
- End-users have to be able to recognise and remember the limitations.
- Affordances are the opportunities of use (or the potential use). These opportunities are
suggested by the system just by looking at it.

Slide 86
Let see an example.
If we consider the door in the picture, We can say that in terms of affordances:
1. We may open and close the door using the handle.
2. The handle (with its shape, size and location) suggests a relationship between it and the
hand of a standing person (push the handle down).
3. In front of the handle, there is also a lock (but it is not clear from the picture how it works).

However, we may also assume that the door can be locked and unlooked.
In terms of constraints:
• the left frame-blocks and prevents the door from moving forward. This physical
limitation informs people that they should push, not pull.
• The handle of the doors usually (convention) can only be pushed down.

Slide 87
Of course how we used affordances and how we design affordances is important. Bad design
does not appropriately use affordances. Like in these pictures.

<<If your room is 210, where you have to go? To the left or to the right?>>>

Slide 88
Question time
Slide 1
L2
High graded answer to the activity L1
Define
The Necker's cube is a bistable visual illusion. The same sensory data can support two
perceptual inferences (one in which corner 1 is closest to the observer, the other in which
corner 2 is closest).
Explain
This exemplifies that perception, as also suggested by Nour & Nour (2017) is the product of
two sources of information: the sensory data (bottom-up) and prior expectations (top-down).
Conclude
In this case, there is a conflict between these two modalities bottom-up and top-down that
could bring to perceive the cube heading towards different directions on the basis of which
modality dominates in a specific moment.

Why this answer is good

+ It answers the question (minimal mandatory requirement)
+ Terminology is correct (correctness)
+ Provide background and refer to the material (appropriateness)
+ It presents the answer in a short, logic and consistent way (logic and
consistency)

Slide 2
Agenda
• Usability and UX Methods and Definition(s)
• Tools and methods for ideation and testing

Slide 3
The decision making that you perform when you are going to design a process
or a product will be always affected by the context of use, the human and
psychological factors involved.
If you do not have enough information (about the context, the process of use, the
people involved etc.) well you could risk designing something (Like the wide
research for a missing woman at night in Iceland) that will be not useful by
wasting resources and energy.

This is the reason why user research, user experience and usability assessment
are important!

Slide 4
User experience: Definition, Issues and solutions

Slide 5
The ISO 9241-210 Defines the concept of User experience as a process in which
people perceptions and responses that result from the use and/or anticipated use
of a product, system or service.

This means that to measure UX we need to consider end-user:

• emotions,
• beliefs,
• preferences,
• perceptions,
• physical and psychological responses,
• behaviours and accomplishments

that occur before, during and after the use of a product or service>>
Anticipated use: Refer to people believes, attitude and Trust toward a product

Slide 6
User experience is different from Usability.

User Experience is focused on creating first value for the end-users, UX

plays on usefulness for people and on their emotion to create and promotes
positive emotions toward a service or a product.

Usability is more about safety, learnability etc.

Slide 7
However, A service/product with poor usability is more likely to provide a low
level of UX
Usability remains vital, and yet the interface-centred methods and perspectives
of human-computer interaction do not address all dimensions of web design. In
short, usability is necessary but not sufficient.

But UX also includes

Usefulness. As practitioners, we can’t be content to paint within the lines drawn
by managers. We must have the courage and creativity to ask whether our
products and systems are useful and to apply our knowledge of craft + medium
to define innovative solutions that are more useful.

Products and systems should be:

Desirable. Our quest for efficiency must be tempered by an appreciation for the
power and value of image, identity, brand, and other elements of emotional
design.
Findable. We must strive to design navigable web sites and locatable objects, so
users can find what they need.
Accessible. Just as our buildings have elevators and ramps, our web sites should
be accessible to people with disabilities (more than 10% of the population).
Today, it’s good business and the ethical thing to do. Eventually, it will become
the law.
Credible. Thanks to the Web Credibility Project, we’re beginning to understand
the design elements that influence whether users trust and believe what we tell
them.
Valuable. Our sites must deliver value to our stakeholders (not only the users).
For non-profits, the user experience must advance the mission. With for-profits,
it must contribute to the bottom line and improve customer satisfaction.

Slide 8
Key points about interaction assessment

Slide 9
User interface (UI) design and User Experience(UX) are two connected but
different aspects. UI is more about design and UX is more about “quantifying”
the experience of people with the product including the experience provided by
the UI
Slide 10
A well-prepared designer can do User Interface
But nobody can do UX alone!

Slide 11
Designers can not deal with UX alone! Even if they know how to perform UX
research. Only a team of experts can do that introducing in the design process
multiple perspectives and insights from in filed research that designers may use
to drive their creative process

Slide 12
Quite often we invest a significant amount of money to develop solutions for
problems that nobody has!

Slide 13
To provide a good experience means to know the needs of the end-users and
stakeholders and to provide value to them in their context.
We need to ask our self
Who really are our users?
What makes them tick?
What are they are hoping to accomplish?
Which are the constraints in the process of use and in the context?

Slide 14
Even stressful events, like performing an MRI scan, can be improved if we
combine usability to improve the experience of the end-users.
This does not mean that we do not have to pose the right question during the
development:
Is this product cost-effective?
it is safe to use?
it reduces the workload, the stress or facilitates people life, and well-being?
it increases the satisfaction of all the stakeholders involved.

These questions are not secondary ones. We need to know the answers to these
questions at the early stages of design!
Slide 15
End-Users
People who are using (actively or passively) a technology or service

Stakeholders
People who are affected or are affected by technology or a service

Example: An assistive technology that is right for the end-users, but bad for the
caregiver is more likely to be abandon

Slide 16
Affordances are perceived possibilities of action (Norman, 1988)
Affordances are the visual clue to explain without a verbal explanation the
function of the object (an opportunity for action)

Slide 17
Good affordances enable users to interact in a natural and comprehensible way

Slide 18
Affordances and Active users
If we do not design a clear pathway for the end-users with signifiers and
perceivable affordances and constraints, the end-users will find their way to use
a system.

Slide 19
How you designing and presenting information is important
To have a prototype of your product, or to design appropriate stimuli for your
experiment, is important because what you show to your participants and how
(methods and questions) is going to affect the reaction of the participants.
How you design the process of use, the space between elements, their
relationship, the colours, the labels and the words you are using to present the
contents are going to convey different information and affordances to
participants.
If you look at this example of the USA electoral map it seems that the RED
(Republican) received more votes than the BLUE (Democrats) but if you
visualize the actual vote of people in a different modality the visual information
is significantly different.
We are priming and biasing people by design, we need to be aware of that and to
control for that.

Slide 20
Questions to answer at early stages of development
THE WHO AND WHAT OF THE SOLUTION
• For who this tech/service is NOT designed for?
• Who are our users? And who are the stakeholders?
• What makes them tick?
• What are they hoping to accomplish? What are the tasks?

THE WHY OF THIS SOLUTION

• What are the social needs?
• There is value in the service/product?
• Is this product cost-effective?
• it reduces the workload, the stress or facilitates people life, and well-being?
• Does it increase the satisfaction of all the stakeholders involved?

THE HOW OF THE SOLUTION

• What happens when it goes wrong? it is safe to use?
• Does it improve the overall experience?
• Does it provide value and usefulness?

Slide 21
How to answer the questions: Testing, testing, testing…

Slide 22
User research at different points
UX is a process
Cannot be measured with one score, or with one method or in one specific
moment. You need a plan!

Usability is property
Quality of interaction

Usability testing occurs during and after the use of a product

Slide 23
User testing is mainly for products already in the market. But some techniques
could be used also in concept/prototyping phases
UX evaluation requires at least a fully functional prototype. However, aspects
such as the intention of use and perceived usefulness which are important to the
overall experience could be measured with raw concepts or mock-ups

Slide 24
Please note that:
USABILITY TESTING GOAL IS TO ASSESS THE INTERACTION
QUALITY: the fact that usability testing also provides evidence to redesign
is a consequence, or a secondary goal, not the main objective of the usability
testing.

Slide 25
The landscape of User Research methods
The graphic presentation shows here methods from the more Qualitative and
attitudinal to the more quantitative and behavioural

[Link]

Slide 26
In this link, you can find useful resources to drive you in the selection of
appropriate methods of assessment: [Link]

Slide 27
A meaningful UX evaluation: Requires multiple evaluation points with
objective and subjective measures, and well-defined procedures to collect data
over time to support design and redesign

Usability: requires a set of well-planned moments of review and test at different

stages (formative or summative)

Slide 28
Usability definition
Slide 29
Usability is usually described in terms of Effectiveness, Efficiency, Satisfaction
in line with the ISO9241-11, 2018:

The extent to which a product can be used by specified users to achieve

specified goals with effectiveness, efficiency and satisfaction in a specified
context of use.

Slide 30
What usually is not reported in usability studies is The context of use!
THIS IS A SERIOUS LIMITATION! No context= no possibility to
replicate

This Accounts for the specificity of the test and the type of participants
• Individual variables (attitudes, habits previous experiences, expectations)
are collected.
• which type of tasks and scenarios you asked participants to perform, how
these tasks were presented to the end-users.
• Which equipment did you use for the test e.g. the same PC or laptop, which
operative system, touch or mouse interaction?
• Which size of the screen/monitor?
• If it is a physical prototype, how this was presented? Which dimensions?
Did you record the audio and the video?
• With which tool for audio recordings?

Slide 31
Context
<< The information about the users (demographic information, their attitudes
preferences etc.), the tasks that they have to perform, information about the
physical and social environments in which a product is used >>

What to report about the context you simulated in the test e.g.:
Data about participants (how much they represent the stakeholders)
questionnaires about users attitudes, preferences, competences
the qualitative observation about people behaviour, attitudes
description of the tasks and their reasons,
information about equipment used in the test,
the observation about the usage of the product in the real environment of use.
Slide 32
Effectiveness
<<Accuracy and completeness with which users achieve certain goals.
Indicators of effectiveness include the quality of solution and error rates.>>
Measured by e.g.:
NUMBER OF ERRORS PERFORMED or VERBALISED,
ISSUES EXPERIENCED,
MISSING TASKS

Slide 33
Efficiency
Measured e.g.:
TIME TO PERFORM CORRECTLY
TASK COMPLETION
LEARNING TIME
This could also include the measure of stress and cognitive workload see for
instance the NASA-TLX or the Rating Scale Mental Effort (RSME)

Slide 34
Satisfaction
<<Extent to which the user's physical, cognitive and emotional responses that
result from the use of a system, product or service meet the user’s needs and
expectations. Satisfaction includes the extent to which the user experience that
results from actual use meets the user’s needs and expectations. Anticipated use
(expectations) can influence satisfaction with actual use>>

Measured e.g.:
STANDARDISED SATISFACTION QUESTIONNAIRE.
This could include also measure of likelihood to recommend

Slide 35
Satisfaction in the context of usability is a very specific concept:
TO MEASURE SATISFACTION YOU MUST USE A STANDARDISE AND
RELIABLE SCALE
To ask qualitative questions regarding preferences, pleasure etc. is also possible
at the end of the usability test but these questions are not a measure of
satisfaction. Just additional information to inform about design preferences

Slide 36
There are free scales you can use
• System Usability Scale (SUS)
[Link]
[Link]

• Usability Metric for User Experience (UMUX)

Finstad, K. (2010). The usability metric for user experience. Interacting with
Computers, 22(5), 323-327.

• User experience questionnaire (UEQ)

[Link]

These scales are validated and reliable, and each one of this scale has a
procedure to calculate and report an overall score of satisfaction from 0 to 100

Slide 37
Three variations of the USABILITY definition (9241-11) to enable the
application with different products. Software, medical devices, Everyday
products

Slide 38
We identified 54 standards which refer to ISO 9241-11. All these standards refer
to the usability umbrella as the framework to assess the interaction with 19
different types of products

Slide 39
When it comes to software, Usability is a sub-set of the quality of interaction:
• Efficiency
• Effectiveness
 • Satisfaction
• Context of use
• Appropriateness recognisability,
• Learnability,
• Operability,
• User error protection,
• User interface aesthetics,
• Accessibility

Slide 40
UX and Usability: Test what? How? When?

Slide 41
As represented in the picture, different methods of assessment are associated
with the development phase of a product

Slide 42
BREAK

Slide 43
Ideation phase: Inform the design and the requirements (context analysis)

Slide 44
Some methods for the ideation phase: Observation, interview, focus group, card
sorting, survey

Slide 45
In focus groups or interview you want to get out information about all the
possibilities of stakeholders

A useful way to gather information about who are the stakeholders is to use
Matrix Clarification Tool1.

1
See pages from 1 to 8 of this document
[Link]
.pdf
This is a simple way to ask during interview or survey regarding all the
people that can benefit from, influence the adoption of technology and their
importance

- INFLUENCE: the power stakeholders have over the adoption and usage
of a tool
- IMPORTANCE: how important is that the product is attractive to solve
the problem of that stakeholder

Slide 46
Technology and services never work in isolation you need to map the process of
use from different perspectives by:
- Observation
- System mapping with stakeholders
- User Journey
- Card sorting to organise contents
- Survey

Moreover, experts analysis with Cognitive walkthrough, heuristic evaluation and

task analysis are associated with system mapping process with stakeholders
which aim to map from a different perspective the use of the product in
(complex) sociotechnical environments

Slide 47
Design and be aware of key tasks and scenarios
From concept to prototype

Slide 48
In the slide, you have a summary of methods for the concept/prototype phase

Slide 49
Task analysis
There are different types of task analysis e.g.,
• Hierarchical looks at the relationship among tasks in the process
• Cognitive looks at the decisions and cognitive effort to perform the tasks
We will see an example in the case study!
For more info:
[Link]
[Link]

Slide 50
Tasks and goals
An interactive system requires a certain set of actions to achieve a goal. For
instance, in this example, a user wants to login into a system.
If you know the system you also know the sequence of correct actions that
people should perform, so that:
• If participants do something outside of the list of task = You can report a
usability issue
• If participants say that they do not know what to do = You can report a
usability issue
• If participants perform the steps but there is a technical problem = You can
report a usability issue

Slide 51
Usually, when you involve participants you present the tasks in the form of
scenarios in which you do not say what they have to do but what they want to
achieve (goal)

Slide 52
Iterative Cycle of Design
After that you have defined the product (ideation) You will start by defining
tasks and performing a review e.g., Heuristics principles.
This will provide you with a set of potential interactive issues that you will try to
solve by a redesign (prototype). Then you could perform a usability test that will
provide you with a list of issues experienced by participants. This will enable
you to comment about the usability of your prototype and to take further steps.

Slide 53
Inspection methods
You can explore the limitations of a system by using principles to rapidly inform
the redesign (e.g., heuristics) but this will be never as powerful as the user
testing. We will discuss this type of assessment in L3

Slide 54
Test with users (Prototype or product)

Slide 55
To involve people and let them interact with a prototype or a product IS NOT
ALWAYS equal to usability testing. For instance, in a participatory design, you
can invite end-users to redesign a product with you, this is a powerful way to
generate ideas and to improve the design, but it is not a form of assessment

Slide 56
There is always a distance between manufacturers and stakeholders point of
view. To reduce this distance it is necessary to show to manufacturers and to the
stakeholders the other perspective.

Slide 57
DIFFERENTLY FROM OTHER METHODS ONLY WITH USABILITY
TESTING:
you are exploring in a formal and reliable/replicable way the user performance
with the product /prototype to measure the usability e.g., efficiency,
effectiveness and satisfaction.

THERE ARE NO OTHER WAYS TO FULLY MEASURE USABILITY!

Slide 58
Why usability is important for every product
[Link]

Slide 59
There are different forms of Usability testing in-presence or in a remote. How
you set up your protocol is important, because you need to collect enough data
to advance your design in a reliable way
Slide 60
Effectiveness: missing steps, verbalization of problems (Verbalization could be
collected during (Concurrent) or after the interaction (retrospective)), issues in
understanding how to progress toward the goal, time to achieve the task
Efficiency: Time to perform correctly or to fail the task
Perceived satisfaction: Satisfaction scales, other important factors for the
specific product e.g., intention to promote, workload etc.
Qualitative perspective: User preference (verbal) and qualitative comments
(during the debriefs)

Slide 61
Level of prototype
When you do not have a functional product you can use a wizard of Oz
technique:
<<user interact with a product and responses are (all or partially)
generated by a human behind-the-scenes.>>

Slide 62
Wizard of Oz
Is a method of testing a system that does not exist (the listening typewriter, IBM
1984)

Human ‘wizard’ simulates system response:

• interprets user input according to an algorithm
• controls computer to simulate the appropriate output
• uses a real or mock interface
• wizard sometimes visible, sometimes hidden

good for:
adding simulated and complex vertical functionality
testing futuristic ideas

Slide 63
Advantages
Data are used to inform how to progress the design before implementing a fully
functional prototype
Good for A/B testing
Good as a driver for focus group and expert review

Disadvantages
Sometimes is less expensive to have a functional prototype (even not
aesthetically pleasing)
Data about efficiency, effectiveness and satisfaction could be affected by the
simulation and could be not representative of the real interaction with the
product.

Slide 64 Of usability test of fully functional prototype

Advantages
Data are used to inform how to progress the design by controlling the usability
You can model the likelihood of people finding issues

Disadvantages
Time and costs

Slide 65
Tips to organise users testing

Slide 66
The starting point of your evaluation should be a good (set of) question(s):

What you want out of your assessment?

Why you are doing the evaluation?

Slide 67
In a test:
You are empirically exploring a system with the end-users observing what
participants are doing, their reaction and the problems they are experiencing or
verbalise in order to achieve a task.

Slide 68
What you are testing?
Are people able to perform correctly the tasks presented as scenarios?

• If participants…
… do something outside of the list of correct steps= Issue
… say that they do not know what to do = Issue
… perform the correct steps but there is an technical problem = Issue
… can not achieve the goal or fail to achieve it= Issue
… verbalised issues due to e.g. information organisation = Issue
… spend an unreasonable amount of time to achieve the task = Issue

Everything that prevents a smooth interaction in terms of time and performance

is a usability issue

Slide 69
You test to obtain data, such as: time to perform, task achievement, task
failure, number and types of errors, verbalised issues, frustration/ satisfaction

These will help you to identify elements of the system/prototype which may be
improved.

Slide 70
Goal:
Identify if something does not work; Test new functions; Test uncertainty;
Decide how to redesign the system in future

Set up the procedure:

-Tasks that people have to perform (know when participants are right or wrong)
-Type of data you want to gather;
-Type of questions you are going to ask.

Minimal equipment
• Conference system
• Interactive Prototype link
• Participant: device PC/Mobile/Tablet
• Audio and video recording
Slide 71
Test don’t judge
You aim to enable a good interactive experience! You are doing a test not to
assess (how good is your product or how terrible are end-users) but to get
information to better support your decision making about the design of your
product.

Slide 72
You may choose different approaches to let people verbalise their
experience:

• Let people talk aloud during the interaction (Concurrent thinking aloud)
• Ask people to interact, video record their screen actions and audio. Ask them
to watch a video of their interaction, and to comment about issues
(Retrospective thinking aloud)

Slide 73
Each usability test is composed at least by the three following three
elements:
- Consent and questionnaire before the interaction (demographic and other
aspects you want to collect)
- Interaction with the product (observation, data recording) and a technique
to enable participants to verbalise issues. Here you should use your
scenarios!
- Debrief phase for final remarks of participants e.g., measures of
satisfaction, cognitive workload and likelihood to use scale etc.

Slide 74
Evaluation protocol
You should know since from the beginning:
• What you want out of the test
• How and why you are going to record and use the data you will collect
(each measure counts!)
• How you are going to report these data in your report.

Try to put down on paper your procedure:

- What participants will do from when they arrive, since when they will
finish the assessment
 - The order of the events: Questionnaires, the order of tasks, etc.
- What you will do: who will observe, who will support the participant,
who will record etc.
- How you will do that: e.g., ask participants to verbalise the issues? how
you are going to answer if they have problems?
- Which equipment you will use to collect data
- How you will store data
- How you are going to analyse the data

Slide 75
As an evaluator you will assist the participant remotely, you should drive and
keep people in the test, but you can not affect people decisions.

Slide 76
SOME EXAMPLE OF DEALING WITH PARTICIPANTS:
If participants struggle to find something or to progress. Do not help them!
You can ask:
Do you remember your goal?
What are you thinking?
What are you doing right now?

If you notice that participants get frustrated because they can not achieve the
goal?
You can say:
<<You believe that this task is achievable? If not, we can go to the next one?
Remember that if the system is not good enough to enable you to find something
is not your fault!>>
<<You are the end-user, your opinion is what matters if you believe that
something is not doable, for us this is already a valuable result.>>

Slide 77
Tips for the testers
• Define the critical and key tasks of the product
• Design the test procedure
• Present the tasks in terms of reliable scenarios to participants
 • Observe, record data, take notes
• Always add standardised evaluation scales –this does not mean that you
can not use qualitative scales tailored on your product, just also add
standardised scales to enable benchmarking and replicability
• Use the data to inform how to improve your redesign

Slide 78
Remember that a usability report should contain
+ data about effectiveness e.g. Errors, Issues, failed tasks
+ data about efficiency e.g. time to perform
+ data about satisfaction e.g. SUS
+ data about participants
+ how you tested (scenarios, tasks, information to replicate your test)

Slide 79
Your report should also explain how you selected your evaluation methods e.g.,
Because :
+ Suit your constraints (budget/time/capacity)
+ May inform the design phase
+ Reliable insights at low costs

Slide 80
Summary
+UX requires a systems perspective
+ UX is a process
+ UX & usability go together
+ Evaluation requires expertise
A designer with some expertise in evaluation and research is not equal to a
User Experience Researcher. Evaluation is not easy! Bad or partial
evaluation and analysis bring to disaster!
Slide 81
END
Slide 1
L3

Slide 2
Case Study
From the article of Borsci et al. 2018 Usability study of pH strips for nasogastric
tube placement

Slide 3
Let’s start with the contextual Analysis

Slide 4
This case study was about the insertion of nasogastric tubes for feeding and
providing medication to patients.
Here you can see an explanation of the procedure.
[Link]

Slide 5
People (elderly, baby and trauma patients) that can not eat for a medical
condition are usually fed and take medicine by Nasogastric tubes inserted
through the nose in the stomach.
The tube could ends-up:
• in the oesophagus (risk of choking people in a coma)
• in the lungs (risk of food in the lungs)
• Tubes inserted in the stomach could migrate or could be moved by patients
when sleeping or for other reasons

Slide 6
Nurses (the ones who really run hospitals) are well trained in avoiding choking
but they need something to confirm that the tube is in the stomach (after the
insertion and over multiple time on a day before every feeding).
There two solutions:
xRay (expensive and dangerous after multiple expositions) and pH strips
Slide 7
X-Ray
Enables to visualise the tube but radio-opacity of tubes is not always
satisfactory, and more than 50% of tubes could either migrate from the stomach
or be inadvertently removed by patients after the x-ray has been taken.
To do multiple x-rays per day or weekly on patients is not cost-effective nor safe
for people well being.

Slide 8
Risk and limitations of pH strips
- No visual confirmation but cost low (1p per strip)
- can be performed multiple times but requires to aspirate gastric contents (not
always possible)
- Errors in use (reading) the pH could be up to 30%

Slide 9
2. Proposed innovation(ideation phase)

Slide 10
We identified 3 major issues with the use of pH strips in clinical settings by
an expert review
• Transfer problem:
• pH strips were developed for lab use (with a high level of light) and
mainly for testing acidity of chemical solutions i.e. these were not
mean to be used in clinical settings with humans

• Clinical related problem:

• Current strip only consider pH -> if patients are taking antacid this
test could be compromised

• The strips as a system:

• currently, recommended strips have 16 possible results presented by
multiple colours (not accessible)
 • To read the result it is required to interpret multiple colours strips and
this could add unnecessary complexity, errors and subjectivity

Slide 11
To solve the issues we identified 3 main requirements for a new strip
- The strip should detect an enzyme which originates only from the stomach
(this could solve the clinical issue)
- A single colour strip: To provide a Yes or No answer to the question: “the
NG tube is in the stomach?” this may reduce the likelihood of errors and
increase perceived usability//effort of decision making (this could solve the
system issue)

Slide 12
How to implement the requirements opened a set of questions e.g.,
• How current strips are used in the field and major issues?
• Professionals attitude and trust in use of current pH strips?
• How much usable are the current pH strips?
• Professionals perspective on the concept (Enzymatic strip)?
• How the Enzymatic strips have to look like? Is colour an issue?

Slide 13
Evaluation protocol
To answer our questions we set up an evaluation protocol

Slide 14
As reported in the slide:
After an initial observation of 24 patients treatments in the hospital (phase 1).
We involved 178 participants in different phases of interview to map and
understand the context of use and the issues in using x-Ray and pH strips We
also surveyed their preferences of design regarding the concept of a new set of
pH strip.
Finally, we performed a simulation test to understand the likelihood of errors in
interpreting the results of pH strips, and a usability test to compare readings
carried out by normal and redesigned Enzymatic Strips.
Slide 15
Let’s look at how we mapped the context of use:
• The process of use of pH strips is not as easy as it seems
• Previous studies showed that in 44% of the cases fluid could not be aspirated
or pH was >5.0 (Taylor 2014).
• Confirmation of tube position by X-ray remains more common than pH -
51% compared with 45% (Taylor 2015)

Slide 16
Observation and interviews: Task analysis
Actually, our task analysis showed a long process composed of nineteen steps
with an if cycle toward the end of the decision-making process of inserting the
tube, double check or send the patient for an xRay

Slide 17
Concerning outcomes from the interviews

Interviews with nurses and professionals showed some concerning outcomes

• 68.4% of professionals stated that when the tube is inserted, in addition to pH

strips confirmation, they used both visual cues and also auscultation (although
these are not generally seen as reliable methods)

• 31.6% of professionals stated that they used jelly or other lubricants, instead
of only water to facilitate the insertion

• In 35% of the cases, the correct pH strips for NG-tube check are not available
in the ward, and professionals have to use a set of incorrect pH strips to
perform the procedure.

Slide 18
Concerning Outcomes from observation
Time to start feeding/medication on average was:
• After a pH strip confirmation (average): 4 hours and 30 minutes
• After an x-Ray request (average): 17 hours and 18 minutes.

After the results from a pH strip, a clinician should decide what to do end
clinicians are not always there. When the confirmation is done by an x-Ray the
x-Ray should be booked, the patient should be moved to the x-ray room, and
experts radiologists should read the results, this process adds time to the
decision making

Slide 19
Data from the survey and interviews
Acceptance analysis of the current pH strip for the NG tube intubation revealed
some issues in terms of anxiety of use and availability of the correct strips in the
field

Slide 20
Data from the survey and interviews
Clinicians were optimist regarding the proposed solution (enzymatic strip)

Slide 21
Simulation test
Our simulation test was performed by asking participants to read 15 pH strips
showed on a screen. Participants were asked to report the pH value of the strip

Slide 22
Results of the simulation test
We identified no significant difference in the performance of reading pH strips
between Health professionals and Lay users with a likelihood of error around
10%.

Slide 23
Factors such as age, expertise, acceptance and attitude do not affect errors when
reading pH strips.
Trust in the use of pH strips affects the number of individual errors in reading
the pH strips (p< 0.5)

Slide 24
The enzymatic strip
Slide 25
Preference of design of the strip:
- 36% preferred the option blue-yellow
- 64%preferred the red-green option

56% of participants expressed concern about the Red and Green solution
<<Blue/yellow better for colour blind, fluid can contain green and red elements
this could be problematic for green/red strip blue/yellow is better>>

<<A control is needed, when the strip is impregnated and the colour does not
change, how I can be sure that the strip is working?>>

Slide 26
Usability test (Real-life simulation):
 Professionals were randomly assigned to 10 impregnated strips at a
different level of pH (5 new strips, 5 normal strips)
 Professionals performed the reading and reported a decision:
 Recheck/retest,
 Feed
 Send the patient for xRay;

Slide 27
Usability test
The 10 stimuli were randomised

Slide 28
Results showed that with the new strip there was a substantial reduction of
unnecessary x-Ray (72% - this may allow saving 32 of the 42 unnecessary xRay
procedures)

Slide 29
Thanks to this UX and usability analysis the project was awarded 2M
Strips are tested in 10 hospitals in the UK (results end of 2022)
Slide 30
Information seekers (CH3)

Slide 31
SEEK FOR THE STRUCTURE
Your handbook stresses that idea that we are seeking for an organised visual
structure. One of the most important goals in structuring information is to
provide a visual hierarchy by arranging:
- Label and break information in sections and subsections
- Provide a visual hierarchy that enables people, when scanning information, to
separate what is relevant to their goals from what is irrelevant i.e., focus
their attention on the relevant information

This organization helps people to deploy strategies to search for information and
achieve they have goals in mind. So that if they find what they are looking for
more quickly because they can easily skip everything else.

Slide 32
Studies1 show that people usually adopt as a strategy on websites both browsing
and searching for information.
Looking at information, design and labelling on an interface, the users can
predict what they will find if they pursue a certain path.
We are able to follow the scent of information

Information Foraging Theory assumes that, when searching for information,

humans are relying on a foraging mechanism that evolved to help our ancestors
find food

Slide 33
How we present information (organisation and aesthetics) makes the difference
in terms of first impressions

We usually have 50 milliseconds to make a good impression with an interface

(Lindgaard et al 2006) 2.

1
Katz, M. A., & Byrne, M. D. (2003). Effects of scent and breadth on use of site-
specific search on e-commerce Web sites
2
Lindgaard, G., Fernandes, G., Dudek, C., & Brown, J. (2006). Attention web
designers: You have 50 milliseconds to make a good first
In this amount of time end-users are assessing the visual complexity and the
prototypically of the information.

After 17milliseconds end-users already knows if a product is complex or simple

visually
After 50milliseconds end-users knows if a product is prototypical or Familiar
(the amount to which an object is representative of a class of objects)

In this example from Tuch and colleagues3 (2012) you can see different website
interfaces categorised in terms of visual complexity (VC) and prototypically
(PT). Where the interface with low complexity and high familiarity or
prototypically is considered the best solution

Slide 34
Design principles and information
As we are information seekers and designers should facilitate the process of
following the scent of information. In this sense, to use principles of design is
often a very good way to review a system before we involve participants.
If principles of design are violated probably prototypically and aesthetics are
usually low and, therefore, it is hard to follow the scent of information

Slide 35
Break

Slide 36
Usability Inspection Methods
How can we make it easier for people to find information?

Slide 37
As suggested by J Nielsen ([Link]
a-heuristic-evaluation/)

3
Tuch, A. N., Presslaber, E. E., Stöcklin, M., Opwis, K., & Bargas-Avila, J. a.
(2012). The role of visual complexity and prototypicality regarding first
impression of websites: Working towards understanding aesthetic judgments.
Heuristic evaluation involves having a small set of evaluators examine the
interface and judge its compliance with recognized usability principles

Slide 38
Several Lists of Heuristics principles were generated over the years to help
designers and practitioners to check and inspect their design before the
interaction with people
The 10 Heuristics of Nielsen are largely used for web design
[Link]
The Principles of Tognazzini are broader and more inclusive of the ones of
Nielsen and could be used for software ([Link]
interaction-design/)
The Wickens principles are based on cognitive principles by aiming at
facilitating the use of interactive displays

We are not going to discuss these lists, but we are going to look at some
principles of these lists, for instance, the principle “recognition is better than
recall” when we are going to talk about memory and attention.

Slide 39
If you think about that you can inspect any product using any solid theoretical
principle, to see if a system follows certain rules

Slide 40
Activity

Slide 41
Affordances
• People can open and close the flame by the knobs.
• People can control the intensity of the flame by the knobs.
• When the stove is not working, the top part of the stove could be used as an
additional surface to place kitchen tools

Limitations
• In terms of physical constraints: each knob is associated with a specific
flame, and the knobs should be turned toward a certain direction to open and
close.
• However, (semantic constraints) there are no signifiers to indicate the
direction in which you can turn the knobs, nor if you should press to open the
gas.
• As a convention (cultural constraints) you can usually turn knob
anticlockwise, but no indications are provided.
• Clear mapping (logical constraints) of the association between knobs and
flames of the stove is missing in the design.
[See the slide]

Slide 42
Card Sorting4
Another way to simplify a product in terms of the architecture of information is
to ask the end-users how they associate or cluster together information.
If you want to know how you can organise the information you can ask people
to organise the contents on an interface (usually you provide them with cards
with a list of categories close card sorting, or you ask them to first identify
categories (open card sorting). Then you ask them to organise the categories
hierarchically.
You can then build a matrix with the answer of all the participants and look at
and quantify how much participants considered certain pieces of information
associated with other information.
This is a good way to enable people to elicit their mental model.

Slide 43
Gestalt Principles

Slide 44
Our brain is an exceptional tool that evolved to detect patterns. We know
that thanks to the Gestalt school which operated during the 20ies

Gestalt ("shape, form“). The findings of the Gestal School suggested that
We perceive objects as well-organized patterns rather than separate components.

The motto of the GESTALT school was: “The whole is greater than the sum of
it’s parts”

4
Schmettow, M., & Sommer, J. (2016). Linking card sorting to browsing
performance–are congruent municipal websites more efficient to
use?. Behaviour & information technology, 35(6), 452-470
Gestalt principles offer a framework and described rules that govern human
perception.
• Gestalt psychology is descriptive: it doesn’t try to explain why we have
these rules or how they come into being.

The Gestalt perception principles were confirmed by neurophysiological

research. As suggested in your handbook <<We are—along with other animals—
“wired” to perceive our surroundings in terms of whole objects>>

Slide 45
Objects that are near to each other (relative to other objects) appear
grouped, while those that are farther apart are not considered part of the
whole
The Gestalt principle of Proximity is that the relative distance between objects in
a display affects our perception of whether and how the objects are organized
into subgroups. Objects that are near each other (relative to other objects) appear
grouped, while those that are farther apart do not.

Slide 46
Example

Slide 47
Another factor that affects our perception of grouping is expressed in the Gestalt
principle of Similarity, where objects that look similar appear grouped, all other
things being equal.

Slide 48
Example

Slide 49
In addition to the two Gestalt principles concerning our tendency to organize
objects into groups, several Gestalt principles describe our visual system’s
tendency to resolve ambiguity or fill in missing data in such a way as to perceive
whole objects. The first such principle, the principle of Continuity, states that our
visual perception is biased to perceive continuous forms rather than
disconnected segments.

Slide 50
Slider controls are a user-interface example of the Continuity principle. We see a
slider as depicting a single range controlled by a handle that appears somewhere
on the slider, not as two separate ranges separated by the handle. Even
displaying different colors on each side of a slider’s handle doesn’t completely
“break” our perception of a slider as one continuous object, although the below
choice of strongly contrasting colors (grey vs. red) certainly strains that
perception a bit.

Slide 51
Related to Continuity is the Gestalt principle of Closure, which states that our
visual system automatically tries to close open figures so that they are perceived
as whole objects rather than separate pieces. Our visual system is so strongly
biased to see objects that it can even interpret a totally blank area as an object.

Slide 52
Example

Slide 53
Symmetry. It states that we tend to parse complex scenes in a way that reduces
complexity. The data in our visual field usually has more than one possible
interpretation, but our vision automatically organizes and interprets the data to
simplify it and give it symmetry. For instance, here we see three groups of
curled brackets, instead of 6 brackets

Slide 54
Example
Slide 55
Figure and ground
The next Gestalt principle that describes how our visual system structures the
data it receives is Figure/Ground. This principle states that our mind separates
the visual field into the figure (the foreground) and ground (the background).
The foreground consists of the elements of a scene that are the object of our
primary attention, and the background is everything else.
The Figure/Ground principle also specifies that the visual system’s parsing of
scenes into figure and ground is influenced by characteristics of the scene. For
example, when a small object or color patch overlaps a larger one, we tend to
perceive the smaller object as the figure and the larger object as the ground

Slide 56
The previous six Gestalt principles concerned the perception of static
(unmoving) figures and objects. One final Gestalt principle—Common Fate—
concerns moving objects. The Common Fate principle is related to the Proximity
and Similarity principles — like them, it affects whether we perceived objects as
grouped. The Common Fate principle states that objects that move together are
perceived as grouped or related.

Slide 57
Example

Slide 58
Please reflect on the fact that: You can use the Gestalt principles to review a
design.

Slide 59
End
Slide 1
L4 Attention Memory & Design

Slide 2tention
Attention

Slide 3
Attention means that you are dedicating (voluntarily or not) cognitive resources
to information (a stimulus or a group of stimuli):
- Visual (the slides of this lesson?)
- Auditory (the sound of my voice)
- A memory e.g., yes you could look at me explaining attention and think to that
time you were on vacation…so you are dedicating resources to the wrong
information…
etc.

Slide 4
It is a complex topic, we will just focus on some key aspects of it from the
cognitive ergonomics point of view!

Attention is selective
you cannot focus on every information and the events around you;

Attention is divisible
you can make the effort to pay attention to (some of the) multiple things you are
doing (multi-task);

Attention can shift voluntarily

you can consciously switch attention from one thing to another;

Attention can shift involuntarily

Sometimes your attention is focused on something, whether you want to or not.

Attention is situated
Often referred to as vigilance or alertness.
Slide 5
Selective attention
Think of it as a spotlight. Everything inside the spotlight is attended to and
processed, while information outside the spotlight is unattended.

Slide 6
Attention is driven by top-down and bottom-up aspects.
Top-down attentional control is driven by endogenous cues, factors that are
internal to the individual: personal goals, knowledge concerning a specific
stimulus, etc.
Bottom-up attentional control is automatically driven by exogenous cues, factors
that are external to the individual.

Slide 7
Attention is attracted to items that differ from their surroundings. Attributes that
may guide our attentional focus when we are searching for information1:
-Colour
-Motion
-Orientation
-Size

Slide 8tention, search for information and information scent

In the real-world, when we search for information in structured scenes, the
structure provides a source of guidance.
This natural way of guide attention is also used when we are searching on an
interface.

This is connected to the idea of information scent and Foraging theory.

If I ask you: Where is your attention guided if you are looking for humans in this
picture?

1
Wolfe, J. M., & Horowitz, T. S. (2017). Five factors that guide attention in
visual search. Nature Human Behaviour, 1(3), 1-8.
Certainly, you will not <<look for people on the top of the wall, not because
they could not be there but because they are unlikely to be there given your
understanding of the scene, whereas you might scrutinize the bench> 2

Therefore your attentional focus will be on certain elements of the picture based
on your goals.

Slide 9
If I ask to look for birds? Were your attention will be guided?

Probably you will look at the sky or the threes, therefore again you will focus
your attention on other elements of the scene

Slide 10
Attention is divisible (divided attention)

We are performing multiple concurrent or parallel tasks (Multitasking)

Our brain is able to attend to different stimuli at the same time, nevertheless
dividing our attentional resources: Negatively affects memory and learning, as
tasks are competing for the limited attentional resources we have.

Slide 11
Information can be designed to grab our attention:
Therefore we will shift our focus of attention (involuntarily) to certain
information. noticeable
Of course, this depends on how salient the information is!
Certain biological items for instance are attracting our attention more than other
elements:
• Threatening stimuli to our survival – animals such as spiders, snakes etc.
• Faces of people
• babies

2
Wolfe, J. M., & Horowitz, T. S. (2017). Five factors that guide attention in
visual search. Nature Human Behaviour, 1(3), 1-8.
Slide 12
If you are designing a product you want to know which information in your
design is going to attract people focus of attention.

We will discuss that during the lesson on Eye-tracking

But you can have a look at these tools I reported here and in Canvas to see how
we can map saliency and try to model the shift of attention of potential users
when they are interacting with products.

[Link]
Index - DeepGaze II ([Link])

Slide 13
Situated attention, often referred to as vigilance or alertness

Vigilance refers to the ability of organisms to maintain their focus of attention

and to remain alert to stimuli over prolonged periods of time

Situated attention finds applications in many domains:

-Air traffic control

-Military radar operations
-Airport screening
-Automated driving

In general situated attention is important for tasks with complex systems in

which should focus on events to avoid potential risks.

Slide 14
Models of attention and SEEV
There are different models of attention. Some of these models propose that
attention is a single pool of resources, and once that these resources are
depleted, our attention will suffer. This means that there will be a task switch, or
we will start paying less attention to a specific task. The more complex
something is, the more resources it requires.
Wickens proposed in 1984 a Multiple resource model

Resources are distributed in separate pools. This means that the attentional
resources that you use to read are not the same ones you use to listen.
Although your reading pool might be almost depleted, you might still have a lot
of resources available to listen to something.

This multiple resource model needs a filter: what we pay attention to? We are
going to focus on this filter of selective attention

Slide 15
Selective attention model (SEEV) Salience Effort Expectancy Value (SEEV

Wickens (2008) suggest that the probability of Attending something (have our
attention focused and remember something) is affected by the

 Salience of information (S)

 Effort (EF) P(A) = S – EF + (EX ∗ V)
 Expectancy (EX)
 Value (V)

Slide 16
 The Salience is the ability of the stimuli (event/information) to grab
attention (exogenous/ BotomUp).
 Of course, your attention may also be focused on certain stimuli because
you want to (specific purposes/goals – endogenous/Top-down)

Slide 17
Effort
It has a negative influence on selective attention. The more the effort requested
the less the attention

Slide 18
Expectations
The likelihood of seeing an event (or obtaining a relevant piece of information)
by paying attention to a specific event or set of stimuli.
This is also associated with the bandwidth (frequency of occurrence/ likelihood
to have access to information).

e.g. your expectation to obtain some relevant information during this lesson to
pass the exam probably affect your selective attention. Also because I am not
going to repeat this lesson (low bandwidth), and this information is not in the
handbook. However, it is in my notes (high bandwidth of the notes)

Slide 19
VALUE
Relevance/usefulness of paying attention to achieve a task or a goal

Slide 20
A summary of SEEV

• Salience of information (S): The stimuli ability to grab attention or my

attention is focused on those stimuli for specific purposes

• Effort (EF): inhibitor of salience the more the effort the less the attention

• Expectancy (EX): The likelihood of seeing an event at (or obtaining a

piece of relevant information) paying attention to a specific event or set of
stimuli

• Value (V): relevance/usefulness of paying attention to achieve a task or a

goal

Slide 21
[Link]

Slide 22
When the conscious perception of an object requires to looking at the object
and to pay attention to the object, while we are focusing our attention on other
elements of the scene we may end-up missing relevant information

(In)attentional blindness shows that

• Even highly salient objects that are fixated by the eyes can be missed by
some people when their attention is elsewhere. This is not associated with
any vision deficit.

• Instead of a complete and detailed figure of the world, we only see a small
part of it. The part we are able to attend i.e., focus our attention on

Slide 23

[Link]

Change blindness is a somewhat different (but related) to the inattentional

blindness phenomenon:

• Without attention to see a change of an event or information it could be

difficult to perceive the change: we may be “blind to the change”.
• Our attention combines pieces of the images and of the information around
us, where we focus our attention make the difference!

Slide 24
 (In)attentional blindness: Events or object in the visual field that are
directly looked at ”(fixated”), and potentially highly valuable, or even highly
salient can be missed by some people when attention is elsewhere.

 Change blindness: is the failure to notice something different about a visual

display/ a scene that changes when you are not looking. Change blindness is
directly related to working memory, individuals who experience the effect of
change blindness fail to notice something different about a visual display
from one moment to the next.

Slide 25
The good design acknowledges and uses our limitation by:
• Providing structured information
• Highlighting salient information
• Providing discriminable information
• Giving access at minimal cost to information

Slide 26
Memory and design

Slide 27
We are discussing these parts of the Information model of Wickens [ see slide]

Slide 28
Psychologists historically have distinguished short-term memory from long-
term memory.
• Short-term memory covers situations in which information is retained for
intervals ranging from a fraction of a second to a few minutes.
• Long-term memory covers situations in which information is retained over
longer periods (e.g., hours, days, years, even lifetimes).

It is tempting to think of short- and long-term memory as separate memory

stores.
Indeed, some theories of memory have considered them separate. After all, in a
digital computer, the short-term memory stores (central processing unit [CPU]
data registers) are separate from the long-term memory stores (random access
memory [RAM], hard disk, flash memory, CD-ROM, etc.). More direct
evidence comes from findings that damage to certain parts of the human brain
results in short-term memory deficits but not long-term ones, or vice versa.

Finally, the speed with which information or plans can disappear from our
immediate awareness contrasts sharply with the seeming permanence of our
memory of important events in our lives, faces of significant people, activities
we have practiced, and information we have studied.

These phenomena led many researchers to theorize that short-term memory is a

separate store in the brain where information is held temporarily after entering
through our perceptual senses (e.g., visual or auditory), or after being retrieved
from long-term memory.
Slide 29
Cognitive research showed that short and long-term memory are functions of a
single memory system—one that is more closely linked with perception than
previously thought3

Slide 30
Short-term memory (STM)
enables the brain to remember a small amount of information for a short period
of time. The shortest type of memory known is the working memory(WM),
which can last just seconds.
WM is what we use to hold information in our head while we engage in other
cognitive processes. An example is remembering the phone number of a new
friend reciting it as you are navigating your phone’s menu system to add the
number to your contacts.

Long term memory (LTM)

As reported in the diagram above, explicit memory is one type of long-term
memory. This is our conscious memory. It contains information about events and
knowledge that you cumulated in your life.
The other kind of long-term memory is implicit, or unconscious memory. These
unconscious memories may be procedural, involving learned motor skills—
learning how to ride a bike or how to type using a keyboard, for example.
Implicit memories can also result from priming, which occurs when exposure to
one stimulus influences your brain’s response to another.

Slide 31
Short and Working memory and design

3
Jonides, J., Lewis, R. L., Nee, D. E., Lustig, C. A., Berman, M. G., & Moore,
K. S. (2008). The mind and brain of short-term memory. Annu. Rev.
Psychol., 59, 193-224.)
Slide 32
Working memory could be defined as a sort of “working attention”4:
• WM is the main component of our short-term memory: STM only holds
information, Working memory enables manipulation of information.
• It is our system to manipulate or organize the material we are holding in
memory. It could be considered an interface between STM and LTM.
• WM is composed of a tiny subset of all the neural activity from our
perceptual systems and our long-term: The memories that we are aware of
right now!
• It integrates information from all of our sensory modalities and our long-
term memory.

Slide 33
The design should help people remember key information from one moment
to the next. People can not remember what they have done, but they need to know
how to perform the actions to achieve their goal.
For instance:
- “Search function” should keep displaying the search terms, and
should be visible.
- Stick to one ‘call to action’: an action can be performed in one way only.
By AVOIDING multiple modalities of interaction
- Keep instructions/procedures visible during the execution of actions.
- keep menus and information panels broad and shallow.

Slide 34
It is not always possible to stick to one call to action
Sometimes we deal with multiple functions/actions that can be performed with
similar commands or combinations of commands so we use a product by
activating a different modality to produce a different result.

A mode can be described as a change of modality in the use of a product.

Our working memory is limited and designed systems have limited space. It
is a matter of finding a balance between limitations (humans and
technological)

4
Baddeley A. (2000) The episodic buffer: a new component of working
memory? Trends Cogn Sci. 2000 Nov 1; 4(11):417-423.
In this example: The Caps Lock Function on the keyboard, when activated
changes the MODALITY OF WRITING. SO YOU CAN WRITE CAPITAL
LETTERS

Slide 35
MODES & Errors
In many digital cameras, pressing the shutter button can either snap a photo or
start a video recording, depending on which MODE is selected.
If you forget in which mode you are in you may end-up shooting a photo instead
of doing a video (or vice versa)

Slide 36
Summary (Modes & Errors)
- MODES are necessary to provide multiple functions with a limited interface
(technology is limited)
- MODE ERRORS are due to poor feedback about the current mode (status) of
the system.

Human working memory is too unreliable and designers to support users to

avoid mode errors have to:

• Ensure that a system gives clear and continuous feedback about the mode;
• Ensure that the system keeps track of the mode and provides
visible/perceivable warnings.

Slide 37
Activity L4 on Canvas

Tell me all that you know regarding the relationship between attention and
saliency of information (15 minutes)

Rubric (How I will assess your answer):

- you used the correct terminology (correctness)
- you connected appropriately the elements described in the question
(appropriateness)
- The answer is short, logic and consistent (logic and consistency)
Slide 38
Possible answer
How we dedicate (voluntarily or not) cognitive resources to information and we
focus our attention is strongly affected by the saliency of information.
Information can be designed to grab our attention and to force us to focus of
attention (involuntarily) on certain aspects.
We can be primed by the design and its structure (exogenous aspects) to focus
on something, or we can voluntarily or involuntarily be affected by internal
aspects or our characteristics to focus our attention on specific information.
Because of that, we could miss relevant information (e.g., (in)attentional
blindness) or miss relevant changes in the information or during events (change
blindness).
Wickens proposed a model to explain how we filter information, called SEEV
(Wickens et al 2003). In this model saliency is one of the elements that affect
how we filter information and how we allocate attention, together with Effort,
Expectancy and Value.

Slide 39
Break

Slide 40
Long Term Memory

Slide 41
Activating Memory for recognition and recall
The human brain was “designed,” through millions of years of natural selection
and evolution, to recognize things quickly.
By contrast, recalling memories—that is, retrieving them without perceptual
support—must not have been as crucial for survival, because our brains are
much worse at that.
The quality of long term memory processes of recognition and recall is linked to
the infrastructure, that we developed through evolution

Activating a memory content means

consists of reactivating the same pattern of neural activity that occurred when
the memory was formed and stored.

Key types of activities:

- recognize patterns (recognition)
- recall information (recall)

Slide 42
Recognition
New perceptions that are very similar to the original ones reactivate the same
patterns of neurons, resulting in recognition if the reactivated perception reaches
awareness.

In the absence of a similar perception, stimulation from activity in other parts of

the brain can also reactivate a pattern of neural activity, which if it reaches
awareness results in recall. we do not recognise if the perception does not reach
awareness

The more often a neural memory pattern is reactivated, the stronger it

becomes— that is, the easier it is to reactivate—which in turn means that the
perception it corresponds to is easier to recognize and recall.

Slide 43
Recall
In the absence of perceptual stimuli (or when something is not clear enough so
that we can recognize it): We may recall information and images (remember or
try to make sense of the information).

When we recall we reactivate a pattern of neural activity in which we stored

previous (similar) information.
This results in a “recall” (when we found a match).
Note: we may be not able to recall information when that information was not
correctly stored

Slide 44
For instance: See and select an option (recognition), is easier than writing a code
(recall)

Slide 45
This natural tendency to recognize and to see the patterns is used in marketing
and advertisement
Slide 46
This heuristic also applies in testing: where open questions exams demand a
recall process (harder) and multiple-choice questions exams demand a
recognition process (easier)

Slide 47-54
Let’s play.
Take a piece of paper
I will read for you some words
try to remember and write down all the words

If you find both these words in your list you have been exposed to a process that
induced a false memory, because none of these words was on the list

Memory is much more unreliable that we believe.

We performed online a Deese-Roediger-McDermott (DRM) False Memory Lists

When presented with a list of semantically related words, people report to also
remember a related but absent word in the list - indicating a false memory – a
memory for an event that never occurred
([Link]

Slide 55
Limitations of LTM
memory is stored in a distributed network, spread among many parts of the
brain. It evolved to serve our ancestors and us very well in getting around in our
world.

However:

• Error-prone: memory is a good picture but not accurate.

• Affected by emotion: Emotions may alter our perception, memory or
ability to recall/recognize
• Free-associative: Memory associations are personal so the way to retrieve
information
• Retroactively alterable
• Easily biased by a variety of factors at the time of recording or retrieval.
Slide 56
Error-Prone
- What is in long-term memory is not an accurate (high-resolution)
recording of our experiences.
- long-term memory can be characterized as using heavy compression
methods that drop a great deal of information.
Images, concepts, events, sensations, actions—all are reduced to
combinations of abstract features.

Slide 57
Weighted by emotions
Emotional reactions to events may alter attention and compromise our way of
memorising details or enhance our ability to associate events with other
memories.

Slide 58
Free associative
Mum, Apple, McDonald…

- You know the meaning of these words

- We have a common/shared understanding of these words
- However, each one of us associates to these words different events,
information, images sensation emotions etc.

Slide 59
Retroactively alterable by new information
New information in line with our prejudice and believes are often affecting your
LTM

Slide 60
Easily bias
The time, the context, the association with previous information, the emotion
can easily bias the process of encoding information (when we stored the
information) and the recall of the information (leading questions).
As a result, we could remember the details or things that were not there.
Slide 61
Long Term Memory and Design

Slide 62
Authentication
Authentication is one functional area in which many systems place burdensome
demands on users’ long-term memory.

Even when we ask questions to help people to remember or recover their

password we are probably asking too much.
When we ask for instance: the name of your elementary school?
Some questions could have several possible answers. Many people had several
elementary schools, childhood friends, or heroes.

Slide 63
WHERE ELSE FAILS STANDARDISE!
Previous knowledge of a certain way of information display would be used
again when users meet another information display of the same signal. Using a
commonly known display element/symbol to convey different information,
simply would not work.

Slide 64
Use pictures where possible to convey function.
People recognize pictures very quickly, which also stimulates the recall of
associated information. For this reason, today’s user interfaces often use pictures
to convey the function.

Recognition is fairly insensitive to the size in which objects and events are
displayed. After all, we have to be able to recognize things independently of
their distance from us.
What is important are features: as long as most of the same features are present
in the new picture that was in the original one, the new perception stimulates the
same neural pattern, resulting in recognition.

Slide 65
END
Slide 1
L5

Slide 2
Human Decision Making and interaction

Slide 3
Where are we in the information model of Wickens and colleagues1: Decision
making and response selection

Slide 4
Dual system
Daniel Khaneman2 dual system theory is a way to simplify the complexity of our
mind and it is a functional exemplification to explain our way of taking
decisions.
We used two systems:
 The fast system (system 1): automatic mind system one - it evolved first
and is the main controller of our perception and behaviour.
 The slow system (system 2) the conscious, monitored mind. System two
came later in evolution and it usually takes a back seat in controlling
human perception and behaviour.

Some scientists prefer the terms emotional mind (System 1) and rational mind
(system 2)

System 1: Like perception judgments and decisions happen quickly

EMOTIONAL MIND: Fast thinking (automatic, frequent, emotional,
stereotypic, subconscious). To deal with unconscious and automatic mind
processes: 2+2= ?

System 2: For important or hard judgments/decisions (slow)

RATIONAL MIND: Control of perception and behaviour. Slow, effortful,
infrequent, logical, calculating, conscious. To deal with conscious and complex
decision(e.g., 55 * 3 -94 = ?). System 2 COULD check system one, but often
doesn’t.

1
Wickens, C. D., Gordon, S. E., Liu, Y., & Lee, J. (1998). An introduction to
human factors engineering.
2
Kahneman, Daniel. Thinking, fast and slow. Macmillan, 2011.
See in your handbook the example on CH 10 (section: WE HAVE TWO
MINDS) regarding the baseball and the bat. In this example, most people report
$10 for the ball, which is obviously wrong if you would check. The correct
answer for the cost of the ball is $5. System 2 can work that out; system 1
cannot.

Slide 5
The fast system (System 1)
judges quickly, and usually its judgements are good enough to allow us to
achieve our goals.

 It judges only on what it is perceived in front of you;

 It filters perceptions based on goals and beliefs;
 It excludes potentially conflicting information before any action of
System 2.
 System 1 seeks a coherent story above all else and often leads us to jump
to conclusions.

System 1 works mainly in terms of shortcuts and heuristics!

Slide 6
The slow system (System 2)
The perceptions and judgments of system 1 come quickly, and are usually good
enough to allow us to get by in most situations.

System 2 is lazy and only sometimes necessary i.e., we are lazy thinkers
We use System 2 only for complex and important situations, usually, we rely on
System 1

System 2: requires a mental and conscious effort while System 1 is always

running in the background and takes no conscious effort.

Slide 7
Why we need both systems:
A fully automatic brain (Only System 1) would be inflexible: it couldn’t
switch goals in mid-action, or quickly adjust its response to rapidly changing
situations and very ineffective in complex situations.
A Fully rational brain (Only System 2) would be slow and very ineffective
to deal with all the tasks we can perform automatically: Think about driving.
Most of the tasks you are doing are automatic, you have learned how to do that
“without thinking”. System 1 is dealing with that, so you can, for instance, think
about other aspects while you are driving e.g., mentally go through the key
concepts for an exam, while you are driving to the university to perform the
exam?

Slide 8
System 2 IS REQUIRED when we need to perform exactly right. When, for
instance, we are in situations that System 1 does not recognize and therefore has
no automatic response, or when system one has multiple conflicting responses
and no quick-and-dirty way to resolve them.

The SLOW system is the controller of the FAST system!

Slide 9
The design should take advantages of the fact that we can act and think fast and
the slow

Slide 10
Designing considering Fast and slow thinking means:
• Support System 1: Indicate system status and users’ progress toward
their goal. e.g., Home>About us, Progress bar. This simplifies
recognition for the users
• Support System 1 and avoid/minimize the use of System 2: Tell
people explicitly and exactly what they need to know. Don’t expect
them to deduce information. Don’t require them to figure things out by a
process of elimination. Give clear instructions.
• Support the use of System 1 - Minimize complexity. Don’t expect
people to optimize combinations of many interacting settings or
parameters.
• Avoid the use of System 2 - Don’t make users diagnose system
problems. People do not want to or are not able to solve complex issues
related to technology. You need to create support tools and an interface
for when things go bad!
• Avoid the use of System 2 - Let people use perception rather than
calculation. Some problems that might seem to require calculation can
be represented graphically, allowing people to achieve their goals with
quick perceptual estimates instead of calculation.
• Avoid the use of System 2 - Let the computer do the math. Don’t
make people calculate things the computer can calculate itself.

• Facilitate System 1 - Make the system familiar. Use standards and

graphics/function people have seen before (Recognized is better than
recall!)

All these practices are about to let people use System 1!

Slide 11
Decision and framing

Slide 12
A decision is not only a matter of Systems 1 and System 2 but it is also
associated with (and affected by) how the information is presented.

Slide 13
Framing is one of the reasons why people’s judgments and preferences are
unstable over time
i.e., to present a choice in one way nudges people to decide for an option; to
present the options differently enables people to decide differently.

The value of the options is certainly important but, how choices /options are
framed (presented) may strongly affect people’s decisions.

Slide 14
In this case,3 the economist subscription was presented with three options.
• Print-only for: $59
• Web subscription-only access for: $125
• Print and web access for: $125

Only 16% of the subscribers opted for option 1, none for option 2, and 84% for
option 3

3
From the behavioural economist Dan Ariely:
[Link]
Slide 15
Then a large majority of the subscribers opted for the less expensive option, as
removing the second option the offer of print and web access resulted in way too
expensive.
So the middle option with only web subscription at the same price as Print and
web was “persuading” people of the convenience of the “print and web option”.

Slide 16
How we present (manipulate) information sets up the context for people to
understand these options. Framing is a powerful mechanism to manipulate
information:
• Negative: Fake news, fake quotations, fake information, fake options;
• Positive: to enable users to perform better, quicker or to increase safety and
give them a sense of control (placebo buttons) in many crossways with
traffic lights.

Slide 17
Exploit our expectations and the limitation of our fast thinking (System 1) is
often accepted to change in a positive way human behaviour
Persuasive systems attempt to change our behaviour by pushing our
motivations, by challenging us, by rewarding etc.

Fogg Behaviour Model4 (FBM)

How a system persuades and pushes people to perform certain actions is
something that a good persuasive system is taking into account.
So we want to change or affect a behaviour (B), and to do that we can use some
triggers (T: design, information, rewards etc.) to motivate (M) people and to
increase their ability (A) to find information, perform certain tasks etc.

Slide 18
When persuasion is used in malicious ways we talk about manipulative design.
This is an approach to design systems and processes to influence or trick users

4
Fogg, B. J. (2009, April). A behavior model for persuasive design. In Proceedings of the 4th
international Conference on Persuasive Technology (pp. 1-7).
into taking particular actions that they might not take otherwise. There are
several types of malicious tricks that are reported in this infographic.

Slide 19
Design Errors and ROI
The bad design of everyday objects is a very problematic (and often
undiscussed) issue.
For instance on October 21 of 2020 about 10.000 units of this chair were retired
from the USA and Canada market because the locking mechanism on the chair
frame can disengage, posing a fall hazard. In total 19 reports of chairs breaking,
including four reports of minor injuries from falls were reported.
All of this was avoidable with a simple test, for instance, by using the prototype
for a period of time before introducing it in the market.
Unfortunately, often everyday products are not tested because most companies
do not see the Return of Investment in the evaluation.

Slide 20
We discussed in this course that our brain can lie to us, and also that “Design
and Systems” can lie to us or use our limitations against us.
We need to be aware of that as designers and as users.
In today complex society we need to learn to rely more and more on the Slow
System. Moreover, errors may always occur but we need to demand Design
Responsibility from companies and institutions.

As designers, you should take a stand. You are going to deal with issues,
sometimes, even important issues. There are two approaches to deal with
problems that are embedded in the core of our society (i.e., unavoidable unless
we redesign the entire system):
1. You can decide to hide the problems from the eyes, make these less
perceivable [Left picture]. This will not mitigate the issue at all, it will
just make appear the system better.
2. You can try to mitigate the issue with design solutions. This will raise
awareness regarding the issue, open up the possibility to be inclusive and
offer some solutions for the end-users [Right picture]
I let you decide where you stand!

Slide 21
ACTIVITY L6
Slide 22
Priming: how a product is presented in terms of information (e.g., packaging,
marketing instructions etc.) may affect people perception of that object
Perceptual frames: we have expectations about objects or events that are
usually encountered in a situation (e.g., you know most rooms in your home
well enough that you need not constantly scrutinize every detail). Our
expectations could be biased by this previous knowledge. For instance, if an
object is moved from one place to another in our house, it could take a while for
us to realise it.

Priming and perceptual frames are both related to the perception of the
information and how this perception may be biased by the presentation of
information(priming) or by our expectations based on our previous knowledge
(perceptual frames).

Framing is related to decision making, and how the presentation of options and
choices affect our decisions. Perceptual frames also may affect directly or
indirectly our decisions. If we believe that something should be in a certain
place, we will maintain this assumption until we will discover that it is false, and
this discovery could have consequences i.e., I can not find the key of the car, I
am going to be late!

These two phenomena have one thing in common: The context.

In fact, the manipulation of the context (Priming) and our expectations regarding
the context (Frames) may affect how we perceived the reality, our decision-
making and ultimately our behaviour.

Slide 23
DESIGN PERSUASION & S1 & S2

Slide 24
Computers are good—reliable, fast, and accurate—at precisely what we are bad
at: remembering, calculating, deducing, monitoring, searching, enumerating,
comparing, and decision making.
People are using computers for that reason. Designers need to build interface
and products to help people to perform rationally their tasks and minimize
the need to use System 2
This is the reason behind the title of the very famous book by Steve Krug:
<<Don’t make me think>>

Slide 25
Decision Support Systems (DSS) – DESIGN FOR SYSTEM 2
In the slide an example of a Dashboard (Information Visualization ) to support
decisions regarding Supply Chain.

In this case, we want to support accurate decisions by providing a way to

manage, filtering and visualize multiple complex data. The management of
Complexity
Key aspects of DDS systems (support of System 2):
 Provide all options. If there are too many to simply list, organize or
abstract them into categories and subcategories, and provide summary
information for those, so people can evaluate and compare entire
categories of options at once.
 Help people find alternatives. Some solutions may be so counterintuitive
that people don’t consider them. Decision support systems can expose
options users might miss and can generate variants on user solutions that
are minor or even major improvements.
 Provide unbiased data. That is, data created or collected in an objective,
reproducible manner.
 l Don’t make people calculate. Perform calculations, inferences, and
deductions for users where possible. Computers are good at that; people
aren’t.
 Check assertions and assumptions. Decisions are based not only on data
but also on assumptions and assertions. Decision support systems—
especially those supporting critical or complex decisions—should let
users declare any assumptions and assertions upon which a decision will
be based, and then should “sanity check” them for the user.

Slide 26
Persuasive Design for good – DESIGN FOR SYSTEM 1
Example from [Link]
You want to buy something. The System provides you with examples of other
products and accessories to wear with the product you are looking at and other
(similar) products you may also like.
The system is designed to influence or persuade people suggesting associations
that may push you to spend more or increase your wish list. This is different
from simplifying the complexity and support decision making. The system is not
presenting you with all the possible choices by giving you the possibility to filter
and managing the information. The System is suggesting to you what else to buy
in terms of predefined choices (based on predefined criteria). This system is
persuading you that attached to the target product there are other items you want
to look at.

In this specific case, if we refer to the model of Fog the target behaviour is to
inform you about what else you can buy associated with a product you are
looking at(Behavior). The motivation is that fashion objects are usually linked to
outfits that you can composed (Motivation), therefore the System is offering you
the ability to connect items and to create your personal choice among the
available options (ability). In this case, we are talking about a legitimate set of
Triggers: The system tells you how you could or should wear the product
(WEAR IT label)” and what else “you may also like”. These triggers are playing
on System 1 (your fast reaction to the triggers) but are legitimate modalities, the
interface is not playing tricks on you, for instance suggesting that there is limited
availability of the product and that you should buy that now.

Slide 27
BAD PERSUASION - DESIGN FOR SYSTEM 1
This example of a web system, by which you can book rooms in hotels all over
the world, is playing on System 1. This interface is pushing continuously the
end-users to believe (without saying that) that the displayed deal is a special
offer (JACKPOT! YOU GOT A GREAT RATE!), and that someone else could
still this offer (3 other people looked at this in the last 10 minutes). This seems
to be a great deal, in fact, as reported in the trigger “someone else just booked in
fact on the same dates 8 minutes ago”.
The behaviour the system is persuading you is to book a room as soon as
possible, without doing further comparisons or searches.
The interface once you selected an offer is not offering you the ability to do a
further comparison with other offers, but only to select inside the offer (Ability).
To motivate you to book in a fast way, several triggers are proposed and these
are all based on unverifiable information that the end-users can not check or
visualize.

This is very close to irresponsible persuasive design. It is not (yet) illegal to

adopt these practices, but we can consider these design patterns something
borderline between persuasion and dark patterns
Slide 28
Action, controls (CH 13) and errors

Slide 29
Selection and Execution of response

Slide 30
FITTS’ LAW you handbook is quite accurate about fits’ Law, therefore, we can
rapidly discuss this law

Slide 31
FITTS was interested in establishing the difficulty of a target selection task
when people are interacting with point and click interfaces. This law helps in
defining some useful principles that we have to consider when designing
interfaces.
When you move an object from one point to another (moving toward a target):
• The larger and the nearer your target is to your starting point, the faster you
can point to it.
• This applies to all the types of controls in which people may point and click.

Slide 32
Following the Fitts’ law: circular menus enables fast cursor-to-target motions. If
the cursor is at the center of the circular menu then it is equidistant from all the
options, and you will have the same level of difficulty to reach all the options.
Despite this is good in theory, and in some cases, this solution is adopted with
good results. Not always interacting with digital devices the cursor may be at the
center of the menu (no advantage)

Slide 33
Fitts’ law predicts that, if the pointer or finger is blocked from moving past the
edge of the screen, targets at the edge will be very easy to hit: people can just
yank the pointer toward the target until the edge stops it, with no need for slow,
fine adjustments at the end of the movement.
Thus, from the point of view of Fitts’ law, targets at the screen’s edge behave as
if they were much larger than they are. However, this edge-pointing detail of the
law applies mainly to desktop and laptop computers, because modern
smartphones and tablet computers don’t have raised edges that physically stop
fingers.

Slide 34
 Make click-targets—graphical buttons, menu items, links—big enough so
that they are easy for people to hit.
 Make the actual click-target at least as large as the visible click-target.
 Checkboxes, radio buttons, and toggle switches should accept clicks on
their labels as well as on the buttons, thereby increasing the clickable area.
 Leave plenty of space between buttons and links so people don’t have
trouble hitting the intended one.
 Place important targets near the edge of the screen to make them very
easy to hit.
 Display choices in pop-up and pie menus if possible.

Slide 35
Complexity advantage and Hick-Hyman law.

Slide 36
When we select a response/action to perform a task:
the time required to make a decision (select and execute the action) is a function
of the number of available options.
i.e., intuitively the more the options the more the time!
However, the time to make a decision when we have 4 equal options (2 bits of
information) is not double when we have 2 options (1 bit of information).

Slide 37
Often a person may transmit more information, or perform faster per unit time,
if: s/he may operate with complex (information-rich) with manageable choices,
than when s/he has to work with several simple (information-reduced) choices.
Slide 38
The time to select an option is roughly proportional (logarithmic) to the number
of possible choices – i.e., more options more time to decide (i.e., Hick’s Law)

The frequency of the decision also matters: People are better at selecting 1
option among 8 (complex decision, with 3 bits), then at selecting one option
among two repeated 4 times (simple decision, 4*1bits). i.e., breaking complexity
in a small set of tasks may add unnecessary frustration

Slide 39
How we handle the complexity? It depends on the context: Is complexity
manageable? Which are the frequent actions?

To show a large set of visible options is perceived by people as complex i.e., too
many options slow down the reaction time of users and expose people to errors
and workload.

To reduce the number of stimuli/options to a manageable group of options

enables end-users to perform a faster decision-making process (and usually
fewer errors) especially when the actions we need to perform are not frequent
ones.

NOT ALWAYS SHOWING SEVERAL OPTIONS IS BAD: If we show a

manageable set of options of frequent actions and minimize the information
regarding the infrequent actions people will interact in a smoother way

Slide 40
Complexity advantage: Implications for design

Slide 41
• KEEP CHOICES SIMPLE!
o Do not give users more choices than are essential (e.g., time is critical)
o Long menus, with lots of rarely chosen options, may not be desirable.
o Offering many choices = longer response time and increased possibility of
mistakes.
 o More items/options typically lead to a greater similarity between items
and confusion.
o Provide the necessary more used information/options
• TO USE A SMALL NUMBER OF COMPLEX CHOICES WITH KEY
(frequent) OPTIONS, IS USUALLY PREFERABLE TO HAVE
MULTIPLE SIMPLE CHOICES
• COMPLEXITY IS PART OF ANY SYSTEM! IT CAN NOT BE
ELIMINATED, BUT IT CAN BE MANAGED TO ENABLE BETTER
PERFORMANCE.
• SOMETIMES IT IS NECESSARY TO ADD IN A CONTROLLED WAY
COMPLEXITY - e.g., When we want to enable the user to think and to
perform slowly and accurately

Slide 42
In online websites the payment process is often performed with sequential steps,
with few elements (sequential choices), instead of presenting to people a long
and complex form, this approach:
• increases the time to perform (login, select, insert in the basket, confirm the
selection, insert data, check-in HTTPS, payment, Payment confirmation
and email for track);
• enables people to have more control over errors (error recovery).

HOW WE MANAGE AND DISTRIBUTE COMPLEXITY AFFECT THE

USER JOURNEY!

Slide 43
In their study Iyengar and Lepper (2000)5 showed in three empirical studies
<<that, although having more choices might appear desirable, it may sometimes
have detrimental consequences for human motivation.>> Their study on
consumer behaviour suggested that the provision of extensive choices, though
initially appealing to choice-makers, may nonetheless undermine choosers'
subsequent satisfaction and motivation.

Netflix is a very good example of choice paradox: Too many choices often lead
to indecision!

5
Iyengar, S. S., & Lepper, M. R. (2000). When choice is demotivating: Can one
desire too much of a good thing?. Journal of personality and social
psychology, 79(6), 995
How much time do you spend to select a movie?

Slide 44
BREAK

Slide 45
Errors

Slide 46
Most basic difference between errors:
 Commission/execution of an error: your action results in an error.
 Omission (lack in executing a necessary action): Because you forget to do
a certain action an error happens.

Slide 47
Skills, Rules Knowledge Model of Rasmussen 6
Taxonomy of unsafe acts, distinguishing between voluntary and involuntary
actions. Diagram on pg 47 slide important!!!

Slide 48
Violations of Rules
• Sometimes someone makes an intentional error, where the knowledge and
awareness of the context or mode were correct, but the wrong action is still
chosen to perform the INTENDED action.
• A violation is when you want to do a specific action/task (intended) but you
DELIBERATELY PERFORM A PROCEDURAL ERROR or use a
wrong approach to achieve the goal (deliberate)
• This type of error constitutes a violation (and could be punishable by law).

6
RASMUSSEN Jens (1983). Skills, rules, and knowledge: signals, signs and
symbols and other distinctions in human performance models. IEEE
Transactions: Systems, Man & Cybernetics, 1983, SMC-13, [Link],
USA), 1994 - Not available from UT Library
Types of Violations:
• Changes in the procedures (rule not apply any more) but new rules are not
available yet, or voluntarily ignored for convenience: ROUTINE
VIOLATION
• People have competing demands and they need to speed up so they decide
to break the rules: SITUATIONAL
• People deal with unexpected circumstances and try to solve a problem
that can not be solved following the rules: EXCEPTIONAL
VIOLATION
• People were asked to do so for ORGANISATIONAL REASONS
• People may decide to violate rules for personal reasons (INDIVIDUAL)

Slide 49
Sometimes people may do the wrong thing, believing it to be right! In these
cases, we have an error in thinking and planning the action.

So these actions are INTENDED because people want to do the specific

action/task, but an ERROR is performed because people do not know that they
are making a mistake, for instance:
• Unusual situations which require to think and to apply a divergent problem-
solving strategy
• People may rely on processes or information that are out of date (a map to
drive on an unfamiliar route with a not updated navigation system),
• Misdiagnose process and take inappropriate corrective action (due to lack
of experience or insufficient/incorrect information etc.)

Slide 50
While a situation may be diagnosed and understood correctly, rule-based errors
may result from a failure to apply the correct rules for the selection of a
response, or by applying bad rules.
e.g., we decide to ignore an alarm in a real emergency, by following the history
of spurious alarms…

Slide 51
Errors may result from slips of action when the correct response is intended but
an incorrect action is actually released i.e., an unintended response “slips” out of
the hands (Norman, 1981).
Slips of this sort are typically the result of poor human factors design, such as
incompatible control–display relationships, confused displays or controls,
coupled with an operator who is well skilled and performing a task in a highly
automated mode, thereby not carefully monitoring his or her action of
selections.

In skill-based errors due to slips: We want to perform an action/task, but we end

up doing something else (UNINTENDED). This unintended action resulted in
an INADVERTENT error.

A simple, frequently-performed physical action goes wrong:

• The wrong key while typing
• Wrong word (Slips of the tongue)
• Click on the wrong choice in a menu (“delete” instead of “save”…)

Slide 52
Similar to slips we can inadvertently forget to perform an important/mandatory
action is forgotten (Omission of an important task).
So without intent, we did not finish appropriately an action because we forget
something and this resulted in an error.
i.e., Nowadays “MS Word” automatically saves documents every ‘tot’ minutes.
In the past, many potential masterpieces were lost because people forget to save
before close.

Slide 53
Controls (Actuators) of interactive systems

Slide 54
Control (actuator): the part of the system that is directly actuated by the operator
e.g. by applying pressure (ISO 9355-1, 1999).
Control is adjusted or manipulated by the human hand to effect change in a
system

Slide 55
What we have discussed in terms of principles and methods for Usability and
UX assessment could be also applied to redesign controls, e.g.,:
• Clustering information (e.g., Card Sorting)
• Hierarchical labelling: organise information in a hierarchical manner
• Demarcation + Similarity and Symmetry to manage and handle the
complexity i.e.,: breaking the interface into chunks, so that it becomes easier
to process.
• Usability inspection with experts + Usability test to review and inform the
redesign

Slide 56
Stereotypes + affordances
Controls are the tools to perform actions. Whenever we deal with controls we
have expectations about:
- how the movement of that control will be associated with the corresponding
action.

These expectations may be defined as stereotypes due to previous knowledge of

similar systems. For instance, in the case of the picture in the slide, the
AFFORDANCE is that the control is turn”able”, the STEREOTYPE is that
turning clockwise equals to increase the volume.

Slide 57
In displays, the user controlling experience is defined by this formula of GAIN

Gain is proportional. It is the result of the relationship between the magnitude of

the input with the magnitude of the output signal at steady state. Many systems
contain a method by which the gain can be altered, providing more or less
"power" to the system.
e.g., the gas pedal in the car is an example of GAIN the more you push down the
pedal the more the speed of the car. However, the relationship between the
action “pushing the gas pedal” and “increase speed of the car” is controllable
and alterable.

We can alter the RATIO of the controls, for two reasons, for instance:
- we want people to perform with the minimal amount of physical effort a quick
movement in the display. Therefore, we want a LOW RATIO (HIGH display
movement/LOW Physical movement) e.g., 3 units: 2 units = A GAIN of 1.5
units of increase in the display per each unit of increase of physical movement
(HIGH GAIN)
- We want to slow down the user to focus their attention and increase accuracy
so we increase the physical movement required (or reduce the display
movement). HIGH RATIO (low display movement/high physical
movement) e.g., 3units: 6unist= A GAIN OF 0.5 units of increase in the
display per each unit of physical movement (LOW GAIN)
In the slide, you have an example about how you can change the GAIN of the
mouse (in Windows system)

Slide 58
When we operate with a system having control means to be able to manage the
stimuli and the reaction of the system, and to be able to continuously correct
actions, manage complexity and lags

[Link]
Slide 1
Reading & Eye Tracking

Slide 2
Speaking and understanding spoken language is a natural human ability, but reading is not.
Over hundreds of thousands—perhaps millions—of years, the human brain evolved the neural
structures necessary to support the acquisition and the execution of spoken languages. As a
result, normal humans are born with an innate ability to learn as toddlers, with no systematic
training, whatever language they are exposed to.
Conversely, When it comes to reading we need to receive a systematic training

Slide 3
Speaking and understanding a language
To acquire the necessary linguistic competences to understand and speak a language there is a
critical period: which is usually up to 8 years old (to achieve full fluency). In this period we
need to be exposed and have the opportunity to practice regularly a certain language.
Up to this period of our development, we may acquire very easily new languages. After this
period our ability to learn a new language decreases constantly.

Have a look at: [Link]

Slide 4
To learn how to write and read require specific training in terms of competences.
At no time during childhood, our brain shows any special innate ability to learn to read or
write. We learn these competencies by instruction and practice, like playing a violin or
reading music.

Slide 5
Reading and writing imply learning how to interact and manipulate with an
artefact: The alphabet.

We developed (as humans) a tool (alphabet) to transport our culture and to

communicate remotely with people.
Therefore we need to learn the script (system of writing) to use and decode
written languages.

If we do not know a script and we are not able to interpret the code (alphabet)
we talk about ILLITERACY. This means that we cannot understand (interact
with) the content of the message, like in the examples I am reporting here from
your handbook.
Slide 6
Recognize patterns and their combination
Learning to read involves training our visual system to recognize patterns—the
patterns of text (characters)
• We do not need to learn the basic visual features like lines, shapes etc.
• We need to learn the combinations of the features (rules) to create
characters — e.g., letters, numeric digits, and other standard symbols.
This means that we need to be able to recognize the patterns created by
the standard arrangement of certain features.
• Then we need to learn how these symbols can be combined to generate
words, sentences, periods etc.

Slide 7
To read we need to perform the following processes:
• To be able to recognize
• Phonemes (the smallest unit of a spoken language)
• Graphemes (the smallest unit of a written language)
• Morphemes (the smallest unit of meaning that cannot be further
divided)
• Acquire the syntax to use words and phrases (features of the
language system).
• to acquire experience about the logic context in which words are used and
their sense (context of words, phrases and sentences)

Slide 8
To handle all these processes concurrently we should learn how to read by
performing in parallel bottom-up and top-down activities:
• Bottom-up (FEATURE DRIVEN)
Once we learn how to recognize the feature (forms of letters) and their
combinations we can read the text
• Top-down (CONTEXT DRIVEN)
We use the context and the meaning of phrases to speed up the recognition
of the feature and the meaning, and to speed up the reading process
Slide 9
Here you have an example from your book of a broken linguistic code.

You are all good readers, therefore with a minimal effort you could make sense
of this text, inferring by the context of the text (the most probable association
between the words):

<<Mary had a little lamb. Its fleece was white as snow. And everywhere that
Mary went. Mary went, Mary went. Everywhere that Mary went. The lamb was
sure to go>>

The context (Top-Down) in reading is important especially for the poor readers
who have little skills / expertise in recognizing and combining the symbols,
therefore they strongly rely on context to read and understand the text. Because
of that reading for poor readers is complex, time consuming and demanding
from the cognitive point of view and their comprehension of the text is minimal.

Also experts readers use the context to extrapolate the overall meaning of the
contents of a text, but they can read and understand better because for them it is
easy to recognize the patterns (minimal workload) therefore they can better
reflect on the words and their meaning.

Slide 10
Design and Reading
Don’t disrupt reading; support it! This means:
1. Ensure that the text of your product enable feature-based processes
(visibility of text): avoid difficult or tiny fonts, patterned backgrounds,
centering, etc.
2. Use restricted, highly consistent vocabularies. Help poor readers with
plain language or simplified language.
3. Format text to create a visual hierarchy: facilitate easy scanning of the
text e.g. use headings, bulleted lists, tables, and visually emphasized
words. (Support information seeking)
Slide 11
Here I am reporting one of the many examples regarding the importance of
facilitating reading with products. In this specific case for medical products 1
The tips about reading are relevant to guide the design of interfaces, but also of
elements such as: product guidelines, packaging, instructions.
Remember that: The text you are using on an interface or associated with a
product is part of your system. Usability analysis should also include the usage,
for instance, of instructions and guidelines to check if issues due to
inappropriate words (or instructions, or label) affect the safety of the interaction
with a system.
This is true for high-risk products (e.g., medical products) but in general for any
type of interactives artefact (digital or physical)

Slide 12
As we already discussed we are information seekers

Slide 13
If you also recall, we discussed about the fact that we can follow the scent of
information

Slide 14
Moreover, we discussed that Information can be designed to grab our
attention and that when we are designing a product we want to know which
information is going to attract people attention (attentional shift).

We also talk about this ([Link] ) that I reported in

Canvas to analyse the saliency of images.

Slide 15
The best way to answer the question regarding what people are looking at when
interacting with a product is to use an ET system.

1
[Link]
authorisation/product-information/product-information-reference-documents-
guidelines
Slide 16
Just because I want you to be aware of that. Usually, we train groups of 20
students with 4 eye trackers at the master level on the usage of ET. As you can
easily understand it is not possible to train all of you on the usage of the fully
functional ET.
An ET system costs between 20 to 35 thousand Euros and therefore there is
limited availability of such tools.

Nevertheless, this year for the first time I decided to introduce the topic of ET at
the bachelor level.

To my knowledge, no other universities (at least in EU) are training students on

ET at the bachelor level. Because it is considered an advanced research method.

Nevertheless, I believe that we found a way to help you in learning the basic
principles behind the use of eye-tracking tools so that (one day) you could apply
this know-how in a controlled setting by using fully functional eye-tracking
(ET) systems.

Slides 17
I want to provide you with initial knowledge and insights about:
• How and when you can use an ET
• Which data you can obtain from this equipment
• You can also have some practical insights about the fully functional ET by
following the optional tutorial online (see CANVAS) regarding how to
assemble an ET and analyse data of such a tool. This is optional but if you are
interested in ET this is a good starting point to learn how to handle some
practical aspects.

At the end of today, you will have the basic knowledge about ET and the metrics
that you can obtain from such a system. This will help you in future courses or
for your bachelor project. In general, the contents you are receiving today could
be considered a good starting point to build upon, and are equivalent to the
initial knowledge that you could receive on the first day of a master-level course
on ET.

Slide 18
To give you some insights regarding ET we created an online tool together with
the BMS LAB called BMSwebET.
The current version of the BMSwebET is the initial prototype. This was not
tested for usability (you are the first end-users). It is not as accurate as a fully
functional ET.
In general, web-based ET systems cannot be as accurate as fully functional
equipment that use specific sensors to collect eye movements, but we can
certainly improve the accuracy of the current prototype.

For instance, can compare people performances in locating information on

an interface

[Link]

In this example, I set up the test so that each participant will see the two
INTERFACES (prototype and original interface), ONE AFTER ANOTHER,
FOR ABOUT 10 SECONDS.
FOR EACH INTERFACE, I ask participants to LOCATE WITH their EYES
THE MENU.

e.g. <<In this test, you will see the two interfaces, one after another, for about 10
seconds. For each interface, please try to locate with your eyes the location of
the menu.>>

Slide 19
If you are interested in this topic and you want to know how to use a fully
functional ET you can access the online tutorial and resources on Canvas.
This is just for your knowledge and expertise.

[Link]

Slide 20
Vision and Eye-tracking
Eye-tracking and reading are strongly connected

Slide 21
Even though our vision appears extremely stable, the eye is constantly moving
around to construct a picture of what we are looking at.
Our vision can be divided into:
• Fixation. This is the pause of the eye movement on a specific area of the
visual field.
• Saccades. These are rapid movements of the eye from one fixation to
another

Slide 22
ET and vision
An ET registers the eye fixations.
• Fixations take place in our foveal area (F). This accounts for nearly half of
the visual information sent to our brain, including colours
• The Foveal area is highly detailed and provides complete clarity about what
we are looking at.
• only objects in the very centre of your vision are in clear focus. This is
called the Foveal Spotlight.
• However, the foveal area is only 8% of our total vision.
• The rest of our vision is composed of parafoveal (Pf) and peripheral
vision(P).
• These regions surround our foveal vision and help us to gain a sense of
what is happening around us.

Slide 23
THE EYE IN READING
Our eyes fix the fovea on a word, pause there for a fraction of a second to allow
basic patterns to be captured and transmitted to the brain for further analysis,
then jump to the next important word.

Eye fixations while reading always land on words! usually near the center, never
on word boundaries.

Very common small connector and function words like “and,” “or,” “is,” “but”
are usually skipped over, their presence either detected in perifoveal vision or
simply assumed.

Slide 24
According to reading researcher Kevin Larson (2004) 2, the reading area in and
around the fovea consists of three distinct zones (for European-language
scripts):
1. Closest to the fixation point is where word recognition takes place. This zone
is usually large enough to capture the word being fixated and often includes
smaller function words directly to the right of the fixated word.
2. The next zone extends a few letters past the word recognition zone. Here
readers gather preliminary information about the next letters in this zone.
2
Larson, K., 2004. The Science of word recognition, [Link]. July 2004, [Link]
[Link]/typography/ctfonts/[Link].
3. The final zone extends out to 15 letters past the fixation point. Information
gathered out this far is used to identify the length of upcoming words and to
identify the best location for the next fixation point.

From a historical point of view, both the knowledge about vision and research
about reading are strongly connected with the development of eye-tracking
systems.

Slide 25
Around 1879 Louis Émile Javal noticed for the first time that readers’ eyes do
not skim fluently through the text while reading, but make quick movements
(saccades) mixed with short pauses (fixations).

In 1898 Delabarre and Huey developed (independently) tools to follow the

movement of the eyes

In 1908, Edmund Huey built a device that could track eye movement during the
reading process and published The Psychology and Pedagogy of Reading.

In 1937, Guy Thomas Buswell, an educational psychologist, used light beams

which were reflected on readers’ eyes and recorded them on film.

In 1967 Alfred Yarbus Publish “Eye Movements and Vision”.

In this important book…[next slide]

Slide 26
Classic experiment of Alfred Yarbus
Yarbus in 1967 asked participants to look at this painting seven times, each time
with a different instruction before starting to look at the image…

Slide 27
Motivation/Goals and fixations
The seven instructions produced very different behaviour of the participant in
terms of eye movement:

(a) Free examination

(b) Estimate the material circumstances of the family?
(c) Give the ages of the people?
(d) What had the family been doing before the arrival of the unexpected visitor?
(e) Remember the clothes worn by the people
(f) Remember the positions of the people and objects in the room
(g) Estimate how long the visitor had been away from the family

Results of Yarbus showed that depending on the type of information that

participants must obtain, the distribution of the points of fixation on the object
changes correspondingly.
Different items of information were usually localized in different parts of an
object.

The GOAL that we have in mind -> CHANGES HOW WE LOOK AT

THINGS!

Slide 28
Break

Slide 29
Eye-tracking (How it works)
Eye-tracking is the process of recording:
• the point of gaze where one person is looking at
• the motion of eyes towards a specific point

Slide 30
Eye-tracking general considerations
In general, with eye tracking, you can understand
• in real-time, what the user is looking at
• PAY ATTENTION THIS IS NOT a measure of the focus of attention!
• It could be considered a measure of attentional shift.

• You can associate the data to:

• Their facial expressions and body language.

Slide 31
Pupil Centre Corneal Reflection
The most commonly used technique to track the eyes is the pupil centre corneal
reflection (PCCR).
The basic concept is to use a light source to illuminate the eye causing highly
visible reflections. The camera may capture an image of the eye showing these
reflections.
A filtered Infra-Red camera is embedded with the LED (light-emitting diode)
when the infrared rays hit the pupil in the cornea region the reflection produces a
glint on the cornea and in the pupil.
The equipment can then calculate a vector formed by the angle between the
cornea and pupil reflections—the direction of this vector, combined with other
geometrical features of the reflections, is then used to calculate the gaze
direction.

Once the image processing software has identified the centre of the pupil and the
location of the corneal reflection, the vector between them is measured.

Slide 32
Calibration and differences
Two general types of eye-tracking techniques are used:
bright-pupil and dark-pupil (depends on the type of eye-trackers).

Slide 33
Bright pupil
The light of the tracker is direct (coaxial with the optical path) and the eye acts
as a retroreflector creating a bright reflection
(this is similar to the red eyes effect in pictures)

Advantages
Good contrast and robust eye tracking
Low interference
disadvantages
Not effective for tracking outdoors (interference of sunlight)

Slide 34
Dark pupil
The light of the tracker is not direct and the reflection of the eye is minimal i.e.,
the pupil appears dark

Advantages
Better for outdoor
Disadvantages
Less accurate than the bright pupil detection

Slide 35
Screen ET - Combination of Bright and Dark pupil tracking
Slide 36
Wearable ET – Dark pupil tracking

Slide 37
Pros and Cons of ET

Advantages:
 Unobtrusiveness doesnt attract attention/distract
 It provides objective measures and understandable metrics
 We can track eyes movement which means that we can track the shift or
the diversion of attention of users during the interaction with an
interface

Disadvantages
 Perceiving an object is not equal to fixate an object with the eyes
 We can look at something but our focus of attention can be somewhere
else. Therefore, to track the Eyes Movements is not equal to track
attention

Slide 38
4 points on ET and eye movements from Andrew Duchowski 3
1. When we look at something: <<… we move our eyes to bring a particular
portion of the visible field of view into high resolution so that we may see in
fine detail whatever is at the central direction of gaze.>>

Slide 39
2. When we move our eyes from one point to another: <<… we divert our
attention to a point so that we can focus our concentration (if only for a very
brief moment) on the object or region of interest.>>

Slide 40
ET tracks the “diversion of attention” not the attention: <<… we may presume
that if we can track someone’s eye movements, we can follow along the path of
attention deployed by the observer.>>

3
Duchowski, A. T. (2007). Eye tracking methodology. Theory and practice. Springer
Slide 41
4. ETs give us an insight on what grabs the attention of people: << This may
give us some insight into what the observer found interesting…what drew their
attention… provide a clue as to how that person perceived whatever scene she or
he was viewing.>>

Slide 42
Data that you can obtain by an ET

Slide 43
Micro metrics
• Fixations (frequency and duration)
• Gaze movements: gaze points are what the eyes are looking at (a cluster
of gaze around a point constitute a fixation)
• Saccades movements: are quick simultaneous movements of both the
eyes between points of fixations
• Pupils dilatation and size: it is usually associated in the literature with the
cognitive workload and task difficulty

• Blink of the eyes: when we are blinking (shutting and opening the eyelids)
there is a signal loss which is accounted for in the ET systems. However, too
much signal loss could be an indicator of errors (hardware, software, light
etc.) during the detection. Exclusion criteria for the data.

Additional resources from BMSLAB

[Link]
[Link]
[Link]
analysis-software/)

Slide 44
Macro Metrics
These macro Metrics are based on a combination of the micro metrics
• Scanpath Maps
• Heat Maps
• Areas of Interests (AOI)
• Time to First Fixation
• Time spent (Dwell Time)
• Ratio
 • Revist

Slide 45
Scan path (gaze plot)
The use of gaze fixation and saccades metrics (time, duration) can tell you the
path used by participants:
Scan path duration, Scan path direction, Spatial density
Are used to visualize: What people are looking at in the product?

Slide 46
Heat map
By using the gaze points we can create heat areas. They are typically displayed
as a colour gradient overlay on the presented image or stimulus. The red, yellow,
and green colours represent in descending order the number of gaze points that
were directed towards parts of the image.

Slide 47
Areas of Interest
Usually, in ETs software you are asked to set up AREAS OF INTEREST (AOI).
These AOI are the areas of the objects you want to compare in terms of micro
metrics.
You can determine which regions of the visual area of the product you want to
extract metrics from.
For instance, in this picture, we can estimate the differences in terms of fixations
for the adverts in different positions.
To define AOI is useful to aggregate metrics to answer: Are people looking at
what we want (or we expect)?

Slide 48
Time to the first fixation
The amount of time that it takes a respondent (or all respondents on average) to
look at a specific AOI from stimulus onset.

This answer the question: how certain aspects of a visual scene are prioritized?

Banovic et al (2014) 4 show (see the picture) that when people have to choose a
product (food) they need significantly less time to fixate on the product they
4
Banović, M., Rosa, P. J., & Gamito, P. (2014). Eye of the beholder: Visual search, attention and product
choice. I see me, you see me: inferring cognitive and emotional processes from gazing behavior, 40-61.
want to choose compared to the products that they want to discard (non –chosen
products) at the first attempt.
The more they learn (stage 2 – after 12 tasks) the less they need to fixate
between the products they want to choose and the products they are not
choosing –i.e., the difference in time to the first fixation disappeared = learning
process.

Slide 49
Time spent on AOI (Dwell Time)
A long duration of looking at a certain region (AOI) can indicate a high level of
interest, while a shorter duration of fixations can indicate that other areas on the
screen might be more interesting or relevant.

This metric can answer questions regarding, e.g., How much time people spent
looking at certain things?
This can inform, for instance, about the importance that participants give to
different actions or information and the priority within different tasks or
information.

Drew et al (2018)5 used eye trackers to observe (during the performance of

different subtasks) the interpretation of radiology results interrupted or not by
competing tasks by using dwell time. They showed that interruptions of tasks
reduce the time spent of radiologists on reading/looking at the medical image
(the main task) and push the radiologist to shift their attention toward other (less
relevant) actions/aspects e.g. dictation.

Slide 50
Ratio
It provides information about how many of your respondents guided their gaze
towards a specific AOI.
It could be a ratio, for instance, of fixations count and duration or a ratio of time
spent on saccade movement (searching) and time of fixation.
Usually, a high ratio indicates that something (e.g., a logo, the login etc.) is easy
to find, while a lower ratio that something requires effort to be fixated. In the
picture, the sign of “Harrods” is more attractive than the “Bus”.
This answer the question: Are people looking at what I want to?

5
Drew, T., Williams, L. H., Aldred, B., Heilbrun, M. E., & Minoshima, S. (2018). Quantifying the costs of
interruption during diagnostic radiology interpretation using mobile eye-tracking glasses. Journal of Medical
Imaging, 5(3), 031406.
Slide 51
Revisit
Is the number of returned gaze to a particular AOI (how many times a
participant look at the area).
Which are the hottest areas (for positive and negative reasons) and which are the
COLD areas.
Of course, the reasons why an area is attractive should be investigated.

Slide 52
Recap
• Eye trackers enable the observation and recording of eye behaviour such
as pupil dilation, fixation and movements.
• Eye trackers may not replace other techniques (qualitative and
quantitative). Eye-tracking is an enhancement!
• Eye trackers are not mind-reading devices, and they can only tell
us what the person looked at but not why
• Just because a fixation was registered (by a device) does not necessarily
mean that the user really saw an object i.e., that they registered
cognitively in their brain the object.

Slide 53
Best practices at a glance
• Have optimal environment and lighting conditions
• Work with a dual-screen configuration
• Clean your computer
• Appropriately train all people involved
• Use a clear and robust procedure
• Simplify your (lab) setup

Slide 54
Do I need the ET
A fundamental question is: Do I need an eye tracker to answer my question?
Keep in mind that eye trackers are useful tools only when:
- Results may bring actionable insights
- There are no simplest methods to answer a research question or a buy-in
boost i.e., higher support and participation

End