Human Computer Interaction (SE613)
Dr. Laila Esheiba
Chapter 6
Emotional
Interaction
Presented By: Marwa Zakaria
Table of contents
01 Introduction 02 Emotions and the User
Experience
03 Expressive Interfaces 04 Annoying Interfaces
and Emotional Design
Affective Computing Persuasive Technologies
05 06
and Emotional AI and Behavioral Change
07 Anthropomorphism
Objective
The main goals of this chapter are to accomplish the following:
1 Explain how our emotions relate to behavior and the user experience.
2 Explain what are expressive and annoying interfaces and the effects they can have
on people.
3 Introduce the area of emotion recognition and how it is used.
4 Describe how technologies can be designed to change people’s behavior.
5 Provide an overview on how anthropomorphism has been applied in interaction
design.
Introduction
Section: 6.1
Have You Ever!
● Receive some bad news, how does it affect you?
● Do you feel upset, sad, angry, or annoyed—or all
of these?
● Does it put you in a bad mood for the rest of the
day?
● How might technology help?
Imagine !
Wearable Technology
• That could detect how you were feeling.
• provides a certain kind of information and
suggestions geared toward helping to improve
• your mood.
• Would you find such a device helpful, or would you find it
unnerving that a machine was trying to cheer you up?
Emotional AI – Emotional Design
Emotional AI
Designing technology to detect and recognize
someone’s emotions automatically from sensing
aspects of their facial expressions, body
movements, gestures, and so forth is called
Emotional AI.
Emotional AI – Emotional Design
Emotional Design
• It examines why people become emotionally attached to certain
products.
• The focus here is on how to design interactive products to evoke
certain kinds of emotional responses in people.
• How social robots might help reduce loneliness, and how to
change human behavior through the use of emotive feedback.
Emotional AI - Affective computing
Applications of Emotional AI
• Health (Patient Care), driving, Medical Diagnosis, and
education. These can be used to determine if someone is
happy, angry, bored, frustrated, and trigger an appropriate
technology intervention, such as making a suggestion to
them to stop and reflect or recommending a particular
activity for them to do.
Emotions and the
02 User Experience
Section 6.2
Consider Shopping Online Experience
Realization Of Needing The need to buy new laptop
Anticipation Of Purchasing Follow the online shopping
steps to get the product
Consider Shopping Online Experience
The Possible Feelings That Might Be Experienced
• Joy or frustration of finding out more about what product.
• Confusion deciding which to choose from potentially hundreds or even
thousands of items.
• The shock of how much it costs and the disappointment that it is too
expensive.
Consider Shopping Online Experience
The Possible Feelings That Might Be Experienced
• Revising your decision may be accompanied by annoyance if you discover
that nothing is as good as the first choice.
• Finally, when you make your decision, a sense of relief is often
experienced.
• The requirement to fill in the many details raises the possibility of making
a mistake.
“what makes people feel happy, sad, annoyed,
anxious, frustrated, motivated, and so on, and then
using this knowledge to inform
the design of different aspects of the user
experience..”
—Emotional Interaction
How to Influence
People’s
Emotions?
• The goal is to make people feel sad or
upset at what they observe and make
them want to do something to help, such
as by donating.
How to trigger a strong emotional
response in the viewer.
How emotions affect behavior and how behavior
affects emotions?
Examine 1
How people express themselves
at different situations
2 Examine
Person’s How people read each
Expressions 3 other’s expressions.
Facial, body language, voice
can trigger emotional tone
responses in others.
Many theorists argue that emotions cause behavior,
fear brings about flight, and that anger initiates the
fight perspective.
How to design App based on user experiences?
25% 80%
Simple And Short- Complex And Long-
lived lasting
Automatic emotions Conscious emotions, tend to be slow to
(also knowns as affect) happen develop and they are often the result of
rapidly, typically within a fraction of a conscious cognitive behavior, such as
second. weighing the odds, reflection, or
contemplation.
Design Model for Emotion & Behaviour
Reflective design
considering the
Behavioral personal value of a
design product in a
use and equates to particular culture
the traditional values
of usability.
Visceral design
Make products look,
feel, and sound good
Swatch Watch
The Design Of A Swatch Can Be Viewed In
Terms Of The Three Levels:
1. Reflective Level
• The use of cultural images and
graphical elements.
2. Behavioral Level
• its affordances so it’s usable
3. Visceral Level
• The brilliant colors, wild designs and
art attract users’ attention
,
Expressive Interfaces and
Emotional Design
Section: 6.3
Features That Make Interface Expressive
Emojis
Emotional
sounds
Responses
virtual agents colors
icons
• The aesthetics interface can have a positive effect
on people’s perception of the system’s usability.
• When the look and feel of an interface is pleasing
and pleasurable—for example through beautiful
graphics or the way that the elements have been
put together—people are likely to be more tolerant
and prepared to wait a few more seconds for a
website to download.
• Good-looking interfaces are generally more
satisfying and pleasurable to use.
Nest thermostat Example
• The popular Nest thermostat provides an automatic way
of controlling home heating that is personalized to the
habits and needs of the occupants.
• Works out how to save money by reducing energy
consumption when not needed.
• Have a minimalist and aesthetically pleasing interface.
• This is very different from earlier generations of
automatic thermostats, which has lots of complicated
buttons and a dull screen.
04 Annoying Interfaces
Section 6.4
Examples of Usability Problems
Getting a printer to work Trying to switch Attaching the ends of
with a new digital camera from watching a DVD cables between
Or inserting a SIM card into to a TV channel smartphones and laptops
a smartphone
Advantages
• An interface metaphor of a warm, cozy
living room, replete with fire and furniture
• Intended to convey a comfortable
feeling.
Disadvantages
• The interface too cute and childish
Advantages
• Cartoon character, with a warm personality
Clippy
Disadvantages Win 98
• Many Microsoft users found it too intrusive,
distracting them from their work.
Solution
• Replaced by virtual assistants who talk in
speech bubbles that have no physical
appearance who the user can talk to via
LiveChat.
Situations That Might Cause Negative
Emotional Responses
• The application doesn’t • User’s expectations are not met.
• Error messages pop up that
work properly or crashes.
are vague
• Or the system doesn’t do
what the user wants it to do • The appearance of an interface is too
noisy
• Poorly laid-out touchpads, • The overuse or automatic playing of
sound effects and music
control panels, that cause
• The system requires users to carry out
users to press the wrong
too many steps to perform a task
keys or buttons
Is there a better way of letting users know when
a link to a website is not working?
Suggested Solution
• Researchers found that people are more
forgiving when a computer says it’s sorry
after making a mistake.
• Several companies now provide
alternative and humorous “error” landing
pages.
• Errors are intended to make light of the
embarrassing situation and to take the
blame away from the user
Should Voice Assistants Teach Kids Good Manners?
• Some parents, are worried that this lack of etiquette
when their kids speak to voice assistant could
develop into a new social norm
• It could transfer over to how they talk to real human
beings.
• One would hope that parents would continue to teach
their children good manners.
• Configure Alexa and other voice assistants to reward
children when they are polite to them
Affective Computing
05 and Emotional AI
Section 6.5
How to use computers to recognize
and express emotions in the same
way as humans do
—Affective Computing
Techniques Used in Affective Computing
Cameras for measuring facial expressions
Biosensors placed on fingers or palms to measure galvanic skin response
Affective expression in speech (voice quality, pitch, loudness)
Body movement and gestures, as detected by motion capture systems
The use of automated facial coding is gaining popularity in commercial settings, especially
in marketing and e-commerce. computer vision and machine learning algorithms
to catalog a user’s emotional reactions to digital content
Facial coding using Affdex software
• If a user screws up their face when an ads.
pops up, this suggests that they feel
disgusted.
• If they start smiling, it suggests that they are
feeling happy.
• The website can then adapt its ad, movie
storyline, or content to what it perceives the
person needs at that point in their emotional
state.
All-the-Feels app
shows the biometric data of a streamer playing a video
games
• A dashboard provides a visualization of the streamer’s heart rate, skin
conductance, and emotions.
• This additional layer of data has been found to enhance the spectator
experience and improve the connection between the streamer and spectators.
Affectiva has also started to analyze drivers’ facial expressions when on the road to improve driver safety.
The emotional AI software perceives if a driver is angry and then suggests deep breath and plays soothing music
Do you think it is ethical
that technology is trying
to read your emotions?
• Human beings will suggest things to each other, often based
on what they think the other is feeling.
• However, some people may not like the idea that an app can
do the same, for example, suggesting what you should eat,
watch, or do based on how it analyzes your facial
expressions.
Persuasive Technologies
06 and Behavioral Change
Section 6.6
A diversity of techniques has been used at
the interface level to draw people’s attention
to certain kinds of information in an attempt
to change what they do or think.
—Persuasive Design
Pop up Ads Recommendations Personal Reminders
Persuasive Design Example
• Amazon’s one-click mechanism that makes it
easy to buy something on its online store.
• It’s recommender systems suggest specific
books, hotels, restaurants, and so forth, that
a reader might want to try based on their
previous purchases, choices.
Nintendo’s Pokémon Pikachu device
• It is designed to motivate children into being more
physically active on a consistent basis.
• The owner of the digital pet that lives in the device was
required to walk, run, or jump each day to keep it alive.
• If the owner did not exercise for a week, the virtual pet
became angry and refused to play anymore.
• This use of positive rewarding and sulking can be a
powerful means of persuasion.
• The children often become emotionally attached to
their virtual pets
Using the HAPIfork in a restaurant
• HAPIfork is a device that was developed to help
someone monitor and track their eating habits.
• It detects that they are eating too quickly, it will
vibrate, providing the eater with real-time feedback
intended to slow them down.
• The assumption is that eating too fast results in
poor digestion and poor weight control.
Using the HAPIfork in a restaurant
• The data collected are turned into a dashboard of graphs and
statistics so that the user can see each week whether their fork
behavior is improving.
Tidy Street public electricity graph
• Sustainable HCI, The global concern about • Tidy Street project in United Kingdom created
climate change has also led a number of HCI a large-scale visualization of the street’s
researchers to design and evaluate various electricity usage on the road surface using
energy-sensing devices that display real- chalk
time feedback to find ways of helping • was updated each day to represent how the
people reduce their energy consumption average electricity usage of the street.
07 Anthropomorphism
Section 6.7
The propensity people have to apply
human qualities to animals
and objects.
—Anthropomorphism
Anthropomorphism Example
• Interactive dolls have been designed to
talk, sense, and understand the world
around them.
• Using sensor-based technologies &
speech recognition.
• For example, the interactive doll Luvabella
exhibits facial expressions, such as
blinking, smiling, and making baby noises
in response to how her owner plays and
looks after her.
A Robot or a Cuddly Pet? Which do you prefer and why?
• Early robot pets were made of hard • For making robot pets cuddly is to enhance
materials that made them look shiny. the emotional experience people receive
• In contrast, a more recent trend has been to through using their sense of touch.
make them look and feel more like real pets • The robot “detects” the way it is touched
by covering them up in fur and making them sensors then responds accordingly, using
behave in more cute, pet-like ways. the ears, breathing, to communicate its
emotional state.
The Zora Robot
• It was a social robot for healthcare to
entertain seniors and to help them get some
exercise:
• Some people found this kind of robot
care a little demeaning. It can never
match the human touch and warmth
that patients need.
• There is no harm in it playing an
entertaining and motivating role
alongside human caregivers.
Thanks!