Human-robot interaction (HRI) studies the interaction between people and robotic systems.

While
robots are traditionally operated using user interfaces gleaned from human-computer interaction,
such as control panels or screen-based interfaces, there is potential to move toward more natural
modes of interaction. These will, to a large extent, be modeled on how people interact with each
other and are composed of verbal and nonverbal ways of interacting.
HRI is a broad church: at one end of the spectrum, it studies how an operator can control one or more
robotic systems through traditional methods and sometimes focuses on the cognitive load imposed
by controlling one or more robots. For example, if an operator coordinates a handful of
semiautonomous drones during a search and rescue operation, how can the cognitive load on the
operator be optimized to maximize the efficiency of the overall mission (e.g., Goodrich et al. 2011)?
On the other end of the spectrum of HRI, one finds research into natural interaction between humans
and robots. This field is also known as social robotics, and the large majority of research efforts in
HRI concentrate on it (Bartneck et al. 2020). The holy grail of social HRI, of course, is the natural
and intuitive interaction between people and artificial systems. On one hand, this is a technical effort,
with results in social signal processing, artificial intelligence, and robotics coming together to create
social robots. But social robotics offers a unique opportunity to study how people respond and
interact with artificial social agents. Social robots take up a singular position in agents we interact
with. The interaction between people has, of course, been the subject of extensive study for more
than a century, and the interaction between animals and people has been researched at length, but
robots are a new and, until recently, unexplored “species.” Until recently, we have known very little
about how people interact with robots, and our relation and interaction with robots is continuously
evolving. Culture, media, education, context, and exposure change our attitudes toward robots and
the ways in which we interact with them. When we meet a robot, several automatic social responses
kick in that color our interaction with the robot; these responses evolved or developed to interact
with other humans and often transfer to our interaction with robots.
This is not unique to robots. We treat all technology to some extent as if it is humanlike, something
known as anthropomorphization, which Clifford Nass called the “media equation.” We relate to
media computers, printers, mobile phones, and of course robots as if they are human (Reeves and
Nass 1996). Everyone has at one time or another muttered at their computer when it crashed or
cursed their printer when the paper jammed, but the media equation theory takes things a little
further by claiming that we not only respond to these media as if they were persons but ascribe
personal qualities to each, such as a personality, expertise, and even gender. And we often do so
without being aware of it. The media equation is taken to the extreme in social robots, as the
appearance of the robot and its behavior (the things it does) have been carefully designed to elicit a
strong social response from us.

Cognitive and Neuroscientific Insights Informing HRI

Social psychology is immediately relevant to the design of social robots, and knowingly
or not, designers and programmers of social robots take concepts and theories from social
psychology into consideration when building robots. Failing to do so usually results in a
disappointing HRI. Whether you wish to create a friendly robot or a horror experience,
you will rely on fundamentals from social psychology when designing the appearance of your robot
and its interaction.
The media equation predicts that people will perceive and treat robots in a humanlike way, but the
fact that we readily interpret animated objects as having humanlike emotions and intentions have
been known for a long time. Fritz Heider and Marianne Simmel (1944), two psychologists working
together in the United States, published an influential paper titled “An Experimental Study of
Apparent Behavior” in which they described a simple and elegant experiment: They asked people to
describe short film clips of moving geometric figures, such as circles and triangles. The figures were
animated by hand and seemed to play out a short story. Everyone who saw the videos ascribed
emotions and intentions to the figures. The original videos from the 1940s can still be found online,
and even now when seeing the videos, people readily see the figures having emotions, intentions, and
motivations, and they see a narrative unfold over the few minutes of video runtime. This is our social
brain interpreting the world around it and, specifically, our theory of mind—our ability to attribute
mental states to others and ourself—overinterpreting moving geometric figures. This concept has
been gratefully used by animators, and some striking examples exist of very minimalist animation
films that show that very little is needed to nudge our social brain into interpreting simple shapes and
movement as having agency (Thomas and Johnston 1995). If you have ever observed a vacuuming
robot moving around the room, you have probably been struck by its animal like appearance as it
scuttles around the room, gently bumping into furniture and working hard at getting specks of dirt
from the floor. These robots are not designed to be social, and yet they still evoke a strong social
response in us. In social robots, designers add elements such as a head, eyes, and reactive responses
to evoke a strong social response in people.
One such social response on which designers rely is pareidolia: the tendency to see human or animal
forms in objects, such as dogs in clouds or the face of Elvis on a piece of burnt toast. Using
magnetoencephalography (MEG), researchers found that the ventral fusiform face area (FFA) in the
brain is involved. The FFA has been implicated in detecting faces of people and animals and is also
involved in distinguishing animate from inanimate visual stimuli (Kanwisher et al. 1999). This area
shows a cortical response 170 ms after we are presented with a human face and shows a similar but
slightly earlier activation of 165 ms when seeing objects that resemble faces (Hadjikhani et al. 2009).
This suggests that seeing face is a very early and automatic response and is not something the brain
puzzles together after extended cognitive processing. As such, we can assume that responses to
robots with a face are early and automatic.

T676Design of Social Robots

One aspect that often arises in robot design is that of neoteny, a juvenile appearance that usually
evokes a caring response and is generally described as “cute.” Young animals, including human
children, have a large head, large eyes, chubby cheeks, a small chin, a flat face, a small nose, and
relatively short arms and legs. Konrad Lorenz (1982) argued that infantile and juvenile features have
a biological function by triggering nurturing responses in adults. We are so keen on neotenous
appearances that we breed domesticated animals to retain neotenous features. Many breeds of
smaller dogs retain juvenile features, such as a short snout and a relatively large head and large eyes,
and consequently are considered cute by most people. The nurturing response is also largely cross-
cultural. The same physical features evoke a similar response in people regardless of culture or
background. This has been used to good effect by robot designers: if a robot is to be likeable,
designers will give it features that evoke a caring response. This not only causes people interacting
with the robot to find it cute but also makes them inclined to feel more generous toward any mistakes
the robot makes. The opposite seems to hold as well. Robots that have adult, or gerontomorphic,
features appear less cute and have less appeal. While there is no research on this yet, it is likely that
they are considered more knowledgeable and authoritative, and therefore it makes sense for robot
designers to give robots that need to radiate authority or trust an adult appearance (see figure 19.1).
Perhaps the most well-known issue in robot design is that of the uncanny valley
(figure 19.2). This effect, first hypothesized by Mori in 1970 (Mori et al. 2012), describes
the familiarity or appeal of a robot as a function of its human likeness. Mori in his original
paper wrote about shinwa-kan, which does not translate well into English but is sometimes described
as familiarity, appeal, likeability, or affinity. When a robot does not resemble a human, it has low
familiarity. This gradually goes up: As human likeness increases, so does familiarity, until the robot
is almost humanlike but not quite. At this point familiarity gets knocked back, and when plotted this
resembles a sharp dip in the familiarity curve. This is known as the uncanny valley. Androids, robots
that have humanlike skin but lack humanlike motions, find themselves firmly in the uncanny valley.
You can climb out of the uncanny valley by making a robot that is almost indistinguishable from a
person. Note that the uncanny valley effect is more pronounced when the robot is moving: the
familiarity or eeriness of the robot is more exaggerated when the robot is animated. Mori never
backed up his hypothesis with data, but later empirical research has shown that the uncanny valley is
indeed real (MacDorman and Ishiguro 2006; MacDorman and Chattopadhyay 2016).
Rosenthal-von der Pütten et al. (2019) studied the neural mechanisms underlying human responses to
artificial agents and, specifically, the uncanny valley response. They suggest that the uncanny valley
requires a neural system that derives human likeness from sensor cues followed by a downstream
system that integrates these signals into a nonlinear value function representing the uncanny valley
response curve. Using functional magnetic resonance imaging (f MRI), they investigated the neural
activity of people when observing people and artificial agents, including robots, while making rated
responses or expressing a preference for stimuli. They found that the ventromedial prefrontal cortex
encoded a representation of the uncanny valley, in which the subjective likability of artificial agents
was a nonlinear function of human likeness. Functionally connected areas in the brain encoded
critical inputs for signals: the temporoparietal junction (TPJ) encoded a linear human likeness
continuum. The TPJ was also found to be active in detecting agency (Mar et al. 2007), belief
attribution, and learning from others (Rosenthal-von der Pütten et al. 2019). In addition, nonlinear
representations of human likeness found in the dorsomedial prefrontal cortex (DMPFC) and fusiform
gyrus (FFG) emphasized a human-nonhuman distinction. The DMPFC is known to show activity
when attributing mental states to others or when assessing performance of others or of the self
(Rosenthal-von der Pütten et al. 2019), while the FFG is implicated in distinguishing animate from
inanimate stimuli (Chaminade et al. 2010). Activation in the amygdala, which in humans is
implicated in the formation and storage of memories associated with emotional events, was found to
predict a negative response to artificial agents. As such, the brain seems to have a direct neural
representation of the uncanny valley, or rather the uncanny valley can be explained by brain
processes that are universal to all people. If the same neural mechanisms implicated in assessing
people, people’s behavior, and the agency of stimuli are also active when we perceive robots, then
this might help us design more effective robots. Generally, what makes people appealing will make
robots appealing, and only cultural conditioning and habituation are likely to change the initial, and
often automatic, responses we have to robots. When discussing the uncanny valley, one cannot
escape mentioning androids and perhaps their more famous ilk, the Geminoids. A Geminoid, a
contraction of Gemini (meaning “twins” in Latin) and android—is modeled after a human being and
as such is their robotic doppelgänger. Hiroshi Ishiguro was the first to build Geminoids, and the
various models that have been built—including ones of himself, his daughter, and a Japanese news
anchor—have been the subject of academic study into the uncanny valley effect. These studies
showed that the uncanny valley effect is sometimes not there or cannot be explained by relying on
appearance alone. Bartneck et al. (2009) had people briefly interact with Hiroshi Ishiguro or with his
Geminoid. While participants could clearly distinguish an android from a human, and unsurprisingly
found the human to be more humanlike, the android was not liked less, which goes against Mori’s
prediction. This result and others suggest that the uncanny valley is a multidimensional phenomenon
and that the two-dimensional plot of figure 19.2 should be revised. Instead, the effect is caused by a
mismatch between different aspects of the robot: a robot that appears human but moves like a robot
causes tension in the observer, which leads to an eerie appearance (Moore 2012).