Human Computer Interaction

 CEN 3721

Lecture 1: Introduction
Debra Lee Davis, M.S., M.A., Ph.D
Intro to Human-Computer Interaction
 What is HCI?
Study of interaction between people and computers
The intersection of computer science, behavioral
science, design, ergonomics, and other fields of study
Goal of HCI: Improve the interactions between users
and computers
Design systems that minimize barriers between the
human cognitive model of what they want to
accomplish, and the computer’s understanding of the
user’s task
Source: MIND: Introduction to Cognitive Science 2nd edition, by Paul Thagard
Human-Computer Interaction (cont)
What do we need to study for successful HCI?
Cognitive science: how the mind works; how do we learn;
how much can we easily remember; what is
pleasant/unpleasant; what motivates/demotivates people;
perceptions
Ergonomics of sight, hearing, and motion: how do we look
at screens, how do we interact with them
Physics of sight, hearing, motion, color
Cultural differences: shapes, colors, and images with
cultural Meanings
Demographic differences: age, gender, physical
abilities/disabilities
Technology: input/output modalities
Human-Machine Interaction
Human Factors: Study of human factors looks at the
design of objects that humans use or interact with
Engineering mantra in the past: “Form-Fit-Function”
Form: shape, size, dimensions, mass, or visual parameters
which uniquely characterize an item
Fit: the ability to physically interface or interconnect with or
become an integral part of another item or assembly
Function: the action(s) that it is designed to perform
Is it still used today?
Yes! (NASA, US Armed Forces, Many large corporations, etc.)
There is still a lot of really bad designs
Cognitive Science: Studying the Mind
 Why study the mind?
Computers can be made more intelligent by reflecting on
what makes people intelligent
How does the mind work? What about the brain? What’s
the difference?
Cognitive Science: the science that explains how people
accomplish various kinds of thinking
Knowledge in the mind consists of Mental
Representations
Mental procedures operate on mental representations to
produce thought and action
Different kinds of mental representations foster different
kinds of mental procedures
Cognition
 Cognition – from Latin base cognitio – “know
together”
 The collection of mental processes and activities
used in perceiving, learning, remembering,
thinking, and understanding and the act of using
those processes
Cognitive Processes
Learning and Memory
Thinking and Reasoning (Planning,
Decision Making, Problem Solving ...)
Language
Vision-Perception
Social Cognition
Emotion
Dreaming and Consciousness
Cognitive Processes - How do we
perceive the world?
Perception is biased by experience

You are meeting with a real-estate manager to discuss

adding 5 new buildings. The manager shows you this map.
What do you see?
You are meeting with an advertising manager. The manager
shows you a sketch of an ad that contains 1 word. What do
you see?
Source for this section: Designing with the Mind in Mind by Jeff Johnson (2010)
Perception is biased by experience
What do you see?

Now you are told: It’s a Dalmatian sniffing the

ground. Now what do you see?
Perception is biased by experience
What does this mean?

New Vaccine Contains Rabies

The area in which you live has a widespread problem
with rabies
OR
You love your pets and want to ensure they are
vaccinated, but you know people who’ve had bad
experiences in the past with vaccine quality
Perception is biased by Context
Read the following:

What letter is ?

What does “Polish silverware” mean?

 [Link]

Surrounding context also works between different senses

Perception is biased by our Goals &
Plans
We tend to focus on our goals & ignore everything else
Adults tend to be more focused on goals than children
Children are more stimulus driven, more distractible but less biased
Exercise: Look for the scissors

Was there a screwdriver there also? What about a tape dispenser? A

vise?
Perception is biased by our Goals &
Plans
Exercise 2: Find a map of campus:

How does this work?

Influencing where
we actively look
Sensitizing our
perceptual system to
certain features
(e.g., red cars)

How many of
you saw this?
Design Implications
Understand our cognitive capabilities and
propensities
Avoid ambiguity
Be consistent
Know who your users are
Understand context
Understand user goals
Disciplines in Cognitive Science
 Computer Science- Artificial Intelligence
 Neuroscience
 Psychology - Cognitive Psychology -
Developmental Psychology – Social Psychology
 Philosophy
 Linguistics
 Anthropology, Education
Cognitive Science
Central hypothesis of cognitive science: Thinking can
best be understood in terms of representational structures
in the mind and computational processes that operate on
those structures (Thagard)

CRUM: Computational-Representational Understanding

of the Mind

The Mind Computer Program

Mental Representations + Data structures + algorithms =
computational procedures = running programs
Thinking
Mental Representations
 Various kinds of mental representations have been
proposed, including:
Rules
Concepts
Images
Analogies
Explanatory pattern:
People have mental representations
People have algorithmic processes that operate on those
Representations
The processes, applied to the representations, produce
the behavior
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules
Concepts
Analogies
Images
Neural connections
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules
Concepts
Analogies
Images
Neural connections
Logic
Why study logic?
Many basic ideas about representation and computation
have come from the study of logic
Considered to be central to work on reasoning
Has substantial representational power
Formal logic began with the Greek philosopher
Aristotle more than 2000 years ago
He studied patterns of inference called syllogisms
Example:
All students are overworked.
Mary is a student
Therefore, Mary is overworked.
Watson’s Selection Task
Have 4 cards, each of which has a number on one side
and a letter on the other side
Rule to be tested: If a card has an ‘A’ on one side, then
it has a 4 on the other side
Which cards must be turned over to determine
whether this rule holds?
 Watson’s Selection Task
Solution:
Must turn over the A to check whether there is a 4 on the
reverse
 Modus ponens: If A then 4 (If P then Q. P, therefore Q)

Must turn over the 7: if there is an A on the reverse, it

refutes the rule in question
 Modus tollens: If A then 4; 7 means not-4; so not-A is

required for the rule to hold (If P then Q. Not Q, therefore Not P)
Turning over the B is irrelevant to the rule
Turning over the 4 is irrelevant to the rule
Watson’s Selection Task
People have less difficulty with this problem if given in
context
Suppose the cards have information about whether
individuals are in a bar on one side, and numbers on
the other side indicating their age
Rule to be tested: If a person is in the bar, then he/she
is 21 years of age or older
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic

Rules
Concepts
Analogies
Images
Neural connections
Rules
Rules are if-then structures
Similar to conditional logic, but rules are more
applicable to the real world
General information about the world: Students are
overworked. If x is a student, then x is overworked.
How to do things: If you register early, then you will get
the courses you want
Linguistic regularities: If a sentence has a plural subject,
then it has a plural verb
Rules
Rules can represent real-world goals
If you want to go home
and you have the bus fare,
then you can catch a bus.
Forward reasoning with the rule simulates backward
reasoning
You go home
if you have the bus fare
and you catch a bus.
Widely used Expert Systems
E.g., financial services, medical diagnostics, etc.
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules

Concepts
Analogies
Images
Neural connections
Concepts
Concepts are representations of typical entities, and are
not strict definitions
Concepts are associated with default inheritance in
hierarchies
Views of the nature of concepts are often described as
frame , script and schema
 Frames (Minsky) - conceptual structures containers with
default values
 Shank and Abelson showed that a great deal of our social
knowledge consists of scripts (e.g., how to do something)
Schemas (Piaget) - cognitive framework or concept that
helps organize and interpret information (e.g., dog, horse)
Concepts
Concepts can be innate, or formed from
experience, from rules, or by combining other
concepts we already have
Innate: basic physical concepts of objects and
behavior; ex. Playing “peek-a-boo” with babies
Learned: child learns to distinguish one type of
animal from other types of animals
In language, concepts are represented by words
and phrases that involve a whole conceptual
system
“Barking up a tree”, “Beat around the bush”, etc.
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules
Concepts

Analogies
Images
Neural connections
Analogies
 Analogical thinking: dealing with a new
situation by adapting a similar familiar situation
There are numerous computational models for
analogical reasoning
Also called case-based reasoning, used in artificial
intelligence
Cognitive science itself uses analogies: models the
computer to explain how the mind works
Can be a fertile source of creative designs
George de Mestral invented Velcro after observing how
burrs stuck to his dog
Reverse engineering is used to figure out how to make
an analogous product
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules
Concepts
Analogies

Images
Neural connections
Images
Human thinking involves pictorial representations
that are different from verbal ones
Mental imagery:
How do you get from your home to FIU?
Images can be visual or non-visual:
Sensory images such as taste and smell
Tactile images: What does a baby’s skin feel like?
Motor images: How do you slam-dunk a basketball?
Emotional images: How did you feel when…? How
would you feel if…?
Images
Vision: easy for humans, but extracting information
from millions of pixels and understanding that
information is very difficult for computers
Optical illusions and cognitive illusions
Necker cube: Is the opening at the top, or in the front?

Newbold’s Animated Necker Cube: [Link]

Vision – how we see color
Color perception is relevant to user interface design:
Our vision is optimized to detect contrasts (edges), not
absolute brightness (from birth!)
Our ability to distinguish colors depends on how colors
are presented
 Paleness: The paler (less saturated) two colors are, the harder
it is to tell them apart
 Color patch size: The smaller or thinner objects are, the

harder it is to distinguish their colors (e.g., text).

 Separation: The more separated color patches are, the more

difficult it is to distinguish their colors

 [Link]
Vision – how we see color
Some people have color-blindness
 Most common: red/green

The user’s display and the viewing conditions affect

color perception

 More Optical Illusions:

Guide for Using Color
Distinguish colors by saturation and brightness as well
as hue (high contrast)
 Use distinctive colors
 Avoid color pairs that color-blind people cannot
distinguish
Use color redundantly with other cues
Separate strong opponent colors
Images
Imagery contains a
great deal more
information in a
concise manner than
can sometimes
reasonably be
described otherwise:
Physical
characteristics
Emotional
characteristics
Message it conveys,
etc.
Approaches to Modeling the Mind
6 main approaches to modeling the mind:
Logic
Rules
Concepts
Analogies
Images

Neural connections
Neural Connections
Physical brain contains neurons which
signal each other through contacts at
specialized points called synapses
Computational approaches emphasize the
importance of connections
Neural networks
Parallel distributed processing
Many cognitive tasks are understood computationally
in terms of processing that simultaneously satisfies
numerous constraints
Challenges for Cognitive Science
CRUM has many shortcomings. It does not
address:
Emotion
Consciousness
Physical Body (movement, hormonal and chemical
influences, etc.)
Influence of our physical environment
CRUM also does not
The mind is a dynamic system, not a computational
system
Humans are innate social creatures