Human Computer Interaction

Dr. Zartasha Baloch

Department of CSE, MUET Jamshoro
Chapter 2: Understanding Computer
Learning Objectives
• In this lecture the students will be able to
• Understanding Computer
• Text entry devices
• Positioning & Pointing devices
• Display devices
• Memory
• Processing & Networks
The Computer
A computer system is made up of various elements

Each of these elements affects the interaction

• input devices – text entry and pointing
• output devices – screen (small & large), digital paper
• virtual reality – special interaction and display devices
• physical interaction – e.g. sound, haptic, bio-sensing
• paper – as output (print) and input (scan)
• memory – RAM & permanent media, capacity & access
• processing – speed of processing, networks
Interacting with computers

to understand human–computer interaction

… need to understand computers!

what goes in and out

devices, paper,
sensors, etc.

what can it do?

memory, processing,
networks
Interactivity?
Long ago in a galaxy far away … batch processing
• punched card stacks or large data files prepared
• long wait ….
• line printer output
… and if it is not right …

Now most computing is interactive

• rapid feedback
• the user in control (most of the time)
• doing rather than thinking …

Is faster always better?

Richer interaction
Sensors and devices everywhere
 Text Entry Devices
keyboards (QWERTY et al.)
chord keyboards, phone pads
handwriting, speech
Keyboards
• Most common text input device
• Allows rapid entry of text by experienced users

• Keypress closes connection, causing a character code to be sent

• Usually connected by cable, but can be wireless
QWERTY Layout
• Standardised layout
• QWERTY arrangement not optimal for typing
– layout to prevent typewriters jamming!
• Alternative designs allow faster typing but large social base of QWERTY typists produces
reluctance to change.
alternative keyboard layouts
Alphabetic
• keys arranged in alphabetic order
• not faster for trained typists
• not faster for beginners either!

Dvorak
• common letters under dominant fingers
• biased towards right hand
• common combinations of letters alternate between hands
• 10-15% improvement in speed and reduction in fatigue
• But - large social base of QWERTY typists produce market pressures not to change
Phone Pad and T9 entry
• use numeric keys with
multiple presses
2–abc 6-mno
3-def 7-pqrs
4-ghi 8-tuv
5-jkl 9-wxyz
hello = 4433555[pause]555666
surprisingly fast!

• T9 predictive entry
• type as if single key for each letter
• use dictionary to ‘guess’ the right word
• hello = 43556 …
• but 26 -> menu ‘am’ or ‘an’
Handwriting Recognition
• Text can be input into the computer, using a pen and a digesting tablet
• natural interaction

• Technical problems:
• capturing all useful information - stroke path, pressure, etc. in a natural manner
• segmenting joined up writing into individual letters
• interpreting individual letters
• coping with different styles of handwriting

• Used in PDAs, and tablet computers …

… leave the keyboard on the desk!
Speech recognition
• Improving rapidly

• Most successful when:

• single user – initial training and learns peculiarities
• limited vocabulary systems

• Problems with
• external noise interfering
• imprecision of pronunciation
• large vocabularies
• different speakers
Numeric keypads

• for entering numbers quickly:

• calculator, PC keyboard
• for telephones
1 2 3 7 8 9

not the same!! 4 5 6 4 5 6

7 8 9 1 2 3

0 # 0 . =
*
telephone calculator
Positioning, Pointing and Drawing
1. Positioning Devices
• These devices help users specify a location on a screen or in a digital
environment.
a. Touchscreens
b. Trackpads/Touchpads
c. Optical and Laser Mice
d. Trackballs
e. Game Controllers
Touchscreens
• Description: Responsive displays that detect the location of a finger or stylus
touch.
• Usage: Smartphones, tablets, ATMs, kiosks, POS systems.
• Advantage: Intuitive and direct manipulation; no need for a separate device.
Trackpads/Touchpads
• Description: Flat surfaces that detect finger movement and gestures.
• Usage: Laptops and some keyboards.
• Advantage: Compact and gesture-enabled (pinch, swipe, scroll).
Optical and Laser Mice
• Description: Devices that track hand motion to control a cursor on the
screen.
• Usage: Desktops, gaming, graphic design.
• Advantage: High precision and wide range of models for different needs.
Trackballs
Description: A stationary device where users roll a ball to move the pointer.
• Usage: CAD applications, gaming, industrial control systems etc.
• Advantage: Space-efficient and ergonomic.
Game Controllers
• Description: A game controller translates physical actions (like pressing
buttons or moving sticks) into digital commands that control objects,
characters, or environments in a game.
• Usage: provide real-time, multidimensional, and immersive control over
game environments.
• Advantage: Instant feedback from buttons and analog inputs supports high
responsiveness.
Stylus and light pen
Stylus
• small pen-like pointer to draw directly on screen
• may use touch sensitive surface or magnetic detection
• used in PDA, tablets PCs and drawing tables

Light Pen
• now rarely used
• uses light from screen to detect location
 Digitizing tablet

• An input device that enables

you to enter drawings and
sketches into a computer.

• A digitizing tablet consists of

an electronic tablet and a
cursor or pen.
Eye Gaze Technology
• An eye gaze device is an assistive input technology that allows a user to control a
computer or digital interface using only eye movement. It tracks where the user is
looking on the screen and translates that into cursor movement, clicks, or other
actions—hands-free and touch-free.

• control interface by eye gaze direction

• e.g. look at a menu item to select it
• uses laser beam reflected off retina
• … a very low power laser!
• potential for hands-free control
• high accuracy requires headset
• cheaper and lower accuracy devices available
sit under the screen like a small webcam
Discrete positioning controls
• in phones, TV controls etc.
• cursor pads or mini-joysticks
• discrete left-right, up-down
• mainly for menu selection
Display Devices
bitmap screens (CRT & LCD)
large & situated displays
digital paper
Bitmap Displays
• A bitmap-base means that the display is made of a fixed number of dots or
• pixels in a rectangular grid. The color or intensity at each pixel is held by the
computer’s video card.
• The more bits per pixel, the more colors/intensities are possible.
Resolution and Colour depth
• Resolution … used (inconsistently) for
• number of pixels on screen (width x height)
• e.g. SVGA 1024 x 768, PDA perhaps 240x400
• density of pixels (in pixels or dots per inch - dpi)
• typically between 72 and 96 dpi
• Aspect ratio
• ration between width and height
• 4:3 for most screens, 16:9 for wide-screen TV
• Colour depth:
• how many different colours for each pixel?
• black/white or greys only
• 256 from a pallete
• 8 bits each for red/green/blue = millions of colours
Anti-aliasing

Jaggies
• diagonal lines that have discontinuities in due to horizontal raster scan process.

Anti-aliasing
• softens edges by using shades of line colour
• also used for text
Cathode Ray Tube
• Stream of electrons emitted from electron gun,
focused and directed by magnetic fields,
hit phosphor-coated
screen which glows

• used in old TVs and

computer monitors
Liquid Crystal Displays
• Smaller, lighter, and … no radiation problems.

• Found on PDAs, portables and notebooks,

… and increasingly on desktop and even for home TV

• also used in dedicated displays:

digital watches, mobile phones, HiFi controls

• How it works …
• Top plate transparent and polarised, bottom plate reflecting.
• Light passes through top plate and crystal and reflects to eye.
• Voltage applied to crystal changes polarisation and hence colour
• N.B. light reflected not emitted => less eye strain
LED
• An LED screen is a display device that uses Light Emitting Diodes (LEDs) to
produce images, text, and video content. These screens are commonly used
in TVs, computer monitors, digital billboards, and large video walls.
• Using tiny light-emitting diodes as individual pixels or as a backlight.
• Controlling the intensity and color of each LED to form images on the
screen.
large displays
• used for meetings, lectures, etc.
• 1. LED Walls (Direct-View LED Displays): Composed of thousands of tiny LEDs
as individual pixels (no backlight needed).
• 2. LCD Video Walls: Multiple LCD screens tiled together with ultra-narrow bezels.
• 3. OLED Displays: Organic light-emitting diodes (self-emissive pixels), used in
premium signage, digital art displays, and interactive screens.
• 4. Interactive Whiteboards (Smart Boards / Digital Panels): Large touchscreen
LCD/LED displays with multi-touch input.
• 5. Projection Systems (Laser/LED Projectors): Projectors that display images on
large flat surfaces or screens.
• 6. Transparent and Flexible Displays (Emerging Tech): OLED or Micro-LED
based used in Retail displays, museums, AR interfaces, smart glass.
Digital Paper
• Digital paper refers to electronic display technology designed
to mimic the appearance and readability of traditional paper appearance
while offering the benefits of digital functionality.
• It combines the look and feel of real paper with features like
note-taking, document viewing, storage, and synchronization. cross
section
Virtual Reality and 3D Interaction

positioning in 3D space
moving and grasping
seeing 3D (helmets and caves)
positioning in 3D space
• cockpit and virtual controls
• steering wheels, knobs and dials … just like real!
• the 3D mouse
• six-degrees of movement: x, y, z + roll, pitch, yaw

• data glove
• fibre optics used to detect finger position
• VR helmets
• detect head motion and possibly eye gaze
• whole body tracking
• accelerometers strapped to limbs or reflective dots and video processing
Positioning in 3D Space
6 Degrees of Freedom (6DoF)?
These refer to the ways an object can move in 3D space:
3 for Position (Translation):
1.X-axis – move left and right
2.Y-axis – move up and down
3.Z-axis – move forward and backward yaw
3 for Orientation (Rotation):
[Link] – tilt up or down (like nodding your head)
[Link] – turn left or right (like shaking your head “no”)
[Link] – rotate clockwise or counter-clockwise (like tilting
your head sideways)

roll
pitch
How a 3D Mouse Works
•You push, pull, twist, or tilt the 3D mouse controller (often a knob or puck).
•The device detects subtle forces and rotations to move or manipulate digital models in
real-time.
•Used mostly with CAD software, 3D modeling, VR interfaces, or drone control.
3D Displays
• 3D can be displayed on normal screens using shadows, depth etc.
• desktop VR
• ordinary screen, mouse or keyboard control
• perspective and motion give 3D effect
• seeing in 3D
• use stereoscopic vision
• VR helmets
• screen plus shuttered specs, etc.
VR headsets
• small TV screen for each eye
• slightly different angles
• 3D effect
VR motion sickness
• time delay
• move head … lag … display moves
• conflict: head movement vs. eyes
• depth perception
• headset gives different stereo distance
• but all focused in same plane
• conflict: eye angle vs. focus
• conflicting cues => sickness
Simulators and
VR caves
• scenes projected on walls
• realistic environment
• hydraulic rams!
• real controls
Physical controls, sensors etc.

special displays and gauges

sound, touch, feel, smell
physical controls
environmental and bio-sensing
Dedicated Displays
• analogue representations:
• dials, gauges, lights, etc.

• digital displays:
• small LCD screens, LED lights, etc.

• head-up displays
• found in aircraft cockpits
• show most important controls
… depending on context
Sounds
• beeps, bongs, clonks, whistles and whirrs

• used for error indications

• confirmation of actions e.g. keyclick

Touch, feel, smell
• touch and feeling important
• in games … vibration, force feedback
• in simulation … feel of surgical instruments
• called haptic devices

Sensory Modality Current Technologies Maturity

Haptic gloves, ultrasound,
Texture (Advanced)
electrotactile
Scent cartridges, wearable
Smell (Emerging)
emitters
Electro/thermal stimulation,
Taste (Experimental)
chemical droplet
BMW iDrive 9
• an advanced infotainment and vehicle control system that allows drivers to interact with and manage their
vehicle's functions through a centralized interface.
• Central Control Knob (Rotary Controller):
• Located on the center console
• Can be rotated, pushed, tilted, or pressed
• Some models have touchpad input on the controller for writing characters
• Touchscreen Display:
• On recent models, the screen supports touch gestures, pinch-to-zoom, and swipe
• Positioned for easy visibility and minimal eye movement
• Voice Control:
• “Hey BMW” command activates natural language interaction (especially in iDrive 7+)
• Allows control of navigation, climate, windows, etc.
• Steering Wheel Buttons:
• Quick access to media, calls, and driver assistance features
Physical Controls
• specialist controls needed …
• industrial controls, consumer products, etc.

easy-clean
smooth buttons

multi-function
control
large buttons
clear dials

tiny buttons
Environment and bio-sensing
• sensors all around us
• car courtesy light – small switch on door
• ultrasound detectors – security, washbasins
• RFID security tags in shops
• temperature, weight, location

• … and even our own bodies …

• iris scanners, body temperature,
• heart rate, galvanic skin response,
• blink rate
Paper: Printing and Scanning
print technology
fonts, page description, WYSIWYG
scanning, OCR
Printing
• image made from small dots
• allows any character set or graphic to be printed,
• critical features:
• resolution
• size and spacing of the dots
• measured in dots per inch (dpi)
• speed
• usually measured in pages per minute
• cost!!
Types of dot-based printers
• dot-matrix printers
• use inked ribbon (like a typewriter
• line of pins that can strike the ribbon, dotting the paper.
• typical resolution 80-120 dpi
• ink-jet and bubble-jet printers
• tiny blobs of ink sent from print head to paper
• typically 300 dpi or better .
• laser printer
• like photocopier: dots of electrostatic charge deposited on drum, which picks up toner (black powder form of ink)
rolled onto paper which is then fixed with heat
• typically 600 dpi or better.
Printing in the workplace
• shop tills
• dot matrix
• same print head used for several paper rolls
• may also print cheques

• Thermal printers
• special heat-sensitive paper
• paper heated by pins makes a dot
• poor quality, but simple & low maintenance
• used in some fax machines
Screen and Page
• WYSIWYG
• what you see is what you get
• aim of word processing, etc.
• but …
• screen: 72 dpi, landscape image
• print: 600+ dpi, portrait
• can try to make them similar
but never quite the same
• so … need different designs, graphics etc, for screen and print
Scanners
• Take paper and convert it into a bitmap

• Two sorts of scanner

• flat-bed: paper placed on a glass plate, whole page converted into bitmap
• hand-held: scanner passed over paper, digitising strip typically 3-4” wide

• Shines light at paper and note intensity of reflection

• colour or greyscale

• Typical resolutions from 600–2400 dpi

Scanners (ctd)
Used in

• desktop publishing for incorporating photographs and other images

• document storage and retrieval systems, doing away with paper storage

+ special scanners for slides and photographic negatives

Optical character recognition
• OCR converts bitmap back into text
• different fonts
• create problems for simple “template matching” algorithms
• more complex systems segment text, decompose it into lines and arcs, and decipher
characters that way
• page format
• columns, pictures, headers and footers
Paper-based interaction
• paper usually regarded as output only

• can be input too – OCR, scanning, etc.

• Xerox PaperWorks
• glyphs – small patterns of /\\//\\\
• used to identify forms etc.
• used with scanner and fax to control applications

• more recently
• papers micro printed - like wattermarks
• identify which sheet and where you are
• special ‘pen’ can read locations
• know where they are writing
 Memory

short term and long term

speed, capacity, compression
formats, access
Short-term Memory - RAM
• Random access memory (RAM)
• on silicon chips
• 100 nano-second access time
• usually volatile (lose information if power turned off)
• data transferred at around 100 Mbytes/sec

• Some non-volatile RAM used to store basic set-up information

• Typical desktop computers:

64 to 256 Mbytes RAM
Long-term Memory - disks
• magnetic disks
• floppy disks store around 1.4 Mbytes
• hard disks typically 40 Gbytes to 100s of Gbytes
access time ~10ms, transfer rate 100kbytes/s

• optical disks
• use lasers to read and sometimes write
• more robust that magnetic media
• CD-ROM
- same technology as home audio, ~ 600 Gbytes
• DVD - for AV applications, or very large files
Blurring boundaries
• PDAs
• often use RAM for their main memory

• Flash-Memory
• used in PDAs, cameras etc.
• silicon based but persistent
• plug-in USB devices for data transfer
speed and capacity
• what do the numbers mean?

• some sizes (all uncompressed) …

• this book, text only ~ 320,000 words, 2Mb
• the Bible ~ 4.5 Mbytes
• scanned page ~ 128 Mbytes
• (11x8 inches, 1200 dpi, 8bit greyscale)
• digital photo ~ 10 Mbytes
• (2–4 mega pixels, 24 bit colour)
• video ~ 10 Mbytes per second
• (512x512, 12 bit colour, 25 frames per sec)
virtual memory
• Problem:
• running lots of programs + each program large
• not enough RAM

• Solution - Virtual memory :

• store some programs temporarily on disk
• makes RAM appear bigger

• But … swopping
• program on disk needs to run again
• copied from disk to RAM
• slows t h i n g s d o w n
Compression
• reduce amount of storage required
• lossless
• recover exact text or image – e.g. GIF, ZIP
• look for commonalities:
• text: AAAAAAAAAABBBBBCCCCCCCC 10A5B8C
• video: compare successive frames and store change
• lossy
• recover something like original – e.g. JPEG, MP3
• exploit perception
• JPEG: lose rapid changes and some colour
• MP3: reduce accuracy of drowned out notes
Storage formats - text
• ASCII - 7-bit binary code for to each letter and character
• UTF-8 - 8-bit encoding of 16 bit character set
• RTF (rich text format)
- text plus formatting and layout information
• SGML (standardized generalised markup language)
- documents regarded as structured objects
• XML (extended markup language)
- simpler version of SGML for web applications
Storage formats - media
• Images:
• many storage formats :
(PostScript, GIFF, JPEG, TIFF, PICT, etc.)
• plus different compression techniques
(to reduce their storage requirements)

• Audio/Video
• again lots of formats :
(QuickTime, MPEG, WAV, etc.)
• compression even more important
• also ‘streaming’ formats for network delivery
methods of access
• large information store
• long time to search => use index
• what you index -> what you can access
• simple index needs exact match
• forgiving systems:
• Xerox “do what I mean” (DWIM)
• SOUNDEX – McCloud ~ MacCleod
• access without structure …
• free text indexing (all the words in a document)
• needs lots of space!!
processing and networks

finite speed (but also Moore’s law)

limits of interaction
networked computing
Finite processing speed
• Designers tend to assume fast processors, and make interfaces more and more
complicated

• But problems occur, because processing cannot keep up with all the tasks it needs to do
• cursor overshooting because system has buffered keypresses
• icon wars - user clicks on icon, nothing happens, clicks on another, then system responds and windows
fly everywhere

• Also problems if system is too fast - e.g. help screens may scroll through text much too
rapidly to be read
Moore’s law

• computers get faster and faster!

• 1965 …
• Gordon Moore, co-founder of Intel, noticed a pattern
• processor speed doubles every 18 months
• PC … 1987: 1.5 Mhz, 2002: 1.5 GHz
• similar pattern for memory
• but doubles every 12 months!!
• hard disk … 1991: 20Mbyte : 2002: 30 Gbyte
• baby born today
• record all sound and vision
• by 70 all life’s memories stored in a grain of dust!
The Myth of the Infinitely Fast Machine
• implicit assumption … no delays
an infinitely fast machine
• what is good design for real machines?
• good example … the telephone :
• type keys too fast
• hear tones as numbers sent down the line
• actually an accident of implementation
• emulate in deisgn
Limitations on Interactive Performance
Computation bound
• Computation takes ages, causing frustration for the user
Storage channel bound
• Bottleneck in transference of data from disk to memory
Graphics bound
• Common bottleneck: updating displays requires a lot of effort - sometimes helped by adding a
graphics co-processor optimised to take on the burden
Network capacity
• Many computers networked - shared resources and files, access to printers etc. - but interactive
performance can be reduced by slow network speed
Networked Computing
Networks allow access to …
• large memory and processing
• other people (groupware, email)
• shared resources – esp. the web

Issues
• network delays – slow feedback
• conflicts - many people update data
• unpredictability
Psychology and the Design of Interactive System
• Some direct applications
• e.g. blue acuity is poor
 blue should not be used for important detail

• However, correct application generally requires understanding of context in

psychology, and an understanding of particular experimental conditions