Visualization

1 Introduction
1.1 Basic information
Motivation.
ˆ Good visualization can convey large amounts of complex data into the human brain.

ˆ This is great to communicate results to others, but also to explore data interactively during
research.

ˆ Our brain/visual system can process some types of gures almost without eort. We should
exploit this.

ˆ In this lecture we learn about how to represent complex data visually such that our brains
can parse it eciently.

Outline of lecture. Two main parts, complementary to each other. To keep lecture enter-
taining, will alternate between the two blocks.

ˆ Part A: Visual data representation

 Scatter and line plots

 Images and functions in 2d
 Distributions
 High-dimensional data
 Graphs and networks
 Vector elds
 Meshes and 3d
 Other plot types

ˆ Part B: Background and discussions

 Data and processing

 Color
 Edward Tufte and minimalism
 The human visual system and perception

1
  Grammar of graphics
 Components of a gure
 Dynamic and interactive visualization
 How to lie with charts

Software toolbox.
ˆ Lecture based on python, numpy; and work mostly with matplotlib

ˆ other relevant plotting libraries are: seaborn, plotly, vega-altair

ˆ other core packages: pandas, imageio, scikit-learn, jupyter

ˆ also worth a look: R and ggplot2

Literature. The following references inuenced this lecture and might be useful.

Alberto Cairo. The Functional Art. New Riders, Berkeley, 2013.

Alberto Cairo. How charts lie. W. W. Norton & Company, 2019.

Andy Kirk. Data Visualisation. SAGE Publications Ltd, 2019.

Robert Spence. Information Visualization. Springer, 2014.

2
Edward Tufte. The Visual Display of Quantitative Information. Graphics Press, Cheshire, 1983.

Jake VanderPlas. Python Data Science Handbook. O'Reilly, 2022. URL [Link]
[Link]/PythonDataScienceHandbook/.
Hadley Wickham, Danielle Navarro, and Thomas Lin Pedersen. ggplot2: Elegant Graphics for
Data Analysis. URL [Link]

Leland Wilkinson. The grammar of graphics. Springer, 2005.

1.2 Some examples

1.2.1 Tables versus plots
ˆ taken from F. J. Anscombe: Graphs in Statistical Analysis, The American Statistician,
1973, 27, 1721

Data series represented as table:

x1 10.00 8.00 13.00 9.00 11.00 14.00 6.00 4.00 12.00 7.00 5.00
y1 8.04 6.95 7.58 8.81 8.33 9.96 7.24 4.26 10.84 4.82 5.68
y2 9.14 8.14 8.74 8.77 9.26 8.10 6.13 3.10 9.13 7.26 4.74
y3 7.46 6.77 12.74 7.11 7.81 8.84 6.08 5.39 8.15 6.42 5.73
x2 8.00 8.00 8.00 8.00 8.00 8.00 8.00 19.00 8.00 8.00 8.00
y4 6.58 5.76 7.71 8.84 8.47 7.04 5.25 12.50 5.56 7.91 6.89

ˆ hard to interpret as numbers in table, try basic statistical analysis

ˆ all xi and yi sequences have same mean and variance

ˆ sequences of pairs (x1 , y1 ), (x1 , y2 ), (x1 , y3 ) and (x2 , y4 ) all yield essentially the same linear
regression:

 same slope, intercept, correlation coecient, standard error for slope estimation
ˆ graphic representation immediately tells us four dierent stories

14 14 14 14
12 12 12 12
10 10 10 10
8 8 8 8
6 6 6 6
4 4 4 4
2 2 2 2
5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20

3
1.2.2 Historical examples
Chinese cartography
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ approx 1100 AD

European cartography
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ 1546 by Petrus Apianus, generalization to two-dimensional plots still took some time

4
Playfair
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ William Playfair: The commercial and political atlas, 1786

Napoleon's march to Moscow.

ˆ taken from [Spence: Information Visualization]

ˆ gure in public domain, available at [Link]

ˆ Charles Joseph Minard, 1869

5
Water pumps in London.
ˆ taken from [Spence: Information Visualization]

ˆ gure in public domain, available at

[Link]
ˆ John Snow, 1854

Regency chart
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ E. J. Marey: La méthode graphique, 1885

6
 ˆ Note: George II was the founder of Göttingen University, Wilhelmsplatz is named after
William IV.

Train timetable.
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ E. J. Marey: La méthode graphique, 1885

Galaxy distribution.
ˆ taken from [Tufte: The visual display of quantitative information]

ˆ example for computerized cartography, 1977

7
1.2.3 Example by Alberto Cairo: world population
ˆ taken from [Cairo: The Functional Art, Chapter 1]

ˆ Cairo read book `The Rational Optimist: How Prosperity Evolves' by Matt Ridley

ˆ chapter on world population made the hypothesis that it will soon stabilize

 rapid decrease in fertility in developing countries

 slight increase (back to `replacement rate' of 2.1 children per woman) in developed
countries

ˆ provided gure did not support the hypothesis, did not display appropriate data, dierent
simultaneous trends cannot be distinguished in summarized data

ˆ showing all individual trajectories not helpful either: all necessary data is shown, but hard
to process visually

ˆ nal version: highlight representatives from dierent clusters. supports hypothesis, allows
for further more detailed exploration (e.g. China, Brazil, Niger)

8
2 Data and processing
Literature. Sources for this section were:
ˆ [Kirk, 2019, Chapter 4]

ˆ Online course `R for data science', [Link]

Sections 5, 7, and 13.

ˆ Pandas documentation, `Getting started', [Link]

tarted/[Link]#getting-started

2.1 Data types and structures

Primitive data types Classication of data types due to psychologist Stanley Stevens, 1946.
No rigorous, universal system; some debate and more rened proposals exist. But the general
idea seems to be consensus. No need to be dogmatic about this. Just be aware of some conceptual
properties of data.

ˆ textual: unstructured passages of text

 responses to `Any other comments?' in a questionaire

 abstract for academic research article
 product description in an online shop most dicult to handle in automated / quan-
titative analysis → natural language processing
 in programming: string types (not primitive in that sense, but highly complex sub-
structure)

ˆ nominal: from a xed set of categories

 gender of survey participant

 meals available on restaurant menu
 city, country of birth
 edges in a graph (described by indies of two vertices)
 in programming: enum types, or int types but we never use ordered comparison or
addition, only use equality test
 can have a hierarchical structure: cities can be grouped into countries
ˆ ordinal: categories can be ordered, but no notion of `distance' or `dierence' between
categories.

 options on a survey to which extent you agree with a statement

 `non-quantitative' size of clothing: XXS to XXL
 rank of police ocer
 rank in a competition (but not the nishing time or score)
 in programming: int or char types, but do not use dierences, only equality and order
comparison

9
 ˆ interval: there is a notion of dierence, but not of ratio

 temperature in degrees Celsius (20◦ is not twice as hot as 10◦ )

 dates (but ratios between dierences can be meaningful)
 in programming: oating point types
ˆ ratio:

 most physical measurements: length, duration, mass, temperature in Kelvin

 in programming: oating point types

Composite data structures. Primitive types can be combined into more complex data struc-
tures.

ˆ lists / tuples: ordered list of entries. Entries may be dierent data types, primitive or
nested lists. In principle can build arbitrarily deep and rich structures.

Simplify things by imposing more structure:

ˆ array: one- or multi-dimensional Cartesian grid of values, usually of a common primitive

type

ˆ struct or composite types: dened by set of elds (or variables). Each eld has a specied
data type (can be primitive or another struct). An instance of a struct must provide a
value for each eld (maybe have a convention for missing values).

ˆ table / `data frame': a list of instances of the same struct; elds/variables are usually
referred to as columns; dierent entries as rows

Composite data often represents some of the following structures:

ˆ images: usually represented as rectangular two-/three-dimensional array of pixel intensities

ˆ time series: measurement of some quantity over time, e.g. temperature evolution through-
out the day. If time interval is regular, it can be represented as one-dimensional array of
values. Otherwise could be list of pairs of time stamp and value (e.g. times when a car
passed on the road + type of car).

ˆ functions: abstract denition of a function is list of associated pairs of input and output
values. Computationally and in experiment the concrete representation can vary extremely.

 store values at xed locations such as grid points (e.g. image, xed-interval time series)
 store pairs of location and values where measurements are available (e.g. weather
measurement stations)
 ⇒ visualizing functions poses several challenges, high dimensionality, irregular avail-
ability of values, representation of uncertainty

ˆ graphs / networks: usually consists of two parts:

1. list of vertices or nodes, with properties such as label, type, weights, . . .

2. list of edges between nodes, with properties such as orientation, type, length, . . .

10
2.2 Data processing / pipeline
ˆ acquisition

 source depends strongly on context (internet, company records, scientic measure-

ments obtained by ourselves or collaborators)

ˆ examination

 is meant here at very fundamental level: what even is my data? which les? what
types, format, relation? amount and range? Can be very relevant `in the wild'
 quality and representativeness? missing values, errors, formatting inconsistencies,
corruption, duplicates, out of date

ˆ cleaning / pre-processing

 x formatting errors, typos, add missing values

 apply lters
 compute derived data (averages, reductions, aggregations, clustering, binning,. . . ),
but often it is too early to know what will be relevant

personal recommendations:

 keep all iterations/versions of the data (if possible/practical), ideally in a format where
changes can be traced transparently (via di or with dedicated analysis script)
 add meta data:
* document source of external data
* for generated / processed data: document used parameters, conventions;
examples: encoded directly in lename; added as comment at beginning
* use meaningful directory structures
 automate and standardize cleaning and pre-processing as much as possible via scripts,
save these scripts with the data for documentation

ˆ data exploration

 more detailed and quantitative than examination: what stories does the data tell?
 is it compatible with hypotheses? what are even good hypotheses? are there statisti-
cally signicant dependencies?
 usually main challenge: nd clear low-dimensional structure encoded in vast high-
dimensional dataset
 `Exploratory data analysis is an attitude, a exibility, and a reliance on display, not
a bundle of techniques, and should be so taught.' (John Tukey: We Need Both
Exploratory and Conrmatory, The American Statistician, 1980)
 statistics, visualization, quick exible movement play a key role, will be demonstrated
throughout lecture

ˆ main step: actual work on data, extracting information required for visualization

ˆ backup, archiving, publishing

11
  use version management, repositories, redundant storage
 when project is done, make sure everything is stored and documented such that you
can later retrace your steps and reproduce all ndings
 in spirit of reproducible research data is also often published along with the paper

2.3 Basic data transformations

This section is not meant to be an exhaustive tutorial, or even a pandas-specic tutorial. It
intends to illustrate a few basic common operations on tabular data. This section is given as a
Python example.

12
3 Color
Additional literature for this section.
E. R. Kandel, J. H. Schwartz, T. M. Jessell, S. A. Siegelbaum, and A. J. Hudspeth. Principles
of Neural Science. McGrawHill, New York, 5th edition, 2013.

3.1 Color perception

YoungHelmholtz theory / trichromacy.
ˆ [Link]

ˆ developed in 19th century

ˆ Young postulated in 1802 the existence of three types of photoreceptors,

 [Link]
 British polymath, obtained medical doctor degree from Uni Göttingen in 1796
 also known for advocating the wave theory of light, introducing the early form of the
double slit experiment

ˆ Helmholtz in 1850: violet, green, red. Their relative signal strength is perceived as color.
Can increase intensity without changing hue.

 [Link]
 also known for: Helmholtz decomposition, equation, . . .

[Link]
ˆ some historical context:

 Maxwell's equations were published in 1850s to 1870s

 experimental validation of dierent wavelength sensitivity in retinal cells of sh in
1956, in humans in 1983

ˆ tetrachromacy: some animals have four types of cone cells, some can see ultraviolet. ex-
ample: goldsh

13
Opponent process theory.
ˆ white vs black, green vs red, yellow vs blue
color perception based by position on these three axes, `there is no reddish green, or bluish
yellow'

[Link]
ˆ proposed by Ewald Hering in 1892, later formalized by Leo Hurvich and Dorothea Jameson

ˆ present in Goethe's discussion of color, 1810 (Goethe made accurate observations with
prisms and the like, but his interpretation was more based on aesthetics than scientic
rigor)

ˆ there is a rough equivalence of the (w-bk), (r-g), (y-b) channels in the earliest visual
processing in the retina, e.g. [Kandel et al., 2013, p. 594]

[Link]
ˆ so not in contradiction of trichromacy, latter is essentially the rst processing step of the
former

3.2 Color models

RGB.
ˆ Additive model, three channels: red, green, blue. Each has intensity between 0 and 1. Can
denote a color as point in [0, 1]3 .

 (0, 0, 0) is black, (1, 1, 1) white, (s, s, s) is grey for s in [0, 1].

 (1, 0, 0), (0, 1, 0) and (0, 0, 1) are `pure' red, green, blue.
 (1, 1, 0): yellow, (0, 1, 1): cyan, (1, 0, 1): magenta; secondary colors
ˆ used mostly for describing colors on screens, basic image storage, websites,. . .

 [0, 1] is usually discretized into 256 discrete values (=8bit), given as natural numbers
between 0 and 255, or as hex code: 00 to FF

14
  color specication in HTML: #RRGGBB
ˆ python example:

 create simple images of color gradients

 load image, extract and visualize color distribution (2d/3d pointcloud)
ˆ application examples:

 use two red and green channel to visualize two densities

 color histogram matching

ˆ RGB is a relative color space:

 depends on what red,green and blue are used for actual mixing. so in principle is
device dependent.
 distances in the `RGB-cube' are not necessarily accurate reections of perceived sim-
ilarity by humans

Absolute color spaces, example: CIE color spaces.

ˆ specied by the International Commission on Illumination, (Commission internationale de
l'éclairage)

ˆ attempt to nd objective descriptions of color, based on `standard observers' and `standard
illuminants'

ˆ specication is copyrighted, precise standards and documentation has to be purchased from

the CIE

ˆ CIELAB: three dimensional, axes are L (lightness), a (red-green), b (yellow-blue), distances

in these coordinates are supposed to be similar to perceived color dierences

ˆ can be transformed to RGB if an identication prole is xed

ˆ similars concepts: Adobe RGB, sRGB

HSL and HSV.

ˆ RGB is intuitive at level of additive mixing of signal to three cone types

ˆ at level of perception other parametrization seems more appropriate:

 hue and saturation describe color in `(blue-yellow)-(red-green)-plane'

15
  lightness / value: interpolation between black,color, and white

[Link]
ˆ transformation between RGB and HSL/HSV values

 note: so HSL/HSV are merely alternative parametrizations for RGB color space, so
also still a `relative' color space
 is a parametrization in which certain `natural' operations are easier to express
ˆ python example:

 construct a hue circle

 simple saturation lter

CMYK.
ˆ subtractive color model. base colors: cyan, magenta, yellow, black (k stands for key, usually
the printing plate with most structural detail)

ˆ color mixing laws from kindergarden

ˆ most commonly used in print

[Link]
ˆ CMYK is `overparametrization': CMY channels would be enough. K is used to save
expensive color ink, and to get deeper black

ˆ again device dependent, conversion to and from RGB requires some registration prole

3.3 Color in visualization

(next paragraphs roughly following [Kirk, 2019, Chapter 9], examples from there unless men-
tioned otherwise)

16
Preliminaries.
ˆ in principle: space of perceivable colors is three-dimensional, could encode three dimensions
via color

ˆ but: color shading and saturation are at the bottom of the ClevelandMcGill scale (more
info later), so only qualitative presentation of data, not precise and quantitative

ˆ also: `unpacking' three dimensions from color within a 2d plot can be challenging for
viewers, due to our diculties of conceptualizing more than three dimensions

Highlighting.
ˆ use color for pop out eect

Nominal data.
ˆ need clearly distinct colors, ideally without obvious order

ˆ simple example: dierent hues for same saturation and value in HSV

ˆ usually the default colors in plotting software bad example:

17
 ˆ how many categories can we visually distinguish? Kirk: at most 12 categories via hue
(p259). General advice: as few as possible, try to have at most 6.

more information can be shown better via annotation, multiple additional plots (e.g. fo-
cussing on sub-groups), interactive visualization

ˆ creative example: relative space usage in Vienna

Ordinal data.
ˆ colors should be ordered in an intuitive way. bad example:

18
Interval or ratio data.
ˆ use color scales / color maps

ˆ signed vs unsigned data ([-1,1] vs [0,1])

ˆ warning: rainbow scale may be very confusing because opposite ends look very similar
(could be useful for cyclic data):

19
3.4 Color maps
Python example.
ˆ value variation for xed hue in HSV
ˆ two opposing hues for signed data
ˆ saturation variations
ˆ summary: in principle can easily create color maps, but they are not perceptually uniform,
do not make the most of our color perception

Example: more general color maps from matplotlib.

ˆ [Link]
ˆ categories: sequential, diverging, cyclic, qualitative
ˆ some background: [Link]
tml

Perceptual uniformity.
ˆ many standard colormaps in various software is not perceptually uniform, can cause sub-
stantial perceptual `gaps'
ˆ this eect can easily be visualized with a `comb pattern' (see python examples)
ˆ matplotlib contains a few perceptually uniform ones, e.g. viridis or plasma
ˆ good resource for more perceptually uniform color maps: [Link]
rg/

Some ways to improve perceptual resolution.

ˆ draw contour lines
ˆ use a high-frequency cyclic scale

20
Normalization / truncation / transformations.

Multidimensional data.
ˆ maps and vector elds: 2d to 2d

S : (x, y) 7→ (f (x, y), g(x, y))

ˆ in polar coordinates:    
f (x, y) cos(φ(x, y))
= r(x, y) ·
g(x, y) sin(φ(x, y))

ˆ can in principle be represented by color function. For example: use φ as hue (scale [0, 2π]
to [0, 1]), use scaled r as value (need to truncate at some maximum value)

ˆ example: unbalanced W1 eld

ˆ comment: can only convey a qualitative picture

3.5 Color deciency

ˆ example palette from [Kirk]:

21
ˆ matplotlib viridis seems to be fairly robust

ˆ simulation of eects at [Link]

-simulator/
ˆ also in Firefox, developer tools (F12), accessibility

22
4 Human visual system and cognition
Literature for this section.
Christopher Healey. Perception in visualization. URL [Link]
healey/PP/.
Christopher Healey and James Enns. Attention and visual memory in visualization and computer
graphics. IEEE Transactions on Visualization and Computer Graphics, 18(7):11701188, 2012.
doi:10.1109/TVCG.2011.127.

Jeremy M. Wolfe, Melissa L.-H. Võ, Karla K. Evans, and Michelle R. Greene. Visual search
in scenes involves selective and nonselective pathways. Trends in Cognitive Sciences, 15(2):
7784, 2011. doi:10.1016/[Link].2010.12.001.
See also [Cairo, 2013, Part II], [Spence, 2014, Chapter 4], [Kandel et al., 2013, Chapters 25-28].

An unhelpful analogy [Cairo, 2013, Part II].

ˆ An unsatisfying description of the eye and brain: What is the meaning of the small picture
drawn in the brain? How do we process and interpret an image in the brain?

ˆ A digital camera as metaphor for the visual system. Eye and retina are lens and ccd chip,
nerves are the wires, brain are processor and hard drive. But the retina is a very peculiar
ccd chip, how is the signal processed in the brain?

ˆ Seeing, perceiving, knowing are distinct from each other.

4.1 The human eye

Overview of the eye.

23
 [Link]

ˆ pupil with lens at the front

ˆ retina with photoreceptors at the back

Photoreceptors.
ˆ rods

 approx 100mio, very sensitive (can respond to a single photon),

 signal is pooled over multiple rod cells, signal is collected over longer time interval ⇒
better sensitivity, less spatial and temporal resolution
 dominant `in the dark'
ˆ cones

 7mio, three types with peak sensitivity in at dierent wavelengths: long, medium,
short ⇒ color detection
 small part of electromagnetic spectrum can be perceived by the eye

[Link]

24
 ˆ distribution of cones and rods

 rods relatively evenly distributed, density gradually decreasing towards the periphery,
two notable gaps.
 fovea: contains highest density of cones, responsible for sharp color vision
 but due to higher sensitivity of rods: at night fovea is not so helpful. astronomers
know: look slightly past a star to see it
 blind spot: optimal nerve leaves retina, neigher cones nor rods

[Link]

Structure of retina and rst optical processing.

ˆ see [Kandel et al., 2013, Chapter 26]

ˆ three layers: photoreceptors and two layers of early processing neurons.

ˆ black layer of pigments at the back to avoid re-scattering of light into eye

ˆ interesting: Receptors are behind the two processing layers. Whether this is on purpose or
by accident is not fully known. In fovea the front layers are pushed aside.

ˆ ⇒ blind spot is necessary consequence of layer ordering: optic nerve must pass through
retina to brain

ˆ approx. 1 million axons in optical nerve, i.e. approx. 1% of number of photoreceptors, ⇒

strong compression and processing must already happen in retina

ˆ ganglion cells (third layer):

 ON and OFF type for faster detection of decrease in intensity

 have center and outer sensitive region, re most rapidly if stimuli are dierent ⇒
simple edge detectors

25
  transient cells for better detection of temporal variations: higher ring rate on increase
of signal, then reduce again (act like a temporal edge detector)

ˆ only 10% of cortical neurons driven by color contrast rather than luminance contrast

[Kandel et al., 2013, Figure 26-10], response of ganglion cells

Saccades and xations.

ˆ As noted above: detailed color vision is only possible with the fovea. Peripheral vision has
much less resolution.

ˆ A simple experiment: close your eyes for a while, open them again and keep them focused
on xed object. While keeping the eyes xed, it is extremely dicult to properly see objects
beyond the `central focus of vision', and to determine their color.

ˆ angular range of vision:

 peripheral vision ∼ 180◦

 fovea ∼ 2◦ (approximately thumbnail arm's length)
 parafovea ∼ 10◦
ˆ So how is the illusion of a `steady wide-view' image in the brain generated?

ˆ Solution: focus of eye moves quickly through scene in front of us:

 a sequence of xations (lasting typially around 200ms) and quick saccades in between
( 20ms) where the eye moves
 location of next saccade mostly chosen unconciously / automatically, based on input
from peripheral vision (bottom-up) and task/context (top-down)
 evolutionary trait, important for survival
 moving and uncommon objects attract our attention

26
  simple example of a deduced design principle for animations: do not start an animation
and simultaneously introduce new text box

ˆ eye tracking experiments by Alfred Lukyanovich Yarbus, Russian psychologist, 1960s

[Link]
[Link]

4.2 Preattentive processing

Seminal experiments.
ˆ Treisman and Gormican, 1990: show a simple image only for 50ms, viewer can still extract
the `gist'

ˆ Kundel and Nodine, 1975: trained radiologists were shown lung X-rays for 200ms. Could
detect anomalies in 70% of cases (97% under unlimited viewing).

ˆ Conclusion: we extract a lot of information from an image in the rst 200ms, before we
can even consciously think about the image.

ˆ This is called preattentive processing. The name refers to processing that happens approxi-
mately within rst 200ms after stimulation, on large multi-element displays. Eye movement
takes about 200ms, so this processing happens before eyes can focus on something, and
must therefore happen in parallel in the low-level visual system.

ˆ By now we know: preattentive processing is in fact not independent of attention (bottom-

up), since the given task of the individual may inuence the exact evaluation of the images
(top-down).

Anne Treisman (1935-2018).

ˆ English cognitive psychologist, taught in Oxford, Berkeley, Princeton. Important contri-
butions to study of visual perception. Devised various experiments and developed `feature
integration theory' for their interpretation.

ˆ Motivated by `edge detection' neurons in the brains of cats (cf. the famous experiment by
David H. Hubel and Torsten Wiesel in 1959, 1981 Nobel prize in medicine)

27
 ˆ Treisman designed experiments with two dierent performance measures:

 Time experiment: Subject must complete a task as fast as possible, while being
accurate. THen increase number of distractors. If solving time is approximately
constant, then task is preattentive. Otherwise, serial search is required and it takes
longer with increasing number of distractors.
 Accuracy experiment: A xed short time (250ms) to solve the task is given. The eye
cannot move in this time. Measure accuracy vs complexity. If accuracy remains high
with increasing complexity, task is preattentive.

Pop-out eect: quickly identifying unique visual properties.

ˆ example: red circle among blue circles (feature: hue)

ˆ example: red circle among red squares (feature: shape)

ˆ identication happens automatically and without exhausting the viewer

pop-out with hue and shape [Healey]

Features for which preattentive processing works (not exhaustive).

ˆ orientation

ˆ length

ˆ closure

ˆ size

ˆ curvature

ˆ density

ˆ number

ˆ hue

ˆ luminance

ˆ intersections

ˆ terminators

ˆ 3d depth

28
 features in preattentive processing [Healey]

Tasks solved by preattentive processing.

ˆ target detection: detect the presence or absence of a `target' element with a unique visual
feature within a eld of distractor elements

ˆ boundary detection: detect a boundary between two groups of elements, where all of the
elements in each group have a common visual property

ˆ region tracking: track one or more elements with a unique visual feature as they move in
time and space

ˆ counting and estimating number of elements with a unique visual feature

Conjunction search.
ˆ nd target with several specic features (e.g. hue and shape) that are not unique. typically
much harder, does usually not happen preattentively.

ˆ exceptions possible in `extreme' cases

29
 conjunction search is much harder [Wolfe]

Feature hierarchy.
ˆ `hue trumps shape' (caveat: color deciency): pop-out eect of some feature may over-
shadow `weaker features'

feature hierarchy between hue and shape [Healey]

`color beats shape' from [Kirk: Data Visualisation]

Ensemble coding.
ˆ humans are good at getting overall impressions at rst glance. examples:

 average size of points

30
  `average emotion' in group of faces
 `setting' of a scene (e.g. kitchen)
ˆ the overall impression might help to perform target search faster, e.g. bread in kitchen.
this suggests that there are several parallel processing pathways in the brain.

Gestalt grouping principles.

ˆ Gestalt psychology is psychological school of thought, emerging in early 20th century Ger-
many and Austria, among other things concerned with principles of perception

ˆ does not hold up to modern scientic standards, unsatisfying in terms of explanations

of phenomena, but did collect a list of observed principles that seem to underlie visual
perception

ˆ famous example: grouping laws

 proximity

 similarity

 connectedness

31
  continuity

 closure

 symmetry

[Link]

4.3 Theoretical models for preattentive processing

Feature integration theory (Treisman).
ˆ simple model: feature maps

ˆ each feature has its own map: where in the image does this feature occur

ˆ combination search needs attentive look-up of master map

ˆ of course: reality not as simple; perception is not as binary; sometimes even conjunction
search can be preattentive

32
Other theoretical models.
ˆ texton theory: things that look distinct in isolation can look almost identical in crowded
arrangements

low vs high NN similarity

ˆ similarity theory: TN vs NN similarity; not purely based on presence of preattentive fea-

tures, conjunction is possible to some extent

ˆ boolean map theory: to some extent we are able to intersect features

4.4 Visual perception and memory

Guided search.
ˆ perception is not just bottom-up; there must be some top-down component

ˆ feature maps: bottom-up; search query: top-down; queries are in format/features provided
by the visual cortex;

ˆ activation map: combination of bottom-up, top-down activity; look at regions of most

activation; this would explain TN-NN results

ˆ also explains why pure bottom-up attempt to predict location of next saccade failed
(saliency theory: Itti and Koch, `Computational Modeling of Visual Attention', Nature
Rev.: Neuroscience, 2001)

ˆ ⇒ preattentive is not entirely independent of attention (recall bread and kitchen example)

Postattentive search.
ˆ Does looking at scene for longer time quickly improve our understanding?

ˆ This is true when we can link objects in the image to familiar representations from long-
term memory (LTM). LTM can be queried almost instantaneously.

ˆ But querying short-term memory is slow (≈ 50ms). So if objects in image cannot be

detected preattentively or recognized semantically via LTM, then allowing some extra time
to look at image in advance does not speed up subsequent target search.

33
 ˆ This is conrmed by a simple experiment: target search for arbitrary conjunction of features
(e.g. `green vertical') among several objects. iIn this case, showing the image for 300ms prior
to showing the target description did not speed up the search.

ˆ But again: reality more complicated. Sometimes there is a positive eect (keyword `con-
textual cuing').

ˆ Our takeaway: if a visualization looks unfamiliar and is not accessible via preattentive
processing, then it will take a long time and mental eort to parse and understand.

Change blindness.
ˆ the photography metaphor from above is inaccurate: it is not true that an increasingly
accurate and faithful model of the scene is built in brain over time as the image is gradually
processed in greater detail. only short-lived models specically guided by the current vision
task are created.

ˆ good experiment: change blindness. if two similar images are shown next to each other (or
separated by a short blink) then we may sometimes not spot the dierence, even when it
is not particularly small or subtle

ˆ obviously our visual system detects the change in the pictures, but our attention is not
drawn to it

ˆ some theories:

 overwriting. new image overwrites old image, everything that is not saved via ab-
straction is lost. but: we would immediately see change if no blink in between (have
time dierence lters in visual cortex, these are tricked by the blink)
 rst impression. abstraction: we do not encode scene as accurate pixel rasterized
image, but more like a naive, abtract vector graphic (`elderly couple in front of sphinx
statue, trees in background'). if change in image does not require change in this
encoding, it is not noticed. example: experiment with movie actor changed mid-lm.
 everything stored, but not compared: illustrated by experiment. subject is asked
for directions by person with basketball. students show up, cause some distraction,
basketball disappears. only few individuals notice that it is suddenly missing, but
more half of the others remember the ball afterwards.

Inattentional blindness.
ˆ if our attention is drawn onto a specic task, other things may be missed, even if they are
obvious

ˆ simple example: individuals had to examine a cross on slide and see which arm was longer.
after two or three trials, a small `critical object' was added to the image. 25% of subjects
missed this; but essentially 100% noticed the object when asked to look out for it

ˆ extreme example: gorilla in ball passing video (available on the internet)

34
Attentional blink.
ˆ subject looks at sequence of images, approx 100ms per image, need to judge if target was
present or not, by pressing some button

ˆ when pre-attentive processing has identied relevant candidate (and thus `activates' con-
scious processing), the pre-attentive processing is disabled for a short interval

ˆ thus a target present shortly after another one is usually missed

4.5 Derived principles and mechanisms for visualization

Becoming aware of our limitations.
ˆ no general purpose vision exists, i.e. one which fully extracts and holds all information in
an image, at least after some time of observation

ˆ our vision seems to provide some specic fast low-level info extraction mechanisms, that
can then be used to solve a specic set of high-level tasks, the latter probably selected by
evolution

Reduce number of saccades required for understanding image.

ˆ recall: look-up in short term memory is slow; saccades take time

ˆ use direct labelling

ˆ use systematic coloring (more on this later)

Preattentive processing.
ˆ obvious application in data-feature mapping, i.e. when mapping data attributes to visual
features;

ˆ if used properly, we can automatically detect distinct trends in dataset, identify `outliers'

ˆ can also be used for highlighting and drawing attention

ˆ be careful to not produce interference between features! recall feature hierarchies.

ˆ also keep in mind similarity theory: aim for good TN-NN-ratios

Familiarity improves understanding.

ˆ if a visualization looks familiar, we can very quickly get a good orientation (see ensemble
coding)

ˆ so re-use visualization techniques that the reader already knows, e.g. small multiple, smooth
animations

35
Attention management.
ˆ make sure that viewers know what to focus on, they may miss important things when
focussing on wrong parts of image

ˆ attention is limited in space and time, do not ask for too much at once

Visual encoding of numbers.

ˆ William S. Cleveland and Robert McGill: `Graphical Perception: Theory, Experimentation,
and Application to the Development of Graphical Methods', Journal of the American
Statistical Association, Vol. 79, No. 387 (1984).

ˆ How accurate are various visual encodings of numbers?

ˆ Example: scale vs disk vs color

ˆ Example: obesity vs BA degrees

Abstraction.
ˆ since internal representation of gure in brain seems to drop many `unnecessary details'
(see change blindness), strong abstraction must happen in visual understanding of scene

ˆ simplify work by using abstracted images.

36
5 Graph visualization
5.1 Graph layout problem
ˆ Some of the previous embedding methods can be seen as graph layout algorithms.

ˆ The graph layout problem is as follows: Given an abstract graph without vertex positions,
nd good vertex positions for visualization of the graph.

ˆ A related problem is manifold learning: In that case vertex positions are given, but lie in
a high-dimensional space. Then we need to nd a low-dimensional embedding.

ˆ Layout algorithms usually work by minimizing some form of loss / objective function. The
best loss will depend on the context. Typical criteria are:

 Vertices should not collide, but be evenly distributed.

 Edges should be kept short, or near some prescribed length (e.g. to preserve pairwise
distances).
 Edge crossings should be avoided.
 Special graph structures (hierarchical structure, bi-partite structure, symmetries, . . . )
should be well reected in the embedding.

Overview on methods. Above we have seen some examples:

ˆ PCA, MDS, Diusion maps: Spectral methods that try to preserve global pairwise dis-
tances, or local neighbourhood.

ˆ UMAP: Tries to preserve local neighbourhood structure and density of samples.

A third common type of methods:

ˆ Force based embeddings: Vertices are modeled as `repelling charges' that want to be far
form each other, edges act like springs that draw vertices together (or that have a preferred
length). Then the `energy' of the embedding is minimized with respect to the positions.

Common aspects and dierences:

ˆ All methods involve parameters that must be chosen reasonably.

ˆ A graph can have multiple intrinsic length scales. It may look like a one-dimensional
manifold at one scale, but 2-dimensional at a smaller scale. At the microscopic scale it will
just look like individual points. Parameter choices will aect which version we see.

ˆ Spectral methods give globally optimal results (PCA, MDS, Diusion maps), but they are
restricted to particular cases (e.g. linear projection of Euclidean distances in PCA). Much
more general losses and embedding classes to minimize over can be conceived, but then
one can typically no longer guarantee global optimality.

37
5.2 Decorated graphs
Graphs considered so far were large but relatively simple in the following sense:
ˆ Vertices were simple points, sometimes with a simple class label. No complex properties.
ˆ Likewise, edges merely represented similarity or proximity between vertices.
Drawing them with simple markers (maybe decorated by color and size) and lines was sucient.
Of course there are more complex graphs where vertices and edges have more structure and
properties:
ˆ Vertices have an `identity': may represent persons, agents, classes, proteins, electronic
devices, . . .
ˆ Edges describe nuanced interactions: administrative relation (supervisor, . . . ), class rela-
tions (derivation, aggregation, . . . ), biochemical pathways, . . .
These additional attributes can be encoded visually in various ways.
ˆ Vertices become boxes, with labels and text, dierent shapes, . . .
ˆ Edges have dierent line styles, arrow tips, text, . . .

5.3 Examples
Some examples were given in the Python notebooks. Here are more examples from dierent
applications.

Software.
ˆ Graph layout documentation in Mathematica
[Link]
ˆ Manifold learning implemented by scikit-learn
[Link]

Analysis of machine learning models.

ˆ Visualization of network structure
[Link]
ˆ Visualzation of a learned Word2Vec embedding [Link]

Biochemistry.
ˆ Biochemical pathways
[Link]
show
ˆ Cytoscape
[Link] [Link]
ˆ Waddington ot
[Link]

38
Meshes, diusion, and learning.
ˆ Intrinsic meshes and their properties
Sharp, Gillespie, Crane: Geometry Processing with Intrinsic Triangulations, ACM SIG-
GRAPH 2021 courses, 2021

ˆ Deep learning for meshes and point clouds via the Laplacian
Sharp, Attaiki, Crane, Ovsjanikov: DiusionNet: Discretization Agnostic Learning on
Surfaces, ACM Trans. Graph., 2022

Social networks.

UML.

Graphical models.
ˆ Pictorial representation of structured high-dimensional probability distributions. Graphs
encode dependency information.

Feynman diagram.
ˆ Pictorial representation of interacting elementary particles

Other graph-type plots.

ˆ Sunburst, Gnome Disk Usage Analyzer and [Link]
t-component
ˆ Sankey diagram, [Link]

ˆ Collection of many nice examples: [Link]

lization-introduction

39
6 Edward Tufte and minimalism
6.1 Introduction
Edward Tufte (1942-).
ˆ Professor emeritus for political science, statistics and computer science at Yale University.

ˆ First inuential book data visualization: The Visual Display of Quantitative Information,
1983, Tufte [1983]

ˆ Promoted a philosophy of minimalist design in information graphics, apparently driven by

a trend that graphics were only perceived as means to dumb down information or to make
statistical data less boring, assuming the audience would be stupid or not interested.

ˆ Uses very polemic language in his books, seems to enjoy deconstruction of bad examples,
likes to formulate lists principles.

ˆ The following section is based on Tufte [1983], examples taken from there, and other
material inspired by this book.

Graphical excellence according to Tufte. Excellence in statistical graphics consists of

complex ideas communicated with clarity, precision, and eciency. Graphical display should

ˆ show the data

ˆ induce the viewer to think about the substance rather than about methodology, graphic
design, the technology of graphic production, or something else

ˆ avoid distorting what the data have to say

ˆ present many numbers in a small space

ˆ make large data sets coherent

ˆ encourage the eye to compare dierent pieces of data

ˆ reveal the data at several levels of detail, from broad overview to the ne structure

ˆ serve a reasonably clear purpose: description, exploration, tabulation, or decoration

ˆ be closely integrated with the statistical and verbal descriptions of the data set.

Graphical excellence is nearly always multivariate.

6.2 Data-ink
Source for this section: [Tufte, 1983, Chapter 4].

40
Example: Evolution of charts by Playfair.

ˆ rst example: 1785, early pages of `The Commercial and Political Atlas'
ˆ second example: created on year later, already much more mature, removed much of the
`background'
ˆ Tufte formulates a fundamental principle: Above all else show the data.

Denition.
ˆ data-ink is the ink in a graphic that represents data / information
 non-data-ink: frames, grids, (unnecessary) ticks, decoration
 data-ink: data points, (necessary) labels, derived data (e.g. marginal distributions,
indication of minimal or maximal values)
data-ink
ˆ data-ink ratio =
total ink

Example: electroencephalogram.

ˆ extreme example: almost exclusively data-ink, but can only be read by specialists

41
Example: sour taste.
ˆ original image taken from: Kuznicki and McCutcheon: Cross-enhancement of the sour
taste on single human taste papillae. Journal of Experimental Psychology: General, 108(1),
6889, 1979. (study eect of sucrose on the perceived intensity of sour taste)

ˆ Tufte's introduction in book: `[The display] compares each long bar with the adjacent short
bar to show the viewer that, under the various experimental conditions, the long bar is
longer.'

ˆ Tufte removes: frames, some ticks and labels, one side of each bar, stars (marking the
longer bars), text decoration (underline)

ˆ lines connecting adjacent bars are kept (data, since they show which experiments belong
together)

ˆ extreme example. My humble opinion: should be seen as illustration of principle rather

than as concrete suggestion

Example: simplication of box plot.

ˆ Tufte: conventional box plot is highly redundant. Suggests minimalistic version, argues
via number of placings of straightedge

ˆ my humble opinion: oversimplied

 (information about) data is there; but weight of ink does not align with weight of data
(usually higher density within quartiles, otherwise not proper plotting device anyway)
 Tufte frequently makes data density calculations: how much numbers are encoded in
a gure and equates this with amount of numbers that are transferred into viewers
brain

42
  but the visual system/brain do not extract a list of numbers from a graphic (at least
not at "rst glance"), but coarse structures and trends
 coarse structure more accurately visually reected by original design

Example: periodic system.

ˆ original image created by science illustrator Roger Hayward for chemistry textbook by
Linus Pauling, 1947 (introduced covalent bond in chemistry, two Nobel prices, chemistry
and peace)

ˆ remove grid, try removing guidelines (but rather not), add individual labels

Conclusion.
ˆ According to Tufte good gures have a high data-ink ratio.

ˆ But minimizing data-ink should be seen as a loose guideline, not a rigorous principle.

ˆ Tufte's suggestion on data ink is not based on scientic studies but more on personal
preferences.

ˆ Redundant data ink may be helpful. Example: train schedule chart.

6.3 Chartjunk: Vibrations, Grids and Ducks

Avoid moiré patterns.

Keep grids subtle. A revised version of the train plan.

43
Ducks.
ˆ a duck is a graphic which is just entirely decoration, e.g. self-promotion of graphical tech-
niques instead of information display

ˆ sometimes a table may be better than a pointless graphic

Example: removing chart junk from a bar chart.

[Link]

6.4 Critical discussion

The following summary is loosely based on a discussion in [Cairo, 2013, Chapter 3].

Dumbing down.
ˆ common misconception: graphics are for `dumbing down' data, ashy presentation of `bor-
ing statistics'

ˆ when struggling to design an interpretable gure, reex is to merely simplify the data
instead of working harder

ˆ Tufte: we should not think that our readers are stupid.

ˆ Of course this is true. But also keep in mind: readers/listeners need time to absorb, process,
pause and digest new information. Not all required background-knowledge may be present
(or has become a little diuse). We get tired. Human brains are not computers, eyes are
not cameras. Motivation, patience, redundancy and good graphics are key to including the
audience.

Edward Tufte vs. Nigel Holmes.

44
 ˆ Nigel Holmes was art director for Time magazine

ˆ example: illustration of diamond prices, 1980s

ˆ `anti-Tufte': very low data-ink ratio, data-density, full of decoration and `chart junk', (and
blatantly sexist)

However:

ˆ Tufte's principles are not rigorously based on scientic research but also on aesthetic pref-
erences

ˆ there is no empirical evidence that data-ink ratio is indeed a good measure for the quality
of a graphic (in terms of readability)

ˆ mixed results in studies:

 Ben-Gurion University, 2007, 87 students: compare bar charts with minimalistic ver-
sions.
No signicant dierence in interpretation performance;
students aesthetically preferred `classical' charts.
 University of Saskatchewan (Canada), 2010, 20 students: compare four Nigel Holmes
illustrations with minimalistic versions.
Subjects interpreted both versions equally well.
After a waiting period, subjects could answer questions about Holme's graphics with
higher accuracy (were not told that they would be questioned)
 these are not conclusive, representative studies (e.g. very small sample groups) but
tempting naive conclusion: decoration may help the brain remember a graphic (and
thus also its data)

45
7 How to lie with charts
ˆ This section follows roughly the book [Cairo, 2019] with most examples are taken from
there. Some examples are taken from [Tufte, 1983].
ˆ Cairo identies several ways that charts may lie to us:
 by being poorly designed
 by displaying dubious data
 by displaying insucient or inappropriate data
 in the way uncertainty is treated
 by suggesting misleading patterns
 by pandering to our expectations and prejudices
ˆ DISCLAIMER: The data and charts shown in this section are often ctional and have
not been veried by me. Due to the sheer mass of examples not each example can be
discussed in detail. The examples should therefore not be misinterpreted as `sources' on
the respective topics but merely as illustrations for certain misrepresentations of data and
relations. The list given in this section is by no means exhaustive.

7.1 Poor design

Skewed encoding.
ˆ graphical representation of numbers is not consistent with numbers
ˆ seen previously: area and radius of bubbles
ˆ example [Tufte, 1983]: missing base line

Tufte's lie factor: size of eect in graphic / size of eect in data.

46
 ˆ example [Tufte, 1983]: fuel economy, miles per gallon requirements

ˆ 1978: 18mpg represented by 0.6 inches

ˆ 1985: 27.5mpg represented by 5.3 inches

ˆ lie factor=(5.3/0.6)/(27.5/18)=5.78 (computed slightly dierently in Tufte's book)

Inconsistent design.

ˆ example [Tufte, 1983]: nobel prices (left: original version, right: xed version)

ˆ number of nobel prizes per decade is shown, but last data point is only over four years

Distorting 3d eects.
ˆ example [Tufte, 1983]: oil prices

ˆ example [Cairo, 2019]: 3d charts for ctional company

47
Why it matters?
ˆ taken from [Cairo, 2019]

ˆ study on the deceptiveness of deceptive graphics: Anshul Vikram Pandey et al.: `How
Deceptive Are Deceptive Visualizations? An Empirical Analysis of Common Distortion
Techniques', New York University Public Law and Legal Theory Working Papers 504 (2015)

ˆ charts comparing the drinking water supply situation in two ctional cities

ˆ subjects were shown representative or skewed versions, all charts in addition contained the
correct numbers

ˆ subjects were subsequently asked to rate the severity of the dierences between the two
cities

ˆ subjects with wrong chart versions were substantially misled, eect weaker in more edu-
cated subjects, but they were also misled

7.1.1 Axes and scaling.

Role of the baseline.
ˆ previously: emphasis that y-axis should start at zero

ˆ but it is not always so easy: zero is sometimes arbitrary

ˆ example [Cairo, 2019]: degrees Celsius or Fahrenheit in global warming plot

48
 ˆ similar example [Cairo, 2019]: what is appropriate vertical scale for national debt?

ˆ relevant vertical scale depends on context

ˆ starting at zero may not be necessary if numbers are not encoded by length (bar chart)
but by position (scatter / line plot)

ˆ but need to make sure that readers are not misled (e.g. do not draw axis at non-zero value,
additional hint in caption)

Aspect ratio.
ˆ changing the aspect ratio of a graphic can change its perception

ˆ example [Cairo, 2019]: life expectancy

Logarithmic scale.
ˆ sometimes non-linear scales of the axes are more appropriate

 allows separation of data points at various orders of magnitude

 allows accurate representation of exponential growth
 careful: can potentially be misleading
ˆ example [Cairo, 2019]: GDP vs life expectancy

ˆ example [Cairo, 2019]: exponential growth

Multiple plots with dierent scales.

ˆ example [Cairo, 2019]: population in the Americas. In a single plot scaling imposed by
large countries makes it hard to see dierences in smaller countries. Breaking the axis is
a common technique but it can be misleading. Alternative: show additional second chart
with a `zoom' on the smaller countries.

49
7.2 Dubious data
Ination.
ˆ adjust monetary values for ination, possibly for purchasing power parity

ˆ example [Tufte, 1983]: oil prices and ination

ˆ example [Tufte, 1983]: budget increase (contains also several design aws, and does not
correct for increase of population)

ˆ example [Tufte, 1983]: Playfair (has also published a chart on the increase of the national
debt, in nominal currency and with dramatic portrait format, but has included also a
`de-sensationalized' landscape format version with inated-adjusted numbers)

Faulty data.
ˆ the displayed data can simply be wrong / contains glitches that were not noticed

ˆ example [Cairo, 2019]: pornhub (pornhub published statistics about access numbers in
various states of the US based on IP addresses. Kansas was a strong outlier. Actual
explanation is not immense porn consumption in Kansas, but that IP addresses that were
assigned to US, but not to a federal state, were incorrectly assigned to Kansas)

Non-representative data / apples and oranges.

ˆ example [Cairo, 2019]: prices for health care (appeared on a news website, conrms bias one
may have as European. no details on data given with chart, only source. After looking up
source: US data is averaged over large number of medical claims. Data for rest of countries
are given by insurance companies that are members of the agency which published the

50
 data, not representative for each country, entirely dierent source. So hard to compare the
numbers.)

7.3 Insucient data

Context: preceding and subsequent data points, related data series.
ˆ example [Tufte, 1983]: trac deaths and stricter speeding enforcement (shown chart only
shows to subsequent data points in one state. is this part of a larger trend? how did the
number evolve in other states without the change in enforcement?)

ˆ example [Cairo, 2019]: Dow Jones since Trump inauguration

Too much summarizing.

ˆ example [Cairo, 2019]: tax cuts and jobs act (Paul Ryan: `average family saves 1182$'.
but how are savings distributed over dierent income groups? larger incomes get a larger
relative tax cut.)

ˆ example [Cairo, 2019]: Barclays gender pay gap (BBC: `women earn up to 43% less than
men at Barclays'. Impression: dierence due to unequal pay for similar positions. more
detailed data: dierence due to much fewer women in senior and management positions.
So still a stark inequality, but must be addressed very dierently.)

Too much data.

ˆ try to hide the message by burying it
ˆ example: all 100+ fertility curves without any highlighting in the example from section 1

51
Absolute numbers vs ratios and conditional probabilities.
ˆ example [Cairo, 2019]: vaccination (ctitious example, inspired by reality): more children
dying due to the vaccine than due to smallpox. but: ignores total number of vaccinated
or non-vaccinated children. conditional probability for death among vaccinated children is
much lower than among un-vaccinated children.

ˆ example [Cairo, 2019]: Charleston church shooting 2015 (Shooter was partially motivated
by statistics of `black-on-white crime'. Chart made it look as if black violent oenders were
specically targeting white people. This ignores the composition of the general population.
See python example.)

What is the right metric for success?.

ˆ example: movies and box oce (ination, number of cinemas, marketing and lming bud-
get,. . . )

7.4 Dealing with uncertainty

ˆ example [Cairo, 2019]: climate change (Bret Stephen in New York Times: `modest warming
since 1880 by 0.85 degrees Celsius and its human cause are indisputable, scientic models
for future trend are subject to uncertainty and should be treated as such' (paraphrasing
by me). But: 0.85 degrees Celsius is by no means modest. While future models carry
uncertainty, they have shown to be conservative in the past, and the general trend in the
overwhelming majority of models points upwards. This should not be discarded by a mere
reference to associated uncertainty.)

ˆ example: election polls

52
 ˆ example: unemployment rate (4.3% to 4.4% in one month means almost nothing. it is
mostly noise. look at longer time-frame to see signal. similar: daily corona virus infection
numbers)

7.5 Suggesting misleading patterns

correlation vs causation.
ˆ example [Cairo, 2019]: Obamacare and unemployment (signing of Aordable Care Act
coincides with a beginning of economic recovery. but no causal relation can be inferred at
this point)

ˆ example [Cairo, 2019]: religion and happiness

 on the country level the two seem to be negatively correlated. but: cannot infer causal
relation.
 maybe confounding factor: inequality
 at the individual level the correlation may even be reversed in countries with high
inequality

ˆ example: spurious correlations: Nicolas Cage, science and strangulation

53
population fallacy.
ˆ example [Cairo, 2019]: cigarettes vs life expectancy (trend at the country level seems like
a positive correlation, because wealth may be a confounding factor. at the individual level
we recover well-established negative correlation)

ˆ example [Cairo, 2019]: Kansas citizens voting against their interest

Simpson's paradox.
ˆ example: kidney stones ([Link]

misreading exponential growth.

ˆ example [Cairo, 2019]: Cambrian explosion (creationists argue that sudden increase in
diversity of genera is proof of intelligent creator, scientic data suggests it is not `sudden'
and part of a longer growth trend)

ˆ in linear scale, a period of exponential growth can look like an `o-on'-situation

7.6 Epilogue
For chart creators: Tufte's principles of graphical integrity.
ˆ The representation of numbers, as physically measured on the surface of the graphic itself,
should be directly proportional to the numerical quantities represented.

ˆ Clear, detailed, and thorough labeling should be used to defeat graphical distortion and
ambiguity. Write out explanations of the data on the graphic itself. Label important events
in the data.

ˆ Show data variation, not design variation.

ˆ In time-series displays of money, deated and standardized units of monetary measurement

are nearly always better than nominal units.

ˆ The number of information-carrying (variable) dimensions depicted should not exceed the
number of dimensions in the data.

ˆ Graphics must not quote data out of context.

54
For chart consumers: Cairo's principles.
With the internet we consume a lot of charts from a wide range of sources, often not professional
or acting in good faith, and often without sucient context. It is impossible to scrutinize every
chart that we see. But we should be wary. Cairo gives some advice [Cairo, end of Chapter 3,
subset of full list]:

ˆ Don't trust any chart built or shared by a source you're not familiar with until you can vet
either the chart or the source, or both.

ˆ Don't trust chart authors and publishers who don't mention the sources of their data or
who don't link directly to them. Transparency is another sign of appropriate standards.

ˆ Expose yourself to sources you disagree with, and assume good faith on their part.

ˆ Don't assume ill intentions when haste, sloppiness, or ignorance is the more likely expla-
nation for a bad chart.

ˆ But trust has its limits. If you begin spotting a pattern of misdeed in a source on your list,
erase it.

ˆ Follow only sources that issue corrections when they ought to and that do it visibly.

ˆ All journalists have political views. Who doesn't? But most try to curb them, and they
do their best to convey, as famed Watergate reporter Carl Bernstein likes to say, `the best
obtainable version of the truth.'

ˆ Telling the dierence between a merely partisan source, there are reliable ones all over
the ideological spectrum, and a hyperpartisan one can be tricky. It will require some
time and eort on your part, but there is a very good clue you can begin with: the
tone of the source's messages, including whether that source employs ideologically loaded,
bombastic, or aggressive language. If it does, stop paying attention to it, even if it's just
for entertainment.

ˆ Expertise matters, but it's also specic. When it comes to arguing over a chart about
immigration, your judgment as a layperson is as valid as that of a mechanical engineer or
someone with a PhD in physics or philosophy. And your opinion is less likely to be accurate
than are the ones expressed by statisticians, social scientists, or attorneys who specialize
in immigration. Embrace intellectual modesty.

ˆ It's easy to be overly critical of charts that depict realities we'd rather not learn about. It's
much harder to read those charts, to assume that their creators likely acted in good faith,
and then to coolly assess whether what the charts show has merit. Don't immediately snap
to judgment against a chart just because you dislike its designers or their ideology.

ˆ Conversely: The more ideologically aligned you are with a publication, the more you should
force yourself to read whatever it publishes with a critical eye. We humans nd comfort in
charts and stories that corroborate what we already believe and react negatively against
those that refute it.

ˆ We are prone to lying to ourselves.

55
Reasoning and rationalization.
Some points from the epilogue in Cairo's book.

ˆ We are not rational thinking machines as we may sometimes think.

ˆ We are prone to lying to ourselves, to all sorts of logical fallacies and mental biases.

ˆ `Forming beliefs is easy; changing them is hard work.' (from `The Believing Brain' by
psychologist Michael Shermer)

ˆ example of cheating students (from `Mistakes Were Made (but Not by Me)' by Carol Tavris
and Elliot Aronson):

ˆ start with two students, initially with mild opinions about cheating

ˆ one day, during exam, both are tempted to cheat. one goes through with it, the other one
not.

ˆ the one who resisted will likely become more self-righteous about not cheating

ˆ the other may start to say cheating is not that bad, or it was justied in this case, because
the exam was unfair, everyone does it, etc. (I'm a good person, so it's impossible that
cheating is really that bad!)

ˆ rationalization is the habit of `defending' views/beliefs we already hold, even against new
evidence. we are very prone to it.

56
8 Animations and interactive visualization
8.1 Motivation
ˆ seen in previous chapters: compressing high-dimensional data into graphics is considerable
challenge
ˆ time oers an additional dimension
ˆ one additional dimension can often be accessed by a pre-dened animation
ˆ even more dimensions accessible via interaction

8.2 Relevant eects in human visual system

Recall: Change blindness.
ˆ example: altimeter
ˆ example: housing lter

Recall: Attentional blink.

ˆ task: identify targets in fast sequence of images
ˆ when pre-attentive processing has identied target (and thus `activates' conscious process-
ing), the pre-attentive processing is disabled for a short interval

Saccadic blindness.
ˆ recall: eye movement composed of xations (≈ 200ms) and fast saccades between them (≈
20ms)
ˆ during saccade we are blind, if we quickly move gaze from one part of room to another, we
do not see scene continuously moving by
ˆ intuitively clear: our visual designs should reduce number of saccades

Rapid serial visual representation (RSVP) [Spence, 2014].

ˆ experimental results
 mode 1: subjects rst look at given reference image, then see rapid sequence of images
(100ms per image), need to determine whether reference image was part of sequence.
success rate: 80%-90%
 mode 2: no prior instruction; subjects are shown reference image right after sequence;
still success rate of ≈ 90%.
 but this drops to 10%-20% if one pauses for ≈ 5s after the sequence, before showing
the reference image
 mode 3: images are still only shown for 100ms, but blank pauses of 1.5s are introduced
between images. success rate ≈ 90%. (unfortunately, unclear from Spence's text if
this experiment was performed with or without pause after sequence, and if sequence
contained fewer pictures)

57
 ˆ interpretation:

 100ms seems to be enough to capture gist of image

 this processing must be pre-attentive
 but not enough time for consolidation in short-term memory

Moving RSVP.
ˆ dierent presentation modes:

 collage
 shot
 oat (with brief `capture' of pictures near center)
ˆ experimental comparison (Corsato at al., An eye tracking approach to image search activ-
ities using RSVP display techniques. ACM Proceedings of conference on advanced visual
interfaces, 2008)

 identify 40 out of 2000 images in the three presentation modes

 compare eye movement (`gaze travel'), success rate
 oat mode is most ecient, why?
 for humans task consists of pre-attentive identication of relevant images; then con-
scious conrmation
 oat mode allows easy identication at center, without gaze movement; during subse-
quent movement pictures are still visible long enough for conrmation; we may even
have enough time to outlast an attentional blink

Visual momentum.
ˆ Spence, Section 5.3.8; Woods: `Visual Momentum: A Concept to Improve the Cognitive
Coupling of Person and Computer', International Journal of Man-Machine Studies 1984

ˆ goal of visualization is to help user build mental model of data

ˆ discontinuous changes in display easily disrupt this model

ˆ examples:

 rotating a scatter plot

 interpolating values in bar charts and similar
 `appearing houses' also fall in this category
ˆ comment: implementing these things in practice can be extremely complex, may consume
most of the coding time, need proper environment

58
Some implications for animations.
ˆ When viewers cannot control playback (e.g. in talk), show animation in loop or at least
several runs. Before understanding content of video, viwers need to decode it and learn its
visual language.

ˆ When animation is running, let viewers watch, do not compete for attention. In short clips
it is often possible to explain what they are supposed to see before, or maybe after a rst
run (→ recall attentional blindness).

ˆ Choose suitable playback speed. Keep in mind: viewers usually see video for the rst time.
Leave extra time when you explain things during the video.

ˆ When an animation shows temporal evolution over time, do not immediately cut away
after last time-step. Give viewers time to study the nal state.

8.3 Control elements

Norman's action cycle.
ˆ Donald A. Norman: The Design of Everyday Things, 1988

ˆ how to design common devices? door handles, coee makers, bicycle brakes

ˆ general ideas also apply to interactive graphics

Design principles.
ˆ aordance, perceived and actual

 what does a control element / device `oer' ?

 should be intuitive, extends to design in general
 there are social conventions and expectations
 typical examples: doors, shower knobs, lamps

ˆ metaphors in graphical user interface: exploit familiarity of users with other concepts for
fast adaptation

 windows mimic overlapping sheets of paper that can be rearranged

 manipulation of objects on screen: drag and drop, scrolling by `pushing'
 input elements on windows are design similar to `analog' forms (by now: probably the
converse is true)
 lesystem with folders mimics large ling cabinet with various drawers, separators,
and dossiers

ˆ feedback, examples:

 `click' animation (or sound?)

 reliable progress bars
 trac light without feedback

59
 ˆ constraints

 example: `swipe animation' at end of picture gallery on phone

 grey out buttons; not make them entirely invisible; make sure `inactive' look is not
confusing

ˆ consistency

 minimize user time for learning how to use the program

 objects that look the same should behave the same
 there are many conventions already on what design elements look like
ˆ what are typical user goals?

 make sure these can be obtained eciently

 example: Deutsche Bahn app, looking for an alternative connection

8.4 Navigating, browsing and ltering complex data

Standard navigation operations.
ˆ open, close, and move windows

ˆ drag and drop

ˆ sort and rearrange

ˆ search and lter

ˆ scroll, pan, zoom

 on `map' move in all four directions

 for text pdf: vertical scrolling more useful than horizontal
ˆ for all of these there are by now conventions on design and control interfaces

ˆ these can usually be combined in natural ways

Context.
ˆ thumbnails of neighbouring pages or images

ˆ document outline / section structure in PDF reader, hyperlinks within document

ˆ hover preview on links (wikipedia, PDF reader)

60
Level of detail.
ˆ travel navigation: before digital navigation assistance, printed maps faced a dicult trade-
o:

 high resolution: impossible to print for large area, would constantly need to turn
pages, dicult to get large scale impression of travel
 low resolution: not enough detail within cities, and at critical junctions
 digital navigation can store all high resolution information, automatically track loca-
tion on map, and choose zoom level based on appropriate level of detail

ˆ engineering drawing: distinguish between full details for all parts, and reduced version that
explains how to replace a certain part

Hierarchies.
ˆ example: hyperbolic graph browser. Better than `regular zoom' since it still provides
context.

ˆ le system explorer

 Directories and lenames themselves are a means of visualization of data. With proper
use we can eortlessly remember (or at least retrieve) the locations of thousands of
les.
 list vs items vs thumbnails

Moving within a program / an app.

ˆ When using software via a graphical interface, users need to move between dierent menus
and query interfaces.

ˆ Multiple instances of same interface might be required at same time. They can be organized
in tabs or oating windows.

ˆ relatively easy on computer: large screen, complex input devices

ˆ extremely challenging on smart phones:

 much smaller screen, touch screen much coarser than mouse, no keyboard, only `back
button'
 navigation often only in `tree graph', cumbersome for complex applications
 recall example: Deutsche Bahn navigator app
ˆ on laptops: mouse less precise

61
8.5 Sensitivity and dynamic exploration
Extend mental model of users.
ˆ user should quickly be able to extend their mental model to include expectations to reac-
tions on changing parameters

ˆ make experience as `smooth' as possible

ˆ exploration should be simplied, in particular in high dimensions

ˆ so ideally: from current point can glimpse a little bit into all directions

Sensitivity data.
ˆ how did parameter change aect change of outcome?

ˆ anticipate if a small parameter change can lead to a desired outcome? show `near misses'

Dynamic exploration.
ˆ short-cut in action cycle, get immediate feedback

8.6 Examples: interactive plotting and animations

Mathematica.
ˆ Manipulate[] function [Link]
ml

PlotLy.
ˆ very interactive even in default mode

ˆ core library implemented in JavaScript, hence simple export of dynamic gures as HTML

ˆ code not yet stable/persistent, subjective: deviating a little from give examples with short
code usually requires a lot more code

Matplotlib: interactive + animations.

ˆ basic zoom/perspective changes in interactive widgets

ˆ animation

ˆ simple dynamic interaction

ˆ feels outdated, essentially works like old-school GUI implementation, substantial boilerplate
code

62
Manim: animations.
ˆ script-based creation of animations in python

ˆ ideal for algorithmic creation of data-based animations

ˆ easy to keep `visual momentum'

ˆ `community' fork available at [Link]

8.7 Examples: Interactive plotting for the web

ˆ there is a plethora of java script charting libraries:
[Link]
ˆ generally much easier now with WebGL and powerful browsers

echarts
ˆ [Link]

ˆ very easy to get started

[Link]
ˆ [Link]

ˆ nice example gallery: `open data show room' of Uni Bern, Institut für Wirtschaftsinformatik
(German only, unfortunately), [Link]
warning: this also contains quite a few ducks in Tufte's sense, e.g. [Link]
[Link]/#

PlotLy / Altair
ˆ for PlotLy see above, Altair is similar

Tensorow Playground.
ˆ [Link]

ˆ interactive arrangement and training of a neural network

8.8 Visualization for websites

Context.
ˆ for creating a website one has to solve visualization problems on (at least) three layers

ˆ layer 1: global structure

 how is the whole content broken down into individual documents? what is the hier-
archical structure of documents?

63
  how is this structure presented to the user? how can users navigate it? (only moving
in the tree, use search function, many internal hyperlinks)

ˆ layer 2: design of a single page/document

 how are navigation, text and embedded media combined?

ˆ layer 3: media within a single page

 embedded media can be (interactive) gures, videos, external content

Some comments on navigation.

ˆ menu structure: tall & thin vs at & fat

 how complex is each choice? how many choices necessary? when looking for a leaf
node, are the names of the parent nodes clear?
 categories can be claried by keywords: `look ahead information' (`residue', Furnas:
Eective view navigation, ACM proceedings of CHI'97, 1997)

ˆ breadcrumbs: visualization of past trajectory of user

 often supplied by `back' button in browser, but in interactive sessions often in conict
with website functionality (re-sending of requests?)

ˆ `see also' suggestions and cross-references

ˆ how often do you use page-internal navigation vs google?

ˆ increasingly popular: virtually no `static predened hierarchical navigation tools'; content

selection solely based on search query interface and machine learning (`users also liked. . . ')

Dark patterns.
ˆ In a perfect world the design of websites would be optimizes for user experience. In practice
they are often designed to encourage the best outcome for the company that runs the
website. When website (or generally application) design intends to bias the user against
acting in their best interest this is referred to as a `dark pattern'.

ˆ example: accepting the default privacy settings (which are very open) is much easier (and
encouraged by suggestive link coloring) than choosing a more restrictive setting

ˆ example: pressuring users into booking overpriced accommodation or transportation tickets

buy creating the impression that availability is very limited; this is often amplied by
dynamic methods, such as pop-up warnings

ˆ example: it is sometimes very easy to start a subscription for a service with very few steps
that are easy to follow, whereas cancellation may be only possible via much more subtle
paths, including articial obstacles (possibly not being possible online at all)

64