1 Introduction hhow like the needle information fs, It always has a point and needs an eye. Thomas F. Gilbert Print-based information visualization has been used effectively for centuries to marshal ‘multifaceted data in the service of making a point visually. The advent of interactive computer graphics, the Internet, and readily avallable sources of data extends that rich tradition and introduces a new kind of expression, interactive visualization. This inno- vative medium has the potential for sustaining a meaningful virtual dialogue between scholars and their audience, using data as the liaison. This chapter introduces the emerging field of interactive visualization by exploring the following topics: + What is information visualization? + Why visualization Is valuable + A brief history of visualization + Information graphics in journalism = Theoretical models for interactive visualization + Descriptive models for interactive visualization + Prescriptive models for interactive visualization + The ASSERT model for interactive visualization What Is Information Visualization? in 1854, an epidemic of cholera spread through the Soho section of London, Killing hundreds of people in a matter of weeks. London was overcrowded by the influx of people to the elty from the country, and sanitary services could not keep up with Mt Fopulation, The streets were literally cesspool of human waste, and the ety leaders peamed that the constant odor in the air was the cause of the outbreak F rondon anesthesiologist named John Snov used data from a local priest, Henry Wiutchesd, and London city epidemiologist Willam Farr to geographically map theFigure 1.1 Portion of John Snow's cholera map (image courtesy John Snow Atchive and Research Companion), spread of the massive cholera outbreak in London. Snow plotted the deaths of cholea victims on a simplified map of the afflicted Soho neighborhood (see figure 1.1) 0 Convince authorities that cholera was waterborne and that the outbreak emanated from an infected water pump on Broad Street The visualized data that Snow presented helped tum the tide from the airbomne mlasma theory of how disease spread to the modern germ theory and Brovided early evidence of visualization’s power to persuade Johnson, 2006, hit, 1997) Farly efforts such as Snow’s simple but et king full advantage of the new technolo, statistics, and computer ization is a subset of sometimes ref ffective map have been growing and are isies afforded by computers, the Intemet, * graphics, known as interactive visualization. interactive vis a larger field known tal 8 information visualization, which is aso ‘erred to as informatics, that crosses the disciplinary boundaries of com Miemces nee tessa, statistics, psychology, cognition, neuroscence, and the bse Gefinitgn ce S20Y fesearchers Stuart Card, jock Mackinlay, and Ben Shnelderman’s ddintion of information visualization is the mest oer cract description. of this dete tong t He te Of computersupported, interactive, visual representation of abso data t0 amply cognitionIntroduction. 3 Implicit in this det increase understandi tion isthe potential for information visualization to externally 1g by (1) Increasing the number of resources available to the viewer while remaining cognizant of the limits of human working memory, 2) reduc ing the need for time-consuming and tedious searching, (3) enhancing pattern recog: nition to detect meaningful trends and conditions, (4) using the properties of the human perception system to effectively communicate meaning, and (5) allowing the user to directly interact with the information to construct his or her own understand- ings (Card, Mackinlay, & Shneiderman, 1999). Information visualization involves the reduction of raw information, such as data, Into simpler graphical elements that use spatial variables, such as position, size, shape, and color, to revea ships and pattezns implicit but hidden within the data, “The goal is to discover the hidden structure of a (typically large) dataset” (Manovich, 2010). \ good visualization can tell a rich and profound story using primary source data to answer meaningful questions about a topic. This aligns with the research practices ‘used by other disciplines such as history and the social sciences and provides an ‘opportunity for scholars to share research in this new medium in other forms beyond visual relatio the written word. this is not to say that the medium in which an argument is presented has no effect con the ultimate perception of the message (McLuhan, 1964). Writing has its own Internal structures, such as linearity, sequence, syntax, abstraction, and cultural heri- ie, that can cast a profound effect on meaning. Different media types wil Be Better suited to representing particular subject matter and characteristics than others, The cleverly described the difficulty matching an argu: “Writing about music Is like popular musician Elvis Costello’ ment with its mediuim of expression by explaining that dancing about architecture” (Staley, 2003). Genres of Visualization 1 variety of reasons. Scientists use visualiza- pifferent groups create visualizations for historians use visualization ‘communicate scientific phenomena, le do, and journalists use Infographics to she general public. This book uses the term atic and interactive visualiza- reality-based. tion to graphically to explain and further explore what peopl clarify and communicate complex ideas tot ‘as a broad category encompassing both st ed interchangeably to represent the ‘often rooted in historical visualization, visualization tions, Data and information are w xd. The examples discussed are elements involve’ 11 to all types of visualization, put the ideas demonstrated are commor her people than Costello, including Martin Mull, Laure 1, This quote has been attributed to 0 vind Miles Davis. Im sticking with Elvis on this one. Anderson, Frank Zappa, Steve Martin,Chapter eu} Figure 1.2 Molecular visualization by Ben} in Grosser, University of Illinois, Scientific Visualization and Simulations Scientific visualization is a subset of visualization that uses quantitative data to cite sraphical images of scientific ies that data represent so of Three-dimensional rende: imagery Mesomen can yx the unig oe paguely when representa only as tale of mbes ringef molecules eigenen ee alow researchers to visually explore thes oterwise might not be abiows tmadlion tothe gol of deseibing phe interactively rich datasets and gain insights that -homena, one of science’s roles isto predict retical foundation. Scientific visualizations in the form “sarily need to rely on real-world data but can use mathemat sential equations to generate data based on theoretical models that ay “what if” scenarios and graphically display the results Historical Visualization 11 our work with historical visualization at the University of Virginia, we define inter active visualization. as the process of te "ting primary sources of information communicateIntroduction Figure 1.3 Screen shot from Visualfyes "Notes from Virginia” project (Image courtesy Scot Hench). ivectly with a viewer to support inquiry ina visual, compelling, and interactive manner. The primary sources of information for historical study ean take many forms, including ps, drawings, letters, legal documents, numeric data consisting of newspaper accounts, and jcating the material can photographs, m population, demographic, and financial data, 1y product of human effort. The method for commun timelines, maps, animations, three-dimensional (3D) The means of interaction with those data database to fully immersive 3D almost be equally varied, from charts, renderings, and movies (see figure 1.3) ic textual queries on a can range from dyn environments, Infographics Infographics isa subs magazines to explain complex depiction using charts and illust xd as two-dimen set of information visualization. Itis often used by newspapers and processes in an attractive, clear, and direct graphical rations in a highly aesthetic manner (see figure 1-4) sional static images with high production values, cused of oversimplifying complex issues Typically rendere many journalistic infogeaphics are often acc and choosing aesthetics over truth.Chapter Figure 1.4 nt * COOP Magazine image courtesy Chris Korbey: [Link] Interactive Visualization Interactive visualizations Most of Its features. Sp dynamically with the vis the user to limit or e datasets together, sho jon and share are another subset of information visualizat mi scal ‘ecifcally, they add the ability for the end ee sualization in a reflexive manner. This interact ae “Xpand the data being explored, to selectively contrat ore deals Ow change over time, or zoom into the data for mi Because of the large num ons, interactive ber of possible combinations for user interaction ae erative, meaning that they can potentially peecas ‘yond what the visualization’s designer had original Pt nade possible by the ever-rising capabilities of cone a “Pensive personal computers, The spread of this technol talizations can be ge into the information bey This interaction is m 'es technology on ine [rrrIntroduction 4 mobile devices such as cell phones and hand-held tablets will undoubtedly provide new opportunities for people to graphically interact with data and pated but often fascinating insights, Why Visualization Is Valuable Information visualization is useful in both organizing and making sense out of data, particularly when there is a lot of it. We ae limited in the amount of analysis we can do by looking at raw data, but when those data are visually structured, we are better able to understand complex and multidimensional relationships, prompting connec tions that may have not been otherwise apparent. We do this by taking advantage of four brain’s massive parallel processing capabilities that the human visual system affords, Numbers alone do not always tell the full story but often are a good place to start. fo find that story, statisticians use very precise mathematical techniques to describe the relationships among groups of numbers, such as the mean, the variance, and the correlation. For the most part, these techniques accurately provide a good representa- tion of the underlying phenomenon the data represent, but that is not always the In 1973 the Yale statistician Frank Anscombe created four unique datasets that and correlations but that, when graphed, showed generated identical means, variances, jphs would have shown very different shapes (Figure 1.5). Ina real dataset, these very different distributions of the samples and suggested very different underlying dynamics. Clearly, the statistical measures did not accurately portray the underlying phenome figure 1.5, the datasets are normally distributed, Ship with one anomaly, and, finally, a curvilinear relationship (Tufte, 1983) The visual system is a powerful pattern-recognizing mechanism that can instantly discern patterns that are unmediated by mathematics. Datardriven, visualizations huvness that innate human ability toward a deeper understanding of thelr meaning, Jon very well in this case. Reading clockwise ‘a very linear relationship with one outlier, a very linear relation. A Brief History of Information Visualization in a graphical context has been practiced for centu: histication and reliance on empirical data. Leonardo 6) directly communicated the correla figure of a man circumscribed within The visualization of information ries, with an ever-increasing sop! 1487 drawing Vitruvian Mam (figure 1 ing the da Vinci's tions of the ideal human proportion us » perfect circle, By the eighteenth century, mercha ata about their world, This information was typically stores ts and governments collected progressively din more quantitativeChapter ° [le | oui it ents a different massive t 1 soil ables of numbers, making interpretation difficult at best. The bee sclentst William Playfair (1759-1823) devised a number of conte Gevices such as pie, bar, and line charts (figure 1.7) that are ie a nicale tabular data with clarity using this much more sccenbie fon a Aithough she is known mainly for her work as a nurse, the wr ary Horence Nightingale was a key figure in using statistical partis Oa dean role that inactivity, malnutrition, and inadequate smaltation eae Gtaus during the Crimean War, She kept detailed records and used i s caus On Playfair’s ple charts to create rose diagrams (figure 1.8), ow KNW coxcombs (Spiegthalter, 2008) The Use Of graphs and charts to vi place with new advances new ideas such as the intuitive. Howeve ao al quantitative data bam mor analyses. Representing even the simplest and quarts as numbers was not mt Isually represent them through the we os 18 statistician Johin Tukey's box and while pth to the dat ple graphical device sle glance. The in statistical Inge, medi , techniques to, y Charting types, such as the pioneers, igure 1.9), provided dey For example, this si facts of a dataset in a sin, al conveys te baste descriptive sae! }box's height is mapped to the lower a eeIntroduction u Figure 1.6 Leonardo da Vinet 1487 Vitruvian Man,2 Chapter EUR Figure 1.7 Early chart by William Playfair, DIAGRAM or snus EAUSRS or MORTALLTY OTHE ARMY sn tHE EAST ova, mein ass Figure 18 rete NightiCgae’ 1855 cours charIntroduction B Figure 1.9 John Tukey's box-and-whisker graph, quartiles, the line within the box shows the median, the middle of the box shows the can average, anil the two whiskers on top and bottom show the range of data. In the beginning of the eighteenth century, the English geologist and former canal builder William Smith created what has been called “the map that changed the world” (igure 1.10). It was the fist published geological map of Britain, where the diferent feological layers were represented as an overlay to the underlying geography using Information Smith garnered from extensively traveling across England (Coppock & hind, 1991), This remarkable map helped introduce the concept that maps could be ised to convey other information aside from geography. Twoedimensional maps took a turn away from geographic accuracy and toward increased utility in 1931, when a draftsman working for the London Underground linew a map (figuee 1.11) in his spare time that forever changed the way subway maps svere dawn, Harty Beck ignored the actual distances between stops and the geographic Flacement of lines, stations, and crossing points in favor of making the sap more tusable for its riders and increa 2001) ng. its communicative impact (Spence, The Impact of Computers the advent of computers that are capal sone opportunity for both designers and statisticians to experiment With @ Det id render complex graphics based on large sets of syond the static drawings of the ble of dravving sophisticated graphics provided cf expression, one that cou! mediun jarich graphical visualizations be data to create dat previous centuries.Introduction 1s Figure 1.11 eery Beck's London Underground map (Image courtesy and copyright TFL/London Transport Animators such as the legendary abstract flm artist John Whitney experimented ‘and animating imagery using the computer. After years with algorithmically creating 1.12), Whitney adopted the of creating abstract films using analog techniques (figure tool to produce a mesmerizing relationship between music and artist in residence In 1968. intial rates of growth and rendering tech- being used to render realistic models ‘The Jet Propulsion Laboratory's animation with the simula: provided photorealistic various planets in its ually witness first digital computer visual design and became IBM's first As processing speeds increased at exponet niques grew in sophistication, computers were a in three dimensions. of complex scientific phenom James Blinn introduced the general public to computer tions created for the Voyager project 1978, These simulations ‘an unmanned spacecraft, approachin 1.13)—something not possible to act depictions of Voyager, travels through space (figure hand ‘with both the rendering fidelity that was inereasing and the increasing processing ct with the graphics being rendered speed! of the computer, in more meaningful ways. Uni nce professors David Evans people were able to intera versity of Utah computer sclMi iT | ih Wi nkCopter v Figure 1.14 Image from Evans and Sutherland simulation. and Ivan Sutherland introduced a new gente of computer tool, the simulator, in which people had an immersive interaction with the computer in applications suchas Might simulators and combat games (Figure 1-14). However, computer graphics were beyond the grasp of nonprogrammers to create mercial companies such as Genigraphics produced systems for making pre wand computer-aided design (CAD) systems for engineering and archi oth of which were available on expensive and uence of the personal computer and accessible ‘and MacroMind’s Director has brought powerful sentation slides tectural drawing such as CADAM, dedicated minicomputers. The confl software such as AutoDesk’s AutoCad hh of casual users. ing industry has had a tremendous impact on the development increasing power and capabilities of modem. etplace has created a critical mass accelerat- 1g chips such as NVIDIA and nls within the rei The computer gami ofsoftware and hardware, driving the ever computer graphics. The large gaming mad igh-performance/low-cost renderin leveloped by game producers. the potential the computer has for the sophis: ation, Rather than be passively content at data elements are compared, con- ‘a good visualization now involves xdvances in software rendering d Iris important not to underestimate ticated interactive communication of inform fesigner’s decisions of wh with the visualization d or hidden, trasted, transformed, aggregated, shown: Chapter ‘dialogue with data that can enable a viewer to generatively explore data ditectly to ights not dictated by the designer, develop in ‘The impact of the Internet The Intemnet and the open-source ethos of free accessibility have made data moe widely available for instantaneous download. A wealth of freely available infomation from US Census data from 1790 to worldwide geographic, economic, and poltal data to real-time social media data from services such as Twitter can be insanty downloaded and used without any requirement for permissions from the pid. These data are increasingly being used by interactive visualization producers topoide Powerful tools for information and inquiry Google and other Internet companies have added simple application prvamning Interfaces (APL) to their popular online services such as maps that encourage people create geographically based visualizations called mashups,? which layer data andmups {ogether in ways that are easier and more accessible to a mass audience than previous Ols'-based desktop systems. This visualization connects realtime police crime et and plots them on a map (figure 1.15) where users can select the kinds of cies 0 View during any given time period. Other freely available API offer a wide range of web-based tools that programmes vith litle effort or experience can use to create timelines, such as MIT's SIMILE poe. These tools have access to online images using Flickrand any numberof charting and visualization APIs including Processing, ProtoVis, and Prefuse, all of which ate freely even a modest level of programming skill to incorporate into his or her Web site. : : = a. are cs genre of Web sites has emerged that make it easy for people who F Usher HOw to create Web sites or programs to produce sophisticated visu on using thelr own data, These Web-apps make it easy to upload datasets and male * Narlety of compelling visualizations that can be shated worldwide. IBM Reseacts Figure Las Oakland Crimespot 8, courtesy of Stamen Design, 2% Image courtesy [Link], Gcosrphie Information System (GIs), ee waaIntroduction 19 et ot a “ASERRY Figure 1.16 Screen shot from Visually project. ple allows users to share and visualize their own datasets using sim snarls, tree-maps, word clouds, and geo: allow red by ManyByes! project Web-based tools to create network diagrams, 1.16). Other tools such as Tableau’ and our own VisualEyes* graphic maps (Figur Interactive visualizations to be rapidly cr for sophisticated data-driven nonprogrammers. The University of Virginia's Sciences, Huma cal Initiatives (SHANTI SHIVA. project” provid services from sites such as Google, Kaltura, Flick, and MIT to provide data, charts, traps, videos, and images together in a simple-to-use interface whee the elements 1,17), The resulting visualizations can easily be linked nities and Arts Network of Technolog Jes access to a number of Web-based interact with one another (figure to or embedded in Web pages or WordPress blogs. 4 [Link]/softwaredata/cognos/manyeyes www [Link]hap een Sal (hoe rat} Figure 1.17 SHIVA projet. Networks. in his influential book, Pattern Language, the architect that the design of the christopher Alexander sete urban landscape was not the product of top-down, hiarci Mandates, bul @ more organic bottom-up process that reflected an “invsble m8 & interconnections” forming the basis of a network (Alexander, Ishikawa, & silvesteit, wedi tim, 2011). This concept resonated with the emergence ofa topology of dn bsPlted new ways to visualize the Web-like connections between newa (gure 1.18). This new shape of information could potentially reveal much about the interconnections between people and ideas do ea, 0 find patterns within anger sets of data prompted a nce ant relation eo, technlaues for effectively visualizing huge datasets and shi : Bere cemeen tndividinl anal geeeesteeiee ee Going beyond ttn eat Os: these network diagrams highlight the conn echon ema try to unitate Felationships between members Visualization in the Academ The a Intertaces of oH Pesonal computer inthe 1980s and the highly ee Interfaces of Unis, aah Windows (all inspired by Xerox FARCS rot ae nat 2€cessible platform for a nee direction for information visualization eerie Interesting, mix of academic sins RrraIntroduction a Figure 1.18 prey network map using data from the CIA Facthook (image courtesy Morttz Stephane. computer science to design are crossing boundaries to mniversites around the world have estab- rocessing from statistics, psychology create new tools, techniques, and projects. U tished centers to keep pace with the rapidly unfolding new developments inp speed! and storage capacities. The University of Maryland’s Hun in 1983 by Ben Shneiderman, has had have consistently pushed the bounds afforded by ewvailability of large sets of raw data, Research verrammber of influential academic and commercial projects for interactive Mion including a toolkit for graphically querying and analyzing datasels vethod called Treemaps for displaying large link-node man Computer Interaction Laboratory" founded 4 an extended history of innovative projects that the personal computer revolution and the increasing at the Laboratory has spawned data visualiza called SpotFire and a mapping mé 6, [Link]Introduction Figure 1.20 Map of Salem witcheraft accusations (Image courtesy Ben Ray/IATH Figure 1.21 The University of Vignla’s Valley of the Shadow project. aChapter 1 Railroad Traffic in Nebraska from Selected Stations (oom) Cm) 5) Figure 1.22 Screen shot of Stanford's Spatial History project. Stanford's Spatial History Project focuses son visualization and history with innovac tive software that intera ctively connects historic maps Project looks at history with a geography-based inter historic data to information such country ihe Stanford Visualization Group" develops innovative da to represent data. They have created a 1 including flare, prefuse, and most recently the new tools for structuring data like Wrangler (ig and data (figure 1 ace that connects a8 the growing role of railways to a growing jons and tools ization toolkits Popular Javascript-based protovis and ure 1.23), quarmation Graphics in the News: Visual Journalism The popular newspaper USA Talay pione ‘graphics in the 1980s in its a Braphs, called infegraphics, py dltiven issues such as wes of a new field: visual jou aesthetics for simplicity ‘red the use of high-profile information nal newspaper. These visually bold ¢ ed simplified but visually engaging views of dees her Politics, and demographics and ashen 1s the advent ualsm. The paper was often criticized ter sacrificing visual and using the images as page fillers ta help m ly nati 10. [Link] /grOup/spatialhistory 11 https isstantord, edIntroduction 25 igure 1.23 A streamgraph visualization by the Stanford Visualization Group. integrity of visual journalism projects, an alliance of over 100 designers and journalists worldwide have endorsed a six-point checklist” from Harvard University’s Neiman Foundation for Journalism (Cairo, 2011). It is the venerable Gray Lady'® that has led the field of innovative information graphics in both the print and online editions of the New York Times. Its online work has dramatically raised the bar of journalistic interactive visualization by consistently providing usable interactive visualizations that utilize state-of-the-art research in both theory and technique. A good collection of the web-based information visualizations can be found at this Web site." The Times has an eclectic staff of 25 editors, artists, 3D modelers, journalists, design- crs, statisticians, cartographers, and computer programmers who develop cutting-edge Infographics to bring data to life for readers of the daily paper and Web site. Speaking ata design conference in Copenhagen, Times Graphics Editor Amanda Cox explained the goals": “We reveal patterns, provide context, and describe relationships ... to create a sense of wonder.” Models for Information Visualization A model is an attempt to describe a complex real-world phenomenon or process by abstracting its important aspects into a more concise and generalizable description. Models are “a set of interrelated guesses about the world,” and a successful model can reduce the number of factors involved to remove confusion but still be able describe the true situation (Lave & March, 1993). 12, [Link]?fuscaction=[Link]&showeaseld 13, A nickname for the New York Times because it has more words than pictures 14, [Link]/new-york-timesinfographics/ 15, Video of Amanda Cox's talk at New Media Days 2010: [Link]Chapter 1 Visualization models generally fall into three camps: e fundamental issues at stake + Theoretical models that try to offer Insight as to the fundamental cate taxonomies of the elements that characterize their + Descriptive models that create taxo Cans ads thr provides framework scat scolas though the pees of creating and evaluating information visualizations. This book is organized around a prescriptive model developed by the author called ASSERT, but it was informed by a long tradition of models outlined in this section, The ASSERT model uses six steps to scaffold the design of inquiry-based interactive visualizations: ask @ question that one might be interested in the answer to; search for evidence 10 support the exploration of that question; structure that information in a ‘meaningful way to explore the question; envision ways to answer the question using that structured data; represent that evidence in a way that provides answers to the question and the opportunity to explore the data; and, finally, fella story that uses the structured information that can offer insight into the question. Theoretical Models A frequent criticism about information visualization is that it lacks the theoretical bases of more matur disciplines. In response, visualization researchers have started ‘dentifying some of the theoretical foundations that support the effectiveness of infor: mation visualizations, pasticularly ine sducation settings. Most research to date has focused on the visuali tion techniques (the strategy) rather than the more foundae tonal scientific underpinnings (the why) behind efficacy, but influences are beginning to surface number of potential Abstraction Innere ithclm (1969) described graphic abstraction as “a means by which a picture Interprets what it portrays.” The level of di pretation isin the eye of the behold tion: the level of mimetic (re letail is presented prima facie, but its inter let. Therefore, there are two dim alistc) detail in the image itself ‘can be interpreted, ranging from idea: things, The Image and interpretation scales are inversely cay realistic and detailed imagery tends to produ sions to abstrac and the way in which it 's and symbols to specific nces of particular ted So that highly and nongeneralizable associa: © broader ideas and situations, inseparably linked into one simultane lat the process of and knowledge, and it involv i se real situations, standing, and th, clement involved in that compreh. ceptions Abstraction isa critical es the extraction of more of thesIntroduction E ‘These traits must be generative in nature, meaning they are not merely descriptive but can build an image that invokes understandings beyond the traits themselves, like Aristotle's principle of entelechy, in which the forms (abstractions) can generate new instances of particulars. Its this abstraction that enables the rich multidimensionality that interactive visualizations can offer beyond the single dimensional thread of the conventional written narrative Interactive visualizations are, at their heart, abstractions of the data and phenom cena they seek to represent, The maps in figure 1.24 represent a continuum from a realistic, highly mimetic depiction, an aerial photograph of Manhattan from Google Earth, to a hand-drawn but still accurate 1865 topographic map from the David Rumsey Collection and, finally, a highly stylized but informationally accurate map by Alexander Cheek” showing the neighborhoods of Manhattan. The degree of that abstraction is an important design question and is dependent on the message to be communicated. The aerial and drawn maps convey a high sense Figure 1.24 Degrees of abstraction in lower Manhattan (neighborhood map courtesy Alexander Check) 16, [Link] 17, [Link]Chapter 1 of accuracy and are useful for locating a point exactly, whereas the more stylized maps give an overall sense of where Manhattan neighborhoods are in relation to one nother. A higher level tion can help isolate and communicate an overateh ing or mos lized idea than a less abstract version might Continua though m n 1 in the world are continuous in nature, our per ceptic h 1 to reference known “mileposts” that divide the tiny par xample, an angle moves smoothly from 0° to 180? ut we often ref ie in phases, (ic, flat, almost flat, acute, almost right any a act angle because they are more meaningfully, but naccurately, abst ure 1.25) | This idea is : the sorites or heup paradox posed by Eubulides,” a phi f Aristotle. He showed his students 4 pile of sand and cap of sand. Fubulides then asked if he removed one grin, was it still a he still agreed it was. He then asked if he continued to remove ceased ta ee ng Bile was gone, could the students identify the step at which the saad eas al te ‘entity the step at which the sand coved his position shows how arbitrary the catego- Ton Process in continuous data can be (Kosko, 1999) ns alter how we make sense of the world n we design visual nderfully illustrates" th Beneralize the actions of simple, realistic to icon: — and are important to keep Nations of continuous phenomena. Scott MeCloud his point by sho he figure being drawn 'c objective to subjective wing the degree to which people can Pon multiple continua: complex to and specific to universal, The simpler 18, S 10 Zeno, who used it ‘sed It asthe basis of his own paradox about 19, Image courtesy Scott McCloud, Understanding Comics: The invisible AnIntroduction ANOTHER 15 THE UNIVERSALITY. OF CARTON, INKGERY.— THE MORE CARTOONY A FACE 15, FOR INSTANCE, THE MORE "PEOPLE TT COULD BE SAID TO DESCRIBE. Figure 1.26 Continuum of representation detail. ie., more abstract) the representation, the more the viewer can see the drawing as more broadly generalizable, and the more detailed the representation, the mote likely the drawing will be viewed as an instance of a particular item or phenomenon (figure 1.26). Cognitive Load Theory Cognitive load theory (CLT) has emerged as one of the most Important factors in understanding, learning, and overall attention. Research suggests that memory comprised of two primary structures—short term and long tenm—both of which are controlled by a central executive. Long-term memory is where instruction goes once it is actually leamed by the user; hence, the aim ofall instruction is to alter long-term, ‘memory, but information must first pass through short-term memory (Swelle, 2005), Short-term memory is able to hold up to seven items (plus or minus two) and, equally important, is able to contrast, combine, or manipulate no more than two to four ele ‘ments at a time, This Is sometimes referted to as working memory. In addition, all the contents of working memory are lost within 20 seconds without rehearsal (Miller, 1956) ‘These limitations in short-term memory have profound effects on how people are able to understand concepts, see patterns in data, and extract relationships between, elements. To overcome this issue, people have developed strategies that connect the limited short-term memory to the larger long-term memory by grouping disparateChapter 1 a VisualSpatal Episoaic Phonological butler sketch pad = on Visual semantic =* Episodic LIM += Language Figure 1.27, he cognitive load theory (adapted from Sweller, 2005) items together into a group, called “chunking” (e.g,, telephone area codes), and imme diately connecting new data to already known information (e.g., linking to storie, schemas, and frames of reference) CLT posits that there are three separate distinct stores of short-term memory: a visual/patial sketchpad, which stores perception of two- and three-dimensional phonological loop, which stores audio and languag ‘isodic buffer, which temporarily integrates the other two (figure 1.27). Certain kinds of Information can be cross-encoded. For example, on-screen text is presented and processed visually, but it may be encoded verbally (Baddeley, 1992 One of the primary goals of information visualization is to comprehensible manner make ge-related (verbal) input; and an present information in Designers of visualization must respect the limitations and {ss ofthe affordances suggested by CLT about people's ability to accept informe tion in order to reach that goal ‘Multiple Representation/Multimedia Research and theory on the use of mul using multiple represe auditory/verbal and vi Itimedia in general sug ‘tations of information to facilitate under ‘sual channels tively independent, the limited ca targeting information delivery sim di est some benefits of standing, Because the Is by which information can be pacity of short-term memory m nultaneously to both. ia learning is that people learn more de words alone. But adding images absorbed are rela ight be increased by The basic premise of multime ly from words and sound and pictures than from just alone will not enhance learning. the potential by targeting the appropriate short term memory eyIntroduction 3m Multimedia cognition literature suggests that using both auditory and visual chan nels will result in higher content retention than using any one channel alone when performed properly. Richard Mayer (2005) outlined the following principles that describe some of the characteristics involved in multiple-channel learning: 1, Multimedia principle People learn better from words and pictures than from words alone. 2, Split attention principle People learn better when words and pictures are physically and temporally integrated. 3, Modality principle People learn better from graphics and narration than from graphics and text. 4. Redundancy principle People learn better when the same information is represented in multiple formats, 5. Segmenting, pretraining, and modality principles People learn better when informa- tion is broken into segments, when they know the names of the main concepts, and when words are spoken rather than written, 6, Coherence, signaling, and spatial/temporal contiguity principles People team better when extraneous material is excluded, when pictures and text are close to each other, and people learn better from graphics and narration, ‘These principles can help designers develop visualizations that build on the benefits of multiple representation and multimedia by adding a contextual modeling layer to “provide « powerful inscription that promotes learning conversations, can be flexibly manipu- lated, add build on existing metaphors and representations” (Gordin & Pea, 1998). Distributed Cognition Distributed cognition stands in opposition to the conventionally held notion that all thinking and understanding is performed within an individual's mind. The theory suggests that artifacts, such as paper, web pages, and tools serve as external scaffolds for internal cognition, In this view, “Google makes us smarter” by providing an environment in which individual minds are supported and carried further by the vast information provided, so that thinking becomes a joint activity working with knowl- edge both “in the head” and "in the world." ‘The ability of the computer to rapidly provide closely linked multiple representa- tions in various forms facilitates its role in enabling forms of distributed cognition. This has the result of enlisting the computer to further reduce the cognitive load requirements and increase the combined ability to understand complex information, ‘The visualization and the interaction act as a bridge between internal and external 20. An allusion to Nicholas Carr's article “Is Google Making Us Stupid?” Atlantic Magazine, August 2008.Chapter 1 5 the emergent property of that interaction (Lit, representation, with cognition Nersessian, & Stasko, 2008). Mental Models People construct mental models that abstract the facts about a specific situatio they can later be compared to new experiences and offer insight into the issues and relationships operating in both cases, A successful visualization should be able to guide the viewer to infer relationships, patterns, and alternative possibilities beyond the explicit goals of its designer, something that had not yet been considered in the “nthe mental moves are not necessaly visual o lola in nature but prove an magne representation ofthe world in which visual characteristics ae blended with concept and hye features extracted from perception «Pylyshyn, 2002) This abstraction makes mental models seul tls for problem solving, as they have ben encode withthe tempor spa, ad causal relationships of a concept, and they become a template for understanding (Rapp, 2005) The ability of mental models to internalize and generalize conceptual under: standing is the “holy grail” in education and allows people to transfer their inteinal representations about one situated experience to another. People integrate new experiences with information they already know, and that inte ation is mediated by mental models that frame previous experiences (Bransford, Brown 2000), & Cocking, Descriptive Models Descriptive models were developed to simplify some facets of organize them according to some factors that hel} simplifications allow people to com, 4 number of attempts to create taxonomies or org characterize thelr fundamental attributes are useful ways to categorize specific inst ated with other but comparable Ip describe that environment. These Pare similar situations with one another, There are nizations of the elements that and overarching q ances of the re valities. Descriptive models eal world so they can be eval: Bertn’s Storing, Communicating, and Processin The e late French cartographer Jacques Ber tion visualization elements into a comprehensive theory encom) ae : . information analysis, and Btaphical theory. pene aici His Semiotog search with aintaine communicating, and processing i effective marshaling of data through visu Eats so Of Graphics (Bertin, 1983) an empirical foundation of the rules 'd that graphs have t graphs have three primary func information tion visualization res of graphic communication, bertin im tlons—storing, the and offered guidelines for al representations,Introduction 2 Shneiderman’ Task by Data Type Taxonomy ‘Ben Shneiderman (1996) described a Task by Data Type Taxonomy in which the nature and structure of the underlying data dictate how the data should be visualized. He proposed seven basic ways to describe the structure of data relative to the way is could be visualized: 1. Onestimensional data types are nominal (named) lists of elements and are typified by documents and names, 2. Two-dimensional data types include maps, images, and other kinds of spatial displays, that utilize the height and width factors to convey information. 53. Thnee-dimensional data types represent the real-world dimensionality of three space like molecular data. 4, Multidimensional data include an arbitrary mapping of multiple related factors, each, ‘with the same number of items represented, such as census information and weather factors. 5. Temporal data have some set of values arranged by the time dimension and are often rendered as timelines and line charts. 6, Tree data are hierarchical collections of items that have some relationship to one another, such as a family tree and computer disk directories. 7. Network data are similar to tree data, where the items have a distinct relationship to one another, but the structure is not strictly hierarchical and is typified by the graphs of social networking tools such as Facebook and Twitter. ‘The idea of a task taxonomy was expanded a decade later by his colleagues, who defined low-level tasks (sort, filter, cluster, etc.), topolagy-based tusks (node relations), attibute-based tasks, browsing tasks, overview tasks, and, finally, high-level tasks (Lee, Plasant, Parr, Fekete, & Henry, 2007). Stonford’s Visualization Zoo Stanford University researchers Jeffrey Heer and colleagues described a taxonomy of visualization display types based on the nature of the underlying data, much like Shneiderman’s Task by Data-type taxonomy, with five basic categorl 1. Time series data are best represented by index charts, stacked-data graphs, small- multiple graphs, and horizon graphs. 2. Statistical distributions can take advantage of box and whisker, q-q, and stem-and- leaf and scatter plots. 3. Maps take advantage of spatial dimer and choropleth maps. 4. Hierarchies are best represented by node-link diagrams, tree graphs, and tree maps. 5, Networks show organization through force-directed graphs, arc diagrams, and matrix ns and include process maps, cartograms,Chapter 1 anize style of visualizations based on 2010) i Kk n, understanding, and evalu Presrpie models seek to provide support in the craton ny anda ons. These models provide an overarching vision of h ation of visualizations. Th P = vidual components can work together to achieve the visualization’s go providing guidance as how to actually accomplish that goal. Prescriptive m not procedures such as recipes, which are step by step and followed in sequence toward 4 goal, but provide generic guidance in the processes that can prove useful Shneiderman’s Visual Information-Seeking Mantra Interactive visualization pioneer Ben Shneiderman offered the “Visual Information: Seeking Mantra” for use as a starting point for designing, interactive visualizations, particularly those with large a mounts of data: overview frst, 20 1m and filter, then details on demand. tn his oft-cited paper first introducing the statement, he wrote the mantra 10 times in a row to underscore how valuable he and his colleay tes had found it when developing some of the earliest examples of interactive visualization The overview step, much like an establishing shot in a feature film, provides an all contextualization of the entire dataset and makes it possible to seek out overall patterns. The ability to zoom in on while filtering out extrane a subset of the data containing, items of interest ‘Ou features allows focus on a p. articular subject and is ampli fied by the ability to show additional details on demand (Shneiderman, 1996) Van Ham and Perer’s Search and Expand Sometimes the amount of data being explored is too large or too varied to mak overview of the data meaningful or even tect sabi on demand model stands in contrast to Shneiderman 1 Mantra by reducing the dataset before an overs This is done by initiating the process with, inethod to create a more manageable and porte called a desree of interest. This subser context, and the nically possible. The search, show context, ‘sual Information-Secking lew is provided a search or some other user interaction tially meaningful subset of the d * can be presented a ® to expand the context an on intere user Is then fre« an overview to pr Subset in more detail depending \d/or further explore the sts (van Ham & Perer, 2009) Chi’ Data State Reference Xerox Pare researcher Ed the visuz h ‘alization process, beginning w Reference mod takes a data-centric view of h the raw data hose data pass through threeIntroduction 35 stages in a linear pipeline: analytical abstraction (generating metadata: data about data), visualization abstraction (extraction of visualizable elements), and finally the view (end product of mappings). Each stage is ushered to the next phase by transformation opera- ‘ors, which transform the data, the visualization of the data, and the visual mapping of the data. This model is useful by creating a taxonomy in which a wide variety of visualizations can be evaluated together (Chi, 2000), Spence’s Selection, Encoding, and Presentation Citing Marcel Proust as the inspiration for his model, “The real vopage of discovery consists: not in seeking new landscapes but in having new eyes,” Robert Spence (2001) outlined a progression, in which raw data pass through Selection, encoding, and final through presentation on the way to the viewer, who then has the ability to manipulate those three steps to make meaning. This changes the locus of control from solely the visualization’s designer to a more collaborative relationship among the data, the designer, and the viewer. ry’ Acquire, Parse, Filter, Mine, Represent, and Interact Ben Fry, developer of the popular Processing” visualization programming language, described a framework for visualization that starts with the collection of data with the goal of answering a question using that data. The procedure to answer the question follows a seven-step sequence: 1. Acquiring the data initiates the process, where raw data or information is obtained in a digital form and made available locally or through an Intemet connection, stuch as US Census data, 2, Parsing is the process of structuring the raw data from its native format into some- thing that can identify the constituent parts, such as tagging or structuring data ele- ments into discrete database fields or spreadsheet columns. 3, Filtering the data removes portions of the structured dataset that are not of interest. This filtering-out process can be done across multiple dimensions. For example, we ‘may only want the census income data for the married men in California from 1920 to 1950, 4. Mining the filtered data will be useful in finding patterns that emerge using statisti cal methods, such as correlation and regression. For example, we may want to compare incomes of men and women using the census data and see if that correlation changes over time, 5. Representing the data visually makes it possible to communicate the patterns found in the mining step by using tables, graphs, network diagrams, and a host of other visualization techniques. 21, [Link]Chapter 1 36 aging by iteratively revisiting the 6, Refining that representation to by Herativey iceman poe night of sgh one By teres ier wth te vswalation adds options to manipulate the reeset i cl ow and what data are being represented, making it easy allowing the user to change h see making ay for them t© discover new rationships or better understand th being presented erting raw data into an interactive natic sequence for conver ‘This model is a pra ; : 7 visualization and has been evidenced in Fry's many excellent visu, Even though it is not explicitly mentioned by name as a step, he believes that all visualizations should begin with a guiding question (Fry, 2004) Roam’s Back of the Napkin Visual thinking consultant and author Dan Roam has developed a framework for solving business problems using drawn images that is a useful model for framing the visualization process as well. The Back of the Napkin model refers to simple drawings to describe ideas, often made on the back of a restaurant napkin. It looks at problems through an iterative loop of looking, seeing, imagining. and showing 1. Look ‘The process of gathering and selecting information with an open attitude by gathering data relevant to the problem, Supports observation and reflection, establishing the dimensions to explore the data by (Le, who, what, when, etc.), and finally classifying the information to determine What parts are important to the problem at hand. 2, See Processing the information relevant to the problem to refine its definition scope into the six ways to see: whovhat, how much, when a. Who and what help define the objects of our inquiry by descriptions of the items in our topic deploying that information in a manner that when, how, and why offering both names and. and their observable characteristics b. How much helps quantify those characte ‘can be compared with one another and hav ©. Where helps situate the objects in the each other tistics into numeric terms so the objects we st world to help expose part-to-whole rel 4. When provides the critical temporal con, ships both among the objects themselve. €. How explores the atistical techniques applied to them. to provide external context and with lationships between them, text that can help, and with the fa lationships expose ¢ sal relation: eT World they exist in and is lore explan, WS Of seeing. How can be thought of how = who/what + how ms rs entry portal to begin asking questo other five ways. Why asks us to synthesi j roe eed om it inthe larger context of our topic cause-and-effect rel tive than Roam’s first four proposed war of the “Four W's" (ie, £. Why provides the by the tory and dedluc s the sum, the information cottected and reflectIntroduction 7 3. Imagine Identifying patterns and components in the data identified and envisag- Ing ways to convert them into graphical representation, This is done by rating them on a set of five continua, called SQVID (unfortunately pronounced squid): a Simple or elaborated: The degree in which the data being represented requires a complex or simple solution, The ultimate goal is to make the complex understandable, not just simpler. The objective is to provide the proper amount of complexity that properly represents the problem without overwhelming the audience. . Quality or quantity: The degree to which the problem is qualitative or quantitative or both, Some problems lend themselves to one or the other, but the richest descrip tions of human endeavors will typically contain both methodologies. «, Vision or execution: The degree to which the problem is represented prescriptively or descriptively 4, Individual or compared: The degree to which individuals or groups will best repre- sent the problem. . Delta or absolute: The degree to which differences in the data are reflected as change in relation to themselves over time or with others, versus concentrating on the actual values of the underlying data. 4, Show ‘The process of choosing an appropriate technique of representing the data/ problem, graphically rendering that technique, and presenting the results for others. This model is not a formal model in the academic sense but, nonetheless is a useful lens to look at visualization problems through and provides a systematic methodology for generating and evaluating projects (Roam, 2008. Elements of User Experience Userexperience designer Jesse James Garrett (2003) has proposed Elements of User Experience, a model with a user-centered perspective for developing Web-based proj- ects, The model has five “planes” ranging from the more abstract Strategy to the more concrete Surface (igure 1.28). 1. Strategy Where user needs and objectives are defined. This is an explicit explora- tion of the specific goals to be achieved by the project from both the designer's and the user's perspective, hopefully with some measure of success. The users are carefully profiled, and a strategy is developed to satisfy them. 2. Scope Where functional specification and content requirements are set The strat- egy from the first plane is translated into specific features and functionality to meet the user's expectations, Decisions are made on what to include and what to ignore in terms of functionality and content. 3, Structure Where the interaction design and information architecture are accom- plished. The disparate pieces of content and functionality need to be merged into aChapter 1 and 'ssues, navigation design informing “where” issue tnformation design presiding over the "what design is performed, This functionalit tion of the prev Where the visual the culminati ; the end user us planes where conten interaction, and aesthetics mee Gatrett’s model has bee ‘dely adopted in the seucrience communities and is equally spplican Ch nd interactive raphics design The ASSERT Model We have developed leations that are the ASSERT mode 1 (figure 1.29 accessibleIntroduction 39 ‘Ask a question ‘Search for information ‘Structure the data Envision the answer Represent the visualization Tell a story Figure 1.29 ‘The ASSERT model.Chapter 1 volved over a number of traditions of historical and social science research. It has evolved yeas in the context of developing historical visualzations at he Vigna Cente fr Digital History (VCH) for museums, historians, and civic organizations and teaching, raduate history seminars where creating historical visualizations were he model has been successfully used in a heh deliverable ofthe couse. Finally anothe? ASSERT symhests important has inteduced by these models and extends the soe, The other mols play addres subsets of erica ses but hd nt is press, ASSERT addreses the fll Scope development In a. pocessden jons, Unlike In the course of creating interactive visualization in the humanities, it became clear that a more holistic perspective would be useful. One that embraced the long-held inquiry-driven research traditions common to the sciences, humanities, and the social sciences; one that could scaffold the process from idea to execution in an iterative fashion; one that was able to freely cross between theoretical and practical bounds in its execution; and finally, a model that included the narrative clement so crucial to create a compelling communicative experience The model has six compone! nts: Ask a question, Search for evidence to answer that Question, Structure that information to answer the question, Envision ways to answer the question using available data, Represent the dat, answer the question, and, finally, Tell the question. It provides a scaffold for Questions through visual representation, a in a meaningful visualization to @ meaningful story using that data to answer F scholars and educators to pose meaningful n and intera tion, 1. Ask a question. Good visualizations ask specific questions about the topic that can be answered ty data. In addition, questions should be able to if you answered them and who the intended 2. Search for evidence to answer the are driven by good questions. The goal Is to accessible and reliable address why you or anyone would care question. For ¢ bef exploced it Important to fd good cane auton and be amare ofeach sees oe innaig? tobe abstated and structed eed to be wansformed pred sn ‘order to be useful as a data source in te deel way of answering ee ‘ach of the questions that are being of information that could nswer that ngths and limit ations: Is it accessible? Is it the question, Information in its + prder to tell 4 compelling story. The data may sampled, or have st @ visualization, n Using the data, stions identified tistical methods applied in This is an exploratory process sing the d ta found and strucIntroduction a tured in the earlier design phases. It involves looking at the relationships in the data to the research question and within parts of the data 5. Represent the data in a compelling visualization. ‘The visible product of the entire visualization process is its representation and is where users will interact with the infor- ‘mation presented. Creating that representation is a careful blend of science, at, and display technology. 6, Tell a meaningful story using the evidence to answer the question. Storytelling is one of the primary ways we make sense of the world. A good visualization answers a ques- tion using primary source evidence to tell a story. That story should be meaningful and able to address the significance criteria (the "so what?” ‘The model offers a prescriptive framework to apply traditional research methodol- ogy used in many disciplines for developing interactive visualizations. The steps and sequence need not be slavishly followed. As in all creative endeavors, there are mul- tiple paths to a successful project, but the framework provides an overarching scaffold for the process, Most steps benefit from iteration to previous steps as more informa- tion is uncovered and other options are further explored. ASSERT is a holistic model in which all of the constituent parts relate to the whole and, in a perfect world, would be considered together. The reality of our humanity forces us to work serially on what is in truth a parallel problem. The model helps scaffold this process. As designers work longer with visualization projects, the indi- vidual steps will begin to join, ones that work well in particular situations will emerge, and new strategies will be adopted as needed. The ASSERT model provides a useful starting point to structure the process, Summary ‘This chapter explores some definitions of what interactive visualization is, such as Stewart Card's “The use of computer-supported, interactive, visual representations of abstract «data to amplify cognition,” and why it isa valuable addition to just analyzing numeric data, as illustrated by Anscombe’S Quartet. There are several gentes of visualizations created to serve different groups and purposes from scientists to journalists, People have been using graphics to visualize information from the time of Leonardo a Vinci and before. Willlam Playfair introduced the world to pie and bar charts in the mid-eighteenth century, John Snow's cholera map effectively used mapping to persuade people toward the germ theory in the mid-nineteenth century, and Harry Beck's iconic subway map helped the use of maps as geometric as opposed to only geographic displays. The computer has played a big role in the evolution of more sophisticated graphics generated by sources ranging from the video artist John Whitney to the spacecraft animations of the Jet Propulsion Laboratory's Jim Blinn and highly realisticChapter 1 L tions by Evans and Sutherland. The Internet has brought a wealth of freely ning with the University of Maryland!’s Human Computer Interaction Laboratory work in interaction usability ani ay of large data sets, the University of Viginia’s Institute for the Advancement of the Humanities work in pioneering the digital erareanes of USA Today and the New York ‘Times have advanced the use of visual journalism and the development of a new genre of information display, the infogeapl A number of theoretical models provide a good foundation for interactive visualiza- tions. Theoretical 1 uch as the issues of abstraction, the cognitive load theory and multime¢ fer useful guidance. Thete are many descriptive and prescrip tive model nthe prescriptive ASSERT model developed by the autho The ASSERT mod ought of more as a web or a circle with links than straight lin s a i a nd assumes that iteration between phases will be common, Starting at any stage can be useful, but ultimately, a clear question to ask >petully answer) ing chapt eating interactive visualizations through the lens of the ASS 1 m steps that define it and exploring the various issues