Lecture 4b – Data Preparation

for Visualization
Course Instructor: Faiza Qamar

Department of Creative Technologies (CT)

Faculty of Computing and Artificial Intelligence (FCAI)
Air University, Islamabad
1
 Raw Data
• Raw data is the initial state of data you have collected, received or
downloaded that has not yet been subjected to any statistical or trans
forming treatment.

2
 Data Acquisition
• There are several distinct origins and methods involved in acquiring data.

• Sometimes you have to do most of the work yourself.

• Other times people make it available for you to access in different ways.

• In each of these cases you need to be assured about the reliability of the
data you are gathering.

3
 Data Acquisition
• When describing the importance of ‘trustworthiness’, there may be
collection issues creating inaccuracies and biases that can affect the quality
of your data at source.
• Example: If a survey about healthcare access only collects responses from urban
hospitals, rural populations are underrepresented.

• You need to be discerning in the degree of trust you place in it, at least to
begin with, until you have a chance to examine it more closely.

4
 Data Examination
• Once you have acquired your data – whether this is all of it or just a starting
point – the second step is to examine it thoroughly.

• Examining your data specifically involves learning about the types of data
you have, the size and range of values held, and its condition.

5
 Data Types
• The five basic data types

6
 Data Types
• An item is an individual entity that is discrete, such as a row in a simple table
or a node in a network

7
 Data Types
• An attribute is some specific property that can be measured, observed, or
logged.
• For example, attributes could be salary, price, number of sales, protein
expression levels, or temperature.

8
 Data Types
• A link is a relationship between items, typically within a network.

9
 Data Types
• A position is spatial data, providing a location in two-dimensional (2D) or
three-dimensional (3D) space.
• For example, a position might be a latitude–longitude pair describing a
location on the Earth’s surface.
• Or three numbers specifying a location within the region of space measured
by a medical scanner.

10
 Data Types
• A grid specifies the strategy for sampling continuous data in terms of both
geometric and topological relationships between its cells.

Topological relationships (connectivity):

which cells are neighbors and how they
connect.

11
 Dataset Types
• A dataset is any collection of information that is the target of analysis.
• The four basic dataset types are tables, networks, fields, and geometry.

• Other ways to group items together include clusters, sets, and lists.

• In real-world situations, complex combinations of these basic types are

common.

12
 Dataset Types
• These datasets are made up of five core data types: items, attributes, links,
positions, and grids.

13
Dataset Types

14
 Dataset Types – Tables
• Tables have cells indexed by items and attributes,
for either the simple flat case or the more complex
multidimensional case.

• A multidimensional table has a more complex

structure for indexing into a cell, with multiple keys.

15
 Dataset Types - Tables
• Each row represents an
item of data, and each
column is an attribute of
the dataset.
• Each cell in the table is
fully specified by the
combination of a row and
a column—an item and an
attribute—and contains a
value for that pair.

CS 366 – Data Visualization 16

 Dataset Types - Networks
• The dataset type of networks is well suited for
specifying that there is some kind of relationship
between two or more items.

• An item in a network is often called a node (vertex).

• A link (edge) is a relation between two items.

17
 Dataset Types - Networks
• In an articulated social network the
nodes are people, and links mean
friendship.
• In a gene interaction network, the
nodes are genes, and links between
them mean that these genes have
been observed to interact with each
other.

18
 Dataset Types - Networks
• Network nodes can have associated attributes, just
like items in a table.
• In addition, the links themselves could also be
considered to have attributes associated with them;
these maybe partly or wholly disjoint from the node
attributes.
• For example: Social Networks:
• Nodes (people): attributes like age, gender, location,
interests.
• Links (friendship, connection): attributes like strength of
relationship, frequency of interaction, type of relationship
(family, coworker, acquaintance).

19
 Dataset Types - Networks
• Networks with hierarchical structure are more
specifically called trees.

• In contrast to a general network, trees do not have

cycles:
• each child node has only one parent node pointing to it.

• One example of a tree is the organization chart of a

company, showing who reports to whom.

20
 Dataset Types – Fields
• The field dataset type also contains
attribute values associated with
cells.

• A field dataset represents data that

varies continuously across space
(2D/3D) or time.

• Continuous phenomena that might

be measured in the physical world
or simulated in software include
temperature, pressure, speed,
force, and density; mathematical
functions can also be continuous.
21
 Dataset Types – Geometry
• The geometry dataset type specifies
information about the shape of items with
explicit spatial positions.

• The items could be points, or one-

dimensional lines or curves, or 2D surfaces
or regions, or 3D volumes.

• Geometry datasets do not necessarily have

attributes, in contrast to the other three
basic dataset types.

22
 Dataset Types – Other Combinations
• A set is simply an un ordered group of items.

• A group of items with a specified ordering could be called a list.

• A cluster is a grouping based on attribute similarity, where items within a

cluster are more similar to each other than to ones in another cluster.

23
 Dataset Availability
• Dataset availability can be either static or dynamic, for any dataset type.
• The default approach to vis assumes that the entire dataset is available all at
once, as a static file.

• However, some datasets are instead dynamic streams, where the dataset
information trickles in over the course of the vis session.
• One kind of dynamic change is to add new items or delete previous items.
Another is to change the values of existing items.

24
Attribute Types

25
 Data Size: Amount and Range
• Once you have established an understanding of the different types of data,
you can switch your examination towards the shape and size of this data.
• Looking at the quantitative attributes across all variables and for all items.

• The main questions you will ask of your data include:

• For quantitative variables, what is the lowest and the highest value in each case?
• In what format are the numeric values presented (i.e. how many decimal places or
comma-formatted)?
• For a categorical variable (nominal or ordinal), how many different values are held?
• If you have textual data, what is the maximum and minimum character length or word
count

26
 Data Condition: Quality and Representativeness
• You also need to examine the condition of your data.

• Visualization tools expect structured, clean data.

• Undiscovered and unresolved issues around the quality of your data will
undermine the trust in and the accuracy of your work.

27
 Features To Look Out For
• Missing values: Are empty cells assumed to be of no value (zero/nothing) or
no measurement (n/a, null)?
• This is a subtle but important difference.

Source: Missing Values. Popular techniques to handle it with code examples

28
 Features To Look Out For
• Erroneous values: Typos and any values that clearly look out of place (such
as a gender value in an ‘age’ column).

Source: Detect Data Entry Errors (and Impute Missing Values) in any Tabular Dataset
29
 Features To Look Out For
• Inconsistencies: Capitalization,
units of measurement, value
formatting.

Source: Reducing Inconsistency in Integrating Data from Different Sources

30
 Features To Look Out For
• Duplicate records.

31
 Features To Look Out For
• Expired values: Values that might have elapsed in their current relevance or
accuracy, like someone’s age or any statistic that would be expected to have
subsequently changed.

• Uncommon system characters or line breaks.

• Leading or trailing spaces: A subtle but particularly evil issue!

• Date issues around format (dd/mm/yy or mm/dd/yy)

32
 Tools for Data Preparation
• Excel/Google Sheets basics
• Easy entry-level tools.
• Features: filtering, sorting, conditional formatting, pivot tables.
• Pros: simple | Cons: limited automation

• Python (pandas)
• Popular for data manipulation.
• Can:
• Clean missing data.
• Convert formats.
• Merge/join datasets.

• R (tidyverse)

CS 366 – Data Visualization 33

 References
• Tamara Munzner - Visualization Analysis and Design (2014)
• Chapter 2
• Data Visualization: A Handbook for Data Driven Design 2nd Edition by Andy
Kirk
• Chapter 4

CS 366 – Data Visualization 34