Basic Data Analysis Using R

O.J. Akintande, PhD

Lecturer & Researcher,
Computational Unit,
Department of Statistics,
University of Ibadan
Outline
1. Introduction to R and RStudio
2. Data Manipulation (filtering, sorting, recoding
and computing new variables)
3. Generating Summary Statistics, Frequency
Tables and Cross Tables
4. Introduction to Data Visualization Using ggplot2
Expected Learning Outcomes
At the end of the course, you will be able to:
● Master R and RStudio environments and navigate easily
● Implement data preprocessing tasks necessary for data
mining, machine learning and other data science tasks.
● Produce quick summary statistics and tables to draw
insights from any dataset
● Produce elegant graphics from datasets to understand
trends and patterns
Why R and RStudio?
 Why RStudio?
● RStudio allows you to
- build scripts
- compile codes
- create plots
- work with various datasets
- create projects for data storage.
● RStudio interface is divided according to your work, into the console, the file
viewer, workspace manager and plots area.
● Effective Integrated Development Environment (IDE) for R language: The
program allows you to author HTML, PDF, documents and slideshows.

[Source: [Link] ]
R & RStudio Interface
 Q3 - Pane Q2 - Pane
★ Source ★ Environment
★ History
★ Connections

Q4 - Pane
★ Console Q1 - Pane
★ Terminal ★ Files
★ Plots
★ Packages
★ Help
★ Viewer
Q1 - Files Pane
1. Deletefiles and folders
2. Createnew folders
3. Rename folders
4. Folder navigation
5. Copy or move files
6. Setworking directory or go to working directory
7. View files
8. Import datasets
Q1 - Plots Pane
1. To view output of
visualizations produced when
typing code into the Console
window or running a script.

Note: Plots can be created

using a variety of different
packages
Q1 - Packages Pane
The Packages pane displays all currently
installed packages alongwith a brief
description and version number for the
package.
○ Packages can also be
removed using the x icon to the
right of the version number
for the package.

○ Clicking on the package name

will display the help file for the
package in the Help tab.

○ Clicking on the check box to

the left of the package
name loads the library so that it
can be used when writing
code in the Console window.
Q1 - Help Pane

Displays linked help documentation for any

packages that you have installed.
Q1 - Viewer Pane

1. To view local web content

2. Can’t be used to view online web
content
Q2 - Environment Pane
● Contains a listing of variables that you have created
for the current session.
➔ Each variable is listed in the tab and can be expanded
to view the contents of the variable.
➔ The rectangle surrounding the variable displays the
columns for the variable.
➔ Clicking the table icon on the far-right
side of the display will open the data in a tabular
viewer.
● Other functionality includes
➔ Opening or saving a workspace
➔ Importing dataset from text files, Excel spreadsheets,
and various statistical package formats.
➔ You can also clear the current workspace.
Q2 - History Pane
❏ Displays a list of all commands that have been
executed in the current session. This tab includes a
number of useful functions including the ability
● to save these commands to a file
● to load historical commands from an
existing file.
● to select specific commands from the
History tab and send them directly to
the console or an open
script.
● to remove items from the History
pane.
Q2 - Connections Pane
❏ Can be used to access existing or create new connections to ODBC
and Spark data sources.
Q3 - Source Pane
❏ Used to create scripts, and display datasets
➔ An R script is simply a text file containing a series of commands that are executed together.
➔ Commands can also be written line by line from the Console pane as well.
➔ When written from the Console pane, each line of code is executed when you click the Enter (Return)
key.
➔ Scripts are executed as a group.
➔ Multiple scripts can be open at the same time with each script occupying a separate tab.
➔ RStudio provides the ability to execute the entire script, only the current line or a highlighted group
of lines.
❏ Used to display datasets.
Q4 - Console Pane
❏ Used to interactively write and run lines of code.
➔ Each time you enter a line of code and click Enter (Return) it will
executethat line of code.
➔ Any warning or error messages will be displayed in the Console
window as well as output from print() statements.
Q4 - Terminal Pane
❏ It provides access to the system shell from within the
RStudio IDE.
❏ It supports xterm emulation, enabling use of
full-screen terminal applications (e.g. text editors,
terminal multiplexers) as well as regular
command-line operations with line editing and shell
history.
➔ There aremany potential uses of the shell
including advanced source control operations,
execution of long-running jobs,remote logins,
and system administration of RStudio.
➔ The Terminal pane is unlike most of the other
features found in RStudio in that it’s capabilities
are platform specific.
➔ In general, these differences can be
categorized as either Windows capabilities or
other (Mac, Linux, RStudio Server).
Customizing the
RStudio Interface
If you don’t like the default RStudio interface, you can customize the appearance. To do so, go
to Tool | Global Options... (RStudio | Preferences on a Mac).
Working Directory
 Working Directory

1. To check your current working directory: type

getwd()
in the console

2. To change the working directory

● Use setwd() function:

setwd("C:/Users/HP/OneDrive - University of Ibadan Laboratory for

Interdisciplinary Statistical Analysis/Trainings/Udemy Courses/Basic
Statistics/Statistics with R")
Working
Directory
Select:
Session | Set Working
Directory | Choose
Directory
 Working
Directory
From within the Files
pane, use the More |
Set As Working
Directory menu
 Reading Data from a
comma-separated values
(csv) File
❖ demo <- [Link]("csv-data-frames/[Link]")
❖ View(demo)
❖ head(demo)
Data Manipulation in R
(filtering, sorting, recoding and computing new variables)
### how to filter (select) your data in base R using brackets
### a new data frame, demo2, will be created each time one filter variable
## select the female subjects
❖ demo2 <- demo[demo$gender == "Female",]
### retain the subjects with the income greater than 100
❖ demo2 <- demo[demo$income > 100,]
❖ View(demo2)
### if you want to keep only the variables 1, 3 and 7
### (age, income and gender)
❖ demo2 <- demo[demo$income > 100, c(1,3,7)]
❖ View(demo2)
### if you want to drop variables 6, 7 and 8
### (car category, gender and retired)
❖ demo2 <- demo[demo$income > 100, -c(6:8)]
❖ View(demo2)
❖ ### two or more filter variables
❖ ### select the female subjects with the income over 100
❖ demo2<-demo[demo$gender=="Female"&demo$income > 100,]
❖ View(demo2)
❖ ### how to filter (select) your data in base R using subsets
❖ ### a new data frame, demo2, will be created each time
❖ ### keep the married subjects only
❖ ### (one filter variable)
❖ demo2 <- subset(demo, marital == "Married")
❖ View(demo2)
❖ ### retain the married subjects aged over 35
❖ ### (two filter variables)
❖ demo2 <- subset(demo, marital == "Married" & age > 35)
❖ View(demo2)
❖ ### keep the first three variables only
❖ ### (age, marital status, income)
❖ demo2 <- subset(demo, marital == "Married" & age > 35, select = c(1:3))
❖ View(demo2)
❖ ### drop variables 4, 5, 6 and 8
❖ ### (education, car price, car category, retired)
❖ demo2 <- subset(demo, marital == "Married" & age > 35, select = -c(4:6, 8))
❖ View(demo2)
 Filtering with Dplyr - A Grammar of Data Manipulation

❖ require(dplyr)
### keep the unmarried subjects only
### (one filter variable)
❖ demo2 <- filter(demo, marital == "Unmarried")
❖ View(demo2)
### keep the unmarried subjects only aged under 50
### (two filter variables)
❖ demo2 <- filter(demo, marital == "Unmarried", age < 50)
❖ View(demo2)
### if you want to keep some variables only,
### you must first specify the variables you want to keep
### suppose we want to keep only the first three variables
### (age, marital status, income)
❖ demo2 <- select(demo, age, marital, income)
❖ View(demo2)
### next we filter our new data frame demo2,
### keeping only the unmarried persons aged under 50
❖ demo2 <- filter(demo2, marital == "Unmarried", age < 50)
❖ View(demo2)
Recode Categorical (Factor) Variables
### how to recode the categorical (factor) variables
### we want to convert the variable gender as follows
### Male = 1, Female = 2
### a new variable gender2 will be created
### first we will use the brackets (base R)
❖ demo$gender2[demo$gender == "Male"] = "1"
❖ demo$gender2[demo$gender == "Female"] = "2"
❖ View (demo)
##########
### we can do the same type of recoding with the plyr package, function revalue
### load the package
❖ require(plyr)
### let's create a new variable, gender3
❖ demo$gender3 = revalue(demo$gender, c("Male"="1", "Female"="2"))
❖ View(demo)
### important: if the variable to recode is not a factor
### we must convert it into a factor before recoding
❖ demo$gender = factor(demo$gender)
### to recode into the same variable (without creating a new one)
### we just use the same variable name in both sides of the revalue function
❖ demo$gender = revalue(demo$gender, c("Male"="1", "Female"="2"))
❖ View(demo)
Recode Continuous Variables
### how to recode a continuous variable into a factor
### we want to create a categorical variable as follows
### subjects with income under 200 - low income
### subjects with income of 200 and more - high income
### a new variable, incat (income category), will be created
❖ demo$incat[demo$income<200] = "Low income"
❖ demo$incat[demo$income>=200] = "High income"
❖ View(demo)
### now we want to create three groups by income
### low income - under 150
### medium income - between 150 and 300
### high income - 300 and more
### so we will have two cut points: 150 and 300
### a new variable, incat2, will be created
❖ demo$incat2 = cut(demo$income, breaks=c(-Inf, 150, 300, Inf), labels=c("Low income", "Medium
income", "High income"))
❖ View(demo)
### by default, the ranges are open on the left, and closed on the right
### namely (-Inf,150], (150, 300] and (300, Inf)
### to get it conversely, use the option right=FALSE
❖ demo$incat2 = cut(demo$income, breaks=c(-Inf, 150, 300, Inf), labels=c("Low income", "Medium
income", "High income"), right = FALSE)
❖ view(demo)
Sort a Data Variable
### how to sort a data frame
### a new data frame, demo2, will be created each time
### sort by income, ascending (default)
❖ demo2 <- demo[order(demo$income),]
❖ View(demo2)
### sort by income, descending
❖ demo2 <- demo[order(-demo$income),]
❖ View(demo2)
### sort by income and age
❖ demo2 <- demo[order(demo$income, demo$age),]
❖ View(demo2)
### sort by income (ascending) and age (descending)
❖ demo2 <- demo[order(demo$income, -demo$age),]
❖ View(demo2)
Compute a New Variable
❖ math <- [Link]("csv-data-frames/[Link]")
❖ View(math)
### how to compute a new variable
### we will create a variable that stores the difference between the two grades
❖ math$diff = math$grade2 - math$grade1
### another variable that stores the average of the two grades
❖ math$avg = (math$grade1 + math$grade2) / 2
❖ head(math)
Compute Statistical Indicators for Numeric
Variable
### how to compute the main statistical indicators
### for a numeric variable in base R
##########
❖ demo <- [Link]("csv-data-frames/[Link]")
❖ View(demo)
####### we will compute these indicators for the variable income
### mean
❖ mean(demo$income)
### or
❖ m <- mean(demo$income)
❖ print(m)
### standard deviation and variance
❖ sd(demo$income)
❖ var(demo$income)
### minimum, maximum and range
❖ min(demo$income)
❖ max(demo$income)
❖ range(demo$income)
❖ max(demo$income) - min(demo$income)
### median
❖ median(demo$income)
### quartiles
❖ quantile(demo$income)
Compute Statistical Indicators for Numeric
Variable with psych Package
### how to compute the main statistical indicators for a
numeric variable
### with the psych package
##########
###### we will compute this indicators for the following
variables
###### age, income and car price
### create a matrix with the variables of interest
❖ demo2 <- cbind(demo$age, demo$income,
demo$carpr)
### give suggestive names to the matrix columns
❖ colnames(demo2) <- c("age", "income", "price")
❖ View(demo2)
### load the psych package
❖ require(psych)
### use the describe function to generate the statistics table
❖ describe(demo2)
### the trimmed mean is computed with a default trim of 0.1
### mad - median absolute deviation (the median of the
absolute deviations from the data median)
######## more options for the describe function
❖ describe(demo2, [Link] = TRUE, trim = 0.1, check = TRUE)
### [Link] - if TRUE it omits the missing values (if FALSE it
deletes the cases)
### trim - sets the trimming fraction
### check - if TRUE it checks for non-numeric data
Compute Statistical Indicators for Numeric
Variable with pastecs Package
### how to compute the main statistical indicators for a numeric variable
### with the pastecs package
###### we will compute this indicators for the following variables
###### age, income and car price
### create a matrix with the variables of interest
❖ demo2 <- cbind(demo$age, demo$income, demo$carpr)
### give suggestive names to the matrix columns
❖ colnames(demo2) <- c("age", "income", "carpr")
❖ View(demo2)
### load the pastecs package
❖ require(pastecs)
### before computing the indicators we set some options (in base R)
options(scipen=100)
## force R to use the standard notation, NOT the exponential notation
options(digits=2)
## make R show only the first two decimals
### run the [Link] function from pastecs
### if we want ALL the statistics we run
❖ [Link](demo2)
### if we want to omit the basic statistics we run
❖ [Link](demo2, basic = FALSE)
### if we want the basic statistics only we can execute
❖ [Link](demo2, desc = FALSE)
 Descriptive Statistics
### we will compute these indicators for the variable income
### mean
❖ mean(demo$income)
### or
❖ m <- mean(demo$income)
❖ print(m)
### standard deviation and variance
❖ var(demo$income)
### minimum, maximum and range
❖ min(demo$income)
❖ max(demo$income)
❖ range(demo$income)
❖ max(demo$income) - min(demo$income)
### median
❖ median(demo$income)

### quartiles
❖ quantile(demo$income)
 Determining the Skewness and Kurtosis
### we will use the variable
income for our examples
### load the package

❖ require(e1071)
### compute the skewness
❖ skewness(demo$income)
### compute the kurtosis
❖ kurtosis(demo$income)
 Computing Quantiles
### we will use the variable income for our example
### compute the following percentiles 17%, 55% and 97%
### use the quantile function in the stats package
### (this package loads automatically when you start R)

❖ quantile(demo$income, probs = c(0.17, 0.55, 0.97))

### to get the quartiles

❖ quantile(demo$income, probs = c(0.25, 0.50, 0.75))

 Determining the Mode
### we will find out the mode for the variable income
### load the package
❖ require(modeest)
❖ mlv(demo$income, method="mfv")

### "mfv" stands for "most frequent value"

### for the discrete variables, the best way to compute
### the mode is to tabulate the frequencies
### as we will learn in a future lecture of this course
 Creating Frequency Tables and Cross
Tables
### we will build a table for the variable educ (education level)
### this table will contain the following:
### absolute frequencies (counts), cumulative absolute frequencies,
### relative frequencies and cumulative relative frequencies
### create the initial table (with the counts only)

❖ mytable <- table(demo$educ, exclude = NULL)

### the missing values will be excluded

print(mytable)
### compute the cumulative counts (using the cumsum function)
❖ cumul <- cumsum(mytable)
❖ print(cumul)
### compute the relative frequencies
❖ relative <- [Link](mytable)
❖ print(relative)
### compute the cumulative relative frequencies
❖ n <- nrow(demo)
### number of rows (cases) of the data frame demo
❖ cumulfreq <- cumul/n
❖ print(cumulfreq)
### create the final table with the cbind function
❖ mytable2 <- cbind(Freq=mytable, Cumul=cumul, Relative=relative,
CumFreq=cumulfreq)
❖ print(mytable2)
 Building Cross Tables using xtabs
### we will build a cross table with the variables gender and carcat (car category)
### load the package
❖ require(gmodels)
❖ CrossTable(demo$gender, demo$carcat, [Link] = FALSE)
### we don't want the chi square contributions
### some other options of the CrossTable function

❖ CrossTable(demo$gender, demo$carcat, digits=3, expected=TRUE,

prop.r=TRUE, prop.c=TRUE,prop.t=TRUE, [Link]=TRUE, chisq =
FALSE, fisher=FALSE, mcnemar=FALSE, [Link]=FALSE)
Introduction to Data Visualization Using ggplot2
 Welcome to ggplot2
❏ R package for producing statistical, or data, graphics with a deep
underlying grammar

❏ Grammar of Graphics
- Independent components that can be composed in many
different ways
- You can create new graphics that are precisely tailored to
your problem

❏ Provides beautiful, hassle-free plots

❏ Designed to work iteratively

❏ Can produce a publication-quality graphic in seconds

 Components of ggplot2
❏ According to ggplot2 concept, a plot can be divided into different fundamental parts:
Plot = data + Aesthetics + Geometry.
❏ The principal components of every plot can be defined as follow:
● data is a data frame
● Aesthetics is used to indicate x and y variables. It can also be used to control the color, the
size or the shape of points, the height of bars, etc…..
● Geometry corresponds to the type of graphics (histogram, box plot, line plot, density plot,
dot plot, ….)
❏ Two main functions, for creating plots, are available in ggplot2 package : a qplot() and ggplot()
functions.
● qplot() is a quick plot function which is easy to use for simple plots.
● The ggplot() function is more flexible and robust than qplot for building a plot piece by piece.
❏
 Types of Graphs for Data Visualization
❏ The ggplot2 package provides methods for visualizing the
following data structures:
1. One variable - x: continuous or discrete
2. Two variables - x & y: continuous and/or discrete
3. Continuous bivariate distribution - x & y (both continuous)
4. Continuous function
5. Error bar
6. Maps
7. Three variables
Data Format and Preparation
 qplot(): Quick Plot with ggplot2
The qplot() function is very similar to the standard R plot() function. It can be used to create
quickly and easily different types of graphs: scatter plots, box plots, violin plots, histogram
and density plots.
qplot(x, y = NULL, data, geom="auto")
qplot(): Quick Plot with ggplot2
qplot(): Quick Plot with ggplot2
Box Plot, Violin Plot and Dot Plot
Box Plot
Violin Plot
Dot Plot
Histogram
Density Plots
Density Plots
ggplot(): Build Plot Piece By Piece
ggplot(): Build Plot Piece By Piece
ggplot(): Build Plot Piece By Piece
ggplot(): Build Plot Piece By Piece
One Variable: Continuous
One Variable: Continuous
One Variable: Continuous
One Variable: Continuous
One Variable: Discrete
Two Variables: Cont. Y, Cont. X
Two Variables: Cont. Y, Cont. X
Two Variables: Cont. Y, Cont. X
Two Variables: Cont. Y, Cont. X