Module-2 R
Module-2 R
To read data into R, you can use several functions depending on the format of the data.
Below are some common examples:
R CSV Files
A Comma-Separated Values (CSV) file is a plain text file which contains a list of data.
These files are often used for the exchange of data between different applications. For
example, databases and contact managers mostly support CSV files.
Storing data in excel spreadsheets is the most common way for data storing, which is
used by the data scientists. There are lots of packages in R designed for accessing data from
the excel spreadsheet. Users often find it easier to save their spreadsheets in comma-separated
value files and then use R's built-in functionality to read and manipulate the data.
R allows us to read data from files which are stored outside the R environment. The file
should be present in the current working directory so that R can read it. We can also set our
directory and read file from there.
In R, getwd() and setwd() are the two useful functions. The getwd() function is used to
check on which directory the R workspace is pointing. And the setwd() function is used to set
a new working directory to read and write files from that directory.
Example
A text file in which a comma separates the value in a column is known as a CSV file. Let's
start by creating a CSV file with the help of the data, which is mentioned below by saving
with .csv extension using the save As All files(*.*) option in the notepad.
Example: [Link]
id,name,salary,start_date,dept
1,Shubham,613.3,2012-01-01,IT
2,Arpita,525.2,2013-09-23,Operations
3,Vaishali,63,2014-11-15,IT
4,Nishka,749,2014-05-11,HR
5,Gunjan,863.25,2015-03-27,Finance
6,Sumit,588,2013-05-21,IT
7,Anisha,932.8,2013-07-30,Operations
8,Akash,712.5,2014-06-17,Financ
R has a rich set of functions. R provides [Link]() function, which allows us to read a CSV
file available in our current working directory. This function takes the file name as an input
and returns all the records present on it.
Let's use our [Link] file to read records from it using [Link]() function.
Example
Example
csv_data<- [Link]("[Link]")
print([Link](csv_data))
print(ncol(csv_data))
print(nrow(csv_data))
From the above output, it is clear that our data is read in the form of the data frame. So we
can apply all the functions of the data frame, which we have discussed in the earlier sections.
Like reading and analyzing, R also allows us to write into the .csv file.R provides a
[Link]( ) function. This function creates a CSV file from an existing data frame. This
function creates the file in the current working directory.
Example
csv_data<- [Link]("[Link]")
#Getting details of those peoples who joined on or after 2014
details <- subset(csv_data,[Link](start_date)>[Link]("2014-01-01"))
# Writing filtered data into a new file.
[Link](details,"[Link]")
new_details<- [Link]("[Link]")
print(new_details)
You can check the default working directory using getwd( ) function and you can also change
the directory using the function setwd( ).
After the setting of the working path, you need to import the data set or a CSV file as
shown below.
Execute the above line of code in R studio to get the data frame as shown below.
To check the class of the variable ‘readfile’, execute the below code.
class(readfile)
---> "[Link]"
In the above image you can see the data frame which includes the information of student
names, their ID’s, departments, gender and marks.
After getting the data frame, you can now analyse the data. You can extract particular
information from the data frame.
>marks <- max(data$[Link]) #this will give you the highest marks
#To extract the details of a student who scored the highest marks,
> data <- [Link]("[Link]")
> Marks <- max(data$[Link])
> retval <- subset(data, Marks. Scored == max(Marks. Scored))
#This will extract the details of the student who secured highest marks
> View(retval)
To extract the details of the students who are in studying in ‘chemistry’ Dept,
R Excel Data
The xlsx is a file extension of a spreadsheet file format which was created by Microsoft to
work with Microsoft Excel. Microsoft Excel is a widely used spreadsheet program that stores
data in the .xls or .xlsx format.
R allows us to read data directly from these files by providing some excel specific packages.
There are lots of packages such as XL Connect, xlsx, gdata, etc.
We will use xlsx package, which not only allows us to read data from an excel file but
also allow us to write data in it.
Install xlsx Package
"xlsx" package installed with [Link] command. When we install the xlsx
package, it will ask us to install some additional packages on which this package is
dependent.
For installing the additional packages, the same command is used with the required package
name. There is the following syntax of install command:
1. [Link]("package name")
Example
1. [Link]("xlsx")
Output
In R, grepl() and any() functions are used to verify the package. If the packages are installed,
these functions will return True else return False. For verifying the package, both the
functions are used together.
For loading purposes, we use the library() function with the appropriate package name. This
function loads all the additional packages also.
Example
Once the xlsx package is loaded into our system, we will create an excel file with the
following data and named it employee.
Apart from this, we will create another table with the following data and give it a name
as employee_info.
Note: Both the files will be saved in the current working directory of the R workspace.
1. [Link](file_name,sheet_index)
Let's see an example in which we read data from our [Link] file.
Example
In R, we can also write the data into our .xlsx file. R provides a [Link]() function to write
data into the excel file. There is the following syntax of [Link]() function:
[Link](data_frame,file_name,[Link],[Link],sheetnames,append)
Here,
o The data_frame is our data, which we want to insert into our excel file.
o The file_names is the name of that file in which we want to insert our data.
o The [Link] and [Link] are the logical values that are specifying whether the
column names/row names of the data frame are to be written to the file.
o The append is a logical value, which indicates our data should be appended or not into
an existing file.
Example
A binary file is a file which contains information present only in the form of bits and
bytes(0's and 1's). They are not human-readable because the bytes translate into characters
and symbols that contain many other non-printable characters. If we will read a binary file
using any text editor, it will show the characters like ð and Ø.
The code is relatively very easy to read binary data into R. To read binary data, we must
know how a piece of information has been parsed into binary.
For example, the binary file of a Microsoft Word program can only be read by the Word
program in a human-readable form. It indicates that human-readable text, there is a lot of
information such as character formatting and page numbers, etc., which are also stored with
alphanumeric characters.
A binary file is a contiguous sequence of bytes. The line break we see in a text file is a
character joining the first line to the next line.
Like CSV and Excel files, we can also write into a binary file. R provides a writeBin( )
function for writing the data into a binary file. There is the following syntax of writeBin()
function:
writeBin(object,con)
Here,
o The ?con' is the connection object which is used to write the binary file.
o The ?object' is the binary file in which we write our data.
We can also read our binary file which we have created before. For this purpose, R provides a
readBin() function for reading the data from a binary file.
1. readBin(con,what,n)
Here,
o The ?con' is the connection object which is used to read the binary file.
o The ?what' is the mode such as character, integer, etc. which represent the bytes to be
read.
o The ?n' is the number of bytes which we want to read from the binary file.
Example
# Creating a connection object to read the file in binary mode using "rb".
[Link] <- file("/Users/ajeet/R/[Link]", "rb")
# Reading the column names. n = 3 as we have 3 columns.
[Link] <- readBin([Link], character(), n = 3)
R JSON File
JSON stands for JavaScript Object Notation. The JSON file contains the data as text in a
human-readable format. Like other files, we can also read and write into the JSON files. For
this purpose, R provides a package named rjson, which we have to install with the help of the
familiar command [Link].
Install rjson package
By running the following command into the R console, we will install the rjson package into
our current working directory.
1. [Link]("rjson")
The extension of JSON file is .json. To create the JSON file, we will save the following data
as employee_info.json. We can write the information of employees in any text editor with its
appropriate rule of writing the JSON file. In JSON files, the information contains in between
the curly braces({}).
Example: employee_info.json
1. {
2. "id":["1","2","3","4","5","6","7","8" ],
3.
"name":["Shubham","Nishka","Gunjan","Sumit","Arpita","Vaishali","Anisha","Gin
ni" ],
4. "salary":["623","552","669","825","762","882","783","964"],
5.
"start_date":[ "1/1/2012","9/15/2013","11/23/2013","5/11/2014","3/27/2015","5/21/
2013",
6. "7/30/2013","6/17/2014"],
7.
"dept":[ "IT","Operations","Finance","HR","Finance","IT","Operations","Finance"]
8. }
Reading the JSON file in R is a very easy and effective process. R provide from JSON()
function to extract data from a JSON file. This function, by default, extracts the data in the
form of a list. This function takes the JSON file and returns the records which are contained
in it.
Let's see an example to understand how fromJSON() function is used to extract data and print
the result in the form of a list. We will consider the employee_info.json file which we have
created before.
Example
R provide, [Link]() function to convert the extracted data into data frame. For further
analysis, data analysts use this function. Let's start an example to see how this function is
used, and in our example, we will consider our employee_info.json file.
Example
Like HTML, XML is also a markup language which stands for Extensible Markup Language.
It is developed by World Wide Web Consortium(W3C) to define the syntax for encoding
documents which both humans and machine can read. This file contains markup tags.
There is a difference between HTML and XML. In HTML, the markup tag describes the
structure of the page, and in xml, it describes the meaning of the data contained in the file. In
R, we can read the xml files by installing "XML" package into the R environment. This
package will be installed with the help of the familiar command i.e., [Link].
1. [Link]("XML")
We will create an xml file with the help of the given data. We will save the following data
with the .xml file extension to create an xml file. XML tags describe the meaning of data, so
that data contained in such tags can easily tell or explain about the data.
Example: xml_data.xml
<records>
<employee_info>
<id>1</id>
<name>Shubham</name>
<salary>623</salary>
<date>1/1/2012</date>
<dept>IT</dept>
</employee_info>
<employee_info>
<id>2</id>
<dept>IT</dept>
</employee_info>
<employee_info>
<id>1</id>
<dept>IT</dept>
</employee_info>
</records>
In R, we can easily read an xml file with the help of xmlParse() function. This function is
stored as a list in R. To use this function, we first need to load the xml package with the help
of the library() function. Apart from the xml package, we also need to load one additional
package named methods.
It's not easy to handle data effectively in large files. For this purpose, we read the data in the
xml file as a data frame. Then this data frame is processed by the data analyst. R provide
xmlToDataFrame() function to extract the information in the form of Data Frame.
Example
In the relational database management system, the data is stored in a normalized format.
Therefore, to complete statistical computing, we need very advanced and complex SQL
queries. The large and huge data which is present in the form of tables require SQL queries to
extract the data from it.
R can easily connect with many of the relational databases like MySql, SQL Server,
Oracle, etc. When we extract the information from these databases, by default, the
information is extracted in the form of data frame. Once, the data comes from the database to
the R environment; it will become a normal R dataset. The data analyst can easily analyze or
manipulate the data with the help of all the powerful packages and functions.
RMySQL Package
RMySQL package is one of the most important built-in package of R. This package provides
native connectivity between the R and MySql database. In R, to work with MySql database,
we first have to install the RMySQL package with the help of the familiar command, which is
as follows:
1. [Link]("RMySQL")
When we run the above command in the R environment, it will start downloading the
package RMySQL.
To work with MySql database, it is required to create a connection object between R and the
database.
For creating a connection, R provides dbConnect() function. This function takes the
username, password, database name, and host name as input parameters. Let's see an example
to understand how the dbConnect() function is used to connect with the database.
Example
R MySQL Commands
In R, we can perform all the SQL commands like insert, delete, update, etc. For performing
the query on the database, R provides the dbSendQuery() function. The query is executed in
MySQL, and the result set is returned using the R fetch () function. Finally, it is stored in R as
a data frame. Let's see the example of each and every SQL command to understand how
dbSendQuery() and fetch() functions are used.
Create Table
R provides an additional function to create a table into the database i.e., dbWriteTable().
This function creates a table in the database; if it does not exist else, it will overwrite the
table. This function takes the data frame as an input.
Example
Select
We can simply select the record from the table with the help of the fetch() and
dbSendQuery() function. Let's see an example to understand how to select query works
with these two functions.
Example
We can select the specific record from the table with the help of the fetch() and
dbSendQuery() function. Let's see an example to understand how to select query works
with where clause and these two functions.
Example
Insert command
We can insert the data into tables with the help of the familiar method dbSendQuery()
function.
Example
Update command
Updating a record in the table is much easier. For this purpose, we have to pass the update
query to the dbSendQuery() function.
Example
Delete command
Below is an example in which we delete a specific row from the table by passing the delete
query in the dbSendQuery() function.
Example
Drop command
Below is an example in which we drop a table from the database by passing the appropriate
drop query in the dbSendQuery() function.
Example
Consider a scenario where a real estate company wants to gather information on properties
listed for sale in a certain area. The company can use web scraping to extract data such as
property type, price, location, square footage, and number of bedrooms and bathrooms from
popular real estate websites like Zillow and Redfin.
The company can then use this data to create a comprehensive database of properties for sale
in the area, which can be used for various purposes such as market analysis, determining fair
market value for potential clients, and identifying trends and patterns in the local real estate
market.
Once the data is collected and stored, the company can also use it to generate custom reports
and visualizations, such as graphs and maps that show the distribution of properties by price,
location, and other relevant factors. This can provide valuable insights into the real estate
market, which can be used to make informed business decisions and provide better services
to clients.
Having, knowledge of HTML and CSS will be an added advantage. It’s observed that most of
the Data Scientists are not very familiar with technical knowledge of HTML and CSS.
Therefore, let’s use an open-source software named Selector Gadget which will be more
than sufficient for anyone in order to perform Web scraping. One can access and download
the Selector Gadget extension([Link] Consider that one has this
extension installed by following the instructions from the website. Also, consider one using
Google chrome and he/she can access the extension in the extension bar to the top right.
Web Scraping in R with rvest
rvest maintained by the legendary Hadley Wickham. We can easily scrape data from webpage
from this library.
library(rvest)
Now, let’s start by scraping the heading field. For that, use the selector gadget to get the
specific CSS selectors that enclose the heading. One can click on the extension in his/her
browser and select the heading field with the cursor.
Once one knows the CSS selector that contains the heading, he/she can use this simple R
code to get the heading.
# Using CSS selectors to scrape the heading section
heading = html_node(webpage, '.entry-title')
Once one knows the CSS selector that contains the paragraphs, he/she can use this simple R
code to get all the paragraphs.
# R program to illustrate
# Web Scraping
# Import rvest library
library(rvest)
# Reading the HTML code from the website
webpage = read_html("[Link] /
data-structures-in-r-programming")
# Using CSS selectors to scrape the heading section
heading = html_node(webpage, '.entry-title')
# Converting the heading data to text
text = html_text(heading)
print(text)
# Using CSS selectors to scrape
# all the paragraph section
# Note that we use html_nodes() here
paragraph = html_nodes(webpage, 'p')
Stastical Graphics:
R – graphs
There are hundreds of charts and graphs present in R. For example, bar plot, box plot, mosaic
plot, dot chart, coplot, histogram, pie chart, scatter graph, etc.
Types of R – Charts
Bar plot or Bar Chart in R is used to represent the values in data vector as height of the bars.
The data vector passed to the function is represented over y-axis of the graph. Bar chart can
behave like histogram by using table() function instead of data vector.
Syntax: barplot(data, xlab, ylab)
where:
data is the data vector to be represented on y-axis
xlab is the label given to x-axis
ylab is the label given to y-axis
Note: To know about more optional parameters in barplot() function, use the below
command in R console:
help("barplot")
Example:
# defining vector
x <- c(7, 15, 23, 12, 44, 56, 32)
# plotting vector
barplot(x, xlab = "GeeksforGeeks Audience",
ylab = "Count", col = "white",
[Link] = "darkgreen",
[Link] = "darkgreen")
Output:
Note: To know about more optional parameters in pie() function, use the below command in
the R console:
help("pie")
Example:
Assume, vector x indicates the number of articles present on the GeeksforGeeks portal in
categories names(x)
Output:
Pie chart in 3D can also be created in R by using following syntax but
requires plotrix library.
Syntax: pie3D(x, labels, radius, main)
Example:
Output:
Histogram
Histogram is a graphical representation used to create a graph with bars representing the
frequency of grouped data in vector. Histogram is same as bar chart but only difference
between them is histogram represents frequency of grouped data rather than data itself.
Syntax: hist(x, col, border, main, xlab, ylab)
where:
x is data vector
col specifies the color of the bars to be filled
border specifies the color of border of bars
main specifies the title name of histogram
xlab specifies the x-axis label
ylab specifies the y-axis label
Example:
# defining vector
x <- c(21, 23, 56, 90, 20, 7, 94, 12,
57, 76, 69, 45, 34, 32, 49, 55, 57)
Output:
Scatter Plot
A Scatter plot is another type of graphical representation used to plot the points to show
relationship between two data vectors. One of the data vectors is represented on x-axis and
another on y-axis.
Syntax: plot(x, y, type, xlab, ylab, main)
Where,
x is the data vector represented on x-axis
y is the data vector represented on y-axis
type specifies the type of plot to be drawn. For example, “l” for lines, “p” for
points, “s” for stair steps, etc.
xlab specifies the label for x-axis
ylab specifies the label for y-axis
main specifies the title name of the graph
Note: To know about more optional parameters in plot() function, use the below command in
R console:
help("plot")
Example:
# plotting
plot(x = orange$age, y = orange$circumference, xlab = "Age",
ylab = "Circumference", main = "Age VS Circumference",
[Link] = "darkgreen", [Link] = "darkgreen",
[Link] = "darkgreen")
Output:
If a scatter plot has to be drawn to show the relation between 2 or more vectors or to plot the
scatter plot matrix between the vectors, then pairs() function is used to satisfy the criteria.
Syntax: pairs(~formula, data)
where,
~formula is the mathematical formula such as ~a+b+c
data is the dataset form where data is taken in formula
Example :
Output:
Box Plot
Box plot shows how the data is distributed in the data vector. It represents five values in the
graph i.e., minimum, first quartile, second quartile(median), third quartile, the maximum
value of the data vector.
Syntax: boxplot(x, xlab, ylab, notch)
where,
x specifies the data vector
xlab specifies the label for x-axis
ylab specifies the label for y-axis
notch, if TRUE then creates notch on both the sides of the box
Note: To know about more optional parameters in boxplot() function, use the below
command in R console:
help("boxplot")
Example:
# plotting
boxplot(x, xlab = "Box Plot", ylab = "Age",
[Link] = "darkgreen", [Link] = "darkgreen")
Output:
# Loading package
library(dplyr)
Data Layer:
In the data Layer we define the source of the information to be visualize, let’s use the mtcars
dataset in the ggplot2 package
library(ggplot2)
library(dplyr)
ggplot(data = mtcars) +
labs(title = "MTCars Data Plot")
Aesthetic Layer:
# Aesthetic Layer
ggplot(data = mtcars, aes(x = hp, y = mpg, col = disp))+
labs(title = "MTCars Data Plot")
Output:
Data visualization with R and ggplot2
Geometric layer:
In geometric layer control the essential elements, see how our data being displayed using
point, line, histogram, bar, boxplot
R
# Geometric layer
ggplot(data = mtcars, aes(x = hp, y = mpg, col = disp)) +
geom_point() +
labs(title = "Miles per Gallon vs Horsepower",
x = "Horsepower",
y = "Miles per Gallon")
Output:
Data visualization with R and ggplot2
Geometric layer: Adding Size, color, and shape and then plotting the Histogram plot
# Adding size
ggplot(data = mtcars, aes(x = hp, y = mpg, size = disp)) +
geom_point() +
labs(title = "Miles per Gallon vs Horsepower",
x = "Horsepower",
y = "Miles per Gallon")
# Histogram plot
ggplot(data = mtcars, aes(x = hp)) +
geom_histogram(binwidth = 5) +
labs(title = "Histogram of Horsepower",
x = "Horsepower",
y = "Count")
Facet Layer:
It is used to split the data up into subsets of the entire dataset and it allows the subsets to be
visualized on the same plot. Here we separate rows according to transmission type and
Separate columns according to cylinder.
# Facet Layer
# Separate rows according to transmission type
p <- ggplot(data = mtcars, aes(x = hp, y = mpg, shape = factor(cyl))) + geom_point()
p + facet_grid(am ~ .) +
labs(title = "Miles per Gallon vs Horsepower",
x = "Horsepower",
y = "Miles per Gallon")
p + facet_grid(. ~ cyl) +
labs(title = "Miles per Gallon vs Horsepower",
x = "Horsepower",
y = "Miles per Gallon")
Output:
Statistics layer
In this layer, we transform our data using binning, smoothing, descriptive, intermediate
ggplot(data = mtcars, aes(x = hp, y = mpg)) +
geom_point() +
stat_smooth(method = lm, col = "red") +
labs(title = "Miles per Gallon vs Horsepower")
Output:
Coordinates layer:
In these layers, data coordinates are mapped together to the mentioned plane of the graphic
and we adjust the axis and changes the spacing of displayed data with Control plot
dimensions.
Output:
This layer controls the finer points of display like the font size and background color
properties.
Example 1: Theme layer – element_rect() function
Output:
ggplot2 provides various types of visualizations. More parameters can be used included in
the package as the package gives greater control over the visualizations of data. Many
packages can integrate with the ggplot2 package to make the visualizations interactive and
animated.
Save and extract R plots:
To save and extract plots in R, you can use the ggsave function from the ggplot2 package.
Here’s an example of how to save and extract plots:
# Create a plot
plot <- ggplot(data = mtcars, aes(x = hp, y = mpg)) +
geom_point() +
labs(title = "Miles per Gallon vs Horsepower")
Output: