Importing data into R

One of the most important features we need to be able to do in R is import existing data,
whether it be .txt files, .csv files, or even .xls (Excel files). If we can’t import data into R,
then we can’t do anything. It is often necessary to import sample textbook data into R
before you start working on your homework.

Reading Tabular Data Files:

If you have a .txt or a tab-delimited text file, or a file with table like structure then you
can easily import it by using the basic R function [Link](). It’s good to
know that the [Link]() function is the most important and commonly used function to
import simple data files into R. It is easy and flexible. Reads a file in table format and
creates a data frame from it.

Syntax: [Link](file, header = FALSE, sep = "", quote = "\"'", dec = ".",
[Link], [Link],....)

Here

• file: You have to specify the file name, or Full path along with file name. You can
also use the URL of the external (online) txt files. For
example, [Link] or “C:/Users/Suresh/Documents/R Programs/[Link]”
• header: If the text file contains Columns names as the First Row then please specify
TRUE otherwise, FALSE
• sep: It is a short form of separator. You have to specify the character that is
separating the fields. ”, “means data is separated by comma. The default separator is
“white space”, that is one or more spaces, tabs, carriage return etc.
• quote: the set of quoting characters. To disable quoting altogether, use quote = "".If
your character values (ex: Last-Name, Occupation, Education column etc) are
enclosed in quotes then you have to specify the quote type. For double quotes we
use: quote = “\””.
• dec: the character used in the file for decimal points.
• [Link]: A Character vector that contains the row names for the returned data frame
Example 1: A data table can resides in a text file. The cells inside the table are
separated by blank characters. Here is an example of a table with 4 rows and 3
columns.
100 a1 b1
200 a2 b2
300 a3 b3
400 a4 b4
Now copy and paste the table above in a file named "[Link]" with a text editor. Then
load the data into the workspace with the function [Link].
> mydata = [Link]("[Link]") # read text file
> mydata # print data frame
V1 V2 V3
1 100 a1 b1
2 200 a2 b2
3 300 a3 b3
4 400 a4 b4
>mydata1=[Link](“[Link]

Example 2:
rain<-
[Link]("C:/Users/SUNITHA/Desktop/[Link]",header=TRUE,sep=",")
o/p: rain
month rain_mm
flow_cmm 1 1 128
15000

2 2 98 12000
3 3 92 11000
4 4 77 9800
5 5 68 7600

Reading CSV Files:

The csv file is a text file in which the values in the columns are separated by a comma.
While R can read excel .xls and .xlsx files these filetypes often cause problems. Comma
separated files (.csv) are much easier to work with. It’s best to save these files as csv
before reading them into
R. If you need to read in a csv with R the best way to do is by using [Link]() function.
Syntax: [Link] (file, header = TRUE, sep = ",", quote = "\"", dec = ".", )
Here

• file: You have to specify the file name, or Full path along with file name. You can also
use the URL of the external (online) txt files. For
example, [Link] or “C:/Users/Suresh/Documents/R Programs/[Link]”
• header: If the text file contains Columns names as the First Row then please specify
TRUE otherwise, FALSE
• sep: It is a short form of separator. You have to specify the character that is
separating the fields. ”, “means data is separated by comma. The default separator is
“white space”, that is one or more spaces, tabs, carriage return etc.
• quote: the set of quoting characters. To disable quoting altogether, use quote = "".If
your character values (ex: Last-Name, Occupation, Education column etc) are
enclosed in quotes then you have to specify the quote type. For double quotes we
use: quote = “\””.
• dec: the character used in the file for decimal points.
Example1: Let's consider the following data present in the file named [Link].
You can create this file using windows notepad by copying and pasting this data. Save the
file as [Link] using the save As All files(*.*) option in notepad.
id,name,salary,start_date,dep
t 1,Rick,623.3,2012-01-01,IT
2,Dan,515.2,2013-09-23,Operations
3,Michelle,611,2014-11-15,IT
4,Ryan,729,2014-05-11,HR
5,Gary,843.25,2015-03-27,Finance
6,Nina,578,2013-05-21,IT
7,Simon,632.8,2013-07-30,Operations
8,Guru,722.5,2014-06-17,Finance
Following is a simple example of [Link]() function to read a CSV file available in
your current working directory −
data <- [Link]("[Link]") print(data)
O/P: id, name,salary, start_date,
dept IT
 3 3Michelle 611.002014-11-15 IT HR
4 4Ryan729.002014-05-11 Finance IT
5 NAGary843.252015-03-27
6 6Nina 578.002013-05-21

• We can also analyze the imported csv file for additional information.
> data = [Link]("[Link]",header=TRUE,sep=",")
>print([Link](data)) #o/p TRUE
>print(ncol(data)) #o/p 5
>print(nrow(data)) #o/p 6
• It’s also possible to choose a file interactively using the function [Link](), which is
easy to select the file while reading.
# Read a csv file
my_data <- [Link]([Link]())
Here you need to enter file name which you wanted to open by browsing.

Importing Data from Excel Files:

Microsoft Excel is the most widely used spreadsheet program which stores data in the .xls or
.xlsx format. R can read directly from these files using some excel specific packages. Few
such packages are - XLConnect, xlsx, gdata etc. We will be using xlsx package.

• Importing and loading “xlsx” package:

You can use the following command in the R console to install the "xlsx" package. It may ask
to install some additional packages on which this package is dependent.
[Link]("xlsx")

• Load the imported package into R workspace

# Load the library into R workspace.

library("xlsx")

• To check if you already installed the package or not, type in the following:
any(grepl("<name of your package>", [Link]())) o/p True - if installed already

• Create an excel file and save it.

Open Microsoft excel. Create an excel file with name [Link] in worksheet named
as sheet1.
• Reading the Excel File
The [Link] is read by using the [Link]() function as shown below. The result is
stored as a data frame in the R environment.

# Read the first worksheet in the file [Link].

data <- [Link]("[Link]", sheetIndex = 1)
print(data)

When we execute the above code, it produces the following result −

id, 1name,salary, start_date, dept Rick623.302012-01-01IT
1 2 Dan515.202013-09-23Operations
2 3 Michelle 611.00
3 4 2014-11-15 IT HR
4 Ryan 729.00 2014-05-11 Finance IT
5 NAGary 843.25 2015-03-27
6 6Nina 578.00 2013-05-21

• We can read an excel file by selecting sheet index or sheet name.

data <- [Link]("[Link]", sheetIndex =

1) or

data <- [Link]("[Link]", sheetName = “sheet1”)

Loading and storing data clipboard:

R has a function writeClipboard that does what the name implies. However, the
argument to writeClipboard may need to be cast to a character type. For example the
code

writeClipboard( ):
> x <- "hello world"
> writeClipboard(x)
copies the string “hello world” to the clipboard as expected. However the code
> x <- 3.14
> writeClipboard(x)
Produces the error message. The solution is to call writeClipboard( [Link](x) ),
casting the object x to a character string.

readClipboard() :
The companion function for writeClipboard is readClipboard.
The command
x <- readClipboard()
will assign the contents of the clipboard to the vector x. Each line becomes an element of x.
The elements will be character strings, even if the clipboard contained a column of numbers
before the readClipboard command was executed. If you select a block of numbers from
Excel, each row becomes a single string containing tabs where there were originally cell
boundaries.

• You can also load directly from the clipboard:

# First copy the data to the clipboard

data <- [Link]('clipboard', header=TRUE)
 # Or:
# data <- [Link]('clipboard')

Ex: display a file by using display file option in file menu, and select any data from that
and press ctrl+c. (data is loaded into clipboard) then execute the following command
to load the clipboard:
data <- [Link]('clipboard')
> data
Subtype Gender Expression
1 A m -0.54
2 A f -0.80
p3 B f -
1.03
4 C m -0.41

• It is possible to write delimited data to terminal (stdout()), so that it can be

copied and pasted elsewhere. Or it can be written directly to the clipboard.
[Link](data, stdout(),
[Link]=FALSE)
"Subtype","Gender","Expression"
"A","m",-0.54
"A","f",-0.8
"B","f",-1.03
"C","m",-0.41
> [Link](data, 'clipboard', [Link]=FALSE)

Saving an R data file:

As you work with your data in R you will eventually want to save it to disk. This will allow
you to work with the data later and still retain the original dataset. It can also allow you to
share your dataset with other analysts. One of the simplest ways to save your data is by
saving it into an RData file with the function save( ). The function save() can be used to
save one or more R objects to a specified file (in .RData or .rda file formats). The function
can be read back from the file using the function load().

Let us create an example dataset. The following R script creates an R data frame
[explained in another topic of this learning infrastructure] for you to practice saving.

x <- c(1:10) # create a numeric vector

y <- c(11:20) # create a numeric vector.
z <- c(21:30) # create a numeric
vector m <- cbind(x, y, z) # create a
matrix
d <- [Link](m) # create a data frame

# create a text vector

t <- c("red", "blue", "red", "white", "blue", "white", "red","blue", "white",
"white") df <- cbind(d, t) # add the text vector to the data frame
Your R session now has a data frame object named df that you can use for the exercises
below. You can save the data frame df [from the above example] using this command:

save(df, file = "[Link]")

While the save( ) command can have several arguments, this example uses only two. The
first argument is the name of your R data object, df in this example. The second argument
assigns a name to the RData file, [Link] in this example. You can use any text as your file
name as long as it does not contain any embedded spaces. While you do not have to use the
.RData extension, this is a recommended practice because the .RData extension will help
RStudio to identify your R datasets. Notice that the file name is enclosed in quotation
marks.

# Saving an object in
RData format save(data1,
file = "[Link]")

# Save multiple objects

save(data1, data2, file = "[Link]")

It’s also possible to specify the file name for saving your work
space: [Link](file = "my_work_space.RData")

To restore your workspace, type this:

load("my_work_space.RData")

# save your command history

savehistory(file="myfile") # default is ".Rhistory"

# recall your command history

loadhistory(file="myfile") # default is ".Rhistory“

Loading R data:

Loading Rdata Files in a Convenient Way. These functions loads an Rdata object saved
as a data frame or a matrix in the current R environment. The function [Link] saves
the loaded object in the global environment while load.Rdata2 loads the object only
specified environments.

[Link](filename, objname)

load.Rdata2(filename, path=getwd())

Arguments
Filename - Rdata file (matrix or data frame)
Objname - Object name. This object will be a global variable in R.
Path - Directory from which the dataset should be loaded
# load a data frame in the file "data_s3.Rdata" and save
this # as the object "dat.s3"
[Link]( filename="data_s3.Rdata", "dat.s3" )
head(dat.s3)

Writing Data to a File

The R base function [Link]() can be used to export a data frame or a matrix to a
file. A simplified format is as follow:

Syntax:
[Link](x, file, append = FALSE, sep = " ", dec = ".", [Link] = TRUE, [Link] =
TRUE)
where
• x: a matrix or a data frame to be written.
• file: a character specifying the name of the result file.
• sep: the field separator string, e.g., sep = “\t” (for tab-separated value).
• dec: the string to be used as decimal separator. Default is “.”
• [Link]: either a logical value indicating whether the row names of x are to be written
along with x, or a character vector of row names to be written.
• [Link]: either a logical value indicating whether the column names of x are to be written
along with x, or a character vector of column names to be written. If [Link] = NA and
[Link] = TRUE a blank column name is added, which is the convention used for CSV
files to be read by spreadsheets.

Ex1: Write data from R to a txt file: [Link](my_data, file = “my_data.txt”, sep = “”)
Ex2: The R code below exports the built-in R mtcars data set to a tab-separated ( sep = “\t”)
file called [Link] in the current working directory:
# Loading mtcars data
data("mtcars")
# Writing mtcars data
[Link](mtcars, file = "[Link]", sep = "\t", [Link] = TRUE, [Link] = NA)

If you don’t want to write row names, use [Link] = FALSE as follow:
[Link](mtcars, file = "[Link]", sep = "\t",[Link] = FALSE)

Writing data to a CSV file:

• [Link]() uses “.” for the decimal point and a comma (“,”) for the separator.
• write.csv2() uses a comma (“,”) for the decimal point and a semicolon (“;”) for
the separator.
The syntax is as follow:
[Link](my_data, file =
"my_data.csv")
 Ex3: write data from R to a csv file: [Link](my_data, file = “my_data.csv”)
Ex4: [Link](xdata, "c:/[Link]", sep="\t")
library(xlsx)
Ex5: [Link](ydata, "c:/[Link]")

What is Data Manipulation in R?

Data structures provide the way to represent data in data analytics. We can manipulate
data in R for analysis and visualization. One of the most important aspects of computing
with data Data Manipulation in R and enable its subsequent analysis and visualization.

How to round off numbers in R:

Although R can calculate accurately to up to 16 digits, you don’t always want to use that
many digits. In this case, you can use a couple functions in R to round numbers. To round a
number to two digits after the decimal point, for example, use the round() function as
follows:
> round(123.456,digits=2)
[1] 123.46
You also can use the round() function to round numbers to multiples of 10, 100, and so
on. For that, you just add a negative number as the digits argument:
> round(-123.456,digits=-2)
[1] -100
If you want to specify the number of significant digits to be retained, regardless of the size
of the number, you use the signif() function instead:
> signif(-123.456,digits=4)
[1] -123.5

Both round() and signif() round numbers to the nearest possibility. So, if the first digit
that’s dropped is smaller than 5, the number is rounded down. If it’s bigger than 5, the
number is rounded up.

If the first digit that is dropped is exactly 5, R uses a rule that’s common in programming
languages: Always round to the nearest even number. round(1.5) and round(2.5) both
return 2, Ex:
> round(-4.5)
[1] -4
> round(-4.6)
[1] -5
> round(-4.4)
[1] -4

Contrary to round(), three other functions always round in the same direction:
• floor(x) rounds to the nearest integer that’s smaller than x. So floor(123.45)
becomes 123 and floor(-123.45) becomes –124.
• ceiling(x) rounds to the nearest integer that’s larger than x. This means
ceiling (123.45) becomes 124 and ceiling(123.45) becomes –123.
 • trunc(x) rounds to the nearest integer in the direction of 0. So trunc(123.65)
becomes 123 and trunc(-123.65) becomes –123.

Merging data in R:
➢ In R you use the merge() function to combine data frames. This powerful function
tries to identify columns or rows that are common between the two different data
frames.
➢ The simplest form of merge() finds the intersection between two different sets of
data. In other words, to create a data frame that consists of those states that are
cold as well as large, use the default version of merge():

syntax: The merge() function takes quite a large number of arguments. These arguments
can look quite intimidating until you realize that they form a smaller number of related
arguments:
➢ x: A data frame.
➢ y: A data frame.
➢ by, by.x, by.y: The names of the columns that are common to both x and y. The default
is to use the columns with common names between the two data frames.
➢ all, all.x, all.y: Logical values that specify the type of merge. The default
value is all=FALSE (meaning that only the matching rows are returned).
➢ That last group of arguments — all, all.x and all.y — deserves some explanation.
These arguments determine the type of merge that will happen.

Ex1: To merge two dataframes (datasets) horizontally, use the merge function. In most
cases, you join two dataframes by one or more common key variables (i.e., an inner join).
➢ # merge two dataframes by id
total <- merge(dataframea,dataframeb,by="id")
➢ # merge two dataframes by id and country
total <- merge(dataframea,dataframeb,by=c("id","country"))

Ex2:Let's create two data frames and merge them:

>exp <- [Link](samples=c("a","b","c"),values=c(2.43,5.32,-1.23))
>backedup <- [Link](patients=c("a","b","c"),marked=c("yes","yes","no"))
➢ Merge two data frames exp and backedup
> xyz <- merge(exp,backedup,by.x="samples",by.y="patients")
>xyz
samples values marked
1 a 2.43 yes
2 b 5.32 yes
3 c -1.23 no

Different types of merging

The merge() function allows four ways of combining data:
• Natural join: To keep only rows that match from the data frames,
specify the argument all=FALSE.
• Full outer join: To keep all rows from both data frames, specify all=TRUE.
 • Left outer join: To include all the rows of your data frame x and only those from y
that match, specify all.x=TRUE.
• Right outer join: To include all the rows of your data frame y and only those from x
that match, specify all.y=TRUE.

Ex: We will create two data frame df1 and df2 to illustrate joins in R. We will create two
data frame df1 and df2 to illustrate joins in R.
> df1 = [Link](CustomerId = c(1:6), Product=c(rep("Toaster”,3), rep("Radio",3)))
> df2 = [Link](CustomerId = c(2,4,6), State = c(rep("Alabama",2), rep("Ohio",1)))
> df1

>df2

We can merge these data frames by using the merge function and its optional parameters:
Natural join: merge(x=df1, y=df2, by = “CustomerId”,all = FALSE)
Outer join: merge(x = df1, y = df2, by = "CustomerId", all = TRUE)
Left outer: merge(x = df1, y = df2, by = "CustomerId", all.x = TRUE)
Right outer: merge(x = df1, y = df2, by = "CustomerId", all.y = TRUE)

Data aggregation in R:

You have a data set and you need to quickly organize it to perform your data analysis.
Where do you start? You could create a table of statistics which summarizes data by
aggregating it. We use aggregate function to do this.
aggregate( ) function: Aggregate is a function in base R which can, as the name suggests,
aggregate the inputted [Link] d.f by applying a function specified by the FUN
parameter to each column of [Link] defined by the by input parameter.
Syntax: aggregate(x, by, FUN, ………)
➢ The first argument to the function is usually a [Link].
➢ The by argument is a list of variables to group by. This must be a list even if there is
only one variable, as in the example.
➢ The FUN argument is the function which is applied to all columns (i.e., variables)
in the grouped data. Because we cannot calculate the average of categorical
variables such as Name and Shift, they result in empty columns, which I have
removed for clarity.

The process involves two stages. First, collate individual cases of raw data together with a
grouping variable. Second, perform which calculation you want on each group of cases.
These two stages are wrapped into a single function.
To perform aggregation, we need to specify three things in the code:
• The data that we want to aggregate
• The variable to group by within the data
• The calculation to apply to the groups (what you want to find out)
Ex: Load the example data by running the following R code:
data=DownloadXLSX("[Link]
Aggregation_data.xlsx", [Link] = FALSE, [Link] = TRUE)
Name Role Shift Salary Age
1 Ann Cook Lunch 1000 19
2 Bob Server Lunch 1200 24
3 Charlie Cook Lunch 1400 29
4 Dave Server Lunch 1500 24
5 Ed Manager Lunch 2200 32
6 Fred Manager Dinner 2000 41
7 Gary Cook Dinner 2000 28
8 Henry Server Dinner 1500 30
9 Ian Cook Dinner 1600 22
10 Jo Server Dinner 1800 25

Perform aggregation with the following R code.

agg = aggregate(data,by = list(data$Role),FUN = mean)
This produces a table of the average salary and age by role, as below.
Group.1 Salary Age
1 Cook 1500.0 24.4
2 Manager 2100.0 36.5
3 Server 1500.0 25.8

How to Rename Columns in R

This page will show you how to rename columns in R with examples using either the
existing column name or the column number to specify which column name to change.
Ex:
> d <- [Link](alpha=1:3, beta=4:6, gamma=7:9)
>d
alpha beta gamma
1 1 4 7
2 2 5 8
3 3 6 9

We can display the names of columns

> names(d)
[1] "alpha" "beta" "gamma"

We can rename a column by using indexing the column name

> names(d)[names(d)=="beta"] = "two"

We can rename more than one column at a time by using rename function
> library(plyr)
> rename(d, c("beta"="two", "gamma"="three"))
>d
alpha two gamma
1 1 4 7
2 2 5 8
3 3 6 9

We can also rename a column by indexing with number.

> names(d)[3] = "three"
>d
alpha two three
1 1 4 7
2 2 5 8
3 3 6 9

HOW TO SORT AND ORDER DATA IN R

One very common task in data analysis and reporting is sorting information, which you
can do easily in R. we use sort( ), order( ) and arrange( ) functions in R to sort the data.
sort() function sorts a vector.
Syntax: sort(x, decreasing = FALSE, [Link] = NA,.........)

x: vector
decreasing: decrease or not
[Link]: if TRUE, NAs are put at last position, FALSE at first, if NA, remove them (default)

...

Sort Vectors:

>x <- c(1,2.3,2,3,4,8,12,43,-4,-1,NA)

How to sort a vector in ascending order

>sort(x)
[1] -4.0 -1.0 1.0 2.0 2.3 3.0 4.0 8.0 12.0 43.0

How to sort a vector in decreasing order

>sort(x,decreasing=TRUE)
[1] 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0

put NA values at last position

>sort(x,decreasing=TRUE, [Link]=TRUE)
[1] 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0 NA

put NA values at first

>sort(x,decreasing=TRUE, [Link]=FALSE)
[1] NA 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0

order( ) function: order() function sorts a vector, matrix or data frame.

Syntax: order(x, decreasing = FALSE, [Link] = NA, ...)
Where:
x: vector
decreasing: decrease or not
[Link]: if TRUE, NAs are put at last position, FALSE at first, if NA, remove them (default)

Ex1: Sort Vectors:

>x <- c(1,2.3,2,3,4,8,12,43,-4,-1,NA)
>order(x)
[1] -4.0 -1.0 1.0 2.0 2.3 3.0 4.0 8.0 12.0 43.0
>order(x,decreasing=TRUE)
[1] 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0
>order(x,decreasing=TRUE, [Link]=TRUE)
[1] 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0 NA
>order(x,decreasing=TRUE, [Link]=FALSE)
[1] NA 43.0 12.0 8.0 4.0 3.0 2.3 2.0 1.0 -1.0 -4.0

Ex2: Order data frame:

>BOD #R built-in dataset, Biochemical Oxygen
Demand Time demand
1 1 8.3
2 2 10.3
3 3 19.0
4 4 16.0
5 5 15.6
6 7 19.8

Sort by "demand" column:

>BOD[with(BOD,order(demand)),]
Time demand
1 1 8.3
2 2 10.3
5 5 15.6
4 4 16.0
3 3 19.0
6 7 19.8

arrange() function:
We learned how to sort the values with the function sort(). The library dplyr has its
sorting function called arrange( ). The arrange() verb can reorder one or many rows,
either ascending (default) or descending.
We can reorder the data of a data table, by the value of one or more columns (i.e., variables).
• Sort a data frame rows in ascending order (from low to high)
using the R function arrange() [dplyr package]
• Sort rows in descending order (from high to low) using arrange() in combination
with the function desc() [dplyrpackage]
Ex:
> arrange(A): Ascending sort of variable A
> arrange(A, B): Ascending sort of variable A and B
> arrange(desc(A), B): Descending sort of variable A and ascending sort of B
> arrange(mtcars, cyl, disp) # mtcars is data set, cyl and disp are columns in that dataset.
> arrange(mtcars, desc(disp)) # descending sort on disp column

Data Manipulation in R
Data structures provide the way to represent data in data analytics. We can manipulate
data in R for analysis and visualization. One of the most important aspects of computing
with data Data Manipulation in R and enable its subsequent analysis and visualization.

Creating Subsets of Data in R

As we know, data size is increasing exponentially and doing an analysis of complete data is
very time-consuming. So the data is divided into small sized samples and analysis of
samples is done. The process of creating samples is called sub-setting. Different methods
of sub-setting in R are:
a. $
The dollar sign operator selects a single element of data. When you use this operator
with a data frame, the result is always a vector.

b. [[
Similar to $ in R, the double square brackets operator also returns a single element, but it
offers the flexibility of referring to the elements by position rather than by name. It can be
used for data frames and lists.
c. [
The single square bracket operator in R returns multiple elements of data. The index
within the square brackets can be a numeric vector, a logical vector, or a character vector.

Selecting Rows/Observations:
R has powerful indexing features for accessing object elements. These features can be used
to select and exclude variables and observations. The following code snippets demonstrate
ways to keep or delete variables and observations and to take random samples from a
dataset.
Selecting (Keeping)
Variables # select variables
v1, v2, v3 myvars <- c("v1",
"v2", "v3") newdata <-
mydata[myvars]

# exclude 3rd and 5th variable

newdata <- mydata[c(-3,-5)]

# delete variables v3 and v5

mydata$v3 <- mydata$v5 <- NULL

# select 1st and 5th thru 10th variables

newdata <- mydata[c(1,5:10)]

Selecting
Observations # first 5
observations newdata
<- mydata[1:5,]

# based on variable values

newdata <- mydata[ which(mydata$gender=='F' & mydata$age > 65), ]

# or
attach(mydat
a)
newdata <- mydata[ which(gender=='F' & age > 65),]
detach(mydata)

Selection using the Subset Function

The subset( ) function is the easiest way to select variables and observations. In the
following example, we select all rows that have a value of age greater than or equal to 20
or age less then
10. We keep the ID and Weight
columns. # using subset function
newdata <- subset(mydata, age >= 20 | age < 10, select=c(ID, Weight))
In the next example, we select all men over the age of 25 and we keep
variables weight through income (weight, income and all columns between
them).

# using subset function (part 2)

newdata <- subset(mydata, sex=="m" & age > 25, select=weight:income)

Commands to Extract Rows and Columns

The following represents different commands which could be used to extract one or more
rows with one or more columns. Note that the output is extracted as a data frame. This
could be checked using the class command.
# All Rows and All
Columns df[,]
# First row and all columns
df[1,]
# First two rows and all columns
df[1:2,]
# First and third row and all columns
df[ c(1,3), ]
# First Row and 2nd and third column
df[1, 2:3]
# First, Second Row and Second and Third
Column df[1:2, 2:3]
# Just First Column with All rows
df[, 1]
# First and Third Column with All rows
df[,c(1,3)]

How to identify and remove duplicate data in R.

You will learn how to use the following R base and dplyr functions:
R base functions
duplicated(): for identifying duplicated elements and
unique(): for extracting unique elements,
distinct() [dplyr package] to remove duplicate rows in a data frame.
 Load the tidyverse packages, which include dplyr:
library(tidyverse)

We’ll use the R built-in iris data set, which we start by converting into a tibble data
frame (tbl_df) for easier data analysis.
my_data <- as_tibble(iris)
my_data

## # A tibble: 150 x 5
## [Link] [Link] [Link] [Link] Species
## <dbl> <dbl> <dbl> <dbl> <fct>
## 1 5.1 3.5 1.4 0.2 setosa
## 2 4.9 3 1.4 0.2 setosa
## 3 4.7 3.2 1.3 0.2 setosa
## 4 4.6 3.1 1.5 0.2 setosa
## 5 5 3.6 1.4 0.2 setosa
## 6 5.4 3.9 1.7 0.4 setosa
## # ... with 144 more rows

Find and drop duplicate elements

The R function duplicated() returns a logical vector where TRUE specifies which elements
of a vector or data frame are duplicates.
Given the following vector:
x <- c(1, 1, 4, 5, 4, 6)
▪ To find the position of duplicate elements in x, use
this: duplicated(x)
## [1] FALSE TRUE FALSE FALSE TRUE FALSE
▪ Extract duplicate
elements:
x[duplicated(x)]
## [1] 1 4
▪ If you want to remove duplicated elements, use !duplicated(), where ! is a logical
negation: x[!duplicated(x)]
## [1] 1 4 5 6
▪ Following this way, you can remove duplicate rows from a data frame based on a
column values, as follow:
# Remove duplicates based on [Link] columns
my_data[!duplicated(my_data$[Link]), ]
## # A tibble: 23 x 5
## [Link] [Link] [Link] [Link] Species
## <dbl> <dbl> <dbl> <dbl> <fct>
## 1 5.1 3.5 1.4 0.2 setosa
## 2 4.9 3 1.4 0.2 setosa
## 3 4.7 3.2 1.3 0.2 setosa
## 4 4.6 3.1 1.5 0.2 setosa
## 5 5 3.6 1.4 0.2 setosa
## 6 5.4 3.9 1.7 0.4 setosa
## # ... with 17 more rows
! is a logical negation. !duplicated() means that we don’t want duplicate rows.

Extract unique elements

Given the following vector:
x <- c(1, 1, 4, 5, 4, 6)
You can extract unique elements as follow:
unique(x)
## [1] 1 4 5 6
It’s also possible to apply unique() on a data frame, for removing duplicated rows as
follow: unique(my_data)
 Remove duplicate rows in a data frame
The function distinct() [dplyr package] can be used to keep only unique/distinct rows
from a data frame. If there are duplicate rows, only the first row is preserved. It’s an
efficient version of the R base function unique().

In this chapter, we describe key functions for identifying and removing duplicate data:
▪ Remove duplicate rows based on one or more column values: my_data
%>% dplyr::distinct([Link])
▪ R base function to extract unique elements from vectors and data frames: unique(my_data)
▪ R base function to determine duplicate elements: duplicated(my_data)

Example 1 : Remove Duplicate Rows based on all the variables (Complete Row)
The distinct function is used to eliminate
duplicates. x1 = distinct(mydata)
In this dataset, there is not a single duplicate row so it returned same number of
rows as in mydata.

Example 2 : Remove Duplicate Rows based on a variable

The .keep_all function is used to retain all other variables in the output data
frame. x2 = distinct(mydata, Index, .keep_all= TRUE)

Example 3 : Remove Duplicates Rows based on multiple variables

In the example below, we are using two variables - Index, Y2010 to determine
uniqueness. x2 = distinct(mydata, Index, Y2010, .keep_all= TRUE)

select( ) Function
It is used to select only desired variables.
syntax : select(data ,
) data : Data
Frame
........Variables by name or by function

Example 1 : Selecting Variables (or Columns)

Suppose you are asked to select only a few variables. The code below selects variables
"Index", columns from "State" to "Y2008".
mydata2 = select(mydata, Index, State:Y2008)

Example 2 : Dropping Variables

The minus sign before a variable tells R to drop the variable.
mydata = select(mydata, -Index, -State)
The above code can also be written like :
mydata = select(mydata, -c(Index,State))

Example 3 : Selecting or Dropping Variables starts with 'Y'

The starts_with() function is used to select variables starts with an
alphabet. mydata3 = select(mydata, starts_with("Y"))
 Adding a negative sign before starts_with() implies dropping the variables starts with
'Y' mydata33 = select(mydata, -starts_with("Y"))
The following functions helps you to select variables based on their names.
Helpers Description
starts_with( Starts with a prefix
)
ends_with() Ends with a prefix
contains() Contains a literal string
matches() Matches a regular expression
num_range( Numerical range like x01, x02, x03.
)
one_of() Variables in character vector.
everything() All variables.

Example 4 : Selecting Variables contain 'I' in their names

mydata4 = select(mydata, contains("I"))

filter( ) Function
It is used to subset data with matching logical conditions.
syntax : filter(data ,
) data : Data
Frame
........Logical Condition

Example 1 : Filter Rows

Suppose you need to subset data. You want to filter rows and retain only those values in
which Index is equal to A.
mydata7 = filter(mydata, Index == "A")
Index State Y2002 Y2003 Y2004 Y2005 Y2006 Y2007 Y2008 Y2009
1 A Alabama 1296530 1317711 1118631 1492583 1107408 1440134 1945229 1944173
2 A Alaska 1170302 1960378 1818085 1447852 1861639 1465841 1551826 1436541
3 A Arizona 1742027 1968140 1377583 1782199 1102568 1109382 1752886 1554330
4 A Arkansas 1485531 1994927 1119299 1947979 1669191 1801213 1188104 1628980

Y2010 Y2011 Y2012 Y2013 Y2014 Y2015

1 1237582 1440756 1186741 1852841 1558906 1916661
2 1629616 1230866 1512804 1985302 1580394 1979143
3 1300521 1130709 1907284 1363279 1525866 1647724
4 1669295 1928238 1216675 1591896 1360959 1329341

Example 2 : Multiple Selection Criteria

The %in% operator can be used to select multiple items. In the following program, we are
telling R to select rows against 'A' and 'C' in column 'Index'.
mydata7 = filter(mydata6, Index %in% c("A", "C"))