Scripting with R, Dataframes and more
Writing first R script
Before I delve more into the topic. I have a request: don’t burn yourself with a lot of Googling. Just focus on
the doucmentation first, and then google relevant topics.
As usual VIM is your friend for text writing. So, here is the first script to print “Hello World!”.
NOTE: activate the virtual environment first, and be in the appropriate directory to write your script. You
would just write one line in the script, so the final file would look like.
cat hello_world.R
print ("Hello World!")
To run the script, you will type:
Please note in the command below: /usr/local/bin/Rscript is specific to my laptop. You should simply
write Rscript
/usr/local/bin/Rscript hello_world.R
[1] "Hello World!"
Incorporating arguments in R scripts
As discussed in the class - for general scripts, you will frequently use command line arguments to feed
necessary or optional things to the script. In R, function commandArgs does the job, and you can use it as
shown below.
cat code_with_arguments.R
args = commandArgs (trailingOnly = T)
print (c(args[1], args[2]))
echo "anything" | /usr/local/bin/Rscript code_with_arguments.R first "argument with space"
[1] "first" "argument with space"
So args[1] and args[2] are strings holding your provided arguments in order. You may ask why
trailingOnly = TRUE. What will happen if it is set to FALSE. Try it yourself.
Reading from strandard input
In this course, I have emphasized a lot on standard inputs. You may not appreciate it now, but as you
progress further in your career you will realize its use.
Now, lets see how to read using standard input.
seq 1 10 > [Link]
cat [Link] | /usr/local/bin/Rscript read_from_stdin.R
1
V1
1 1
2 2
3 3
4 4
5 5
6 6
cat read_from_stdin.R
d_t = [Link](file ("stdin"), sep = "\t", header = F, stringsAsFactors = F)
print (head(d_t))
As you used to read tabular data in Python like read_csv, here you have the equivalence [Link] or
[Link]. To look for help of any function you can just put a question mark sign in front (?[Link]).
The above code printed first 6 lines from file with V1 as the column name. With argument header = F in
[Link] function, R automatically assigns headers as V1, V2 and so on. This is the first time you are
getting introduced to a data frame structure in R. Essentially, it is a matrix with nice column and row names.
Dataframes
It is the most widely used data structure in R. In fact, Pandas in Python is an inspiration from R. So, lets
take a deep dive into dataframes. We can take existing dataframes available in R. For example, mtcars, is a
dataframe with measurement of 11 features for 32 cars. You can see the description of column names by
typing ?mtcars.
mtcars
## mpg cyl disp hp drat wt qsec vs am gear carb
## Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
## Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4
## Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1
## Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
## Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2
## Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
## Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4
## Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
## Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
## Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
## Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
## Merc 450SE 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3
## Merc 450SL 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3
## Merc 450SLC 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3
## Cadillac Fleetwood 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4
## Lincoln Continental 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4
## Chrysler Imperial 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4
## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1
## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2
## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1
## Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
## Dodge Challenger 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2
## AMC Javelin 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2
## Camaro Z28 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4
## Pontiac Firebird 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
## Fiat X1-9 27.3 4 79.0 66 4.08 1.935 18.90 1 1 4 1
## Porsche 914-2 26.0 4 120.3 91 4.43 2.140 16.70 0 1 5 2
2
## Lotus Europa 30.4 4 95.1 113 3.77 1.513 16.90 1 1 5 2
## Ford Pantera L 15.8 8 351.0 264 4.22 3.170 14.50 0 1 5 4
## Ferrari Dino 19.7 6 145.0 175 3.62 2.770 15.50 0 1 5 6
## Maserati Bora 15.0 8 301.0 335 3.54 3.570 14.60 0 1 5 8
## Volvo 142E 21.4 4 121.0 109 4.11 2.780 18.60 1 1 4 2
Some general use functions:
# Dimension of a dataframe
dim(mtcars)
## [1] 32 11
# number of rows
nrow(mtcars)
## [1] 32
# number of columns
ncol(mtcars)
## [1] 11
# indexing : like get rows 3 to 6 and columns 5 to 7
print(mtcars[3:6, 5:7])
## drat wt qsec
## Datsun 710 3.85 2.320 18.61
## Hornet 4 Drive 3.08 3.215 19.44
## Hornet Sportabout 3.15 3.440 17.02
## Valiant 2.76 3.460 20.22
# Get columns by name
mtcars[, c("mpg", "cyl")]
## mpg cyl
## Mazda RX4 21.0 6
## Mazda RX4 Wag 21.0 6
## Datsun 710 22.8 4
## Hornet 4 Drive 21.4 6
## Hornet Sportabout 18.7 8
## Valiant 18.1 6
## Duster 360 14.3 8
## Merc 240D 24.4 4
## Merc 230 22.8 4
## Merc 280 19.2 6
## Merc 280C 17.8 6
## Merc 450SE 16.4 8
## Merc 450SL 17.3 8
## Merc 450SLC 15.2 8
## Cadillac Fleetwood 10.4 8
## Lincoln Continental 10.4 8
## Chrysler Imperial 14.7 8
## Fiat 128 32.4 4
## Honda Civic 30.4 4
## Toyota Corolla 33.9 4
## Toyota Corona 21.5 4
3
## Dodge Challenger 15.5 8
## AMC Javelin 15.2 8
## Camaro Z28 13.3 8
## Pontiac Firebird 19.2 8
## Fiat X1-9 27.3 4
## Porsche 914-2 26.0 4
## Lotus Europa 30.4 4
## Ford Pantera L 15.8 8
## Ferrari Dino 19.7 6
## Maserati Bora 15.0 8
## Volvo 142E 21.4 4
# what if you ask for a single column?
mtcars[, "mpg"]
## [1] 21.0 21.0 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 17.8 16.4 17.3 15.2 10.4
## [16] 10.4 14.7 32.4 30.4 33.9 21.5 15.5 15.2 13.3 19.2 27.3 26.0 30.4 15.8 19.7
## [31] 15.0 21.4
# It just became a vector, you lost the row id information. So, to keep it
# intact, you have to use an addition flag. See below:
mtcars[, "mpg", drop = F]
## mpg
## Mazda RX4 21.0
## Mazda RX4 Wag 21.0
## Datsun 710 22.8
## Hornet 4 Drive 21.4
## Hornet Sportabout 18.7
## Valiant 18.1
## Duster 360 14.3
## Merc 240D 24.4
## Merc 230 22.8
## Merc 280 19.2
## Merc 280C 17.8
## Merc 450SE 16.4
## Merc 450SL 17.3
## Merc 450SLC 15.2
## Cadillac Fleetwood 10.4
## Lincoln Continental 10.4
## Chrysler Imperial 14.7
## Fiat 128 32.4
## Honda Civic 30.4
## Toyota Corolla 33.9
## Toyota Corona 21.5
## Dodge Challenger 15.5
## AMC Javelin 15.2
## Camaro Z28 13.3
## Pontiac Firebird 19.2
## Fiat X1-9 27.3
## Porsche 914-2 26.0
## Lotus Europa 30.4
## Ford Pantera L 15.8
## Ferrari Dino 19.7
4
## Maserati Bora 15.0
## Volvo 142E 21.4
This is a nicely behaved dataframe, as in there are no NA or NaNvalues. Many times, you run into those cases.
To analyze those datasets, you usually handle those instances by i) either removing entire row, ii) setting
those to lets say zero, iii) replace by the mean of of the column or any other stratagies.
Let’s explore one such dataset.
head(airquality)
## Ozone Solar.R Wind Temp Month Day
## 1 41 190 7.4 67 5 1
## 2 36 118 8.0 72 5 2
## 3 12 149 12.6 74 5 3
## 4 18 313 11.5 62 5 4
## 5 NA NA 14.3 56 5 5
## 6 28 NA 14.9 66 5 6
[Link] function is your friend here.
head([Link](airquality))
## Ozone Solar.R Wind Temp Month Day
## [1,] FALSE FALSE FALSE FALSE FALSE FALSE
## [2,] FALSE FALSE FALSE FALSE FALSE FALSE
## [3,] FALSE FALSE FALSE FALSE FALSE FALSE
## [4,] FALSE FALSE FALSE FALSE FALSE FALSE
## [5,] TRUE TRUE FALSE FALSE FALSE FALSE
## [6,] FALSE TRUE FALSE FALSE FALSE FALSE
Now you want to extract all the indices which are TRUE. To get that, you would use which function.
head(which([Link](airquality)))
## [1] 5 10 25 26 27 32
You see this is returning a vector, by walking to the dataframe column by column. See in the first 33 rows,
the first column (Ozone) has NA values exactly at those indices (5, 10, 25, 26, 27,32).
head(airquality, n = 33)
## Ozone Solar.R Wind Temp Month Day
## 1 41 190 7.4 67 5 1
## 2 36 118 8.0 72 5 2
## 3 12 149 12.6 74 5 3
## 4 18 313 11.5 62 5 4
## 5 NA NA 14.3 56 5 5
## 6 28 NA 14.9 66 5 6
## 7 23 299 8.6 65 5 7
## 8 19 99 13.8 59 5 8
## 9 8 19 20.1 61 5 9
## 10 NA 194 8.6 69 5 10
## 11 7 NA 6.9 74 5 11
## 12 16 256 9.7 69 5 12
## 13 11 290 9.2 66 5 13
## 14 14 274 10.9 68 5 14
## 15 18 65 13.2 58 5 15
## 16 14 334 11.5 64 5 16
5
## 17 34 307 12.0 66 5 17
## 18 6 78 18.4 57 5 18
## 19 30 322 11.5 68 5 19
## 20 11 44 9.7 62 5 20
## 21 1 8 9.7 59 5 21
## 22 11 320 16.6 73 5 22
## 23 4 25 9.7 61 5 23
## 24 32 92 12.0 61 5 24
## 25 NA 66 16.6 57 5 25
## 26 NA 266 14.9 58 5 26
## 27 NA NA 8.0 57 5 27
## 28 23 13 12.0 67 5 28
## 29 45 252 14.9 81 5 29
## 30 115 223 5.7 79 5 30
## 31 37 279 7.4 76 5 31
## 32 NA 286 8.6 78 6 1
## 33 NA 287 9.7 74 6 2
But, to set zero value at all cells with NA, we need to know row and column ids. To acheive this, you would
add one agrument in the which function.
head(which([Link](airquality), [Link] = T))
## row col
## [1,] 5 1
## [2,] 10 1
## [3,] 25 1
## [4,] 26 1
## [5,] 27 1
## [6,] 32 1
cp_air = airquality
cp_air$aqi = [Link](cp_air)
print(head(cp_air))
## Ozone Solar.R Wind Temp Month Day aqi
## 1 41 190 7.4 67 5 1 1
## 2 36 118 8.0 72 5 2 2
## 3 12 149 12.6 74 5 3 3
## 4 18 313 11.5 62 5 4 4
## 5 NA NA 14.3 56 5 5 5
## 6 28 NA 14.9 66 5 6 6