Assignment-2
Data Analytics
Pre-
Proc
essi
ng
In R
Name :- Purushottam Kumar
Roll :- 2020178043
MCA 2nd Sem (Regular)
Submission Date : 18-July-2021
Pre-processing : Data preprocessing is a technique, which is used to transform the raw data in a
useful and efficient format.
Steps Involved in Data Pre-processing:
1. Data Cleaning: Handling irrelevant data or missing data or noisy data etc.
(a). Missing Data : When some data is missing in the data.
(b). Noisy Data : Noisy data is a meaningless data that can’t be interpreted by machines.
2. Data Transformation: Transforming the data in appropriate forms suitable for mining process.
3. Data Reduction: To increase the storage efficiency and reduce data storage & analysis costs.
-------------*-----------------*----------------*---------------*---------------*------------------*---------------*------------
Here I have used a dataset of country that contains four columns. Mainly it is focused on the
possibility of purchase. It has a column named as ‘purchased_item’. We have to predict if an item
got purchased or not depending on the country, age and salary of a person.
Here I have performed the following operations on dataset :
Dealing with missing data
Dealing with categorical data
Splitting the dataset into training and testing sets
Scaling the features
Step 1 : Importing The Dataset
dataset = [Link]('[Link]')
Here we can see, this is a simple dataset consisting of four
features. The dependent factor is the ‘purchased_item’ column.
We have to predict if an item got purchased or not depending
on the country, age and salary of a person. Also, the
highlighted cells with value ‘NA’ denotes missing values in
the dataset.
Step 2 : Dealing With Missing Values
dataset$age = ifelse([Link](dataset$age),ave(dataset$age, FUN = function(x)
mean(x, [Link] = 'TRUE')),dataset$age)
dataset$salary = ifelse([Link](dataset$salary), ave(dataset$salary, FUN =
function(x) mean(x, [Link] = 'TRUE')), dataset$salary)
Here It will check for missing values in the age and salary columns and update the missing cells with the column-wise
average.
dataset$column_header : Selects the column in the dataset specified after $ (age and salary).
[Link](dataset$column_header) : This method returns true for all the cells in the specified column with no values.
ave(dataset$column_header, FUN = function(x) mean(x, [Link] = ‘TRUE’)) : Ths method calculates the average of the
column passed as argument.
Output :
In addition, I don’t need decimal places for age so I will round it up using the following code. The argument 0 in the
round function means no decimal places.
dataset$age = [Link](format(round(dataset$age, 0)))
After executing the above code Out Would be following :
Step 3 : Dealing With Categorical Data
Categorical variables represent types of data, which may be divided into groups. For Ex :- gender, age group, educational level
etc.
In our dataset, we have two categorical features, nation, and purchased_item. In R we can use the factor method to convert texts
into numerical codes.
dataset$nation = factor(dataset$nation, levels = c('India','Germany','Russia'), labels = c(1,2,3))
dataset$purchased_item = factor(dataset$purchased_item, levels = c('No','Yes'), labels = c(0,1))
factor(dataset$olumn_header, levels = c(), labels = c()) : the factor method converts the categorical features in the
specified column to factors or numerical codes.
levels: the categories in the column passed as a vector. Example c(‘India’,’Germany’,’Russia’)
labels: The numerical codes for the specified categories in the same order. Example c(1,2,3))
Output:
Step 4 : Splitting The Dataset Into Training And Testing Sets
I am using the caTools library in R to split the dataset to training_set and test_set
[Link]('caTools') #install once
library(caTools) # importing caTools library
[Link](123)
split = [Link](dataset$purchased_item, SplitRatio = 0.8)
training_set = subset(dataset, split == TRUE)
test_set = subset(dataset, split == FALSE)
[Link](): The seed function preserves the uniqueness of the split i.e, for each seed value, the split will be unique. It is
similar to the random_state argument in python.
[Link](dataset$dependent_factor, SplitRatio = 0.8) : This method will return boolean values with the length of
the original dataset in the specified SplitRatio .0.8 gives 80 percentage Trues and 20 percentage Falses. For example, the
above code block will assign the variable split with values [TRUE TRUE TRUE TRUE TRUE FALSE TRUE TRUE
FALSE TRUE TRUE TRUE TRUE TRUE FALSE]
subset(dataset, split == TRUE) : This method will return a subset of the dataset passed as an argument where the split is
True. (80 percent of the original dataset with respect to the given code)
subset(dataset, split == FALSE): This method will return a subset of the dataset passed as an argument where the split is
False. (20 percent of the original dataset with respect to the given code)
Step 5 : Scaling The Features
training_set[,3:4] = scale(training_set[,3:4])
test_set[,3:4] = scale(test_set[,3:4])
The scale method in R can be used to scale the features in the dataset. Here I am scaling only the non-factors which are
the age and the salary.
Output:
1. Training_set:
2. Test_set: