Notes

Unit 02: Python Basics

B. 0, 1, 2, 3, 4
C. 0, 2, 4
D. 1,2,3,4,5

7. How many times the print statement is executed in the givencode?

x=5
while (x <= 5):
print(x)

A. 1 time
B. 5 times
C. Infinite Loop
D. None of the above

8. How many times the print statement is executed in the given code?
x = 10
while (x <= 10):
print(x)
x=x-2

A. 5 times
B. 10 time
C. Infinite Loop
D. None of the above

9. How many times the print statement is executed in the given code?
x = 10
while (x <= 10):
print(x)
continue
x=x-5
break

A. 10 times
B. 2 times
C. Infinite Loop
D. None of the above

10. How many times the print statement is executed in the given code?
x = 10
while (x <= 10):
print(x)
break
x=x-5
continue

LOVELY PROFESSIONAL UNIVERSITY 27

Notes
Notes
Machine Learning

A. 1 time
B. 5 times
C. 10 times
D. Infinite Loop

11. What is the output of the followingcode?

x = 15
y = 20
if ( not(x > y) and (y > x) ):
print("All good")
else:
print("All bad")

A. All good
B. All bad
C. Both (A) and (B)
D. None of the above

12. What is the output of the followingif-elif codes?

x=75
if (x >= 90):
print("Grade O")
elif (x >= 80):
print("Grade A")
elif (x >= 70):
print("Grade B")
elif (x >= 60):
print("Grade C")

A. Grade O
B. Grade A
C. Grade B
D. Grade C

13. What is the output of the following nested-if code? Ans: Grade B
x=90
if (x >= 90):
print("Grade O")
else:
if (x >= 80):
print("Grade A")
else:
if (x >= 70):
print("Grade B")

28 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 02: Python Basics

else:
if (x >= 60):
print("Grade C")

A. Grade O
B. Grade A
C. Grade B
D. Grade C

14. Assume the function already defined as given below.

def MyFunc(n):
sum = 0
for i in range(n):
sum = sum + i
return (sum)
What is the value of x if you execute the following code?
x = MyFunc(10)

A. 0
B. 45
C. 55
D. None of the above

15. Assume the function already defined as given below.

def MyFunct(x, y):
x = x + 10
y = y + 10
return(x,y)
What is the value of t if you execute the following code?
t = MyFunct(10,20)

A. (10, 20)
B. (20, 30)
C. Nothing is printed.
D. None of the above

Answers for Self Assessment

1. D 2. A 3 D 4. B 5. C

6. C 7. C 8. C 9. C 10. A

11. A 12. C 13. C 14. B 15. B

LOVELY PROFESSIONAL UNIVERSITY 29

Notes
Notes
Machine Learning

Review Questions
1. Explain the Datatypes and their functionalities.
2. Differentiate conditional and unconditional statements. Give the respective name of the
statements.
3. Illustrate finite and infinite loop. Give reasons for getting infinite loop.
4. How do you receive the output from the function? Explain with an example.
5. Why do you need Recursive Function? How it stops the recursive operation.

Further readings
 John Zelle, “Python Programming: An Introduction to Computer Science“, Second
Edition, Franklin, Beedle and Associates Inc, 2009.

Web Links

 [Link]
 [Link]
 [Link]
 [Link]
 [Link]

30 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Dr. VDevendran, Lovely Professional University Unit 03: Data Pre-Processing

Unit 03: Data Pre-Processing

CONTENTS
Objectives
Introduction
3.1 Introduction to Data Analysis
3.2 Importing the data
3.3 Summarizing the Dataset
3.4 Data Visualization
3.5 Exporting the data
3.6 Data Wrangling
3.7 Exploratory Data Analysis (EDA)
Summary
Keywords
Self Assessment
Answers for Self Assessment
Review Questions
Further readings

Objectives
 To understandthe concepts of Data Preprocessing and Data Analysis.
 To understandthe fundamentals of datasetand downloading from the website.
 To understand the python code for preprocessing of data.
 To understand the process of data wrangling with examples.
 To know the different aspects of exploratory data analysis.

Introduction
Data preprocessing is a process of preparing the raw data and making it suitable for a machine-
learning model. It is the first and crucial step while creating a machine-learning model because the
real world data generally contains noises, missing values and may be in an unusable format, which
cannot be directly used for machine learning model. Hence, the data preprocessing is required
tasks for cleaning the data and making it suitable for a machine-learning model, which also
increases the accuracy and efficiency of a machine-learning [Link] this unit, we will discuss and
understand the fundamentals of data preprocessing and the necessary steps and approaches in
doing the preprocessing. Also, we explore the concept of data analysis and we try to understand
how the data wrangling and exploratory data analysis helps for effective data preprocessing.

3.1 Introduction to Data Analysis

Data analysis plays a crucial role in processing data in making them as useful information. Data
analysis is the process, which includescleaning the data, changing the data and processing raw data
and extracting relevant information that helps for machine learning. We cannot preprocess
effectively unless we understand better about the data given to us. Hence, the data analysis became

LOVELY PROFESSIONAL UNIVERSITY 31

Notes
Machine Learning

important in the approach of data preprocessing. The process of data analysis consists of the
following steps.

 Gathering the Requirement for Data: This helps you to decide the need for the data, what
type of data you want to use, and what data you plan to analyze.
 Data Collection:It’s time to collect the data from your sources. Data collection will be done
from your identified requirements.
 Data Cleaning: It’s time to clean up the collected data. Assume that some of your collected
data is useful and some of data is not useful. The cleaning techniques are given in Fig 1. This
process is where you remove white spaces, duplicate records, and basic errors. Data cleaning
is mandatory before sending the information for data analysis.

Fig 1. The approaches of data preprocessing

Data Analysis: You will understand the data better and better in all the possible ways. You can also
use tools, which includes Excel, Python, R, Rapid Miner and etc. You will create many graphs /
diagrams as the outcome of your data analysis.
Data Interpretation: Assume that you have your data analysis results. Now, you need to interpret
them and come up with the best courses of action based on your findings.
Data Visualization: This is a fancy way of saying like “graphically show your information in a way
that people can read and understand it.” You can use charts, graphs, maps, bullet points, or other
methods.
There are few more types, which are commonly used in the worlds of technology and business and
the same is given below.
Diagnostic Analysis: This answers the question, “Why did this happen?” Ideally, the analysts find
similar patterns that existed in the past, and consequently, use those solutions to resolve the present
challenges hopefully.
Predictive Analysis: This answers the question, “What is most likely to happen?” By using patterns
found in older data as well as current events, analysts predict future events.
Prescriptive Analysis: Mix all the insights gained from the other data analysis types, and you have
prescriptive analysis. Sometimes, an issue can’t be solved solely with one analysis type, and instead
requires multiple insights.
Statistical Analysis: This answers the question, “What happened?” This analysis covers data
collection, analysis, modeling, interpretation, and presentation using dashboards.
Text Analysis: Also called “data mining,” text analysis uses databases and data mining tools to
discover patterns residing in large datasets. It transforms raw data into useful business information.
Text analysis is arguably the most straightforward and the most direct method of data analysis.
Although there are many data analysis methods available, they all fall into one of two primary
types of data analysis. They are qualitative analysis and quantitative analysis. The qualitative data

32 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 03: Data Pre-Processing

analysis method derives data via words, symbols, pictures, and observations. This method doesn’t
use statistics. But, the Quantitative Data Analysisproduces different numbers as the result of data
analysis with the help of statistical methods. Statistical data analysis methods collect raw data and
process it into numerical data.

3.2 Importing the data

For our practice, we can load the data directly from the UCI Machine Learning repository
([Link] This data /dataset can be imported into python using
pandas as shown below. The dataset downloaded is known as “Pima Indian Dataset” using the
following steps.
Step 1 Declaring the Pandas Library
Step 2 File is assigned to a variable name
Step 3 Assigning the Columns Names or Column Headings.
Step 4 Importing the PIMA Dataset ( File Name is pima_indians.csv )
Observe the following code for importing the dataset using python.
import pandas
data = ‘pima_indians.csv’
names = ['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin', ‘Outcome’]
dataset = pandas.read_csv(data, names=names)

3.3 Summarizing the Dataset

The dataset may be understood by the following observations. It is also known as summary of the
[Link] are given in the bulletins.

 Basic Information about the dataset is obtained from the following code.
print([Link] ())

 Dimensions of Dataset can be obtained using the following code.

print([Link])

 Listing all top 10 data, the following code helps.

print([Link](10))

 Listing all bottom 10 data, the following code helps.

print([Link](10))

 View the Statistical Summary from this code.

print([Link]())

3.4 Data Visualization

Data visualization is the process of representing data using visual elements like charts, graphs, etc.
The data can be better understood if we provide and summarize using the beautiful diagrams,
which is known as data visualization. The sample for the visualization is given in Fig 2. Generally,
there are two types of plots exists and used for data visualization. They are univariate and
multivariate.
Let us explore the first type of visualization i.e., univariate plots.

LOVELY PROFESSIONAL UNIVERSITY 33

Notes
Machine Learning

Fig 2 A Sample Data Visualization

Univariate Plots
Here, 'uni' means one and ‘variate’ indicates a variable. Therefore, univariate plot is a form of
diagram / graph that only involves single [Link] is given in Fig 3.

Fig. 3 An example for univariate plots

You can use the following code for this purpose.
import pandas
import [Link] as plt
data = 'iris_df.csv'
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = pandas.read_csv(data, names=names)
[Link](kind='box', subplots=True, layout=(2,2), sharex=False, sharey=False)
[Link]()
You can create a histogram of each input variable to get an idea of the distribution using the
commands shown below:
[Link]()
[Link]()

Multivariate Plots
Multivariate plots help us to understand the interactions between the variables. For example, we
look at different variables (or factors) and how they might impact certain situations or outcomes.

34 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 03: Data Pre-Processing

Consider the marketingscenario, you might look at how the variable_1i.e., “money spent on
advertising” impacts the variable_2 i.e., “number of sales”. Here, we are considering two variables
for the analysis and the same is put up in the visualization just like Fig 4.

Fig. 4 A sample for multivariate plots

3.5 Exporting the data

After the data preprocessing is completed, we need to store the updated / modified data into the
hard drive permanently. For example, the data should move from the python Jupyter to Hard Disk.
The most common format is a csv file or excel [Link] built in functions to_csv() and to_excel() of
pandas can be used in order to export [Link] syntax can be understood from the following code.

 Exporting data as a csv file

df.to_csv('C:/Deva/[Link]')

 Exporting data as a excel file

df.to_excel('C:/Deva/[Link] ')
Consider the code for creating a dataframe and exporting the contents into permanent file.
import pandas as pd
data = {'product': ['computer', 'tablet', 'printer', 'laptop'], 'price': [850, 200, 150, 1300]}
df = [Link](data)
df.to_csv('C:\MyFolder\[Link]', index=False, header=True)
print(df)

3.6 Data Wrangling

Data wrangling is one of the most important tasks in Machine Learning and also in data science.
The process of gathering, collecting and transforming the original / raw data into another format is
called data wrangling. This is made for better understanding, better decision-making, better
accessing and better analysis in less time. There are few concepts that can help for effective data
wrangling.
Data exploration: In this process, the data is studied, analyzed and understood by visualizing
representations of data.
# Import pandas package
import pandas as pd
# Assign data
data = {'Name': ['Jai', 'Princi', 'Gaurav','Anuj', 'Ravi', 'Natasha', 'Riya'],'Age': [17, 17, 18, 17,
18, 17, 17],'Gender': ['M', 'F', 'M', 'M', 'M', 'F', 'F'],'Marks': [90, 76, 'NaN', 74, 65, 'NaN', 71]}
# Convert into DataFrame

LOVELY PROFESSIONAL UNIVERSITY 35

Notes
Machine Learning

df = [Link](data)
# Display data
df
Dealing with missing values:
# Compute average
c = avg = 0
for ele in df['Marks']:
if str(ele).isnumeric():
c += 1
avg += ele
avg /= c
# Replace missing values
df = [Link](to_replace="NaN",
value=avg)
Reshaping data:
# Categorize gender
df['Gender'] = df['Gender'].map({'M': 0,'F': 1, }).astype(float)
Filtering data:
# Filter top scoring students
df = df[df['Marks'] >= 75]

# Remove age row

df = [Link](['Age'], axis=1)
Merge operation is used to merge raw data and into the desired format as follows. Here the field is
the name of the column, which is similar on both data-frame.
[Link]( data_frame1,data_frame2, on="field ")
WRANGLING DATA USING MERGE OPERATION
# Import module
import pandas as pd
# Creating Dataframe
details = [Link]({ 'ID': [101, 102, 103, 104, 105,106, 107, 108, 109, 110],
'NAME': ['Jagroop', 'Praveen', 'Harjot','Pooja', 'Rahul', 'Nikita', 'Saurabh', 'Ayush', 'Dolly',
"Mohit"],'BRANCH': ['CSE', 'CSE', 'CSE', 'CSE', 'CSE','CSE', 'CSE', 'CSE', 'CSE', 'CSE']})
# Creating Dataframe
fees_status = [Link]( {'ID': [101, 102, 103, 104, 105,106, 107, 108, 109,
110],'PENDING': ['5000', '250', 'NIL','9000', '15000', 'NIL','4500', '1800', '250', 'NIL']})
# Merging Dataframe
print([Link](details, fees_status, on='ID'))
DETAILS STUDENTS DATA WHO WANT TO PARTICIPATE IN THE EVENT:
# Import module
import pandas as pd
# Initializing Data

36 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 03: Data Pre-Processing

student_data = {'Name': ['Amit', 'Praveen', 'Jagroop','Rahul', 'Vishal', 'Suraj','Rishab',

'Satyapal', 'Amit', 'Rahul', 'Praveen', 'Amit'],'Roll_no': [23, 54, 29, 36, 59, 38,12, 45, 34, 36, 54,
23],'Email': ['xxxx@[Link]', 'xxxxxx@[Link]','xxxxxx@[Link]',
'xx@[Link]','xxxx@[Link]', 'xxxxx@[Link]','xxxxx@[Link]',
'xxxxx@[Link]','xxxxx@[Link]', 'xxxxxx@[Link]','xxxxxxxxxx@[Link]',
'xxxxxxxxxx@[Link]']}
# Creating Dataframe of Data
df = [Link](student_data)
# Printing Dataframe
print(df)
DATA WRANGLED BY REMOVING DUPLICATE ENTRIES:
# import module
import pandas as pd
# initializing Data
student_data = {'Name': ['Amit', 'Praveen', 'Jagroop', 'Rahul', 'Vishal', 'Suraj', 'Rishab',
'Satyapal', 'Amit','Rahul', 'Praveen', 'Amit'],'Roll_no': [23, 54, 29, 36, 59, 38,12, 45, 34, 36, 54,
23],'Email': ['xxxx@[Link]', 'xxxxxx@[Link]', 'xxxxxx@[Link]', 'xx@[Link]',
'xxxx@[Link]', 'xxxxx@[Link]','xxxxx@[Link]',
'xxxxx@[Link]','xxxxx@[Link]', 'xxxxxx@[Link]', 'xxxxxxxxxx@[Link]',
'xxxxxxxxxx@[Link]']}
# creating dataframe
df = [Link](student_data)
# Here [Link]() function displays duplicate entries in Rollno column.
non_duplicate = df[~[Link]('Roll_no')]
# printing non-duplicate values
print(non_duplicate)

3.7 Exploratory Data Analysis (EDA)

Thisconcept isused by data scientists to analyze and investigate the datasets. They will summarize
the important characteristics of the dataset using data visualization methods. This analysis will
make data scientists very easy and comfortable to discover some new patterns, spot anomalies in
the existing patterns, test their hypothesis and etc.,The main purpose of EDA is to help look at data
before making any assumptions. It can help identify obvious errors, as well as better understand
patterns within the data, detect outliers or anomalous events, find interesting relations among the
[Link] scientists can use exploratory analysis to ensure the results they produce are valid
and applicable to any desired business outcomes and goals. Analysis also helps stakeholders by
confirming they are asking the right questions. Analysis can help answer questions about standard
deviations, categorical variables, and confidence intervals. Once Analysis is complete and insights
are drawn, its features can then be used for more sophisticated data analysis or modeling, including
machine [Link] are four primary types of Exploratory Data Analysis, which are given
below.
Univariate non-graphical:
The data that come from making a particular measurement on all of the subjects in a sample
represent our observations for a single characteristic such as age, gender, speed at a task, or
response to a stimulus.
Univariate graphical:
Graphical methods are required to provide a full picture of the data. The few commonly used
methods are as follows. Stem-and-leaf plots, which show all data values and the shape of the
distribution. Histograms, a barplot in which each bar represents the frequency (count) or

LOVELY PROFESSIONAL UNIVERSITY 37

Notes
Machine Learning

proportion (count/total count) of cases for a range of [Link] plot, which graphically depicts the
five-number summary of minimum, first quartile, median, third quartile, and maximum.
Multivariate non-graphical:
Multivariate data arises from more than one variable. Multivariate non-graphical EDA techniques
generally show the relationship between two or more variables of the data through cross-tabulation
or statistics.
Multivariate graphical:
Multivariate data uses graphics to display relationships between two or more sets of data. The most
used graphic is a grouped bar plot or bar chart with each group representing one level of one of the
variables and each bar within a group representing the levels of the other [Link] common
types of multivariate graphics include, Scatter plot, which is used to plot data points on a
horizontal and a vertical axis to show how much one variable is affected by [Link]
chart, which is a graphical representation of the relationships between factors and a [Link]
chart, which is a line graph of data plotted over [Link] chart, which is a data visualization that
displays multiple circles (bubbles) in a two-dimensional [Link] map, which is a graphical
representation of data where values are depicted by color.

Summary
 The concepts of Data Analysis areintroduced.
 We understood the fundamentals of dataset and downloading from the website.
 The process of data wrangling is discussed with examples.
 We came to know about different aspects of exploratory data analysis and their types.
 We have seen the necessary python code for preprocessing, data visualization and others.
 Necessary Python code is given to explain the data preprocessing and other relevant concepts.

Keywords
 Data Analysis
 Import and Export
 Data Preprocessing
 Data Wrangling
 Exploratory Data Analysis

Self Assessment
1. Data Analysis is a process of?
A. Inspecting data
B. Cleaning data
C. Transforming data
D. All of the above

2. How many main statistical methodologies are used in data analysis?

A. 2
B. 3
C. 4
D. 5

3. Data Analytics uses ______________ to get insights from data.

38 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 03: Data Pre-Processing

A. Statistical figures
B. Numerical aspects
C. Statistical methods
D. None of the mentioned above

4. Text Analytics, also referred to as Text Mining?

A. True
B. False

5. Correlation is the relationship between ______ variables.

A. One
B. Two
C. Zero
D. All of the mentioned above

6. Which of the following is the correct extension of the Python file?

A. .python
B. .pl
C. .py
D. .p

7. Which keyword is used for function in Python language?

A. Function
B. def
C. Fun
D. Define

8. What is a hypothesis?
A. A statement that the researcher wants to test through the data collected in a study
B. A research question the results will answer
C. A theory that underpins the study
D. A statistical method for calculating the extent to which the results could have happened by
chance

9. Customer analytics refers to _________.

A. Customer Relationship Management: churn analysis and prevention
B. Marketing: cross-sell, up-sell
C. Pricing: leakage monitoring, promotional effects tracking, competitive price responses
D. All of the mentioned above

10. Which of the following graph can be used for simple summarization of data?
A. Scatter plot
B. Overlaying
C. Bar plot
D. All of the mentioned

LOVELY PROFESSIONAL UNIVERSITY 39

Notes
Machine Learning

11. Result analysis are relatively easy to replicate or reproduce.

A. True
B. False

12. Which of the following gave rise to need of graphs in data analysis?
A. Data visualization
B. Communicating results
C. Decision making
D. All of the mentioned

13. What will be output of given code?

df = [Link]( { ‘c1’ : [ 12, 34, 45], ‘c2’ : [32, 21, 44], ‘c3’ : [74, 41, 20] } )
print([Link])

A. Index( [0,1,2], dtype = ‘int’ )

B. RangeIndex( start=0, stop=3, step=1 )
C. 012
D. None of the above

14. The plot method on Series and Data Frame is just a simple wrapper around _______.
A. [Link]()
B. [Link]()
C. [Link]()
D. None of the mentioned

15. Which of the following is not true about series and data frames?
A. Both are size mutable.
B. Both can be derived from pandas.
C. Both can be reshaped into different forms.
D. Both can be created by passing data in form of list, dictionaries and ndarray.

Answers for Self Assessment

1. D 2. A 3 C 4. A 5. B

6. C 7. B 8. A 9. D 10. C

11. B 12. D 13. B 14. B 15. A

Review Questions
1. Explain the importance of data analysis.
2. Give the different approaches for data cleaning.
3. Give the python code for importing the data from UCI repository.
4. Differenciateunivariate and multivariate analysis with examples.
5. Whydata wrangling is used?Give the various steps involved in this.

40 LOVELY PROFESSIONAL UNIVERSITY

 Notes
Unit 03: Data Pre-Processing

6. How to remove the duplicate entries from the dataset?

7. Illustrate the fundamentals of exploratory data analysis.
8. Give the types of exploratory data analysis.

Further Readings
 John Zelle, “Python Programming: An Introduction to Computer Science“, Second
Edition, Franklin, Beedle and Associates Inc, 2009.
 Applied Machine Learning by MadanGopal, McGraw Hill Education, India, 2018.
 Machine Learning by Tom Mitchell, McGraw Hill Education, India, 2017.
 Principles of Soft Computing by S. N. Sivanandam and S. N. Deepa, Wiley, India, 2018.

Web Links

 [Link]
 [Link]
 [Link]
 [Link]
 [Link]
visualization-techniques-in-data-science/

LOVELY PROFESSIONAL UNIVERSITY 41

 Notes

Dr. VDevendran, Lovely Professional University Unit 04: Implementation of Pre-processing

Unit 04 : Implementation ofPre-processing

CONTENTS
Objectives
Introduction
4.1 Importing the Data
4.2 Summarizing the Dataset
4.3 Data Visualization
4.4 Exporting the Data
4.5 Data Wrangling
Summary
Keywords
Self Assessment
Answers for Self Assessment
Review Questions
Further Readings

Objectives
 To implementthe concepts of Data Preprocessing and Data Analysis.
 To implementthe importing and exporting of datasets.
 To understand the python code for preprocessing of data.
 To draw different types of graphs using matplotlib and pandas packages.
 To understand the process of data wrangling with examples.

Introduction
Data preprocessing is a process of preparing the raw data and making it suitable for a machine-
learning model. It is the first and crucial step while creating a machine-learning model because the
real world data generally contains noises, missing values and may be in an unusable format, which
cannot be directly used for machine learning model. Hence, the data preprocessing is required
tasks for cleaning the data and making it suitable for a machine-learning model, which also
increases the accuracy and efficiency of a machine-learning [Link] this unit, we will discuss and
understand the fundamentals of data preprocessing and the necessary steps and approaches in
doing the preprocessing. Also, we explore the concept of data analysis and we try to understand
how the data wrangling and exploratory data analysis helps for effective data preprocessing.

4.1 Importing the Data

For our practice, we can load the data directly from the UCI Machine Learning repository
([Link] We have downloaded the Iris dataset from this and
stored in our Desktop. The actual path is mentioned to read the file for importing into python as in
Fig 1.

42 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Fig 1. Python code for reading dataset

4.2 Summarizing the Dataset

The dataset may be understood from the [Link] Information about the dataset is
obtained from the following code as in Fig 2.

Fig 2. Python code for reading basic information

Dimensions of Dataset can be obtained using the following code as in Fig 3.

Fig 3. Python code for shape of the dataset

Listing all top 10 data, the following code helps as in Fig 4.

LOVELY PROFESSIONAL UNIVERSITY 43

 Notes

Unit 04: Implementation of Pre-processing

Fig 4. Python code for getting top 10 rows of dataset

Listing all bottom 10 data, the following code helps as in Fig 5.

Fig 5. Python code for getting bottom 10 rows of dataset

View the Statistical Summary from this code as in Fig 6.

Fig 6. Python code for getting statistical data of a dataset

4.3 Data Visualization

Data visualization is the process of representing data using visual elements like charts, graphs,
[Link] data can be better understood if we provide and summarize using the beautiful diagrams,
which is known as data [Link], there are two types of plots exists and used for
data visualization. They are univariate and multivariate. We can select first four columns from the
iris data set for the visualization using iloc function as in Fig 7.

44 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Fig 7. Getting First four columns from the iris data set

The last column is the target / labels used for classification purposes. The following code will help
us if we want to know what it is. We can use it if it is necessary. Fig 8 depicts the python code.

Fig 8. Getting Last column from the iris data set

Univariate Plots
Let us explore the first type of visualization i.e., univariate plots. Here, 'uni' means one and‘variate’
indicates a variable. Therefore, univariate plot is a form of diagram / graph that only involves
single variable as in Fig 9, 10 and 11.

LOVELY PROFESSIONAL UNIVERSITY 45

 Notes

Unit 04: Implementation of Pre-processing

Fig 9 Bar Chart

Fig 10 Histogram Graphs

Fig 11 Box Charts

46 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Multivariate Plots
Multivariate plots help us to understand the interactions between the variables. Here, we are
considering two variables for the analysis and the same is put up in the visualization using
matplotlib package as in Fig 12.

Fig 12 Scatter plots

4.4 Exporting the Data

After the data preprocessing is completed, we need to store the updated / modified data into the
hard drive permanently. For example, the data should move from the python Jupyter to Hard Disk.
The most common format is a csv file or excel [Link] built in functions to_csv() and to_excel() of
pandas can be used in order to export [Link] syntax can be understood from the following code.
Exporting data as a csv fileas in Fig 13.

Fig 13 Exporting the data as csv file

Exporting data as a excel file as in Fig 14.

LOVELY PROFESSIONAL UNIVERSITY 47

 Notes

Unit 04: Implementation of Pre-processing

Fig 14 Exporting data as Excel File.

4.5 Data Wrangling

Data wrangling is one of the most important tasks in Machine Learning and also in data science.
The process of gathering, collecting and transforming the original / raw data into another format is
called data wrangling. This is made for better understanding, better decision-making, better
accessing and better analysis in less time. There are few concepts that can help for effective data
wrangling. Let us consider this below for converting dictionary data type into dataframe data type.
This helps to explore more on data using available python libraries as in Fig 15.

Fig 15 Conversion into DataFrame

Dealing with missing values:

48 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Fig 16 Calculating Average of Marks

Fig 17 Replacing Nan with Average Value

Fig 18 Giving 0 or 1 for Gender Column

Fig 19 Filtering the Data

WRANGLING DATA USING MERGE OPERATION

LOVELY PROFESSIONAL UNIVERSITY 49

 Notes

Unit 04: Implementation of Pre-processing

Fig 20 Sample preparation of dataframe from “data”

Fig 20 Sample preparation of dataframe from “details”

Fig 21 Sample preparation of dataframe from “fees_status”

Fig 22 Merging of dataframe from “details” and “fees_details”

DETAILS STUDENTS DATA

50 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Fig 23 Student Data before duplicate checking

DATA WRANGLED BY REMOVING DUPLICATE ENTRIES:

Fig 24 Student Data without duplicates

Summary
 Implemented the concepts of Data Preprocessing and Data Analysis.
 Implemented the importing and exporting of datasets.
 Understood the python code for preprocessing of data.
 We understood how to draw different types of graphs using matplotlib and pandas
packages.
 Understood the process of data wrangling with examples.

Keywords
 Import and Export
 Data Preprocessing
 Pandas
 Matplotlib
 Data Wrangling

LOVELY PROFESSIONAL UNIVERSITY 51

 Notes

Unit 04: Implementation of Pre-processing

52 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Self Assessment
Q1) Data Analysis is a process of?
A. Inspecting data
B. Cleaning data
C. Transforming data
D. All of the above

Q2) How many main statistical methodologies are used in data analysis?
A. 2
B. 3
C. 4
D. 5

Q3) Data Analytics uses ______________ to get insights from data.

A. Statistical figures
B. Numerical aspects
C. Statistical methods
D. None of the mentioned above

Q4) Text Analytics, also referred to as Text Mining?

A. True
B. False

Q5) Correlation is the relationship between ______ variables.

A. One
B. Two
C. Zero
D. All of the mentioned above

Q6) Which of the following is the correct extension of the Python file?

A .python
B .pl
C .py
D .p

Q7) Which keyword is used for function in Python language?

A Function
B def
C Fun
D Define

LOVELY PROFESSIONAL UNIVERSITY 53

 Notes

Unit 04: Implementation of Pre-processing

Q8) What is a hypothesis?

A. A statement that the researcher wants to test through the data collected in a study
B. A research question the results will answer
C. A theory that underpins the study
D. A statistical method for calculating the extent to which the results could have happened by
chance

Q9) Customer analytics refers to _________.

A. Customer Relationship Management: churn analysis and prevention
B. Marketing: cross-sell, up-sell
C. Pricing: leakage monitoring, promotional effects tracking, competitive price responses
D. All of the mentioned above

Q10) Which of the following graph can be used for simple summarization of data?

A Scatter plot
B Overlaying
C Bar plot
D All of the mentioned

Q11) Result analysis are relatively easy to replicate or reproduce.

A True
B False

Q12) Which of the following gave rise to need of graphs in data analysis?

A Data visualization
B Communicating results
C Decision making
D All of the mentioned

Q13)What will be output of given code?

df = [Link]( { ‘c1’ : [ 12, 34, 45], ‘c2’ : [32, 21, 44], ‘c3’ : [74, 41, 20] } )
print([Link])

A Index( [0,1,2], dtype = ‘int’ )

B RangeIndex( start=0, stop=3, step=1 )
C 012
D None of the above

Q14)The plot method on Series and DataFrame is just a simple wrapper around _______.

A [Link]()
B [Link]()
C [Link]()
D None of the mentioned

54 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Q15)Which of the following is not true about series and dataframes?

A Both are size mutable.

B Both can be derived from pandas.
C Both can be reshaped into different forms.
D Both can be created by passing data in form of list, dictionaries and ndarray.

Answers for Self Assessment

1. D 2. A 3 C 4. A 5. B

6. C 7. B 8. A 9. D 10. C

11. B 12. D 13. B 14. B 15. A

Review Questions
1. Explain the importance of data analysis.
2. Give the different approaches for data cleaning.
3. Give the python code for importing the data from UCI repository.
4. Differentiateunivariate and multivariate analysis with examples.
5. Whyis data wrangling used?Give the various steps involved in this.
6. How to remove the duplicate entries from the dataset?
7. Illustrate the fundamentals of exploratory data analysis.
8. Give the types of exploratory data analysis.

Further Readings
John Zelle, “Python Programming: An Introduction to Computer Science“, Second
Edition, Franklin, Beedle and Associates Inc, 2009.
Applied Machine Learning by MadanGopal, McGraw Hill Education, India, 2018.
Machine Learning by Tom Mitchell, McGraw Hill Education, India, 2017.
Principles of Soft Computing by S. N. Sivanandam and S. N. Deepa, Wiley, India, 2018.

Web Links

 [Link]
 [Link]
 [Link]
 [Link]
m
 [Link]
visualization-techniques-in-data-science/

LOVELY PROFESSIONAL UNIVERSITY 55

 Notes
Dr. Rajni Bhalla, Lovely Professional University Unit 05: Physical Layer

Unit 05: Physical Layer

CONTENTS
Objectives
Introduction
5.1 What is the Purpose of a Regression Model?
5.2 Types of Regression Analysis
5.3 Multiple Linear Regression
5.4 Assumptions for Multiple Linear Regression
Summary
Keywords
Self Assessment
Answers for Self Assessment
Review Questions
Further Readings

Objectives
 learn what is regression analysis.
 understand the purpose of regression analysis.
 learn different types of regression analysis

Introduction
Regression analysis is the method that is most frequently used to address regression issues in
machine learning. It is based on data modelling and comprises choosing the line that fits the data
the best and travels the least distance between each data point while passing through all of the data
points. Although there are other regression analysis methods, logistic and linear regression are the
most frequently employed. In the end, the nature of the data will dictate the kind of regression
analysis model we use.

5.1 What is the Purpose of a Regression Model?

When knowledge of the independent variables is available, regression analysis is used to either
predict the value of the dependent variable or to determine how an independent variable will affect
the dependent variable.

5.2 Types of Regression Analysis

There are several regression analysis prediction methods accessible. The number of independent
variables, the shape of the regression line, and the kind of dependent variable are other factors that
influence the approach choice.

56 LOVELY PROFESSIONAL UNIVERSITY

Notes
Machine Learning

Figure 1 Types of regression Analysis

1. Linear Regression
Linear regression, which presumes a linear relationship between a dependent variable (Y) and
an independent variable (X), is the modelling technique that is most frequently utilised. It uses
a best-fit line, commonly referred to as a regression line. Y = c+m*X + e, where 'c' stands for the
intercept,'m' for the line's slope, and 'e' for the error term, is the formula for the linear
relationship.
One dependent variable and more than one independent variable can be used in a complex
linear regression model, which can be simple (just one dependent variable and one
independent variable).

Figure 2 Linear Regression

2. Logistic Regression
The logistic regression method is appropriate when the dependent variable is discrete. In
other words, this method is used to determine the likelihood of events that are mutually
exclusive, such as pass/fail, true/false, 0/1, and so on. Thus, the probability has a value

LOVELY PROFESSIONAL UNIVERSITY 57