UNIT 1: PYTHON PROGRAMMING-II

Title: Python Programming-II Approach: Hands on, Team Discussion, Web

search, Case studies
Summary:
This chapter provides a comprehensive review of fundamental python programming and
techniques. Students will gain hands-on experience with essential Python libraries,
preparing them for more advanced data analysis and machine learning tasks which can be
incorporated in their capstone project.

Learning Objectives:
1. Review the basics of the NumPy and Pandas library, including arrays, and essential
functions.
2. Efficiently import and export data between CSV files and Pandas Data Frames.
3. Implement Linear Regression algorithm, including data preparation, and model
training.

Key Concepts:
1. Recap of NumPy library
2. Recap of Pandas library
3. Importing and Exporting Data between CSV Files and Data Frames
4. Handling missing values
5. Linear Regression algorithm
Learning Outcomes:
Students will be able to:
1. Apply the fundamental concepts of the NumPy and Pandas libraries to perform data
manipulation and analysis tasks.
2. Import and export data between CSV files and Pandas Data Frames, ensuring data
integrity and consistency.
Prerequisites: Foundational understanding of Python from class XI and familiarity with the
basic programming.

1
1.1. Python Libraries
Python libraries are collections of pre-written codes that we can use to perform common tasks,
making our programming life easier. They are like toolkits that provide functions and methods to
help us avoid writing code from scratch.
In the realm of data science and analytics, two powerful libraries stand out for their efficiency and
versatility: NumPy and Pandas. These libraries form the backbone of data manipulation and
analysis in Python, enabling users to handle large datasets with ease and precision.

Let's recap a few of these libraries (covered in Class XI) that are incredibly valuable in the realm
of Artificial Intelligence, Data science and analytics.

1.1. 1 NumPy Library

NumPy, short for Numerical Python is a powerful library in Python used for numerical computing.
It is a general-purpose array-processing package.

In NumPy, the number of dimensions of the array is called the rank of the array

1.1.2 Pandas Library

Where and why do we use the Pandas library in Artificial Intelligence?

Suppose we have a dataset containing information about various marketing

campaigns conducted by the company, such as campaign type, budget, duration,
reach, engagement metrics, and sales performance. Pandas is used to load the
dataset, display summary statistics, and perform group-wise analysis to understand
the performance of different marketing campaigns. We can visualize the sales
performance and average engagement metrics for each campaign type using
Matplotlib, a popular plotting library in Python.

Pandas provides powerful data manipulation and aggregation functionalities, making

it easy for us to perform complex analyses and generate insightful visualizations. This
capability is invaluable in AI and data-driven decision-making processes, allowing
businesses to gain actionable insights from their data.

2
[Link] Pandas Data Structures
Pandas generally provides two data structures for manipulating data, They are:
● Series
● Data Frame

i) Creation of a Series from Scalar Values- A Series can be created using scalar values as shown
below:

ii) Creation of a DataFrame from NumPy arrays

array1=[Link]([90,100,110,120])
array2=[Link]([50,60,70])
array3=[Link]([10,20,30,40])
marksDF = [Link]([array1, array2, array3], columns=[ 'A', 'B', 'C', 'D'])
print(marksDF)

iii) Creation of a DataFrame from dictionary of array/lists:

import pandas as pd
# intialise data of lists.
data = {'Name':['Varun', 'Ganesh', 'Joseph', 'Abdul','Reena'],
'Age':[37,30,38, 39,40]}
# Create DataFrame
df = [Link](data)
# Print the output.
print(df)

The dictionary keys become column labels by default in a DataFrame, and the lists become the
columns of data.

3
iv) Creation of DataFrame from List of Dictionaries
# Create list of dictionaries
listDict = [{'a':10, 'b':20}, {'a':5,'b':10,'c':20}]
a= [Link](listDict)
print(a)
There will be as many rows as the number of dictionaries present in
the list.

[Link] Dealing with Rows and Columns

i) Adding a New Column to a DataFrame:

ResultSheet={'Rajat': [Link]([90, 91, 97],index=['Maths','Science','Hindi']),
'Amrita': [Link]([92, 81, 96],index=['Maths','Science','Hindi']),
'Meenakshi': [Link]([89, 91, 88],index=['Maths','Science','Hindi']),
'Rose': [Link]([81, 71, 67],index=['Maths','Science','Hindi']),
'Karthika': [Link]([94, 95, 99],index=['Maths','Science','Hindi'])}
Result = [Link](ResultSheet)
print(Result)

To add a new column for another student ‘Fathima’, we can write the following statement:
Result['Fathima']=[89,78,76]
print(Result)

ii) Adding a New Row to a DataFrame:

[Link]['English'] = [90, 92, 89, 80, 90, 88]
print(Result)

4
[Link][] method can also be used to change the data values of a row to a particular value.
For example, to change the marks of science.
[Link]['Science'] = [92, 84, 90, 72, 96, 88]
print(Result)

[Link] Deleting Rows or Columns from a DataFrame

To delete a row, the parameter axis is assigned the value 0 and for deleting a column, the
parameter axis is assigned the value 1.

Result = [Link]('Hindi', axis=0) #delete the row “Hindi”

print(Result)

#delete multiple columns

Result = [Link](['Rajat','Meenakshi','Karthika'], axis=1)
print(Result)

[Link] Attributes of DataFrames

We are going to use following data as example to understand the attributes of a DataFrame.
import pandas as pd
# creating a 2D dictionary
dict = {"Student": [Link](["Arnav","Neha","Priya","Rahul"],
index=["Data 1","Data 2","Data 3","Data 4"]),
"Marks": [Link]([85, 92, 78, 83],
index=["Data 1","Data 2","Data 3","Data 4"]),
"Sports": [Link](["Cricket","Volleyball","Hockey","Badminton"],
index=["Data 1","Data 2","Data 3","Data 4"])}
# creating a DataFrame
df = [Link](dict)
# printing this DataFrame on the output screen
print(df)

5
i) [Link]
>>>[Link]

ii) [Link]
>>>[Link]

iii) [Link]
>>>[Link]
(4,3)
iv) [Link](n)
>>>[Link](2)

v) [Link](n)
>>>[Link](2)

1.2. Import and Export Data between CSV Files and DataFrames

CSV files, which stand for Comma-Separated Values, are simple text files used to store tabular
data. Each line in a CSV file represents a row in the table, and each value in the row is separated
by a comma. This format is widely used because it is easy to read and write, both for humans and
computers.
In Python, CSV files are incredibly important for data analysis and manipulation. We often use the
Pandas library to load, manipulate, and analyze data stored in CSV files. Pandas provides powerful
tools to read CSV files into Data Frames, which are data structures that allow us to perform
complex operations on the data with ease. This makes CSV files a go-to format for data scientists
and analysts working with Python.

6
1.2.1 Importing a CSV file to a DataFrame
Using the read_csv() function, we can import tabular data from CSV files into pandas DataFrame
by specifying a parameter value for the file name (e.g. pd.read_csv("[Link]")).
Let us create a DataFrame from the “[Link]” file.
Follow the following steps to upload the csv file in the google colab file
Step1: Open the Google colab from the following link [Link]
And create a new file from the File menu

Step 3: Click on the ↑ and

select the csv file

Step 2: Click on the Folder

icon

The csv file is uploaded

Once the csv file is uploaded then we can execute the following code to convert the csv to
DataFrame.
import pandas as pd
df=pd.read_csv("[Link]")
print(df)

7
On Python IDE we can directly give the complete path name with the csv file in parenthesis.

import pandas as pd
import io
df = pd.read_csv('C:/PANDAS/[Link]',sep =",", header=0)
print(df)

1.2.2 Exporting a DataFrame to a CSV file

We can use the to_csv() function to save a DataFrame to a text or csv file. For example, to save the
DataFrame df created in above coding
print(df)

Program to export this data-

df.to_csv(path_or_buf='C:/PANDAS/[Link]', sep=',')
This creates a file by the name [Link] in the hard disk. When we open this file in any text editor or
a spreadsheet, we will find the above data along with the row labels and the column headers, separated
by comma.
On Google Colab we can write the following code
df.to_csv("[Link]",index=False)
The [Link] is created

8
1.3. Handling Missing Values
The two most common strategies for handling missing values explained in this section are:
i) Drop the row having missing values OR
ii) Estimate the missing value
Checking Missing Values
Pandas provide a function isnull() to check whether any value is missing or not in the DataFrame.
This function checks all attributes and returns True in case that attribute has missing values,
otherwise returns False
Drop Missing Values
Dropping will remove the entire row (object) having the missing value(s). This strategy reduces
the size of the dataset used in data analysis, hence should be used in case of missing values on
few objects.
The dropna() function can be used to drop an entire row from the DataFrame
Estimate the missing value
Missing values can be filled by using estimations or approximations e.g a value just before (or
after) the missing value, average/minimum/maximum of the values of that attribute, etc. In some
cases, missing values are replaced by zeros (or ones).
The fillna(num) function can be used to replace missing value(s) by the value specified in num.
For example, fillna(0) replaces missing value by 0. Similarly fillna(1) replaces missing value by 1.

1.4. CASE STUDY

Let's examine a scenario where we have the marks of certain students, but some data is missing
in the columns due to specific circumstances. For example, Meera and Suhana couldn't attend the
Science and Hindi respectively exam due to fever, Joseph participated in a national-level science
exhibition on the day of the AI exam.

Let us feed the data in Python

ResultSheet={'Maths': [Link]([90,91,97,89,65,93],
index=['Heena','Shefali','Meera','Joseph','Suhana','Bismeet']),
'Science':[Link]([92,81,[Link],87,50,88],
index=['Heena','Shefali','Meera','Joseph','Suhana','Bismeet']),
'English': [Link]([89, 91, 88,78,77,82],
index=['Heena','Shefali','Meera','Joseph','Suhana','Bismeet']),
'Hindi': [Link]([81, 71, 67,82,[Link],89],
index=['Heena','Shefali','Meera','Joseph','Suhana','Bismeet']),
'AI': [Link]([94, 95, 99,[Link],96,99],
index=['Heena','Shefali','Meera','Joseph','Suhana','Bismeet'])}
marks = [Link](ResultSheet)
print(marks)

9
#check for missing values
>>>print([Link]())

We can see there are three “True” values. So three pieces of data are missing.

>>>print(marks['Science'].isnull().any())
True
print(marks['Maths'].isnull().any())
False
#To find the total number of NaN in the whole dataset
>>>[Link]().sum().sum()
3

#apply dropna() for the above case

drop=[Link]()
print(drop)

#Estimate the missing value

FillZero = [Link](0)
print(FillZero)

10
1.5. PRACTICAL ACTIVITY (**For Advanced Learners)
Activity: To implement Linear Regression algorithm
Dataset: [Link]

import pandas as pd
df=pd.read_csv(‘USA_Housing.csv')
[Link]()

Upon examining the count, it's evident that all columns contain 5000 values. This suggests that
there are no missing values in any of the columns.

EXPLORATORY DATA ANALYSIS

11
From the above output, we understand that 4000 rows (80% of 5000 rows will be used for training
the model)

12
Applying the Linear Regression Algorithm

13
We observe that there is a difference between the actual and predicted value.
Further, we need to calculate the error, evaluate the model and test the accuracy of the model.
This will be covered in the next chapter.

EXERCISES
A. Objective type questions
1. Which of the following is a primary data structure in Pandas?
a) List
b) Tuple
c) Series
d) Matrix

2. What does the fillna(0) function do in Pandas?

a) Removes rows with missing values
b) Fills missing values with zeros
c) Estimates missing values based on averages
d) Converts all data to zero

14
3. In Linear Regression, which library is typically used for importing and managing data?
a) NumPy
b) Pandas
c) Matplotlib
d) Scikit-learn

4. What is the correct syntax to read a CSV file into a Pandas DataFrame?
a) [Link]("[Link]")
b) pd.read_csv("[Link]")
c) pandas.read_file("[Link]")
d) pd.file_read("[Link]")

5. What is the result of the [Link] function?

a) Data type of the DataFrame
b) Number of rows and columns in the DataFrame
c) Memory usage of the DataFrame
d) Column names of the DataFrame

6. Which function can be used to export a DataFrame to a CSV file?

a) export_csv()
b) to_file()
c) to_csv()
d) save_csv()

B. Short Answer Questions

1. What is a DataFrame in Pandas?
2. How do you create a Pandas Series from a dictionary?
3. Name two strategies to handle missing values in a DataFrame.
4. What does the head(n) function do in a DataFrame?
5. What is the role of NumPy in Python programming?
6. Explain the use of the isnull() function in Pandas.

C. Long Answer Questions

1. Describe the steps to import and export data using Pandas.
2. Explain the concept of handling missing values in a DataFrame with examples.
3. What is Linear Regression, and how is it implemented in Python?
4. Compare NumPy arrays and Pandas DataFrames.
5. How can we add new rows and columns to an existing DataFrame? Explain with code
examples.
6. What are the attributes of a DataFrame? Provide examples.

15
D. Case study
1. A dataset of student marks contains missing values for some subjects. Write Python code
to handle these missing values by replacing them with the mean of the respective
columns.

2. Write Python code to load the file into a Pandas DataFrame, calculate the total sales for
each product, and save the results into a new CSV file. Click in the link below to access
[Link] dataset.
[Link]

3. In a marketing dataset, analyze the performance of campaigns using Pandas. Describe

steps to group data by campaign type and calculate average sales and engagement
metrics.

4. A company has collected data on employee performance. Some values are missing, and
certain columns are irrelevant. Explain how to clean and preprocess this data for analysis
using Pandas.

16