BCA SEM- 4 PYTHON WEBSOL UNIT - 4

Unit IV

Pandas: Overview of Series and DataFrames, reading data from csv file, DataFrame
operations working with data using functions like head, tail , info, shape, reshape, columns,
isnull, dropna, mean, sum, describe, value_counts, corr, loc, iloc, apply.

Overview of Series and DataFrames

Pandas is a Python library used for working with data sets.

It has functions for analyzing, cleaning, exploring, and manipulating data.

The name "Pandas" has a reference to both "Panel Data", and "Python Data Analysis" and
was created by Wes McKinney in 2008.

Pandas is open-source Python library which is used for data manipulation and analysis. It
consist of data structures and functions to perform efficient operations on data. It is well-
suited for working with tabular data such as spreadsheets or SQL tables. It is used in
data science because it works well with other important libraries. It is built on top of the
NumPy library as it makes easier to manipulate and analyze. Pandas is used in other
libraries such as:
 Matplotlib for plotting graphs
 SciPy for statistical analysis
 Scikit-learn for machine learning algorithms.
 It uses many functionalities provided by NumPy library.

Here is a various tasks that we can do using Pandas:

 Data Cleaning, Merging and Joining: Clean and combine data from multiple sources,
handling inconsistencies and duplicates.
 Handling Missing Data: Manage missing values (NaN) in both floating and non-
floating point data.
 Column Insertion and Deletion: Easily add, remove or modify columns in a
DataFrame.
 Group By Operations: Use "split-apply-combine" to group and analyze data.

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 1

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

 Data Visualization: Create visualizations with Matplotlib and Seaborn, integrated with
Pandas.

1. Pandas Series
A Pandas Series is one-dimensional labeled array capable of holding data of any type
(integer, string, float, Python objects etc.). The axis labels are collectively called indexes.
Pandas Series is created by loading the datasets from existing storage which can be a SQL
database, a CSV file or an Excel file. It can be created from lists, dictionaries, scalar values,
etc.
import pandas as pd
import numpy as np

ser = [Link]()
print("Pandas Series: ", ser)

data = [Link](*'w', 'e', 'b', 's', 'o',’l’+)

ser = [Link](data)
print("Pandas Series:\n", ser)

2. Pandas DataFrame
Pandas DataFrame is a two-dimensional data structure with labeled axes (rows and
columns). It is created by loading the datasets from existing storage which can be a SQL
database, a CSV file or an Excel file. It can be created from lists, dictionaries, a list of
dictionaries etc.
import pandas as pd

df = [Link]()
print(df)

lst = ['class', 'at', 'websol', 'for', 'bca', 'sem',

'mgsu']

df = [Link](lst)
print(df)

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 2

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

Why Use Pandas

Pandas allows us to analyze big data and make conclusions based on statistical theories.
Pandas can clean messy data sets, and make them readable and relevant.
Relevant data is very important in data science.
Data Science: is a branch of computer science where we study how to store, use and analyze data
for deriving information from it.

import pandas as pd sub: marks

result = { 0 co 75
'sub:': ["co", "c++", "java"], 1 c++ 99
'marks': [75, 99, 59]
} 2 java 59

ans = [Link](result)

print(ans)

Series

A Pandas Series is like a column in a table.

It is a one-dimensional array holding data of any type.

import pandas as pd 0 10

a = [10, 70, 20] 1 70

2 20
ans = [Link](a)
dtype: int64
print(ans)

Labels

If nothing else is specified, the values are labeled with their index number. First value has
index 0, second value has index 1 etc.

This label can be used to access a specified value.

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 3

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

Create Labels

With the index argument, you can name your own labels.

import pandas as pd x 10

a = [10, 70, 20] y 70

z 20
myvar = [Link](a, index = ["x", "y", "z"])
dtype: int64
print(myvar)

import pandas as pd hindi 42

marks = {"hindi": 42, "eng": 38, "maths": 39}

eng 38
maths 39
myvar = [Link](marks)
dtype: int64
print(myvar)

DataFrame

A Pandas DataFrame is a 2 dimensional data structure, like a 2 dimensional array, or a table

with rows and columns.

import pandas as pd id marks

data = { 0 101 50
"id": [101, 102, 103], 1 102 40
"marks": [50, 40, 45]
} 2 103 45

#load data into a DataFrame object:

df = [Link](data)

print(df)

Head:

The head() method in Pandas is used to return the first n rows of a Pandas object, such as a
DataFrame or Series. This method is particularly useful for quickly inspecting the beginning
of a dataset, especially when dealing with large amounts of data

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 4

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

import pandas as pd Name Age City

0 Alice 25 New York
# Create a sample DataFrame 1 Bob 30 London
data = {'Name': ['Alice', 'Bob', 'Charlie', 2 Charlie 22 Paris
'David', 'Eve', 'Frank'],
'Age': [25, 30, 22, 35, 28, 40],
'City': ['New York', 'London', 'Paris',
'Tokyo', 'Berlin', 'Rome']}
df = [Link](data)

# Display the first 3 rows

print([Link](3))

tail()
In pandas, the method is used to retrieve the last n rows of a DataFrame or Series. It is
particularly useful for quickly inspecting the end of a dataset, especially when working with
large amounts of data.

import pandas as pd

data = {'col1': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],

'col2': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J']}
df = [Link](data)

print([Link]())
print([Link](3)) # Returns the last 3 rows

.info()

The .info() method in Pandas provides a concise summary of a DataFrame. This method is
crucial for initial data exploration and understanding the structure of your dataset.

The information contains the number of columns, column labels, column data types, memory
usage, range index, and the number of cells in each column (non-null values).

import pandas as pd <class '[Link]'>

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 5

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

RangeIndex: 4 entries, 0 to 3
data = {'Name': ['Alice', 'Bob', 'Charlie', Data columns (total 3 columns):
'Dave'], # Column Non-Null Count Dtype
'Age': [25, 30, None, 40], --- ------ -------------- -----
'City': ['New York', 'London', 'Paris', 0 Name 4 non-null object
'Tokyo']} 1 Age 3 non-null float64
df = [Link](data) 2 City 4 non-null object
dtypes: float64(1), object(2)
[Link]() memory usage: 192.0+ bytes

shape
In pandas, the shape attribute is used to determine the dimensions of a DataFrame or Series. It
returns a tuple representing the number of rows and columns (for a DataFrame) or the
number of elements (for a Series).

import pandas as pd (3, 2)

d = {"col1" : [1, 2 ,3], "col2" : [4, 5, 6]}

df = [Link](data = d)
print([Link])

reshape() (for Series/NumPy arrays):

While not a direct Pandas DataFrame method, the reshape() method is available for the
underlying NumPy arrays of Pandas Series and DataFrames. It allows changing the
dimensions of the array without altering its data.
import pandas as pd # import pandas library
import numpy as np import pandas as pd
array = [2, 4, 6, 8, 10, 12]
s = [Link]([1, 2, 3, 4, 5, 6]) series_obj = [Link](array)
reshaped_array = [Link]((3, 2)) arr = series_obj.values
print(reshaped_array) # reshaping series
reshaped_arr = [Link]((3, 2))
# show
reshaped_arr

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 6

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

dropna()
The dropna() method in Pandas is used to remove missing values (NaN, None, or NaT) from
a DataFrame or Series. This method is a crucial part of data cleaning and preparation.

import pandas as pd import pandas as pd

import numpy as np
import numpy as np
df = [Link]({'A': [1, [Link], 3], 'B': # Create a sample DataFrame
data = {'A': [1, 2, [Link], 4],
[4, 5, None]})
'B': [5, [Link], 7, 8],
print([Link]()) 'C': [9, 10, 11, [Link]]}
df = [Link](data)
print("Original DataFrame:\n", df)

# Drop rows with any NaN values (default

behavior)
df_rows_dropped = [Link]()
print("\nDataFrame after dropping rows with
any NaN:\n", df_rows_dropped)
import pandas as pd

# create a DataFrame with missing values

data = {'A': [1, 2, None, 4, 5],
'B': [1, 2, 3, None, 5]}

df = [Link](data)

# drop missing values

df_dropped = [Link]()

print(df_dropped)

sum()
In Pandas, the sum() method is used to calculate the sum of values in a DataFrame or
Series. It offers flexibility in summing across different axes and handling missing values.

import pandas as pd import pandas as pd

data = { data = [[10, 18, 11], [13, 15, 8], [9, 20, 3]]
'A': [1, 2, 3],
'B': [4, 5, 6], df = [Link](data)
'C': [7, 8, 9]
} print([Link]())

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 7

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

df = [Link](data)

# Sum along columns (default: axis=0)

col_sum = [Link]()

isnull()

The isnull() function in Pandas is used to detect missing or null values within a Series or
DataFrame. It returns a boolean object of the same shape as the input, where True indicates
the presence of a null value (such as None or NaN), and False indicates a non-null value.

Usage:

 For a DataFrame: [Link]()

 For a Series: [Link]()

import pandas as pd
# Creating the Series
sr = [Link]([11, 21, 8, 18, 65, None, 32, 10, 5,
24, None])
# detect missing values
result = [Link]()

# Print the result

print(result)

mean()
the mean() method is used to calculate the arithmetic mean (average) of data within a
DataFrame or Series.

import pandas as pd

data = {
'Math': [85, 90, 78, 92],
'Physics': [92, 88, 84, 90],
'Chemistry': [75, 80, 85, 70]
}
df = [Link](data)

# Calculate mean of each column

column_means = [Link]()
print("Mean of each column:\n",
column_means)

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 8

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

describe()
In pandas, the describe() method provides a quick statistical summary of a DataFrame or
Series. By default, it generates descriptive statistics for numerical columns, but it can also be
configured to include object (e.g., string) or categorical data.

For Numerical Data:

When applied to numerical columns, describe() returns a DataFrame containing the following
statistics for each column:

 count: The number of non-null entries.

 mean: The average value.

 std: The standard deviation, indicating the spread of the data.

 min: The minimum value.

 25%: The 25th percentile (first quartile, Q1).

 50%: The 50th percentile (median, Q2).

 75%: The 75th percentile (third quartile, Q3).

 max: The maximum value.

import pandas as pd 0 1 2
count 3.000000 3.000000 3.000000
data = [[10, 18, 11], [13, 15, 8], [9, 20, 3]] mean 10.666667 17.666667 7.333333
std 2.081666 2.516611 4.041452
df = [Link](data) min 9.000000 15.000000 3.000000
25% 9.500000 16.500000 5.500000
print([Link]()) 50% 10.000000 18.000000 8.000000
75% 11.500000 19.000000 9.500000
max 13.000000 20.000000 11.000000

columns
In Pandas, [Link] attribute returns the column names of a DataFrame. It gives
access to the column labels, returning an Index object with the column labels that may be
used for viewing, modifying, or creating new column labels for a DataFrame.

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 9

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

import pandas as pd Name of Columns of Pandas DataFrame

Index(['Weight', 'Name', 'Age'], dtype='object')
df = [Link]({
'Weight': [45, 88, 56, 15, 71],
'Name': ['Sam', 'Andrea', 'Alex', 'Robin', 'Kia'],
'Age': [14, 25, 55, 8, 21] })

result = [Link]

print("\n Name of Columns of Pandas

DataFrame")
print(result)

loc and iloc

loc and iloc are two primary methods used for selecting data from a DataFrame or
Series. They differ fundamentally in how they address rows and columns:

 .loc (Label-based indexing):

o loc stands for "location" and uses labels (index names and column names) to select data.

o When slicing with loc, both the start and end labels are inclusive.

o It is suitable for selecting data based on specific row and column names, or for filtering data
based on conditions applied to column values.

 .iloc (Integer-location based indexing):

o iloc stands for "integer location" and uses integer positions (0-based indices) to select data.

o When slicing with iloc, the start position is inclusive, but the end position is exclusive
(similar to standard Python list slicing).

o It is useful when you need to select data based on its absolute position within the DataFrame,
regardless of the labels.

import pandas as pd 20
data = {'col1': [10, 20, 30], 'col2': [40, 50, 60]}
df = [Link](data, index=['rowA', 'rowB', 'rowC'])

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 10

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

# Select row 'rowB' and column 'col1'

print([Link]['rowB', 'col1'])
import pandas as pd 50
data = {'col1': [10, 20, 30], 'col2': [40, 50, 60]}
df = [Link](data, index=['rowA', 'rowB', 'rowC'])
print(df)
# Select the element at row index 1 (rowB) and column index 0 (col1)
print([Link][1, 1])

The loc() function is label based data The iloc() function is an indexed-based
selecting method which means that we have selecting method which means that we have
to pass the name of the row or column to pass an integer index in the method to
which we want to select select a specific row/column.

corr()
Pandas [Link]() is used to find the pairwise correlation of all columns in the
Pandas Dataframe in Python. Any NaN values are automatically excluded. To ignore any
non-numeric values, use the parameter numeric_only = True.

import pandas as pd A B C

# Create a sample DataFrame

A 1.000000 1.000000 -0.173205
data = { B 1.000000 1.000000 -0.173205
'A': [10, 20, 30, 40, 50],
'B': [2, 4, 6, 8, 10], C -0.173205 -0.173205 1.000000
'C': [15, 12, 18, 11, 14]
}
df = [Link](data)

# Calculate the correlation matrix

correlation_matrix = [Link]()
print(correlation_matrix)

The resulting DataFrame displays the correlation coefficients between each pair of
columns. A value close to 1 indicates a strong positive linear relationship, a value close to -1
indicates a strong negative linear relationship, and a value near 0 suggests a weak or no linear
relationship. The diagonal values are always 1, representing the correlation of a column with
itself

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 11

 BCA SEM- 4 PYTHON WEBSOL UNIT - 4

apply()
The apply() function in the Pandas library in Python is a versatile tool used to apply a
function along an axis of a DataFrame or to each element of a Series. It enables efficient data
transformations and manipulations.

import pandas as pd x 120

y 1454
def calc_sum(x): dtype: int64
return [Link]()

data = {
"x": [50, 40, 30],
"y": [300, 1112, 42]
}

df = [Link](data)

x = [Link](calc_sum)

print(x)

value_counts()

The value_counts() method in Pandas is used to count the occurrences of unique values
within a Series or a DataFrame column. It returns a new Series where the index represents the
unique values and the values represent their respective counts. By default, the results are
sorted in descending order of frequency, and NaN (missing) values are excluded.

Python is popular for data analysis thanks to its powerful libraries and Pandas is one of the
best. It makes working with data simple and efficient. The Index.value_counts() function
in Pandas returns the count of each unique value in an Index, sorted in descending order so
the most frequent item comes first. By default, it ignores any missing (NA) values.

import pandas as pd python 2

import numpy as np java 1
php 1
idx = [Link](['python', 'java', 'php','python']) Name: count, dtype: int64
print(idx.value_counts())

Dr. Amit Vyas sir WEBSOL 9214525215 BCA SEM - 4 Page 12