Data Engineering with

Python
Dachuri Chaitanya
[Link] (IIT Roorkee), AI Expert, 20 yrs Exp.
[Link]
 Faculty Introduction
• Post graduate from IIT Roorkee

• 20 yrs Industry Experience

• 10+ yrs Teaching Experience

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Course Specialties
Session recording Code notebook

Course Material Assignments

Highlight IMP Question hour

Essence from experience Accessible Trainer

1 month Affordable price

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Download PYTHON
[Link]

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Install PYTHON
1 2

3 4

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Verify PYTHON
In Command prompt: In Start Menu:

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 What is PYTHON ?
• Python is an interpreted, object-oriented, high level programming language.

Compiled PL Interpreted PL Non OOPL OOPL HL PL LL PL

PL does not
PL supports OOP Machine/Byte
Program Program supports OOP Program
(written by developer) (written by developer) features (written by developer) Code
features
Ex: C Ex: C++, Java, Python Ex: Perl, Python, Ruby Ex: Assembly Language
Compiling
Interpret & run
Line by line.
Machine/Byte Interpretation OOP Features
Code generates byte - Objects
code, saves in - Encapsulation
RAM/Drive
Running - Inheritance
(Faster) - Polymorphism

Output Output
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
Ex: C, C++, Java Ex: JS, Perl, Python
 Why PYTHON ?
• Developer Productivity • Huge Support Libraries

• Portable • Open Source

• Software Quality • Powerful

• Easy to Learn • Easy to Use

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 PYTHON is… Interpreter
P V M : Python Virtual Machine

Software code
package

PYTHON P
[Link] or V Output
Python M
Interpreter

Software code
Source code byte code
package
1. Reads line by line of source code ([Link])
[Link] Output
2. Generates byte code for every line ([Link]) [Link] PVM

3. Executes the byte code by PVM

Python Interpreter

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 How to run PYTHON script
python [Link] To get o/p using this command

Using windows command prompt 1) Python installation path must be set in PATH system variable.
2) Source code ([Link]) must be available in CWD: C:\Users\Lenovo

python [Link]
or
Executes the python D:\code\[Link] To get o/p using this command

py [Link] python script

1) Python installation path must be set in PATH system variable.
2) NO need to have source code ([Link]) in CWD.
Current Working Run Command
Directory (CWD) syntax
Example:
py [Link] To get o/p using this command

1) NO need to set python installation path in PATH environment

variable
2) Source code ([Link]) must be available in CWD.

py D:\code\[Link] To get o/p using this command

1) NO need to set python installation path in PATH environment variable.

Output
2) NO need to have
(Print statements) D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
source code ([Link]) in CWD.
numpy

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 ndarrays, ndarray vs List
A numpy array is a grid of homogeneous values.

numpy array Python List

Homogeneous Data Heterogeneous Data

Fixed Length (No elements change possible) Variable Length (add elements, delete elements, change elements)

Supports vectorized operations Does not supports vectorized operations

Faster results even with huge datasets Slow results with huge data

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Create ndarrays
array(list/tuple/array):
[Link]([1, 2, 3]),
[Link]([[1, 2, 3], [4, 5, 6]])

asarray(ndarray) – [Link](arr1)

arange(int) – [Link](10), [Link](5, 15)

ones(…) – [Link](7), [Link]((2, 6))

ones_like(ndarray) – np.ones_like(arr1)

zeros(…) – [Link](5), [Link]((2, 7))

zeros_like(ndarray) – np.zeros_like(arr1)

eye(int) – [Link](6)

indentity(int) – [Link](7)

[Link] – shape of the array. Ex: (2, 3) array() - [Link]([1, 2, 3], dtype=np.float64)

[Link] – dimension of the array. Ex: 2 arange() – [Link](7, dtype=np.float64)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert) Creates a new array with same
[Link] – data type of elements in array ar2 = [Link](np.float64) elements of arr1 and given dtype
 Arithmetic, comparison, indexing operations
arr + 2 arr2d = [Link]([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

arr * arr arr2d[0][2], arr2d[0, 2]

arr – 4 Arithmetic arr2d[0:2], arr2d[1:], arr2d[:2], arr2d[:]

arr – arr
Accessing rows (elements) of 2D array
1/arr

arr**2 arr2d[0:2, 1:], arr2d[1:, 2:], arr2d[1, :2]

Accessing sub
Indexing, slicing
arr1 < arr2
portion of 2D array. – 2D array
arr == 0
Comparison

arr[5] Indexing, slicing

– 1D array
arr[5:8]
View of arr. Not a new array.
arr[5:8] = 12 Common for 1D, 2D, 3D

arr[:8], arr[5:], arr[:]

Common for 1D, 2D, 3D
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
arr[5:8].copy() To create a new array
 Contd…
Indexing, Matrix, Random operations
arr3d = [Link]([[[1, 2, 3], [4, 5, 6]], arr2d = [Link]((8, 4))
[[7, 8, 9], [10, 11, 12]]]) for i range(8):
Indexing, slicing arr2d[i] = i
arr3d[0], arr3d[1, 0] – 3D array
arr2d

names = [Link](['bob', 'joe', 'will', arr2d[[4, 3, 0, 6]]

'bob', 'will', 'joe', 'joe’]) Fancy Indexing
arr2d[[-3, -5, -7]] = 0

data = [Link]([[4, 7], [0, 2], [-5, 6], [0, arr2d[[1, 5, 7], [0, 3, 1]] = 0 ➔ (1, 0), (5, 3), (7, 1)
0], [1, 2], [-12, -4], [3, 4]])

names == ‘bob’, names != ‘joe’, ~(names == arr = [Link](15).reshape((3, 5)) Matrix operations
‘will’)
arr.T Matrix Transpose. Matrix multiplication
data[names==‘bob’]
[Link](arr1, arr2) or arr1.T @ arr2
data[~(names==‘joe’)]=7 samples = [Link].standard_normal(size=(4, 4))

data[data<0] = 0 rng = [Link].default_rng(seed=2709) Random number

generator
data[(names==‘bob’)|(names==‘will’)] data = rng.standard_normal((2, 3))

data[names==bob, 1] Boolean #Generating array of 2, 3 shape where every element is randomly choosen between 0 to 100
Indexing data(IIT=Roorkee
D. Chaitanya Reddy [Link](100,
Alumni, AI Expert) size=(2, 3))
data[names==‘joe’, 1:] data = [Link](100, size=15)
 Universal Functions
Takes 2 arrays and
[Link](arr), [Link](arr) [Link](arr1, arr2) returns single array

[Link](arr),
Unary ufunc [Link](arr1, 4)

[Link](arr) [Link](arr1, arr2) Binary ufunc

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Conditional Logic, Statistical
xarr = [Link]([1.1, 1.2, 1.3, 1.4, 1.5]) rng = [Link].default_rng(seed=78322)
arr = rng.standard_normal((5, 4))
yarr = [Link]([2.1, 2.2, 2.3, 2.4, 2.5])

cond = [Link]([True, False, True, True, False])

Regular python code:

[(x if c else y) for x, y, c in zip(xarr, yarr,
cond)] [Link](arr) → -0.38142524555500124

[Link](arr), [Link]()
Array operation code:
rarr = [Link](cond, xarr, yarr) [Link](arr), [Link](axis=1), [Link](axis=0)

rarr = [Link](cond, xarr, 2) [Link](arr), [Link](axis=0), [Link](axis=1)

rarr = [Link](cond, 4, yarr)

rarr = [Link](cond, 2, -2)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Boolean array methods, Sorting, Set Logic
arr = rng.standard_normal(100) names = [Link](['bob', 'joe', 'will', 'bob', 'will',
'joe', 'joe’])
(arr > 0).sum() #no of positive values
[Link](names)
(arr < =0).sum() #no of non positive values

values = [Link]([6, 0, 0, 3, 2, 5, 6])

bools = [Link]([False, False, True, False])
result = np.in1d(values, [2, 3, 6]) #returns boolean array
[Link]() #checks atleast one True Boolean array
[Link]() #checks all are True
methods ages = [Link]([4, 2, 2, 5, 6, 10, 8])

arr = rng.standard_normal(6) np.intersect1d(values, ages)

Sorting
Set Logic on arrays
[Link](), [Link](arr) np.union1d(values, ages)

arr2d = rng.standard_normal((5, 3))

[Link](axis=0), [Link](axis=1)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Reshaping, concatenation, split, repeat
[Link]((arr1, arr2)) Same as
arr = [Link](18) [Link]([arr1, arr2], axis=0)
Reshaping [Link]((arr1, arr2)) Same as
[Link](6, 3) [Link]([arr1, arr2], axis=1)

[Link](6, 3).reshape(2, 9) np.r_[arr1, arr2] Same as [Link]((arr1, arr2))

If -1 Inferred from data, which will np.c_[arr1, arr2] Same as [Link]((arr1, arr2))
[Link](3, -1)
be 6 in this example.
[Link](arr2)
arr = [Link].standard_normal((5, 2)) Splitting
arr2d = [Link](15).reshape(3, 5)
f, s, t = [Link](arr, [1, 3]) Indices at which to split the array
[Link]() into pieces.
Converting into 1D array.

[Link](‘C’), [Link](‘F’) arr = [Link](3) Repeating

[Link]() [Link](3) Repeat the elements
Inside the array
arr1 = [Link](6).reshape(2, 3) [Link]([2, 3, 4])
Concatenation
arr2 = [Link](6, 12).reshape(2, 3) arr2d = [Link].standard_normal((2, 2))
Default concatenate is
[Link]([arr1, arr2]) with axis-0 (rows) [Link](2, axis=0), [Link]([2, 3], axis=1)

[Link]([arr1, arr2], axis=0) [Link](arr2d, 2)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
[Link]([arr1, arr2], axis=1) [Link](arr2d, (3, 2))
 Broadcasting
Definition: It’s a feature that governs the operations
work between arrays of different shapes.
arr1 = [Link](12).reshape(4, 3)

arr2 = [Link](12, 24).reshape(4, 3)

res = arr1 – arr2 Operation between arrays

of same shape

mean = [Link](0)

res = arr1 – mean Operation between arrays

of different shapes

Rule:
Two arrays are compatible for broadcasting if for
each tailing dimension (starting from the end) the
axis length match or if either of the lengths is 1.
Broadcasting is performed over the missing or
length 1 dimensions.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Broadcasting
arr1 = [Link](12).reshape(4, 3)

mean = [Link](1)

mean = [Link](4, 1)

res = arr1 – mean

arr1 = [Link](24).reshape(3, 4, 2)

arr2 = [Link](100, 108).reshape(4, 2)

res = arr1 + arr2

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

pandas

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Introduction
• pandas adopts many behaviors from numpy. import pandas as pd

s1 = [Link]([4, 7, -5, 3])

Series Basics

• pandas is designed for working with tabular or print(s1)

heterogeneous data.
[Link], [Link]

s2 = [Link]([4, 7, -5, 3], index=['d', 2, 'a’, 9])

• where numpy best suited for working with
homogeneously typed numerical array data. s2['d'], s2[9], s2[2], s2['d’] = 6

s2[[9, 'd', 2]], s2[s2>0], s2 * 2, [Link](s2)

• Contains 2 data structures: Series, DataFrame ‘d’ in s2, 10 in s2

sdata = {‘Ohio’: 35000, ‘Texas’: 71000, ‘Hyderabad’:

16000, ‘Vizag’: 5000}
• Series is a one dimensional array like object s3 = [Link](sdata), s3.to_dict()
containing sequence of homogeneous values and
associated array of data labels called index. states = [‘California’, ‘Ohio’, ‘Hyderabad’, ‘Vizag’]
s4 = [Link](sdata, index=states)

s3 + s4

[Link] = ‘Population’, [Link] = ‘state’

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Series, DataFrames Adding new column with boolean values
data = {'state': ['Ohio', 'Ohio', 'Ohio', 'Nevada', df1['eastern'] = df1['state'] == 'Ohio’
'Nevada', 'Nevada'],
Delete column
'year': [2000, 2001, 2002, 2001, 2002, 2003], del df1['eastern']
'pop': [1.5, 1.7, 3.6, 2.4, 2.9, 3.2]}
data = {'Ohio': {2000: 1.5, 2001: 1.7, 2002: 3.6},
df1 = [Link](data) 'Nevada': {2001: 2.4, 2002: 2.9}}
Reordering columns/final
list of columns
[Link](data, columns=['year', 'state', 'pop’]) df2 = [Link](data), df2.T Reordering indexes/final
list of indexes
[Link](data, columns=['year', 'state', 'pop', [Link](data, index=[2002, 2001, 2003])
'debt’])
[Link] = 'year’, [Link] = 'state’
DataFrame Basics
[Link](), [Link]()
2002 in [Link], 'Ohio’ in [Link]
Accessing columns in DF
[Link], df1['pop’], [Link]

[Link][0], [Link][2] Accessing rows by position

Adding new column
df1['debt'] = 16.5, df1['debt'] = [Link](6.0)

debt_ser = [Link]([-1.2, -1.5, -1.7], index=['one',

'four', 'five’])
These are NOT indexes of DF.
df1['debt'] = debt_ser

dser = [Link]([-1, -5, -7], index=[1, D.

4,Chaitanya
5]) Reddy (IIT Roorkee Alumni, AI Expert)
df1['debt'] = dser
 Contd…
Series, DataFrames
obj = [Link]([4.5, 7.2, -5.3, 3.6], index=['d', 'b', reindex function arguments
'a', 'c’]) Reordering indexes/final
list of indexes
obj2 = [Link](['a', 'b', 'c', 'd', 'e’])

Series Reindexing
obi = [Link](['blue', 'purple', 'yellow'], index=[0,
2, 4])

obj3 = [Link]([Link](7), method='ffill’)

df1 = [Link]([Link](9).reshape((3, 3)),

index=['a', 'c', 'd'], columns=['Ohio', 'Texas',
'California’]) Reordering Row indexes/final
list of Row indexes in DF obj = [Link]([Link](5.), index=['a', 'b', 'c',
df2 = [Link](index=['a', 'b', 'c', 'd’]) 'd', 'e’])

Reordering columns /final list of columns in DF new_obj = [Link]('c’), [Link](['d', 'c’])

df3 = [Link](columns=['Newyork', 'Texas',
'California’]) df3 = [Link](index=[‘a’, ‘d’]) Dropping Entries
(elements, row,
states = ['Newyork', 'Texas', 'California’] df4 = [Link](columns=[‘Texas’])
columns)
df4 = [Link](states, index='columns')
df4 = [Link]([‘Texas’, ‘Ohio’], axis=1)
DataFrame
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
Reindexing df4 = [Link](‘Texas’, axis=‘columns’)
 Contd…
Series, DataFrames Selecting rows by
positional indexes
s1 = [Link]([4.5, 7.2, -5.3, 3.6], index=['d', 'b',
[Link][0], [Link][[3, 1]]
'a', 'c’])
Series indexing, Selecting rows & columns
by positional indexes
s1[‘d’], s1[1], s1[2:4], selection, filtering [Link][2, [2, 0, 3]], [Link][[3, 2], [2, 1, 0]]
s1[['b', 'c', 'a’]], s1[[1, 3]], s1[s1>0]
df3>5, df3[df3['two']>5], [Link][[Link]>7]
loc indexes exclusively with actual
[Link][['b', 'a', 'c’]] index labels. [Link][[Link]>8] Error!!!, Because iloc can’t be
iloc indexes exclusively with integers used with boolean arrays
[Link][[1, 2, 0]] (with actual integer indexes / with position
incase of non integer indexes).
loc, iloc can modify the dataframes in place.
df3 = [Link]([Link](16).reshape((4, 4)),
index=['Ohio', 'Colorado', 'Utah', 'New York'],
columns=['one', 'two', 'three', 'four’])
Selecting
[Link], df3['two’], df3[['three', 'one’]] column(s)
Selecting rows by
actual indexes
DataFrame
[Link]['Ohio’], [Link][['Ohio', 'Utah’]],
indexing,
selection,
[Link]['Colorado', ['three', 'one’]] filtering
Selecting rows & columns
by actual indexes
[Link][['Colorado', 'New York'], ['three', 'one']]
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Series, DataFrames
df3 = [Link]([Link](16).reshape((4, 4)),
s1 = [Link]([7.3, -2.5, 3.4, 1.5], index=['a', 'c',
index=['Ohio', 'Colorado', 'Utah', 'New York'],
'd', 'e’])
columns=['one', 'two', 'three', 'four’])
s2 = [Link]([-2.1, 3.6, -1.5, 4, 3.1], index=['a',
[Link] = 7, df3['one'] = 12, df3[['one', ‘four']] = 12
'c', 'e', 'f', 'g’])
Updating column values
s1 + s2
[Link]['Ohio'] = -4, [Link][['Ohio', 'Utah']] = 5
df1 = [Link]([Link](9.).reshape((3, 3)),
[Link][1] = -5, [Link][[0, 1]] = 9
columns=list('bcd'), index=['Ohio', 'Texas',
'Colorado’])
Updating row values
Updating Rows, Columns df2 = [Link]([Link](12.).reshape((4, 3)),
Updating values in columns=list('bde'), index=['Utah', 'Ohio', 'Texas',
selective rows & columns 'Oregon’])
Both are same.
[Link][df3['three']>5]=25
df1 + df2, [Link](df2)
Substitutes the passed
[Link][df3['three']>5, 'four'] = -1 value for any missing
[Link](df2, fill_value=-10)
values.
[Link][df3['three']>5, ['one', 'four']] = 100
[Link](columns=[Link], fill_value=0)

Arithmetic operations
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
with Series & DFs
 Contd…
Series, DataFrames
df1 = [Link]([Link](4, 3),
s1 = [Link](range(4), index=['d', 'a', 'b', 'c’])
columns=list('bde'), index=['Utah', 'Ohio', 'Texas',
'Oregon’])
s1.sort_index() Sorting indexes, column headers
applying abs to every
[Link](df1) element in df1 df1 = [Link]([Link](8).reshape((2, 4)),
index=['three', 'one'], columns=['d', 'a', 'b', 'c’])
Applying function on 1D
def diff(x):
array (Row / Column) in DF df1.sort_index(), df1.sort_index(axis=0)
return [Link]() - [Link]()
df1.sort_index(axis=1)
Applying diff function
[Link](diff) across the rows.
s2 = [Link]([4, [Link], 7, [Link], -3, 2])
[Link](diff, axis=0) Sorting values and missing
Applying diff function s2.sort_values() values (NaN) moved to last.
across the columns.
[Link](diff, axis=1) Sorting values but,
def my_format(el): s2.sort_values(na_position='first’) NaN moved to first.
Applying function element
return f'{el:0.2f}’
wise in a DF, a Column in DF Sorting values
Applying my_format function
[Link](my_format)on all elements in DF df2 = [Link]({'b': [4, 7, -3, 2], 'a': [0, 1, 0,
1]})
Applying my_format function on
df1['e'].map(my_format) all elements in a column in DF df2.sort_values('b’)

df2.sort_values(['a',
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert) 'b'])
 Contd…
Series, DataFrames
s1 = [Link](range(5), index=['a', 'a', 'b', 'b',
'c’])

[Link].is_unique Series, DF with

duplicate Indexes
s1['a’]

df = [Link]([Link](4, 3), index=['a',

'a', 'b', 'b’])

[Link]['a']

df1 = [Link]([[1.4, [Link]], [7.1, -4.5],

[[Link], [Link]], [0.75, -1.3]], index=['a', 'b', 'c',
'd'], columns=['one', 'two’]) Sum of each elements
across the rows
[Link](), [Link](axis=0)

[Link](axis=1), [Link](axis=1, skipna=False)

[Link](axis=1, skipna=True) Statistical

calculations

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Series, DataFrames
s1 = [Link](['c', 'a', 'd', 'a', 'a', 'b', 'b', 'c',
'c’])

[Link](), s1.value_counts(), [Link](['b', 'c’])

get unique get count Check the values in

values of values given list or not

s1[[Link](['b', 'c'])]
Unique Values, Value
counts, Membership

df1 = [Link]({'Qu1': [1, 3, 4, 3, 4], 'Qu2': [2,

3, 1, 2, 3], 'Qu3': [1, 5, 2, 4, 4]})
Getting count of values in
a column.
df1['Qu1'].value_counts()
Getting count of combination
of values across all columns
df1.value_counts() in a DF.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

Data Loading

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Reading CSV/Text files
1st row is NOT treated
food = pd.read_csv('[Link]’) as column header

food = pd.read_csv('[Link]’, header=None)

Making this column as
Own column headers name for indexes.

food = pd.read_csv('[Link]’, names=['Name', 'Sci. Name', 'Category', 'Sub Category’])

food = pd.read_csv('[Link]’, names=['Name', 'Sci. Name', 'Category', 'Sub Category’], index_col='Name’)

food = pd.read_csv('[Link]’, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],

index_col=['Category', 'Sub Category’])
Multiple index columns with names
from the names list.

food = pd.read_csv('[Link]’, header=3, names=['Name', 'Sci. Name', 'Category', 'Sub Category’])

Row number(s) to use as the column
names, and the start of the data.

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],

skiprows=10)

Skip the no. of rows from start.

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],
skipfooter=7)
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
Skip the no. of rows from bottom of file.
 Contd…
Reading CSV/Text files
food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’], nrows=50)

No. of rows to read.

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],

keep_default_na=False)

Python have many default sentinels (empty data/invalid data/etc…) like NA, NULL, etc… All these values should
be loaded/read as NaN by default. keep_default_na=False means, all those sentinels are loaded/read as is.

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],

na_values=['Unfermented milks’])
Add additional sentinels while
reading/loading data.

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub Category’],

keep_default_na=False, na_values=['Unfermented milks’])
With all default sentinels are reading/loading as
is, Adding some other values as sentinels while
reading/loading data.

sentinels = {'Category': ['Herbs and Spices'], 'Sub Category': ['Unfermented milks’]}

food = pd.read_csv('[Link]’, header=0, names=['Name', 'Sci. Name', 'Category', 'Sub
Category’],na_values=ssentinels)
defining the column wise sentinels
and used.
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Reading CSV/Text files
food = pd.read_csv('[Link]’, sep='|')
separator char to use
to separate columns
food = pd.read_csv('[Link]’, sep='\s+')
separator can be a regular
expression too.

chunker = pd.read_csv('[Link]’, chunksize=100)

[Link] type object that contains DataFrames. Each DF has 100 rows.

for piece in chunker:

print(piece)
Iterating through each DF in
chunker.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Writing to CSV/Text files
food.to_csv(r'[Link]')
All NaN values should
be written as NULL.
food.to_csv(r'[Link]’, sep='|’)
Do NOT write Row
index information.
food.to_csv(r'[Link]’, sep='|', na_rep='NULL’)
Do NOT write Column
headers information.
food.to_csv(r'[Link]’, sep='|', na_rep='NULL’, index=False)

Only write the subset of

food.to_csv(r'[Link]’, sep='|', na_rep='NULL’, index=False, header=False) columns from available columns.

food.to_csv(r'[Link]’, sep='|', na_rep='NULL’, index=False, columns=['FOOD NAME', 'SCIENTIFIC NAME',

'GROUP'])

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Reading Excel files
file = [Link](r'[Link]')
All sheet names
file.sheet_names
present in the Excel

df1 = [Link](sheet_name='Sheet1')
Read the data from the
More popular & versatile specified sheet name into DF
than ExcelFile()
1st row is NOT treated
df1 = pd.read_excel('[Link]’, sheet_name='Sheet1')
as column header

Making this column as

df1 = pd.read_excel('[Link]’, sheet_name='Sheet1', header=None) name for indexes.

Changing the dtype of

df1 = pd.read_excel('[Link]’, sheet_name='Sheet1', index_col='ID')
a given column.

df1 = pd.read_excel('[Link]’, sheet_name='Sheet1', index_col='ID', dtype={'Income': str})

df1 = pd.read_excel('[Link]’, sheet_name='Sheet1', index_col='ID', dtype={'Income': str, 'ID': float})

Changing the dtype of

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert) given multiple column.
 Writing to Excel files
Name of the sheet in newly
bikes.to_excel(r'[Link]') creating Excel file.

All NaN values should

bikes.to_excel(r'[Link]’, sheet_name='Bikes_info')
be written as NULL.

Do NOT write Row

bikes.to_excel(r'[Link]', sheet_name='Bikes_info', na_rep='NULL') index information.

Do NOT write Column

bikes.to_excel(r'[Link]', sheet_name='Bikes_info', na_rep='NULL', index=False) headers information.

bikes.to_excel(r'[Link]', sheet_name='Bikes_info', na_rep='NULL', index=False, header=False)

bikes.to_excel(r'[Link]', sheet_name='Bikes_info', na_rep='NULL', index=False, columns=['ID', 'Income',

'Children', 'Cars'])

Only write the subset of

columns from available columns.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 JSON file Single columns values,
no column header.

df1 = pd.read_json(r'[Link]’, orient='index’)

Indication of
expected JSON format.

Row Index names

Multiple columns values with column headers.

df1 = pd.read_json(r'[Link]', orient=‘index’)

Row Index names

3 rows with
same columns.
No Row index
names. df1 = pd.read_json(r'[Link]', orient='columns’)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
JSON file
df1 = pd.read_json(r'[Link]', orient='columns')

Not in proper order

These 2 column values
are extended to all 3
rows.

3 rows of a
table.
Proper code for correct Reading of JSON:
import json
with open('[Link]','r') as f:
data = [Link]([Link]())

df1 = pd.json_normalize(data, record_path =['students'],

meta=['school_name', 'class'])

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
JSON file
These 2 column
values are
extended to all
3 rows.

import json Needs to read

with open(r'[Link]', 'r') as f: differently
data = [Link]([Link]())

df1 = pd.json_normalize(data, record_path =['students'],

meta=['school_name', 'class', ['info', 'president'], ['info', 'contacts',
'email’]])
3 rows of a
table.

df1.to_json('[Link]')

df1.to_json('[Link]', orient='records')

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Interacting with DB
Connect to DB Execute the query.
Close the connection

1 2 3

import sqlalchemy as sqla

1 Need username,
user = 'root' password, host, port,
password = 'python' database name details.
host = '[Link]'
port = '3306'
database = 'pythonTraining'
url = 'mysql://{0}:{1}@{2}:{3}/{4}'.format(user, password, host, port, database)
con = sqla.create_engine(url)

2
df1 = pd.read_sql('select * from students', con)

df1.to_sql(table_name, con, if_exists='replace', index=False)

If table exists.
Table name into which D. Chaitanya Reddypossible values:
(IIT Roorkee Alumni, fail, replace, append
AI Expert)
want to load DF data.
Web Scraping

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Reading & Writing HTML file
tables = pd.read_html(r'[Link]
Can use html link or html file.
len(tables),
Number of tables present in the
html page.

type(tables)

List – contains the list of all tables

in DataFrame format present in the
html page.

tables[0], tables[2]

df1.to_html()

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Reading XML file
df1 = pd.read_xml(r'[Link]')
We can use xml
string also here.

df1 = pd.read_xml(r'[Link]')
Parse required
set of nodes

df1 = pd.read_xml(r'[Link]')
df1.to_xml(r'[Link]', row_name='store',
df1 = pd.read_xml(r'[Link]', attrs_only=True) attr_cols=['slNo'], elem_cols=['foodItem', 'price',
'quantity'])
df1 = pd.read_xml(r'[Link]', elems_only=True)

df4 = pd.read_xml(r'[Link]')

df4 = pd.read_xml(r'[Link]', xpath='./store')

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Writing to XML file
df1.to_xml(r'[Link]') Each node name

Do not write row

df1.to_xml(r'[Link]', row_name='store') indexes into XML

List of columns as
df1.to_xml(r'[Link]', row_name='store', index=False)
attributes of node

df1.to_xml(r'[Link]', row_name='store', index=False, attr_cols=['slNo']) List of columns

as child nodes

df1.to_xml(r'[Link]', row_name='store', attr_cols=['slNo'], elem_cols=['foodItem', 'price', 'quantity'])

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

Data Cleaning

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Data Cleaning
s1 = [Link]([1.2, -3.5, [Link], None, 0]), [Link]()

s2 = [Link](['Jack', [Link], None, 'Mango’]), [Link]() Dropping NaN

values in Series

[Link](), s1[[Link]()], s1[[Link]()]

Both gives same
final result

df1 = [Link]([[1., 6.5, 3.], [1., [Link], [Link]], [[Link], [Link], [Link]], [None, 6.5, 3.]])

[Link](), [Link](how='all'), [Link](how='any’), [Link](axis=0, how='any’)

[Link](axis=1, how='any'), [Link](axis=1, how='all’)

df2 = [Link]([Link](7, 3)) Dropping NaN

[Link][:4, 1] = [Link] values in DF
[Link][:2, 2] = [Link]

[Link](thresh=2)
Number of rows to drop
which contains any missing
value.
Does not [Link]
Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
columns
 Contd…
Data Cleaning
s1 = [Link]([1, [Link], 3.5, [Link], 7])
Filling NaN values in
Series
[Link](0), [Link]([Link]())

df1 = [Link]([Link](6, 3))

[Link][2:, 1] = [Link]
[Link][4:, 2] = [Link]

[Link](2.5)
Filling NaN values in
Fill values column wise DF
using dictionary.
[Link]({0: 0.5, 2: 10.34})

Fill with
previous values
[Link](method='ffill’)
Fill with previous
values with limit.
[Link](method='ffill', limit=2)

[Link](axis=1, method='ffill', limit=1)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Cleaning
Keep the first duplicate
Keep the last duplicate
s1 = [Link]([1, 2, 3.5, 2, 7]) value and drop the rest
value and drop the rest
duplicates
duplicates Removing duplicate
[Link](), s1.drop_duplicates(), s1.drop_duplicates(keep='last')
values in Series

df1 = [Link]({'k1': ['one'] * 3 + ['two'] * 4 + ['one'], 'k2': [1, 1, 2, 3, 3, 4, 4, 2]})

[Link]() Removing duplicate

values in DF
Drop the rows based on duplicates
in combination of all columns.
df1.drop_duplicates()

Drop the rows based on duplicates

in k1 column only.
df1.drop_duplicates(subset=['k1’])

Drop the rows based on duplicates

in k1 & k2 columns only.
df1.drop_duplicates(subset=['k1', 'k2’])

df1.drop_duplicates(subset=['k1', 'k2'], keep='last')

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Cleaning
s1 = [Link]([1, 2, -3.5, 2, 7]*10) Updating the outliers
(value is greater than 3
Detecting the outliers (value
or less than -3) to
is greater than 3 or less
correct values.
than -3) based on req. Detecting & filtering
[Link]()>3, s1[[Link]()>3], s1[[Link]()>3] = [Link](s1)*3
outliers in Series

df1 = [Link]([Link](1000, 4))

Filtering the DF based on

the outliers detection in Detecting & filtering
2 column.
df1[df1[2].abs()>3], df1[2][df1[2].abs()>3] → col = df1[2]; col[[Link]()>3] outliers in DF columns
Filtering the 2 column
based on the outliers
detection in 2 column.

df1[2][df1[2].abs()>3] = [Link](df1[2]) * 3 → col = df1[2]; col[[Link]() >3] = [Link](col)*3

Updating the outliers

(value is greater than 3
or less than -3) to
correct D.
values.
Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
Data Transformation

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Data Transformation
map() apply() applymap()

Applying a function on Applying a Applying a

all elements in a function across function on all
column in DF (Series) the rows/columns. elements in DF
Ex: Ex: Ex:
df1['e'].map(my_format) [Link](diff) [Link](my_format)

s1 = [Link]([1., -999., 2., -999., -1000., 3.])

Transforming / Replacing
[Link](-999, [Link])
values in Series

[Link]([-999, -1000], [Link])

[Link]([-999, -1000], [[Link], 0.8])

[Link]({-999: [Link], -1000:0.8}) D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Transformation
df1 = [Link]([Link](12).reshape((3, 4)), index=['ohio', 'colorado', 'new York'], columns=['one',
'two', 'three', 'four'])

Name of the function

[Link] Name of the function that updates the
that updates the index column text to upper
text to titled text. case text.
def transform(x):
return x[:4].upper()
[Link](index=[Link], columns=[Link])
[Link] = [Link](transform)

Renaming Axis Indexes

[Link](index={'OHIO': 'INDIANA'}, columns={'three':'peekaboo'})

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Transformation
ages = [Link]([20, 22, 25, 27, 21, 23, 37, 31, 61, 45, 41, 32])

bins = [Link]([18, 25, 35, 60, 100])

cats = [Link](ages, bins)

category All Categories Category value

indexes counts

[Link], [Link], pd.value_counts(cats)

Discretization & Binning

groups = ['Youth', 'Youth Adult', 'Middle Aged', 'Senior']
cats = [Link](ages, bins, labels=groups)

Giving names
to categories

[Link], pd.value_counts(cats)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Transformation
emails = {'Dave': 'dave@[Link]', 'Steve': 'steve@[Link]', 'Rob': 'rob@[Link]', 'Wes': [Link]}
emails = [Link](emails)

Manipulating String
[Link]()
values

[Link]('gmail')

[Link][:5], [Link][2:8]

[Link]()

[Link]('@', '-’)

[Link]()
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Transformation
fruits = ['apple', 'orange', 'apple', 'apple'] * 2
N = len(fruits)
rng = [Link].default_rng(seed=12345)

df = [Link]({'fruit': fruits, 'basket_id': [Link](N), 'count': [Link](3, 15, size=N), 'weight':

[Link](0, 4, size=N)}, columns=['basket_id', 'fruit', 'count', 'weight'])

Categorical data
df['fruit']

df['fruit'] = df['fruit'].astype('category')
df['fruit’]

df['fruit'].[Link] df['fruit'].[Link] df['fruit'].value_counts()

pd.get_dummies(df['fruit'])
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
Data Wrangling

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Data Wrangling
s1 = [Link]([Link](size=9), index=[['a', 'a', 'a', 'b', 'b', 'c', 'c', 'd', 'd'], [1, 2, 3, 1,
3, 1, 2, 2, 3]])

[Link]
Hierarchical
Accessing Accessing elements indexing in Series
elements using using both outer &
outer index inner index

s1['b'] s1['b':'c'] [Link][['b', 'd']] [Link][:, 2]

Converting Series
to DF.
[Link]() [Link]().stack()

df = [Link]([Link](12).reshape((4, 3)), index=[['a', 'a', 'b', 'b'], [1, 2, 1, 2]], columns=[['Ohio',

'Ohio', 'Colorado'], ['Green', 'Red', 'Green']])

[Link] = ['key1', 'key2'] [Link] = ['state', 'color'] Hierarchical

indexing in DF
[Link]

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Wrangling
df1 = [Link]({'key': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1': range(7)})

Joining / Merging
DFs

[Link](df1, df2)

df2 = [Link]({'key': ['a', 'b', 'd'], 'data2': range(3)})

inner join outer join left join right join

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Wrangling
df1 = [Link]({'key': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1': range(7)})

Joining / Merging
DFs

Default value is
‘inner’

df2 = [Link]({'key': ['a', 'b', 'd'], 'data2': range(3)}) [Link](df1, df2, how='outer')

inner join outer join left join right join

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Wrangling
df1 = [Link]({'key': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1': range(7)})

Joining / Merging
DFs

df2 = [Link]({'key': ['a', 'b', 'd'], 'data2': range(3)}) [Link](df1, df2, how='left')

inner join outer join left join right join

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Wrangling
df1 = [Link]({'key': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1': range(7)})

Joining / Merging
DFs

df2 = [Link]({'key': ['a', 'b', 'd'], 'data2': range(3)}) [Link](df1, df2, how='right')

inner join outer join left join right join

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Wrangling
[Link](df1, df2, on='key’)

Explicitly giving the common

column name to merge both DFs
Joining / Merging
DFs
df3 = [Link]({'lkey': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1': range(7)})
df4 = [Link]({'rkey': ['a', 'b', 'd'], 'data2': range(3)})

[Link](df3, df4, left_on='lkey', right_on='rkey’)

Can provide different column

names from both DFs to merge.
[Link](df3, df4, left_on='lkey', right_on='rkey', how='outer’)

left = [Link]({'key1': ['foo', 'foo', 'bar'], 'key2': ['one', 'two', 'one'], 'lval': [1, 2, 3]})
right = [Link]({'key1': ['foo', 'foo', 'bar', 'bar'], 'key2': ['one', 'one', 'one', 'two'], 'rval': [4,
5, 6, 7]})

[Link](left, right, on=['key1', 'key2'], how='outer')

Can merge on multiple common

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
columns present in both DFs.
 Contd…
Data Wrangling
left = [Link]({'key': ['a', 'b', 'a', 'a', 'b', 'c'], 'value': range(6)})
right = [Link]({'group_val': [3.5, 7]}, index=['a', 'b'])

[Link](left, right, left_on='key', right_index=True)

Joining / Merging
DFs with Row index
Merging both DFs with ‘key’ column in
left DF and row index in right DF.

While merging with index with right DF, the row index values of left DF preserved.

While merging with index with left DF, the row index values of right DF preserved.

While merging with columns from both DFs, row index values are NOT preserved.

left2 = [Link]([[1., 2.], [3., 4.], [5., 6.]], index=['a', 'c', 'e'], columns=['Ohio', 'Nevada'])
right2 = [Link]([[7., 8.], [9., 10.], [11., 12.], [13, 14]], index=['b', 'c', 'd', 'e'],
columns=['Missouri', 'Alabama’])
[Link](left2, right2, left_index=True, right_index=True, how='outer’)

Merge by row Can merge on row index of both DFs

index by default.
join method
D. Chaitanya in DFAlumni,
Reddy (IIT Roorkee can AI
also
Expert)be used for merging. But,
[Link](right2, how='outer') the input args are slightly to be differed.
 Contd…
Data Wrangling
s1 = [Link]([0, 1], index=['a', 'b'])
s2 = [Link]([2, 3, 4], index=['c', 'd', 'e'])
s3 = [Link]([5, 6], index=['f', 'g’])

[Link]([s1, s2, s3]) Resulting a new Series Concatenating

Series
[Link]([s1, s2, s3], axis='columns') Resulting a new
DF
s4 = [Link]([s1, s3])
[Link]([s1, s4], axis='columns', join='inner’)
To indicate which rows are copied
from which Series.
[Link]([s1, s2, s3], axis='columns', keys=['one', 'two', 'three’])

df1 = [Link]([Link](6).reshape(3, 2), index=['a', 'b', 'c'], columns=['one', 'two'])

df2 = [Link](5 + [Link](4).reshape(2, 2), index=['a', 'c'], columns=['three', 'four’])

[Link]([df1, df2], axis=1, keys=['level1', 'level2’])

To indicate which columns are
Concatenating
list of DFs to copied from which DF. DFs
concatenate
[Link]({'level1': df1, 'level2': df2}, axis=1)
dict of DFs to concatenate, along
with keys mentioned as keys of dict
Names of multi level
[Link]([df1, df2], axis=1, keys=['level1', 'level2'], names=['upper', 'lower']) columns in resulted DF
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Wrangling
a = [Link]([[Link], 2.5, [Link], 3.5, 4.5, [Link]], index=['f', 'e', 'd', 'c', 'b', 'a'])
b = [Link]([Link](len(a), dtype=np.float64), index=['f', 'e', 'd', 'c', 'b', 'a’])

Resulting a new array with takes elements from Combining Series

[Link]([Link](a), b, a) a Series if they are NOT NaN. Else, take from
b Series

Same result as [Link]()

a.combine_first(b)

df1 = [Link]({'a': [1., [Link], 5., [Link]], 'b': [[Link], 2., [Link], 6.], 'c': range(2, 18, 4)})
df2 = [Link]({'a': [5., 4., [Link], 3., 7.], 'd': [[Link], 3., 4., 6., 8.]})

df1.combine_first(df2)

Union of all rows and columns of both DFs. Combining DFs

But, result DF takes value from df1 if not
NaN. Otherwise, takes from df2.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

Data Visualization

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Data Visualization
import [Link] as plt ar2d = [Link](24).reshape(3, 8)
ar2d = [Link](float)
ar1 = [Link]([3, 4, 5.6, 0, -2, -2, 4, 5, 3])
ar2d[0, 1] = 5 ar2d[1, 1] = 10 ar2d[2, 1] = 8
[Link](ar1) ar2d[0, 2] = -1 ar2d[1, 2] = 0 ar2d[2, 2] = -3.2
ar2d[0, 3] = 5 ar2d[1, 3] = 0 ar2d[2, 3] = 3.6
ar2d[0, 4] = 5 ar2d[1, 4] = 0 ar2d[2, 4] = 0
ar2d[0, 5] = 8 ar2d[1, 5] = 2 ar2d[2, 5] = -1
ar2d[0, 6] = 1 ar2d[1, 6] = -7 ar2d[2, 6] = 1
ar2d[0, 7] = -5 ar2d[1, 7] = 2.5 ar2d[2, 7] = 7.5

ar2d = [Link](12, 2) Plotting a 2-D

Y – axis:
Values c array data
[Link](ar2d)
of array

Y – axis:
This is X-axis: indexes of Values in
one plot values in the array the each
column of
the array
Plotting a simple
X-axis: indexes of
1-D array data values
D. Chaitanyain column
Reddy ofAlumni, AI Expert)
(IIT Roorkee
the array
 Contd…
Data Visualization
fig = [Link]() 1. plots can resides in a figure object.
2. Can’t have empty figure. Every figure must have at least one plot in it.

1. Adding 6 plots (2 x 3) into figure.

Making figures
ax1 = fig.add_subplot(2, 3, 1)
2. Assigning 1st plot to ax1

ax2 = fig.add_subplot(2, 3, 2) Assigning 2nd plot to ax2

ax3 = fig.add_subplot(2, 3, 3)
ax4 = fig.add_subplot(2, 3, 4) Assigning the 3rd, 4th, 5th, 6th
ax5 = fig.add_subplot(2, 3, 5) plots to ax3, ax4, ax5, ax6

Not created 6th plot at all.

ax6 = fig.add_subplot(2, 3, 6)

In Jupyter notebook, all the figure / plot related commands

must be entered and executed in the same cell.
All empty plots gets
added to figure.

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Visualization
[Link]([Link].standard_normal(50).cumsum(), color='black', linestyle='dashed') Making different
plots.
line plot plot data line color line style

[Link]([Link].standard_normal(100), bins=20, color='black', alpha=0.3)

Histogram plot data Histogram Line color Transparency of over

bins laid plot.

[Link]([Link](30), [Link](30)+3*[Link].standard_normal(30))

Scatter plot X-axis Y-axis

data data

fig

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Visualization
fig, axes = [Link](2, 3) 1. Returns a tuple contains figure and array More about
of indexes of plots in the figure.
2. fig contains 6 empty plots (2 x 3) in it.
figures

Share same X-axis to all 4 plots

fig, axes = [Link](2, 2, sharex=True, sharey=True)
Share same Y-axis to all 4 plots
for i in range(2):
for j in range(2):
axes[i, j].hist([Link].standard_normal(500), bins=50, color='black', alpha=0.5)

Way to access
each plot in fig

ZERO vertical space between plots

fig.subplots_adjust(wspace=0, hspace=0)

ZERO horizontal space between plots

line plot
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
 Contd…
Data Visualization
fig = [Link]() Adding a single empty
More about
ax = fig.add_subplot() plot in a figure plots

[Link]([Link].standard_normal(30).cumsum(), color='black', linestyle='--', marker='o')

possible values: possible values: possible values:

black, green, blue, orange, solid o x + . *
Yellow, lightblue, Purple, dashed, dotted
lightgreen, Any hex color -
code (Ex: #AAA02A) --
Giving name to
figr = [Link]() data drawn in plot
ax1 = figr.add_subplot()
[Link]([Link].standard_normal(30).cumsum(), color='blue', linestyle='--', label='Default')
[Link]([Link].standard_normal(30).cumsum(), color='orange', linestyle='-', drawstyle='steps-post',
label='Steps-post')
[Link]()
Add the legend information in Style by which data
plot is drawn in plot

figr = [Link]()
ax1 = figr.add_subplot()
data = [Link].standard_normal(30).cumsum()
[Link](data, color='blue', linestyle='--', label='Default')
[Link](data, color='orange', linestyle='-', drawstyle='steps-post',
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)label='Steps-post')
[Link]()
 Contd…
Data Visualization More about
fig, ax = [Link]() Adding labels to the
Adding the X-axis plots X-axis ticks and
ticks details. display style of X-
[Link]([Link].standard_normal(1000).cumsum())
ax.set_xticks([0, 250, 500, 750, 1000]) ticks labels
ax.set_xticklabels(['one', 'two', 'three', 'four', 'five'], rotation=30, fontsize=8)
ax.set_xlabel('Stages') Setting labels
ax.set_ylabel('bitcoin profit') to X, Y axes Displaying The font size of
ax.set_title('My first matplotlib plot’) labels as 30o
the labels.
Setting the
Title to plot
Plot properties can also be
[Link](title='My first matplotlib plot', xlabels='Stages') set using this method.
fig, ax = [Link]()
[Link]([Link](100).cumsum(), color='black', linestyle='solid', label='one')
Drawing three different
[Link]([Link](100).cumsum(), color='blue', linestyle='dashed', label='two')
data in the same plot
[Link]([Link](100).cumsum(), color='red', linestyle='dotted', label='three')
[Link]()
[Link]('figure', figsize=(10, 10)) Setting the figure default
Many other properties:
properties at global level.
ticks
matplotlib
grid configuration
[Link]('font', family='monospace', weight='bold', size=8) legend
Setting font properties at global level etc…

[Link]('lines', linewidth=1.5, linestyle='-', color='red’)

Setting lines properties at global level
D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)
[Link]('axes', titlelocation='center', titlesize='small', titlecolor='black')
 Contd…
Data Visualization
s1 = [Link]([Link].standard_normal(10).cumsum(), index=[Link](0, 100, 10)) Plotting
Series
[Link](), [Link](kind='line'), [Link](kind='bar'), [Link](kind='barh’)

[Link](kind='line', title='Normalized Profit')

fig, axes = [Link](2, 1)

data = [Link]([Link](size=16), index=list('abcdefghijklmnop'))
[Link](ax=axes[0], color='black', alpha=0.7)
[Link](ax=axes[1], color='blue', alpha=0.2)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 Contd…
Data Visualization
df = [Link]([Link].standard_normal((10, 4)).cumsum(0), columns=['A', 'B', 'C', 'D'], index=[Link](0,
100, 10))
Plotting
[Link]() DataFrames

df1 = [Link]([Link](size=(6, 4)), index=['one', 'two', 'three', 'four', 'five', 'six'],

columns=[Link](['A', 'B', 'C', 'D'], name='Genus’))

[Link](), [Link](stacked=True, alpha=0.5)

[Link](), [Link](stacked=True, alpha=0.5)

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)

 END

D. Chaitanya Reddy (IIT Roorkee Alumni, AI Expert)