FACULTY OF INFORMATION TECHNOLOGY

Semester 1, 2025/2026
 Intrduction

 NumPy

 Pandas

 SciPy

 Matplotlib

 Seaborn

Py – NLU 2
 NumPy (Numerical Python): a powerful library for numerical
computing (i.e., arrays, matrices, and many mathematical
functions) in Python
 Pandas: a data manipulation and analysis library built on top of
NumPy (two primary data structures: Series, DataFrame)
 SciPy (Scientific Python) builds on NumPy and provides additional
functionality (i.e., optimization, integration, interpolation,
eigenvalue problems, algebraic equations, differential equations,
…) for scientific and technical computing
 Matplotlib/Seaborn: a plotting library for creating static, animated,
and interactive visualizations in Python

Py – NLU 3
4
 Fundamental package for scientific computing with Python

 N-dimensional array object

 Linear algebra, Fourier transforms, random number

capabilities

 Building block for other packages (e.g. SciPy)

 Open source

Py – NLU 5
 One dimension arrays (1D) represent vectors.

 Two-dimensional arrays (2D) represent matrices.

 Higher dimensional arrays represent tensors.

Py – NLU 6
 NumPy is an important library for:

◦ Data Science

◦ Machine learning

◦ Signal and image processing

◦ Scientific and engineer computing

Py – NLU 7
 Array indexing is the same as accessing an array element

 Access an array element by referring to its index number

 The indexes in NumPy arrays start with 0

import numpy as np

arr = [Link]([1, 2, 3, 4])

print(arr[0])

Py – NLU 8
 Use comma separated integers representing the dimension
and the index of the element

import numpy as np

arr2d = [Link]([[1,2,3,4,5], [6,7,8,9,10]])

arr3d = [Link]([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])

print(arr2d[1, 4]) # for 2d array

print(arr2d[1, -1]) # for 2d array
print(arr3d[0, 1, 2]) # for 3d array

The third element of the second array of the first array

Py – NLU 9
 Slicing: takes elements from one given index to another given
index

 Syntax: [start:end:step]
◦ start – index position from where the slicing will start, default 0
◦ stop – index position till which the slicing will end , default length of
array in that dimension
◦ step – number of steps, i.e. the start index is changed after every n
steps, and array slicing is performed on that index, default 1

Py – NLU 10
 Example: import numpy as np

arr = [Link]([1, 2, 3, 4, 5, 6, 7])

arr2d = [Link]([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])

print(arr[1:5])
print(arr[4:])
print(arr[:4])
print(arr[-3:-1])
print(arr[1:5:2])
# for 2d array
print(arr[1, 1:4])
print(arr[0:2, 2])
print(arr[0:2, 1:4])

Py – NLU 11
 NumPy has some extra data types, and refer to data types
with one character

i - integer m - timedelta
b - boolean M - datetime
u - unsigned integer O - object
f - float S - string
c - complex float U - unicode string
V - fixed chunk of memory for other type ( void )

Py – NLU 12
 Example:

import numpy as np

arr = [Link]([1, 2, 3, 4], dtype='U')# unicode string

arr1 = [Link]([1.1, 2.1, 3.1])# float
newarr = [Link]('i') # int

print(arr)
print([Link])
print(newarr)
print([Link])

Py – NLU 13
 The copy owns the data and any changes made to the copy
will not affect original array

◦ any changes made to the original array will not affect the copy.

 The view does not own the data and any changes made to the
view will affect the original array,

◦ any changes made to the original array will affect the view.

Py – NLU 14
 Example (copy):
import numpy as np

arr = [Link]([1, 2, 3, 4, 5])

x = [Link]()
arr[0] = 42

print(arr)
print(x)

x[0] = 100
print(arr)
print(x)

Py – NLU 15
 Example (view):
import numpy as np

arr = [Link]([1, 2, 3, 4, 5])

x = [Link]()
arr[0] = 42

print(arr)
print(x)

x[0] = 100
print(arr)
print(x)

Py – NLU 16
 The shape of an array is the number of elements in each
dimension

import numpy as np
arr = [Link]([[1, 2, 3, 4], [5, 6, 7, 8]])
arr1 = [Link]([1, 2, 3, 4], ndmin=5)

print(arr)
print([Link])
print(arr1)
print('Shape of array :', [Link])

Py – NLU 17
 Reshaping means changing the shape of an array

 The shape of an array is the number of elements in each

dimension

 By reshaping we can add or remove dimensions or change

number of elements in each dimension

 The result is typically a view of the original array, not a copy

 Rule: The elements required for reshaping are equal in both

shapes
Py – NLU 18
 Syntax: Syntax : [Link](shape)
Argument : It take tuple as argument, tuple
is the new shape to be formed
Return : It returns [Link]

 pass -1 for unknown dimension, only one unknown

dimension
import numpy as np

arr = [Link]([1, 2, 3, 4, 5, 6, 7, 8])

newarr = [Link](2, 2, -1)

print([Link](2, 4).shape)
print([Link])
print([Link](3,
Py – NLU 3).shape) 19
 Using basic for loop to iterate 1D/2D/… arrays
◦ ➔ difficult for arrays with very high dimensionality
 Using nditer to iterate over multi-dimensional arrays

import numpy as np

array = [Link]([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

for element in [Link](array):

print(element)

Py – NLU 20
 Using ndenumerate to iterate over multi-dimensional arrays
with indexes
 Applying a function along a specific axis, similar to iteration
along a specific axis
import numpy as np

array = [Link]([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

for index, element in [Link](array):

print(index, element)

# Sum along the columns (axis=0)

result = np.apply_along_axis([Link], axis=0, arr=array)
print(result) Py – NLU 21
 How NumPy treats arrays with different shapes during
arithmetic operations?

 Broadcasting rules: Two dimensions are compatible when

◦ they are equal, or import numpy as np
a = [Link]([1.0, 2.0, 3.0])
◦ one of them is 1
b = 2.0
print(a*b)

Py – NLU 22
 Example:

Py – NLU 23
 Example:

a(4x3)+b(3) = ?

a(4x3)+b(4) = ?

Py – NLU 24
 Putting contents of two or more arrays in a single array
 Using concatenate() function to join a sequence of arrays
along with the axis. Default axis 0

import numpy as np

arr1 = [Link]([[1, 2], [3, 4]])

arr2 = [Link]([[5, 6], [7, 8]])

arr3 = [Link]((arr1, arr2), axis=0) # along cols

arr4 = [Link]((arr1, arr2), axis=1) # along rows

print(arr3)
print(arr4)

Py – NLU 25
 Using stack() function to join a sequence of arrays along with
the axis. Default axis 0

import numpy as np

arr1 = [Link]([1, 2, 3])

arr2 = [Link]([4, 5, 6])

arr3 = [Link]((arr1, arr2), axis=0)

arr4 = [Link]((arr1, arr2), axis=1)
print(arr3)
print(arr4)

Py – NLU 26
 Using hstack() to stack along rows; vstack() to stack along
columns; dstack() to stack along height (depth)
import numpy as np

arr1 = [Link]([1, 2, 3])

arr2 = [Link]([4, 5, 6])

arr3 = [Link]((arr1, arr2))

arr4 = [Link]((arr1, arr2))
arr5 = [Link]((arr1, arr2))
print(arr3)
print(arr4)
print(arr5)
Py – NLU 27
 Reverse operation of Joining
 Breaks one array into multiple ones
 Use array_split() for splitting arrays
 Syntax: array_split(ary, indices_or_sections, axis=0)
import numpy as np

arr1d = [Link]([1, 2, 3, 4, 5, 6])

arr2d = [Link]([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]])
newarr1d = np.array_split(arr1d, 3)
newarr2d = np.array_split(arr2d, 3)
newarr2d1 = np.array_split(arr2d, 3, axis=1) # along rows

print(newarr1d)
print(newarr2d)
print(newarr2d1) Py – NLU 28
 Using hsplit() opposite of hstack(), similar for vsplit() and
dsplit() – for 3d arrays
 Example:
import numpy as np

arr1d = [Link]([1, 2, 3, 4, 5, 6])

arr2d = [Link]([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]])

newarr1d = [Link](arr1d, 3)
newarr2d = [Link](arr2d, 3)

print(newarr1d)
print(newarr2d)
Py – NLU 29
 Using where() to search for indexes of a given value in an
array
 Using searchsorted() to performs a binary search in the array
◦ Option side='right' to return the right most index instead

import numpy as np

arr = [Link]([6, 7, 8, 9])

x1 = [Link](arr % 2 == 0)
x2 = [Link](arr, 7, side='right')

print(x1)
print(x2) Py – NLU 30
 Using sort() to sort a specified array
◦ returns a copy of the array, leaving the original array unchanged
 Example:

import numpy as np

arr1d = [Link](['banana', 'cherry', 'apple'])

arr2d = [Link]([[3, 2, 4], [5, 0, 1]])

print([Link](arr1d))
print([Link](arr2d))

Py – NLU 31
 Boolean Indexing: Efficiently filter elements based on
conditions
 Multiple Conditions: Combine conditions with logical
operators
import numpy as np

arr = [Link]([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

arr2d = [Link]([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

filtered_array1 = arr[arr > 5]

filtered_array2 = arr[(arr > 3) & (arr < 8)]
filtered_array2d = arr2d[arr2d > 5]

print(filtered_array1)
print(filtered_array2)
print(filtered_array2d)
Py – NLU 32
 [Link]: Filter and optionally replace elements
 Fancy Indexing: Filter using a list of specific indices
 [Link]: Filter along a specific axis using indices.
import numpy as np

arr = [Link]([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

arr2d = [Link]([[1, 2, 3], [4, 5, 6], [7, 8,
np. where(condition, [x, y, ]/) 9]])
indices = [0, 2, 4, 6, 8]
Condition: filtered_array1 = [Link](arr > 5, arr, -1)
• True, yield x, otherwise yield y. filtered_array2 = arr[indices]
filtered_array3 = [Link](arr, indices)
#
print(filtered_array1)
print(filtered_array2)
Py – NLU
print(filtered_array3) 33
 [Link](): sum of elements in array
 [Link](): minimum value of array
 [Link](axis=0): maximum value of an array row
 [Link](axis=1): cumulative sum of the elements
 [Link](): mean
 [Link](): median
 [Link](): correlation coefficient
 [Link](): standard deviation

Py – NLU 34
 Construct an array by repeating it the number of times

 Syntax: [Link](arr, reps)

◦ arr: The input array

◦ reps: The number of repetitions of A along each axis.

import numpy as np
a = [Link]([0, 1, 2])
print([Link](a, 2))
[0 1 2 0 1 2]
Py – NLU 35
 Repeat each element of an array after themselves
 Syntax: [Link](a, repeats, axis=None)
 Where,
◦ a: array_like, Input array.
◦ repeats: int or array of ints - The number of repetitions for each
element.
◦ axis: int, (optional) - The axis along which to repeat values. By default,
use the flattened input array, and return a flat output array.

Row: axis = 0; Column: axis = 1

Py – NLU 36
 Example: import numpy as np
[3 3 3 3]
print([Link](3, 4))

x = [Link]([[1,2], [3,4]])
[1 1 2 2 3 3 4 4]
print([Link](x, 2))
[[1 1 1 2 2 2]
print([Link](x, 3, axis=1)) [3 3 3 4 4 4]]
print([Link](x, [1, 2], axis=0))
[[1 2]
[3 4]
[3 4]]
Py – NLU NumPy reference — NumPy v2.1 Manual 37
38
 Pandas is a Python library used for working with data sets
◦ a powerful module that is optimized on top of NumPy

 A freely available library for loading, manipulating, and

visualizing sequential and tabular data, such as time series or
micro-arrays

 The name "Pandas" has a reference to both "Panel Data", and

"Python Data Analysis" and was created by Wes McKinney in
2008

Py – NLU 39
 Loading and saving with “standard” tabular file formats:
◦ CSV (Comma-separated Values)
◦ TSV (Tab-separated Values)
◦ Excel files
◦ Database formats, etc.

 Flexible indexing and aggregation of series and tables

 Efficient numerical/statistical operations (e.g. broadcasting)
 Pretty, straightforward visualization

Py – NLU 40
 Pandas provides a couple of very useful datatypes:
◦ Series represents 1D data, like time series, calendars, the output of
one-variable functions, etc.
◦ DataFrame represents 2D data, like a column-separated-values (CSV)
file, a microarray, a database table, a matrix, etc.

 Each column of a DataFrame is a Series.

◦ That’s why we will see how the Series data type works first.
◦ Most of what we will say about Series also applies to DataFrames.

Py – NLU 41
 A Series is a one-dimensional array with a labeled axis, that
can hold arbitrary objects.

 The axis is called the index, and can be used to access the
elements; it is very flexible, and not necessarily numerical.

 It works partially like a list and partially like a dict.

Py – NLU 42
 It is possible to specify just the series data, associating an
implicit numeric index.

import pandas as pd
s = [Link](["a", "b", "c"])
print(s)

Py – NLU 43
 It is possible to specify both the series data and the explicit
index, separately:

import pandas as pd
s = [Link](["a", "b", "c"], index=[2, 5, 8])
print(s)

Py – NLU 44
 It is possible to specify both the series data and the index, as
a single dictionary:

import pandas as pd

s = [Link]({"a": "A", "b": "B", "c": "C"})

print(s)

Py – NLU 45
 If given a single scalar (e.g. an integer), the series constructor
will replicate it for all indices (that need to be specified)

import pandas as pd

s = [Link](3, index=range(5))

print(s)

Py – NLU 46
 For a given Series, we can access it through either the position
(as a list) or the index (as a dict)

import pandas as pd
days = ["mon", "tue", "wed", "thu", "fri"]
working_hours = [6, 7, 5, 8, 9]
s = [Link](working_hours, index=days)

print(s["mon"])
s["tue"] = 3

print(s[1])

Py – NLU 47
 If a label is not contained, an exception is raised.
 Using the get method, a missing label will return None or
specified default

import pandas as pd
days = ["mon", "tue", "wed", "thu", "fri"]
working_hours = [6, 7, 5, 8, 9]
s = [Link](working_hours, index=days)

print(s["sat"])
print([Link]("sat")) KeyError: “sat”
None
Py – NLU 48
 We can also slice the positions, like we would do with a list.
◦ Note that both the data and the index are extracted correctly. It also
works with labels.
wed 5
import pandas as pd thu 8
days = ["mon", "tue", "wed", "thu", "fri"] fri 9
working_hours = [6, 7, 5, 8, 9] dtype: int64
s = [Link](working_hours, index=days) tue 7
print(s[-3:]) wed 5
print(s["tue":"thu"]) thu 8
dtype: int64

Py – NLU 49
 The first and last n elements can be extracted also using
head() and tail().

import pandas as pd
days = ["mon", "tue", "wed", "thu", "fri"]
working_hours = [6, 7, 5, 8, 9]
s = [Link](working_hours, index=days)
mon 6
tue 7
print([Link](2)) wed 5
print([Link](3)) thu 8
fri 9
dtype: int64

Py – NLU 50
 We can also explicitly pass a list of positions. Tuples do not
work, because they are interpreted as potential indexes.

import pandas as pd mon 6

days = ["mon", "tue", "wed", "thu", "fri"] tue 7
working_hours = [6, 7, 5, 8, 9] wed 5
s = [Link](working_hours, index=days) dtype: int64
mon 6
print(s[[0, 1, 2]]) wed 5
print(s[["mon", "wed", "fri"]]) fri 9
dtype: int64

Py – NLU 51
 The Series class automatically broadcasts arithmetical
operations by a scalar to all of the elements.

print(s) print(s+1) print(s*2)

Py – NLU 52
 The concept of operator broadcasting was taken from the
numpy library, and is one of the key features for writing
efficient, clean numerical code in Python.

 In a way, it is a “generalized” version of scalar products (from

linear algebra).

 The rules governing how broadcasting is applied can be pretty

complex(and confusing). For the moment, we will cover
constant broadcasting only

Py – NLU 53
 Besides numerical operators, we can apply boolean
conditions. The result is called a mask.
 Masks can be used to filter the elements of a Series according
to a given condition.
print(s) print(s>=6) print(s[s>=6])

Py – NLU 54
 Operations between multiple time series are automatically
aligned by label, meaning that elements with the same label
are matched prior to carrying out the operation.
print(s[1:]) print(s[:-1]) print(s[1:]+s[:-1])

Py – NLU 55
 The index of the resulting Series is the union of the indices of
the operands. What happens depend on whether a given label
appears in both input Series or not:
◦ For common labels (in our case "tue", "wed", "thu"), the output Series
contains the sum of the aligned elements.
◦ For labels appearing in only one of the operands ("mon" and "fri"), the
result is a NaN, i.e. not-a-number.

 NaN is just a symbolic constant that specifies that the object

is a ,number-like entity with an invalid or undefined value.

Py – NLU 56
 There are different strategies for dealing with nan‘s. There is
no “best” strategy: you have to pick one depending on the
problem you are trying to solve.

t = s[1:]+s[:-1] nt = [Link]() zt = [Link](0.0)

print(t) print(nt) print(zt)

Py – NLU 57
 Through the method add, it is possible to assign a fill value to
the missing entries of the series to be added, in order to get a
real sum

print(s[1:]) print(s[:-1]) print(s[1:].add(s[:-1], fill_value=0))

Py – NLU 58
 print([Link]()): 35 mon 6
tue 7
 print([Link]()): 15120 wed 5
thu 8
 print([Link]()): 9
fri 9
 print([Link]()): 4 dtype: int64
 print([Link]()): 7.0
 print([Link]()): 2.5
 print([Link]()): 1.58
 print([Link]()): 7.0
 print([Link]()):

Py – NLU 59
 print([Link](0.5)): 7.0
 print([Link]( [0.25, 0.5, 0.75] ))

 print(s[s >= [Link](0.5) ])

Py – NLU 60
 # Pearson
 corr. print([Link](s)): 1.0
 # Spearman
 corr. print([Link](s, method="spearman" ): 1.0
 # Autocorrelation
 # with time lag
 print([Link](lag=0)): 1.0
 print([Link](lag=1)): 0.08
 print([Link](lag=2)): -0.24

Py – NLU 61
 A quick way to get several useful statistics is to use
[Link](). Anyway, the list of statistical methods associated
with Series is larger than this.
count 5.000000
mean 7.000000
std 1.581139
print([Link]()) min 5.000000
25% 6.000000
50% 7.000000
75% 8.000000
max 9.000000
dtype: float64

Py – NLU 62
import pandas as pd
import [Link] as plt
days = ["mon", "tue", "wed", "thu", "fri"]
working_hours = [6, 7, 5, 8, 9]
s = [Link](working_hours, index=days)
[Link]()
[Link]()

import pandas as pd
import [Link] as plt
days = ["mon", "tue", "wed", "thu", "fri"]
working_hours = [6, 7, 5, 8, 9]
s = [Link](working_hours, index=days)
[Link](kind="bar")
[Link]()
Py – NLU 63
 Pandas DataFrame is the 2D analogue of a Series: it is
essentially a table of heterogeneous objects.
 A DataFrame holds three major attributes:
◦ the index, which holds the labels of the rows
◦ the columns, which hold the labels of the columns
◦ the shape, which describes the dimension of the table
 When we extract a column from a DataFrame you get a proper
Series, and we can operate on it using all the tools presented
in the previous section.
 Further, most (not all) of the operations that we can do on a
Series, we can also do on an entire DataFrame.
Py – NLU 64
 An example of a data frame
 Structure:
◦ Rows — representing a singular data entry
point
◦ Columns — corresponding to a grouping
relating to a singular quality of each given
data point that are usually titled
◦ Index — a unique identifier for each data
entry

Py – NLU 65
 Use the following ways to create a Pandas DataFrame:

◦ Creating an Empty DataFrame

◦ Using Python Dictionary

◦ Using Python List

◦ From a File

Py – NLU 66
import pandas as pd
# Create an empty DataFrame
df = [Link]()
# Add a column named 'id' with data
df['id'] = [1, 2, 3, 4, 5]
# Add another column named 'name' with data
df['name'] = ["An", "Minh", "Phuong", "Vu", "Vinh"]
# Add another column named 'age' with data
df['age'] = [21, 19, 20, 24, 32]
# Add a row with data
df = [Link]({'id': 6, 'name': "Binh", 'age': 26}, ignore_index=True)

# Add another row with data

df = [Link]({'id': 7, 'name': "Bao", 'age': 29}, ignore_index=True)
print(df)

Py – NLU 67
import pandas as pd
d = { "x": [Link]([0, 0], index=["a", "b"]),
"y": [Link]([0, 0], index=["b", "c"])}
df = [Link](d)
print(df)  If the index of the input Series do not

print([Link]) match, since label alignment applies, the

print([Link]) missing values are treated as NaN‘s.
print([Link])

Py – NLU 68
import pandas as pd
d = { "column1": [1., 2., 6., -1.],
"column2": [0., 1., -2., 4.] }
df = [Link](d)
print(df)  The columns are taken from the keys

print([Link])  The index is set to the default one

print([Link])
print([Link])

Py – NLU 69
import pandas as pd
d = { "column1": [1., 2., 6., -1.],
"column2": [0., 1., -2., 4.] }
df = [Link](d, index=["a", "b", "c", "d"])
print(df)  A custom index can be specified the
print([Link]) usual way
print([Link])
print([Link])

Py – NLU 70
import pandas as pd  The columns are taken from the keys of
d = [ {"a": 1, "b": 2}, the dictionaries
{"a": 2, "c": 3},]  The index is the default one.
df = [Link](d)
 Since not all common keys appear in all
print(df)
input dictionaries, missing values (i.e.
print([Link])
NaN‘s) are automatically added.
print([Link])
print([Link])

Py – NLU 71
import pandas as pd
df = pd.read_csv("[Link]")
print([Link])
print([Link])
print([Link])

Py – NLU 72
  Help on function read_csv in module [Link]:
read_csv(filepath_or_buffer, sep=’,’, delimiter=None, header=’infer’, names=None, index_col=None,
usecols=None, squeeze=False, prefix=None, mangle_dupe_cols=True, dtype=None, engine=None,
converters=None, true_values=None, false_values=None, skipinitialspace=False, skiprows=None,
nrows=None, na_values=None, keep_default_na=True, na_filter=True, verbose=False,
skip_blank_lines=True, parse_dates=False, infer_datetime_format=False, keep_date_col=False,
date_parser=None, dayfirst=False, iterator=False, chunksize=None, compression=’infer’,
thousands=None, decimal=b’.’, lineterminator=None, quotechar=’"’, quoting=0, escapechar=None,
comment=None, encoding=None, dialect=None, tupleize_cols=False, error_bad_lines=True,
warn_bad_lines=True, skipfooter=0, skip_footer=0, doublequote=True, delim_whitespace=False,
as_recarray=False, compact_ints=False, use_unsigned=False, low_memory=True, buffer_lines=None,
memory_map=False, float_precision=None)

Read CSV (comma-separated) file into DataFrame

Py – NLU 73
 We can also create a DataFrame using other file types like
JSON, Excel spreadsheet, SQL database, etc.

 The methods to read different file types are listed below:

◦ JSON - read_json()

◦ Excel spreadsheet - read_excel()

◦ SQL - read_sql()

Py – NLU 74
Operation Syntax Result
Select column df[col] Series
Select multiple columns df[[col1,col2]] DataFrame
Select row by label [Link][label] Series
Select row by integer location [Link][loc] Series
Slice rows df[5:10] DataFrame
Select rows by boolean vector df[bool_vec] DataFrame

Py – NLU 75
 loc: selects rows using row labels (e.g. age)

 iloc: selects rows using their integer positions (starting from 0

and going up by one for each row).

Py – NLU 76
 For simplicity, we extract a random sample taken from the iris
dataset
import numpy as np
import pandas as pd
[Link](0)
df = pd.read_csv("[Link]")
small = [Link][[Link]([Link][0])].head()
print([Link])
print(small)
Brief explanation:
• [Link]() to generate a random permutation of the indices
from 0 to [Link][0],
• take the first 5 rows of the permuted df using the head() method.

Py – NLU 77
 It is possible to access the columns of small with the []
notation. The result is a Series.
 If the name of the column is compatible with the Python
conventions for variable names, we can also treat columns as
if they were actual attributes of the data frames

print(small["Name"]) OR print([Link])

Py – NLU 78
 It is possible to extract multiple columns in one go, by
specifying a list of columns; the result is a DataFrame

print(small[["SepalLength", "PetalLength"]])

Py – NLU 79
 To extract a row, it is possible to use the loc and iloc
attributes by specifying a label or a position, respectively.

 The result is a Series

print([Link][114]) OR print([Link][0])

Py – NLU 80
 To extract a row, it is possible to use the loc and iloc
attributes by specifying a label or a position, respectively.

 The result is a DataFrame

Py – NLU 81
 To extract multiple rows, it is possible to use the loc and iloc
attributes by specifying a list of labels or positions.
 The result is a Dataframe

print([Link][[114,62,33]]) OR print([Link][[0,1,2]])

Py – NLU 82
 Broadcasting is applied automatically to all rows, or to the
entire table.
print(small["SepalLength"] +small["SepalWidth"])
print(small+small)

Py – NLU 83
 Masking works as well.
print(small["PetalLength"][[Link] > 5])
print(small["Name"][[Link] == "Iris-virginica"])
print(small[["Name","PetalLength","SepalLength"][
[Link] == "Iris-virginica"])

Py – NLU 84
 Statistics can be computed on rows, columns, or the whole
table
print([Link][114][:-1].mean()) # Exclude last column, Name
print([Link]())
print([Link]())

Py – NLU 85
print(small[["Name", "PetalLength", "SepalLength"]]
[[Link] > [Link]()])

Py – NLU 86
 Merging different dataframes is performed using the merge()
function.
 Merging means that, given two tables with a common column
name, first the rows with the same column value are matched;
then a=new
sequences table is created by concatenating the matching
[Link]({
rows.
"id": ["Q99697", "O18400", "P78337", "Q9W5Z2"],
"seq": ["METNCR", "MDRSSA", "MDAFKG", "MTSMKD"],
})
names = [Link]({
"id": ["Q99697", "O18400", "P78337", "P59583"],
"name": ["PITX2_HUMAN", "PITX_DROME", "PITX1_HUMAN",
"WRK32_ARATH"],
})
Py – NLU 87
  Syntax:

[Link](right, how='inner', on=None, left_on=None, right_on=None, …)

 Where,
◦ right: DataFrame or named Series Object to merge with.
◦ how: {‘left’, ‘right’, ‘outer’, ‘inner’, ‘cross’}, default ‘inner’ Type of
◦ on (label or list): Column or index level names to join on. These must
be found in both DataFrames.
◦ left_on (label or list, or array-like): Column or index level names to join
on in the left DataFrame.
◦ right_on (label or list, or array-like): Column or index level names to
join on in the right DataFrame.
Py – NLU 88
print(sequences)
print(names)
print([Link](sequences, names, on="id", how="inner"))

Mismatched ids are dropped

Py – NLU 89
employees_with_salary2 = [Link](salary,
left_on="id",
right_on="id")
employees_with_salary2

Py – NLU 90
print(sequences)
print(names)
print([Link](sequences, names, on="id", how="left"))

The ids are taken from the left table

Py – NLU 91
employees_with_salary_left = [Link](salary,
left_on='id',
right_on='id',
how='left')
employees_with_salary_left

Py – NLU 92
print(sequences)
print(names)
print([Link](sequences, names, on="id", how=“right"))

The ids are taken from the right table

Py – NLU 93
employees_with_salary_right = [Link](salary,
left_on='id',
right_on='id',
how='right')
employees_with_salary_right

Py – NLU 94
print(sequences)
print(names)
print([Link](sequences, names, on="id", how=“outer"))

All ids are retained

Py – NLU 95
employees_with_salary_outer = [Link](salary,
left_on='id',
right_on='id',
how='outer')
employees_with_salary_outer

Py – NLU 96
 The groupby method is essential for efficiently performing
operations on groups of rows.

 Given the Iris dataset, we want to compute the average of the

four columns for each of the three different Iris species.

 The result should be a DataFrame with 3 species (rows) by 4

columns (petal/sepal length/width)

Py – NLU 97
import pandas as pd
iris = pd.read_csv("[Link]")
print([Link]())

Py – NLU 98
 Iterating over the grouped variable returns (value-of-Name,
DataFrame) tuples:
◦ The 1st item is the value of the column Name shared by the group
◦ The 2nd item is a DataFrame including only the rows in that group.

grouped = [Link]([Link])
for group in grouped:
print(group[0], group[1].shape)

Py – NLU 99
 It is possible to apply some transformation (e.g. mean()) to
the individual groups automatically, using the aggregate()
method directly on the grouped variable. The result of
aggregate() is a dataframe.

iris_mean_by_name = [Link]([Link]())
print(iris_mean_by_name)

Py – NLU 100
101
 SciPy is a collection of mathematical algorithms and
convenience functions built on Numpy data structures

 Organized into sub-packages covering different scientific

computing areas

 A data-processing and prototyping environment rivaling

MATLAB

Py – NLU 102
 Special functions  Linear Algebra ([Link])
([Link])  Sparse Eigenvalue Problems
 Integration ([Link]) with ARPACK Compressed
Sparse Graph Routines
 Optimization ([Link])
([Link])
 Statistics ([Link])
 Interpolation  Multi-dimensional image
([Link]) processing ([Link])
 Fourier Transforms  File IO ([Link])
([Link])
 Weave ([Link])
 Signal Processing
And more. . .
([Link]) 

Py – NLU 103
 A free alternative to MATLAB
 The power of the full Python language
◦ Object-oriented
◦ Procedural
◦ Functional (almost)
 More reasons come:
◦ MATLAB-like plotting
◦ Call C/C++/Fortran code directly
◦ MPI-style parallel programming (take my graduate course!)

Py – NLU 104
 Slightly different from [Link]. Always uses
BLAS/LAPACK support, so could be faster.

 Some more functions.

 Functions can be slightly different.

Py – NLU 105
 General purpose minimization: CG, BFGS, least-squares
 Constrainted minimization; non-negative least-squares
 Minimize using simulated annealing
 Scalar function minimization
 Root finding
 Check gradient function
 Line search

Py – NLU 106
 Mean, median, mode, variance, kurtosis

 Pearson correlation coefficient

 Hypothesis tests (t-test, Wilcoxon signed-rank test,

Kolmogorov-Smirnov)

 Gaussian kernel density estimation

See also SciKits (or scikit-learn).

Py – NLU 107
 Sparse matrix classes: CSC, CSR, etc.

 Functions to build sparse matrices sparse.

 linalg module for sparse linear algebra sparse.

 csgraph for sparse graph routines.

Py – NLU 108
 Convolutions
 B-splines
 Filtering
 Continuous-time linear system
 Wavelets
 Peak finding

Py – NLU 109
Methods for loading and saving data
 Matlab files
 Matrix Market files (sparse matrices)
 Wav files

Py – NLU 110
111
 Plotting library for Python

 Works well with Numpy

 Syntax similar to Matlab

Py – NLU 112
 Data visualization is the graphical representation of information and
data.
◦ Can be achieved using visual elements like figures, charts, graphs, maps, and
more.
 Data visualization tools provide a way to present these figures and
graphs.
 Often, it is essential to analyze massive amounts of information and
make data-driven decisions.
◦ converting complex data into an easy-to-understand representation.

Py – NLU
 Matplotlib is one of the most powerful tools for data visualization in
Python.
 Matplotlib is an incredibly powerful (and beautiful!) 2-D plotting
library.
◦ It is easy to use and provides a huge number of examples for tackling unique
problems
 In order to get matplotlib into your script, import [Link] as plt
 However, if it is not installed, you may need to install it:
◦ Easiest way to install matplotlib is using pip.
◦ Type the following command in the command prompt (cmd) or your Linux shell;
 pip install matplotlib
 Note that you may need to run the above cmd as an administrator

Py – NLU
 Strives to emulate MATLAB
◦ [Link] is a collection of command style functions that make
matplotlib work like MATLAB.
 Each pyplot function makes some change to the figure:
◦ e.g., creates a figure, creates a plotting area in the figure, plots some lines in the
plotting area, decorates the plot with labels, etc.
 Note that various states are preserved across function calls
• Whenever you plot with matplotlib, the two main code lines should
be considered:
– Type of graph
• this is where you define a bar chart, line chart, etc.
– Show the graph
• this is to display the graph
Py – NLU
 Matplotlib allows you to make easy things
 Can generate plots, histograms, power spectra,
bar charts, error charts, scatterplots, etc., with
just a few lines of code.

Py – NLU 116
import [Link] as plt

#create data for plotting

x_values = [0, 1, 2, 3, 4, 5 ]
y_values = [0, 1, 4, 9, 16,25]

#the default graph style for plot is a line

[Link](x_values, y_values)

#display the graph

[Link]()

Py – NLU
 Note: if you provide a single list or array to the plot()
command,
◦ then matplotlib assumes it is a sequence of y values, and
import [Link] as plt
◦ automatically generates the [Link]([1, 2, 3, 4])
x values for you. [Link]('some numbers')
[Link]()

 Since python ranges start with 0,

the default x vector has the same length as y but starts with
0.
◦ Hence the x data are[0, 1, 2, 3].

Py – NLU 118
 text() : adds text in an arbitrary location
 xlabel(): adds text to the x-axis
 ylabel(): adds text to the y-axis
 title() : adds title to the plot
 clear() : removes all plots from the axes.
 savefig(): saves your figure to a file
 legend() : shows a legend on the plot
All methods are available on pyplot and on the axes instance
generally.

Py – NLU 119
import [Link] as plt
y1 =[]
y2 =[]
x = range(-100,100,10)
for i in x: [Link](i**2)
for i in x: [Link](-i**2)

[Link](x, y1)
[Link](x, y2)
[Link]("x")
[Link]("y") Incrementally
[Link](-2000, 2000) modify the figure.
[Link](0) # horizontal line
[Link](0) # vertical line

[Link]("[Link]") Save your figure to a file

[Link]() Show it on the screen

Py – NLU 120
 import [Link] as plt

x = [1, 2, 3, 4]
y = [1, 4, 9, 16]

[Link](x, y)

no return value?

• We are operating on a “hidden” variable representing the

figure.
• This is a terrible, terrible trick.
• Its only purpose is to pander to MATLAB users.
• I’ll show you how this works in the next lecture
Py – NLU 121
# importing the required module
import [Link] as plt

# x axis values
x = [1,2,3]
# corresponding y axis values
y = [2,4,1]
# plotting the points
[Link](x, y)

# naming the x axis

[Link]('x - axis')
# naming the y axis
[Link]('y - axis')
• Define the x-axis and corresponding y-axis values as
# giving a title to my graph lists.
[Link]('My first graph!') • Plot them on canvas using .plot() function.
• Give a name to x-axis and y-axis using .xlabel() and
# function to show the plot .ylabel() functions.
[Link]() • Give a title to your plot using .title() function.
• Finally, to view your plot, we use .show() function.

Py – NLU 122
import [Link] as plt

# line 1 points
x1 = [1,2,3]
y1 = [2,4,1]
# plotting the line 1 points
[Link](x1, y1, label="line 1")

# line 2 points
x2 = [1,2,3]
y2 = [4,1,3]
# plotting the line 2 points
[Link](x2, y2, label = "line 2")

# naming the x axis

[Link]('x - axis')
# naming the y axis
[Link]('y - axis') • Here, we plot two lines on same graph. We differentiate
# giving a title to my graph between them by giving them a name(label) which is
[Link]('Two lines on same graph!') passed as an argument of .plot() function.
# show a legend on the plot • The small rectangular box giving information about type
[Link]() of line and its color is called legend. We can add a
legend to our plot using .legend() function.
# function to show the plot
[Link]()

Py – NLU 123
import [Link] as plt

# x axis values
x = [1,2,3,4,5,6]
# corresponding y axis values
y = [2,4,1,5,2,6]

# plotting the points

[Link](x, y, color='green', linestyle='dashed', linewidth = 3,
marker='o', markerfacecolor='blue', markersize=12)

# setting x and y axis range

[Link](1,8)
[Link](1,8)

# naming the x axis

[Link]('x - axis')
# naming the y axis
[Link]('y - axis')

# giving a title to my graph

[Link]('Some cool customizations!')

# function to show the plot

[Link]()

Py – NLU 124
import [Link] as plt

#Create data for plotting

values = [5, 6, 3, 7, 2]
names = ["A", "B", "C", "D", "E"]

[Link](names, values, color="green")

[Link]()

 When using a bar graph, the change in code will be from

[Link]() to [Link]() changes it into a bar chart.

Py – NLU 125
 We can also flip the bar graph horizontally with the following
import [Link] as plt

#Create data for plotting

values = [5,6,3,7,2]
names = ["A", "B", "C", "D", "E"]

# Adding an "h" after bar will flip the graph

[Link](names, values,
color="yellowgreen")
[Link]()

Py – NLU 126
import [Link] as plt
# heights of bars
height = [10, 24, 36, 40, 5]
# labels for bars
names = ['one','two','three','four','five']

# plotting a bar chart

c1 =['red', 'green']
c2 =['b', 'g'] # we can use this for color
[Link](left, height, width=0.8, color=c1)
# naming the x-axis
[Link]('x - axis')
# naming the y-axis
[Link]('y - axis')
• Here, we use [Link]() function to
# plot title plot a bar chart.
[Link]('My bar chart!') • Also give some name to x-axis
# function to show the plot
[Link]() coordinates by defining tick_labels

Py – NLU 127
import [Link] as plt

# frequencies
ages=[2,5,70,40,30,45,50,45,43,40,44,60,7,13,57,18,90,77,32,21,20,40]
# setting the ranges and no. of intervals
range = (0, 100)
bins = 10
# plotting a histogram
[Link](ages, bins, range, color='green',histtype='bar',rwidth=0.8)
# x-axis label
[Link]('age')
# frequency label
[Link]('No. of people')
# plot title
[Link]('My histogram')
# function to show the plot
[Link]()

Py – NLU 128
  Looking at the code snippet, I added two new arguments:
◦ Bins — is an argument specific to a histogram and allows the user to customize how
many bins they want.
◦ Alpha — is an argument that displays the level of transparency of the data points.

import [Link] as plt

#generate fake data

x =
[2,1,6,4,2,4,8,9,4,2,4,10,6,4,5,7,7,3,2,7,5,3,5,9,2,1]

#plot for a histogram

[Link](x, bins = 10, color='blue', alpha=0.5)
[Link]()

Py – NLU 129
import [Link] as plt

#create data for plotting

x_values = [0,1,2,3,4,5]
y_values = [0,1,4,9,16,25]

[Link](x_values, y_values, s=30, color=“blue")

[Link]()

 Can we see the pattern? Now the code changed from [Link]() to
[Link]().

Py – NLU 130
import [Link] as plt

# x-axis values
x = [1,2,3,4,5,6,7,8,9,10]
# y-axis values
y = [2,4,5,7,6,8,9,11,12,12]
# plotting points as a scatter plot
[Link](x, y, label= "stars", color="green", marker="*", s=30)
# x-axis label
[Link]('x - axis')
# frequency label
[Link]('y - axis')
# plot title
[Link]('My scatter plot!')
# showing legend
[Link]()
# function to show the plot
[Link]()
Py – NLU 131
import [Link] as plt
# defining labels
activities = ['eat','sleep','work','play']
# portion covered by each label
slices = [3, 7, 8, 6]
# color for each label
colors = ['r', 'y', 'g', 'b']
# plotting the pie chart
[Link](slices, labels = activities, colors=colors,
startangle=90, shadow = True, explode = (0, 0, 0.1, 0),
radius = 1.2, autopct = '%1.1f%%')
# plotting legend
[Link]()
# showing the plot
[Link]()
Py – NLU 132
# importing the required modules
import [Link] as plt
import numpy as np

# setting the x - coordinates

x = [Link](0, 2*([Link]), 0.1)
# setting the corresponding y - coordinates
y = [Link](x)

# potting the points

[Link](x, y)

# function to show the plot

Examples taken from:
[Link]()
Graph Plotting in Python | Set 1

Py – NLU 133
 Different plots in the same figure using the subplots function

Py – NLU 134
 Adding an image to the plot

Py – NLU 135
136
 Python data visualization library
 Easily create the most common types of plots

Py – NLU 137
 Simplified Syntax
 Statistical Plots
 Built-in Themes and Color Palettes
 Works Well with Pandas DataFrames
 Complex Visualizations Made Easy
 Automatic Estimation and Aggregation
 Good for Exploratory Data Analysis (EDA)

Py – NLU 138
 What is the
difference between
Matplotlib and
Seaborn?

Py – NLU 139
import seaborn as sns
import [Link] as plt
height = [62, 64, 69, 75, 66, 68, 65, 71, 76,
73]
weight = [120, 136, 148, 175, 137, 165,
154, 172, 200, 187]
[Link](x=height, y=weight)
[Link]()

* Samuel Norman Seaborn ( sns )

Py – NLU 140
import seaborn as sns
import [Link] as plt

gender = ["Female", "Female", "Female",

"Female", "Male", "Male", "Male", "Male",
"Male", "Male"]
[Link](x=gender)
[Link]()

Py – NLU 141
Py – NLU 142
 Seaborn provides a wide range of plot types that can be used
for data visualization and exploratory data analysis
 Plot types:
◦ Univariate – x only (contains only one axis of information)
◦ Bivariate – x and y (contains two axis of information)
◦ Trivariate – x, y, z (contains three axis of information)

Py – NLU 143
  Scatter Plot: A scatter plot is used to visualize the relationship
between two variables.
◦ Seaborn's scatterplot() function provides a simple way to create scatter
plots.

import seaborn as sns

tips = sns.load_dataset("tips")

[Link](x="total_bill", y="tip",
data=tips)

Py – NLU 144
  Scatter Plot: A scatter plot is used to visualize the relationship
between two variables (customizing the ‘hue’ and ‘size’)
import seaborn as sns

tips = sns.load_dataset("tips")
[Link](x="total_bill", y="tip", hue="sex",
size="size", sizes=(50, 200), data=tips)
# add labels and title
[Link]("Total Bill")
[Link]("Tip")
[Link]("Relationship between Total Bill and Tip")
# display the plot
[Link]()

Py – NLU 145
  Line Plot: A line plot is used to visualize the trend of a
variable over time.
◦ Seaborn's lineplot() function provides a simple way to create line plots.

import seaborn as sns

fmri = sns.load_dataset("fmri")

[Link](x="timepoint", y="signal", data=fmri)

Py – NLU 146
  Line Plot: A line plot is used to visualize the trend of a variable over
time (customized by using ‘event’, ‘region’ columns)
import seaborn as sns
import [Link] as plt

fmri = sns.load_dataset("fmri")
# customize the line plot
[Link](x="timepoint", y="signal", hue="event",
style="region", markers=True, dashes=False, data=fmri)
# add labels and title
[Link]("Timepoint")
[Link]("Signal Intensity")
[Link]("Changes in Signal Intensity over Time")
# display the plot
[Link]() Py – NLU 147
  Histogram: A histogram is used to visualize the distribution of
a variable.
◦ Seaborn's histplot() function provides a simple way to create
histograms.

import seaborn as sns

iris = sns.load_dataset("iris")

[Link](x="petal_length", data=iris)

Py – NLU 148
  Histogram: A histogram is used to visualize the distribution of
a variable.
import seaborn as sns

import [Link] as plt

iris = sns.load_dataset("iris")
# customize the histogram
[Link](data=iris, x="petal_length", bins=20,
kde=True, color="green")
# add labels and title
[Link]("Petal Length (cm)")
[Link]("Frequency")
[Link]("Distribution of Petal Lengths in Iris Flowers")
# display the plot
[Link]() Py – NLU 149
  Box Plot: A box plot is used to visualize the distribution of a
variable.
◦ Seaborn's boxplot() function provides a simple way to create box plots.

import seaborn as sns

tips = sns.load_dataset("tips")

[Link](x="day", y="total_bill", data=tips)

Py – NLU 150
  Box Plot: (customized by including ‘time’ column)
import seaborn as sns
import [Link] as plt
# load the tips dataset from Seaborn

tips = sns.load_dataset("tips")
# customize the color scheme using the "palette"
parameter
[Link](x="day", y="total_bill", hue="time", data=tips,
palette="Set3", linewidth=1.5, fliersize=4)
# add a title, xlabel, and ylabel to the plot using Matplotlib
functions
[Link]("Box Plot of Total Bill by Day and Meal Time")
[Link]("Day of the Week")
[Link]("Total Bill ($)")
# display the plot
[Link]() Py – NLU 151
  Bar plot: used to visualize the relationship between a
categorical variable and a continuous variable ( each bar
represents the mean or median (or any aggregation) of the
continuous variable for each category).
◦ Seaborn's barplot () function provides a simple way to create bar plots.

import seaborn as sns

titanic = sns.load_dataset("titanic")

[Link](x="class", y="fare", data=titanic)

Py – NLU 152
  Bar Plot: (customized by including ‘sex’ column)
import seaborn as sns
import [Link] as plt

titanic = sns.load_dataset("titanic")
# customize the bar plot
[Link](x="class", y="fare", hue="sex", ci=None,
palette="muted", data=titanic)
# add labels and title
[Link]("Class")
[Link]("Fare")
[Link]("Average Fare by Class and Gender on the
Titanic")
# display the plot
[Link]() Py – NLU 153
  Violin Plot: A violin plot is similar to a box plot, but provides a
more detailed view of the distribution of the data.
◦ Seaborn's violinplot() function provides a simple way to create violin
plots.

import seaborn as sns

# load the iris dataset from Seaborn

iris = sns.load_dataset("iris")
# create a violin plot of petal length by species
[Link](x="species", y="petal_length", data=iris)
# display the plot
[Link]()

Py – NLU 154
  Heatmap: A heatmap is used to visualize the correlation
between different variables.
◦ Seaborn's heatmap() function provides a simple way to create
heatmaps.

import seaborn as sns

import [Link] as plt

# Load the dataset

tips = sns.load_dataset('tips').select_dtypes(include='number')
# Create a heatmap of the correlation between variables
corr = [Link]()
[Link](corr)
# Show the plot
[Link]() Py – NLU 155
 Pairplot: A pairplot is used to visualize the relationship
between multiple variables.
◦ Seaborn's pairplot() function provides a simple way to create pairplots.

import seaborn as sns

import [Link] as plt
# Load iris dataset
iris = sns.load_dataset("iris")
# Create pair plot
[Link](data=iris)
# Show plot
[Link]()

Py – NLU 156
  Pairplot: customized by using ‘hue’ and ‘diag_kind’ parameter
import seaborn as sns

import [Link] as plt

# Load iris dataset

iris = sns.load_dataset("iris")
# Create pair plot with custom settings
[Link](data=iris, hue="species",
diag_kind="kde", palette="husl")
# Set title
[Link]("Iris Dataset Pair Plot")
# Show plot
[Link]()
Py – NLU 157
 FacetGrid: a powerful seaborn tool that allows you to visualize the
distribution of one variable as well as the relationship between two
variables, across levels of additional categorical variables.
import seaborn as sns
# load the tips dataset
tips = sns.load_dataset('tips’) # create a FacetGrid for day vs total_bill
g = [Link](tips, col="day")# plot histogram for total_bill in each day
[Link]([Link], "total_bill")

Py – NLU 158
FACULTY OF INFORMATION TECHNOLOGY