Introduction to Python Programming
Python Libraries: NumPy and Pandas
Intro to AI and Data Science
NGN 112 – Fall 2025
Department of Electrical Engineering
College of Engineering
American University of Sharjah
Prepared by Dr. Tamer Shanableh, CSE and Dr. Jamal A. Abdalla, CVE
Material mainly based on “Python for Programmers” by Paul Deitel and
Harvey Deitel, Pearson; Illustrated edition, ISBN-10 : 0135224330
Last Updated on: 29th of Sept. 2025
Table of Content
2
Python Libraries
NumPy Library
Pandas
DataFrames
Python Libraries
3
Popular libraires in Python for Data Science :
Python Libraries for Data Processing and Model Deployment
• Pandas
• NumPy
Python has many software libraries that can • Sci-Kit Learn
be imported into your program. • SciPy
• PyCaret
A software library is a collection of pre- • Tensorflow
written code, such that programmers do not • OpenCV
reinvent the wheel. Python Libraries for Data Mining and Data Scraping
• SQLAlchemy
You have previously used “import math” where • Scrapy
math is the name for the math library in • BeautifulSoup
Python. Python Libraries for Data Visualization
• Matplotlib
• Seaborn
• Ggplot
• Plotly
• Altair
Source: [Link]
data-science-in-python/196
Importing Libraries
4
▪ Import the whole library:
import numpy
myarr = [Link]([1,2,3,4])
▪ Import the whole library with an alias (as abbreviation):
import numpy as np
myarr = [Link]([1,2,3,4])
Importing a Specific Object
5
▪ Import a specific function or an object:
import [Link] as plt
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
# Create a line plot
[Link](x, y)
6 Numpy Arrays
6
The NumPy Library
7
▪ NumPy is a popular open-source library in Python
for data science and artificial intelligence.
▪ It is a standard way of working with numeric data in
Python.
▪ It can be used for creating and manipulating N-
dimensional arrays
• One-dimensional (1D) for lists of numbers
• Three-dimensional (3D) for images (R, G, B)
• Four-dimensional (4D) for videos (a sequence of 3D images)
Creating NumPy Arrays
8
▪ First step, importing the NumPy library.
import numpy as np
▪ Creating 1D arrays:
# Create a 1D array
numpy_array = [Link]([10,20,30])
# Create a 1D array from a list of numbers
data = [10, 20, 30, 40, 50]
numpy_array = [Link](data)
Numpy 2D arrays
9
▪ Creating a 2D array. Think of 2D arrays as an “Array
of Arrays” or a matrix.
import numpy as np
# Create a 2D array
arr_2D = [Link]([
[10, 20, 30, 4],
[30, 12, 67, 44],
[24, 10, 32, 0]
])
print(arr_2D)
# Print the dimensions of the array
print('Shape: ', arr_2D.shape)
# Find the number of rows and columns
rArray, cArray = arr_2D.shape
print('The number of rows is',rArray)
print('The number of columns is',cArray)
Reshaping NumPy Arrays
10
▪ You can use the NumPy reshape function to transform a 1D array
into a multidimensional array (row-wise).
▪ Example: we can reshape a 12-element 1D array into a 4x3 2D
array
▪ Clearly, reshaping a 12-element 1D array into a 4x4 2D array
will not work, and this will generate an error.
import numpy as np
arr = [Link]([1,2,3,4,5,6,7,8,9,10,11,12])
print('arr contains: \n', arr)
arr_2D = [Link](4,3)
print('arr_2D contains: \n', arr_2D)
Transposing NumPy Arrays
11
▪ You can use the np transpose function to replace rows with columns
in a 2D array
▪ The first row becomes the first column, the second row becomes the
second column, and so forth.
import numpy as np
arr = [Link]([1,2,3,4,5,6,7,8,9,10,11,12])
print('arr contains: \n', arr)
arr_2D = [Link](4,3)
print('arr_2D contains: \n', arr_2D)
#------------------------------------
arr_2D_transposed = [Link](arr_2D)
print('arr_2D_transposed contains: \n', arr_2D_transposed)
NumPy Sorting
12
▪ Use the sort function to order the array values in ascending order.
▪ The sort option axis determines whether to sort the whole array (axis
= None), sort row-wise (axis = 1), or sort column-wise (axis = 0).
#Numpy Example: sort # Sort the whole array
method rst = [Link](arr_2D,axis=None)
import numpy as np print('sort the whole Array: \n', rst)
arr_2D = [Link]([ # Sort row-wise (axis = 1)
[10, 20, 30, 4], rst = [Link](arr_2D,axis=1)
[30, 12, 67, 44], print('Row-wise sorting: \n',rst)
[24, 10, 32, 0]
]) # Sort column-wise (axis = 0)
print(arr_2D) rst = [Link](arr_2D,axis=0)
print('Column-wise sorting: \n',rst)
NumPy Calculation Functions
13
▪ We can use the sum, min, max, mean, std, and var functions on NumPy
arrays. The function also makes use of the axis option.
▪ An example of using of sum is shown below.
import numpy as np
grades = [Link]([[87,96, 70], [100, 87, 90], [94,77,
90],[100, 81, 82]])
print('The grades are: \n', grades)
sum = [Link](axis=1) # row-wise
print('Summation row-wise:\n',sum)
sum = [Link](axis=0) # col-wise
print('Summation col-wise:\n',sum)
sum = [Link](axis=None) # all
print('Summation of all grades:\n',sum)
NumPy Calculation Functions
14
▪ An example of using of min is shown below.
import numpy as np
grades = [Link]([[87,96, 70], [100, 87, 90], [94,77,
90], [100, 81, 82]])
print('The grades are: \n', grades)
min = [Link](axis=1) # row-wise
print('min row-wise:\n',min)
min = [Link](axis=0) # col-wise
print('min col-wise:\n',min)
min = [Link](axis=None) # all
print('min of all grades:\n',min)
Indexing and Slicing (1/4)
15
▪ Arrays in NumPy use a zero-indexing scheme.
▪ This scheme applies to row and column indexing.
import numpy as np
grades = [Link]([[87,96, 70], [100, 87, 90], [94,77,
90],[100,81, 82]])
print('The grades are: \n', grades)
# Select one grade using: grade[row index, col index]
print('grades[0,0] = ', grades[0,0])
print('grades[1,2] = ', grades[1,2])
# Select one row of grades using: grade[row index]
print('grades[3] = ', grades[3])
Indexing and Slicing (2/4)
16
▪ Multiple rows can be selected from a NumPy array.
▪ Select multiple sequential rows of grades using array_name[row
index from : row index to]. However, this will exclude the row
with the last index as shown in the example below.
import numpy as np
grades = [Link]([[87,96, 70],[100, 87, 90],[94,77,90],
[100, 81, 82]])
print('The grades are: \n', grades)
# Select multiple sequential rows of grades using :
grade[row index from : row index to]
print('grades[0:2] = \n', grades[0:2]) #up to but not
including row 2
Indexing and Slicing (3/4)
17
▪ You can select a subset of columns in NumPy arrays
• grades[:,0] means select all rows, column 0
• grades[:, 0:2] means select all rows,
columns 0,1 (up to but not including 2)
import numpy as np
grades = [Link]([[87,96, 70], [100, 87, 90], [94,77,90],
[100, 81, 82]])
print('The grades are: \n', grades)
print('First column | grades[:,0] = \n', grades[:,0])
print('Last 2 columns| grades[:, 1:3] = \n', grades[:,1:3])
Adopted from [Link]
exercises/numpy/[Link]
Indexing and Slicing (4/4)
18
▪ Python allows negative indices in arrays
▪ One particularly important case is the access of the last
column using the negative column index of ‘-1’
import numpy as np
grades = [Link]([[87,96, 70], [100, 87, 90], [94,77,90],
[100, 81, 82]])
print('The grades are: \n', grades)
print('First column | grades[:,0] = \n', grades[:,0])
print('Last column | grades[:, -1] = \n', grades[:,-1])
19 Pandas: Series and DataFrames
Pandas Series and DataFrames (1/2)
20
▪ NumPy arrays are optimized for homogenous numeric data.
▪ However, in machine learning (ML) applications, we need to
provide:
• Support for heterogeneous types (e.g., numbers and strings).
• Support for missing data.
• Support for headers and indices (as shown in the next slide).
▪ Pandas is the commonly used library for dealing with such
data.
▪ It provides support for:
• Series: for 1D collections (enhanced 1D array).
• DataFrames: for 2D collections (enhanced 2D array).
Pandas Series and DataFrames (2/2)
21
Index value
Index header header header header
Rest of columns are called “values”
First column is called “index”
Pandas Series (1/2)
22
▪ A Series is an enhanced 1D array.
▪ It can be indexed using integers like NumPy or strings.
import pandas as pd
grades = [Link]([87, 100, 94])
print('Grades Series:\n',grades)
print('First grade: ',grades[0])
Output (index and value):
0 87
1 100
2 94
First grade: 87
Pandas Series (2/2)
23
▪ Series type provides statistical functions like count, mean,
min, max, and std.
▪ For a full numerical summary, you can use the describe
function.
import pandas as pd
grades = [Link]([87, 100, 94])
print('Grades Series:\n',grades)
print('Count: ', [Link]())
print('Mean: ', [Link]())
print('Min: ', [Link]())
print('Max: ', [Link]())
print('Std: ', [Link]())
# for an overall summary you can use:
print('Description:\n',[Link]())
Series with a Custom Index
24
▪ You can use custom indices with the index
argument. Index value
import pandas as pd
grades = [Link]([87, 100, 94],
index=['First', 'Second', 'final'])
print(grades)
Output:
First 87
Second 100
final 94
Accessing Series Using String Indices
25
▪ In the previous example, a Series with custom indices can be accessed
via square brackets [ ] containing a custom index value.
import pandas as pd
grades = [Link]([87, 100, 94], index=['First',
'Second', 'final'])
print('Grade of first = ',grades['First']) # or
print('Grade of first = ',grades[0])
# You can also access all values and all indices
print('Series values are: ', [Link])
print('Series indices are: ', [Link])
Output:
Grade of first = 87
Grade of first = 87
Series values are: [ 87 100 94]
Series indices are: Index(['First', 'Second', 'final'],
dtype='object')
DataFrames
26
▪ DataFrames are enhanced 2D
Index header header header header
arrays
▪ They can have custom indices
(rows) and headers
▪ Each column in a DataFrame is
a Series
▪ DataFrames can be created
by reading files or using
dictionaries (dict).
Creating DataFrames From Files
27
▪ Pandas provides a read_csv() function to read data
stored as a .csv file into a Pandas DataFrame.
▪ Pandas supports many different file formats, including .csv
and Excel (.xlsx):
myDataFrame = pd.read_csv("[Link]")
myDataFrame = pd.read_excel("[Link]")
▪ After reading a file, you can display the first and last 5 rows
using [Link]()
▪ To save data from DataFrames to files, use:
myDataFrame.to_csv("[Link]")
myDataFrame.to_excel("[Link]")
Creating DataFrames From Files in Colab
28
▪ How to upload files to Google Colab?
Click to upload a file
I uploaded this file
▪ After uploading the file, you can read it (read_csv or
read_excel ).
▪ Files saved by the user (to_csv or to_excel) are also located in
the same place in Google Colab.
Creating DataFrames From Public Data (1/3)
29
▪ We will use the Iris sample data, which contains information on 150
Iris flowers, 50 each from one of three Iris species: Setosa,
Versicolour, and Virginica.
▪ Each flower is characterized by five attributes:
1. sepal_length in centimeters
2. sepal_width in centimeters
3. petal_length in centimeters
4. petal_width in centimeters
▪ Each flower belongs to one type, which is the last column in the
DataFrame:
(Setosa, Versicolour, Virginica)
▪ Data is available online at:
[Link]
Iris Flowers Dataset
30
Creating DataFrames From Public Data (2/3)
31
▪ Check the comments in the code below.
import pandas as pd
#The argument header=None says that this dataset does not
contain a header yet, so we will add one next
data = pd.read_csv('[Link]',header=None)
# data = pd.read_csv([Link]
learning-databases/iris/[Link]’)
#You can then add column headers
[Link]=['sepal_length','sepal_width','petal_length','pe
tal_width','class']
#And display the first 5 rows to make sure that the reading
is successful
[Link]()
Creating DataFrames From Public Data (3/3)
32
Output:
Accessing DataFrame Columns and Rows (1/4)
33
▪ You can use access one column in a Dataframe using the
header’s name.
Output:
# Access one column using a header’s name petal_length
columns:
print('petal_length 0 1.4
columns:\n',data['petal_length']) 1 1.4
2 1.3
3 1.5
4 1.4
▪ You can use access one row using iloc function 145
...
5.2
146 5.0
# Access one row using the .iloc function 147 5.2
print('\n\nFirst row:') 148 5.4
149 5.1
print([Link][0])
Output: First row:
sepal_length 5.1
sepal_width 3.5
petal_length 1.4
petal_width 0.2
class Iris-setosa
Accessing DataFrame Columns and Rows (2/4)
34
▪ You can use access a sequential slice of rows using iloc function.
# Access a sequential slice of rows using the .iloc
function
print('\n\nFirst 5 rows:')
print([Link][0:5]) # up to but not including 5
Output:
First 5 rows:
sepal_length sepal_width petal_length petal_width class
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
Accessing DataFrame Columns and Rows (3/4)
35
▪ You can access a sequential slice of rows and columns using iloc
function.
# Access a sequential slice of rows and columns using the
.iloc function
print('\n\nFirst 5 rows and first 2 columns:')
# Print up to but not including row 5, up to but not
including col 2
#.iloc[ rows from:to , cols from:to ]
print([Link][0:5 , 0:2 ])
Output: First 5 rows and first 2
columns:
sepal_length sepal_width
0 5.1 3.5
1 4.9 3.0
2 4.7 3.2
3 4.6 3.1
4 5.0 3.6
Accessing DataFrame Columns and Rows (4/4)
36
# Access a sequential slice of rows and columns using the
.iloc function
print('\n\nFirst 5 rows and first 2 columns:’)
# Print up to but not including row 5, and cols 0,1 and the
last column
#.loc[ rows from:to , [cols indices] ]
print([Link][0:5 , [0,1,-1]])
Output: sepal_length sepal_width class
0 5.1 3.5 Iris-setosa
1 4.9 3.0 Iris-setosa
2 4.7 3.2 Iris-setosa
3 4.6 3.1 Iris-setosa
4 5.0 3.6 Iris-setosa
DataFrames Boolean Indexing (1/5)
37
▪ Pandas provide a powerful selection feature called Boolean
indexing.
▪ That is, you can use a Boolean expression that returns True/False
to filter a DataFrame.
▪ Let us start by extracting the numeric data from our DataFrame.
data_numeric = [Link][:, 0:4]
data_numeric.head()
Output:
First 5 rows and first 2 columns:
sepal_length sepal_width class
0 5.1 3.5 Iris-setosa
1 4.9 3.0 Iris-setosa
2 4.7 3.2 Iris-setosa
3 4.6 3.1 Iris-setosa
4 5.0 3.6 Iris-setosa
DataFrames Boolean Indexing (2/5)
38
▪ Pandas checks every element to determine
whether its value is greater than or equal to
# from the previous slide
data_numeric = [Link][:, 0:4] 5.0.
#Filter the dataFrame, locate values >=
▪ If True then it includes it in the new DataFrame
5.0 (rst in the example above).
rst = data_numeric[data_numeric >= 5.0]
print(rst) ▪ Elements for which the condition is False are
represented as NaN (not a number) in the new
Output: DataFrame.
sepal_length sepal_width petal_length petal_width
0 5.1 NaN NaN NaN
1 NaN NaN NaN NaN
2 NaN NaN NaN NaN
3 NaN NaN NaN NaN
4 5.0 NaN NaN NaN
.. ... ... ... ...
145 6.7 NaN 5.2 NaN
146 6.3 NaN 5.0 NaN
147 6.5 NaN 5.2 NaN
148 6.2 NaN 5.4 NaN
149 5.9 NaN 5.1 NaN
[150 rows x 4 columns]
DataFrames Boolean Indexing (3/5)
39
▪ In a Boolean expression, you can use:
▪ AND, which is the & operator
▪ OR, which is the | operator
data_numeric = [Link][:, 0:4]
rst = data_numeric[data_numeric >= 5.0]
print([Link]())
#Other examples (data_numeric >= 3.0) AND (data_numeric <=
5.0):
rst = data_numeric[(data_numeric >= 3.0) & (data_numeric <=
5.0)]
print([Link]())
#Other examples (data_numeric < 3.0) OR (data_numeric > 5.0):
rst = data_numeric[(data_numeric < 3.0) | (data_numeric > 5.0)]
print([Link]())
DataFrames Boolean Indexing (4/5)
40
▪ In a Boolean expression, you can use the .loc function to filter rows
according to Boolean criteria.
import pandas as pd
data = pd.read_csv('[Link]',header=None)
[Link]=['sepal_length','sepal_width','petal_length','petal_width','cla
ss']
print('----------------------------------------------')
#Select row where sepal_length >= 5.0
rst = [Link][ data.sepal_length >= 5.0 ]
print('Select row where sepal_length >= 5.0')
print([Link]())
print('----------------------------------------------')
#Select row where sepal_length >= 5.0 AND & data.sepal_width >= 3.5
rst = [Link][ (data.sepal_length >= 5.0) & (data.sepal_width >= 3.5)]
print('Select row where sepal_length >= 5.0 & data.sepal_width >= 3.5')
print([Link]())
DataFrames Boolean Indexing (5/5)
41
Output: ----------------------------------------------
Select row where sepal_length >= 5.0
sepal_length sepal_width petal_length petal_width class
0 5.1 3.5 1.4 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
5 5.4 3.9 1.7 0.4 Iris-setosa
7 5.0 3.4 1.5 0.2 Iris-setosa
10 5.4 3.7 1.5 0.2 Iris-setosa
----------------------------------------------
Select row where sepal_length >= 5.0 & data.sepal_width >= 3.5
sepal_length sepal_width petal_length petal_width class
0 5.1 3.5 1.4 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
5 5.4 3.9 1.7 0.4 Iris-setosa
10 5.4 3.7 1.5 0.2 Iris-setosa
14 5.8 4.0 1.2 0.2 Iris-setosa
DataFrames Statistics (1/2)
42
▪ Similar to Series, you can use the describe() function to print out
statistics.
▪ In DataFrames, the statistics are calculated by column (for the
numeric columns only).
print([Link]())
Output:
sepal_length sepal_width petal_length petal_width
count 150.000000 150.000000 150.000000 150.000000
mean 5.843333 3.054000 3.758667 1.198667
std 0.828066 0.433594 1.764420 0.763161
min 4.300000 2.000000 1.000000 0.100000
25% 5.100000 2.800000 1.600000 0.300000
50% 5.800000 3.000000 4.350000 1.300000
75% 6.400000 3.300000 5.100000 1.800000
max 7.900000 4.400000 6.900000 2.500000
DataFrames Statistics (2/2)
43
▪ Similar to Series, you can use the mean(), min(), max(), std(), var().
▪ In DataFrames, the statistics are calculated by column (for the
numeric columns only).
▪ If the DataFrame has strings, the functions will return an error.
Output:
print('Avg per col:') Avg per col:
sepal_length 5.843333
print(data_numeric.mean()) sepal_width 3.054000
print('Std per col:') petal_length 3.758667
print(data_numeric.std()) petal_width 1.198667
print('Min per col:')
Std per col:
print(data_numeric.min()) sepal_length 0.828066
print('Max per col:') sepal_width 0.433594
print(data_numeric.max()) petal_length 1.764420
petal_width 0.763161
…
Converting DataFrames to/from NumPy (1/3)
44
▪ There are cases where you need to convert a DataFrame into a NumPy Array and vice
versa
▪ This is needed in machine learning tasks like classification and regression that you will
study later.
▪ Let us start by converting a DataFrame into a NumPy array using the to_numpy()
function.
import pandas as pd
data = pd.read_csv('[Link]',header=None)
[Link]=['sepal_length','sepal_width','petal_length','petal_width', 'class']
#Convert a dataFrame into a numPy array
numpy_from_dataFrame = data.to_numpy()
print(numpy_from_dataFrame)
#OR: Convert the first 4 columns of a dataFrame into a numPy array
numpy_from_dataFrame_numeric = [Link][:, 0:4].to_numpy()
print(numpy_from_dataFrame_numeric)
Converting DataFrames to/from NumPy (2/3)
45
Output of Output of
data.to_numpy(): [Link][:, 0:4].to_numpy():
[[5.1 3.5 1.4 0.2 'Iris-setosa’] [[5.1 3.5 1.4 0.2]
[4.9 3.0 1.4 0.2 'Iris-setosa'] [4.9 3. 1.4 0.2]
[4.7 3.2 1.3 0.2 'Iris-setosa'] [4.7 3.2 1.3 0.2]
[4.6 3.1 1.5 0.2 'Iris-setosa'] [4.6 3.1 1.5 0.2]
[5.0 3.6 1.4 0.2 'Iris-setosa'] [5. 3.6 1.4 0.2]
[5.4 3.9 1.7 0.4 'Iris-setosa'] [5.4 3.9 1.7 0.4]
[4.6 3.4 1.4 0.3 'Iris-setosa'] [4.6 3.4 1.4 0.3]
[5.0 3.4 1.5 0.2 'Iris-setosa'] [5. 3.4 1.5 0.2]
[4.4 2.9 1.4 0.2 'Iris-setosa'] [4.4 2.9 1.4 0.2]
[4.9 3.1 1.5 0.1 'Iris-setosa'] [4.9 3.1 1.5 0.1]
[5.4 3.7 1.5 0.2 'Iris-setosa'] [5.4 3.7 1.5 0.2]
[4.8 3.4 1.6 0.2 'Iris-setosa'] [4.8 3.4 1.6 0.2]
… …
Converting DataFrames to/from NumPy (3/3)
46
▪ To convert a NumPy array into a DataFrame, we can use the
command [Link]().
▪ Notice how you can add columns (which are the headers),
using the argument columns = […]
dataFrame_from_numpy = [Link](numpy_from_dataFrame,
columns = ['sepal_length', 'sepal_width', 'petal_length',
'petal_width','class'])
dataFrame_from_numpy.head()
Output:
Creating DataFrames using Dictionaries (1/2)
47
▪ A dictionary can be used to create a Dataframe as shown below.
▪ The dictionary keys will be the Dataframe headers.
▪ The values become the element values in the corresponding column.
Output:
import pandas as pd Name Age Gender
0 Braund, Mr. Owen Harris 22 male
df = [Link]({ 1 Allen, Mr. William Henry 35 male
"Name":["Braund, Mr. Owen Harris", 2 Bonnell, Miss. Elizabeth 58 female
"Allen, Mr. William Henry",
"Bonnell, Miss. Elizabeth"],
"Age":[22, 35, 58],
"Gender":["male","male", "female"]
})
print(df)
[Link]()
Creating DataFrames using Dictionaries (2/2)
48
# Example 2
import pandas as pd
my_dictionary={
"Name": ["Dr. Sami Batata",
"Prof. Marwa Halawah",
"Mr. Fawzi Kamal"],
"Age": [29, 40, 60],
"Gender": ["male", "female", "male"]}
df = [Link]( my_dictionary)
print(df)
Output:
Name Age Gender
0 Dr. Sami Batata 29 male
1 Prof. Marwa Halawah 40 female
2 Mr. Fawzi Kamal 60 male