Machine Learning and Data

Mining

Python Lab
Seminars 1
Introduction to Google Colab
and Data Manipulation

Seminar1
Part (A) Navigating Google Colab
Environment
Part (B) Coding in Google Colab
Environment
Part (C) Data Manipulations in Google
Colab

1
Python Lab Seminar 1
Part (A) Navigating Google Colab Environment
What is Google Collaboratory and Jupyter Notebook?
Google Colab is a convenient and easy-to-use way to run Jupyter notebooks on the cloud, and
their free version comes with some access to GPUs. Jupyter Notebook (formerly known as
IPython Notebook or ipynb) is used to create interactive notebook documents that can contain
live code, equations, visualisations, media and other computational outputs. Jupyter Notebook
is often used by data scientists and students to document and demonstrate coding workflows
or simply experiment with code.

Why use Colab Notebook (.ipynb files ) and not Python Integrated
Development Environment (.py files)?
Colab notebooks allow you to combine executable code and rich text in a single document,
along with images, HTML, and more. When you create your own Colab notebooks, they are
stored in your Google Drive account. You can easily share your Colab notebooks with co-
workers or friends, allowing them to comment on your notebooks, edit them, and, more
importantly, see your results and replicate your work. This is called reproducibility. Machine
learning-related methods and processes often use randomisation functions when applied,
which hinders the reproduction of the same exact results when Python code is shared and run
on different machines among scientists. Using Colab notebooks allows for reproducibility of
results on different machines.

Colab notebooks are similar to Jupyter Notebook, is an open-source web application that
allows users to create and share documents that contain live code and results (code outputs),
including equations, visualisations, and narrative text. On the other hand, Python IDLE is an
integrated development environment (IDE) that provides a basic interface for writing and
executing Python code. (.py) files saved in Python IDLE are suitable for reproducing your
machine-learning results.

Therefore, from now on, you will save your Python Machine Learning experiments as Colab
Notebook (.ipynb). We can use Google Collaboratory Cloud Environment as a convenient
platform because Google Colab hosts these notebooks, so we don’t use our computer
resources to run the notebook.

Starting with Google Colab Notebooks Requires a Google Account

1- To launch Google Colab [Link]
2- Log in with your Google account or sign up for one.

2
3- Select New Notebook to create a new Python Colab Notebook. If you are not signed into
Google, you will be asked to do so. The other presented option, Recent, shows you the last
Python notebooks you worked with. Google Drive and GitHub allow you to load your Python
Notebooks from your Google Drive and GitHub; The Upload option allows you to load your
Python Notebooks from your PC hard drive filing system.

4- Once you start a new notebook, you can modify the notebook name according to your
preference by editing [Link]. Ensure you keep the file extension .ipynb

3
5- Download your Mall Customers Dataset from your Blackboard seminar folder. This will be
downloaded to your PC hard drive in your download folder. Remember, we are not working on
the PC environment right now; we are working on a cloud environment; therefore, we need to
transfer the dataset from your PC to the cloud hard drive  , This is called “Upload”.

6- In Colab, Select the “Files” icon to expand the files side bar.

7- To upload your Dataset from your PC (local machine) storage drive to the Colab Cloud
storage, select the “Upload to session storage” icon. Select you Mall Customer Dataset
from your directory then click “Open”.

4
8- Wait for a second, and you shall see your dataset file in the Files sidebar.

9- Sometimes, your dataset is not on your local machine. Instead, it is in your Google Drive.
Therefore, another way to upload your dataset is to mount your Google Drive first! To test this
functionality, go to your Google Drive and upload your Customers Mall Dataset to your Google
Drive. Select “+New” then “Upload file” to upload your dataset to google drive.

5
10- Once your dataset is saved to your Google Drive, from the Files sidebar, In Colab, select
“Mount Drive”. Grant Colab permissions to access your Google Drive “Connect to Google
Drive”, this will point the Colab cloud to use your Google Drive as its storage drive .

6
11- Wait for a few moments, and you will see a new directory in the Files sidebar, “drive” by
clicking on it, you can expand it to see all your files on Google Drive, one of which is your Mall
Customer Dataset.

12- Remember that mounting and retrieving files from Google Drive in Colab is much faster
than manually uploading files into Colab; this is very useful when using large datasets like
image datasets, which are multiple Giga Bytes in size. Using the “Upload” function in Colab for
these large files means a significant amount of waiting time.

13- Like a PC, the Colab environment runs hardware on a CPU; you can accelerate your
hardware using a GPU or even a TPU. CPUs, Graphics Processing Units (GPUs), and Tensor
Processing Units (TPUs) are all processors that perform computing tasks. CPUs are general-
purpose chips, GPUs are specialised for accelerated computing tasks like graphic rendering
and AI workloads, and TPUs are Google's custom Application-Specific Integrated Circuit
(ASICs) designed specifically for AI-based computing tasks. GPUs have the ability to break
complex problems into thousands or millions of separate tasks and work them out all at once,
while TPUs were designed specifically for neural network loads and have the ability to work
quicker than GPUs while also using fewer resources. Depending on your data types and size,
use these to your advantage; machine learning modelling is a resource-hungry operation and
requires faster processing power.

7
14- To use a GPU or a TPU for your Colab environment, go to the “Runtime” dropdown menu,
select “Change runtime type”, and select your preferred processing unit. Some are freely
available on the free Colab version, while others require a subscription. Our Mall customer
Dataset is a small .csv file, so for now, stay on “CPU”.

15- When creating machine learning models, it is important to keep an eye on the
environment computing resources that you are using, especially if they are expected to run
for a long period of time (i.e., hours); you need to ensure sufficient resources are there to
complete the machine learning task and avoid runtime interruptions and termination. To view
your runtime resources from the “Runtime” dropdown, select “View resources”. If you are
straining/depleting your environmental resources, consider “Change runtime type”.
Additional computing resources may cost additional subscription payment, but you do not
need this for our module.

8
Part (B) Coding in Google Colab Environment

1- To code in Colab, type your code in the black coding cell. Try it, copy and paste the
following python code into the coding cell.
print ('Hello, Welcome to your Machine Learning Module')

2- To run your code, you can select the “Run cell” button or press “Ctrl + Enter” from your
Windows keyboard. From Mac OS, press “Command + Enter” and you will see the output
below the code cell.

9
3- To add additional code cells, you can click “Insert code cell below” or you can click the
“Add code cell button” after hovering on the cell itself.

4- You may want to write notes, comments, or further explanations about the code cell you
created. To add a text cell above or below the code cell, you can simply use “Add text cell”.
You add and edit text and attach images, links, and emojis, too. This organises your notebook.
Simply double-click in the text box, edit, and then click the ESC button on your keyboard.

10
5- To delete code cells or text cells, you can click on the cell, and immediately you shall see
the “Cell menu”, from the cell menu, you can perform so many functions on the cell,
including delete, copy, paste, move cells and others in “More cell actions” including
customising your Google Colab workspace view.

6- Paste the following code in a cell code and explore the “Open editor settings” option
from the “cell menu” to add line numbers to each line of code in the code cell.

print ('Hello, Welcome to your Machine Learning Module')

x = 12
y=0
z=x+y
print ('adding', x,' to', y, 'is', z)

11
7- There are many operations from “Code cell output actions” you can perform on your
code output display, such as “hide/show”, “Clear selected output”, and “View output in
full screen” Try this code:

while True:
print ('Forever Loop ')

The above code will create an infinite loop due to the absence of a break. You can interrupt
the execution if your program takes too long to run by clicking “Interrupt” or using the
keyboard shortcut Ctrl + M + I. Alternatively, you can view the output on the full screen. See
Appendix (A) for more useful keyboard shortcuts for Colab notebooks if you want to try
them.

8- For environment configurations and package installation, we can run some shell
commands with ! to tell the configuration of the Colab environment.
!cat /proc/cpuinfo

12
More importantly, ! is used for installing new libraries into the Google Colab environment
with !pip

13
Now, try installing the following data manipulations library by running the command !pip
install pandas

Note for your coursework: When the task asks for the code
output, paste only the output from the output cell, NOT the code
cell. If the coursework task asks for the code block or line, this is
copied from the code cell.

Part (C) Data Manipulations in Google Colab

Data Manipulation is changing data into a more organised format according to one’s
requirements. Thus, Data Manipulation involves the processing of data into useful information.
Through the Pandas library, data manipulation becomes easy. Hence, let’s understand Data
Manipulation with Pandas in more detail. We will also use the Mall_Customers dataset to
show the syntax of these functions in the work.

14
Pandas is a powerful, fast, and open-source library built on NumPy. It is used for data manipulation and real-world
data analysis in Python. Easy handling of missing data, Flexible reshaping and pivoting of data sets, and size
mutability make pandas a great tool for performing data manipulation and handling the data efficiently.
1- Load your dataset in Colab with Pandas library: Reading CSV file using pd.read_csv and loading data. You
must import pandas as using pd for the shorthand.
#Importing pandas library
import pandas as pd
#Loading data into a DataFrame
data_frame=pd.read_csv('/FilePath/..../Mall_Customers.csv’)

Remember, we are using the dataset in the Colab storage, not your local machine, so we need to find its file path in
the Colab environment. Hover over your data file in the expanded file sidebar, click on the “Kebab menu” icon, then
select “Copy path”, then paste it in the read_csv function argument. Depending on the directory where your Dataset
file is stored on Colab, the file dataset path may vary. In my case, the file path ’/content/Mall_Customers.csv’. Then,
run the cell.

#Importing pandas library

import pandas as pd
#Loading data into a DataFrame
data_frame=pd.read_csv('/content/Mall_Customers.csv')

2- Accessing your dataset values: As part of data exploration, you can print the rows and
columns of your dataset on the screen to see their values. Copy and paste this cell into your
Colab. By default, data_frame.head() displays the first five rows and data_frame.tail() displays the last five rows. If
we want to get the first ‘n’ number of rows, then we use data_frame.head(n) is similar to the syntax used to print the
last n rows of the data frame, data_frame.tail(n)

Code cell:
#displaying first five rows
data_frame.head()

Output cell:

15
3- You can display the column names of the data_frame by applying the list function
Code cell:
# Program to print all the column names of the data_frame
list(data_frame.columns)

Output cell:

4- Data formatting issues: having correct data types is critical to machine learning.
Every machine learning algorithm processes a range of variables with specific data types;
without the correct data types, the machine learning algorithm throws an error. Exploring data
types tells you if there are any formatting problems in values that your dataset holds. The
functions info() prints the summary of a data_frame that includes the data type of each column.

Code cell:
data_frame.info()

Output cell:

5- Recorded data value errors: Formatting errors are not the only issue you may
experience in your dataset. There can be value errors. Value error can render your
machine learning entity's interpretation untrustworthy. One way to find errors in your dataset
is by looking at its descriptive stats. The describe() function outputs descriptive statistics, which include
those that summarise the central tendency, dispersion, and shape of a dataset’s distribution, excluding NaN values. By
default, for numeric data, the result’s index will include count, mean, std, min, and max, as well as lower, 50, and
upper percentiles. For object data (e.g. categorical values), the result’s index will include count, unique, top, and
freq. We will talk more about variable types in your first lecture. Copy, paste and run the following code cell in you
Colab notebook.

Code cell:
data_frame.describe()

Output cell:

16
Code cell:
data_frame.describe(include='object')

Output cell:

6- Dataset dimensions: when asked about your data dimension, you are asked about the
number of rows and the number of columns. You can use the shape method to obtain only
that information. In machine learning, we call columns features and the rows
instances.

Code cell:
data_frame.shape

Output cell:

7- Missing data values: one common issue you will find in datasets is missing data values.
Depending on the portion and cause of missing values, scientists can decide on a suitable
method to mitigate their missingness. Using suitable methods will likely produce reliable
analysis. Therefore, to find the number of missing values in the dataset, use
data_frame.isnull( ).sum( ). In the below example, if the dataset doesn’t contain any null
values, each column’s output is 0.

Code cell:
data_frame.isnull().sum()

Output cell:

17
To find the percentage of missing data values per variable, we understand that the length of
the dataset is the number of rows, the number of customers. Therefore, missing values in a
column occupy a percentage of the variable length

Code cell:
data_frame.isna().sum()/len(data_frame)*100

Output cell:

7- Removing Rows (instances): By using the drop(index) function, we can drop the row at
a particular index. If we want to replace the data_frame with the row removed.

Code cell:
#Removing 4th indexed value from the data_frame
data_frame.drop(4, inplace = True)
data_frame.head()

Output cell:

To remove multiple rows (instances), we can include a list of indices for those instances. For
example, remove instances 1 and 3.

Code cell:
data_frame.drop(data_frame.index[[1,3]], inplace=True)

8- Removing unnecessary variables from your analysis: Not all variables are useful for analysis; there are
unnecessary variables; we will discuss these in our lecture. This function can also be used to remove the columns of a
data frame by adding the attribute axis =1 and providing the list of columns we would like to remove. data_frame =
data_frame.drop('column_name', axis=1)

Code cell:
data_frame.drop('CustomerID',axis=1, inplace=True)
data_frame.head()

Output cell:

18
To remove multiple unnecessary variables, we use a list within the drop function that contains
a list of the column names to be removed. Assume you want to remove two more variables:
Gender and Age, try the following

Code cell:
data_frame.drop(['Gender', 'Age'],axis=1, inplace=True)
data_frame.head()

Output cell:

When trying to make changes to a Pandas data frame using a function, we use 'inplace=True'
if we want to commit the changes to the data frame. The use of inplace=True can be
summarised in the following condition:

9- Renaming columns: sometimes, you may be required to change the names of your
columns. To rename the columns, we list the original names of the columns to change and
then apply the rename() function using attribute axis=1

Code cell:
#Importing pandas library
import pandas as pd
#Loading data into a DataFrame
data_frame=pd.read_csv('/content/Mall_Customers.csv')
data_frame.rename({'CustomerID':"ID", 'Gender':"Sex", 'Annual Income
(k$)':"Salary"}, axis=1, inplace=True)
data_frame.head()

Output cell:

10- Renaming categorical values (labels) in variables: Some machine learning

algorithms cannot process categorical variables in a string format; thus, they can throw an
error. For that, we convert the (map) or (encode) each category to a numeric type of value.

19
For example, Sex has two categorical values: Male and Female, we can map them to 1 and 2
respectively in that variable.

Code cell:
data_frame['Sex'] = data_frame['Sex'].map({'Male': 1 , 'Female': 2})
data_frame.head()

Output cell:

In the case of having a variable with a large number of categories (labels), mapping them
individually can become cumbersome (difficult to handle). Therefore, you can use a
LabelEncoder() function to automatically map all categories to numeric labels based on their
alphabetical order.
Let’s reload our original dataset in its original form and rename the variables again. To
understand the difference between map() and LabelEncoder() functions.

Code cell:
import pandas as pd
data_frame=pd.read_csv('/content/Mall_Customers.csv')

data_frame.rename({'CustomerID':"ID", 'Gender':"Sex", 'Annual Income

(k$)':"Salary"}, axis=1, inplace=True)

data_frame.head()

Output cell:

Let’s check the unique values in the variable “Sex”; for that, you can use the unique()
function.
Code cell:
data_frame['Sex'].unique()

Output cell:

Now let’s import the library sklearn, which has the preprocessing package with the
LableEncoder() function to encode the categorical values for the variable “Sex”

20
Code cell:
from sklearn import preprocessing
label_encoder = [Link]()

label_encoder = [Link]()
data_frame['Sex']= label_encoder.fit_transform(data_frame['Sex'])

data_frame.head()

Output cell:

When comparing [Link]() results output from the map() and LabelEncoder() functions,
you notice that in the map(), Males and Females were encoded to 1 and 2, respectively.
However, the LabelEncoder() encoded Males as 1 and Females as 0; this numeric ranking is
due to the alphabetical order of the categories, F before M.

11- Filtering: this is another way of dropping instances. However, we can use filtering if we
know the exact values or range of values whose instances we want to remove from the data.
This can be useful if we identify a range of values that are errors or not required for the
analysis. We can exclude them from the analysis by filtering them out from the dataset.
For example, in our data, we are interested in analysing low-spending customers; we may
want to use machine learning to find similarities among them (this will be done later during
the semester).
A common operation in data analysis is to filter values based on a condition or multiple
conditions. Pandas provides a variety of ways to filter data points (i.e. rows). First, Let’s
remove the spaces in the Spending Score (1-100) variable by renaming it
Spending_Score.

Code cell:
data_frame.rename({' Spending Score (1-100)':"Spending_Score"}, axis=1,
inplace=True)
data_frame.head()

Output cell:

Now, we can use the logical operators in the column Spending_Score values to filter all
customers whose spending score was below 70. Let’s call their dataset low_spenders_data.
Once they are filtered, check the minimum and maximum values for the Spending_Score
variable in their descriptive stats.

Code cell:

21
low_spenders_data= data_frame[data_frame.Spending_Score < 70]
low_spenders_data.describe()

Output cell:

Pandas allow for combining multiple logical operators. For instance, we can apply conditions
on both the Salary and Spending_Score columns. Perhaps you want to target high earners
(above the average salary) with low spending scores (below 70). Let’s call their dataset
high_earners_low_spenders_data

Code cell:
high_earners_low_spenders_data = data_frame[(data_frame.Salary > 59.76) &
(data_frame.Spending_Score < 70)]
high_earners_low_spenders_data.describe()

Output cell:

12- Variable Construction: Data Scientists usually use values in other variables to construct
a new variable that holds new values. We can add new columns to our dataset. Let’s call it
“New Column” holding a single value of 1 for all rows.

Code cell:
#Creates a new column with all the values equal to 1
data_frame['NewColumn'] = 1
data_frame.head()

Output cell:

22
However, having a variable with values that do not vary is considered useless and
unnecessary. We can create a function that converts values from another column to fill in the
values for the newly constructed variable. For example, let’s construct a new variable,
“Customer Satisfaction”, and assume that customers with a spending score above 70 are
“satisfied” while anyone else is “unsatisfied”.

To build this box, we create a function in Python. A function in Python is defined by df

followed by the name of the function. In our case, our function box is called “ Satisfaction”,
and the argument it will be modifying or testing is the “value” of the Spending_Score
variable for customers. Let’s code this satisfaction function:

Code cell:
def satisfaction(value):
if value > 70:
return "Satisfied"
else:
return "Unsatisfied"

Now, we know that the “value” comes from the Spending_Score variable; therefore, we
point (apply) the function to the Spending_Score variable from our dataset data_frame.
This will convert the values in the Spending_Score variable to either “Satisfied” or
“Unsatisfied”.

Code cell:
data_frame['Spending_Score'].apply(satisfaction)

Output cell:

23
Note: Pandas truncated the output view of the rows; you are not able to see the full column
values. You can undo this by applying the set_options function to pandas to display more
rows.

Code cell:
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('[Link]', 150)

Now let’s assign you’re the new values of Spending_ Score to a newly constructed column
“Customer_Satisfaction”

Code cell:
data_frame['Customer_Satisfaction'] = data_frame['Spending_Score'].apply(satisfaction)
data_frame.head()

Output cell:

13- Saving you prepared dataset: Once you have prepared and cleaned your dataset, you
should save it so that you don’t need to rerun the code again. This is productive, so you can
apply machine learning. To export your Pandas data_frame to a .csv file, use the template:
data_frame.to_csv(r '/Exported NewFile Path/[Link]', index=False)

You can use the same path as the original dataset, but ensure you change the file name so that you don’t overwrite
the original dataset with the prepared one. Let’s call the clean dataset, Prepared_Mall_Customers. Also note, that
you can save your clean data with or without the instance indexes.

Code cell:
#This will save the dataset without the raws indeces
data_frame.to_csv(r'/content/Prepared_Mall_Customers.csv', index=False)

Output cell:

24
The job is not complete yet; remember, you saved your clean dataset on the Colab cloud; you
must download it now onto your local machine storage; otherwise, if you terminate Colab, it
won’t retain any dataset; for privacy and legality, Google Colab deletes all used datasets
upon exit. To save the clean dataset onto your local machine, from the expanded File
sidebar, drop the kebab menu next to your saved clean dataset, then select
“Download” and select the directory path to your preferred folder in your local machine
storage. In many cases it will be downloaded to your download folder directly via your
browser.

25
26
Appendix
Google Colab Keyboard Shortcuts
o To move the cell up ctrl+m K.
o To move the cell down ctrl+m J.
o To create a new cell below ctrl+m b.
o To create a new cell above ctrl+m a.
o To delete a cell ctrl+m d.
o To convert a text cell to a code cell ctrl + m + y.
o To convert a code cell to text cell ctrl + m + m (double tap m)
o To replace within cell ctrl + shift + h
o To replace within entire notebook ctrl + h
o Ctrl + Shift + p command palette
o Ctrl + M + C: Copy the selected cell.
o Ctrl + M + X: Cut the selected cell.
o Ctrl + M + V: Paste the copied/cut cell below the selected cell.
o Ctrl + M + D: Delete the selected cell.
o Ctrl + M + Z: Undo the last cell deletion (very handy if you accidentally delete
something important).
o Ctrl + m + i to interrupt m
o Ctrl + m + l to toggle line numbers
o Ctrl + m + o to toggle output

27