Unit 7 : Advance Libraries
1
Machine Learning Libraries
* Python is one of the most popular programming language
known in market for ML due to its vast collection of libraries.
* Python libraries that used in Machine Learning are:
* Numpy
* Pandas
* Matplotlib
* Scikit-learn
* Theano
* TensorFlow
* Keras
* PyTorch
* Scipy
Silver Oak University 2 Computer Engineering
Numpy
*NumPy is a very popular python library for large multi-
dimensional array and matrix processing, with the help of a
large collection of high-level mathematical functions.
*It is very useful for fundamental scientific computations in
Machine Learning.
*It is particularly useful for linear algebra, Fourier transform,
and random number capabilities.
*High-end libraries like TensorFlow uses NumPy internally for
manipulation of Tensors.
Silver Oak University 3 Computer Engineering
Example
# Python program using NumPy
# for some basic mathematical
# operations
import numpy as np
# Inner product of vectors
# Creating two arrays of rank 2 print([Link](v, w), "\n")
x = [Link]([[1, 2], [3, 4]])
y = [Link]([[5, 6], [7, 8]]) # Matrix and Vector product
print([Link](x, v), "\n")
# Creating two arrays of rank 1
v = [Link]([9, 10]) # Matrix and matrix product
w = [Link]([11, 12]) print([Link](x, y))
Silver Oak University 4 Computer Engineering
Alais
Array using numpy
dot product of arrays
Silver Oak University 5
Print the dimension of x and y
Silver Oak University 6
Print the datatype of v
Silver Oak University 7
Other Numpy Methods
X=[Link]() Copy the array Y into X
X= [Link]() Make a view of array Y and Save in
to X
[Link] Print the shape of a 2-D array X
Y= [Link](4, 3) Convert the 1D array X into 2D
array and save it as Y
[Link](X,Y) Join two arrays
[Link]((arr1, arr2), Join two 2-D arrays along rows
axis=1) (axis=1)
[Link]() Same as concate
numpy. hstack() to stack along rows
Silver Oak University 8 Computer Engineering
Other Numpy Methods
numpy. vstack() to stack along columns
[Link]() to stack along height or depth
numpy.array_split(X,3) Split the array in 3 parts
[Link](X== 4) Return the index where 4 exist in
array X
[Link](X) Sort the array X
Silver Oak University 9 Computer Engineering
Pandas
*Pandas is a popular Python library for data analysis. It is not directly
related to Machine Learning.
*As we know that the dataset must be prepared before training.
*In this case, Pandas comes handy as it was developed specifically for
data extraction and preparation.
*It provides high-level data structures and wide variety tools for data
analysis.
*It provides many inbuilt methods for groping, combining and filtering
data.
* Pandas allows us to analyze big data and make conclusions based on
statistical theories.
* Pandas can clean messy data sets, and make them readable and
relevant.
* Relevant data is very important in data science.
Silver Oak University 10 Computer Engineering
Example
# Python program using Pandas for
# arranging a given set of data
# into a table
# importing pandas as pd
import pandas as pd
data = {"country": ["Brazil", "Russia", "India", "China", "South Africa"],
"capital": ["Brasilia", "Moscow", "New Dehli", "Beijing", "Pretoria"],
"area": [8.516, 17.10, 3.286, 9.597, 1.221],
"population": [200.4, 143.5, 1252, 1357, 52.98] }
data_table = [Link](data)
print(data_table)
Silver Oak University 11 Computer Engineering
alais
Convert the data dictionary into
dataframe.
12
Pandas
* Locate Row:
* As you can see from the result above, the DataFrame is like
a table with rows and columns.
* Pandas use the loc attribute to return one or more specified
row(s)
Silver Oak University 13 Computer Engineering
Pandas
* Locate Row:
* As you can see from the result above, the DataFrame is like
a table with rows and columns.
* Pandas use the loc attribute to return one or more specified
row(s)
Silver Oak University 14 Computer Engineering
Pandas
* Named Indexes
* With the index argument, you can name your own indexes.
Silver Oak University 15 Computer Engineering
Pandas
* Named Indexes
* With the index argument, you can name your own indexes.
Silver Oak University 16 Computer Engineering
Pandas
* Load Files Into a DataFrame :
* If your data sets are stored in a file, Pandas can load them
into a DataFrame.
* Supported files :
* CSV
* EXCEL
* ETC…..
Silver Oak University 17 Computer Engineering
Pandas
* Load Files Into a DataFrame :
* If your data sets are stored in a file, Pandas can load them
into a DataFrame.
import pandas as pd
df = pd.read_csv('[Link]')
print(df.to_string())
Silver Oak University 18 Computer Engineering
Pandas
Data Cleaning :
* Data cleaning means fixing bad data in your data set.
* Bad data could be:
* Empty cells
* Data in wrong format
* Wrong data
* Duplicates
Silver Oak University 19 Computer Engineering
Pandas
* Remove Rows with empty cell :
import pandas as pd
df = pd.read_csv('[Link]')
new_df = [Link]()
print(new_df.to_string())
Silver Oak University 20 Computer Engineering
Pandas
* Replace Empty Values : Another way of dealing with empty
cells is to insert a new value instead. This way you do not
have to delete entire rows just because of some empty
cells. The fillna() method allows us to replace empty cells
with a value:
import pandas as pd
df = pd.read_csv('[Link]')
[Link](130, inplace = True)
Note: we can also replace the empty cell in specific column. Data can be
also replaced with mean , medium or mode of data.
Silver Oak University 21 Computer Engineering
Pandas
* Fixing Wrong Data : "Wrong data" does not have to be
"empty cells" or "wrong format", it can just be wrong, like if
someone registered "999" instead of "1.99".
Update the 450 with 45 using below code of
python :
[Link][7, 'Duration'] = 45
Silver Oak University 22 Computer Engineering
Pandas
* Removing Duplicates : To discover duplicates, we can use
the duplicated() method. The duplicated() method returns a
Boolean values for each row. It Returns True for every row
that is a duplicate, otherwise False:
print([Link]())
* To remove duplicates, use the drop_duplicates() method.
df.drop_duplicates(inplace = True)
Silver Oak University 23 Computer Engineering
Matpoltlib
*Matpoltlib is a very popular Python library for data
visualization.
*Like Pandas, it is not directly related to Machine Learning. It
particularly comes in handy when a programmer wants to
visualize the patterns in the data.
*It is a 2D plotting library used for creating 2D graphs and
plots. A module named pyplot makes it easy for programmers
for plotting as it provides features to control line styles, font
properties, formatting axes, etc.
*It provides various kinds of graphs and plots for data
visualization, viz., histogram, error charts, bar chats, etc,
Silver Oak University 24 Computer Engineering
Exapmle
# Python program using Matplotib
# for forming a linear plot
# importing the necessary packages and modules
import [Link] as plt
import numpy as np
# Prepare the data
x = [Link](0, 10, 100)
# Plot the data
[Link](x, x, label ='linear')
# Add a legend
[Link]()
# Show the plot
[Link]()
Silver Oak University 25 Computer Engineering
SciPy
*SciPy is a very popular library among Machine Learning
enthusiasts as it contains different modules for optimization,
linear algebra, integration and statistics.
*There is a difference between the SciPy library and the SciPy
stack.
*The SciPy is one of the core packages that make up the SciPy
stack.
*SciPy is also very useful for image manipulation.
Silver Oak University 26 Computer Engineering
Example
# Python script using Scipy
# for image manipulation
# Saving the tinted image
imsave('D:/Programs / cat_tinted.jpg',
from [Link] import imread, imsave, img_tint)
imresize
# Read a JPEG image into a numpy array
# Resizing the tinted image to be 300 x 300
img = imread('D:/Programs / [Link]') # path of pixels
the image
img_tint_resize = imresize(img_tint, (300, 300))
print([Link], [Link])
# Saving the resized tinted image
# Tinting the image
imsave('D:/Programs / cat_tinted_resized.jpg',
img_tint = img * [1, 0.45, 0.3] img_tint_resize)
Silver Oak University 27 Computer Engineering
Scikit-learn
*scikit-learn is one of the most popular ML libraries for classical
ML algorithms.
*It is built on top of two basic Python libraries, viz., NumPy
and SciPy.
*Scikit-learn supports most of the supervised and unsupervised
learning algorithms.
*Scikit-learn can also be used for data-mining and data-
analysis, which makes it a great tool who is starting out with
ML.
Silver Oak University 28 Computer Engineering
Example
* # Python script using Scikit-learn
* # for Decision Tree Clasifier
*
* # Sample Decision Tree Classifier
* from sklearn import datasets
* from sklearn import metrics
* from [Link] import DecisionTreeClassifier
*
* # load the iris datasets
* dataset = datasets.load_iris()
*
* # fit a CART model to the data
* model = DecisionTreeClassifier()
* [Link]([Link], [Link])
* print(model)
*
* # make predictions
* expected = [Link]
* predicted = [Link]([Link])
*
* # summarize the fit of the model
* print(metrics.classification_report(expected, predicted))
* print(metrics.confusion_matrix(expected, predicted))
Silver Oak University 29 Computer Engineering
Theano
*We all know that Machine Learning is basically mathematics
and statistics.
*Theano is a popular python library that is used to define,
evaluate and optimize mathematical expressions involving
multi-dimensional arrays in an efficient manner.
*It is achieved by optimizing the utilization of CPU and GPU. It
is extensively used for unit-testing and self-verification to
detect and diagnose different types of errors.
*Theano is a very powerful library that has been used in large-
scale computationally intensive scientific projects for a long
time but is simple and approachable enough to be used by
individuals for their own projects.
Silver Oak University 30 Computer Engineering
TensorFlow
*TensorFlow is a very popular open-source library for high
performance numerical computation developed by the Google
Brain team in Google.
*As the name suggests, Tensorflow is a framework that involves
defining and running computations involving tensors.
*It can train and run deep neural networks that can be used to
develop several AI applications.
*TensorFlow is widely used in the field of deep learning
research and application.
Silver Oak University 31 Computer Engineering
Keras
*Keras is a very popular Machine Learning library for Python.
*It is a high-level neural networks API capable of running on
top of TensorFlow, CNTK, or Theano.
*It can run seamlessly on both CPU and GPU. Keras makes it
really for ML beginners to build and design a Neural Network.
One of the best thing about Keras is that it allows for easy and
fast prototyping.
Silver Oak University 32 Computer Engineering
Python Frameworks
* Python Programming language has many applications when it
comes to implementation.
* Web development being one of the applications, there is a
pressing need to understand which framework will serve your
purpose in the best way possible.
* A framework is a collection of modules or packages which
helps in writing web applications.
* While working on frameworks in python we don’t have to
worry about the low level details such as protocols, sockets or
thread management.
Silver Oak University 33 Computer Engineering
34
Python Frameworks
* Frameworks automate the common implementation of common
solutions which gives the flexibility to the users to focus on the
application logic instead of the basic routine processes.
*Frameworks make the life of web developers easier by giving them
a structure for app development. They provide common patterns in
a web application that are fast, reliable and easily maintainable.
Silver Oak University 35 Computer Engineering
Python Frameworks
* Advantages Of Frameworks
* Open-source
* Good documentation
* Efficient
* Secure
* Integration
Silver Oak University 36 Computer Engineering
operations involved in a web
application using a web
framework:
* Url Routing – Routing is the mechanism of mapping the URL directly to
the code that creates the web page.
* Input form handling and validation – Suppose you have a form which
takes some input, the idea is to validate the data and then save it.
* Output formats with template engine – A template engine allows the
developers to generate desired content types like HTML, XML, JSON.
* Database connection – Database connection configuration and
persistent data manipulation through an ORM.
* Web security – Frameworks give web security against cross-site request
forgery aka CSRF, sql injection, cross-site scripting and other common
malicious attacks.
* Session storage and retrieval – Data stored in the session storage gets
cleared when the page session ends.
Silver Oak University 37 Computer Engineering
Use of Framework?
* Frameworks make it easier to reuse the code for common
HTTP operations.
* They structure the projects in a way so that the other
developers with the knowledge of the framework can easily
maintain and build the application.
Silver Oak University 38 Computer Engineering
Use of Framework?
* Frameworks make it easier to reuse the code for common
HTTP operations.
* They structure the projects in a way so that the other
developers with the knowledge of the framework can easily
maintain and build the application.
Silver Oak University 39 Computer Engineering
Frameworks In Python
Depending upon the sort of functionalities and key features they
provide to the user, these are top 5 frameworks in python, both
micro-frameworks and full-stack frameworks.
● Django
● Web2Py
● Flask
● Bottle
● CherryPy
Silver Oak University 40 Computer Engineering
Django
● Django is a free and open-source full-stack python framework, it includes all
the necessary features by default.
● It follows the DRY principle, which says don’t repeat yourselves.
● Django uses its ORM mappers to map objects to database tables.
● An ORM or object relational mapper is a code library which helps you
manipulate the data from a database using the object-oriented paradigm.
● The main databases that django works on are PostgreSQL, MySQL, SQLite,
Oracle. It can also work with other databases using the third party drivers.
Silver Oak University 41 Computer Engineering
Django
Some of the exemplary features of django web frameworks are following:
● Authentication
● URL routing
● Template engine
● ORM
● Database Schema migrations
Django also follows MVC-MVT architecture,
MVC-MVT architecture:
MVT is slightly different from MVC, Although Django takes care of the
controller part which is the code that controls the interactions between
the model and the view. And the template is HTML file mixed with Django
template language.
Developer provides the model, view and the template. User then maps it
to the url and then the rest is done by django to serve it to the user.
Silver Oak University 42 Computer Engineering
Flask
Flask is a micro-framework. It is lightweight and its modular
design makes it easily adaptable to developer’s needs. It has a
number of out of the box features listed below:
● Built-in development server
● A fast debugger
● Integrated support for unit testing
● RESTful request dispatching
● Jinja2 templating
● Secure cookies support
● Unicode-based
● WSGI compliance
● Ability to plug any ORM
● HTTP request handling
Silver Oak University 43 Computer Engineering
CherryPy
● CherryPy is an open-source framework. It follows the
minimalist approach in building web applications.
● It makes building web applications similar to writing an
object oriented program.
● CherryPy allows us to use any type of technology for
creating templates and data access.
● It is still able to handle sessions, cookies, statics, file
uploads and everything else a web framework typically can.
Silver Oak University 44 Computer Engineering
CherryPy
● key features of CherryPy:
a. An HTTP WSGI compliant thread pooled web server
b. It has simplicity of running multiple HTTP servers at once
c. A flexible plugin system
d. Caching
e. Encoding
f. Authentication
g. Built-in support for profiling, coverage and testing
Silver Oak University 45 Computer Engineering