Huawei Confidential
Foreword
In this course, we are going to learn the third-party libraries of
Python programming, including NumPy, pandas, matplotlib,
SciPy, and scikit-learn.
Huawei Confidential
Objectives
Huawei Confidential
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
Third-party Libraries Installing
Input the following pip commands in your command prompt in
Windows, or terminal in Linux and MacOS.
pip install numpy
pip install pandas
pip install matplotlib
pip install scipy
pip install scikit-learn
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
NumPy
[Link]
NumPy (Numerical Python) is an open source Python library
that’s used in almost every field of science and engineering.
Huawei Confidential
NumPy
The core of the NumPy package, is the ndarray object, a homogeneous
n-dimensional array object, with methods to efficiently operate on it.
NumPy is partially written in Python, but most of the parts that
require fast computation are written in C or C++.
The NumPy API is used extensively in Pandas, SciPy, Matplotlib,
scikit-learn, scikit-image and most other data science and scientific
Python packages.
Huawei Confidential
Why NumPy?
NumPy can be used to perform a wide variety of mathematical
operations on arrays. It adds powerful data structures to Python
that guarantee efficient calculations with arrays and matrices.
Python lists are slow to process. NumPy ndarray is up to 50x
faster than traditional Python lists.
NumPy supplies an enormous library of high-level mathematical
functions that operate on ndarrays.
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
Pandas
[Link]
Huawei Confidential
Pandas
Pandas is an open source, BSD-licensed library providing
high-performance, easy-to-use data structures and data
analysis tools for the Python programming language.
Provides high-level building block for doing practical data
analysis in Python.
Powerful and flexible open source data analysis tool.
Huawei Confidential
Features of pandas
Accessible to everyone
Free for users to use and modify
Flexible
Powerful
Easy to use
Fast
Huawei Confidential
Data structures of pandas
There are three main data structures in pandas:
Series
DataFrame
Panel
The most widely used pandas data structure is DataFrame.
Huawei Confidential
Series
Series is a one-dimensional labeled array capable of holding any
data type (integers, strings, floating point numbers, Python
objects, etc.). The axis labels are collectively referred to as the
index.
Series is ndarray-like.
Series is dict-like.
Huawei Confidential
DataFrame
DataFrame is a two-dimensional labeled data structure with
columns of potentially different types.
Like Series, DataFrame accepts many different kinds of input:
Dict of 1D ndarrays, lists, dicts, or Series
2-D [Link]
Structured or record ndarray
A Series
Another DataFrame
Huawei Confidential
Panel
A panel is a 3D container of data.
The three axes of a panel are:
items − axis 0, each item corresponds to a DataFrame contained inside.
major_axis − axis 1, it is the index (rows) of each of the DataFrames.
minor_axis − axis 2, it is the columns of each of the DataFrames.
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
Matplotlib
[Link]
Matplotlib is a comprehensive library for creating static, animated,
and interactive visualizations in Python.
Huawei Confidential
Matplotlib
Matplotlib is a library for making plots of arrays in Python.
Although Matplotlib is written primarily in pure Python, it makes
heavy use of NumPy and other extension code to provide good
performance.
Matplotlib is designed with the philosophy that you should be
able to create simple plots with just a few commands.
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
SciPy
[Link]
SciPy: Scientific Python, is
the Scientific computing
tools for Python.
SciPy is a Python-based
ecosystem of open-source
software for mathematics,
science, and engineering.
Huawei Confidential
SciPy
CORE PACKAGES:
NumPy Pandas
SciPy library IPython
Matplotlib SymPy
Huawei Confidential
CORE PACKAGES:
NumPy, the fundamental package for numerical computation. It defines the
numerical array and matrix types and basic operations on them.
The SciPy library, a collection of numerical algorithms and domain-specific
toolboxes, including signal processing, optimization, statistics, and much more.
Matplotlib, a mature and popular plotting package that provides publication-
quality 2-D plotting, as well as rudimentary 3-D plotting.
pandas, providing high-performance, easy-to-use data structures.
IPython, a rich interactive interface, letting you quickly process data and test ideas.
SymPy, for symbolic mathematics and computer algebra.
Huawei Confidential
SciPy library
The SciPy library is one of the core packages that make up the SciPy stack. It
provides many user-friendly and efficient numerical routines, such as routines for
numerical integration, interpolation, optimization, linear algebra, and statistics.
SciPy is a collection of mathematical algorithms and convenience functions built on
the NumPy extension of Python. It adds significant power to the interactive Python
session by providing the user with high-level commands and classes for
manipulating and visualizing data.
Huawei Confidential
Contents
1. Third-party Libraries Installing
2. NumPy
3. Pandas
4. Matplotlib
5. SciPy
6. Scikit-Learn
Huawei Confidential
Huawei Confidential
Scikit-learn
[Link]
Scikit-learn is an open source machine learning library that supports numerous
machine learning algorithms. It also provides various tools for model fitting, data
preprocessing, model selection and evaluation, and many other utilities.
Huawei Confidential
Features of scikit-learn
Fitting and predicting: estimator basics
Scikit-learn provides dozens of built-in machine learning algorithms and models,
called estimators. Each estimator can be fitted to some data using its fit method.
Transformers and pre-processors
Machine learning workflows are often composed of different parts. A typical
pipeline consists of a pre-processing step that transforms or imputes the data,
and a final predictor that predicts target values.
Huawei Confidential
Features of scikit-learn
Pipelines: chaining pre-processors and estimators
Transformers and estimators (predictors) can be combined together into a
Pipeline. The pipeline offers the same API as a regular estimator: it can be
fitted and used for prediction with fit and predict.
Using a pipeline will also prevent us from data leakage, i.e. disclosing some
testing data in your training data.
Model evaluation
Fitting a model to some data does not entail that it will predict well on unseen
data. This needs to be directly evaluated. Scikit-learn provides many other
tools for model evaluation, in particular for cross-validation.
Huawei Confidential
Features of scikit-learn
Automatic parameter searches
All estimators have parameters (often called hyper-parameters) that can be
tuned. The generalization accuracy of an estimator often critically depends on
a few parameters.
Quite often, it is not clear what the exact values of these parameters should
be since they depend on the data at hand. Scikit-learn provides tools to
automatically find the best parameter combinations (via cross-validation).
Huawei Confidential
Summary
This chapter introduces Know how to install python third-party
libraries, the concepts and basic usages of NumPy, pandas,
matplotlib, SciPy, and scikit-learn.
Huawei Confidential
More Information
Online learning website
[Link]
Huawei Knowledge Base
[Link]
Huawei Confidential
把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.
Thank you. Copyright© 2020 Huawei Technologies Co., Ltd.
All Rights Reserved.
The information in this document may contain predictive
statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors
that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Huawei Confidential