GRADUATE SCHOOL OF MANAGEMENT

AND BUSINESS ADMINISTRATION

Final approval by Order No. 4677/MES/CAB of July 5, 2017
Accredited by the African and Malagasy Council for Higher Education (CAMES)
BP: 2339– Brazzaville– CONGO
Email: esgae@[Link] Website: [Link]

Department of License

DATA ANALYSIS WITH PYTHON

Course
Bachelor's Degree - Database Administration

Teachers
Teaching team
 COURSE MATERIALS

Data analysis
with python

This document was produced by: Mr. PREFILS NGUENGORO KEKARAT

COMPUTER ENGINEER.
 DATA ANALYSIS WITH PYTHON

Academic Year: 2021-2022

GENERAL OBJECTIVE: This course aims to provide the student with all the
necessary knowledge to analyze and visualize data with python.

SPECIFIC OBJECTIVES :
At the end of this course, the student should be capable of:
•To use Python's scientific libraries;
To analyze data with Numpy;
•To read a dataset with Pandas/explore a dataframe;
Manipulating data with Pandas;
To learn how to handle missing values;
To learn how to draw, customize, and interpret curves from
real data
To create the bar chart, the scatter plots,
histograms and box plots;
To tackle concrete and real data science projects.

CONTENT

CHAPTER I: INTRODUCTION

CHAPTER II: PYTHON RECAP

CHAPTER III: NUMPY LIBRARY

CHAPTER IV: DATA ANALYSIS WITH NUMPY

CHAPTER V: PANDAS LIBRARY

CHAPTER VI: DATA MANIPULATION WITH PANDAS

CHAPTER VII: SERIES WITH PANDAS

[Link] NGUENGORO Computer Engineer 2

 DATA ANALYSIS WITH PYTHON

CHAPTER VIII: DATAFRAMES WITH PANDAS

CHAPTER IX: GRAPHICS/THE CURVES

CHAPTER X: BAR CHARTS AND NUAGE GRAPHS

DEPOINTS

CHAPTER XI: HISTOGRAMS AND BOX PLOTS

[Link] NGUENGORO Computer Engineer 3

 DATA ANALYSIS WITH PYTHON

Table of Contents
CHAPTER I: INTRODUCTION
1. INSTALLATION PYTHON + JUPYTER NOTEBOOK .............................................. 6
2. Presentation Jupyter Notebook........................................................................................ 7
CHAPTER II: PYTHON RECAP
1. Variables & Data types........................................................................................ 8
Lists and dictionaries...................................................................................................... 8
3. Conditions IF/ELSE ........................................................................................................ 9
4. Loops FOR ................................................................................................................. 10
5. Functions Python........................................................................................................... 11
CHAPTER III: Numpy Library............................................................. 12
1. Introduction to Numpy ................................................................................................... 12
Predefined variabless ............................................................................................................ 12
Variable pi: NumPy allows obtaining the value of pi ................................. 12
2. Paintings with Numpy................................................................................................... 12
3. Size of a table......................................................................................................... 12
4. Lecture from a dataset with numpy
5. Types data with Numpy ..................................................................................... 13
Extraction of a value from a Numpy array ...................................................... 14
Slicing 2D Arrays ............................................................................................. 14
7. Extraction from a vector of values from a Numpy array...................................... 15
CHAPTER IV: Data Analysis with Numpy
1. Objectives 15
2. Comparisons ................................................................................................................ 16
3. Selection elements
Trigonometric functionss .................................................................................................. 17
Hyperbolic functions.......................................................................................... 17
Various functions
Useful functions for complex numberss ...................................................................... 18
CHAPTER V: Pandas Library
1. Introduction ................................................................................................................... 18
2. Presentation the datasett .................................................................................................. 18
CHAPTER VI: Data Manipulation with pandas ......................................................... 20
1. Introduction ................................................................................................................... 20
2. Transformation of a column ...................................................................................... 20
3. Operations mathematics between columnss .20

M. Prefils NGUENGORO Computer Engineer 4

 DATA ANALYSIS WITH PYTHON

CHAPTER VII: Series with Pandas

Structures data.................................................................................................... 22
2. Indexing with integerss .22
3. Reindexing of a series object ...................................................................................... 22
4. Trie of a series object..................................................................................................... 22
5. Transformation of a column ...................................................................................... 22
6. Alignment data................................................................................................ 23
CHAPTER VIII: DataFrames with pandas............................................................................ 23
1. Index shared by all columns......................................................... 23
2. Method Apply() ........................................................................................................... 23
CHAPTER IX: Graph: the curves ................................................................................ 24
1. Introduction 24
2. Creation of a curve ................................................................................................... 24
3. Definition of the domain of the axes .................................................................................... 25
4. Addition of a titlee .............................................................................................................. 25
5. Addition of a legend
6. Labels on the axes
7. Display of several curves ..................................................................................... 26
CHAPTER X: Bar charts and scatter plots.................................. 26
CHAPTER XI: Histograms and Box Plotss .......................................................... 28

M. Prefils NGUENGORO Computer Engineer 5

 DATA ANALYSIS WITH PYTHON

CHAPTER I: INTRODUCTION

1. INSTALLATION PYTHON + JUPYTER NOTEBOOK

Anaconda is therefore a Python distribution made for Data Science.

He will therefore install: Python and the Data Science libraries we will need:
Matplotlib, Scipy, Numpy, Pandas, the Jupyter notebook, which I recommend to you
eager to use.
Installation on Windows:
Download the installation file forWindowsor MacOs, thenlaunch it in
double-clicking on the file that has downloaded.
Answer the questions that are asked of you. The default options are generally
acceptable, no need to modify them. Once the installation is complete, check that
Everything went well when launching the Jupyter program.
Installation under Linux
Download the installer for Linux.
Enter the following instruction in the command:
bash ~/Downloads/Anaconda3-5.3.0-Linux-x86_64.h
L’installeur affiche :“In order to continue the installation process, please review the
license agreement."Display the terms of the license for use, scroll all the way down.
and click on yes to accept.
Répondez "yes" à la question qui suit :
Do you wish the installer to prepend the Anaconda3 install location?
to PATH in your /home/ec2-user/.bashrc? [yes|no]
The previous command added to the .bashrc file the path (the folder) in which
Anaconda is located. Thus, you can start Anaconda directly by typing
only "anaconda" in your console! This operation will take effect at
restart of your computer, or as soon as this command is executed:
source .bashrc
Check that everything went well by launching the Jupyter program.
Launch Jupiter
On macOS, launch Anaconda Navigator via the Launchpad.
On Linux, open a console and run: anaconda-navigator.
On Windows, launch Anaconda Navigator from your program list.

M. Prefils NGUENGORO Computer Engineer 6

 DATA ANALYSIS WITH PYTHON

2. Présentation Jupyter Notebook

Jupyter Notebook is a powerful tool that allows users of the Python language to create
and to share interactive documents containing dynamic and executable code,
visualizations of content, documentation texts and equations. The term 'notebook'
is related to the intrinsic nature of the tool that allows writing small pieces of executable code
(called 'cells'), to document them to explain what they do and to display the data
resulting from their execution. All of this is stored in a shareable document with others
users.
It is therefore particularly useful for prototyping algorithms or testing snippets of
code, in order to analyze the results and possibly add them to your main project. With
a notebook, no need to organize your code into functions, with a main, etc. You just have
need to write your code in the predefined blocks and execute it, to 'run some
Python
Jupyter Notebook for data analysis

Jupyter Notebook is a very popular application for all people working

in data analysis. It allows a team to create and share reports.
in the form of a document/code, thus facilitating productivity and collaboration.

[Link] NGUENGORO Computer Engineer 7

 DATA ANALYSIS WITH PYTHON

CHAPTER II: PYTHON REMINDER

1. Variables & Data Types

Numeric variables
Numerical variables can be broken down into two distinct types:
The integers, which correspond to the set of positive whole numbers
or negatives (1, 2, 0, 123, -3, etc.);
Decimal numbers, which, in addition to integers, include the set of
decimal numbers (2.50 ; 5.99 ; -1.20 ; 2/3 ; etc.).
Character chairs
You will now explore a bit more the strings, which allow you to store
text in your variables. First of all, a bit of semantics: we call this
strings, because Python does not consider these variables as text as
such, but like a string of characters put together. Here's how you can define
strings in Python:
ville = "Brazzaville"
film = 'intouchables'
stringVide = ""

2. Lists and Dictionaries

M. Prefils NGUENGORO Computer Engineer 8

 DATA ANALYSIS WITH PYTHON

A list is a data structure that contains a series of values. Python

authorizes the construction of lists containing values of different types (for example
integer and string), which gives them great flexibility. A list is
declared by a series of values (don't forget the quotes, single or double, if
it consists of strings separated by commas, all enclosed by
crochets. Here are some examples:

>>> animaux = ["girafe", "tigre", "singe", "souris"]

>>> tailles = [5, 2.5, 1.75, 0.15]
>>> mixte = ["girafe", 5, "souris", 0.15]
animals
['girafe', 'tigre', 'singe', 'souris']
sizes
[5, 2.5, 1.75, 0.15]
mixed
['girafe', 5, 'souris', 0.15]

Dictionaries prove to be very useful when you need to handle

complex structures to describe and the lists present their limits. The
Dictionaries are unordered collections of objects, meaning there is no
notion of order (i.e. no index). Values of a dictionary are accessed by keys.
This seems a bit confusing? Look at the following example:

>>> ani1 = {}
>>> ani1["nom"] = "girafe"
>>> ani1["taille"] = 5.0
>>> ani1["poids"] = 1100
ani1
{'nom': 'girafe', 'taille': 5.0, 'poids': 1100}

3. Conditions IF/ELSE

This concept is one of the most important in programming. The idea is to say that it
if variable a has this value then do this else do that.

Let's take an example, we will assign a value to a variable and if this value is greater than
5, so we will increment the value by 1

>>> a = 10
>>> if a > 5:
... a=a+1
...
a
11

Condition if and else

It is possible to give instructions regardless of the possible choices with the keywordelse.
>>> a = 20
>>> if a > 5:
... a=a+1
... else:
... a = a - 1

[Link] NGUENGORO Computer Engineer 9

 DATA ANALYSIS WITH PYTHON

...
a
21

Condition elif
It is possible to add as many specific conditions as desired by adding the keyword
elif, contraction of 'else' and 'if', which could be translated as 'otherwise'.
>>> a = 5
>>> if a > 5:
... a=a+1
... elif a == 5:
... a = a + 1000
... else:
... a=a-1
...
a
1005

4. FOR Loops

In programming, we often need to repeat an instruction multiple times.

Essential to any programming language, loops will help us achieve this
task compactly and efficiently.

Imagine, for example, that you want to display the elements of a list one after the other.
others. Given the current state of your knowledge, you would need to type something like:

animaux = ["girafe", "tigre", "singe", "souris"]

print(animals[0])
print(animals[1])
print(animals[2])
print(animals[3])

If your list contains only 4 items, this is still doable, but imagine if it has more.
contains 100 even 1000! To remedy this, we need to use loops. Look at the example
following:
>>> animaux = ["girafe", "tigre", "singe", "souris"]
for animal in animals:
... print(animal)
...
giraffe
tiger
monkey
mouse

Let's discuss in detail what happened in this example:

The variable animal is called an iteration variable, it successively takes on the different
values of the list animals at each iteration of the loop. We will see a little later in this
chapter where you can choose the name you want for this variable. It is created by
Python the first time the line containing the for is executed (if it already existed its

M. Prefils NGUENGORO Computer Engineer 10

 DATA ANALYSIS WITH PYTHON

content would be overwritten). Once the loop is finished, this animal iteration variable will not be
not destroyed and will thus contain the last value of the list animals (here the string of
mouse characters).

Note well the types of variables used here: animals is a list on which we iterate, and
animal is a string because each element of the list is a string.
We will see later that the iteration variable can be of any type according to the list.
traversed. In Python, a loop always iterates over an object called sequential (that is to say a
object made up of other objects) such as a list. We will also see later other objects.
sequences on which we can iterate in a loop.

Already, pay attention to the colon character ':' at the end of the line starting with
for. This means that the for loop awaits a block of instructions, namely all the
instructions that Python will repeat in each iteration of the loop. This block is called
of instructions the body of the loop. How do we indicate to Python where this block starts and
Does it end? This is indicated only by the indentation, that is to say the offset towards the
right of the (or the) line(s) of the instruction block.

5. Python Functions

During your data analysis, you will regularly have to use multiple times
groups of instructions for a very specific purpose. One of the fundamental principles for everyone
Computer programmers aim to achieve maximum results for a minimum effort.
of efforts: a saying even says that a good programmer is a lazy programmer. It is
thanks to this somewhat "lazy" but incredibly effective principle, the idea of came about
functions. Functions allow grouping several instructions into a block that
will be called by a name.
Functions are not unique to Python; they are present in all programming languages.
IT. They particularly enable:
To reuse a piece of code that has already been written via the function name – it is therefore not
necessary to rewrite the entire portion of code each time;
To lighten a code and make it more readable!

There are many built-in functions in Python! In addition to those already seen, there are
for example:
len()a function that returns the length of an element. Do you remember the
String characters? Using this function on a string allows for
example of knowing how many characters it contains;
type()allows to display the type of a variable;
pow(a, b)allows calculating a raised to the power of b. It is equivalent to the notationa**b;
abs()returns the absolute value of a number.

Here are some examples to illustrate the use of these functions:

[Link] NGUENGORO Computer Engineer 11

 DATA ANALYSIS WITH PYTHON

CHAPTER III : Numpy Library

1. Introduction to Numpy

The NumPy library([Link] allows for numerical calculations

with Python. It introduces an easier management of arrays of numbers.

To use NumPy, you must first place yourself in an environment that

understand this library, see Introduction to Python.

First, you need to import the numpy package with the following instruction:

import numpy as np

Predefined variables
Variable pi: NumPy allows you to obtain the value of pi.

[Link]
3.141592653589793

2. Arrays with Numpy

Arrays (in English, array) can be created withc [Link]().We use brackets
to delimit lists of items in tables

a = [Link]([1, 2, 3, 4])

Display:
a
[1, 2, 3, 4]
type(a)
[Link]

3. Size of a table

The funcon [Link]()(in English) returns the size of the array.

a = [Link]([2,5,6,8])
[Link](a)

[Link] NGUENGORO Computer Engineer 12

 DATA ANALYSIS WITH PYTHON

(4,)
b = [Link]([[1, 2, 3],
[4, 5, 6]]
[Link](b)
(2, 3)

4. Reading a dataset with numpy

Reading a text file

>>> b = [Link]('data_a.txt')

Reading from a binary file.

>>> c = [Link]('data_a.npy')

5. Data types with Numpy

The data type - dtype in NumPy is different from primitive data types in
Python, for example, dtype has the type with a higher resolution that is useful in computation
data.

[Link] NGUENGORO Computer Engineer 13

 DATA ANALYSIS WITH PYTHON

6. Extracting a value from a Numpy array

When working with arrays, we often need to retrieve a portion of an array.

To do this, Python allows you to extract slices from an array using a technique called
slicing (slicing, in French). It involves indicating indices in brackets to define
the start and the end of the tranche must be separated by colons:
a = [Link]([12, 25, 34, 56, 87])
a[1:3]
[25, 34]

In the range [n:m], the element at indexn is included, but not the one with indexm. A way to
memorizing this mechanism involves considering that the limits of the slice are defined by
the numbers of the positions located between the elements, as shown in the diagram below:

It is also possible not to have a beginning or an end.

a[1:]
array([25, 34, 56, 87])
a[:3]
array([12, 25, 34])
a[:]
[12, 25, 34, 56, 87]

Slicing of 2D arrays

a = [Link]([[1, 2, 3],
[4, 5, 6]])
a[0,1]
2
a[:,1:3]
array([[2, 3],
[5, 6]]
a[:,1]

[Link] NGUENGORO Computer Engineer 14

 DATA ANALYSIS WITH PYTHON

array([2, 5])
a[0,:]
[1, 2, 3]

7. Extraction of a vector of values from a Numpy array

8. Extracting an array of values from a Numpy array

An important point to consider when we manipulate arrays, and in general

general when we manipulate objects, is that when we assign an array T1 to a
another table T2, the result of this assignment is not the copy of the values stored in T1
in the table T2, but only the copy of the address where T1 is stored in the variable T2!
In other words, after assigning T1 to T2, both arrays point to the same
memory addresses and the modification of one of the two arrays affects the state of the other array.
The following program example shows what actually happens during the assignment of a
table in another.
import numpy as np

vecteur_1 = [Link]([1,2,3])
vector_2 = vector_1
vecteur_2[2] = 99
print("vecteur_1 =", vecteur_1)
print("vecteur_2 =", vecteur_2)

CHAPTER IV: Data Analysis with Numpy

1. Objectives

Data analysis requires organizing the data in a more or less structured format.
common and corresponds to the table format. Also, with the incessant growth of
data sources, the size of the files where this data is stored is becoming increasingly

[Link] NGUENGORO Computer Engineer 15

 DATA ANALYSIS WITH PYTHON

conséquente. Avec la bibliothèque Pandas, le Data Scientist peut s’affranchir de toutes ces
questions relatives au format et à la taille des données pour ne se concentrer que sur le problème
to solve.

2. Comparisons
The equality comparison operator == allows you to check if two quantities are equal or
no. The operator == returns True if the quantities are equal and false if the quantities are not.
equals. We can use the operator == with the all() function to check if all the
elements of the two tables are equal or not. The following code example shows us
how we can compare two arrays for equality with the == operator in Python.

3. Selection of elements

[Link](condlist, choicelist, default=0)

Return an array derived from the elements of the selection list, based on the conditions.

Condlist : The list of conditions that determine from which table of the list of
select the output elements are extracted. When multiple conditions are met, the
the first encountered in condlist is used.
Choice list :The list of tables from which the output elements are extracted. It
must be the same length as condlist.
Default : The item inserted into the output when all conditions are evaluated to False.

4. Comparisons with multiple conditions

The [Link]()is used to select certain elements from an array after

having applied a specified condition. Suppose we have a scenario in which
we need to specify several conditions in a single [Link]() function. We
we can use the & operator for this purpose. We can specify multiple conditions to
the interior of the [Link]() function by enclosing each condition in a pair of
parentheses and using an operator & between them.

5. Replacing values in a numpy array

If we need to replace all values above a certain threshold in an array

Numpy, we can use the [Link]() function. We can specify the limits.
upper and lower of an array using the [Link]() function. The function
[Link]() returns an array where elements lower than the specified limit are replaced.
by the lowest limit. Elements above the specified limit are replaced by the
greatest limit. The following code example shows us how to replace values at
the interior of a NumPy array by the [Link]() function.
import numpy as np

array = [Link]([1,2,3,4,5,5,6,7,8,8,9,9])

result = [Link](array, 0, 5)

[Link] NGUENGORO Computer Engineer 16

 DATA ANALYSIS WITH PYTHON

print(result)

6. Replacement of empty strings

Nous pouvons utiliser la méthode replace()data typestrto replace some

substrings in a different output.

replace() takes two parameters, the first parameter is the regex pattern with which
you want to match the strings, and the second parameter is the string
for the corresponding string replacements.

There is also a third optional parameter in replace() that accepts an integer for
set the maximum number of replacements to execute. If you set 2 as a parameter
count, the replace() function will only match and replace 2 instances in the
chain.

7. Data type conversion

After creating the data instance, you can change the type of the element to a
another type with the astype() method, such as from integer to float and so on.

>>> import numpy as np

test = [Link]([11, 12, 13, 14], dtype='int32')
>>> x = [Link]('float32')
>>> x
array([11., 12., 13., 14.], dtype=float32)
>>> test, [Link]
(array([11, 12, 13, 14]), dtype('int32'))

8. Mathematical calculations with Numpy

Trigonometric functions
[Link](x) sinus
[Link](x) cosine
[Link](x) tangent
arcsine
[Link](x) arccosine
[Link](x) arctangent
Hyperbolic functions
[Link](x) hyperbolic sine
[Link](x) hyperbolic cosine
[Link](x) hyperbolic tangent
[Link](x) hyperbolic arcsine
[Link](x) hyperbolic arccosine
[Link](x) hyperbolic arctangent

[Link] NGUENGORO Computer Engineer 17

 DATA ANALYSIS WITH PYTHON

Various functions
x**n x to the power of n, example: x**2
[Link](x) square root
[Link](x) exponential
[Link](x) natural logarithm
[Link](x) absolute value
[Link](x) sign
Useful functions for complex numbers
[Link](x) real part
[Link](x) imaginary part
[Link](x) module
[Link](x) argument in radians
[Link](x) complex conjugate

CHAPTER V: Pandas Library

1. Introduction

Pandas is an open-source library for data manipulation and analysis.

in a simple and intuitive way in Python. It was developed by Wes McKinney in 2008.
while he was working at AQR Capital Management. At the end of 2009, it was put
in open source and is now actively used in the field of theBig dataand of the
data science car this offers high performance and productivity to its users.

2. Presentation of the dataset

A Dataset is a two-dimensional data structure, meaning that the data is

aligned in a tabular fashion in rows and columns.

3. Reading a CSV file with pandas

To read the file, there is the pandas function.read_csv():

>>> import pandas as pd
>>> data = pd.read_csv('[Link]')

4. Exploration of the DataFrame

The DataFrame is a data structure that organizes data in rows and columns.
which makes it a two-dimensional data structure. You can imagine it as a

[Link] NGUENGORO Computer Engineer 18

 DATA ANALYSIS WITH PYTHON

spreadsheet or an SQL table, or even a dictionary of Series objects. It is

usually the most used pandas object. Like a series, a DataFrame can be constructed
from many different types:
Undict 1D ndarrays, lists, dicts, or Series;
A two-dimensional [Link];
Structured ndarray;
A series ;
Or another Dataframe.

5. Selection of a line

[Link] will return a single row in the form of a series when it receives a single label.
of line.
>>> [Link]['Line1']
state TX
color green
food Lamb
age 2
height 70
score 8.3
Name: Niko, dtype: object

[Link] data types

The types are not native to pandas. They are the result of a tight architectural coupling.
between pandas and numpy. the type of a column must not be related in any way to the python type of
the object contained in the column. Here we have [Link] floating points. The type will be
floatThen, we useastypefor the 'throw' to object.

7. Selection of multiple lines

To select multiple rows, place all the row labels you wish.
select from a list and transmit it [Link]'s selectLine1andLine2.

8. Selecting a column
To select a single column of data, simply put the name of the column.
between the brackets. Let's choose the food column:

>>> df[col1]
Jane Steak
Niko Lamb
Aaron Mango
Penelope Apple
Dean Cheese
Christina Melon
Cornelia Beans
Name: col1, dtype: object

[Link] of multiple columns

[Link] NGUENGORO Computer Engineer 19

 DATA ANALYSIS WITH PYTHON

It is possible to select multiple columns with just the indexing operator.

passing a list of column names. Let's selectCol1, Col2andCol3:
>>> df[['Col1', 'Col2', 'Col3']]

CHAPTER VI: Data Manipulation with pandas

1. Introduction

Pandasis the essential library for handling data. It allows for manipulation
both the data in table form that it can retrieve or export in different
formats. It also allows for easy creation of graphs.

2. Transformation of a column

The best way to convert one or more columns of a DataFrame into values.
The goal of using pandas is to_numeric(). This function will try to change objects
non-numeric (such as strings) into integers or floating-point numbers, as appropriate.

3. Mathematical operations between columns

During operations on dataframes, the names of the rows and columns are
automatically aligned:
df1 = [Link]({'A': [1, 2], 'B': [3, 4]}, index = ['a',
'c'])
df2 = [Link]({'A': [1, 2], 'C': [7, 5]}, index = ['b',
'c'])

df1 + df2give
A B C
a NaN NaN NaN
b NaN NaN NaN
c 4.0 NaN NaN

Possible operations:
df1 + df2
2 * df + 3
1 / dfelement-wise operation.
df **2square of each element.
for boolean dataframes [Link]({'A': [1, 0, 0], 'B': [0,
1, 1]}, dtype = bool):
-dfnot.
df1 & df2and.
df1 | df2or.
df1 ^ df2or exclusive.

[Link] NGUENGORO Computer Engineer 20

 DATA ANALYSIS WITH PYTHON

comparison operations:df1 is equal to df2, [Link](df2), [Link](df2), df1 is less than or equal to df2,
df1 is less than df2, df1 is less than or equal to df2equality, non-equality, <, <=, >, >=. They return dataframes.
booleans.
one can also dodf1 is equal to df2, but be careful, it also returns a dataframe of booleans.
Boolean reductions:

(df > 0).all()returns a series with one element per column that is True if
all values are > 0

(df > 0).any()returns a series with one element per column

Any operating system that supports graphical interfaces must constantly monitor
the environment to detect events such as pressing a key on the keyboard
or on a mouse button. The operating system then informs the programs about it.
Execution course. Each program then determines whether it should respond to these events.

4. Sorting a DataFrame

Sort according to row or column labels:

df.sort_index(axis = 0, ascending = False)returns a dataframe with the rows

sorted in descending order of labels (the default is ascending):
A BC D
a3 5.3 9 1.5 15
a2 2.7 10 5.4 7
a1 1.1 2 3.3 4

df.sort_index(axis = 1, ascending = False)returns a dataframe with the columns

sorted in descending order of labels:
D CB A
a1 4 3.3 2 1.1
a2 7 5.4 10 2.7
a3 15 1.5 9 5.3

Sort by values:
df.sort_values(by = 'C')returns a dataframe with the rows sorted in such a way that the
column 'C' should be in ascending order:
A BC D
a3 5.3 9 1.5 15
a1 1.1 2 3.3 4
a2 2.7 10 5.4 7

[Link] NGUENGORO Computer Engineer 21

 DATA ANALYSIS WITH PYTHON

CHAPTER VII: Series with Pandas

1. Data Structures

DataFrames
A DataFrame is a very important data structure in pandas. A DataFrame is
a two-dimensional table. One could even say that it is a collection of 'pandas Series'
».

2. Indexing with integers

Indexing of Pandas DataFrames can be done by their index position.

class [Link](data=None, dtype=None, copy=False, name=None, tupleize_cols=True,
**kwargs)

[Link]([1, 2, 3])
Int64Index([1, 2, 3], dtype='int64')

3. Reindexing a series object

df.reset_index()returns a reindexed dataframe from 0 to n - 1, but keeps one column

index with the old values of the index!

df.reset_index(drop = True)returns a reindexed dataframe from 0 to n - 1

df.reset_index(drop=True, inplace=True)modify the dataframe directly.

For a dataframe that has a column 'A':df.set_index('A')returns a dataframe where

the index is column A and column A has disappeared from the dataframe itself

4. Sorting a series object

To sort a series of pandas, the methodSeries.sort_values()is used.

Syntax: Series.sort_values (axis = 0, ascending = True, inplace = False, kind = 'quicksort',
sorted

5. Transformation of a column

The best way to convert one or more columns of a DataFrame into values
Numerical values are used to utilize pandas. to_numeric(). This function tries to convert objects.
non-numeric (such as strings) into integers or floating-point numbers, as appropriate.

[Link] NGUENGORO Computer Engineer 22

 DATA ANALYSIS WITH PYTHON

6. Data alignment

The Pandas library is useful for performing exploratory data analysis in Python.
A pandas dataframe represents data in a tabular format. We can perform
perform operations on the data and display them. In this article, we will align the columns
on the left in Pandas. When we display the dataframe, we can align the
data in the columns on the left, the right, or the center.
import pandas as pd

data = {'Name' : ['Tania', 'Ravi',

'Surbhi', 'Ganesh'],

'Articles' : [50, 30, 45, 33],

'Location' : ['Kanpur', 'Kolkata',

'Kolkata', 'Bombay']}

df = [Link](data)
display(df)

CHAPTER VIII: DataFrames with pandas

1. Index shared by all columns

[Link][(data["col1"]=="Female") & (data["col2"]=="Not Graduate") & (data["col3"

]=="Y"), ["Gender","Education","Loan_Status"]

2. Method Apply()

It is one of the main functions for playing with data and creating new variables.
returns a value after processing each row/column of a DataFrame with a
function. The function can be a default function or user-defined.
For example, here, apply can be used to find the missing values of each row and
column :

Create a new function that determines if the parameter value is missing:

def num_missing(x):
return the sum of [Link]()
We apply this function for each column:
Missing values by column:

[Link] NGUENGORO Computer Engineer 23

 DATA ANALYSIS WITH PYTHON

print([Link](num_missing, axis=0)) #axis=0 defines that the function will be applied correctly
on each column
Then apply for each line:
Missing values by row:
print([Link](num_missing, axis=1).head()) #axis=1 defines that the function will be properly
applied to each line

CHAPTER IX: Graph: the curves

1. Introduction

To plot curves, Python alone is not enough and we need libraries.

NumPy and Matplotlib used in this course. If you do not have these libraries,
you can check the pageIntroduction to Pythonto install the appropriate environment for
this course.

On this page, we present two syntaxes: the 'PyLab' syntax which is close to that
of Matlab and the 'standard' syntax that is recommended in the new versions of
matplotlib.

For the 'PyLab' syntax, simply do:

from pylab import *

It is then possible to directly access the functions of NumPy and matplotlib.

For the "standard" syntax, you need to import the numpy package and the pyplot module from
matplotlib. We must then specify the libraries when calling the functions. For in
to learn more about the concept of importation, you can consult the pageModules and
imports.

2. Creation of a curve

The instruction plot()allows to draw curves that connect points whose abscissas and
Coordinates are provided in tables.
import numpy as np
import [Link] as plt

x = [Link]([1, 3, 4, 6])
y = [Link]([2, 3, 5, 1])
[Link](x, y)

M. Prefils NGUENGORO Computer Engineer 24

 DATA ANALYSIS WITH PYTHON

[Link]()

3. Definition of the domain of the axes

It is possible to independently set the domains of the abscissas and ordinates using
the functionsxlim()andylim()
xlim(xmin, xmax)
ylim(ymin, ymax)

Example1 :
import numpy as np
import [Link] as plt

x = [Link](0, 2*[Link], 30)

y = [Link](x)
[Link](x, y)
[Link](-1, 5)

[Link]()

4. Adding a title

We can add a title using the instructionon title()

import numpy as np
import [Link] as plt

x = [Link](0, 2*[Link], 30)

y = [Link](x)
[Link](x, y)
Cosine function

[Link]()

5. Adding a caption

import numpy as np
import [Link] as plt

x = [Link](0, 2*[Link], 30)

y = [Link](x)
[Link](x, y, label="cos(x)")
[Link]()

[Link]()

6. Labels on the axes

Labels on the axes can be added with the functionsxlabel()andylabel()

[Link] NGUENGORO Computer Engineer 25

 DATA ANALYSIS WITH PYTHON

import numpy as np
import [Link] as plt

x = [Link](0, 2*[Link], 30)

y = [Link](x)
[Link](x, y)
x-axis
y-axis

[Link]()

7. Displaying multiple curves

To display multiple curves on the same graph, one can proceed as follows:
import numpy as np
import [Link] as plt

x = [Link](0, 2*[Link], 30)

y1 = [Link](x)
y2 = [Link](x)
[Link](x, y1)
[Link](x, y2)

[Link]()

CHAPTER X: Bar Charts and Cloud Charts

points

Draw a bar chart

import [Link] as plt

names = ['A', 'B', 'C'] # nom des barres
values = [1, 10, 100]
[Link](names, values) ; [Link]() # Tracer

2. Add error bars

names = ['Mammifères', 'Reptiles', 'Poissons']

values = [6, 20, 3]
ic = [0.5,0.7,0.5]
[Link](names, values, color = "#A0AAE4", edgecolor="red",
linewidth=3
yerr=ic, ecolor = "green", capsize = 10) ; [Link]()

3. Stacked bar chart

import [Link] as plt

[Link] NGUENGORO Computer Engineer 26

 DATA ANALYSIS WITH PYTHON

largeur_barre = 0.8
y1 = [2,8,9,7]
y2 = [5,9,4,2]
x = range(len(y1)) # position in the x-axis of the bars
Trace
[Link](x, y1, width = bar_width, color = "#3ED8C9")
[Link](x, y2, width = bar_width, bottom = y1, color =
#EDFF91
[Link](range(len(y1)), ['A', 'B', 'C', 'D']) ; [Link]()

4. Grouped bar chart

largeur_barre = 0.3 # Largeur de chaque barre :

attention if the value is too high, there will be no
difference between each pack of bars
y1 = [1, 2, 5]
y2 = [1, 6, 4]
x1 = range(len(y1)) # Position of the bars of category 1
x2 = [i + bar_width for i in x1] # Position of the bars
the cat 2
import [Link] as plt
[Link](x1, y1, width = bar_width, color = 'orange', #
Bars cat 1
edgecolor = 'black', linewidth = 2
[Link](x2, y2, width = bar_width, color = 'yellow', #
Bars cat 2
edgecolor = ['black' for i in y1], linewidth = 2)
[Link]([r + bar_width / 2 for r in range(len(y1))], #
Etiquettes
['Qualité', 'Environnement', 'Social'])
[Link]()

you :
Sans_engrais = [13,15,10]
Avec_engrais = [14,10,11]
import pandas as pd
mydata = [Link]({"Sans engrais":Sans_engrais,"Avec
fertilizer: With_fertilizer
[Link] = ["A","B","C"]
from pandas import plotting
This function creates
automatically a grouped bar chart from a
dataframe (rot : label orientation)
[Link]()

[Link] NGUENGORO Computer Engineer 27

 DATA ANALYSIS WITH PYTHON

5. Scatter plots: linear regression chart

import seaborn as sns

import [Link] as plt
df = sns.load_dataset('iris')
[Link](x=df["sepal_length"], y=df["sepal_width"], fit_reg=True, order=4)
[Link]()

6. Color and shape of scatter plot graphs

[Link]( x="sepal_length", y="sepal_width", data=df, fit_reg=False, hue='species',

legend=False)
[Link](loc='lower right')
[Link]()

CHAPTER XI: Histograms and Box Plots

1. Histograms

Create a histogram with Python

Example of simulated values with numpy
from [Link] import normal as norm
x = norm(50,20,57)
Histogram trace
import [Link] as plt
[Link](x, color='red', edgecolor='black',
linewidth=1.2,bins=20
[Link]()

Histogram with density curve - density plot

import seaborn as sns # Import the seaborn module

[Link](x, hist=True, kde=True,
bins=int(180/5), color = 'darkblue', # bins : thickness of
bopites
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 4}) # kde_kws = épaisseur de la courbe
[Link]()

2. Box plots

M. Prefils NGUENGORO Computer Engineer 28

 DATA ANALYSIS WITH PYTHON

In this boxplot section, we will take the example of a game of

data to simulate:

sx = ["M"]*5 + ["F"] * 5 + ["M"]

age = [50,60,62,30,40,51,18,74,68,57,35]
taille = [163,167,165,170,171,177,180,190,176,178,173]
See how to simulate values
import pandas as pd
mydata = [Link]({"age":age,"sx":sx,'taille':taille})

Simple mustache boxes

import [Link] as plt

[Link]([Link]) ; [Link]("Boxplot of ages") ;
[Link]().xaxis.set_ticklabels(['A']) ; [Link]()
This last line [Link]() allows to add some
labels on boxes

Multiple mustache boxes

import [Link] as plt

[Link](121)
[Link]([Link])
boxplot of ages
Ages in years
[Link](122)
[Link]([Link])
[Link](150, 200)
boxplot of sizes
Sizes in cm
[Link]()

BIBLIOGRAPHY :

M. Prefils NGUENGORO Computer Engineer 29

 Data Analysis with Python

Fabio Nelli Python Data Analytics Data Analysis and Science Using Pandas, matplotlib, and
the Python Programming Language
Learn data analysis with Python by Nebra Site du zero, 2012

Salesforce, D. (2016). THE ANALYSIS OF YOUR DATA.

2. [Link]. (2012). Apprenez à l ’analyse de donnée avec python. Site du

zero

M. Prefils NGUENGORO Computer Engineer 30