Dr. B.
R Ambedkar National
Institute of
Technology ,Jalandhar
Internship ( 2 Months)
26-05-2022 to 23-07-2022
Name : Mayank Pathak
Roll No : 19113054
Project Topic : Data Analysis with
Python
Instructor : K.S.
Page | 1
Rana
Acknowledgement
Page | 2
Firstly , I would like to thank Rail Coach Factory for giving me
such a great opportunity to do my internship project in their
esteemed organization at its Technical Training Centre,
Kapurthala.
I would like to express my sincere gratitude and indebtedness
to my mentor, Mr. K.S Rana for his invaluable guidance,
encouragement and comforting behaviour, which helped me
complete my internship successfully.
I couldn’t forget the books and internet which provided me with
substantive matter.
Finally , I would like to thank my family and friends for all the
support and encouragement . I would also like to thank my
fellow students for many helpful discussions and good ideas
along the way.
Page | 3
Contents
Data Analysis Using Python
……………………….. 3
Python And Libraries
……………………….. 7
NumPy …………………… 7
Pandas …………………… 10
Matplotlib …………………… 11
Seaborn …………………… 12
Weather Dataset Analysis
………………………..13
Covid-19 Dataset Analysis
………………………..19
IPL Dataset Analysis
…………………………22
Netflix Dataset Analysis
…………………………28
Census Dataset Analysis
…………………………37
Important Visualisation Tools
…………………………42
Page | 4
DATA ANALYSIS
Data analysis is a process of inspecting, cleansing,
transforming, and modelling data with the goal of
discovering useful information ,informing conclusions,
and supporting decision-making.
Data analysis has multiple facets and
approaches ,encompassing diverse techniques under a
variety of names, and is used in different business,
science, and social science domains.
Page | 5
Data requirements
The data is necessary as inputs to the analysis, which is
specified based upon the requirements of those
directing the analysis (or customers, who will use the
finished product of the analysis
Data collection
Data is collected from a variety of sources. The
requirements may be communicated by analysts to
custodians of the data; such as, Information Technology
personnel within an organization. The data may also be
collected from sensors in the environment, including
traffic cameras, satellites, recording devices, etc. It
may also be obtained through interviews, downloads
from online sources, or reading documentation.
Page | 6
Data processing
The phases of the intelligence cycle used to convert
raw information into actionable intelligence or
knowledge are conceptually similar to the phases in
data analysis.
Data cleaning
Once processed and organized, the data may be
incomplete, contain duplicates, or contain errors. The
need for data cleaning will arise from problems in the
way that the datum are entered and stored. Data
cleaning is the process of preventing and correcting
these errors. Common tasks include record matching,
identifying inaccuracy of data, overall quality of
existing data, deduplication, and column segmentation.
Exploratory data analysis
Once the datasets are cleaned, they can then be
analysed. Analysts may apply a variety of techniques,
referred to as exploratory data analysis, to begin
understanding the messages contained within the
obtained data. The process of data exploration may
result in additional data cleaning or additional requests
for data; thus, the initialization of the iterative phases
mentioned in the lead paragraph of this section.
Descriptive statistics, such as, the average or median,
can be generated to aid in understanding the data.
Data visualization is also a technique used, in which the
analyst is able to examine the data in a graphical
format in order to obtain additional insights, regarding
the messages within the data.
Modelling and algorithms
Page | 7
Mathematical formulas or models (known as
algorithms), may be applied to the data in order to
identify relationships among the variables; for example,
using correlation or causation. In general terms, models
may be developed to evaluate a specific variable based
on other variable(s) contained within the dataset, with
some residual error depending on the implemented
model's accuracy
Python And Libraries
Python is a high-level, interpreted, general-purpose
programming language. Its design philosophy
emphasizes code readability with the use of significant
Page | 8
indentation. Python is dynamically-typed and garbage-
collected.
NumPy Library
NumPy, which stands for Numerical Python, is a library
consisting of multidimensional array objects and a
collection of routines for processing those arrays. Using
NumPy, mathematical and logical operations on arrays
can be performed.
Operations using NumPy
Using NumPy, a developer can perform the following
operations −
Mathematical and logical operations on arrays.
Fourier transforms and routines for shape
manipulation.
Operations related to linear algebra. NumPy has in-
built functions for linear algebra and random
number generation.
1. Converting a list to n-dimensional NumPy array
numpy_array = [Link](list_to_convert)
2. Use of [Link] and [Link]
[Link] is used to create new dimensions of
size 1.
For Eg:
a_numpy = [Link](a)
row_vector = a_numpy[:,[Link]]
col_vector = a_numpy[[Link],:]
Similarly, [Link] can be used to reshape any
array. For eg:
Page | 9
a = range(0,15)
b = [Link](3,5)
3. Converting any data type to NumPy array
Use [Link]. For eg
b = [Link](a)
4. Get an n-dimensional array of zeros.
a = [Link](shape,dtype=type_of_zeros)
type of zeros can be int or float as it is required
eg.
a = [Link]((3,4), dtype = np.float16)
5. Get an n-dimensional array of ones.
Similar to [Link]:
a = [Link]((3,4), dtype=np.int32)
6. [Link] and [Link]
[Link](shape_as_tuple,value_to_fill,dtype=type_y
ou_want)
a = [Link]((2,3),1,dtype=np.float16)
a would be:
array([[1., 1., 1.],
[1., 1., 1.]], dtype=float16)
2. [Link](shape_as_tuple,dtype=int)
a = [Link]((2,2),dtype=np.int16)
a would be:
array([[25824, 25701],
[ 2606, 8224]], dtype=int16)
Page | 10
7. Getting an array of evenly spaced values with
[Link] and [Link]
linspace:
[Link](start,stop,num=50,endpoint=bool_valu
e,retstep=bool_value)
[Link](1,2,num=5,endpoint=False,retstep=Tr
ue)
[Link](start=where_to_start,stop=where_to_st
op,step=step_size)
8. Finding the shape of the NumPy array
[Link]
9. Knowing the dimensions of the NumPy array
x = [Link]([1,2,3])
10. Finding the number of elements in the NumPy
array
x = [Link]((3,2,4),dtype=np.int16)
11. Get the memory space occupied by an n-
dimensional array
[Link]
12. Finding the data type of elements in the
NumPy array
x = [Link]((2,3), dtype=np.int16)
[Link] will produce
dtype('int16')
13. How to create a copy of NumPy array
Page | 11
Use [Link]
y = [Link]([[1,3],[5,6]])
x = [Link](y)
14. Get transpose of an n-d array
Use array_name.T
15. Multiply two NumPy matrices
Use [Link] to take matrix product of 2-D
matrices
16. Dot product of two arrays
[Link](matrix1, matrix2)
a = [Link]([[1,2,3],[4,8,16]])
17. Get cross-product of two numpy vectors
z = [Link](x, y)
Pandas Library
Pandas is a software library written for the Python
programming language for data manipulation and
analysis. In particular, it offers data structures and
operations for manipulating numerical tables and time
series.
Library features:
Page | 12
Data Frame object for data manipulation with
integrated indexing.
Tools for reading and writing data between in-
memory data structures and different file formats.
Data alignment and integrated handling of missing
data.
Reshaping and pivoting of data sets.
Label-based slicing, fancy indexing, and sub
setting of large data sets.
Data structure column insertion and deletion.
Group by engine allowing split-apply-combine
operations on data sets.
Data set merging and joining.
Hierarchical axis indexing to work with high-
dimensional data in a lower-dimensional data
structure.
Time series-functionality: Date range generation
[6] and frequency conversions, moving window
statistics, moving window linear regressions, date
shifting and lagging.
Provides data filtration.
Matplotlib: Visualization With Python
Matplotlib is a comprehensive library for creating
static, animated, and interactive visualizations in
Python. Matplotlib makes easy things easy and
hard things possible.
Create publication quality plots.
Make interactive figures that can zoom, pan,
update.
Customize visual style and layout.
Export to many file formats .
Page | 13
Embed in JupyterLab and Graphical User
Interfaces.
Use a rich array of third-party packages built
on Matplotlib.
Seaborn Library
Seaborn is a Python data visualization library based on
matplotlib. It provides a high-level interface for drawing
attractive and informative statistical graphics.
It provides beautiful default styles and color
palettes to make statistical plots more attractive. It is built on
the top of matplotlib library and also closely integrated to the
data structures from pandas.
Seaborn aims to make visualization the central
part of exploring and understanding data. It provides dataset-
oriented APIs, so that we can switch between different visual
representations for same variables for better understanding of
dataset.
Different categories of plot In Seaborn
Relational plots: This plot is used to understand the
relation between two variables.
Categorical plots: This plot deals with categorical
variables and how they can be visualized.
Distribution plots: This plot is used for examining
univariate and bivariate distributions
Regression plots: The regression plots in seaborn are
primarily intended to add a visual guide that helps to
Page | 14
emphasize patterns in a dataset during exploratory
data analyses.
Matrix plots: A matrix plot is an array of scatterplots.
Multi-plot grids: It is an useful approach is to draw
multiple instances of the same plot on different subsets
of the dataset.
Weather Dataset Analysis
Weather data set is a time series data set with per hour
information about the weather conditions of a particular
location. It records temperature ,dew point
temperature ,Relative Humidity ,Visibility, windspeed ,pressure
Page | 15
and conditions. The data is available as a CSV file. We are going
to analyse this data using pandas data frame.
Page | 16
Page | 17
Page | 18
Page | 19
Page | 20
COVID-19 Dataset Analysis With Python
This data is available as a CSV file, downloaded from Kaggle.
We will analyse this data using pandas data frame .
Page | 21
Page | 22
Page | 23
IPL 2008-2020 Dataset Analysis With
Python
Data is taken from kaggle and contains ball-by-ball information
from IPL 2008 to IPL 2022 .We are going to analyse this data
using pandas data frame.
Page | 24
Page | 25
Page | 26
Page | 27
Page | 28
Page | 29
Page | 30
Netflix Dataset Analysis With Python
This Netflix Dataset has information about the TV shows and
movies available on Netflix till 2021. This Dataset is available
on kaggle website for free.
Page | 31
Page | 32
Page | 33
Page | 34
Page | 35
Page | 36
Page | 37
Page | 38
Page | 39
Census Dataset Analysis 2011 With
Python
The data used here is of 2011 India Census of each district. This
data is available as a CSV file, downloaded from kaggle.
Page | 40
Page | 41
Page | 42
Page | 43
Page | 44
Data Visualisation Techniques
Page | 45
Page | 46
Page | 47
Page | 48
Page | 49
Page | 50
Page | 51