DATA SCIENCE

 According to 2020 research, it is measured that 1.7MB (Megabyte) of

data is getting generated every sec, per person. If we consider the same,
then 700-800GB (gigabyte) of data is getting generated per year for each
person.
 Because of the above reason the complex data is getting generated. To
work with respect to the complex data, data science is essential.
BASIC DEFINITION OF DATA SCIENCE:
Data science is a process, where we train the model so that model can
predict the future.

PROCESS INVOLVED IN DATA

SCIENCE
DATA DATA FEATURE
EXTRACTION CLEANING ENGINEERING

DATA ANALYSIS BUSINESS MACHINE

AND INTELLIGENCE LEARNING
VISUALIZATION

TESTING
DEPLOYMENT

1
TERMINOLOGIES:
DATA EXTRACTION:
It is a process where we collect a data from the respective source.[depending on
the project]
 Where the data can be structured (proper order) and unstructured
(randomly).
 Tools used are SQL, Excel and PowerBI.
DATA CLEANING:
In this step we will be working on the missing values, removing unwanted
values and collecting the required values.
 Tools used are Pandas, Numpy and PowerBI.
FEATURE ENGINEERING:
It is a process where we will create a data from an existing data.
 Tool used is Pandas.
DATA ANALYSIS AND VISUALIZATION:
In this step, we will be learning and understanding the data completely with the
help of graphical representation. [Visualization is the process of converting raw
data into graphical or pictorial form.]
 Tools used are matplotlib, seaborn, PowerBI.
BUSINESS INTELLIGENCE:
This means, having the knowledge of the particular domain.
MACHINE LEARNING:
This process is considered as a heart of data science, where we make machine to
understand the data so that it can predict the future.
 This process involves deep learning, neural network, natural language
processing and soon.
 Tool used is SK-learn/Scikit learn

2
TESTING:
It is very important step where we will come to know whether machine got
trained properly or not.
DEPLOYMENT:
Submitting the final product to the client or the users.

DEFINITION OF DATA SCIENCE:

Data science is a multidisciplinary field where we use algorithms and scientific
methods to train the model, so that the machine can predict the future.

ROLES AND RESPONSIBILITIES OF DATA SCIENCE

DATA ENGINEER:
 No prior experience is required.
 Should be good at SQL ,basics of Excel, PowerBI
 The work of data engineer is data extraction and data cleaning
 Programming knowledge is not required.
DATA ANALYST:
 No prior experience is required
 Should be good at python programming, python libraries, like pandas,
numpy, matplotlib and soon
 The work of data analyst is data cleaning, feature engineering and data
analysis and visualization.
DATA SCIENTIST:
 Prior experience is required in IT domain
 Should be very good at python programming and all the libraries which
we use for data science
 The work of data scientist is to apply business intelligence, machine
learning, testing and deployment.

3
NOTE:
Along with above skills, good communication is required.

SOFTWARE USED:
Google Colab, short for Colaboratory, is a free cloud-based platform provided
by Google that allows users to write and execute Python code collaboratively in
a Jupyter Notebook environment. Google Colaboratory notebook, is designed to
facilitate machine learning (ML) and data science tasks by providing a virtual
environment.
 Official website of google colab is [Link]

 As soon as we click on the above link, this page will be displayed

 To create new file, click on file then new notebook

4
 To change file name, double click on it then enter a new name

 Text cell is used to write description about the program, to get that click
on +text
5
 To write program code cell can be used, to get that click on +code

 After writing the program, click on run button so that it gets executed.
 Click on runtimerun all to execute every cell at a time.

 WHY GOOGLE COLAB:

 Normal software may hang, due to complex data or it may take more time
to give the output and it also occupy more storage.
 To overcome the issue of storage, we use google colab as google colab
occupies dynamic memory in the cloud.
 It is also very easy to share the file.

6
 NUMPY
 NumPy is a package, which is used for array operation. NumPy is a
powerful Python library for numerical computing. It provides support for
arrays, matrices, and a wide range of mathematical functions to operate
on these data structures efficiently. NumPy is widely used in scientific
computing, data analysis, machine learning, and other areas where
numerical computations are prevalent. It's often used alongside other
libraries like SciPy, Matplotlib, and pandas to build comprehensive data
analysis and visualization pipelines
DIFFERENCE BETWEEN MODULE AND PACKAGE

MODULE PACKAGE

 It is a python file, where we  It is a collection of module or

can store variable, function, files.
class and soon.
 To access the functionality,  To access the functionality of
we need to use package
Syntax:  Installation is required.
import module_name  pip install
package_name
[for all the software
which supports python]
 conda install
package_name
[for anaconda software]
 We should import the
package in software or the
file
Syntax:
import package_name

HOW TO INSTALL PACKAGE

 Search for the software you are using for python
 E.g.: IDLE, Visual studio code, Pycharm
 Go to the open file location.

7
 Select the location in the location bar and then type cmd.
 As soon as you type cmd, command prompt will popup.
 There we will notice the location like this
C:\Users\Admin\AppData\Roaming\Microsoft\Windows\Start
Menu\Programs\Python 3.9>
 After the location ,type pip install package_name
 C:\Users\Admin\AppData\Roaming\Microsoft\Windows\Start
Menu\Programs\Python 3.9>pip install package_name
 Now open the software and then type import package_name
NOTE:
 In google colab, Packages will be inbuilt and we just need to import
package, so that we can access the functionality of it.
 To access the functionality of numpy, we have to import it first
Syntax:
import numpy

 For easy access we can give the alias name for numpy as np
import numpy as np

 To check the version of numpy, we can use

 np.__version__ {np is alias name of numpy, within double
underscore, we should write version.}

8
 DIFFERENCE BETWEEN LIST AND ARRAY

LIST ARRAY

 It can store both homogeneous  It stores only homogeneous

and heterogeneous values. values.
 It is very complex to use.  It is very easy to use as
compared to list.
 Only limited operations can be  All the possible operations can
done using operators and inbuilt be done.
functions.

 Memory usage is more.  Memory usage is less.

METHODS USED IN NUMPY

1. array() :
 It is an inbuilt method, which is used to convert list into an array.
 If variable consist of heterogeneous values, then array will convert every
value into the same datatype.
Syntax:
[Link](var/value)
 Using array method it is possible to create both unidimensional and
multidimensional arrays.
 While writing the program most of the time, we will be using
unidimensional and two dimensional arrays.

9
10
2. ndim : [n dimension ]
 It is an inbuilt variable, which is used to get the dimension of the
mentioned array.
 The output will be in the form of integer.
Syntax:
array_var.ndim

3. shape:
 It is an inbuilt variable, which is used to et the number of rows and
columns from an array.
Syntax:
array_var.shape

4. reshape() :
 It is an inbuilt method, which is used to change the number of rows and
columns of an existing array.

11
  In this case,the product of number of rows and columns should be equal
to the number of elements present in an existing array.
Syntax:
array_var.reshape(rows,columns)

NOTE:
 If we want flatten or convert multi-dimensional array to uni-dimensional
then we will use
Syntax:
Array_var.reshape(-1)

 To convert uni dimension array to n dimension array, we will make use of

ndmin.

12
 5. arange() :
 It is an inbuilt method, which is used to create the sequence of integers
between the specified limits.
Syntax:
[Link](starting value, ending value, step size)
 Here by default starting value is “0” and step size is ”1”

6. linspace() :
 It is an inbuilt method, which is used to create the equally spaced values
between the given limits.
Syntax:
[Link](starting value, ending value, no._of_values)

7. randint() :
 This function is used to generate, random integer number between the
given limits, where it includes both starting and ending values.
Syntax:
[Link](starting value, ending value, n)
 Here mentioning the values of n is optional, if we don’t mention then
function returns single output. If we mention n value, then those many
values will get generated.
13
 Here randint function is present inside random module, so access the
functionality of randint we have to call random.

8. rand() :
 This function is used to generate a random number between 0 to 1.
Syntax:
[Link](n)

 Here mentioning the values of n is optional, if we don’t mention then

function returns single output. If we mention n value, then those many
values will get generated.
9. randn():
 This function is used to generate a random number between -1 to 1.
Syntax:
[Link](n)

10. copy():
 It is an inbuilt method which is used to copy the entire content of one
array to another variable , so that the modification with respect to source
variable will not affect destination variable.
Syntax:
Dest_var=array_var.copy()

14
 11. view():
 It is an inbuilt method which is used to copy an array to another variable,
but in this case modification with respect to array variable will affect
destination variable.
Syntax:
Dest_var=array_var.view()

INDEXING AND SLICING ON ARRAY

1. UNIDIMENSIONAL ARRAY:
 To access an elements using indexing ,the following syntax is used
Syntax:
array_var[Index]
 To access group of values sequentially from an array , the following
syntax is used.
Syntax:
array_var[start_index:end_index,updation]

2. MULTIDIMENSIONAL ARRAY:

15
 To access an element from 2 dimensional array,we need to use
Syntax:
array_var[row][column]
 To extract group of values from multiple rows of 2 dimensional array , the
following syntax is used
Syntax:
array_var[row_slice][column_slice]

OPERATIONS ON ARRAY

 It is possible to perform multiple operations on array has compared to list.

 Operations on array can be divided into 3 types
 Array to Array operation
 Array to Constant operation
 Inbuilt functions on Array

Array to Array operations

 We known that, in python many of the operators will not support 2 list
collections. But all those operations can be done on 2 or more arrays.
Syntax:
Array1 operator array2 operator……
 Where, length of the arrays should be same.

16
 Array with constant
 It is also possible to perform operations on array with constant value or
scalar value.
 If array consist of single value datatype then constant can be any single
value datatype
Syntax:
array operator constant

in-built functions on array

 using inbuilt functions, it is possible to apply the same operation on each
and every value present in an array.
Syntax:
np.function_name(array)

17
18