Module 4 & 5

➔ Data Science is a combination of multiple disciplines that uses statistics, data analysis, and

machine learning to analyze data and to extract knowledge and insights from it.

➔ Applied focus on extracting knowledge from data to inform decision making.

➔ Focuses on the skills needed to collect, manage, store, distribute, analyze, visualize, reuse

data and on data storytelling.

➔ There are many visual representations of Data Science.

➔ Data Science is about finding patterns in data, through analysis, and make future

predictions.

➔ By using Data Science, companies are able to make:

● Better decisions (should we choose A or B)

● Predictive analysis (what will happen next?)

● Pattern discoveries (find pattern, or maybe hidden information in the data)

Data Science Lifecycle
 Business Analytics Data Science Big Data

It is the statistical study of It is the study of data using Big data simply refers to extremely
business data to gain insights. statistics, algorithm and large data [Link] by its
technology. velocity, variety, and volume)

Uses Mostly structured data. Uses both structured and Data might be unstructured , semi-
unstructured data. structured or structured.

Does not involve much coding. Coding is widely used. Data comes from various sources
such as online purchase, multimedia
forms, sensors, text files and others.

Studies trends and patterns Studies almost every trend and Contains numerous trends and
specific to business. pattern. patterns.
Data Mining vs Data Science

➔ Data mining is the process of identifying patterns in large datasets.

➔ It involves methods at the intersection of database systems, statistics, and machine

learning.

➔ The overall goal of this interdisciplinary subfield of computer science and statistics is to

extract information from large datasets or libraries of data by using sophisticated

mathematical algorithms and transform them into a comprehensible structure for

further use.
➔ Data science is a field that deals with unstructured, structured data, and semi-

structured data. It involves practices like data cleansing, data preparation, data

analysis, and much more.

Data Analytics

➔ Data analytics is the science of examining raw data to reach certain conclusions.

➔ Involves applying an algorithmic or mechanical process to derive insights and running

through several data sets to look for meaningful correlations.

➔ It is used in several industries, which enables organizations and data analytics companies to

make more informed decisions, as well as verify and disprove existing theories or models.

➔ The focus of data analytics lies in inference, which is the process of deriving conclusions that

are solely based on what the researcher already knows.

Lifecycle
➔ Phase 1: Discovery –The data science team learn and investigate the problem.

● Develop context and understanding.

● Come to know about data sources needed and available for the project.

● The team formulates initial hypothesis that can be later tested with data.
Phase 2: Data Preparation-

● Steps to explore, preprocess, and condition data prior to modeling and analysis.

● It requires the presence of an analytic sandbox, the team execute, load, and transform, to get data

into the sandbox.

● Data preparation tasks are likely to be performed multiple times and not in predefined order.

Phase 3: Model Planning –

● In this phase, data science team develop data sets for training, testing, and production purposes.

● Team builds and executes models based on the work done in the model planning phase.
Phase 4: Model Building –

● Team develops datasets for testing, training, and production purposes.

● Team also considers whether its existing tools will suffice for running the models or
if they need more robust environment for executing models.
Phase 5: Communication Results –

● After executing model team need to compare outcomes of modeling to criteria

established for success and failure.
● Team considers how best to articulate findings and outcomes to various team
members and stakeholders, taking into account warning, assumptions.
● Team should identify key findings, quantify business value, and develop narrative to
summarize and convey findings to stakeholders.
Phase 6: Operationalize –

● The team communicates benefits of project more broadly and sets up pilot project

to deploy work in controlled way before broadening the work to full enterprise of

users.
Roles in Data Science Pojects
Data Scientist

➔ Data Scientists find and interpret rich data sources, merge data sources, create

visualizations, and use machine learning to build models that aid in creating actionable

insight from the data.

➔ In short, they apply the scientific discovery process, including hypothesis testing, to obtain

actionable knowledge related to a scientific or business problem.

Data Engineer

➔ Make the appropriate data accessible and available for data science efforts. They design,
develop, and code data-focused applications that capture data, as well as clean the data.
➔ This role also helps to ensure consistency of datasets (e.g., meaning of attributes across
datasets).

Data Science Architect

➔ Design and maintain the architecture of data science applications and facilities. In other words,
this role creates and manages relevant data models, data storage systems and processes
workflows. In conjunction with the Data Engineer, they manage and merge large amounts of
data and their related sources.
Data Science Developer
➔ Data Science Developers design, develop, and code large data (science) analytics applications to
support scientific or enterprise/business processes.
➔ This role enables models to be deployed (i.e., use a model in production) and requires some
expertise in data science, as well as knowledge of how to effectively develop software
applications.
➔ Sometimes this role is known as a machine learning engineer. Regardless, they help bridge the
worlds of data science and software development.
Data Science Product Owner
As explained in Nick’s post, 10 Reasons why you (probably) need a Data Science Product Manager, the
product person is the central point of product leadership – the person who decides which features and
functionality to build, the order in which to build them, and what aspects of them to observe and analyze. The
product owner is responsible for prioritizing what work gets done, ensuring that each work item is clearly
defined from a business context, and that the upcoming work and priorities of the team are visible and
transparent.

In addition, the product owner must agree that the tasks in the done column are actually done. In short, the
product owner represents all the stakeholders for the project. While a product owner is often the product
manager, it is possible to have separate these roles, in that the product managers has a more strategic focus
on the product’s vision, company objectives, and the market (as compared to product owners which are more
tactical and directly involved within the day-to-day data science team by translating a product manager’s
strategy into actionable tasks.
Data/Business Analyst
Data/Business Analysts analyze a large variety of data to extract information
about system, service, or organization performance and present them in
usable/actionable form. They better shape a problem for the data scientist to
explore. Note the difference between a data analyst and data scientist.
Subject Matter Expert
Subject matter experts are people with extensive knowledge of how to apply the analytics within a
specific organizational context. This role is accountable to ensure the desired insights are actionable.
Module 5
➔ Exploratory Data Analysis (EDA) is an approach/philosophy for data analysis

that employs a variety of techniques (mostly graphical).

➔ EDA is an iterative cycle.

1. Generate questions about your data.

2. Search for answers by visualising, transforming, and modelling your data.

3. Use what you learn to refine your questions and/or generate new questions.
➔ EDA is a state of mind.
➔ EDA is an important part of any data analysis, even if the questions are
handed to you on a platter,
➔ because you always need to investigate the quality of your data.
Typical Data Formats

➔ EDA is fundamentally a creative process. And like most creative processes, the key
to asking quality questions is to generate a large quantity of questions. It is difficult to
ask revealing questions at the start of your analysis because you do not know what
insights are contained in your dataset.
➔ To make the discussion easier, let’s define some terms:
● A variable is a quantity, quality, or property that you can measure.
● A value is the state of a variable when you measure it. The value of a
variable may change from measurement to measurement.
● An observation is a set of measurements made under similar conditions
(you usually make all of the measurements in an observation at the same
time and on the same object). An observation will contain several values,
each associated with a different variable. I’ll sometimes refer to an
observation as a data point.
 ➔ Tabular data is a set of values, each associated with a variable and an observation. Tabular data
is tidy if each value is placed in its own “cell”, each variable in its own column, and each
observation in its own row.
➔ Variation
Variation is the tendency of the values of a variable to change from measurement to
measurement.
You can see variation easily in real life; if you measure any continuous variable twice, you will get
two different results. This is true even if you measure quantities that are constant, like the speed
of light.
➔ Covariation
If variation describes the behavior within a variable, covariation describes the behavior between
variables. Covariation is the tendency for the values of two or more variables to vary together in a
related way. The best way to spot covariation is to visualise the relationship between two or more
variables.