STATISTICS
FOR
DATA SCIENCE
Courtesy:
[Link]
scientists-and-data-analysts-69d93a05aae7
Anjali Jivani
WHAT IS STATISTICS?
“Statistics is the grammar of science.”
As stated by Karl Pearson, a British mathematician
“Statistics is a group of methods used to collect, analyze,
present, and interpret data to make decisions.”
“Statistics is a branch of applied mathematics that involves
the collection, description, analysis, and inference of
conclusions from quantitative data.”
2
WHAT IS STATISTICS?
After so many years of statistics it is now redefined by Sheldon Ross
as –
“The art of learning from data. It is concerned with the
collection, subsequent description, their analysis and
drawing conclusions from data.”
The importance of statistics in data science and data analytics
cannot be underestimated.
Statistics provides tools and methods to find structure and to give
deeper data insights.
Both Statistics and Mathematics love facts and hate guesses.
Statistics will allow you to think critically, and be creative when
using the data to solve business problems and make data-driven
decisions.
3
WHAT IS DATA SCIENCE?
4
WHAT IS DATA SCIENCE?
5
WHAT IS DATA SCIENCE?
“Data Science is the study of data to extract meaningful
insights for business.”
It is a multidisciplinary approach that combines principles and
practices from the fields of mathematics, statistics, artificial
intelligence, and computer engineering to analyze large amounts
of data.
While Data Science focuses on extracting insights from data,
Artificial Intelligence focuses on building intelligent systems that
can perform tasks that normally require human intelligence.
For example, finance companies can use a customer's banking
and bill-paying history to assess creditworthiness and loan risk.
6
MAJOR BRANCHES OF
STATISTICS
Descriptive Statistics
The part of statistics concerned with the description and
summarization of data.
It consists of methods for organizing, displaying, and describing
data by using tables, graphs, and summary measures.
Inferential Statistics
The part of statistics concerned with drawing of conclusions
from data.
It consists of methods that use sample results to help make
decisions or predictions about a population.
When you infer from data there is a possibility of chance i.e.
‘probability’.
7
Inferential vs. Descriptive
8
9
Sr. No. DATA SCIENCE DATA ANALYTICS
1. Mostly deals with structured Mostly deals with structured data
and unstructured data
2. Statistical skills are necessary Statistical skills are minimal
3. Data Science makes use of Data Analytics does not use DM
Data Mining and ML or ML to get the insight of data.
algorithms to get insights.
4. The scope of data science is The Scope of data analysis is
large. micro i.e., small.
5. Data science deals with Data Analytics makes use of
explorations and new existing resources.
innovations.
6. In-depth knowledge of Basic Programming skills is
programming is required for necessary for data analytics.
data science.
7. Ask the right questions Find actionable data
8. Uses Big Data Uses Big Data
10
DATA SCIENCE vs DATA ANALYTICS
Data Science and (Big) Data Analytics are unique fields, with the major
difference being the scope. Data Science is an umbrella term for a group
of fields that are used to mine large datasets.
Data analytics software is a more focused version of this and can even
be considered part of the larger process. Analytics is devoted to realizing
actionable insights that can be applied immediately based on existing
queries.
Another significant difference between the two fields is a question
of exploration. Data science isn’t concerned with answering specific
queries, instead parsing through massive datasets in sometimes
unstructured ways to expose insights.
Data analysis works better when it is focused, having questions in mind
that need answers based on existing data.
Data science produces broader insights that concentrate on which
questions should be asked, while big data analytics emphasizes
discovering answers to questions being asked.
More importantly, data science is more concerned about asking
questions than finding specific answers.
11
TYPES OF DATA ANALYTICS
1. Descriptive Analytics tells you what happened in
the past.
2. Diagnostic Analytics helps you understand why
something happened in the past.
3. Predictive Analytics predicts what is most likely to
happen in the future.
4. Prescriptive Analytics recommends actions you
can take to affect those outcomes.
12
TYPES OF DATA ANALYTICS
E.g.
Descriptive – how many centuries has Virat Kohli
scored?
Diagnostic – What was the reason behind such good
performance? His fitness, training schedule, diet,
...etc.
Predictive – trying to guess (predict) how many
centuries will Virat Kohli score in the World Cup.
Prescriptive – how do we get Virat Kohli to score more
centuries in the world cup?
13
DESCRIPTIVE ANALYTICS
Descriptive analytics looks at data statistically to tell you what
happened in the past. Descriptive analytics helps a business
understand how it is performing by providing context to help
stakeholders interpret information. This can be in the form of data
visualizations like graphs, charts, reports, and dashboards.
How can descriptive analytics help in the real world? In a healthcare
setting, for instance, say that an unusually high number of people
are admitted to the emergency room in a short period of time.
Descriptive analytics tells you that this is happening and provides
real-time data with all the corresponding statistics (date of
occurrence, volume, patient details, etc.).
Descriptive Analytics, which use data aggregation and data mining
to provide insight into the past and answer: “What has happened?”
14
DIAGNOSTIC ANALYTICS
Diagnostic analytics takes descriptive data a step further and
provides deeper analysis to answer the question: Why did this
happen? Often, diagnostic analysis is referred to as root cause
analysis. This includes using processes such as data discovery, data
mining, and drill down and drill through.
In the healthcare example mentioned earlier, diagnostic analytics
would explore the data and make correlations. For instance, it may
help you determine that all of the patients’ symptoms—high fever,
dry cough, and fatigue—point to the same infectious agent. You now
have an explanation for the sudden spike in volume at the ER.
15
PREDICTIVE ANALYTICS
Predictive analytics takes historical data and feeds it into a machine
learning model that considers key trends and patterns. The model is then
applied to current data to predict what will happen next.
Back in our hospital example, predictive analytics may forecast a surge in
patients admitted to the ER in the next several weeks. Based on patterns in
the data, the illness is spreading at a rapid rate.
Predictive Analytics: Understanding the future
Predictive analytics has its roots in the ability to “predict” what might
happen. These analytics are about understanding the future. Predictive
analytics provides companies with actionable insights based on data.
Predictive analytics provides estimates about the likelihood of a future
outcome. It is important to remember that no statistical algorithm can
“predict” the future with 100% certainty. Companies use these statistics to
forecast what might happen in the future. This is because the foundation of
predictive analytics is based on probabilities.
16
PRESCRITPIVE ANALYTICS
Prescriptive analytics takes predictive data to the next level. Now
that you have an idea of what will likely happen in the future, what
should you do? It suggests various courses of action and outlines
what the potential implications would be for each.
Back to our hospital example: now that you know the illness is
spreading, the prescriptive analytics tool may suggest that you
increase the number of staff on hand to adequately treat the influx
of patients.
In summary: Both descriptive analytics and diagnostic analytics
look to the past to explain what happened and why it happened.
Predictive analytics and prescriptive analytics use historical data to
forecast what will happen in the future and what actions you can
take to affect those outcomes. Forward-thinking organizations use a
variety of analytics together to make smart decisions that help your
business—or in the case of our hospital example, save lives.
17
STATISTICS VS. ML
STATISTICS MACHINE LEARNING
Problem → Data Data → Solution
In Stats you have a problem and you get the data to solve that
problem
In ML we have data and we want to know what the data can tell
us
When asked to a Statistician applying for a Data Scientist’s job,
Interviewer asked ‘Here is the data – what can you tell me
about it?’
The Statistician said, ‘What do you want to know?’
Interviewer ‘I want to hire you to know about the data.’
Statistician ’How can I tell you anything unless you tell me what
you want to know’
This keeps on repeating!!
18
The reason being both these areas handle data differently
STATISTICS VS. ML
STATISTICS MACHINE LEARNING
Problem → Data Data → Solution
In Stats, the question is cheap and the data is
expensive! (you are paid to collect the data) – Why is
my product not selling?
In ML, data is cheap and question is expensive (you
are paid to ask the right question) – What does my
data tell me?
19
DATA SCIENCE vs ML
20
DATA SCIENCE vs ML
Besides the obvious career as a data scientist, there are plenty of
other data science jobs to choose from.
Data scientist: Uses data to understand and explain the phenomena
around them, to help organizations make better decisions.
Data analyst: Gathers, cleans, and studies data sets to help solve
business problems.
Data engineer: Build systems that collect, manage, and transform
raw data into information for business analysts and data scientists.
Data architect: Reviews and analyzes an organization’s data
infrastructure to plan databases and implement solutions to store
and manage data.
Business intelligence analyst: Gathers, cleans, and analyzes sales
and customer data, interprets it, and shares findings with business
teams.
21