Data analytics is the process of collecting, cleaning, transforming, and interpreting

data to discover useful information, find patterns, and support informed decision-
making. It involves using tools and techniques to analyze raw data, which can be
used to make predictions, optimize processes, and drive business growth.

What it involves
 Data collection: Gathering raw data from various sources.
 Data cleaning and transformation: Preparing and organizing the data for
analysis.
 Data analysis: Applying statistical analysis, machine learning, and other
techniques to find patterns and insights.
 Data visualization: Presenting findings in easily understandable formats like
charts and graphs.
 Interpretation and communication: Explaining the meaning of the data to
stakeholders and making recommendations.

Types of data analytics

 Descriptive analytics: Explains what happened in the past.
 Diagnostic analytics: Explains why something happened.
 Predictive analytics: Forecasts what is likely to happen in the future.
 Prescriptive analytics: Recommends how to act to achieve a desired outcome.

Descriptive statistics
Descriptive statistics summarize and organize data by describing its basic features,
providing a snapshot of a dataset's characteristics. Key measures include central
tendency (mean, median, mode), which identifies the typical value, and measures
of dispersion (range, variance, standard deviation), which describe the data's
spread. It is a crucial first step in data analysis for understanding data and can be
presented using tables and graphs like histograms and bar charts.
 Measures of central tendency
 Mean: The average of all values, calculated by summing all values and dividing
by the count.
 Median: The middle value in a dataset that has been ordered from lowest to
highest.
 Mode: The value that appears most frequently in the dataset.

Measures of dispersion (variability)

 Range: The difference between the highest and lowest values in the dataset.
 Variance: A measure of how much the data points deviate from the mean.
 Standard Deviation: The average distance of each data point from the mean.
Frequency distributions
 Frequency distribution: A table or graph that shows how often each value occurs
in a dataset.
 Histograms and bar charts: Visual representations of frequency distributions that
help show the shape, or distribution, of the data.

Why use descriptive statistics

 To get a quick understanding of a dataset's characteristics.
 To identify potential issues like outliers or missing data.
 To provide a foundation for inferential statistics.
 To communicate data insights in a meaningful and easy-to-understand way through
summaries, tables, and graphs.

Standard deviation in data science is a crucial statistical measure that quantifies the
amount of variation or dispersion in a dataset. It shows how much individual data
points differ from the mean (average) of the set; a low standard deviation indicates
that data points are clustered tightly around the mean, while a high standard
deviation means they are more spread out over a wider range. It is used to
understand data variability, identify outliers, compare datasets, and assess the
reliability of a mean.
Key concepts

 Measure of spread: Standard deviation is a key metric for understanding

the spread of data. It is often represented by the Greek letter sigma

 Low vs. High standard deviation:

o Low

Data points are close to the average, indicating a consistent, less

variable set.

o High

Data points are spread out over a wider range, indicating greater
variability.

 Original units: A major advantage is that it returns to the original units of

measurement, making it easier to interpret compared to variance (

Uses in data science

 Data analysis: It provides a quick way to understand the consistency of a

dataset.
 Outlier detection: By showing how far a value is from the mean, it can be
used to identify outliers.
 Comparison: It is used to compare the dispersion of two or more datasets,
especially when their units and means are the same.
 Forecasting: In business, it can be used in sales forecasting to understand
the variability of sales data and predict future trends.
 Reliability: A distribution with a smaller standard deviation is considered to
have a more reliable mean.

 To calculate it manually:
1. Find the mean (average) of the data points.
2. For each data point, calculate the squared difference between the point
and the mean.
3. Find the average of these squared differences (this is the variance).
4. Take the square root of the result to get the standard deviation.
 Skewness and kurtosis are fundamental statistical measures in data science that
describe the shape of a data distribution. Skewness quantifies the asymmetry of a
distribution, while kurtosis measures its tailedness or peakedness. Understanding
both is crucial for tasks like feature engineering, model building, and identifying
potential outliers and anomalies.

Skewness
 Definition: Measures the lack of symmetry in a dataset's distribution.
 Positive Skewness (Right Skew): The right tail is longer or fatter than the
left. The bulk of the data is on the left, and the mean is greater than the median,
which is greater than the mode (Mean > Median > Mode).
 Negative Skewness (Left Skew): The left tail is longer or fatter than the right. The
bulk of the data is on the right, and the mean is less than the median, which is less
than the mode (Mean < Median < Mode).
 Zero Skewness: The distribution is perfectly symmetrical, like a normal
distribution.
 Data Science Application:
o Helps understand how data is spread out around the mean.
o Identifies potential issues like a disproportionate number of high or low values in
the data.
o Can be used to decide on transformations, such as a logarithmic transform, to
normalize a skewed variable for better model performance, as shown in this
Medium article.

Kurtosis
 Definition:
Measures the "tailedness" of the distribution, indicating the presence of extreme
values (outliers).
 Three main types:
 Leptokurtic: A distribution with a sharper peak and heavier tails than a normal
distribution. It has more extreme values, leading to higher kurtosis values.
 Platykurtic: A distribution with a flatter peak and lighter tails than a normal
distribution.
 Mesokurtic: A normal distribution, which has a kurtosis value of 3 (when using
Fisher's definition, where the normal distribution's kurtosis is zero, it's called
"excess kurtosis").
 Data Science Application:
 Helps identify the likelihood of extreme outliers.
 A high kurtosis value can indicate a higher risk in financial data due to a greater
probability of extreme price changes.
 Is important for statistical modeling assumptions, as many models assume
normality, which kurtosis can help assess.
How they work together
 Both skewness and kurtosis provide insights into data distribution that go beyond
basic measures like mean and variance.
 Together, they offer a more complete picture of the data's shape, helping analysts
detect anomalies, understand the potential for outliers, and determine if the data is
suitable for certain statistical methods or models.

BOX PLOT

A box plot, or box-and-whisker plot, is a data science tool that visualizes a

dataset's distribution using a five-number summary: minimum, first quartile (Q1),
median (Q2), third quartile (Q3), and maximum. It's used to quickly understand
data spread, central tendency, and identify potential outliers, which are data points
falling outside the whiskers (usually at the interquartile range). Box plots are
particularly useful for comparing the distributions of multiple groups
simultaneously.

Key components of a box plot

 Box:

Represents the interquartile range (IQR), with the left side as Q1 and the
right side as Q3. The length of the box shows the spread of the middle 50%
of the data.
  Median (Q2):

A line inside the box that marks the middle value of the dataset, representing
the 50th percentile.

 Whiskers:

Lines extending from the box that typically show the range of the data
excluding outliers. The whiskers extend to the minimum and maximum
values that are within

 Outliers:

Individual points plotted beyond the whiskers to indicate values that are
unusually high or low compared to the rest of the data.

PIVOT TABLE
In data science, a pivot table is a powerful tool for summarizing, aggregating, and
analyzing large datasets by reorganizing them into a new, understandable format. It
allows data scientists to quickly perform exploratory data analysis (EDA) by grouping,
filtering, and calculating descriptive statistics like sums, averages, and counts from raw
data. Libraries like Pandas in Python provide a pivot_table() function that serves this
purpose, enabling users to turn rows into columns and perform complex aggregations for
data interpretation and decision-making.

Key functions and use cases

 Data summarization: Pivot tables allow you to condense large amounts of data
into a more manageable summary table, which is crucial for getting a quick
overview of the data.
 Aggregation: They can calculate various aggregations on grouped data, such as
the sum, count, average, minimum, or maximum values.
 Data exploration: Pivot tables are a core part of EDA, helping data scientists
understand relationships between variables and identify patterns, trends, or outliers
by slicing and pivoting the data from different perspectives.
 Data manipulation: The process of creating a pivot table involves defining rows,
columns, and values, and the tool can handle complex arrangements, including
partial sums or margins.
 Interactive analysis: Pivot tables can be made interactive, allowing users to easily
change the structure (e.g., dragging and dropping fields), filter data, and see how
the summary changes in real-time.
ANOVA
ANOVA, or Analysis of Variance, is a statistical method in data science used to compare the
means of three or more groups to determine if there are significant differences between them. It
works by analyzing the variance within each group and the variance between the groups, and a
common application is to check if an independent variable has a statistically significant effect on
a dependent variable. ANOVA is particularly useful for avoiding the increased error rate that
comes from running multiple t-tests when comparing more than two groups.

 How it works: ANOVA partitions the total variability in a dataset into two
sources: the variation between the groups (e.g., different treatments) and the
variation within the groups (random variation).
 Hypothesis testing: It tests the null hypothesis that all group means are equal
against the alternative that at least one group mean is different.
 Key output: The test produces an F-statistic, which is a ratio of the between-group
variance to the within-group variance. A high F-statistic suggests the groups are
significantly different.
 Types: The two most common types are:
o One-Way ANOVA: Compares groups based on a single independent variable.

o Two-Way ANOVA: Examines the effect of two or more independent variables on

a dependent variable and their interaction.

 Data science applications:

o Comparing the average performance of different ad campaigns.

o Determining if different website designs lead to statistically different conversion

rates.

o Evaluating the impact of different marketing strategies on customer sales.