`Statistics Revision:

Kurtosis

Skewness

Null Hypothesis

Alternate Hypothesis

Data Transformation for Skewed data

Z Transform

Min Max

Hypothesis Testing:
If a hypothesis is to be done for “Average score of predictive analytics is 90”

Then it is a 1 sample Hypothesis and the Null Hypothesis will be H0 = 90

Alternate Hypothesis will be “The Average Score in not equal to 90”

When P value < 0.05 then we reject the Null Hypothesis H0.

The significance value alpha reflects the risk to reject the Null Hypothesis

P Value
Significance value alpha
Confidence Interval

Data Mining:
Digging deep in the data to get the hidden information

“If Trump imposes another 25% Tariff on India, then what should India do?”

1. List out the products India was exporting to the US.

2. List out the products based on export revenue volume.
3. List of the Countries where we can export these products.
4. Analysing the demographics of that country.
5. Analysing the purchasing power of the people in that country.
6. Analysing the existing competition.
Box Plot:
Interquartile Range (IQR) = Q3 – Q1

Q2 = Median

Q1 = 25TH percentile

Q3 = 75th percentile

Data Type:
Qualitative:

1. Nominal
2. Ordinal

Quantitative:

1. Discrete
2. Continous

Regression:

If dependent variable is numerical then we use “Regression”.

If dependent variable is qualitative then we use “Classification models”.

Dependent Variable = Independent Variable 1 + Independent Variable 2

Different Types

CRISP-DM: Elaborate view

Business Data Data

Modeling Evalutaion Deployement
Understanding Undestanding Preparation
Difference Between Census and Sampling
Types of Sampling Method:
1. Probabilistic method
a. Simple Random Sampling
b. Systematic Sampling
c. Stratified Sampling
d. Cluster Sampling
e. Multistage Sampling
2. Non-Probabilistic Sampling
a. Convenience Sampling
b. Judgemental / Positive Sampling
c. Quota Sampling
d. Snowball Sampling

What is Correlation?
We do correlation to measure the strength and direction of the relationship between two variables
— without necessarily assuming one causes the other.

What is Causation?
Causation means that a change in one variable directly causes a change in another variable.

Example 1 — Causation

 Variable X: Hours of study

 Variable Y: Exam marks

If an experiment shows that students who study more consistently score higher (and all
other factors like IQ, prior knowledge, and distractions are controlled), we can say:

Increased study time causes higher exam marks.

Key difference

 Correlation: Variables move together, but we don’t know why.

 Causation: One variable actually produces a change in the other.

Residual Value:
It is the difference between the Actual Value and the Values predicted also known as Error.

Predicted Line: Y = mX + C
Assumptions of Regression:
The Four assumptions in Regression are:

1. Linearity
2. Independence
3. Normality
4. Equal Variance or Homoscedastic

Residual Analysis
 Residuals vs fitted
 Normal Q-Q
 Scale Location
 Residuals vs leverage

Durbin Watson Statistics:

The Durbin–Watson (DW) statistic is a test used in regression analysis to detect autocorrelation
(specifically, first-order autocorrelation) in the residuals.

1. What is autocorrelation?

 Autocorrelation means that the residual (error) from one observation is correlated with the
residual from another observation, especially the one just before it.

 It’s common in time-series data, where today's errors might be related to yesterday's errors.

 If autocorrelation exists, it violates the independence assumption of regression, which can

make statistical tests unreliable.

2. DW statistic formula

DW=∑t=2n(et−et−1)2∑t=1net2DW = \frac{\sum_{t=2}^{n} (e_t - e_{t-1})^2}{\sum_{t=1}^{n}

e_t^2}DW=∑t=1net2∑t=2n(et−et−1)2

where:

 ete_tet = residual at time ttt

 et−1e_{t-1}et−1 = residual at previous time

3. Interpreting Durbin–Watson

DW values range from 0 to 4:

DW Value Interpretation

~2 No autocorrelation

0 to <2 Positive autocorrelation

DW Value Interpretation

>2 to 4 Negative autocorrelation

Example:

 DW = 0.9 → strong positive autocorrelation

 DW = 2.05 → no autocorrelation

 DW = 3.1 → negative autocorrelation

4. Why it matters

 Autocorrelation inflates the significance of predictors (makes p-values too small).

 In business forecasting (sales, stock prices, etc.), failing to detect autocorrelation can lead to
overconfident and misleading predictions.

Outliers and High Leverage Values

We make Box Plot and check the Outliers

1. Outlier

 Definition: A data point whose Y-value (dependent variable) is unusually far from the value
predicted by the regression model.

 Outliers have large residuals.

 They may or may not influence the regression line heavily.

Example:
If most houses priced around ₹50–60 lakh for 1,000 sq ft, but one house sells for ₹90 lakh, that’s an
outlier in price.

Identification:

 Standardized residuals: If > +3 or < -3, it’s likely an outlier.

 Studentized residuals: Values > ±2 (or ±3 for large samples).

 Plot residuals: Look for points far above/below the rest.

2. High-Leverage Value

 Definition: A data point with an unusual X-value (independent variable), far from the mean
of X.

 Even if its Y-value fits the line well, it has the potential to strongly influence the slope and
intercept.
  High leverage points pull the regression line toward themselves.

Example:
If most houses are between 800–1,200 sq ft, but one is 3,000 sq ft, that’s a high-leverage point —
even if its price fits the trend.

Identification:

 Leverage (Hat) values hiih_{ii}hii:

hii>2(k+1)nh_{ii} > \frac{2(k+1)}{n}hii>n2(k+1)

indicates high leverage.

o kkk = number of predictors

o nnn = sample size

 Scatter plot of X vs. Y: Look for X-values far from the pack.

3. Outlier vs. High-Leverage

Feature Outlier High-Leverage Value

Which variable? Unusual Y-value Unusual X-value

Impact Can distort fit if influential Can strongly change slope

Detection Large residual Large hat value

4. Influential Point

 A point can be both an outlier and have high leverage — these are called influential points.

 Detection: Cook’s Distance (> 1 or much larger than others) indicates strong influence on the
model.

Python Packages for Linear Regression:

Steps for Performing Linear Regression:

1. Load the packages
2. Read the file
3. Data Cleaning
4. Missing Value Analysis
5. Outliers
6. Format
7. After data is clean, check Correlation
8. Regression LM
 9. Model Development
a. Training Dataset
b. Test Dataset
10. Developing model based in training data
11. Performance Evaluation

Degree of Freedom:
Number of independent variables in a dataset that are free to vary when estimating a parameter

F(1 , 22) means there are one Dependent variable and 22 independent variables

R squared value is that 1 unit of change in ad results in 0.84 units of sales

Multivariate Linear Regression:

Interaction Effect: If the relationship of one variable is affected by another variable, then it is called
Interaction

Steps for predictive analytics

1. Download all libraries
2. Load the data
3. Prepare the data – Missing data, Mean, Median, Outliers
4. Check for errors: RMS, Mean absolute error
5. Cross Validation
6. Predict

Cross Validation
Cross-validation is a technique for assessing how well a predictive model, like those used in machine
learning, will perform on new, unseen data. It works by repeatedly splitting the dataset into training
and validation subsets, training the model on the training data, and then testing its performance on
the validation data. By repeating this process with different partitions, cross-validation provides a
more robust estimate of the model's true performance and helps to prevent overfitting, where a
model learns the training data too well and fails to generalize to new information.

Difference between R-Square and Adjusted R-Square:

R-squared measures the proportion of variance in the dependent variable explained by the
independent variables, but it always increases with more predictors, potentially leading to
overfitting. Adjusted R-squared, on the other hand, accounts for the number of predictors in a model
by penalizing the addition of non-significant variables, providing a more accurate and generalized
measure of model fit, especially for models with multiple independent variables.

Variance Inflation Factors

VIF values, or Variance Inflation Factors, are statistical metrics that measure the extent of
multicollinearity in a multiple regression model, indicating how much the variance of a regression
coefficient is inflated due to correlations among predictor variables. A VIF of 1 means no correlation,
while higher values, often considered problematic above 5 or 10, suggest a variable is highly
redundant and may make it difficult to determine its individual effect on the dependent variable,
potentially leading to unstable and unreliable regression results.

Forward Regression
Forward regression, or forward selection, is a stepwise regression method that starts with an empty
model and iteratively adds predictor variables one at a time. In each step, the variable that most
improves the model's predictive accuracy (according to a predefined statistical criterion, like an F-
statistic) is added, and the process continues until no more variables offer a statistically significant
improvement, at which point the model stops growing.

Stepwise Regression
Stepwise regression is an automated statistical technique for selecting the best predictor variables
for a regression model, creating a parsimonious (simple) and accurate model. It iteratively adds or
removes variables based on predefined statistical criteria, such as t-tests or F-tests, to determine
which ones are most statistically significant in explaining the outcome. The process can involve
Forward Selection (starting with no variables and adding them), Backward Elimination (starting with
all variables and removing them), or a combination of both.

Principal Component Analysis (PCA)

It is a statistical technique used for dimensionality reduction.1 Its main purpose is to simplify
complex, high-dimensional data by transforming it into a smaller set of new, uncorrelated variables
called principal components.2

Think of it like summarizing a long, detailed book. You don't rewrite the entire book, but you capture
the main plot points (the principal components) so you can understand the story without reading
every single page. You lose some minor details, but you retain the most important information.3
Why is PCA Required?

PCA is required to address two common problems in data analysis:4

1. To Simplify Data and Reduce Dimensionality 📉

Modern datasets can have hundreds or even thousands of features (or dimensions). Working with
such high-dimensional data is difficult for several reasons:

 Computational Cost: More features mean more processing power and time are needed to
train a model.

 Overfitting: With too many features, a machine learning model might start to learn the noise
in the data instead of the underlying patterns, leading to poor performance on new data.5

 Visualization: It's impossible to visualize data in more than three dimensions.

PCA helps by identifying the most important patterns in the data and combining the original features
into a few principal components.6 For example, it might take 50 original features and reduce them to
just 3 or 4 principal components that still capture over 95% of the information.

2. To Address Multicollinearity 🔗

Multicollinearity occurs when independent variables in a regression model are highly correlated. This
is problematic because it becomes difficult to determine the individual effect of each variable on the
outcome.7

For instance, in a dataset of houses, the total_square_footage and the number_of_bedrooms are
likely to be highly correlated. PCA solves this by creating new, uncorrelated principal components. 8 By
using these components in a model instead of the original variables, you remove the issue of
multicollinearity, leading to more stable and reliable model results.9

Multicollinearity
Multicollinearity is a statistical phenomenon that occurs in multiple regression analysis when two or
more independent variables are highly correlated with each other. In simpler terms, it means that
one predictor variable in a model can be linearly predicted from the others with a substantial degree
of accuracy.

This high correlation between predictor variables can cause several problems in a regression model,
making it difficult to assess the individual effect of each independent variable on the dependent
variable.

In regression analysis, the additive method and the interaction method are two ways to model the
relationship between independent variables and a dependent variable. The core difference lies in
how they treat the combined effect of the predictors.
Additive Method
The additive method assumes that the effect of each independent variable on the dependent
variable is independent of the other independent variables in the model. In other words, the total
effect is simply the sum of the individual effects of each predictor.

Think of it like ordering a pizza 🍕. With an additive model, the total cost is the base price plus the
price of pepperoni plus the price of mushrooms. The cost of adding mushrooms doesn't change just
because you already added pepperoni.

The formula for a simple additive model with two independent variables (X1 and X2) looks like this:

Y=β0+β1X1+β2X2+ϵ

Here, β1 represents the change in the dependent variable Y for a one-unit change in X1, regardless of
the value of X2. The effects of X1 and X2 are separate and simply add up.

Interaction Method
The interaction method assumes that the effect of one independent variable on the dependent
variable depends on the level of another independent variable. The relationship is synergistic,
meaning the combined effect of the variables is different from the sum of their individual effects.

Let's go back to the pizza analogy. An interaction effect would be like a "combo deal." If you order
both pepperoni and mushrooms, you might get a discount. The effect of adding mushrooms on the
total price changes if you also have pepperoni.

To capture this, an interaction term (the product of the two independent variables) is added to the
model. The formula for a model with an interaction effect is:

Y=β0+β1X1+β2X2+β3(X1⋅X2)+ϵ

 β1 and β2 are the main effects.

 β3 is the interaction effect. It quantifies how the effect of X1 on Y changes as X2 changes

(and vice versa). If β3 is statistically significant, it indicates that an interaction is present.

Key Differences Summarized

Feature Additive Method Interaction Method

The effect of one predictor is The effect of one predictor depends on the
Assumption
independent of others. level of another.

The effects are separate and

Relationship The effects are synergistic or interdependent.
cumulative.

Includes only main effects (e.g., X1,X2 Includes main effects and an interaction term
Model Term
). (e.g., X1⋅X2).
Feature Additive Method Interaction Method

Straightforward: "A one-unit increase More complex: "The effect of X1 on Y changes

Interpretation
in X causes a Y increase of β1." by β3 for every one-unit increase in X2."

Scree Plot
A scree plot is a simple line graph used in statistical methods like Principal Component Analysis (PCA)
and Factor Analysis to help determine the optimal number of components or factors to retain from
your data.1

Essentially, it's a tool for dimensionality reduction. It plots the eigenvalues (which represent the
amount of variance explained by each component) on the y-axis against the number of the
component on the x-axis.2 The components are always ordered from largest eigenvalue to smallest.3

How to Interpret a Scree Plot

The main goal is to find the "elbow" of the plot.4 This is the point where the curve of the line begins
to flatten out.5

1. The Steep Slope: The components on the steep part of the slope are the most significant.6
They each explain a large and distinct portion of the variance in the data. You should keep
these components.

2. The Elbow: This is the point of inflection where the steep decline transitions to a flatter
slope.7 It represents the cut-off point.

3. The Rubble/Scree: The components after the elbow are on the flat part of the slope. They
explain very little variance and are often considered "noise" or random variability. You should
discard these components.

The term "scree" is a geological metaphor.8 The significant components are like the solid part of a
mountain, while the less significant ones are like the pile of loose rock debris, or "scree," that has
fallen off the mountain and lies at its base.

In the example plot above, the elbow is clearly visible at Component 3. After this point, the line
flattens out, indicating that components 3, 4, 5, and beyond contribute very little additional
information. Therefore, based on this scree plot, you would decide to retain two principal
components.

Rotated Components
RC1, RC2, and RC3 are the rotated components (or factors) that have been extracted from your
original data.

Think of them as new, summary variables created by the analysis. Each one represents an underlying
pattern or theme shared by a group of your original variables.
How to Interpret Them from Your Output

You can understand what each component represents by looking at the Factor Loadings table. A high
loading value (typically > 0.5) means that the original variable is strongly related to that component.

Based on your image:

 RC1 (Rotated Component 1): This component is strongly associated with wheelbas, width,
and length. You could interpret RC1 as a "Vehicle Size" or "Dimensions" factor.

 RC2 (Rotated Component 2): This component is strongly associated with engine_s and
horsepow. You could interpret RC2 as an "Engine Power" factor.

 RC3 (Rotated Component 3): This component is strongly associated with curb_wgt and
fuel_cap. You could interpret RC3 as a "Weight and Capacity" factor.

Varimax and Promax are two different types of rotation methods used in statistical techniques like
Factor Analysis or Principal Component Analysis (PCA). Rotation is a step performed after the initial
factors are extracted to make the results simpler and more interpretable.

The main difference between them is that Varimax is an orthogonal rotation method, while Promax
is an oblique rotation method.

Varimax Rotation
Varimax rotation simplifies the interpretation of factors by maximizing the variance of the loadings
on each factor. This means it tries to make the loadings either very high or very low (close to 0).

The key feature of Varimax is that it is an orthogonal rotation. This means it keeps the resulting
factors uncorrelated with each other. The axes representing the factors are maintained at a 90-
degree angle.

 When to use it: Use Varimax when you have a strong theoretical reason to believe that the
underlying factors are independent of each other.

 Advantage: The results are often easier to interpret because the factors are distinct and not
correlated.

Promax Rotation
Promax is an oblique rotation method. This means it allows the resulting factors to be correlated
with each other. The axes representing the factors are not restricted to 90-degree angles and can be
positioned closer together.

Promax actually starts with an orthogonal Varimax rotation and then allows the factors to become
correlated to achieve a better fit to the data.

 When to use it: Use Promax when you expect the underlying factors to be related. This is
common in many real-world scenarios, especially in social sciences (for example, anxiety and
depression are distinct but related concepts).
  Advantage: It can provide a more accurate and realistic representation of the data if the
underlying factors are truly correlated.

Chi-square
The Chi-square (written as χ2) test is a statistical hypothesis test used to determine if there is a
significant association between two categorical variables. In simpler terms, it helps you understand if
the observed frequencies in a sample are different from what you would expect by chance.

The core idea of the Chi-square test is to compare the observed values (the data you collected) with
the expected values (the data you would anticipate if there were no relationship between the
variables).

There are two main types of Chi-square tests:

1. Chi-square Goodness-of-Fit Test

This test is used when you have one categorical variable and want to determine if the observed
frequency distribution fits a specific theoretical or expected distribution.

Example: A company claims that their bags of marbles contain 30% red, 30% blue, and 40% green
marbles. You buy a bag of 100 marbles and find 35 are red, 25 are blue, and 40 are green. A Chi-
square goodness-of-fit test can tell you if your observed counts are significantly different from the
company's claim.

2. Chi-square Test for Independence

This is the more common type of Chi-square test. It is used to determine if there is a significant
association between two categorical variables.

Example: You want to know if there is a relationship between a person's favorite ice cream flavor
(chocolate, vanilla, strawberry) and their gender (male, female). You survey a group of people and
record their preferences. A Chi-square test for independence can determine if the choice of ice
cream flavor is independent of gender.

How the Chi-square Statistic is Calculated

The formula for the Chi-square statistic is:

χ2=∑E(O−E)2

Where:

 O is the observed frequency (the actual count in your data).

 E is the expected frequency (the count you would expect if there were no relationship
between the variables).

 Σ means you sum the results for all categories.

A larger Chi-square value indicates a greater difference between the observed and expected
frequencies, which suggests that there is a relationship between the variables.
How to Interpret the Results

The calculated Chi-square value is then used to find a p-value. The p-value tells you the probability of
observing your data if there were no real association between the variables.

 A small p-value (typically ≤ 0.05) suggests that the observed association is statistically
significant, and you can reject the null hypothesis (the hypothesis that there is no
association).

 A large p-value (typically > 0.05) suggests that the observed association is not statistically
significant, and you cannot reject the null hypothesis.

In summary, the Chi-square test is a valuable tool for analyzing categorical data to see if there are
meaningful relationships between variables.

Classification:
Clustering:
Clustering in machine learning is an unsupervised learning technique that groups similar data points
into clusters based on their inherent similarities, without needing pre-existing labels or
categories. It's a method of data analysis that helps uncover patterns and natural groupings within
large, unlabelled datasets, enabling users to understand underlying trends, identify segments, and
simplify complex data.

Major Types of Clustering

1. Partitional Clustering
a. Description: Divides data into a specified number of non-overlapping subsets
(clusters).
b. Example Algorithms: K-Means, K-Medoids.
2. Hierarchical Clustering
a. Description: Creates a hierarchy of nested clusters, forming a tree-like structure
(dendrogram).
b. Sub-Types:
c. Agglomerative (Bottom-Up): Starts with each data point as an individual cluster and
merges them.
d. Divisive (Top-Down): Starts with one large cluster and recursively splits it into smaller
clusters.
e. Example Algorithms: Agglomerative Clustering.
3. Density-Based Clustering
a. Description: Groups data points that are closely packed together, identifying clusters
of arbitrary shapes and detecting outliers.
b. Example Algorithms: DBSCAN.
4. Model-Based Clustering
a. Description: Assumes that the data is generated from a mixture of probability
distributions and then tries to find the parameters of these distributions to cluster
the data.
b. Example Algorithms: Gaussian Mixture Model (GMM).
Types of transformation:

1. Min Max
2. Z-score normalisation

Methods of finding the distance:

1. Euclidean
2. Manhattan
3. Minkowski

Limitation:

 For very large data sets, computing and storing the n x n matrix may be expensive and slow.
 It is sensitive to outliers and missing data.
 It has low stability. Reordering data or dropping a few records can result in a different
solution.
 When the metric is changed from one measure to another, results may vary.

Classification:
KNN
K-Nearest Neighbors (KNN) is a simple classification algorithm that predicts the class of a new data
point by taking a majority vote of its 'k' closest neighbors.

## The "New Neighbor on the Block" Analogy

Imagine you move into a new neighborhood and want to guess if your neighbors are likely to be
"social" or "private."

 k=3: You decide to look at the 3 houses closest to you.

 Observe: You see that 2 of these neighbors are "social" (they have barbecues, say hello) and
1 is "private" (keeps to themselves).

 Predict: Based on the majority, you predict that you've moved into a "social" part of the
neighborhood.

## The KNN Algorithm

1. Choose a number 'k': Decide how many neighbors will influence the prediction (e.g., k=5).

2. Calculate Distances: For a new data point, calculate its distance (commonly Euclidean
distance) to every other point in the existing dataset.

3. Find the 'k' Nearest Neighbors: Identify the 'k' data points with the shortest calculated
distances.

4. Vote for the Class: Count the number of neighbors belonging to each class.
 5. Assign the Class: Assign the new data point to the class that gets the most votes.

Naïve Baiyes
The Naive Bayes classifier is a simple and powerful probabilistic algorithm based on Bayes' theorem.
It's used for classification tasks.

The "naive" part of the name comes from its core assumption: it assumes that all the features of a
data point are independent of one another, given the class. This is a strong assumption that is often
not true in the real world, but the algorithm works surprisingly well anyway.

## The "Fruit Identification" Analogy 🍎

Imagine you have a mystery fruit and you want to classify it as either an Apple or a Banana. You have
three features to consider: its color, shape, and taste.

A Naive Bayes classifier would solve this by thinking like this:

1. Look at Past Data (Training): First, it looks at a hundred fruits you've already identified. It
calculates the probabilities for each feature independently.

o Probability of being an Apple: 50 out of 100 fruits were apples (P(Apple) = 0.5).

o Probability of being 'Red' if it's an Apple: 40 out of 50 apples were red (P(Red |
Apple) = 0.8).

o Probability of being 'Round' if it's an Apple: 45 out of 50 apples were round

(P(Round | Apple) = 0.9).

2. Analyze the New Fruit: Now, you get a new fruit that is Red, Round, and Sweet.

3. Calculate Probabilities: The algorithm naively assumes these features are unrelated. It
calculates the probability of this fruit being an Apple versus a Banana.

o For "Apple": It multiplies the probabilities: P(Apple) * P(Red | Apple) * P(Round |

Apple) * P(Sweet | Apple)

o For "Banana": It does the same: P(Banana) * P(Red | Banana) * P(Round | Banana) *
P(Sweet | Banana)

4. Make a Prediction: The algorithm compares the final probability scores. Whichever class has
the higher probability is the prediction. Because the new fruit's features (Red, Round) are
much more common in apples, the "Apple" score will likely be higher.

## How the Naive Bayes Algorithm Works

1. Calculate Prior Probabilities: For each class, calculate its probability of occurring in the
dataset (e.g., the overall percentage of apples).
 2. Calculate Likelihoods: For each feature, calculate the conditional probability of that feature
occurring given a certain class (e.g., the probability of being "Red" given that the fruit is an
"Apple").

3. Apply Bayes' Theorem: For a new data point, use Bayes' theorem to calculate the posterior
probability for each class. This combines the prior probabilities and the likelihoods.

4. Make a Prediction: The class with the highest posterior probability is chosen as the final
prediction.

## Types of Naive Bayes

 Gaussian Naive Bayes: Used for continuous features (like height or weight), assuming they
follow a normal (Gaussian) distribution.

 Multinomial Naive Bayes: Commonly used for text classification (like spam filtering), where
features are typically word counts.

 Bernoulli Naive Bayes: Used when features are binary (e.g., "yes" or "no," "present" or
"absent").

Decision Tree
A Decision Tree is a versatile and intuitive classification model that predicts the class of a data point
by learning simple decision rules from the features. It operates like a flowchart, where each internal
node represents a "test" on a feature, each branch represents the outcome of the test, and each leaf
node represents a class label.

## The "Should I Play Tennis?" Analogy 🎾

Imagine you want to decide whether to play tennis today. Your decision depends on a few factors:
the weather outlook, the humidity, and the wind. A decision tree would help you make this choice by
asking a series of simple, yes/no questions.

1. Start at the Root: The first question is, "What is the weather outlook?" This is the root node.
Let's say the options are Sunny, Overcast, or Rain.

2. Follow the Branches: If the answer is "Overcast," the tree might immediately lead to a leaf
node that says, "Yes, play tennis."

3. Ask More Questions: If the answer is "Sunny," the tree might lead to another internal node
asking, "Is the humidity high?"

4. Reach a Decision: If the humidity is high, the tree might conclude, "No, don't play tennis." If
the humidity is normal, it might conclude, "Yes, play tennis."

The algorithm learns this structure by analyzing past data to find the best questions to ask at each
step to most effectively separate the data into distinct classes.
## How a Decision Tree is Built

The process of building a decision tree involves recursively splitting the data into smaller and smaller
subsets.

1. Select the Best Feature to Split: The algorithm starts with the entire dataset and chooses the
feature that best splits the data into the most "pure" subgroups. A pure subgroup is one that
contains data points of mostly a single class. Common criteria for measuring the quality of a
split are Gini impurity and entropy.

2. Create a Decision Node: A node is created for the selected feature.

3. Split the Data: The dataset is divided into subsets based on the values of the chosen feature.

4. Repeat for Each Subset: The algorithm repeats steps 1-3 for each subset, creating a new
branch and a new decision node. This process continues until a stopping condition is met,
such as when a subgroup is pure or a predefined depth of the tree is reached.

## Key Terminology

 Root Node: The topmost node that represents the entire dataset.

 Internal Node: A node that represents a test on a feature.

 Branch: The link between nodes that represents the outcome of a test.

 Leaf Node (or Terminal Node): A node that represents a final class label or decision.

 Pruning: The process of removing branches from a tree to reduce its complexity and prevent
overfitting.

Confusion Matrix
A confusion matrix is a table that summarizes the performance of a classification model by showing
its correct and incorrect predictions for each class. It provides a detailed breakdown of how
"confused" your model is.

## The "Spam Filter" Analogy 📧

Imagine you have a spam filter for your email. Its job is to classify every incoming email as either
"Spam" or "Not Spam." After it has classified 100 emails, you manually check its work.

A confusion matrix organizes the results into a simple 2x2 grid to answer four key questions:

1. How many spam emails did it correctly label as "Spam"? (True Positives)

2. How many real emails did it correctly label as "Not Spam"? (True Negatives)

3. How many real emails did it incorrectly label as "Spam"? (False Positives - a mistake!)

4. How many spam emails did it incorrectly let into your inbox as "Not Spam"? (False Negatives
- another mistake!)
## The Four Components of a Confusion Matrix

To understand the matrix, you need to know these four terms. Let's assume the "positive" class is
"Spam."

1. True Positives (TP):

o What it is: The model correctly predicted the positive class.

o Example: The email was spam, and the model correctly flagged it as spam.

2. True Negatives (TN):

o What it is: The model correctly predicted the negative class.

o Example: The email was not spam, and the model correctly left it in your inbox.

3. False Positives (FP) - Type I Error:

o What it is: The model incorrectly predicted the positive class.

o Example: The email was not spam, but the model incorrectly sent it to your spam
folder. This is an annoying error because you might miss an important email.

4. False Negatives (FN) - Type II Error:

o What it is: The model incorrectly predicted the negative class.

o Example: The email was spam, but the model incorrectly let it into your inbox.

## Why is it Useful?

A confusion matrix is more informative than a simple accuracy score. It allows you to calculate
several important performance metrics:

 Accuracy: (TP + TN) / Total. What fraction of predictions were correct overall?

 Precision: TP / (TP + FP). Of all the emails the model flagged as spam, how many actually
were spam? (Measures the quality of the positive predictions).

 Recall (Sensitivity): TP / (TP + FN). Of all the actual spam emails, how many did the model
correctly identify? (Measures the model's ability to find all positive instances).

 F1-Score: The harmonic mean of Precision and Recall, providing a single score that balances
both.

Accuracy:
Accuracy = (True Positive + True Negative) / (TP + TN + FP + FN)
Balanced and Unbalanced confusion matrix:
A balanced confusion matrix is one generated from a dataset where the classes are represented in
roughly equal proportions. An unbalanced confusion matrix comes from a dataset where one class
significantly outnumbers the other(s). The terms "balanced" and "unbalanced" refer to the
underlying data, not the structure of the matrix itself.

## The "Team Photos" Analogy 📸

Imagine you're a photographer taking pictures of two school sports teams for the yearbook:

 Balanced Scenario: You take pictures of the basketball team (12 players) and the volleyball
team (12 players). The number of students in each category ("basketball" vs. "volleyball") is
equal. A confusion matrix evaluating a model trained on this data would be considered
balanced.

 Unbalanced Scenario: You take pictures of the football team (50 players) and the chess club
(5 players). The "football" category vastly outnumbers the "chess" category. A confusion
matrix from this data would be unbalanced.

## How They Look in Practice

The key difference is visible in the "Actual Values" totals for each class.

Balanced Confusion Matrix Example

Let's say you have 200 data points total: 100 cats (Positive) and 100 dogs (Negative). The matrix
might look like this:

Predicted: Cat Predicted: Dog Total Actual

Actual: Cat 90 (TP) 10 (FN) 100

Actual: Dog 5 (FP) 95 (TN) 100

Here, the total number of actual cats and dogs is equal.

Unbalanced Confusion Matrix Example

Now, let's say you're detecting a rare disease. You have 200 patients: 10 have the disease (Positive)
and 190 do not (Negative).

Predicted: Disease Predicted: No Disease Total Actual

Actual: Disease 8 (TP) 2 (FN) 10

Actual: No Disease 5 (FP) 185 (TN) 190

Notice the huge difference in the "Total Actual" column. This is an unbalanced problem.

## Why This Distinction is Crucial

In an unbalanced scenario, the standard accuracy metric can be very misleading.

In the disease example, a lazy model that predicts "No Disease" every single time would get 190 out
of 200 predictions correct, giving it a 95% accuracy! However, it completely fails at its actual purpose,
which is to find patients who have the disease.

For unbalanced datasets, it's much more important to look at other metrics derived from the
confusion matrix, such as:

 Precision: How reliable are the positive predictions?

 Recall (Sensitivity): How well does the model find all the positive cases?

 F1-Score: A balance between precision and recall.

Categorical data to factors

Transforming categorical data into factors is the process of converting text-based categories (like
"Low," "Medium," "High") into a numerical format that statistical models can understand, while still
retaining the original category labels and their order.

## The "T-Shirt Drawer" Analogy 👕

Imagine you have a drawer full of t-shirts with sizes written on them: "Small," "Medium," "Large,"
"Small," "Large."

 Categorical Data: The raw text ("Small," "Medium," "Large") is your categorical data. A
computer can't easily perform math on these words.

 Transforming to a Factor: You decide to organize them. You create a system where:

o 1 = Small

o 2 = Medium

o 3 = Large

Now, your drawer of shirts can be represented by the numbers: 1, 2, 3, 1, 3. This is what a factor
does. It stores the data as integers but remembers that "1" means "Small," "2" means "Medium,"
and so on. It also understands the inherent order: Large > Medium > Small.

## How to Do It in R
In R (a popular statistical programming language), this is a very common and straightforward task.
You use the [Link]() function.

Let's say you have a vector of survey responses:

responses <- c("Agree", "Disagree", "Neutral", "Agree")

To transform this into a factor, you simply run:

responses_factor <- [Link](responses)

What happens under the hood? R now sees the data like this: [1, 2, 3, 1]. It also stores the "levels" or
labels: 1="Agree", 2="Disagree", 3="Neutral". This is much more efficient for modeling algorithms to
work with.

## Why is This Important?

1. Modeling Requirement: Most regression and classification models require numerical input.
Factors provide this numerical representation while preserving the categorical nature of the
data.

2. Establishing Order: For ordinal data (like "Low," "Medium," "High"), converting to a factor
allows you to explicitly define the order, which is crucial for some models to interpret the
data correctly.

3. Efficiency: Storing categories as integers is more memory-efficient than storing them as long
text strings, especially in large datasets.

Dummy Variables
Dummy variables are numerical variables (0s and 1s) used to represent categorical data in a
regression model. Since regression models can only process numbers, dummy variables act as on/off
switches to incorporate categories like "City" or "Product Type" into the analysis.

Sensitivity:
Sensitivity, also known as Recall or the True Positive Rate (TPR), is a performance metric for a
classification model. It measures the model's ability to correctly identify all relevant instances of the
positive class.

In simple terms, sensitivity answers the question: "Of all the actual positive cases, what proportion
did our model successfully find?"

## The Airport Security Analogy ✈️

Imagine an airport security scanner's job is to detect prohibited items (the "positive" class).

 10 prohibited items pass through the scanner today.

  The scanner successfully beeps and flags 8 of them.

 It fails to detect 2 of them (these are the "misses" or False Negatives).

In this scenario, the scanner's sensitivity (or recall) is 8 out of 10, or 80%. It correctly identified 80%
of all the prohibited items it should have found.

A high sensitivity is crucial in situations where failing to detect a positive case has severe
consequences, such as in medical diagnoses or fraud detection.

## Formula for Sensitivity

Sensitivity is calculated using the values from a confusion matrix.

Sensitivity (Recall)=True Positives (TP)/ (True Positives (TP) + False Negatives (FN) )

Where:

 True Positives (TP): The number of positive cases the model correctly identified.

 False Negatives (FN): The number of positive cases the model incorrectly labeled as negative
(the ones it "missed").

Specificity:
Specificity, also known as the True Negative Rate (TNR), is a performance metric for a classification
model. It measures the model's ability to correctly identify all relevant instances of the negative
class.

In simple terms, specificity answers the question: "Of all the actual negative cases, what proportion
did our model successfully identify?"

## The Spam Filter Analogy 📧

Imagine a spam filter's job is to correctly identify emails that are "Not Spam" (the "negative" class)
and leave them in your inbox.

 100 legitimate emails are sent to you today.

 The filter correctly leaves 95 of them in your inbox.

 It incorrectly sends 5 of them to the spam folder (these are the "false alarms" or False
Positives).

In this scenario, the filter's specificity is 95 out of 100, or 95%. It correctly identified 95% of all the
legitimate emails.

A high specificity is crucial in situations where a "false alarm" is costly or very undesirable, such as a
medical test that wrongly tells a healthy person they have a disease.
## Formula for Specificity

Specificity is calculated using the values from a confusion matrix.

Specificity=True Negatives (TN)/ (True Negatives (TN) + False Positives (FP) )

Where:

 True Negatives (TN): The number of negative cases the model correctly identified.

 False Positives (FP): The number of negative cases the model incorrectly labeled as positive
(the "false alarms").

Precision:
Precision is a performance metric for a classification model that measures the accuracy of the
positive predictions. It answers the question: "Of all the predictions I made for the positive class,
how many were actually correct?"

## The "Spam Filter" Analogy 📧

Imagine a spam filter flags 10 emails and moves them to your spam folder. You check the folder and
find:

 8 of the emails were actually spam. (True Positives)

 2 of the emails were important, legitimate emails. (False Positives - the "false alarms")

In this case, the filter's precision is 8 out of 10, or 80%. When it predicted an email was spam, it was
correct 80% of the time.

A high precision is crucial when the cost of a false positive is high. For a spam filter, low precision is
annoying because you might miss important emails. For a medical diagnosis, low precision is
dangerous because it could lead to a healthy person receiving unnecessary and potentially harmful
treatment.

## Formula for Precision

Precision is calculated using the values from a confusion matrix.

Precision=True Positives (TP)+False Positives (FP)True Positives (TP)

Where:

 True Positives (TP): The number of positive cases the model correctly identified.

 False Positives (FP): The number of negative cases the model incorrectly labeled as positive
(the "false alarms").
Detection Rate:
Detection rate is another name for Recall or Sensitivity. It's a performance metric that measures a
model's ability to find all the actual positive cases within a dataset.

It answers the question: "Of all the true positive instances that exist, what percentage did my
model successfully detect?"

## The "Fishing Net" Analogy 🎣

Imagine you are a fisher trying to catch a specific type of fish, let's say "tuna" (the positive class).

 There are 100 tuna in the area of the sea you're fishing in.

 You cast your net and pull in 75 tuna.

 You missed 25 tuna that are still in the water.

In this scenario, your detection rate is 75 out of 100, or 75%. Your net successfully detected 75% of
all the tuna that were available to be caught.

A high detection rate is critical when the cost of not finding a positive case is very high. For example,
in fraud detection, you want to detect as many fraudulent transactions as possible. In medical
screening, you want to find as many actual cases of a disease as you can.

## Formula for Detection Rate

The detection rate is calculated from the values in a confusion matrix.

Detection Rate (Recall)=True Positives (TP)+False Negatives (FN)True Positives (TP)

Where:

 True Positives (TP): The number of positive cases the model correctly identified (the tuna
you caught).

 False Negatives (FN): The number of positive cases the model missed (the tuna still in the
sea).

Logistic Regress:
Log(odds) = b0 + b1x1 + b2x2 + ….. + bqxq

P= odds/(1+odds)
Odd is the ratio of event occurrence to not occurence

Multi Nominal Logistic Regression:

Odd of person who is not employed and his point of

Z = (x – x’)/ỽ

Pseudo R^2

Null Deviance

Residual Deviance It should be as low as possible (N – k - m)

Overfitting

Regularization

Overfitting When model has good accuracy on training data

Regularization Method of reducing the overfitting

Data transformation using min max c score

Lower AIC is better model