Machine Learning
Working with Imbalance Data :
Lecture By Krish Naik:
Install library : pip install imbalance Learn
precision (exactness) and recall (completeness)
We explored 5 different methods for dealing with
imbalanced datasets:
1. Change the performance metric
2. Change the algorithm
3. Oversample minority class
4. Undersample majority class
5. Generate synthetic samples
It is always good to do oversampling/up sampling than
downsampling because we may lose important data.
It appears for this particular dataset random forest and
SMOTE are among the best of the options we tried here.
These are just some of the many possible methods to try
when dealing with imbalanced datasets, and not an
exhaustive list. Some others methods to consider are
collecting more data or choosing different resampling
ratios — you don’t have to have exactly a 1:1 ratio!
You should always try several approaches and then
decide which is best for your problem.
Extrapolation is a technique to create data Artificial/synthetic way to handle imbalance data.
If we are working with imbalanced data then accuracy is not best metric to measure, we should
consider other metrics.
Precision : TP / (TP+ FP)
Recall/ Sensitivity : TP/(TP+FN)
Specificity : TN/ (TN+FP)
F score: 2* ((precision*recall)/(precision+recall))
Ada Boost: It is a forest of stumps rather than trees. Stumps are not much great at accurate
classification.
Stump: A tree with just one node and two leaves is called a stump. Stumps can take only one variable to
make decision, Thus, Stumps are technically weak Learners.
Three ideas behind Adaboost are :
1) Ada boost combines a lot of weak learners to make classification. The weak learners are almost
always stumps.
2) Some stumps get more say in the classification than others.
3) Each stump is made by taking previous stumps mistakes into account.
Identifying outlier in Data:
An outlier is an observation that is unlike the other observation.
It is rare or distinct or doesn’t fit in some way.
Outliers can have many causes, such as:
1. Measurement or input error.
2. Data corruption.
3. Ture outlier observation.
Standard Normal Distribution:
It is a special case of normal distribution, where the mean is 0 and standard deviation is 1.
Feature Selection:
Q) What is Feature selection and its importance?
Q) Different types of feature selections techniques?
1) Chi square test:
A chi-square (χ2) statistic is a test that measures how expectations compare to
actual observed data (or model results). It’s a non parametric test with ordinal or
nominal data
There are two types
i) Chi-square goodness of fit test.
ii) Chi-square test of independence.
Chi-square goodness of fit test :
If you think that data might have come from a binomial distribution, or a Poisson
distribution, or a normal distribution, for example, goodness of fit test whether the
distribution really does come from that distribution. You can never prove that the
distribution is what you think it might be, but rejecting the hypothesis, shows that it
isn’t.( the goodness-of-fit test uses Chi-Square to see if some empirical distribution
matches some hypothesized distribution such as a normal distribution)
The test is applied when you have one categorical variable from a single
population.
When to Use the Chi-Square Goodness of Fit Test
The chi-square goodness of fit test is appropriate when the following conditions are met:
The sampling method is simple random sampling.
The variable under study is categorical.
The expected value of the number of sample observations in each level of the
variable is at least 5.
Example : [Link]
Q) chi square goodness of fit test for
Chi-square test of independenc: (Categorical correlation)
(website : [Link]
The Chi-Square test of independence is used to determine if there is a significant
relationship between two nominal (categorical) variables. The frequency of each category
for one nominal variable is compared across the categories of the second nominal variable.
The data can be displayed in a contingency table where each row represents a category
for one variable and each column represents a category for the other variable.
The test is applied when you have two categorical variables from a single
population.
A very small chi square test statistic means that your observed data fits your
expected data extremely well. In other words, there is a relationship.
A very large chi square test statistic means that the data does not fit very well. In
other words, there isn’t a relationship.
= Chi-Square test of Independence
= Observed value of two nominal variables
= Expected value of two nominal variables
When to Use Chi-Square Test for Independence
The test procedure described in this lesson is appropriate when the following conditions are
met:
The sampling method is simple random sampling.
The variables under study are each categorical.
If sample data are displayed in a contingency table, the expected frequency count for
each cell of the table is at least 5.
[Link]
206b1f0b8223
Categorical Feature Selection
There are two popular feature selection techniques that can be used for categorical input
data and a categorical (class) target variable.
They are:
Chi-Squared Statistic.
Mutual Information Statistic.
Calculate Chi-Square value
Summarizing the observed values and calculated
expected values into a table and determine the Chi-
Square value.
We can see Chi-Square is calculated as 2.22 by using the
Chi-Square statistic formula.
5. Accept or Reject the Null Hypothesis
With 95% confidence that is alpha = 0.05, we will check
the calculated Chi-Square value falls in the acceptance or
rejection region.
Having degrees of freedom =1(calculated with
contingency table) and alpha =0.05 the Chi-Square value
is 3.84.
The Chi-Square values can be determined with the Chi-
Square table.
The chi-square distribution is the right side since the
difference in Observed and Expected is large.
In the above fig, we can see Chi-Square ranges from 0 to
inf and alpha ranges from 0 to 1 in the opposite
direction. We will reject the Null hypothesis if Chi-Square
value falls in the error region (alpha from 0 to 0.05 ).
So here we are accepting the null hypothesis since the
Chi-Square value is less than the critical Chi-Square
value.
To conclude the two variables are independent, Gender
variable cannot be selected for training the model.
Difference between Filter, Wrapper and
Embedded methods
Filter vs. Wrapper vs. Embedded methods
Forward selection :
In forward selection, we start with a null model and then
start fitting the model with each individual feature one at
a time and select the feature with the minimum p-value.
Now fit a model with two features by trying combinations
of the earlier selected feature with all other remaining
features. Again select the feature with the minimum p-
value. Now fit a model with three features by trying
combinations of two previously selected features with
other remaining features. Repeat this process until we
have a set of selected features with a p-value of
individual feature less than the significance level.
In short, the steps for forward selection technique are
as follows :
1. Choose a significance level (e.g. SL = 0.05 with a 95%
confidence).
2. Fit all possible simple regression models by
considering one feature at a time. Total ’n’ models are
possible. Select the feature with the lowest p-value.
3. Fit all possible models with one extra feature added to
the previously selected feature(s).
4. Again, select the feature with minimum p-value.
if p_value < significance level then go to Step 3,
otherwise terminate the process.
ML process:
1) Define objective : what we want to predict.
2) Collect data
3) Prepare Data : make to clean your data, bad data in
bad data out.
Numpy - Cleaning
Pandas – Data manipulation, Tranform data,
Matplotlib, seaborn - analyze data.
4) Select Algorithm.
Scikit Learn
5) Train model
6) Test model
7) Predict
8) Deploy.