Boosting and its three techniques
Introduction to Boosting
Boosting is an ensemble learning technique that improves the predictive power of weak
learners (models that perform slightly better than random guessing). It works by combining
multiple weak models iteratively, where each subsequent model corrects the mistakes of the
previous ones. The final strong model is a weighted sum of all weak learners, leading to
improved accuracy and generalization.
Boosting algorithms adjust weights assigned to misclassified samples, ensuring that difficult
cases receive more focus in subsequent iterations. Three popular boosting techniques are
AdaBoost (Adaptive Boosting), Gradient Boosting, and XGBoost (Extreme Gradient
Boosting).
Naive Bayes classifier
The Naive Bayes classifier is a simple but effective probabilistic learning algorithm based on
the Bayes theorem with strong independence assumptions among the features. Despite the
naive in its name, Naive Bayes has been proved a great classifier not only for text related
tasks like spam filtering, and sport events classification but also to predict medical diagnosis.
This article will give you an overview as well as more advanced use and implementation of
Naive Bayes in machine learning.
• Naïve Bayes is a classification algorithm for categorical variables, which is based on
the well-known Bayes theorem.
• Used mostly in high-dimensional text classification
• The Naïve Bayes Classifier is a simple probabilistic classifier and it has very few
number of parameters which are used to build the ML models that can predict at a
faster speed than other classification algorithms.
• It is a probabilistic classifier i.e., it predicts based on the likelihood of an object.
• Naïve Bayes Algorithm: It is used in spam filtration, Sentimental analysis, classifying
articles and many more.
Why is it called Naïve Bayes?
Naïve Bayes is a classification algorithm based on Bayes Theorem that assumes independent
predictors.
• Naïve: Named as Naïve because it assumes the presence of one feature does not
affect other features. For example if the fruit is classified on color shape and taste
basis, then red round tailing sweet like fruit identified as apple. Which is why every
single feature helps in identifying that it is an apple without relying on the other
feature.
• Bayes: Named Bayes for the basis in Bayes’ Theorem.
Bayes’ Theorem
Types of Naïve Bayes Model:
Three types of Naive Bayes Model are –
• Gaussian : The Gaussian naive Bayes assumes that features follow a normal
distribution. This assumes that the predictors X are continuous and have been
sampled from a Gaussian distribution.
• Multinomial : Multinomial Naïve Bayes classifier is used when the data has
multinomial distribution. It is basically used for document classification, like a
particular document which deals with what category like Sports, Politics or education
etc.
• Bernoulli : The Bernoulli classifier functions the same as the Multinomial classifier,
but its predictor variables are Booleans that refer to independent. For example if
some word exists in a document or not so. This model is also one of the most
popular for document classification tasks.
Practical Applications
• Email Filtering: In email filtering, we use the naive Bayes algorithm to classify emails
as spam or not.
• Text Classification: Document categorization, sentiment analysis, language detection
• Medical Diagnosis: helps to diagnose diseases according to patient’s symptoms and
past data.