Scribd
Upload
5 views15 pages

Ensemble Methods Lecture Notes

The lecture notes discuss ensemble methods aimed at improving predictive performance by combining multiple models to reduce variance and enhance stability. They highlight the benefits of parallel training, flexibility in introducing randomness, and the low risk of overfitting due to the use of weak base models. Overall, ensemble methods provide a robust approach to model building in predictive analytics.

Uploaded by

Daksh Bhardwaj
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views15 pages

Ensemble Methods Lecture Notes

The lecture notes discuss ensemble methods aimed at improving predictive performance by combining multiple models to reduce variance and enhance stability. They highlight the benefits of parallel training, flexibility in introducing randomness, and the low risk of overfitting due to the use of weak base models. Overall, ensemble methods provide a robust approach to model building in predictive analytics.

Uploaded by

Daksh Bhardwaj
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Week 7 lecture notes

Lecture Notes: Ensemble Methods


1. Motivation for Ensembles

 Goal: Improve predictive performance by combining multiple models rather than


relying on a single classifier/regressor.
 Statistical benefit: Primarily reduces variance, leading to more stable estimates and
often better empirical accuracy.
 Typical effect: Unstable (high-variance) base learners become robust when
ensembled.
 Implementation notes:

 Embarrassingly parallel: Each can be trained independently, ideal for distributed


systems.
 Flexibility: Can introduce extra randomness—e.g., different random initializations,
feature subsets per bag—to increase diversity.
 Overfitting immunity: Since each base model is weak, adding more bags rarely leads
to overfitting.

You might also like