Scribd
Upload
3 views1 page

5 Overfitting Regularization

The document discusses underfitting and overfitting in machine learning, highlighting that underfitting occurs when a model is too simple, while overfitting happens when a model learns noise instead of patterns. It emphasizes the bias-variance tradeoff and the importance of finding a balance for better generalization. Common strategies to prevent overfitting include regularization, dropout, early stopping, increasing training data, and simplifying models.

Uploaded by

sanskruti
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views1 page

5 Overfitting Regularization

The document discusses underfitting and overfitting in machine learning, highlighting that underfitting occurs when a model is too simple, while overfitting happens when a model learns noise instead of patterns. It emphasizes the bias-variance tradeoff and the importance of finding a balance for better generalization. Common strategies to prevent overfitting include regularization, dropout, early stopping, increasing training data, and simplifying models.

Uploaded by

sanskruti
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Overfitting, Underfitting & Regularization

Balancing model complexity to generalize well

Underfitting
Underfitting occurs when a model is too simple to capture the underlying pattern in the data. It performs poorly on both
training and test data. This often happens when the model has too few parameters, insufficient training, or is a poor fit
for the problem's complexity.

Overfitting
Overfitting occurs when a model learns the training data too well — including its noise and random quirks — rather than
the true underlying pattern. It performs very well on training data but poorly on new, unseen data because it has
essentially 'memorized' rather than 'generalized.'

The Bias-Variance Tradeoff


Underfitting is associated with high bias (overly simplistic assumptions), while overfitting is associated with high
variance (overly sensitive to training data fluctuations). The goal of model building is to find a sweet spot that minimizes
both, producing a model that generalizes well to new data.

Common Ways to Prevent Overfitting


• Regularization (L1/L2): adds a penalty for overly large model weights, discouraging complexity.
• Dropout: randomly disables neurons during training in neural networks.
• Early stopping: halts training once performance on a validation set stops improving.
• More training data: helps the model learn genuine patterns instead of noise.
• Simpler models or feature selection: reduces unnecessary complexity.

Practical Signal to Watch


A large gap between training performance and validation/test performance is the classic warning sign of overfitting.
Tracking both curves during training is one of the simplest and most effective diagnostic habits in machine learning.

You might also like