Scribd
Upload
2 views1 page

Regularization

Regularization is a technique used in machine learning to prevent overfitting by adding a penalty to the model's error calculation based on the size of its parameters. This encourages simpler models that generalize better to new data. The two main types of regularization are L1 (Lasso), which can eliminate unimportant features, and L2 (Ridge), which reduces the influence of parameters without eliminating them.

Uploaded by

af.abdullah51
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views1 page

Regularization

Regularization is a technique used in machine learning to prevent overfitting by adding a penalty to the model's error calculation based on the size of its parameters. This encourages simpler models that generalize better to new data. The two main types of regularization are L1 (Lasso), which can eliminate unimportant features, and L2 (Ridge), which reduces the influence of parameters without eliminating them.

Uploaded by

af.abdullah51
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Regularization

A technique to prevent overfitting in machine learning models

What Is Overfitting?
Overfitting happens when a model learns the training data too closely, including its noise and random
quirks, instead of learning the true underlying pattern. Such a model performs very well on training data but
poorly on new, unseen data.

What Is Regularization?
Regularization is a technique that discourages a model from becoming overly complex. It works by adding
an extra penalty to the model's error calculation, based on the size of the model's parameters (the internal
values the model learns). This penalty forces the model to keep its parameters small, resulting in a simpler
model that is less likely to chase noise and more likely to generalize well to new data.

How It Works (Step by Step)


1. Normally, a model tries to minimize prediction error alone.
2. Regularization adds a second term: a penalty based on parameter size.
3. The model now must balance two goals: fitting the data well, and keeping parameters small.
4. This balance prevents the model from relying too heavily on any single feature.

Two Main Types

Type Penalty Based On Effect

L1 (Lasso) Absolute value of Can shrink some parameters to exactly zero


parameters — removes unimportant features
automatically

L2 (Ridge) Squared value of Shrinks all parameters smaller, but rarely to


parameters exactly zero — reduces influence without
elimination

Bottom Line
Regularization trades a small increase in training error for a large improvement in how well the model
generalizes to new, unseen data. It is one of the primary tools used to control overfitting in machine learning
models.

You might also like