Introduction to Machine Learning

Machine Learning is a technique that allows computers to learn from data and
make decisions without explicit programming. It works by identifying patterns in
data and using them to make predictions. It is used in areas such as:
• Image Recognition
• Speech Processing
• Language Translation
• Recommender Systems

Need for Machine Learning

Machine Learning is important because traditional programming cannot handle
complex tasks or large amounts of data efficiently. ML overcomes this by learning
from data and making predictions without fixed rules. It is needed for the
following reasons:
1. Solving Complex Business Problems
Traditional programming struggles with tasks like language understanding and
medical diagnosis. ML learns from data and predicts outcomes easily.
Examples:
• Image and speech recognition in healthcare.
• Language translation and sentiment analysis.
2. Handling Large Volumes of Data
The internet generates huge amounts of data every day. Machine Learning
processes and analyzes this data quickly by providing valuable insights and real
time predictions.
Examples:
• Fraud detection in financial transactions.
• Personalized feed recommendations on Facebook and Instagram from
billions of interactions.
3. Automate Repetitive Tasks
ML automates time consuming, repetitive tasks with high accuracy hence reducing
manual work and errors.
Examples:
• Gmail filtering spam emails automatically.
• Chatbots handling order tracking and password resets.
• Automating large scale invoice analysis for key insights.
4. Personalized User Experience
ML enhances user experience by tailoring recommendations to individual
preferences. It analyze user behavior to deliver highly relevant content.
Examples:
• Netflix suggesting movies and TV shows based on our viewing history.
• E-commerce sites recommending products we're likely to buy.
5. Self Improvement in Performance
ML models evolve and improve with more data helps in making them smarter over
time. They adapt to user behavior and increase their performance.
Examples:
• Voice assistants like Siri and Alexa learning our preferences and accents.
• Search engines refining results based on user interaction.
• Self driving cars improving decisions using millions of miles of driving data.
How Machines Learn from Data
A machine learns by finding patterns in data and improving over time without
explicit programming. It adapts with experience to make more accurate
predictions. This learning happens through the following steps:
1. Data Input: Machine needs data like text, images or numbers to analyze.
Good quality and enough quantity of data are important for effective
learning.
 2. Algorithms: Algorithms are mathematical methods that help the machine
find patterns in data. Different algorithms help different tasks such as
classification or regression.
3. Model Training: During training, the machine adjusts its internal settings to
better predict outcomes. It learns by reducing the difference between its
predictions and actual results.
4. Feedback Loop: Machine compares its predictions with true outcomes and
uses this feedback to correct errors. Techniques like gradient descent help it
update and improve.
5. Experience and Iteration: Machine repeats training many times with data
helps in refining its predictions with each pass, more data and iterations
improve accuracy.
6. Evaluation and Generalization: Tested on new data to ensure real world
performance
Data is the foundation of machine learning because models learn patterns and
make predictions from it. Good quality and diverse data help improve accuracy,
performance and real-world results.
Types of Machine Learning
There are mainly three types of machine learning which are as follows:
• Supervised Learning: Learns from labeled data where correct outputs are
already known to make predictions or classifications.
• Unsupervised Learning: Learns from unlabeled data by finding hidden
patterns, similarities, or groups automatically.
• Reinforcement Learning: Learns through trial and error by receiving
rewards for correct actions and penalties for wrong ones.
To know more about types refer to: Types of Machine Learning
Benefits of Machine Learning
Machine Learning improves processes by automating tasks and extracting insights
from data, making systems smarter and more efficient.
• Automates repetitive tasks and improves productivity
 • Finds patterns in large data for better decisions
• Provides customized recommendations and experiences
• Enables robots and systems to perform complex tasks accurately
Challenges
• ML models learn from training data and if the data is biased, model’s
decisions can be unfair so it’s important to select and monitor data carefully.
• Since it depends on large amounts of data, there is a risk of sensitive
information being exposed so protecting privacy is important.
• Complex ML models can be difficult to understand which makes it difficult
to explain why they make certain decisions. This can affect trust and
accountability.
• Automation may replace some jobs so retraining and helping workers learn
new skills is important to adapt to these changes.
Applications
• In healthcare it diagnoses diseases, predicts outcomes and personalizes
treatments
• In finance detects fraud, supports trading and assesses credit risk
• Recommends products, forecasts demand and analyzes customer behavior
• Powers self-driving cars, optimizes routes and predicts maintenance
• Recommends content and enables image/speech recognition
• Detects defects and predicts machine failures
Chapter 1:
1.1 Limitations of basic regression and classification models Regularization
Techniques:
Regularization in Machine Learning
Regularization is a technique used in machine learning to prevent overfitting,
which otherwise causes models to perform poorly on unseen data. By adding a
penalty for complexity, regularization encourages simpler and more generalizable
models.
 • Prevents overfitting: Adds constraints to the model to reduce the risk of
memorizing noise in the training data.
• Improves generalization: Encourages simpler models that perform better
on new, unseen data.

Types of Regularization
There are mainly 3 types of regularization techniques, each applying penalties in
different ways to control model complexity and improve generalization.
1. Lasso Regression
A regression model which uses the L1 Regularization technique is called LASSO
(Least Absolute Shrinkage and Selection Operator) regression. It adds the absolute
value of magnitude of the coefficient as a penalty term to the loss function(L). This
penalty can shrink some coefficients to zero which helps in selecting only the
important features and ignoring the less important ones.
Cost=1n∑i=1n(yi−yi^)2+λ∑j=1m∣wj∣Cost=n1∑i=1n(yi−yi^)2+λ∑j=1m∣wj∣
Where
• mm: Number of Features
• nn: Number of Examples
• yiyi: Actual Target Value
• y^iy^i: Predicted Target Value
Note: These formulas apply to linear models. In neural networks, the number of
weights is much larger than the number of features, but the same regularization
principles (L1, L2) still apply on all weights.
Lets see how to implement this using python:
• X, y = make_regression(n_samples=100, n_features=5, noise=0.1,
random_state=42): Generates a regression dataset with 100 samples, 5
features and some noise.
• X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42): Splits the data into 80% training and 20% testing sets.
• lasso = Lasso(alpha=0.1): Creates a Lasso regression model with
regularization strength alpha set to 0.1.
from sklearn.linear_model import Lasso
from sklearn.model_selection import train_test_split
from [Link] import make_regression
from [Link] import mean_squared_error

X, y = make_regression(n_samples=100, n_features=5, noise=0.1,

random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

lasso = Lasso(alpha=0.1)
[Link](X_train, y_train)

y_pred = [Link](X_test)

mse = mean_squared_error(y_test, y_pred)

print(f"Mean Squared Error: {mse}")
print("Coefficients:", lasso.coef_)
Output:

Lasso Regression
The output shows the model's prediction error and the importance of features with
some coefficients reduced to zero due to L1 regularization.
2. Ridge Regression
A regression model that uses the L2 regularization technique is called Ridge
regression. It adds the squared magnitude of the coefficient as a penalty term to the
loss function(L). It handles multicollinearity by shrinking the coefficients of
correlated features, reducing their variance and preventing any single feature from
dominating the model.
Cost=1n∑i=1n(yi−y^i)2+λ∑j=1mwj2Cost=n1∑i=1n(yi−y^i)2+λ∑j=1mwj2
Where,
• nn: Number of examples or data points
• mm: Number of features i.e predictor variables
• yiyi: Actual target value for the ithith example
• y^iy^i: Predicted target value for the ithith example
• wiwi: Coefficients of the features
• λλ: Regularization parameter that controls the strength of regularization
Lets see how to implement this using python:
• ridge = Ridge(alpha=1.0): Creates a Ridge regression model with
regularization strength alpha set to 1.0.
from sklearn.linear_model import Ridge
from [Link] import make_regression
from sklearn.model_selection import train_test_split
from [Link] import mean_squared_error

X, y = make_regression(n_samples=100, n_features=5, noise=0.1,

random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

ridge = Ridge(alpha=1.0)
[Link](X_train, y_train)
y_pred = [Link](X_test)

mse = mean_squared_error(y_test, y_pred)

print("Mean Squared Error:", mse)
print("Coefficients:", ridge.coef_)
Output:

Ridge Regression
The output shows the MSE showing model performance. Lower MSE means better
accuracy. The coefficients reflect the regularized feature weights.
3. Elastic Net Regression
Elastic Net Regression is a combination of both L1 as well as L2 regularization. It
combines both L1 (absolute values) and L2 (squared values) penalties on the
coefficients. With the help of an extra hyperparameter that controls the ratio of the
L1 and L2 regularization.
Cost=1n∑i=1n(yi−y^i)2+λ((1−α)∑j=1m∣wj∣+α∑j=1mwj2)Cost=n1∑i=1n(yi−y^i
)2+λ((1−α)∑j=1m∣wj∣+α∑j=1mwj2)
Where
 • nn: Number of examples (data points)
• mm: Number of features (predictor variables)
• yiyi: Actual target value for the ithith example
• y^iy^i: Predicted target value for the ithith example
• wiwi: Coefficients of the features
• λλ: Regularization parameter that controls the strength of regularization
• αα: Mixing parameter where 0≤α≤10≤α≤1 and αα= 1 corresponds to Lasso
(L1L1) regularization, αα= 0 corresponds to Ridge (L2L2) regularization and
Values between 0 and 1 provide a balance of both L1 and L2 regularization
Lets see how to implement this using python:
• model = ElasticNet(alpha=1.0, l1_ratio=0.5) : Creates an Elastic Net
model with regularization strength alpha=1.0 and L1/L2 mixing ratio 0.5.
from sklearn.linear_model import ElasticNet
from [Link] import make_regression
from sklearn.model_selection import train_test_split
from [Link] import mean_squared_error

X, y = make_regression(n_samples=100, n_features=10, noise=0.1,

random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

model = ElasticNet(alpha=1.0, l1_ratio=0.5)

[Link](X_train, y_train)

y_pred = [Link](X_test)
mse = mean_squared_error(y_test, y_pred)
print("Mean Squared Error:", mse)
print("Coefficients:", model.coef_)
Output:

Elastic Net Regression

The output shows MSE which measures how far off predictions are from actual
values (lower is better) and coefficients show feature importance.
Benefits of Regularization
Now, let’s see various benefits of regularization which are as follows:
• Prevents Overfitting: Regularization helps models focus on underlying
patterns instead of memorizing noise in the training data.
• Enhances Performance: Prevents excessive weighting of outliers or
irrelevant features helps in improving overall model accuracy.
• Stabilizes Models: Reduces sensitivity to minor data changes which ensures
• consistency across different data subsets.
• Prevents Complexity: Keeps model from becoming too complex which is
important for limited or noisy data.
• Handles Multicollinearity: Reduces the magnitudes of correlated
coefficients helps in improving model stability.
• Promotes Consistency: Ensures reliable performance across different
datasets which reduces the risk of large performance shifts.

Lasso vs Ridge vs Elastic Net - ML

Regularization methods like Lasso, Ridge and Elastic Net help improve linear
regression models by preventing overfitting which address multicollinearity and
helps in feature selection. These techniques increase the model’s accuracy and
stability. In this article we will see explanation of how each technique works and
their differences.
Ridge Regression (L2 Regularization)
Ridge regression is a technique used to address overfitting by adding a penalty to
the model's complexity. It introduces an L2 penalty (also called L2 regularization)
which is the sum of the squares of the model's coefficients. This penalty term
reduces the size of large coefficients but keeps all features in the model. This
prevents overfitting with correlated features.
Formula for Ridge Regression:

where:
• The first term calculates the prediction error.
• The second term penalizes large coefficients controlled by λλ.
Example: Let’s assume we are predicting house prices with features like size,
location and number of rooms. The model might give coefficients like:
• β1β1= 5 (Size coefficient)
• β2β2= 3 (Number of rooms coefficient)
• λλ= 0.1 (regularization strength).
The penalty term for Ridge would be calculated as:

This penalty shrinks the coefficients to reduce overfitting but does not remove any
features.
Lasso Regression (L1 Regularization)
Lasso regression addresses overfitting by adding an L1 penalty i.e sum of absolute
coefficients to the model's loss function. This encourages some coefficients to
become exactly zero helps in effectively removing less important features. It also
helps to simplify the model by selecting only the key features.
Formula for Lasso Regression:

where:
• The first term calculates the prediction error.
• The second term encourages sparsity by shrinking some coefficients to zero.
Example: Let’s assume the same house price prediction example but now using
Lasso. Assume:
• β1β1= 5 (Size coefficient)
• β2β2= 0 (Number of rooms coefficient is irrelevant and should be removed)
• λλ= 0.1 (regularization strength).
The penalty term for Lasso would be:
λ⋅∣β1∣=0.1⋅∣5∣=0.1⋅5=0.5λ⋅∣β1∣=0.1⋅∣5∣=0.1⋅5=0.5
Here Lasso forces β2β2= 0 removing the Number of Rooms feature entirely from
the model.
Elastic Net Regression (L1 + L2 Regularization)
Elastic Net regression combines both L1 (Lasso) and L2 (Ridge) penalties to
perform feature selection, manage multicollinearity and balancing coefficient
shrinkage. This works well when there are many correlated features helps in
avoiding the problem where Lasso might randomly pick one and ignore others.
Formula for Elastic Net Regression:

where:
• The first term calculates the prediction error.
• The second term applies the L1 penalty for feature selection.
 • The third term applies the L2 penalty to handle multicollinearity.
It provides a more stable and generalizable model compared to using Lasso or
Ridge alone.
Example: Let’s assume we are predicting house prices using Size and Number of
Rooms. Assume:
• β1β1= 5 (Size coefficient)
• β2β2= 3 (Number of rooms coefficient)
• λ1λ1= 0.1 (L1 regularization).
• λ2λ2= 0.1 (L2 regularization).
The penalty term for Elastic Net would be:

This penalty shrinks both coefficients but because of the mixture of L1 and L2 it
does not force any feature to zero unless absolutely necessary.

Lasso vs Ridge vs Elastic Net

Now lets see a tabular comparison between these three for better understanding.

Ridge Elastic Net

Features Lasso Regression Regression Regression

L1 + L2 Penalty:
L1 Penalty: Lasso L2 Penalty:
Elastic Net uses
uses the absolute Ridge uses the
both absolute and
values of square of the
square penalties
coefficients. coefficients.
Penalty Type together.
 Ridge Elastic Net
Features Lasso Regression Regression Regression

It completely
It makes all
removes It removes some
coefficients
unnecessary features and
smaller but
features by setting reduces others by
doesn’t set them
Effect on their coefficients balancing both.
to zero.
Coefficients to zero.

It is good when It is best for when

It is best when we all features we features are
want to remove matter but we correlated and
irrelevant features want to reduce feature selection
Best Use Case their impact is needed

Alpha + L1_ratio:
Two parameters.
Alpha: Controls Alpha: Similar to
Alpha controls
how much Lasso which
regularization
regularization is helps in
strength and
applied. A higher controlling the
L1_ratio adjusts
alpha means more strength of
the balance
shrinkage. regularization.
Hyperparameters between Lasso
involved and Ridge.

High bias, low Low bias, high Balance of bias

Bias and Variance variance variance and variance

It is great for It works well It combines

automatically when features are Lasso’s feature
Strengths
choosing related but selection and
 Ridge Elastic Net
Features Lasso Regression Regression Regression

important shouldn’t be Ridge’s handling

features. completely of correlations.
removed.

It keeps all
features which
It can sometimes
may not help in It is a bit harder to
remove useful
high-dimensional tune due to having
features if not
data with two parameters.
tuned properly.
irrelevant
Weaknesses features.

Imagine we have
If we have 100 If we have
100 features to
features it will features like
predict house
reduce the impact “size” and
prices. It will set
of every feature “rooms” that are
the coefficients of
but won’t similar it will
irrelevant features
completely remove one and
(like house color)
remove any. shrink the other.
Example to zero.