UNIT 1

REGRESSION
Supervised Machine Learning:
It is an ML technique where models are trained on labeled data i.e output variable is
provided in these types of problems. Here, the models find the mapping function to map input
variables with the output variable or the labels.
Regression and Classification problems are a part of Supervised Machine Learning.
Regression: -
Regression Analysis in Machine learning Regression analysis is a statistical method to
model the relationship between a dependent (target) and independent (predictor) variables
with one or more independent variables. More specifically, Regression analysis helps us to
understand how the value of the dependent variable is changing corresponding to an
independent variable when other independent variables are held fixed. It predicts
continuous/real values such as temperature, age, salary, price, etc.
We can understand the concept of regression analysis using the below example:
Example: Suppose there is a marketing company A, who does various advertisement every year
and get sales on that. The below list shows the advertisement made by the company in the last
5 years and the corresponding sales:
 Regression models "learn" by finding the best possible line/curve that minimizes the
error between the predicted values and the actual values in your training data.
 Error (or "Residual"): The difference between the actual value (yactual) and the
predicted value (ypredicted).
 Cost Function (or "Loss Function"): A mathematical way to measure the total error of
our model. Our goal is to make this number as small as possible.
o Common Cost Function for Regression: Mean Squared Error (MSE)
 MSE=N1∑i=1N(yactual,i−ypredicted,i)2

Steps in Building a Regression Model:

1. Collect Data: Gather your input features and the corresponding output values.
2. Data Preprocessing:
o Handle missing values.
o Deal with outliers.
o Feature Scaling (important for some algorithms): Making sure all features are on
a similar scale (e.g., all between 0 and 1).
3. Split Data: Divide your data into:
o Training Set: Used to train the model (let it learn the patterns).
 o Testing Set: Used to evaluate how well the trained model performs on new,
unseen data. This is crucial to avoid overfitting (where the model memorizes the
training data but can't generalize).
4. Choose a Regression Model: Linear Regression, Polynomial Regression, etc.
5. Train the Model: The model learns the relationships from the training data.
6. Evaluate the Model: Use metrics to see how well your model performs on the testing
set.
Evaluation Metrics (How Good is Our Prediction?):
 Mean Absolute Error (MAE): Average of the absolute differences between actual and
predicted values.
o MAE=N1∑i=1N∣yactual,i−ypredicted,i∣
o Easier to interpret as it's in the same units as the output.
 Mean Squared Error (MSE): We talked about this! Average of the squared differences.
o MSE=N1∑i=1N(yactual,i−ypredicted,i)2
o Penalizes larger errors more.
 Root Mean Squared Error (RMSE): The square root of MSE.

o RMSE=MSE
o Also in the same units as the output, making it more interpretable than MSE
while still penalizing large errors.
 The formula for a simple linear regression line is: Ypredicted=mX+b
Where:
 Ypredicted is the value of Y our line predicts for a given X.
 m is the slope of the line.
 X is the input variable.
 b is the Y-intercept (the value of Y when X is 0).
The Formulas for 'm' and 'b' (OLS Estimators)
To find the values of m and b that achieve this minimization, we use these formulas:
1. Calculate the Slope (m):
m=n∑(X2)−(∑(X))2n∑(XY)−∑(X)∑(Y)
Let's break down what each part means:
 n: The total number of data points (pairs of X and Y values).
 ∑(XY): Sum of the products of each X value multiplied by its corresponding Y value.
 ∑(X): Sum of all X values.
 ∑(Y): Sum of all Y values.
 ∑(X2): Sum of the squares of each X value.
 (∑(X))2: The square of the sum of all X values (calculate ∑(X) first, then square the result).
2. Calculate the Y-intercept (b):
b=Yˉ−mXˉ
Where:
 Yˉ (pronounced "Y-bar"): The mean (average) of all Y values (Yˉ=n∑(Y)).
 Xˉ (pronounced "X-bar"): The mean (average) of all X values (Xˉ=n∑(X)).
 m: The slope you just calculated using the first formula.

Step-by-Step Example
Let's use the "Hours Studied vs. Exam Score" example again:
Student Hours Studied (X) Exam Score (Y)
1 2 50
2 3 60
3 5 75
4 6 80
5 7 90
Export to Sheets
Step 1: Create a table to calculate the necessary sums.
Student X (Hours) Y (Score) XY X^2
1 2 50 100 4
2 3 60 180 9
 Student X (Hours) Y (Score) XY X^2
3 5 75 375 25
4 6 80 480 36
5 7 90 630 49
Sum ∑(X)=23 ∑(Y)=355 ∑(XY)=1765 ∑(X2)=123
Export to Sheets
From the table, we also know:
 n=5 (number of data points)
Step 2: Calculate the Slope (m)
m=n∑(X2)−(∑(X))2n∑(XY)−∑(X)∑(Y) m=5×123−(23)25×1765−23×355
m=615−5298825−8165 m=86660 m≈7.674
So, the slope m is approximately 7.674. This means for every additional hour studied,
the exam score is predicted to increase by about 7.674 points.
Step 3: Calculate the Y-intercept (b)
First, find the means: Xˉ=n∑(X)=523=4.6 Yˉ=n∑(Y)=5355=71
Now, use the formula for b: b=Yˉ−mXˉ b=71−7.674×4.6 b=71−35.3004 b≈35.6996
So, the Y-intercept b is approximately 35.70. This means if a student studies 0 hours,
their predicted exam score is about 35.70.
Step 4: Write the Regression Equation
Now you have your m and b values, so your simple linear regression equation is:
Ypredicted=7.674X+35.70
The formula for a simple linear regression line is: Ypredicted=mX+b
Where:
 Ypredicted is the value of Y our line predicts for a given X.
 m is the slope of the line.
 X is the input variable.
 b is the Y-intercept (the value of Y when X is 0).
The Formulas for 'm' and 'b' (OLS Estimators)
To find the values of m and b that achieve this minimization, we use these formulas:
1. Calculate the Slope (m):
m=n∑(X2)−(∑(X))2n∑(XY)−∑(X)∑(Y)
Let's break down what each part means:
 n: The total number of data points (pairs of X and Y values).
 ∑(XY): Sum of the products of each X value multiplied by its corresponding Y value.
 ∑(X): Sum of all X values.
 ∑(Y): Sum of all Y values.
 ∑(X2): Sum of the squares of each X value.
 (∑(X))2: The square of the sum of all X values (calculate ∑(X) first, then square the result).
 2. Calculate the Y-intercept (b):
b=Yˉ−mXˉ
Where:
 Yˉ (pronounced "Y-bar"): The mean (average) of all Y values (Yˉ=n∑(Y)).
 Xˉ (pronounced "X-bar"): The mean (average) of all X values (Xˉ=n∑(X)).
 m: The slope you just calculated using the first formula.

Step-by-Step Example
Let's use the "Hours Studied vs. Exam Score" example again:
Student Hours Studied (X) Exam Score (Y)
1 2 50
2 3 60
3 5 75
4 6 80
5 7 90
Export to Sheets
Step 1: Create a table to calculate the necessary sums.
Student X (Hours) Y (Score) XY X^2
1 2 50 100 4
2 3 60 180 9
3 5 75 375 25
4 6 80 480 36
5 7 90 630 49
Sum ∑(X)=23 ∑(Y)=355 ∑(XY)=1765 ∑(X2)=123
Export to Sheets
From the table, we also know:
 n=5 (number of data points)
Step 2: Calculate the Slope (m)
m=n∑(X2)−(∑(X))2n∑(XY)−∑(X)∑(Y) m=5×123−(23)25×1765−23×355
m=615−5298825−8165 m=86660 m≈7.674
So, the slope m is approximately 7.674. This means for every additional hour studied,
the exam score is predicted to increase by about 7.674 points.
Step 3: Calculate the Y-intercept (b)
First, find the means: Xˉ=n∑(X)=523=4.6 Yˉ=n∑(Y)=5355=71
Now, use the formula for b: b=Yˉ−mXˉ b=71−7.674×4.6 b=71−35.3004 b≈35.6996
So, the Y-intercept b is approximately 35.70. This means if a student studies 0 hours,
their predicted exam score is about 35.70.
 Step 4: Write the Regression Equation
Now you have your m and b values, so your simple linear regression equation is:
Ypredicted=7.674X+35.70