Module 3: Regression
Algorithms
1. Linear Regression
Definition:
Linear Regression is a supervised learning algorithm that models the relationship
between a dependent variable (target) and one or more independent variables
(features) using a straight line.
Key Points:
• Formula: y=b0+b1x
o y: Dependent variable
o x: Independent variable
o b0: Intercept
o b1: Coefficient (slope)
• Objective: Minimize the residual sum of squares to find the best-fit line.
• Applications:
o Predicting house prices
o Forecasting sales
o Stock market trends
Steps to Implement:
1. Load and preprocess the data.
2. Fit a linear model to the data.
3. Predict target values using the model.
Code Implementation:
# Import libraries
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error, r2_score
# Load dataset
# Example: Using a dummy dataset
data = {'Hours': [1, 2, 3, 4, 5], 'Scores': [20, 40, 60, 80, 100]}
df = [Link](data)
# Split dataset
X = df[['Hours']] # Independent variable
y = df['Scores'] # Dependent variable
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
# Train model
model = LinearRegression()
[Link](X_train, y_train)
# Predict
y_pred = [Link](X_test)
# Evaluate
print("Mean Squared Error:", mean_squared_error(y_test, y_pred))
print("R-squared:", r2_score(y_test, y_pred))
2. Logistic Regression
Definition:
Logistic Regression is a classification algorithm used to predict the probability of a
target variable belonging to a certain class.
Key Points:
Formula:
The output is a probability, mapped to [0,1][0, 1] using the sigmoid function.
• Objective: Find the best-fit logistic function to classify data points.
• Applications:
o Spam email detection
o Loan default prediction
o Disease diagnosis
Steps to Implement:
1. Load and preprocess the data.
2. Fit a logistic regression model to the data.
3. Predict probabilities and classify based on a threshold.
Code Implementation:
# Import libraries
from [Link] import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score, confusion_matrix
# Load dataset
iris = load_iris()
X = [Link][:, :2] # Taking only the first two features for simplicity
y = ([Link] == 0).astype(int) # Binary classification: Is the species
Setosa?
# Split dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
# Train model
log_model = LogisticRegression()
log_model.fit(X_train, y_train)
# Predict
y_pred = log_model.predict(X_test)
# Evaluate
print("Accuracy:", accuracy_score(y_test, y_pred))
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))
3. Evaluation Metrics of Regression Algorithms
Definition:
Evaluation metrics are used to measure the performance of regression models by
comparing predicted values with actual values.
Key Metrics:
1. Mean Absolute Error (MAE):
o Formula:
oInterpretation: Measures the average absolute error.
2. Mean Squared Error (MSE):
o Formula:
o Interpretation: Penalizes larger errors more than smaller ones.
3. Root Mean Squared Error (RMSE):
o Formula:
o Interpretation: Gives the error in the same unit as the target variable.
4. R-squared (R²):
o Formula:
Interpretation: Measures the proportion of variance explained by the
model.
Code Implementation:
# Example: Evaluating a regression model
from [Link] import mean_absolute_error, mean_squared_error
# Sample actual and predicted values
y_actual = [3, -0.5, 2, 7]
y_pred = [2.5, 0.0, 2, 8]
# Calculate metrics
mae = mean_absolute_error(y_actual, y_pred)
mse = mean_squared_error(y_actual, y_pred)
rmse = [Link](mse)
# Print results
print("Mean Absolute Error:", mae)
print("Mean Squared Error:", mse)
print("Root Mean Squared Error:", rmse)
Summary Table
Algorithm Key Use Case Output Evaluation Metrics
Linear Predict continuous MSE, RMSE, R²,
Best-fit line
Regression values Adjusted R²
Logistic Classify binary/multi- Accuracy, Confusion
Probability/Classification
Regression class Matrix