CS-402: ADVANCED MACHINE LEARNING
Lecture 07: Linear vs Logistic Regression Foundations
Semester: Fall 2026 | Instructor: Prof. Malhotra
1. Linear Regression (Continuous Target Variables)
Used when we want to predict a continuous numeric output based on independent input variables. The model
assumes a linear relationship between the inputs and output.
• Mathematical Form: y = β0 + β1x1 + β2x2 + ... + ε
• Cost Function: Mean Squared Error (MSE), which measures the average squared difference between actual
and predicted values.
• Optimization: Gradient Descent algorithm is used to minimize the cost function by iteratively adjusting
weights (β).
EXAM TIP / CORE ASSUMPTION
Linear regression strictly assumes that there is no multicollinearity among independent variables, and the
residuals are normally distributed! Keep this in mind for the mid-term quiz.
2. Logistic Regression (Categorical Target Variables)
Despite the name "regression," this is primarily a classification algorithm used to predict the probability of binary
outcomes (0 or 1, True or False).
• Sigmoid Function Activation: It maps any real-valued number into a probability range between 0 and 1.
# Standard Sigmoid Activation Formula Implementation
import numpy as np
def sigmoid(z):
return 1 / (1 + [Link](-z))
# Output maps smoothly between 0 and 1
print(sigmoid(2.5))
3. Direct Comparison Matrix
Output TypeCore EquationEvaluation Metric
Page 1
Logistic
Feature Linear Regression
Regression
Continuous value (e.g., house price, Probability / Category (e.g., Spam vs Not
temperature) Spam)
Linear polynomial function Log-odds / Sigmoid transform curve
R-squared, RMSE, MAE Accuracy, Precision, Recall, AUC-ROC
Page 2