Module 2: REGRESSION MODEL
Q. WHAT IS REGRESSION MODEL?
A regression model is a statistical model that estimates the relationship between one dependent variable and one or more
independent variables using a line.
Use regression analysis for one of two purposes:-
1. Predict the value of the dependent variable when you know the independent variables
2. Predict the effect of an independent variable on the dependent variable
Ex:- If study hours affect exam scores, the independent variable is study hours and the dependent variable is exam scores
Independent Variable:
An independent variable is the variable you manipulate or vary in an experimental study to explore its effects
It’s called “independent” because it’s not influenced by any other variables in the study
Independent variables are also known as predictors, factors, treatment variables, explanatory variables, input
variables, x-variables, right-hand variable
Dependent Variable:
A dependent variable is the variable that changes as a result of the independent variable manipulation
The dependent variable (DV) is what you want to use the model to explain or predict
It’s also known as the response variable, outcome variable, and left-hand variable
Regression is classifies into Two types:-
1. Simple Regression and Multiple Regression
2. Linear Regression and Nonlinear Regression
Type of Regression Description
1. Simple Regression Regression analysis for studying only two variables at a time.
2. Multiple Regression Regression analysis for studying more than two variables at a
time.
3. Linear Regression Regression curve is a straight line.
4. Nonlinear Regression Regression curve is not a straight line.
Q. What are the similarities and differences between linear regression and logistic regression?
Linear Regression Logistic Regression
Linear Regression is a supervised regression model. Logistic Regression is a supervised classification model.
Here no threshold value is needed. Here a threshold value is added.
Least Square estimation method is used for estimation Maximum likelihood estimation method is used for estimation
of accuracy. of accuracy.
Here when we plot the training datasets, a straight line Any change in the coefficient leads to a change in both the
can be drawn that touches maximum plots. direction and the steepness of the logistic function. It means
positive slopes result in an S-shaped curve and negative slopes
result in a Z-shaped curve.
Linear regression is used to estimate the dependent Whereas logistic regression is used to calculate the probability
variable in case of a change in independent variables. of an event. For example, User will purchase the product or not.
For example, predict the price of houses.
Applications of linear regression: Applications of logistic regression:
- Financial risk assessment - Medicine
- Business insights - Credit scoring
- Market analysis - Hotel Booking
- Gaming
- Text editing
Feature Linear Regression Logistic Regression
Type of Used for Regression (continuous Used for Classification (categorical output)
Problem output)
Output Type Produces continuous values Produces probabilities (values between 0 and 1)
Model Type Linear model. Linear model but with a logistic (sigmoid) function to map
outputs to probabilities.
Dependent Continuous variable. Binary or categorical variable.
Variable
Relationship Assumes a linear relationship Assumes a linear relationship between independent variables
between independent and dependent and the log odds of the dependent variable.
variables.
Equation Y= b0+ b1X1 +b2X2+ ⋯+bnXnY
Form
Decision No fixed boundary, can predict any Uses a threshold (e.g., 0.5 for binary classification)
Boundary value
Loss Function Mean Squared Error (MSE) Log Loss (Cross-Entropy)
Use Case Predicting house prices, sales Classifying spam vs. non-spam emails, disease detection
revenue, etc.
SIMPLE LINEAR REGRESSION:-
Simple Linear Regression:
Simple Linear Regression is a mathematical method that explores the linear relationship between a dependent variable
and one independent variable.
The objective is to find the "line of best fit" (regression line) that minimizes the distance between actual data points and
predicted values.
This line represents the relationship between the dependent variable (Y) and independent variable (X), providing a
linear model.
Use Cases:
1. Evaluating Trends and Sales Estimates:
Simple linear regression is commonly used in business to evaluate trends and make forecasts.
Ex:- If a company’s sales have consistently increased each month, linear regression can be applied with sales
data on the Y-axis and time on the X-axis. The resulting line shows the upward trend in sales and can be used
to predict future sales.
2. Analyzing the Impact of Price Changes:
It can be used to assess the impact of price changes on consumer behavior.
Ex:- A company may record the quantity sold at different price levels and use linear regression to analyze the
relationship. The regression line shows how consumer demand decreases as prices increase, helping the
company make informed pricing decisions.
Equation for the Line of Best Fit: y=a+bx
Where:
o y = Dependent variable
o x = Independent variable
o b = Slope of the line (rate of change of y with respect to x)
o a = Y-intercept (point where the line crosses the Y-axis)
Calculating the Line of Best Fit:
To determine the line of best fit, the values of aaa (intercept) and bbb (slope) must be calculated based on
the data.
Least Squares - Simple Linear Regression:-
The least squares method is used to determine the best-fitting line or curve by minimizing the
sum of the squared differences (residuals) between observed data points and the predicted
values.
It is a fundamental statistical technique for deriving the regression line or best-fit line based on a
given dataset.
Since fitting curves to data may not always result in a unique solution, the goal is to find a curve
with the least deviation from all data points, achieved through the least squares method.
This method is widely applied in data fitting to minimize the sum of squared errors (residuals).
Residuals are the differences between the observed values and the predicted values from the
model, representing the error in the predictions.