Multiple Linear Regression
Linear Regression:
It is the basic and commonly used type for predictive analysis. It is a statistical
approach to modeling the relationship between a dependent variable and a given
set of independent variables.
These are of two types:
1. Simple linear Regression
2. Multiple Linear Regression
Multiple Linear Regression attempts to model the relationship between two or
more features and a response by fitting a linear equation to observed data. The
steps to perform multiple linear Regression are almost similar to that of simple
linear Regression. The Difference Lies in the evaluation. We can use it to find out
which factor has the highest impact on the predicted output and how different
variables relate to each other.
Here : Y = b0 + b1 * x1 + b2 * x2 + b3 * x3 + …… bn * xn
Y = Dependent variable and x1, x2, x3, …… xn = multiple independent variables
Assumption of Regression Model :
Linearity: The relationship between dependent and independent variables
should be linear.
Homoscedasticity: Constant variance of the errors should be maintained.
Multivariate normality: Multiple Regression assumes that the residuals are
normally distributed.
Lack of Multicollinearity: It is assumed that there is little or no
multicollinearity in the data.
Dummy Variable:
As we know in the Multiple Regression Model we use a lot of categorical data.
Using Categorical Data is a good method to include non-numeric data into the
respective Regression Model. Categorical Data refers to data values that represent
categories-data values with the fixed and unordered number of values, for instance,
gender(male/female). In the regression model, these values can be represented by
Dummy Variables.
These variables consist of values such as 0 or 1 representing the presence and
absence of categorical values.
Dummy Variable Trap:
The Dummy Variable Trap is a condition in which two or more are Highly
Correlated. In the simple term, we can say that one variable can be predicted from
the prediction of the other. The solution of the Dummy Variable Trap is to drop one
of the categorical variables. So if there are m Dummy variables then m-1 variables
are used in the model.
D2 = D1-1
Here D2, D1 = Dummy Variables
Method of Building Models :
All-in
Backward-Elimination
Forward Selection
Bidirectional Elimination
Score Comparison
Backward-Elimination :
Step #1: Select a significant level to start in the model.
Step #2: Fit the full model with all possible predictors.
Step #3: Consider the predictor with the highest P-value. If P > SL go to STEP 4,
otherwise the model is Ready.
Step #4: Remove the predictor.
Step #5: Fit the model without this variable.
Forward-Selection :
Step #1 : Select a significance level to enter the model(e.g. SL = 0.05)
Step #2: Fit all simple regression models y~ x(n). Select the one with the lowest P-
value.
Step #3: Keep this variable and fit all possible models with one extra predictor
added to the one(s) you already have.
Step #4: Consider the predictor with the lowest P-value. If P < SL, go to Step #3,
otherwise the model is Ready.
Steps Involved in any Multiple Linear Regression Model
Step #1: Data Pre Processing
1. Importing The Libraries.
2. Importing the Data Set.
3. Encoding the Categorical Data.
4. Avoiding the Dummy Variable Trap.
5. Splitting the Data set into Training Set and Test Set.
Step #2: Fitting Multiple Linear Regression to the Training set
Step #3: Predict the Test set results.
Code 1 :
Python3
import numpy as np
import matplotlib as mpl
from mpl_toolkits.mplot3d import Axes3D
import [Link] as plt
def generate_dataset(n):
x = []
y = []
random_x1 = [Link]()
random_x2 = [Link]()
for i in range(n):
x1 = i
x2 = i/2 + [Link]()*n
[Link]([1, x1, x2])
[Link](random_x1 * x1 + random_x2 * x2 + 1)
return [Link](x), [Link](y)
x, y = generate_dataset(200)
[Link]['[Link]'] = 12
fig = [Link]()
ax = fig.add_subplot(projection ='3d')
[Link](x[:, 1], x[:, 2], y, label ='y', s = 5)
[Link]()
ax.view_init(45, 0)
[Link]()
output :