Course Code: AI-802
Programming Assignment: Linear Regression
Course: Machine Learning
Instructor: Dr. Adeel Nisar
Total Marks: 100
Guidelines
The assignment is divided into 3 parts. Each part carries marks.
Work must be your own. No copy, no plagiarism (0 marks if found).
Use libraries like numpy, pandas, matplotlib, seaborn only for data handling and plotting.
Do not use built-in regression models (e.g., sklearn.linear_model.LinearRegression).
Submit as a Colab/Jupyter Notebook with proper comments.
Objective
The purpose of this assignment is to help you understand the working of Linear Regression by
implementing it from scratch in Python. You will load datasets, define the hypothesis function,
implement the cost function, apply gradient descent to optimize parameters, and visualize the
results.
Part 1: Dataset Load and preprocessing (Marks=15+15)
You can either load datasets directly from online sources (like in the Boston housing example
or by using built-in datasets such as California housing), or you can use your own dataset files
(CSV, Excel, etc.) downloaded and saved locally.
Univariate Dataset (one feature, one target)
Example: Anscombe’s dataset (built into Seaborn).
Multivariate Dataset (multiple features, one target)
Example: Boston Housing dataset (from CMU source).
Preprocessing:
Before applying linear regression, you must preprocess the dataset:
Handle Missing Values
Feature Scaling (Normalization/Standardization)
Splitting Data
Course Code: AI-802
Part 2: Implementing Linear Regression (Marks=15+15+15)
You need to implement the following functions in Python.
1. Hypothesis Function
Ŷ = �� + �
Ŷ → predicted value (what the model guesses).
�→ input value.
� → slope (weight/parameter, tells how steep the line is).
� → intercept (bias/parameter, tells where the line cuts the y-axis).
� (��) − � �
�= � �
� � −( �)
�−� �
�=
�
2. Cost Function
Use Mean Squared Error (MSE) as the cost function:
�
�
� �, � = (�� − (��� + �))�
��
�=�
� → cost (total error we want to minimize).
� → number of data points.
�� → actual value.
(��� + �) → predicted value for input �� .
The square ensures errors are always positive.
3. Optimization Function (Gradient Descent)
Use Gradient Descent to minimize the cost function:
�� ��
� ∶= � − � , � ∶= � − �
�� ��
∶= → “update” (means we replace old value with new).
� → learning rate (small step size).
��
��
→ how much cost changes when slope mm changes.
Course Code: AI-802
��
��
→ how much cost changes when intercept cc changes.
Part 3: Computing Optimal Parameters & Visualization (Marks=15+10)
1. Compute Optimal Parameters
o Run your gradient descent implementation.
o Report the final θ values (parameters) and minimized cost.
2. Visualization
You can use the following code snippet to compare actual vs predicted values:
Sample code:
import [Link] as plt
def plot_regression_line(X, y, y_pred):
[Link](X, y, color="blue", label="Actual Prices")
[Link](X, y_pred, color="red", label="Predicted Prices")
[Link]("Feature (X)")
[Link]("Target (y)")
[Link]("Actual vs Predicted Prices")
[Link]()
[Link]()
Course Code: AI-802