Machine Learning

Assignment No. 1

Q1. Difference Between Supervised and Unsupervised Learning?

Aspect Supervised Learning Unsupervised Learning
Input Data Uses labeled data (input features + correspondingUses
outputs)
unlabeled data (only input features, no outputs)

Goal Predicts outcomes or classifies data based on known

Discovers
labels hidden structures or grouping in data

Computational Complexity
Less complex, as the model learns from labeled data
More complex, as the model must find patterns

Types Classification, Regression Clustering and Association

Testing the Model Model can be tested and evaluated using labeled test
Cannot
databe tested in the traditional sense, as there are no lab

Q2. Steps in Development of ML and its Application?

1. Problem Definition:
Clearly define the problem to be solved. Identify the objective and expected output.
2. Data Collection:
Gather relevant data from sources such as databases, sensors, APIs, or the internet. The quality
of data directly affects model performance.
3. Data Preprocessing:
Clean the data by handling missing values, noise, and outliers. Convert data into a suitable format.
4. Feature Selection and Engineering:
Select important features that influence the output. Create new features to improve model
performance.
5. Model Selection:
Choose an appropriate ML algorithm (e.g. Linear Regression, Decision Trees, SVM). The choice
depends on the problem type and data size.
6. Model Training:
Train the model using training data. The algorithm learns patterns from the data.
7. Model Evaluation:
Evaluate the model using test data. Performance metrics such as accuracy, precision, recall, or
RMSE are used.
8. Model Optimization:
Tune hyperparameters to improve performance. Techniques like cross-validation are applied.
9. Deployment:
Deploy the trained model into a real-world environment. The model starts making predictions on
new data.
10. Monitoring and Maintenance:
Continuously monitor model performance. Update or retrain the model when data changes.

Applications of ML:
Healthcare: Medical image analysis, Disease prediction
Finance: Fraud detection, Stock market prediction
Education: Student performance analysis, Online exam proctoring
E-Commerce: Product recommendations, Customer behavior analysis
 Transportation: Self-driving cars, Traffic prediction
Social Media: Fake news detection, Content recommendation

Q3. Note with Diagram on Overfitting and Underfitting?

Underfitting
Underfitting happens when the model fails to learn important patterns. It performs poorly on both training
and testing data.
Underfitting happens due to:
• Model is too simple • Very high regularization penalty added to loss functions to prevent overfitting •
Features are weak or missing • Not enough training • High bias
Bias-Variance in Underfitting: Underfitting mainly occurs due to high bias. High bias means model
makes strong assumptions. It ignores patterns and learns an overly simple representation.

Overfitting
Overfitting happens when the model learns too much from the training data, including noise and outliers.
It performs very well on training data but poorly on test data.
Overfitting happens due to:
• Model too complex • Too many features • Very little data • High variance
Bias-Variance in Overfitting: Overfitting is mainly caused by high variance. High variance means model
reacts too strongly to training data. It learns noise as patterns. Low bias because the model is extremely
flexible.

Q4. Solve the Following (Confusion Matrix Problem)?

Given Confusion Matrix (Total = 150):

Predicted: NO Predicted: YES Total

Actual: NO TN = 40 FP = 10 50

Actual: YES FN = 10 TP = 90 100

Total 50 100 150

1) Accuracy:
= (TP + TN) / (TP + TN + FP + FN) = (90 + 40) / (90 + 40 + 10 + 10) =
130/150 = 0.8667 = 86.67%

2) Precision:
= TP / (TP + FP) = 90 / (90 + 10) = 90/100 = 0.90 = 90%

3) Recall:
= TP / (TP + FN) = 90 / (90 + 10) = 90/100 = 0.90 = 90%

4) Specificity:
= TN / (TN + FP) = 40 / (40 + 10) = 40/50 = 0.80 = 80%

5) F1 Score:
 = 2 × Precision × Recall / (Precision + Recall) = 2 × 0.9 × 0.9 / (0.9 +
0.9) = 1.62/1.8 = 0.90 = 90%

6) RMSE:
= √((FP + FN) / (TP + TN + FP + FN)) = √(20/150) = √0.1333 = 0.365 = 36%

Q5. Application of SVD (Singular Value Decomposition)?

1. Image Compression
SVD is used to compress images by keeping only the largest singular values and removing smaller
ones. Reduces storage space and maintains image quality.

2. Noise Reduction
Small singular values usually represent noise in data. By removing them, data becomes cleaner and
improves accuracy of models.

3. Principal Component Analysis (PCA)

SVD is used to compute PCA. Reduces dimensionality, removes redundant features, and keeps
maximum variance.

4. Recommender Systems
SVD is used in collaborative filtering techniques. It decomposes user-item rating matrices to find
hidden patterns. Eg: Netflix, Amazon.

5. Solving Linear Systems

SVD is used to solve overdetermined, underdetermined, and least squares problems.

6. Face Recognition
Used in Eigenfaces method. Extracts important facial features, reduces dimensionality of image data,
and is used in biometric systems.

Q6. Diagonalise the Matrix A = [[4, 2], [3, 3]]?

Step 1: Find Eigenvalues using |A - λI| = 0

|A - λI| = |4-λ 2 | | 3 3-λ| = 0
(4-λ)(3-λ) - 6 = 0 4(9-λ) - λ(9-λ) - 6 = 0 12 - 4λ - 3λ + λ² - 6 = 0 λ² -
7λ + 6 = 0 ∴ λ = 1, 6 → λ1 = 1, λ2 = 6

Step 2: Eigen Vector for λ = 1

[A - 1·I][x] = 0 → [3 2][x1] = 0 [3 2][x2] 3x1 + 2x2 = 0 → x1 = -2/3 x2
Let x2 = 3 → x1 = -2 x1 = [2, -3]■

Step 3: Eigen Vector for λ = 6

[A - 6·I][x] = 0 → [-2 2][x1] = 0 [ 3 -3][x2] -2x1 + 2x2 = 0 → x1 = x2
Let x2 = 1 → x1 = 1 x2 = [1, 1]■

Step 4: Form Modal Matrix P and Diagonal Matrix D

P = [2 1 ] D = [1 0] [-3 1] [0 6] This is the diagonalized form. ✓

Q7. Find Eigen Value and Vector of A = [[14, -10], [5, -1]]?
Step 1: Find Eigenvalues using |A - λI| = 0
|14-λ -10| | 5 -1-λ| = 0
(14-λ)(-1-λ) - (-10)(5) = 0 -14 - 14λ + λ + λ² + 50 = 0 λ² - 13λ + 36 = 0 λ
= 4 and 9 → λ1 = 4, λ2 = 9

Step 2: Eigen Vector for λ1 = 4

[14-4 -10][x] = 0 → [10 -10][x] = 0 [5 -1-4][y] [ 5 -5][y] 10x - 10y = 0
→ x = y Eigenvector x1 = [1, 1]■

Step 3: Eigen Vector for λ2 = 9

[14-9 -10][x] = 0 → [5 -10][x] = 0 [5 -1-9][y] [5 -10][y] 5x - 10y = 0 →
x = 2y Let y = 1 → x = 2 Eigenvector x2 = [2, 1]■

Q8. Fit Least Square Straight Line to the Following Data?

Data: x = [2, 7, 9, 1, 5, 12], y = [13, 21, 23, 14, 15, 21]

x y x² y² xy
2 13 4 169 26

7 21 49 441 147

9 23 81 529 207

1 14 1 196 14

5 15 25 225 75

12 21 144 441 252

Σ = 36 107 304 2001 721

Normal Equations (n = 6):

Σy = n·a0 + a1·Σx → 107 = 6·a0 + 36·a1 Σxy = a0·Σx + a1·Σx² → 721 = 36·a0
+ 304·a1
Solving: a0 = 12.446, a1 = 0.89
Regression Line: y = 12.446 + 0.89x ✓

Q9. Define Regularized Regression and its Types?

Regularized regression is a type of regression technique in which a penalty term is added to the loss
function to prevent overfitting and improve model generalization.

Types of Regularization:
1. Lasso Regression (L1)
A regression model which uses the L1 Regularization technique called LASSO (Least Absolute
Shrinkage and Selection Operator) regression. It adds the absolute value of magnitude of the
coefficient as a penalty term to the loss function (L).
Loss = (1/n)Σ(yi - ■i)² + λΣ|wj|

2. Ridge Regression (L2)

 It uses the L2 regularization technique called ridge regression. It adds the squared magnitude of the
coefficient as a penalty term to the loss function. Shrinks coefficients toward zero.
Loss = (1/n)Σ(yi - ■i)² + λΣwi²

3. Elastic Net Regression

It is a combination of both L1 as well as L2 regularization. Useful when features are highly correlated.
Loss = Σ(y - ■)² + λ1Σ|β| + λ2Σβ²

Q10. What is SVM? Explain How Margin is Computed and Optimal

Hyperplane is Decided?
Support Vector Machine (SVM) is a supervised machine learning algorithm for classification and
regression that finds the optimal hyperplane separating classes by maximizing the margin — the
distance between the closest data points of different classes.

How Margin is Computed:

The margin is the perpendicular gap between the separating hyperplane and the nearest data points.
Distance from a point xi to the hyperplane defined by w■x + b = 0 is |w■xi + b| / ||w||
For support vectors, where w■xi + b = ±1, this distance is 1/||w||
The total margin between the two classes = 2/||w||

How Optimal Hyperplane is Decided:

The objective is to maximize the margin (2/||w||), which is equivalent to minimizing the norm of the
weight vector ||w||.
The optimization must satisfy:
w■xi + b ≥ 1 for positive class samples (yi = 1) w■xi + b ≤ -1 for
negative class samples (yi = -1) ensuring no points fall inside the
margin.
This is solved using quadratic programming to find the optimal w and b that defines the maximum
margin hyperplane.
If data is not perfectly separable, a regularization parameter C is introduced to allow some
classification errors (soft margin) to maintain a robust boundary.

— End of ML Assignment 1 Notes —