M.
Tech Machine Learning - Module 1 & 2 Detailed Notes
Module 1: Introduction to Machine Learning
[...Content from Module 1 remains unchanged...]
Module 2: Regression, Classification & Advanced Algorithms
1. Linear Regression
1.1 Simple Linear Regression (SLR)
• Models relationship between two variables: one independent (X) and one dependent (Y).
• Equation: Y = β0 + β1 X + ϵ
• β0 : intercept, β1 : slope, ϵ : error term.
• Assumes linearity, independence, homoscedasticity, and normality of residuals.
• Evaluation Metrics: MSE, MAE, R².
Example: Given data: | X (Hours Studied) | Y (Marks) | |------------------|-----------| | 1 | 2 | | 2 | 4 | | 3 | 5
||4|4||5|5|
Fit a line: Y = β0 + β1 X Using least squares method, you get β0 = 2.2 , β1 = 0.6 So, Y = 2.2 +
0.6X
Visualization: A 2D plot with points and best-fit regression line.
1.2 Multiple Linear Regression (MLR)
• Models relationship between multiple independent variables and one dependent variable.
• Equation: Y = β0 + β1 X1 + β2 X2 + ... + βn Xn + ϵ
Example: Predict house price using area (X1), bedrooms (X2): P rice = 50000 + 150X1 + 10000X2
1.3 Evaluating Regression Fit
• R-squared (R²): Proportion of variance explained.
• Adjusted R²: Penalizes irrelevant features.
• RMSE/MAE: Lower is better.
1.4 Non-linear Regression
• Example: Polynomial Regression Y = 1 + 2X + 0.5X 2
Visualization: Parabolic curve fitted to data points.
1.5 Applications
• Sales Forecasting: Model sales vs ad budget.
• Real Estate: Model price vs area, locality, age.
• Concrete Strength: Regression using mix ratios.
1
2. Multi-Class and Multi-Label Classification
Multi-Class Example: Classifying digits 0–9. Multi-Label Example: Movie = Action + Comedy.
Techniques: - OvR: Train one binary classifier per class. - Binary Relevance: Separate classifiers for each
label.
3. Decision Tree Learning
3.1 Representation
Tree with: - Root = Feature - Branches = Feature values - Leaf = Class label
3.2 Algorithms
• ID3:
• Uses entropy: E(S) = −p+ log2 p+ − p− log2 p−
• Information Gain = Entropy(parent) - Weighted sum of children
Example: | Outlook | Humidity | Play? | |---------|----------|-------| | Sunny | High | No | | Overcast|
Normal | Yes | | Rainy | High | Yes |
Entropy(Play) = 0.918. Compute gains to choose splitting feature.
Visualization: Draw decision tree from example above.
3.3 Hypothesis Space Search
Greedy strategy builds tree by choosing best splits.
3.4 Inductive Bias
• Prefers shorter trees.
• Prefers attributes with high info gain.
4. Instance-Based Learning: KNN
Example: Test point: [5.5, 2.3] Use k=3, compute Euclidean distance to all training points, pick 3 closest,
vote for majority label.
Visualization: 2D scatter plot with different classes, circle around test point.
5. Support Vector Machine (SVM)
5.1 Maximum Margin Classifier
• Find hyperplane: w T x + b =0
2
• Margin = 2 / ||w||
Visualization: Show separating hyperplane and support vectors.
5.2 Dual Formulation
Maximize dual form using Lagrange multipliers.
5.3 Soft Margin (with noise)
Allows slack variables ξi , adds penalty C∑ ξi
5.4 Nonlinear SVM (Kernel Trick)
Maps data to higher dimension without computing explicitly.
Kernel Functions: - Polynomial: (xT z + c)d - RBF: exp(−γ∣∣x − z∣∣2 )
Visualization: 2D dataset non-separable in input space but separable in transformed space.
This updated version includes solved examples and visualizations for all major topics in Module 2. Let
me know if you'd like diagrams as images, MCQs, or numericals for practice.