Detailed Machine Learning Notes (Units I–V)
UNIT I: Introduction to Machine Learning
Machine Learning (ML) is a branch of Artificial Intelligence that enables computers to learn from
data.
Topics covered:
• Evolution of ML and AI
• Supervised, Unsupervised and Reinforcement Learning
• Learning by Rote and Learning by Induction
• Types of Data: Structured, Semi-structured and Unstructured
• Data Acquisition and Preprocessing
• Feature Engineering and Feature Selection
• Data Representation Techniques
• Model Selection, Training and Testing
• Model Evaluation Metrics: Accuracy, Precision, Recall, F1-score
• Datasets: Training, Validation and Test Sets
Advantages: Automation, prediction, pattern discovery.
Applications: Healthcare, finance, transportation, agriculture and cybersecurity.
Important Questions:
1. Explain stages in Machine Learning.
2. Compare supervised and unsupervised learning.
3. Explain feature engineering with examples.
UNIT II: Nearest Neighbor Based Models
Nearest Neighbor methods classify data based on similarity.
Distance Measures:
1. Euclidean Distance
2. Manhattan Distance
3. Minkowski Distance
4. Hamming Distance
K-Nearest Neighbor (KNN):
Step 1: Choose K.
Step 2: Calculate distance from query point.
Step 3: Select nearest neighbors.
Step 4: Perform voting/classification.
KNN Regression:
Prediction is the average of neighboring values.
Advantages:
• Easy to implement
• No training phase
Disadvantages:
• Slow for large datasets
• Sensitive to irrelevant features
Performance Metrics:
Classification: Accuracy, Precision, Recall.
Regression: MAE, MSE, RMSE, R² Score.
UNIT III: Decision Trees and Bayes Classifier
Decision Trees:
Tree structure consisting of root node, internal nodes and leaf nodes.
Impurity Measures:
• Entropy
• Information Gain
• Gini Index
Random Forest:
Ensemble of multiple decision trees.
Advantages:
• Higher accuracy
• Reduced overfitting
Bayes Theorem:
P(A|B)=P(B|A)P(A)/P(B)
Naive Bayes:
Assumes conditional independence between features.
Applications:
• Spam filtering
• Sentiment analysis
• Document classification
Bias-Variance Tradeoff:
Low bias and low variance are desirable for better generalization.
UNIT IV: Linear Discriminants for Machine Learning
Linear Discriminants:
Used to separate classes using linear decision boundaries.
Perceptron:
Single-layer neural classifier.
Learning Rule:
Adjust weights based on classification error.
Support Vector Machines (SVM):
Finds optimal hyperplane with maximum margin.
Types:
• Linear SVM
• Non-linear SVM
Kernel Trick:
Transforms data into higher-dimensional space.
Common Kernels:
• Linear
• Polynomial
• Radial Basis Function (RBF)
Regression Models:
• Linear Regression
• Logistic Regression
Multi-Layer Perceptron (MLP):
Input Layer → Hidden Layer(s) → Output Layer
Backpropagation:
Used to update weights by minimizing error.
UNIT V: Clustering
Clustering groups similar data points together.
Types:
1. Partitional Clustering
2. Hierarchical Clustering
3. Density-based Clustering
K-Means Algorithm:
Step 1: Choose K clusters.
Step 2: Initialize centroids.
Step 3: Assign points to nearest centroid.
Step 4: Recompute centroids.
Step 5: Repeat until convergence.
Hierarchical Clustering:
• Agglomerative (bottom-up)
• Divisive (top-down)
Fuzzy C-Means:
Data points can belong to multiple clusters.
Expectation Maximization (EM):
Probabilistic clustering technique.
Spectral Clustering:
Uses graph theory and eigenvectors.
Applications:
• Customer segmentation
• Image segmentation
• Bioinformatics