Machine Learning Algorithms:
Regression Algorithms
Linear Regression
1. Collect input features (X) and target values (y).
2. Initialize weights and bias.
3. Predict output using: y=wx+by = wx + by=wx+b.
4. Calculate error using Mean Squared Error (MSE).
5. Update weights using Gradient Descent.
6. Repeat until error is minimized.
Polynomial Regression
1. Convert input features into polynomial features.
2. Apply Linear Regression on transformed data.
3. Predict output using polynomial equation.
4. Calculate loss (MSE).
5. Optimize coefficients using Gradient Descent.
Support Vector Regressor (SVR)
1. Map data into higher-dimensional space using kernel.
2. Define an ε-insensitive tube.
3. Find the hyperplane that fits most points within ε margin.
4. Penalize points outside the margin.
5. Optimize using quadratic programming.
Decision Tree Regressor
1. Select the best feature using variance reduction.
2. Split data into subsets.
3. Repeat splitting recursively.
4. Stop when minimum samples or max depth is reached.
5. Predict value as mean of leaf node values.
Random Forest Regressor
1. Create multiple bootstrap samples from data.
2. Train a Decision Tree on each sample.
3. Select random features at each split.
4. Aggregate predictions from all trees.
5. Compute final prediction as average.
Ridge Regression
1. Standardize the data.
2. Add L2 regularization term to loss function.
3. Fit linear model with penalty on large coefficients.
4. Minimize cost function.
5. Produce stable predictions with reduced overfitting.
Classification Algorithms
K-Nearest Neighbors (KNN)
1. Choose the value of K.
2. Calculate distance between test point and all training points.
3. Select K nearest neighbors.
4. Count class labels among neighbors.
5. Assign the most frequent class.
Decision Tree Classifier
1. Select best feature using Gini Index / Entropy.
2. Split dataset based on feature values.
3. Repeat recursively for child nodes.
4. Stop when pure node or depth limit is reached.
5. Assign class label at leaf node.
Random Forest Classifier
1. Create multiple random samples from dataset.
2. Train a Decision Tree for each sample.
3. Use random feature selection at splits.
4. Predict class from each tree.
5. Choose final class by majority voting.
Support Vector Classifier (SVC)
1. Map data to higher-dimensional space.
2. Find optimal hyperplane with maximum margin.
3. Identify support vectors.
4. Use kernel trick if data is non-linear.
5. Classify based on which side of hyperplane.
Logistic Regression
1. Take linear combination of input features.
2. Apply sigmoid function.
3. Convert output to probability (0–1).
4. Use threshold (0.5) for class prediction.
5. Optimize parameters using Gradient Descent.
Clustering Algorithms
K-Means Clustering
1. Choose number of clusters (K).
2. Initialize K centroids randomly.
3. Assign each point to nearest centroid.
4. Recalculate centroids.
5. Repeat until centroids stabilize.
DBSCAN
1. Choose ε (epsilon) and MinPts.
2. Identify core points.
3. Expand clusters from core points.
4. Mark border points.
5. Label remaining points as noise.
Hierarchical Clustering
1. Treat each data point as a cluster.
2. Compute distance between clusters.
3. Merge closest clusters.
4. Repeat until one cluster remains.
5. Cut dendrogram to form clusters.
Dimensionality Reduction
PCA
1. Standardize the data.
2. Compute the covariance matrix.
3. Compute eigenvalues and eigenvectors.
4. Sort eigenvectors by eigenvalues.
5. Transform data into new space.
LDA
1. Compute mean vectors for each class.
2. Compute within-class scatter matrix (Sw).
3. Compute between-class scatter matrix (Sb).
4. Solve LDA eigenvalue equation.
5. Compute eigenvectors.
6. Project data onto LDA direction.
t-SNE
1. Compute pairwise similarities in high-dimensional space.
2. Convert similarities into probabilities.
3. Map points to low-dimensional space (2D/3D).
4. Optimize layout using Gradient Descent.