Scribd
Upload
5 views1 page

Supervised ML Algorithms Notes

Supervised Machine Learning involves training models on labeled data to predict outputs for unseen data, with key types including classification and regression. Various algorithms are categorized into linear models, tree-based models, support vector machines, instance-based methods, probabilistic models, and neural networks, each with specific improvements for performance. General techniques for enhancing model performance include data quality management, balancing, cross-validation, hyperparameter tuning, regularization, and early stopping.

Uploaded by

iqrafatima
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views1 page

Supervised ML Algorithms Notes

Supervised Machine Learning involves training models on labeled data to predict outputs for unseen data, with key types including classification and regression. Various algorithms are categorized into linear models, tree-based models, support vector machines, instance-based methods, probabilistic models, and neural networks, each with specific improvements for performance. General techniques for enhancing model performance include data quality management, balancing, cross-validation, hyperparameter tuning, regularization, and early stopping.

Uploaded by

iqrafatima
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Supervised Machine Learning (Detailed Notes)

1. What is Supervised ML?


- Definition: Supervised ML wo learning hai jahan model ko labeled data diya jata hai. - Matlab
training data ke sath input (X) aur output (Y) dono available hoty hain. - Goal = Model ne unseen
data ke liye correct output predict karna.
- Types: 1. Classification → Predict categorical output (Yes/No, Spam/Not spam). 2. Regression →
Predict continuous output (House Price, Temperature). 2. Supervised ML Algorithms
(A) Linear Models
1. Linear Regression → Continuous prediction (salary, price). Improve: Feature scaling, Polynomial
features, Regularization.
2. Logistic Regression → Classification (spam detection). Improve: Feature scaling, L1/L2
regularization. (B) Tree-Based Models
3. Decision Tree → Easy to interpret, risk of overfitting. Improve: Pruning, Ensemble.
4. Random Forest → Collection of decision trees. Improve: More trees, Feature selection.
5. Extra Trees → Faster & less variance. Improve: Increase estimators.
6. Gradient Boosting → Sequential trees, high accuracy. Improve: Tune learning rate, Early
stopping.
7. XGBoost → Fast boosting, best for large datasets. Improve: Hyperparameter tuning, Early
stopping.
8. LightGBM → Very large datasets, fast training. Improve: min_data_in_leaf, Feature selection.
9. CatBoost → Handles categorical data. Improve: GPU training, Cross validation. (C) Support
Vector Machines
10. SVM → High-dimensional data, uses kernels. Improve: Feature scaling, Kernel trick, C tuning.
(D) Instance-Based
11. kNN → Small datasets, recommendation systems. Improve: Scaling, Best k selection. (E)
Probabilistic
12. Naive Bayes → Text classification, spam filtering. Improve: Correct NB type, Handle imbalance.
(F) Neural Networks
13. Neural Networks → Complex patterns (images, NLP). Improve: Dropout, BatchNorm, Early
stopping. (G) Ensemble Methods
14. Bagging → Reduce variance (Random Forest).
15. Boosting → Reduce bias (XGBoost, GBM).
16. Stacking/Voting → Combine models for accuracy. 3. General Performance Improvement
Techniques
- Data Quality: Clean missing values, remove outliers. - Data Balancing: SMOTE, Oversampling,
Undersampling. - Cross Validation: K-Fold, Stratified K-Fold. - Hyperparameter Tuning: Grid
Search, Random Search, Bayesian Optimization. - Regularization: L1, L2, Elastic Net. - Early
Stopping: Prevent overfitting. Quick Formula:
- Simple Data → Linear / Logistic
- Small Data → kNN, Naive Bayes
- Medium/Complex → Random Forest, SVM
- Big Data → XGBoost, LightGBM, CatBoost
- Deep Patterns → Neural Networks

You might also like