Cheat Sheet: Building Supervised Learning Models
Common supervised learning models
Process Name Brief Description Code Syntax
Process: This method trains one from [Link] import OneVsOneClassifier
classifier for each pair of classes. from sklearn.linear_model import LogisticRegression
model = OneVsOneClassifier(LogisticRegression())
Key hyperparameters:
- `estimator`: Base classifier (e.g.,
One vs One logistic regression)
classifier (using Pros: Can work well for small datasets.
logistic regression) Cons: Computationally expensive for
large datasets.
Common applications: Multiclass
classification problems where the
number of classes is relatively small.
from [Link] import OneVsRestClassifier
from sklearn.linear_model import LogisticRegression
model = OneVsRestClassifier(LogisticRegression())
Process: Trains one classifier per class,
where each classifier distinguishes
between one class and the rest.
Key hyperparameters:
- `estimator`: Base classifier (e.g.,
Logistic Regression)
One vs All - `multi_class`: Strategy to handle
classifier (using multiclass classification (`ovr`) or
logistic regression) Pros: Simpler and more scalable than
from sklearn.linear_model import LogisticRegression
One vs One. model_ova = LogisticRegression(multi_class='ovr')
Cons: Less accurate for highly
imbalanced classes.
Common applications: Common in
multiclass classification problems such
as image classification.
Process: A tree-based classifier that
from [Link] import DecisionTreeClassifier
splits data into smaller subsets based on model = DecisionTreeClassifier(max_depth=5)
feature values.
Key hyperparameters:
- `max_depth`: Maximum depth of the
Decision tree
tree
classifier
Pros: Easy to interpret and visualize.
Cons: Prone to overfitting if not pruned
properly.
Common applications: Classification
tasks, such as credit risk assessment.
Process: Similar to the decision tree
classifier, but used for regression tasks
from [Link] import DecisionTreeRegressor
to predict continuous values. model = DecisionTreeRegressor(max_depth=5)
Key hyperparameters:
- `max_depth`: Maximum depth of the
tree
Decision tree
Pros: Easy to interpret, handles
regressor
nonlinear data.
Cons: Can overfit and perform poorly
on noisy data.
Common applications: Regression
[Link] xMjc1MzF9.f-9bRaBjoFwk9I_vmp5CSmshq8MKN2vXf-uyoI2c1PU 27/08/25, 12 19
Page 1 of 4
:
1
tasks, such as predicting housing
prices.
Process: A linear classifier that finds
the optimal hyperplane separating
classes with a maximum margin.
from [Link] import SVC
Key hyperparameters: model = SVC(kernel='linear', C=1.0)
- `C`: Regularization parameter
- `kernel`: Type of kernel function
(`linear`, `poly`, `rbf`, etc.)
Linear SVM
- `gamma`: Kernel coefficient (only for
classifier
`rbf`, `poly`, etc.)
Pros: Effective for high-dimensional
spaces.
Cons: Not ideal for nonlinear problems
without kernel tricks.
Common applications: Text
classification and image recognition.
Process: Classifies data based on the
majority class of its nearest neighbors.
Key hyperparameters:
- `n_neighbors`: Number of neighbors
from [Link] import KNeighborsClassifier
to use model = KNeighborsClassifier(n_neighbors=5, weights='uniform')
- `weights`: Weight function used in
prediction (`uniform` or `distance`)
K-nearest - `algorithm`: Algorithm used to
neighbors compute the nearest neighbors (`auto`,
classifier `ball_tree`, `kd_tree`, `brute`)
Pros: Simple and effective for small
datasets.
Cons: Computationally expensive as
the dataset grows.
Common applications:
Recommendation systems, image
recognition.
Process: An ensemble method using
multiple decision trees to improve
accuracy and reduce overfitting.
from [Link] import RandomForestRegressor
Key hyperparameters: model = RandomForestRegressor(n_estimators=100, max_depth=5)
- `n_estimators`: Number of trees in the
forest
- `max_depth`: Maximum depth of each
Random Forest
tree
regressor
Pros: Less prone to overfitting than
individual decision trees.
Cons: Model complexity increases
with the number of trees.
Common applications: Regression
tasks such as predicting sales or stock
prices.
Process: A gradient boosting method
that builds trees sequentially to correct
errors from previous trees.
Key hyperparameters: import xgboost as xgb
- `n_estimators`: Number of boosting model = [Link](n_estimators=100, learning_rate=0.1, max_depth=5)
rounds
- `learning_rate`: Step size to improve
accuracy
XGBoost - `max_depth`: Maximum depth of each
regressor tree
Pros: High accuracy and works well
with large datasets.
Cons: Computationally intensive,
[Link] xMjc1MzF9.f-9bRaBjoFwk9I_vmp5CSmshq8MKN2vXf-uyoI2c1PU 27/08/25, 12 19
Page 2 of 4
:
1
complex to tune.
Common applications: Predictive
modeling, especially in Kaggle
competitions.
Associated functions used
Method Name Brief Description Code Syntax
from [Link] import OneHotEncoder
encoder = OneHotEncoder(sparse=False)
encoded_data = encoder.fit_transform(categorical_data)
Transforms categorical features into a one-hot
OneHotEncoder
encoded matrix.
from [Link] import accuracy_score
accuracy = accuracy_score(y_true, y_pred)
Computes the accuracy of a classifier by
accuracy_score
comparing predicted and true labels.
from [Link] import LabelEncoder
encoder = LabelEncoder()
encoded_labels = encoder.fit_transform(labels)
Encodes labels (target variable) into numeric
LabelEncoder
format.
from [Link] import plot_tree
plot_tree(model, max_depth=3, filled=True)
plot_tree Plots a decision tree model for visualization.
from [Link] import normalize
normalized_data = normalize(data, norm='l2')
Scales each feature to have zero mean and unit
normalize
variance (standardization).
from [Link].class_weight import compute_sample_weight
[Link] xMjc1MzF9.f-9bRaBjoFwk9I_vmp5CSmshq8MKN2vXf-uyoI2c1PU 27/08/25, 12 19
Page 3 of 4
:
1
weights = compute_sample_weight(class_weight='balanced', y=y)
Computes sample weights for imbalanced
compute_sample_weight
datasets.
from [Link] import roc_auc_score
auc = roc_auc_score(y_true, y_score)
Computes the Area Under the Receiver Operating
roc_auc_score Characteristic Curve (AUC-ROC) for binary
classification models.
Author
Jeff Grossman
Abhishek Gagneja
[Link] xMjc1MzF9.f-9bRaBjoFwk9I_vmp5CSmshq8MKN2vXf-uyoI2c1PU 27/08/25, 12 19
Page 4 of 4
:
1