Complete Scikit-Learn Models: Hyperparameters

and Tuning Ranges

Table of Contents
1. Introduction
2. Classification Models (44 models)
3. Regression Models (55 models)
4. Clustering Models (12 models)
5. Preprocessing & Transformation (84 models)
6. Hyperparameter Tuning Best Practices

Introduction
This comprehensive guide covers all 207 estimators available in scikit-learn, categorized into Classification (44),
Regression (55), Clustering (12), and Preprocessing/Transformation (84) models. Each section provides detailed
hyperparameter information with ideal tuning ranges.

Classification Models

Tree-Based Classifiers

Decision Tree Classifier

Hyperparameter Description Typical Range Default Notes

Function to measure 'gini', 'entropy', 'gini' faster, 'entropy'

criterion 'gini'
quality of split 'log_loss' more informative

Start with 3-10, increase if

max_depth Maximum depth of tree 3-30, None None
underfitting

Minimum samples to Higher values prevent

min_samples_split 2-20 2
split node overfitting

Minimum samples at 1-5 optimal for

min_samples_leaf 1-10 1
leaf generalization

Features considered 'sqrt', 'log2', int, 'sqrt' recommended for

max_features None
for best split float classification

random_state Random seed Any integer None For reproducibility

Random Forest Classifier
Hyperparameter Description Typical Range Default Notes

More trees = better

n_estimators Number of trees 100-1000 100
performance, slower training

Split quality 'gini', 'entropy',

criterion 'gini' 'gini' generally sufficient
function 'log_loss'

Maximum tree
max_depth 3-30, None None 10-20 often optimal
depth

Minimum samples
min_samples_split 2-20 2 5-10 works well
to split

Minimum samples
min_samples_leaf 1-10 1 1-5 typical range
at leaf

'sqrt' standard for

max_features Features per split 'sqrt', 'log2', float 'sqrt'
classification

Extra Trees Classifier

Typical
Hyperparameter Description Default Notes
Range

n_estimators Number of trees 100-500 100 Similar to RandomForest

'gini', Less critical than

criterion Split quality measure 'gini'
'entropy' RandomForest

Generally deeper than

max_depth Maximum depth 3-30, None None
RandomForest

Minimum samples to Less sensitive than

min_samples_split 2-20 2
split RandomForest

Ensemble Classifiers

AdaBoost Classifier
Hyperparameter Description Typical Range Default Notes

Number of weak More estimators can help

n_estimators 50-500 50
learners performance

Weight applied to each Lower rates need more

learning_rate 0.01-2.0 1.0
classifier estimators

'SAMME',
algorithm Boosting algorithm 'SAMME.R' SAMME.R generally better
'SAMME.R'
Gradient Boosting Classifier
Typical
Hyperparameter Description Default Notes
Range

Number of boosting
n_estimators 100-500 100 Use early stopping
stages

Lower values need more

learning_rate Shrinkage rate 0.01-0.3 0.1
estimators

max_depth Maximum tree depth 3-8 3 Shallow trees work well

Fraction of samples for 0.8-0.9 can improve

subsample 0.5-1.0 1.0
fitting generalization

Histogram Gradient Boosting Classifier

Typical
Hyperparameter Description Default Notes
Range

Similar to
learning_rate Boosting learning rate 0.01-0.3 0.1
GradientBoosting

Maximum boosting
max_iter 100-500 100 Replaces n_estimators
iterations

max_depth Maximum tree depth 3-10 None None means unlimited

min_samples_leaf Minimum samples in leaf 1-50 20 Higher than traditional GB

Linear Classifiers

Logistic Regression
Hyperparameter Description Typical Range Default Notes

Inverse regularization Log scale: [0.001, 0.01,

C 0.001-1000 1.0
strength 0.1, 1, 10, 100]

penalty Regularization type 'l1', 'l2', 'elasticnet', None 'l2' 'l1' for feature selection

Optimization 'liblinear', 'lbfgs', 'newton- Depends on penalty and

solver 'lbfgs'
algorithm cg', 'sag', 'saga' data size

Increase if convergence
max_iter Maximum iterations 100-10000 100
issues

Ridge Classifier
Hyperparameter Description Typical Range Default Notes

Regularization Log scale

alpha 0.001-1000 1.0
strength recommended
 Hyperparameter Description Typical Range Default Notes

Deprecated, use
normalize Normalize features True/False False
preprocessing

'auto', 'svd', 'cholesky', 'lsqr',

solver Solver algorithm 'auto' 'sag' for large datasets
'sparse_cg', 'sag', 'saga'

SGD Classifier
Hyperparameter Description Typical Range Default Notes

'hinge', 'log_loss',
'hinge' for SVM, 'log_loss' for
loss Loss function 'squared_hinge', 'hinge'
logistic
'perceptron'

Regularization Higher values increase

alpha 0.0001-1.0 0.0001
parameter regularization

Learning rate 'constant', 'optimal',

learning_rate 'optimal' 'optimal' usually works well
schedule 'invscaling', 'adaptive'

Initial learning Only for

eta0 0.01-1.0 0.0
rate constant/invscaling/adaptive

Support Vector Machines

SVC (Support Vector Classifier)

Hyperparameter Description Typical Range Default Notes

Regularization Log scale: [0.1, 1, 10, 100,

C 0.1-1000 1.0
parameter 1000]

'linear', 'poly', 'rbf', Start with 'rbf', try 'linear' for

kernel Kernel type 'rbf'
'sigmoid' high dims

gamma Kernel coefficient 'scale', 'auto', 0.001-10 'scale' Only for rbf, poly, sigmoid

degree Polynomial degree 2-5 3 Only for 'poly' kernel

Linear SVC
Hyperparameter Description Typical Range Default Notes

Regularization
C 0.01-100 1.0 Log scale tuning
parameter

penalty Regularization norm 'l1', 'l2' 'l2' 'l1' for feature selection

'hinge', 'hinge' is standard SVM

loss Loss function 'squared_hinge'
'squared_hinge' loss

Dual or primal False when n_samples

dual True/False True
optimization > n_features
Nu-SVC
Hyperparameter Description Typical Range Default Notes

Upper bound on error Alternative to C

nu 0.01-1.0 0.5
fraction parameter

'linear', 'poly', 'rbf',

kernel Kernel type 'rbf' Same as SVC
'sigmoid'

Kernel-specific
gamma Kernel coefficient 'scale', 'auto', 0.001-10 'scale'
parameter

Naive Bayes Classifiers

Gaussian NB
Hyperparameter Description Typical Range Default Notes

priors Prior probabilities array-like, None None If None, adjusted from data

var_smoothing Variance smoothing 1e-12 to 1e-6 1e-9 Prevents zero probabilities

Multinomial NB
Hyperparameter Description Typical Range Default Notes

Additive smoothing Laplace smoothing, 0.1-2.0

alpha 0.01-10.0 1.0
parameter typical

array-like,
class_prior Prior probabilities None If None, uniform priors
None

Whether to learn class

fit_prior True/False True False for uniform priors
priors

Bernoulli NB
Hyperparameter Description Typical Range Default Notes

alpha Additive smoothing 0.01-10.0 1.0 Similar to MultinomialNB

binarize Threshold for binarizing 0.0-1.0, None 0.0 None if data already binary

fit_prior Learn class priors True/False True False for uniform priors

Complement NB
Hyperparameter Description Typical Range Default Notes

alpha Smoothing parameter 0.01-10.0 1.0 Same as other NB variants

fit_prior Learn class priors True/False True Complement NB specific

norm Perform feature normalization True/False False Can improve performance

Neural Network Classifiers

MLP Classifier
Hyperparameter Description Typical Range Default Notes

Hidden layer (50,), (100,), Start simple, increase

hidden_layer_sizes (100,)
architecture (100,50) complexity

'relu', 'tanh',
activation Activation function 'relu' 'relu' most common
'logistic'

Log scale: [0.0001, 0.001,

learning_rate_init Initial learning rate 0.0001-0.1 0.001
0.01]

Higher values prevent

alpha L2 regularization 0.0001-1.0 0.0001
overfitting

max_iter Maximum iterations 200-2000 200 Increase if not converging

batch_size Batch size 32-512, 'auto' 'auto' Powers of 2 for efficiency

K-Nearest Neighbors

KNeighbors Classifier
Hyperparameter Description Typical Range Default Notes

Number of Odd numbers avoid ties, 3-15

n_neighbors 3-30 5
neighbors typical

'distance' gives closer

weights Weight function 'uniform', 'distance' 'uniform'
neighbors more weight

Neighbor search 'auto', 'ball_tree', 'ball_tree' for high

algorithm 'auto'
algorithm 'kd_tree', 'brute' dimensions

Power parameter for

p 1-2 2 1=Manhattan, 2=Euclidean
distance

Radius Neighbors Classifier

Hyperparameter Description Typical Range Default Notes

Radius of Dataset
radius 0.1-10.0 1.0
neighborhood dependent

weights Weight function 'uniform', 'distance' 'uniform' Similar to KNN

'auto', 'ball_tree', 'kd_tree',

algorithm Search algorithm 'auto' Same as KNN
'brute'
Additional Classifiers

Gaussian Process Classifier

Hyperparameter Description Typical Range Default Notes

RBF(), Matern(), Gaussian process

kernel Kernel instance RBF()
etc specific

Optimizer for log-

'fmin_l_bfgs_b', None disables
optimizer marginal- 'fmin_l_bfgs_b'
callable, None optimization
likelihood

More restarts =
Number of
n_restarts_optimizer 0-10 0 better
restarts
optimization

Maximum
For approximation
max_iter_predict iterations in 50-500 100
methods
Newton's method

Regression Models

Linear Regression Models

Linear Regression
Hyperparameter Description Typical Range Default Notes

fit_intercept Calculate intercept True/False True Usually keep True

normalize Normalize features True/False False Deprecated, use preprocessing

copy_X Copy X or overwrite True/False True Memory vs speed tradeoff

n_jobs Number of parallel jobs -1, 1-n_cores None -1 uses all cores

Ridge Regression
Hyperparameter Description Typical Range Default Notes

Regularization Log scale: [0.001, 0.01,

alpha 0.001-1000 1.0
strength 0.1, 1, 10, 100]

Calculate
fit_intercept True/False True Usually True
intercept

Normalize Use StandardScaler

normalize True/False False
features instead

'auto', 'svd', 'cholesky', 'lsqr',

solver Solver algorithm 'auto' 'sag' for large datasets
'sparse_cg', 'sag', 'saga'
Lasso Regression
Hyperparameter Description Typical Range Default Notes

alpha Regularization strength 0.001-100 1.0 Start with [0.001, 0.01, 0.1, 1, 10]

fit_intercept Calculate intercept True/False True Usually True

normalize Normalize features True/False False Use preprocessing

max_iter Maximum iterations 1000-10000 1000 Increase for convergence

Feature selection 'cyclic', 'random' faster for large

selection 'cyclic'
method 'random' datasets

Elastic Net
Hyperparameter Description Typical Range Default Notes

alpha Regularization strength 0.001-100 1.0 Overall regularization

l1_ratio Elastic net mixing 0.0-1.0 0.5 0=Ridge, 1=Lasso, 0.5=balanced

fit_intercept Calculate intercept True/False True Usually True

max_iter Maximum iterations 1000-10000 1000 For convergence

Bayesian Ridge
Hyperparameter Description Typical Range Default Notes

alpha_1 Gamma prior parameter 1e-8 to 1e-2 1e-6 Shape parameter for alpha

alpha_2 Gamma prior parameter 1e-8 to 1e-2 1e-6 Rate parameter for alpha

lambda_1 Gamma prior parameter 1e-8 to 1e-2 1e-6 Shape parameter for lambda

lambda_2 Gamma prior parameter 1e-8 to 1e-2 1e-6 Rate parameter for lambda

n_iter Maximum iterations 100-1000 300 For convergence

Tree-Based Regression

Decision Tree Regressor

Hyperparameter Description Typical Range Default Notes

Split quality 'mse', 'mae', 'mae' more robust to

criterion 'mse'
function 'friedman_mse', 'poisson' outliers

max_depth Maximum depth 3-20, None None Start with 3-10

Minimum samples Higher prevents

min_samples_split 2-20 2
to split overfitting
 Hyperparameter Description Typical Range Default Notes

Minimum samples
min_samples_leaf 1-10 1 1-5 optimal range
at leaf

Features for best 'auto', 'sqrt', 'log2', int, n_features/3 for

max_features None
split float regression

Random Forest Regressor

Hyperparameter Description Typical Range Default Notes

More trees = better

n_estimators Number of trees 100-500 100
performance

criterion Split quality function 'mse', 'mae' 'mse' 'mae' for outlier robustness

max_depth Maximum tree depth 3-30, None None 10-20 often good

Minimum samples to
min_samples_split 2-20 2 5-10 works well
split

'auto', 'sqrt', int, 'auto' = n_features for

max_features Features per split 'auto'
float regression

Extra Trees Regressor

Hyperparameter Description Typical Range Default Notes

n_estimators Number of trees 100-500 100 Similar to Random Forest

criterion Split criterion 'mse', 'mae' 'mse' Less critical than Random Forest

max_depth Maximum depth 3-30, None None Can be deeper than RF

Ensemble Regression

AdaBoost Regressor
Hyperparameter Description Typical Range Default Notes

Number of More can improve

n_estimators 50-500 50
estimators performance

Lower needs more

learning_rate Weight shrinkage 0.01-2.0 1.0
estimators

'linear', 'square',
loss Loss function 'linear' 'linear' most robust
'exponential'
Gradient Boosting Regressor
Hyperparameter Description Typical Range Default Notes

Number of boosting
n_estimators 100-1000 100 Use early stopping
stages

Lower values need more

learning_rate Shrinkage parameter 0.01-0.3 0.1
estimators

max_depth Maximum tree depth 3-8 3 Shallow trees work well

0.8-0.9 can help

subsample Fraction of samples 0.5-1.0 1.0
generalization

'ls', 'lad', 'huber',

loss Loss function 'ls' 'huber' robust to outliers
'quantile'

Support Vector Regression

SVR (Support Vector Regression)

Hyperparameter Description Typical Range Default Notes

Regularization Controls training error vs

C 0.1-1000 1.0
parameter smoothness

Epsilon in epsilon-
epsilon 0.01-1.0 0.1 Width of epsilon-tube
SVR

'linear', 'poly', 'rbf',

kernel Kernel type 'rbf' Choose based on data
'sigmoid'

gamma Kernel coefficient 'scale', 'auto', 0.001-10 'scale' For rbf, poly, sigmoid

Linear SVR
Hyperparameter Description Typical Range Default Notes

Regularization Log scale

C 0.01-100 1.0
parameter tuning

Epsilon
epsilon 0.01-1.0 0.1 Tube width
parameter

'epsilon_insensitive', Standard
loss Loss function 'epsilon_insensitive'
'squared_epsilon_insensitive' SVR loss

False when
Dual or primal
dual True/False True n_samples >
optimization
n_features
Nu-SVR
Hyperparameter Description Typical Range Default Notes

Upper bound on error Alternative

nu 0.01-1.0 0.5
fraction parameterization to C

Regularization
C 0.1-1000 1.0 Used with nu
parameter

'linear', 'poly', 'rbf',

kernel Kernel type 'rbf' Same as SVR
'sigmoid'

Neural Network Regression

MLP Regressor
Hyperparameter Description Typical Range Default Notes

hidden_layer_sizes Hidden layer sizes (50,), (100,), (100,50) (100,) Start simple

'relu', 'tanh',
activation Activation function 'relu' 'relu' most common
'logistic'

learning_rate_init Initial learning rate 0.0001-0.1 0.001 Log scale tuning

alpha L2 regularization 0.0001-1.0 0.0001 Prevents overfitting

Maximum Increase if not

max_iter 200-2000 200
iterations converging

Additional Regressors

Huber Regressor
Hyperparameter Description Typical Range Default Notes

epsilon Robustness parameter 1.1-2.0 1.35 Controls outlier sensitivity

alpha Regularization strength 0.0001-1.0 0.0001 L2 regularization

max_iter Maximum iterations 100-1000 100 For convergence

RANSAC Regressor
Hyperparameter Description Typical Range Default Notes

Minimum samples If None, uses minimum

min_samples 0.1-0.9, int None
for fit needed

Auto-computed, Outlier detection

residual_threshold Residual threshold None
float threshold

Number of random
max_trials Maximum iterations 100-1000 100
selections
 Hyperparameter Description Typical Range Default Notes

stop_probability Stopping probability 0.95-0.99 0.99 Probability of good fit

Theil-Sen Regressor
Typical
Hyperparameter Description Default Notes
Range

fit_intercept Calculate intercept True/False True Usually True

copy_X Copy input data True/False True Memory vs speed

Maximum
max_subpopulation 1000-10000 10000 Computational limit
subpopulation

If None, uses all

n_subsamples Subsamples for median int, None None
combinations

Clustering Models

Density-Based Clustering

DBSCAN
Hyperparameter Description Typical Range Default Notes

Most important
Maximum distance
eps 0.1-10.0 0.5 parameter, dataset
between samples
dependent

Samples in
Higher values = fewer,
min_samples neighborhood for core 3-20 5
denser clusters
point

'euclidean',
Choose based on data
metric Distance metric 'manhattan', 'euclidean'
type
'cosine'

Algorithm for neighbor 'auto', 'ball_tree', 'ball_tree' for high

algorithm 'auto'
search 'kd_tree', 'brute' dimensions

Leaf size for tree Affects speed and

leaf_size 10-50 30
algorithms memory

OPTICS
Hyperparameter Description Typical Range Default Notes

Samples in
min_samples 3-20 5 Similar to DBSCAN
neighborhood

Maximum distance Upper bound for

max_eps 1.0-inf inf
between samples reachability
 Hyperparameter Description Typical Range Default Notes

'euclidean',
metric Distance metric 'euclidean' Same as DBSCAN
'manhattan', 'cosine'

'xi' more robust

cluster_method Extraction method 'xi', 'dbscan' 'xi'
than 'dbscan'

For cluster
xi Steepness threshold 0.01-0.1 0.05
extraction

HDBSCAN
Hyperparameter Description Typical Range Default Notes

Minimum Smallest grouping

min_cluster_size 5-100 5
cluster size considered cluster

Conservative If None, equals

min_samples 1-20 None
point threshold min_cluster_size

Distance DBSCAN-like flat

cluster_selection_epsilon 0.0-1.0 0.0
threshold clustering

'euclidean',
Various metrics
metric Distance metric 'manhattan', 'euclidean'
supported
'cosine'

Centroid-Based Clustering

K-Means
Hyperparameter Description Typical Range Default Notes

Use elbow method or

n_clusters Number of clusters 2-20+ 8
silhouette analysis

Initialization 'k-means++', 'k-

init 'k-means++' usually better
method 'random', ndarray means++'

Number of More runs = better result,

n_init 1-20 10
initializations slower

max_iter Maximum iterations 100-1000 300 For convergence

Tolerance for
tol 1e-6 to 1e-2 1e-4 Relative tolerance
convergence

'lloyd', 'elkan', 'auto', 'elkan' faster for well-

algorithm Algorithm variant 'lloyd'
'full' separated clusters
Mini-Batch K-Means
Typical
Hyperparameter Description Default Notes
Range

n_clusters Number of clusters 2-20+ 8 Same as K-Means

batch_size Size of mini-batches 100-1000 1024 Larger batches = better quality

Usually fewer needed than K-

max_iter Maximum iterations 100-1000 100
Means

Number of random
n_init 1-10 3 Fewer than K-Means
initializations

Bisecting K-Means
Hyperparameter Description Typical Range Default Notes

Target number of Same as K-

n_clusters 2-20+ 8
clusters Means

Initialization 'k-means++', Different from

init 'random'
method 'random' regular K-Means

Cluster selection 'biggest_inertia', How to choose

bisecting_strategy 'biggest_inertia'
strategy 'largest_cluster' cluster to split

Maximum
For each
max_iter iterations per 100-1000 300
bisection
split

Hierarchical Clustering

Agglomerative Clustering
Hyperparameter Description Typical Range Default Notes

Number of
n_clusters 2-20+ 2 Target number of clusters
clusters

Linkage 'ward', 'complete',

linkage 'ward' 'ward' minimizes variance
criterion 'average', 'single'

'euclidean', 'l1', 'l2', Must be compatible with

metric Distance metric 'euclidean'
'manhattan', 'cosine' linkage

Connectivity Constrains which samples

connectivity None, array-like None
matrix can be linked
Other Clustering Methods

Affinity Propagation
Hyperparameter Description Typical Range Default Notes

Controls message passing

damping Damping factor 0.5-0.99 0.5
stability

max_iter Maximum iterations 100-1000 200 For convergence

Iterations without
convergence_iter 10-50 15 Before stopping
change

None, array-like, Points' preference to be

preference Preference values None
float exemplars

Spectral Clustering
Hyperparameter Description Typical Range Default Notes

n_clusters Number of clusters 2-20+ 8 Target clusters

Eigenvalue None, 'arpack', 'lobpcg', None chooses

eigen_solver None
decomposition 'amg' automatically

Dimension of Usually same as

n_components 2-100 n_clusters
projected subspace n_clusters

gamma Kernel coefficient 0.01-10.0 1.0 For RBF kernel

Affinity matrix 'nearest_neighbors', 'rbf', How to build

affinity 'rbf'
construction 'precomputed' similarity matrix

Mean Shift
Hyperparameter Description Typical Range Default Notes

Most critical parameter, estimate

bandwidth Bandwidth parameter 0.1-10.0 None
from data

None, array-
seeds Initial kernel locations None If None, uses all points
like

Discretize initial
bin_seeding True/False False Can speed up algorithm
locations

Minimum bin
min_bin_freq 1-10 1 For bin seeding
frequency
Birch
Hyperparameter Description Typical Range Default Notes

n_clusters Number of clusters 3-20+, None 3 If None, just builds CF tree

threshold Subcluster radius threshold 0.1-10.0 0.5 Controls tree structure

branching_factor Maximum CF subclusters 10-100 50 Per non-leaf node

compute_labels Compute cluster labels True/False True Whether to label points

Preprocessing & Transformation

Feature Scaling

Standard Scaler
Hyperparameter Description Typical Range Default Notes

copy Copy data or in-place True/False True False modifies original data

with_mean Center data to zero mean True/False True False for sparse data

with_std Scale to unit variance True/False True False keeps original variance

MinMax Scaler
Hyperparameter Description Typical Range Default Notes

feature_range Target range (0,1), (-1,1), etc. (0,1) Desired min/max values

copy Copy data True/False True Memory vs speed tradeoff

clip Clip to feature range True/False False Handle unseen data

Robust Scaler
Hyperparameter Description Typical Range Default Notes

Quantile range for (5.0, 95.0), (25.0, (25.0,

quantile_range Robust to outliers
scaling 75.0) 75.0)

Memory
copy Copy data True/False True
consideration

unit_variance Scale to unit variance True/False False Additional scaling

Max Abs Scaler
Hyperparameter Description Typical Range Default Notes

copy Copy data True/False True Only hyperparameter

Normalizer
Hyperparameter Description Typical Range Default Notes

norm Norm to use 'l1', 'l2', 'max' 'l2' Type of normalization

copy Copy data True/False True Memory consideration

Dimensionality Reduction

PCA (Principal Component Analysis)

Hyperparameter Description Typical Range Default Notes

Number of 2-min(n_samples, None keeps all

n_components None
components n_features) components

copy Copy data True/False True Memory vs speed

Whiten Components have unit

whiten True/False False
components variance

'auto', 'full', 'arpack',

svd_solver SVD solver 'auto' Algorithm choice
'randomized'

random_state Random seed Any integer None For reproducibility

Incremental PCA
Hyperparameter Description Typical Range Default Notes

n_components Number of components 2-n_features None None keeps all

whiten Whiten components True/False False Unit variance components

copy Copy data True/False True Memory consideration

batch_size Batch size for fitting 100-1000 None For large datasets

Kernel PCA
Hyperparameter Description Typical Range Default Notes

Number of
n_components 2-n_samples None None keeps all
components

'linear', 'poly', 'rbf', 'sigmoid', Nonlinear

kernel Kernel type 'linear'
'cosine' transformations
 Hyperparameter Description Typical Range Default Notes

gamma Kernel coefficient 0.001-10.0 None For rbf, poly, sigmoid

degree Polynomial degree 2-5 3 For poly kernel

coef0 Independent term 0.0-10.0 1 For poly, sigmoid

Truncated SVD
Hyperparameter Description Typical Range Default Notes

Number of 2-min(n_samples, Desired

n_components 2
components n_features) dimensionality

algorithm SVD solver 'randomized', 'arpack' 'randomized' 'randomized' faster

Number of For randomized

n_iter 5-20 5
iterations algorithm

random_state Random seed Any integer None Reproducibility

Feature Selection

Variance Threshold
Hyperparameter Description Typical Range Default Notes

threshold Variance threshold 0.0-1.0 0.0 Features below threshold removed

Select K Best
Hyperparameter Description Typical Range Default Notes

f_classif, f_regression, chi2, Choose based on

score_func Scoring function f_classif
mutual_info_classif problem type

Number of features
k 1-n_features, 'all' 10 Top k features
to select

Select Percentile
Hyperparameter Description Typical Range Default Notes

score_func Scoring function f_classif, f_regression, chi2 f_classif Univariate scoring

percentile Percentile of features 1-100 10 Percentage to keep

RFE (Recursive Feature Elimination)
Hyperparameter Description Typical Range Default Notes

Any sklearn Must have coef_ or

estimator Base estimator None
estimator feature_importances_

1-n_features,
n_features_to_select Features to select None If None, selects half
None

Features to remove 1-n_features,

step 1 Absolute or fraction
per iteration 0.0-1.0

RFECV (RFE with Cross-Validation)

Hyperparameter Description Typical Range Default Notes

With coef_ or
estimator Base estimator Any sklearn estimator None
feature_importances_

Features to
step 1-n_features, 0.0-1.0 1 Per iteration
remove

int, cross-validation
cv Cross-validation None Determines CV strategy
generator

scoring Scoring metric string, callable None Model evaluation

Feature Engineering

Polynomial Features
Typical
Hyperparameter Description Default Notes
Range

Higher degrees = more

degree Polynomial degree 2-4 2
features

Only interaction
interaction_only True/False False No powers, only products
features

include_bias Include bias column True/False True Constant term

K-Bins Discretizer
Hyperparameter Description Typical Range Default Notes

n_bins Number of bins 3-20 5 Per feature or global

encode Encoding method 'onehot', 'onehot-dense', 'ordinal' 'onehot' Output format

strategy Binning strategy 'uniform', 'quantile', 'kmeans' 'quantile' How to define bins
One Hot Encoder
Hyperparameter Description Typical Range Default Notes

categories Categories per feature 'auto', list of arrays 'auto' Known categories

None, 'first', 'if_binary',

drop Drop strategy None Avoid collinearity
array

sparse Return sparse matrix True/False True Memory efficiency

Unknown category For unseen

handle_unknown 'error', 'ignore' 'error'
strategy categories

Ordinal Encoder
Hyperparameter Description Typical Range Default Notes

Categories
categories 'auto', list of arrays 'auto' Known categories
per feature

Unknown
'error',
handle_unknown category 'error' For new categories
'use_encoded_value'
handling

Value for When

unknown_value int, float None
unknown handle_unknown='use_encoded_value'

Text Processing

TF-IDF Transformer
Typical
Hyperparameter Description Default Notes
Range

norm Normalization 'l1', 'l2', None 'l2' Row normalization

Use inverse document

use_idf Enable IDF reweighting True/False True
frequency

Add one to document

smooth_idf True/False True Prevents zero divisions
frequencies

sublinear_tf Apply sublinear TF scaling True/False False log(TF) instead of TF

Count Vectorizer
Hyperparameter Description Typical Range Default Notes

max_features Maximum features 1000-50000, None None Vocabulary size limit

min_df Minimum document frequency 1-100, 0.0-1.0 1 Ignore rare terms

max_df Maximum document frequency 0.5-1.0, int 1.0 Ignore common terms

ngram_range N-gram range (1,1), (1,2), (1,3) (1,1) Unigrams, bigrams, etc.
 Hyperparameter Description Typical Range Default Notes

stop_words Stop word removal None, 'english', list None Common word filtering

Missing Value Imputation

Simple Imputer
Hyperparameter Description Typical Range Default Notes

Missing value What represents

missing_values NaN, int, str NaN
placeholder missing

Imputation 'mean', 'median',

strategy 'mean' How to fill missing
strategy 'most_frequent', 'constant'

Fill value for When

fill_value Any value None
constant strategy='constant'

copy Copy data True/False True Memory consideration

KNN Imputer
Hyperparameter Description Typical Range Default Notes

n_neighbors Number of neighbors 3-20 5 For imputation

weights Weight function 'uniform', 'distance' 'uniform' Neighbor weighting

metric Distance metric 'euclidean', 'manhattan' 'euclidean' For finding neighbors

copy Copy data True/False True Memory vs speed

Hyperparameter Tuning Best Practices

Grid Search and Random Search

Grid Search CV
Parameter Description Typical Values Notes

param_grid Parameter grid Dict or list of dicts Exhaustive search space

cv Cross-validation folds 3, 5, 10 Balance bias-variance

scoring Scoring metric 'accuracy', 'f1', 'roc_auc' Problem-specific

n_jobs Parallel jobs -1, 1-n_cores -1 uses all cores

refit Refit on best params True/False True for final model

Randomized Search CV
Parameter Description Typical Values Notes

param_distributions Parameter distributions Dict of distributions [Link] distributions

n_iter Number of iterations 50-200 Budget for search

cv Cross-validation 3, 5, 10 Same as GridSearch

scoring Scoring metric Various Problem-dependent

random_state Random seed Any integer Reproducibility

Advanced Tuning Techniques

Halving Grid Search

Parameter Description Typical Values Notes

param_grid Parameter grid Dict Same as GridSearch

factor Reduction factor 2, 3 Resource reduction rate

resource Resource parameter 'n_samples', estimator param What to increase

max_resources Maximum resources int, 'auto' Resource limit

Bayesian Optimization (using external libraries)

Optuna: n_trials, sampler, pruner
Hyperopt: max_evals, algo, trials
Scikit-Optimize: n_calls, acq_func, n_initial_points

General Tuning Guidelines

Parameter Search Strategies

1. Start Simple: Begin with default parameters, establish baseline
2. Coarse-to-Fine: Wide ranges first, then narrow around best values
3. Log Scales: Use for regularization parameters (alpha, C, learning_rate)
4. Cross-Validation: Always use proper validation to avoid overfitting
5. Computational Budget: Balance search thoroughness with time constraints
Common Parameter Ranges by Algorithm Type
Regularization: Log scale from 1e-6 to 1e2
Learning Rates: Log scale from 1e-5 to 1e-1
Tree Depths: Linear scale from 3 to 30
Neighbors: Linear scale from 3 to 50
Iterations: Linear scale, use early stopping when possible

Performance Considerations
Parallel Processing: Use n_jobs=-1 for faster searches
Early Stopping: Implement for iterative algorithms
Validation Strategy: Choose appropriate CV based on dataset size and problem
Memory Management: Consider memory usage for large parameter grids

Model-Specific Tuning Order

1. Tree-based: max_depth → min_samples_split → max_features → n_estimators
2. SVM: C → gamma (if RBF) → kernel choice
3. Neural Networks: hidden_layer_sizes → learning_rate → alpha → activation
4. Ensemble: Base model parameters → ensemble-specific parameters
This comprehensive guide covers all 207 scikit-learn estimators with their hyperparameters and recommended tuning
ranges. Remember that optimal values depend on your specific dataset and problem domain.