PRACTICAL REFERENCE

Machine Learning
Essentials
The minimum, high-leverage knowledge you actually need to start
contributing to a real ML project — concepts, workflow, and code patterns,
without the filler.

Core Concepts Data & Features Models Evaluation Deep Learning Project Workflow

F O C U SE D STU D Y N OTE S • B UI LD- RE AD Y

1
— What's Inside

Read top to bottom. Each section ends where you have just enough to be dangerous. Star anything you'll touch directly in your project.

01 The Mental Model — What ML Actually Is

02 The ML Project Workflow

03 Data: The Part That Decides Everything

04 Features & Preprocessing

05 Core Algorithms You Must Know

06 Training: How Models Learn

07 Overfitting, Underfitting & the Bias–Variance Tradeoff

08 Evaluation & Metrics

09 Neural Networks & Deep Learning

10 The Practical Toolkit (Libraries & Code)

11 Contributing to a Real Project — Checklist

2
01 The Mental Model

Strip away the hype: ML is writing programs that learn rules from examples instead of you hand-coding the rules.

Traditional programming: you write rules, feed data, get answers. Machine learning: you feed data and answers, and the algorithm produces
the rules (the "model"). You then apply that model to new, unseen data.

The three families

Type You give it It learns to Examples

Supervised Inputs + correct labels Map input → output Spam detection, price prediction, image classification

Unsupervised Inputs only Find structure / groups Customer segmentation, anomaly detection, topic discovery

Reinforcement Environment + rewards Take actions to maximize reward Game agents, robotics, recommendation tuning

KEY DISTINCTION

Within supervised learning, classification predicts a category (spam / not spam) and regression predicts a continuous number (house price).
Knowing which one your task is determines your model choice and your metrics.

Vocabulary you'll hear daily

Feature — an input variable (a column). Also called a predictor or attribute.
Label / Target — the thing you're predicting (the answer).
Sample / Instance — one row of data (one example).
Model — the learned function that maps features to predictions.

Parameters — values the model learns during training (e.g., weights).

Hyperparameters — settings you choose before training (e.g., learning rate, tree depth).
Inference — using a trained model to make predictions.

02 The ML Project Workflow

Every project, big or small, follows this loop. Most real work lives in steps 1–3, not the modeling.

# Stage What actually happens

1 Define the problem What are we predicting? Is it classification or regression? What does success look like as a number?

2 Get & explore data Collect data, inspect it, plot distributions, find missing values and outliers (EDA).

3 Prepare data Clean, handle missing values, encode categories, scale, engineer features, split into train/val/test.

4 Choose & train a model Start simple (a baseline), fit it on training data.

5 Evaluate Measure performance on held-out data with the right metric.

6 Tune Adjust hyperparameters, try better features or models. Iterate.

7 Deploy & monitor Ship the model; watch for drift and degradation over time.

GOLDEN RULE

Always build a dumb baseline first (e.g., "predict the average" or "predict the most common class"). If your fancy model can't beat it, something
is wrong. Baselines tell you whether the problem is even learnable.

T H E # 1 B E G I N N E R M I S TA K E

Data leakage — letting information from the test set (or the future) sneak into training. The classic version: scaling or imputing using statistics
computed over the whole dataset before splitting. Always split first, then fit transformations on the training set only.

03 Data: The Part That Decides Everything

"Garbage in, garbage out" is the most reliable law in ML. Models are downstream of data quality.

The three-way split

Set Purpose Typical size

Training set Model learns from this ~60–80%

Validation set Tune hyperparameters & compare models ~10–20%

Test set Final, one-time honest score ~10–20%

WHY THREE SETS?

If you tune on the test set, you're "cheating" — the score stops reflecting real-world performance. The test set should be opened once, at the very
end.

Cross-validation
When data is limited, k-fold cross-validation gives a more reliable estimate: split the data into k parts, train on k–1 and validate on the remaining
one, rotating through all folds, then average the scores. k=5 or k=10 is standard.

Common data problems & fixes

Problem Typical fix

Missing values Drop rows/columns, or impute (mean/median/mode, or model-based)

Outliers Investigate, cap (clip), transform (log), or remove if errors

Imbalanced classes Resample (over/under), class weights, or use AUC/F1 instead of accuracy

Duplicates De-duplicate before splitting

Inconsistent formats Standardize units, dates, casing, categories

04 Features & Preprocessing

Most models can't eat raw data. Feature engineering is often what separates a winning model from a mediocre one.

Encoding categorical data

One-hot encoding — turn each category into a 0/1 column. Best for unordered categories (colors, cities) with few unique values.
Label / ordinal encoding — map categories to integers. Use only when there's a real order (small < medium < large).
Target / frequency encoding — for high-cardinality features; encode by statistics (careful with leakage).

Scaling numeric data often required

Method What it does When

Standardization (StandardScaler) Mean 0, std 1 Default for most algorithms

Normalization (MinMaxScaler) Squash to [0, 1] Neural nets, bounded ranges

W H E N S C A L I N G M AT T E R S

Distance- and gradient-based models (KNN, SVM, logistic/linear regression, neural networks) need scaling. Tree-based models (Decision
Trees, Random Forest, XGBoost) don't care about scale — a nice convenience.

Feature engineering ideas

Create interactions / ratios (e.g., price per square meter).
Extract parts from dates (day-of-week, month, is_weekend).
Bin continuous values into ranges where it helps.
Aggregate (counts, means per group). 4

Text → numbers via TF-IDF or embeddings.

05 Core Algorithms You Must Know

You don't need all of them on day one. Know what each is for, its strengths, and when to reach for it.

Linear Regression Logistic Regression

Predicts a number by fitting a straight line/plane. Simple, fast, Despite the name, it's classification. Outputs a probability via the
interpretable. Great baseline for regression. sigmoid. Strong, interpretable baseline for yes/no problems.

K-Nearest Neighbors (KNN) Decision Tree

Classifies by majority vote of the k closest points. No training step; Splits data with if/else questions. Very interpretable but overfits
slow at prediction. Needs scaling. easily on its own.

Random Forest Gradient Boosting (XGBoost / LightGBM)

Many decision trees averaged together (bagging). Robust, strong Trees built sequentially, each fixing the last one's errors. Often the
default, little tuning. A workhorse. top performer on tabular data.

Support Vector Machine (SVM) K-Means (unsupervised)

Finds the boundary with the widest margin between classes. Groups data into k clusters by similarity. The go-to for
Powerful on smaller, clean datasets. segmentation.

P R A C T I C A L D E FA U LT

For tabular data, start with Random Forest or Gradient Boosting — they win most of the time with minimal fuss. For images, audio, or text,
reach for neural networks (Section 09).

06 Training: How Models Learn

Most models learn by minimizing error. Understanding this loop demystifies the whole field.

The learning loop

1. Make predictions with current parameters.

2. Measure how wrong they are with a loss function.

3. Compute the direction that reduces the loss (the gradient).
4. Nudge parameters in that direction (this is gradient descent).

5. Repeat until the loss stops improving.

Loss functions (what "wrong" means)

Task Common loss

Regression Mean Squared Error (MSE), Mean Absolute Error (MAE)

Classification Cross-Entropy (a.k.a. Log Loss)

Key training knobs

Learning rate — step size. Too high → overshoots and diverges; too low → painfully slow. The most important hyperparameter.
Epoch — one full pass over the training data.
Batch size — how many samples per parameter update.

Gradient Descent variants — Stochastic (SGD), Mini-batch, and adaptive optimizers like Adam (a great default).

WAT C H F O R

If training loss is dropping but validation loss starts rising, you're overfitting — stop training (early stopping) or regularize. See the next section.

07 Overfitting, Underfitting & Bias–Variance

This single idea explains 80% of why models fail. Internalize it.

Underfitting (high bias) Overfitting (high variance)

Model is too simple to capture the pattern. Bad on both training Model memorizes the training data, including noise. Great on
and test data. Fix: more complex model, better features, train training, poor on test. Fix: more data, simpler model,
longer. regularization, dropout.

THE TRADEOFF

You want the sweet spot: complex enough to learn the signal, simple enough to generalize to new data. The gap between training and validation
scores is your diagnostic — a large gap means overfitting.

Regularization (your anti-overfitting toolkit)

L1 (Lasso) — shrinks some weights to exactly zero; also does feature selection.
L2 (Ridge) — shrinks all weights smoothly toward zero.
Dropout (neural nets) — randomly switches off neurons during training.
Early stopping — halt when validation performance stops improving.
Get more data — almost always the most effective fix.

08 Evaluation & Metrics

Choosing the wrong metric is how teams ship models that look great and fail in production. Accuracy alone is often a trap.

Classification metrics
Everything starts with the confusion matrix: True Positives (TP), True Negatives (TN), False Positives (FP), False Negatives (FN).

Metric Meaning Use when

Accuracy % correct overall Classes are balanced

Precision Of predicted positives, how many were right False positives are costly (e.g., flagging good email as spam)

Recall Of actual positives, how many you caught False negatives are costly (e.g., missing a disease)

F1 Score Harmonic mean of precision & recall You need balance; imbalanced classes

ROC-AUC Ranking quality across thresholds Comparing classifiers, imbalanced data

THE ACCURACY TRAP

If 99% of emails are legit, a model that predicts "never spam" scores 99% accuracy while being useless. On imbalanced data, lean on precision,
recall, F1, or AUC.

Regression metrics

Metric Meaning

MAE Average absolute error — easy to interpret, robust to outliers

MSE / RMSE Penalizes large errors more heavily; RMSE is in original units

R² (R-squared) Proportion of variance explained (1.0 = perfect, 0 = no better than the mean)

09 Neural Networks & Deep Learning

When data is large and unstructured (images, text, audio), deep learning shines. Here's the essential map.

The building blocks

6
Neuron — computes a weighted sum of inputs, adds a bias, applies an activation function.
Layers — neurons stacked; networks have an input layer, hidden layers, and an output layer.
 Activation functions — add non-linearity. ReLU is the default for hidden layers; sigmoid/softmax for output probabilities.
Backpropagation — the algorithm that computes gradients efficiently so the network can learn.

Key architectures

Architecture Best for

MLP / Dense network Basic tabular or vector data

CNN (Convolutional) Images, spatial data

RNN / LSTM Sequences, time series (older approach)

Transformer Text, and now nearly everything — the architecture behind modern LLMs

TRANSFER LEARNING — YOUR SHORTCUT

You rarely train from scratch. Take a model pretrained on huge data and fine-tune it on your smaller dataset. This is how most real deep-learning
projects work today, and it saves enormous time and compute.

FRAMEWORKS

PyTorch (dominant in research and increasingly industry) and TensorFlow/Keras are the two main libraries. For your first project, either works —
pick one and stick with it.

10 The Practical Toolkit

The Python stack you'll use 95% of the time. Get comfortable here and you can build almost anything.

Library Role

NumPy Fast numerical arrays — the foundation

Pandas Loading, cleaning, and manipulating tabular data

Matplotlib / Seaborn Visualization and EDA

scikit-learn Classic ML: models, preprocessing, metrics, splitting — your main tool

PyTorch / TensorFlow Deep learning

XGBoost / LightGBM State-of-the-art gradient boosting for tabular data

A complete scikit-learn workflow

END_TO_END.PY

7
 from sklearn.model_selection import train_test_split, cross_val_score
from [Link] import StandardScaler
from [Link] import RandomForestClassifier
from [Link] import classification_report
from [Link] import make_pipeline

# 1. Split FIRST (prevents data leakage)

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y)

# 2. Pipeline ties preprocessing + model together

model = make_pipeline(
StandardScaler(),
RandomForestClassifier(n_estimators=200, random_state=42)
)

# 3. Cross-validate to estimate performance

scores = cross_val_score(model, X_train, y_train, cv=5)
print(f"CV accuracy: {[Link]():.3f} +/- {[Link]():.3f}")

# 4. Train on full training set

[Link](X_train, y_train)

# 5. Evaluate ONCE on the held-out test set

y_pred = [Link](X_test)
print(classification_report(y_test, y_pred))

WHY PIPELINES?

A Pipeline applies the scaler's training statistics to the test data automatically — this is the clean, leak-free way to preprocess. Use them by
default.

11 Contributing to a Real Project

Knowledge in hand — here's how to actually plug into a project and add value without getting lost.

Before you write any model code

Read the existing code & docs. Understand the problem framing, data sources, and how they measure success.
Reproduce their current results. Run the existing pipeline end-to-end first — if you can't reproduce, you can't improve.
Find the metric. Know exactly what number the team is trying to move.
Look at the data yourself. Don't trust assumptions — plot it, check for leakage and imbalance.

Where beginners add the most value

Contribution Why it's high-impact

Better data cleaning / EDA Improves every downstream model; rarely glamorous, always needed

New engineered features Often beats fancier models for tabular problems

A stronger baseline Gives the team a clear yardstick

Better evaluation / error analysis Reveals where the model fails so it can be fixed

Reproducibility (seeds, configs, docs) Saves the whole team time

Habits that mark a good contributor

Set random seeds ( random_state=42 ) so results are reproducible.

Version your data and experiments — track what changed and why.
Change one thing at a time so you know what caused an improvement.
Always compare against the baseline, not just your last run.
Keep the test set sacred — touch it only at the very end.
Document assumptions in code comments and commit messages. 8

T H E O N E - L I N E S U M M A RY
Start simple, measure honestly, prevent leakage, and iterate one change at a time. Do that consistently and you'll outperform people who jump
straight to complex models.

SUGGESTED FIRST MOVE

Pick a small dataset, run the Section 10 pipeline end-to-end, beat a baseline, and write up what you found. That single exercise exercises every
concept in this document — and it's exactly the workflow you'll repeat on the real project.