What it is and why we use it
Data augmentation creates new, label-preserving training examples
by transforming existing ones. The goal is to teach your model the right
invariances (like “a cat is still a cat if the photo is slightly shifted or a bit
darker”) and to reduce overfitting without collecting more data.
Purpose and effects
Regularization: reduces overfitting by making every epoch see fresh
variants of the data.
Inductive bias: bakes in desired invariances and robustness to noise,
viewpoint, lighting, accents, typos, drift, and so on.
Effective dataset growth: turns small or imbalanced datasets into
much richer training streams.
Tradeoff: heavy or unrealistic augmentation can slow learning or hurt
accuracy if it breaks the label or realism.
Practical considerations (for any modality)
1. Label preservation first: only apply transforms that do not change
the class/target.
2. Magnitude and probability: tune how strong and how often to apply
each transform (use small-to-moderate strengths by default).
3. Consistency across inputs and labels: for segmentation, keypoints,
boxes, audio alignments, apply the same transform to the input and
its labels/annotations.
4. Order matters: E.g., geometric changes before color/intensity
changes is a good default in vision.
5. Split hygiene: never augment validation/test.
6. Domain realism: prefer transforms that mimic real variation in your
domain.
7. Monitoring: watch training vs validation curves; if validation drops
while training rises, your policy is likely too strong or label-breaking.
8. On-the-fly vs offline: on-the-fly (random each epoch) is usually
better than saving augmented copies.
Common techniques and when to use them
Images / vision
Geometric
Flip, rotate (small angles), translate, random crop/resize,
scale, shear, perspective.
o Use when: object identity should not depend on exact pose or
position.
o Cautions: keep rotations small if “up vs down” matters; update
boxes/keypoints.
Photometric
Brightness, contrast, saturation, hue, gamma, grayscale, color
jitter, white-balance shift.
o Use when: lighting and color vary in the wild.
o Cautions: keep ranges realistic; medical imaging often needs
carefully bounded intensity shifts.
Noise and blur
Gaussian noise, JPEG artifacts, motion blur, defocus.
o Use when: cameras or pipelines introduce noise or compression.
o Cautions: too much blur can erase class cues.
Erase and mix
Cutout/Random Erasing, GridMask, Mixup, CutMix, Mosaic,
AugMix.
o Use when: you want occlusion robustness (Cutout) or smoother
decision boundaries (Mixup/CutMix).
o Cautions: for detection/segmentation, adjust labels; Mixup can
conflict with tasks needing crisp boundaries.
Key implementation note
For detection/segmentation/keypoints, synchronize transforms across
image and labels; libraries like Albumentations or torchvision’s v2 ops help.
A simple worked example (vision)
Task: CIFAR-like 32×32 image classification.
Desired invariances: small translations, horizontal mirror, mild lighting
change.
Baseline: train without augmentation → fast fit, overfitting after a few
epochs.
Augmented pipeline (per image, per epoch):
1. Random crop with padding 4: pad 4 pixels on each side, then crop
back to 32×32.
o Teaches translation invariance of roughly 12.5% of width/height.
2. Random horizontal flip (p = 0.5): mirrors left–right.
3. Color jitter (small): brightness ±10%, contrast ±10%.
o Keeps color shifts realistic.
What changes:
Each epoch, the same base image appears slightly shifted, sometimes
mirrored, and with mild lighting differences.
The network cannot memorize the exact pixel pattern; it must learn
shape and texture cues that survive these changes.
Validation accuracy improves and stays stable longer.
Minimal Keras example (on-the-fly augmentation)
import tensorflow as tf
from [Link] import layers, Sequential
# Augmentation block (runs on GPU, per-batch, per-epoch)
data_augment = Sequential([
[Link](32, 32), # for larger inputs; skip if already
32x32
[Link]("horizontal"),
[Link](factor=0.1),
[Link](factor=0.1),
])
# Small CNN
def make_model():
return Sequential([
[Link](shape=(32, 32, 3)),
data_augment,
[Link](1./255),
layers.Conv2D(32, 3, padding="same", activation="relu"),
layers.Conv2D(32, 3, padding="same", activation="relu"),
layers.MaxPooling2D(),
layers.Conv2D(64, 3, padding="same", activation="relu"),
layers.Conv2D(64, 3, padding="same", activation="relu"),
layers.GlobalAveragePooling2D(),
[Link](10, activation="softmax"),
])
# Train: [Link](train_ds, validation_data=val_ds, epochs=...)
# Ensure val_ds/test_ds DO NOT include augmentation; only Rescaling/normalization.
Quick recipe to choose a policy
1. List the real-world variations you expect and whether the label
should ignore them.
2. Add two or three simple transforms that encode those invariances.
3. Tune magnitudes and probabilities with a small grid search.
Bottom line: augmentations should look like believable alternate views of
the same example.