Chapter Two

Data & Data

Preprocessing
Outline
Data Cleaning:
Understanding Data Data Cleaning:
Handling Missing
and Attributes Handling Noisy Data
Values

Feature Engineering Data Reduction & Text Representation

& Representation Selection for ML

Preparing Data for

Classification

© 2026 • Introduction to Machine Learning 2

 • Data are facts, numbers, or text that can be processed
by a computer
• Data Objects:
• Also called: samples, examples, instances, data points,
tuples
• Represent an entity of interest
• Examples:
What is Data? • Sales database: customers, store items, sales
transactions
• Medical database: patients, treatments, diagnoses
• University database: students, professors, courses
• Attributes:
• Also called: dimensions, features, variables
• Describe characteristics of data objects
• Database analogy: rows = objects, columns = attributes

© 2026 • Introduction to Machine Learning 3

 • A data field representing a characteristic or
feature of a data object
• Customer Object: John Smith
Attribute: Customer ID → "JS001"
Attribute: Name → "John Smith"
Understanding Attribute: Age → 35
Attribute: Income → $75,000
Attributes Attribute: Marital Status → "Married"
Attribute: Credit Rating → "Good“
• Attribute Vector (Feature Vector):
• A set of attributes used to describe a given object
• [JS001, John Smith, 35, 75000, Married, Good]

© 2026 • Introduction to Machine Learning 4

Types of Attributes

Attributes are The attribute type

classified based on determines which
the set of possible mathematical
values they can take. operations are valid!

© 2026 • Introduction to Machine Learning 5

© 2026 • Introduction to Machine Learning

Values represent categories, states, or "names of things" with no meaningful order

Properties:

Only equality comparison is meaningful

No ordering or ranking

No arithmetic operations possible

Nominal Also called "categorical" attributes

Examples:
Attributes • Hair color: {black, brown, blond, red, grey, white}
• Marital status: {single, married, divorced, widowed}
• Occupation: {teacher, engineer, doctor, artist}
• Zip codes: {90210, 10001, 60601}
• ID numbers: any unique identifiers
Statistical Operations Allowed:
• Mode, frequency distribution, chi-square test

Statistical Operations NOT Allowed:

• Mean, median, standard deviation

6
 A special case of nominal attributes with only two states or outcomes

Two Types:

1. Symmetric Binary:
Binary Attributes

• Both outcomes equally important

• No inherent preference
• Example: Gender (Male/Female), Smoker (Yes/No)

© 2026 • Introduction to Machine Learning

2. Asymmetric Binary:
• One outcome more important than the other
• Convention: assign "1" to the important outcome
• Example: Medical test (positive/negative), where positive is the important state
Representation:

Typically coded as 0 and 1

Important for distance calculations in ML algorithms

7
 • Values have a meaningful order (ranking) but the
magnitude between successive values is not known
• Properties:
• Order is important
• Relative position matters
• Differences between values are not uniform or
known
Ordinal Attributes • Examples:
• Size: {small, medium, large}
• Academic grades: {A, B, C, D, F}
• Military rank: {private, corporal, sergeant,
lieutenant}
• Customer satisfaction: {very unsatisfied,
unsatisfied, neutral, satisfied, very satisfied}
• T-shirt sizes: {XS, S, M, L, XL, XXL}

© 2026 • Introduction to Machine Learning 8

 • Statistical Operations Allowed:
• Median, percentiles, rank correlation
• Cannot compute mean meaningfully

Ordinal
• Why can't we compute mean?
• The difference between "small" and "medium"
Attributes may not equal the difference between "medium"
and "large"
• Example: Satisfaction ratings - is the gap between
"satisfied" and "very satisfied" the same as
between "neutral" and "satisfied"?

© 2026 • Introduction to Machine Learning 9

 • Quantity-based attributes with numerical values
• Two Subtypes:
1. Interval-Scaled:
• Measured on a scale with equal-sized units
• Order is meaningful
• NO true zero point (zero is arbitrary)

Numeric Attributes -
• Addition and subtraction are valid
• Multiplication and division are NOT meaningful
Interval vs. Ratio • Examples:
• Temperature in Celsius or Fahrenheit (0° doesn't mean
"no temperature")
• Calendar dates
• IQ scores
• Example: 100°C is not twice as hot as 50°C! We can
say it's 50° warmer, but not "twice as hot."

© 2026 • Introduction to Machine Learning 10

Numeric Attributes - Ratio-Scaled (continued)
2. Ratio-Scaled:
• Inherits all properties of interval scales
• HAS a true zero point
• All arithmetic operations are valid (addition, subtraction, multiplication, division)
• Ratios make sense
• Examples:
• Temperature in Kelvin (0K means absolute zero)
• Height, weight, length
• Age (0 years means birth)
• Counts (number of customers, sales quantity)
• Monetary quantities
• Example: A 6-foot person is 20% taller than a 5-foot person.
A baseball game lasting 3 hours is 50% longer than a game lasting 2 hours.

© 2026 • Introduction to Machine Learning 11

 • The attribute type determines:
 Which preprocessing techniques are appropriate
Attribute Types  Which distance measures can be used
(Continued)  What statistical analyses are valid
 How the attribute should be encoded for ML algorithms

Attribute Type Description Examples Operations

Nominal Categories, no order Zip codes, eye color =, ≠

Binary Two states Yes/No, Male/Female =, ≠

Ordinal Ordered categories Rankings, grades =, ≠, <, >

Interval Ordered, equal units Temperature (°C), dates +, -

Ratio True zero point Height, weight, age +, -, ×, ÷

© 2026 • Introduction to Machine Learning 12

Discrete vs.
Continuous
Attributes
 Why Data Preprocessing?
• Today’s real-world databases are highly susceptible to noisy, missing, and
inconsistent data due to their typically huge size and their likely origin from
multiple, heterogeneous sources.
• Low-quality data will lead to low-quality mining results,
• Data has quality if it satisfies the requirements of its intended use,
• There are many factors comprising data quality.
• These include: accuracy, completeness, consistency, timeliness, believability, and
interpretability,
• The methods for data preprocessing are organized into the following categories:
data cleaning, data integration, data reduction, and data transformation,
• These can help identify erroneous values and outliers, which will be useful in the
data cleaning and integration steps,
• Data processing techniques, when applied before mining, can substantially
improve the overall quality of the patterns mined and/or the time required for the
actual mining.

© 2026 • Introduction to Machine Learning 14

 ▪ Some data preparation is needed for all mining tools
▪ The purpose of preparation is to transform data
sets so that their information content is best
exposed to the mining tool
▪ Error prediction rate should be lower (or the same)
Why Data ▪
after the preparation as before it
No quality data no quality results!
Preprocessing? ▪ Quality decisions must be based on quality data
▪ ML algorithms required data at high quality
▪ Data need to be formatted for a given software tool
▪ Data need to be made adequate for a given method

© 2026 • Introduction to Machine Learning 15

Why Data Preprocessing?

Data in the real world is dirty

• Incomplete: lacking attribute values, lacking certain attributes of interest, or containing only aggregate data
• Noisy: containing errors or outliers that deviate from the expected
• Inconsistent: containing discrepancies in codes or names: lack of compatibility (e.g Some attributes representing a given concept may
have different names in different databases)

16
• Duplicate: Multiple records for same entity

No quality data  no quality mining results!

• Quality decisions must be based on quality data

• Data warehouse needs consistent integration of quality data

To minimize such problems, employ data cleaning routines.

• Before starting data preprocessing, it will be advisable to have overall picture of the data at high level summary such as
• General property of the data
• Which data values should be considered as noise or outliers
• This can be done with the help of descriptive data summarization

© 2026 • Introduction to Machine Learning

© 2026 • Introduction to Machine Learning

• Consequences of Dirty Data:

Biased or incorrect model
predictions
Why Data Misleading patterns and
Preprocessing? relationships
Wasted computational
resources
Poor business decisions

17
 ▪ Accuracy: How well does a piece of information
reflect reality? [correct/wrong]
▪ Completeness :Does it fulfill your expectations
of what’s comprehensive? [recorded/not]
Measures for data ▪ Consistency: Does information stored in one
place match
quality: relevant data stored elsewhere?

A ▪ Timeliness: Is your information available when

you need it?[Is data up-to-date?]

multidimensional ▪ Validity: Is information in a specific format,

does it follow business rules?
view ▪ Uniqueness: Is this the only instance in
which this information appears in the dataset?
▪ Believability: how trustable the data are correct?
▪ Interpretability: how easily the data can be
understood?

© 2026 • Introduction to Machine Learning 18

© 2026 • Introduction to Machine Learning

Multi-Dimensional Measure of
Data Quality
• Broad Categories:
Intrinsic (accuracy, believability)
Contextual (timeliness, completeness)
Representational (interpretability)
Accessibility (can we get to the data?)

19
 Major Tasks in Data Preprocessing

Data cleaning Data integration Data transformation Data reduction

Fill in missing values, Integration of multiple Normalization (scaling to a Obtains reduced
smooth noisy data, identify databases, data cubes, files, specific range) representation in volume
or remove outliers, and or notes Aggregation but produces the same or
resolve inconsistencies similar analytical results
Data discretization: with
particular importance,
especially for numerical
data
Data aggregation,
dimensionality reduction,
data compression,
generalization

© 2026 • Introduction to Machine Learning 20

Major Tasks in Data
Preprocessing

• Data Preparation as a step in the

Knowledge Discovery Process

© 2026 • Introduction to Machine Learning 21

 DATA
CLEANING
 Missing Data

Data is not always available! Missing data may be due to:

• Equipment Malfunction
• Sensor failure
• Network interruption
• Power outage during data collection
• Data Entry Problems
• Operator forgot to enter value
• Misunderstanding of data entry requirements
• System didn't enforce completeness
• Data Deletion
• Data inconsistent with other records and thus deleted
• Privacy concerns (data removed intentionally)
• Not Considered Important
• Certain data not seen as necessary at entry time
• Later becomes important but is missing
• Historical Changes
• Changes over time not recorded
• Only current value kept, history lost

Example: Sales database with missing customer income - many customers may refuse to provide income information

© 2026 • Introduction to Machine Learning 23

Missing Data

Missing data may need to be inferred

There are always MVs in a real dataset, which may have

an impact on modelling, in fact, they can destroy it!

Some tools ignore missing values, others use some

metric to fill in replacements
© 2026 • Introduction to Machine Learning 24
 How to Handle Missing Data
1. Ignore the Tuple
• Remove records with missing values
• Simple but potentially wasteful
• When to use: Class label is missing (classification tasks)
• When NOT to use: High percentage of missing values, missing values vary by
attribute
2. Fill In Manually
• Human reviews and fills missing values
• Pros: Accurate if done correctly
• Cons: Tedious, time-consuming, often infeasible for large datasets
3. Use a Global Constant
• Fill with "unknown", "N/A", or 0
• Pros: Simple, fast
• Cons: Can create artificial patterns; the constant may be interpreted as a meaningful
value by the model
• Example: Filling all missing incomes with 0 - now 0 has special meaning!

© 2026 • Introduction to Machine Learning 25

 How to Handle Missing Data
4. Use Attribute Mean/Median
• Fill with the mean (for symmetric distributions) or median (for skewed distributions)
• Pros: Simple, preserves overall mean
• Cons: Reduces variance, ignores relationships with other attributes
• Example: Fill missing Age with average age of all customers
5. Use Mean for Same Class
• Fill with mean of all samples belonging to the same class
• Pros: More accurate than global mean, preserves class distinctions
• Cons: Requires class labels, still ignores other attributes
• Example: Fill missing Age with average age of customers in same income bracket
6. Use Most Probable Value
• Predict missing value using regression, Bayesian inference, or decision trees
• Pros: Most sophisticated, uses relationships among attributes
• Cons: Complex, may introduce bias if prediction model is flawed
• Example: Predict income based on education, occupation, and age

© 2026 • Introduction to Machine Learning 26

Noisy Data - What is Noise?
Noise is random error or variance in a measured variable

Noise refers to modification of original values

Examples: distortion of a person’s voice when talking on a poor phone and “snow” on television screen

Incorrect attribute values may be due to

• faulty data collection instruments
• data entry problems
• data transmission problems
• technology limitation
• inconsistency in naming convention

Other data problems which requires data cleaning

• duplicate records
• incomplete data
• inconsistent data

© 2026 • Introduction to Machine Learning 27

 How to Handle Noisy Data?
Binning method:

• first sort data and partition into (equi-depth) bins

• then one can smooth by bin means, smooth by bin median, smooth by bin boundaries, etc.

Clustering

• Detect and remove outliers

Combined computer and human inspection

• Detect suspicious values and check by human

Regression

• Smooth by fitting the data into regression functions

© 2026 • Introduction to Machine Learning 28

 • Equal-width (distance) partitioning:
It divides the range into N intervals of
equal size: uniform grid
if A and B are the lowest and highest
Simple values of the attribute, the width of
Discretization intervals will be: W = (B-A)/N.
The most straightforward
Methods: Binning But outliers may dominate
presentation
Skewed data is not handled well.

© 2026 • Introduction to Machine Learning 29

 Simple Discretization
Methods: Binning

• Equal-depth (frequency) partitioning:

It divides the range into N intervals,
each containing approximately same
number of samples
Good data scaling
Managing categorical attributes can be
tricky.

© 2026 • Introduction to Machine Learning 30

Binning Methods for Data
Smoothing
• Original Data (Sorted prices in dollars): 4, 8, 9, 15, 21, 21, 24, 25, 26, 28, 29, 34

1. Partition into 3 equal-depth bins

• Bin 1: 4, 8, 9, 15
• Bin 2: 21, 21, 24, 25
• Bin 3: 26, 28, 29, 34

2. Step 2: Smoothing by Bin Means

• Bin 1 mean = (4+8+9+15)/4 = 36/4 = 9 → Bin 1: 9, 9, 9, 9
• Bin 2 mean = (21+21+24+25)/4 = 91/4 = 22.75 ≈ 23 → Bin 2: 23, 23, 23, 23
• Bin 3 mean = (26+28+29+34)/4 = 117/4 = 29.25 ≈ 29 → Bin 3: 29, 29, 29, 29

3. Step 3: Smoothing by Bin Boundaries

• Identify min and max in each bin
• Replace each value with nearest boundary
• Bin 1: boundaries 4 and 15 → 4,4,4,15
• Bin 2: boundaries 21 and 25 → 21,21,25,25
• Bin 3: boundaries 26 and 34 → 26,26,26,34

© 2026 • Introduction to Machine Learning 31

 Noisy Data - Regression Method
• Linear regression
• Best line to fit two variables
• Find line that best fits the data
• Replace actual values with predicted values from the
regression line
• Multiple linear regression
• More than two variables
• Fit to a multidimensional surface
• Advantages:
• Uses relationship between variables
• Can handle missing values by prediction
• Smooths out random variation

© 2026 • Introduction to Machine Learning 32

 Noisy Data - Clustering Method
• Group similar objects to detect outliers
• Process:
1. Partition data into clusters of similar objects
2. Points far from any cluster are potential outliers
3. Remove or adjust outliers
• Advantages:
• Detects outliers in multidimensional space
• Doesn't assume any functional form
• Can identify clusters of outliers
• Disadvantages:
• Choice of clustering algorithm affects results
• Parameter selection (number of clusters) can be challenging

© 2026 • Introduction to Machine Learning 33

 How to Handle Inconsistent Data?

Manual correction using external Semi-automatic using various tools

references
To detect violation of known functional dependencies
and data constraints
To correct redundant data

© 2026 • Introduction to Machine Learning 34

 Outlier

• Outliers are data objects with characteristics

that are considerably different than most of
the other data objects in the data set
• Legitimate but extreme values

© 2026 • Introduction to Machine Learning 35

FEATURE ENGINEERING & REPRESENTATION
• Why Feature Engineering?
• The process of transforming raw data into features that better represent the underlying problem to the
predictive models
• Why It's Critical:
 ML algorithms only understand numbers
 Raw data often contains non-numeric or unscaled values
 Proper representation can dramatically improve model performance
 Feature engineering often separates good models from great ones

© 2026 • Introduction to Machine Learning 36

 Categorical Encoding - Why Needed?
• ML algorithms work with numbers, not categories
• Categories like "Red", "Blue", "Green" must be converted to numbers
• Naive conversion (Red=1, Blue=2, Green=3) implies an order that doesn't
exist
Color Wrong Encoding (Numeric Mapping)
Red 1
Blue 2
Green 3

• Problem:
The model thinks Green > Blue > Red.
It assumes an order and magnitude that does not actually exist.
• Two Main Solutions:
Label Encoding (for ordinal data)
One-Hot Encoding (for nominal data)

© 2026 • Introduction to Machine Learning 37

Label Encoding
• Mapping each category to a different integer while preserving order
• Use when dealing with ordinal attributes where the order of values
matters.
• How It Works:
Assign integers preserving the natural order
Larger integer = higher rank/order
• Advantages: Simple, memory-efficient (one column instead of many)
• Disadvantages: Implies equal spacing between categories (may not be
true)

© 2026 • Introduction to Machine Learning 38

One-Hot Encoding
• Creating binary columns for each category, with exactly one "1" per row
• When to Use: Nominal attributes with no order
• How It Works:
 For a categorical variable with k categories, create k new binary variables
 Each new variable represents one category
 For each record, set exactly one variable to 1 (the category it belongs to)
 All others set to 0
• Important: For k categories, use only k-1 binary variables to avoid multicollinearity (dummy variable trap)

© 2026 • Introduction to Machine Learning 39

 One-Hot Encoding - Example
Before Encoding (Original Data):
CustomerID Color Size City

1 Red M New York

2 Blue L Boston

3 Green S New York

4 Red XL Chicago

After One-Hot Encoding (for Color and City):

CustomerID Size Color_Red Color_Blue Color_Green City_NY City_Boston City_Chicago

1 M 1 0 0 1 0 0

2 L 0 1 0 0 1 0

3 S 0 0 1 1 0 0

4 XL 1 0 0 0 0 1

© 2026 • Introduction to Machine Learning 40

One-Hot Encoding - Pros and Cons

Advantages:
• No implied order between categories
• Preserves all information
• Works well with linear models

Disadvantages:
• Increases dimensionality (k columns instead of 1)
• Can cause "curse of dimensionality" for high-cardinality features
• Sparse representation (mostly zeros)
• Can lead to multicollinearity if all k columns are used

© 2026 • Introduction to Machine Learning 41

Numeric Scaling - Why Needed?
• Attributes often have different units and scales
• Algorithms using distance measures (KNN, SVM, K-means) are sensitive to scale
• Attributes with larger ranges dominate distance calculations
Person Age (years) Income ($)

A 25 50,000

B 35 51,000

• KNN algorithm calculating distance between A and B:

• Age difference: |25-35| = 10
• Income difference: |50,000-51,000| = 1,000
• Total distance ≈ √(10² + 1000²) ≈ 1000
• Income contributes 99.99% of the distance! Age barely matters.
• With scaling: Both features contribute equally.

© 2026 • Introduction to Machine Learning 42

 Min-Max Normalization

• Scales all values to a fixed range, usually [0, 1]

• Scaled value = (original - min) / (max - min)
• Example with Age (range 25-42):

Original Calculation Scaled

25 (25-25)/(42-25) = 0/17 0.00

30 (30-25)/(42-25) = 5/17 0.29

35 (35-25)/(42-25) = 10/17 0.59

42 (42-25)/(42-25) = 17/17 1.00

• Now both Age and Income are between 0 and 1 → They contribute equally to distance
calculations!
• When to use: When you know the min/max values and want data in a specific range.

© 2026 • Introduction to Machine Learning 43

Z-Score Normalization (Standardization)
• Centers data around 0 with standard deviation of 1
• Scaled value = (original - mean) / standard deviation
• How many standard deviations above/below the mean?
• Most common choice; works well when data has outliers.
• Example with Age (mean=32, standard deviation=6):

Original Calculation Scaled Meaning

26 (26-32)/6 = -6/6 -1.0 1 SD below mean

32 (32-32)/6 = 0/6 0 Exactly at mean

38 (38-32)/6 = 6/6 1.0 1 SD above mean

44 (44-32)/6 = 12/6 2.0 2 SD above mean

© 2026 • Introduction to Machine Learning 44

 Min-Max vs Z-Score - Simple

Aspect Min-Max Z-Score

Result range Fixed [0,1] No fixed range, can be negative

After scaling mean Not necessarily 0 Exactly 0

After scaling SD Not necessarily 1 Exactly 1

Sensitive to outliers? Yes Less sensitive

Neural networks, when you need Most ML algorithms (KNN, SVM,

When to use
bounded range Linear Regression)

© 2026 • Introduction to Machine Learning 45

 • Sampling: obtaining a small sample s to represent
the whole data set N
• Allow a mining algorithm to run in complexity that
is potentially sub-linear to the size of the data

Sampling • Key principle: Choose a representative subset of the

data
Simple random sampling may have very poor
performance in the presence of skew
Develop adaptive sampling methods, e.g.,
stratified sampling

© 2026 • Introduction to Machine Learning 46

 Types of Sampling
Simple random sampling

• There is an equal probability of selecting any particular item

Sampling without replacement

• Once an object is selected, it is removed from the population

Sampling with replacement

• A selected object is not removed from the population

Stratified sampling:

• Partition the data set, and draw samples from each partition (proportionally, i.e., approximately the
same percentage of the data)
• Used in conjunction with skewed data

© 2026 • Introduction to Machine Learning 47

 Text Representation
• Machine Learning algorithms cannot understand raw text directly. We
need to convert text into numbers.
• Two Fundamental Approaches:
Aspect Bag-of-Words (BoW) TF-IDF

Count of words in Weighted importance

Core Concept
document of words

Word
What it captures Word distinctiveness
presence/frequency

Handles common No - treats all words Yes - reduces their

words? equally impact

© 2026 • Introduction to Machine Learning 48

 • A method to convert text into numbers by counting
words
• Ignores grammar, word order, and sentence structure
completely

Bag-of-Words • Treats each document as a "bag" of individual words

• It works by:
(BoW)  Creating a vocabulary of all unique words
across all documents
 For each document, counts how many times
each word appears
 Results in a vector where each position
represents one word's count

© 2026 • Introduction to Machine Learning 49

Bag-of-Words (BoW)

• Key characteristics:
 Simple and easy to understand
 Every word is treated equally important
 Common words like "the" and "is" dominate
the counts
 Produces very large, mostly-zero vectors
(sparse)
• When to use:
 Small datasets where simplicity matters
 Learning the basics of text representation
 Quick prototyping before trying advanced
methods

© 2026 • Introduction to Machine Learning 50

TF-IDF (Term Frequency - Inverse Document Frequency)

An improved version of Bag-of-Words that weights words by importance

Combines two ideas: how often a word appears in a document, and how rare it is across all documents

Words that appear frequently in ONE document are important FOR that document

Words that appear in MANY documents are NOT distinctive

TF-IDF balances both factors automatically

How it works:
• Starts with word counts just like Bag-of-Words
• Then reduces the weight of words that appear everywhere
• Increases the weight of words that are unique to specific documents
• Common stop words naturally get weighted down to near zero

© 2026 • Introduction to Machine Learning 51

 TF-IDF (Term Frequency - Inverse
Document Frequency)

• Key characteristics:
 Automatically handles common words without
needing a stop word list
 Highlights words that make each document unique
 Still ignores word order, but captures word
importance better
 More informative representation than simple counts
• When to use:
 Most text classification tasks (spam detection,
sentiment analysis)
 Search engines and document retrieval
 Large datasets with varied vocabulary
 Any situation where distinguishing documents
matters

© 2026 • Introduction to Machine Learning 52

Dataset preparation for
Classification
• Proper procedure in some
classification system
development involves three sets
of data :
o Training set
o Validation set
o Test set
• Generally, the larger the
training data the better the
classifier

© 2026 • Introduction to Machine Learning 53

© 2026 • Introduction to Machine Learning

Unbalanced Data
• Sometimes, classes have very unequal frequency
 medical diagnosis: 90% healthy, 10% disease
 eCommerce: 99% don’t buy, 1% buy
• Majority class classifier can be 97% correct, but useless
• If we have two classes that are very unbalanced, then it will be a bias to
evaluate our classifier method
• With two or more classes, a good approach to make a balance between the
class instances is to build BALANCED train and test sets. 54
 Building
Classification
Model

© 2026 • Introduction to Machine Learning 55

 Tips: Dataset size

Before we start building Classification model, we should check how

good is the size of the dataset we have
Given balanced dataset, the next most important aspect of goodness is
size of the data set
The model should be able to converge during learning the parameters
from the dataset
If not, appropriate measure should be taken and care must
be given while reporting performance
We will see learning curve analysis that best suit to detect goodness of
the size of the training dataset
© 2026 • Introduction to Machine Learning 56
Tips: Dataset size
 What to do with small data?
• Having small data but balanced can be approached in
different ways to relay on the performance
• Note that the total data set we have will be divided into
three for training, testing and validation
• The following are the techniques to minimize the effect
of the dataset size
• k-fold cross validation: randomly dividing the
set of observations into k groups, or folds, of
approximately equal size. The first fold is treated
as a test set, and the method is fit on the remaining
k − 1 folds.
• Data augmentation: techniques used to increase
the amount of data by adding slightly modified
copies of already existing data
• What to do with small data: Using K-fold cross
validation 10-fold is the recommended

© 2026 • Introduction to Machine Learning 57