DATA MINING

MODULE II
Data Preprocessing

Complete Study Booklet

Based on: Han, Pei & Tong — Data Mining: Concepts and Techniques, 4th Edition
Covers: Data Types • Statistics • Similarity & Distance • Data Cleaning • Integration •
Transformation • Dimensionality Reduction
Table of Contents
Chapter 1: Data, Measurements, and Data Preprocessing
Chapter 2: Types of Data (Nominal, Ordinal, Interval, Ratio, Discrete, Continuous)
Chapter 3: Statistics of Data (Central Tendency, Dispersion, Covariance, Correlation)
Chapter 4: Similarity and Distance Measures
Chapter 5: Data Quality and Data Cleaning
Chapter 6: Data Integration
Chapter 7: Data Transformation (Normalization, Discretization, Aggregation)
Chapter 8: Dimensionality Reduction (PCA, Attribute Subset Selection, Nonlinear Methods)
BONUS: Formula Sheet • Comparison Tables • 20 Exam Questions • Viva Questions
Chapter 1: Data, Measurements, and Data
Preprocessing

1.1 What Is Data Mining? — Context

Data mining is the process of discovering interesting patterns and knowledge from massive
amounts of data. Before any mining can happen, raw data must be understood and prepared —
this is the role of data preprocessing.
According to Han et al. (2023), the typical KDD (Knowledge Discovery in Databases) process
involves: data cleaning → data integration → data selection → data transformation → pattern
discovery → evaluation → knowledge presentation.

1.2 Why Is Data Preprocessing Required?

Real-world data is characterised by three unavoidable problems:
• Incompleteness: Attribute values may be missing (e.g., customer income not recorded),
attributes of interest may be absent, or only aggregate data may be available.
• Noise: Data may contain errors or outliers — values that deviate from expected
distributions due to faulty instruments, human error, or transmission issues.
• Inconsistency: Discrepancies in data can arise from different naming conventions, data
codes, formats, or from integration of multiple sources.

Low-quality input data leads to low-quality mining results — famously expressed as 'garbage in,
garbage out'. Preprocessing addresses these issues systematically.

🔍 Key Concept: The Four Preprocessing Tasks

1. Data Cleaning — fill missing values, smooth noise, handle outliers, resolve inconsistencies
2. Data Integration — merge data from multiple heterogeneous sources into a coherent store
3. Data Transformation — normalise, discretise, aggregate, or otherwise reshape data
4. Dimensionality Reduction — reduce the number of attributes while preserving information

⚠️ Common Mistake
Students often skip preprocessing and jump to mining. This produces unreliable models.
Preprocessing can account for 60–80% of the total effort in a real data mining project.

🧠 Memory Trick
Remember the 4 Ds: Dirty → Clean → Different Sources → Merge → Different Range →
Transform → Too Many Features → Reduce.
Chapter 2: Types of Data

2.1 Data Objects and Attributes

A data set is made up of data objects. A data object represents an entity (e.g., a customer, a
product, a student). Data objects are described by attributes. An attribute is a data field
representing a characteristic or feature. The terms attribute, dimension, feature, and variable
are often used interchangeably.
The type of an attribute is determined by the set of possible values it can have. Han et al.
identify four main attribute types: Nominal, Binary, Ordinal, and Numeric (which includes Interval
and Ratio).

2.2 Nominal Attributes

Definition
Nominal means 'relating to names'. The values of a nominal attribute are symbols or names that
represent categories, codes, or states. There is NO meaningful ordering among values.
Also known as: Categorical attributes

Examples
• hair_color: black, brown, blond, red, auburn, gray, white
• marital_status: single, married, divorced, widowed
• occupation: teacher, dentist, programmer, farmer
• map_color: red, yellow, green, pink, blue

Mathematical Properties
No mathematical operations (mean, difference) are meaningful. Only mode (most common
value) is a valid central tendency measure.
Dissimilarity: d(i,j) = (p - m) / p
where m = number of matching attributes, p = total attributes.

⚠️ Common Mistake
Integers can represent nominal values (e.g., 0 = black, 1 = brown). This does NOT make them
numeric — arithmetic on these integers is meaningless!
2.3 Binary Attributes
Definition
A binary attribute is a nominal attribute with exactly two states: 0 and 1, where 0 = absent and 1
= present. When states correspond to true/false, they are called Boolean.

Symmetric vs Asymmetric
Type Definition Example
Symmetric Both states equally important; no gender (male/female)
preference for 0 or 1
Asymmetric One state (usually 1, the rarer one) is HIV test
more important (positive/negative)

Similarity Formula for Asymmetric Binary

Using a 2x2 contingency table where q = both 1, r = i=1/j=0, s = i=0/j=1, t = both 0:
Jaccard Coefficient: sim(i,j) = q / (q + r + s)
Asymmetric Dissimilarity: d(i,j) = (r + s) / (q + r + s)
Note: The Jaccard coefficient ignores t (negative-negative matches) since they are
uninformative in asymmetric settings.

2.4 Ordinal Attributes

Definition
An ordinal attribute has possible values with a meaningful order or ranking, but the magnitude
(distance) between successive values is unknown.

Examples
• drink_size: small < medium < large (but we don't know HOW MUCH larger)
• grade: F < D < C < B < A
• customer_satisfaction: 1 (very dissatisfied) < 2 < 3 < 4 < 5 (very satisfied)
• professional_rank: assistant < associate < full professor

Valid Statistical Measures

Mode and Median are valid. Mean cannot be defined (no equal intervals).
Dissimilarity Calculation
Step 1: Replace each value xif with its rank rif ∈ {1, …, Mf}
Step 2: Normalise rank to [0,1]:
z_if = (r_if - 1) / (M_f - 1)
Step 3: Apply any numeric distance measure (e.g., Euclidean) to z values.

2.5 Numeric Attributes

A numeric attribute is quantitative — it represents a measurable quantity. There are two
subtypes:

2.5.1 Interval-Scaled Attributes

Measured on a scale with equal-size units. Values can be positive, zero, or negative.
Differences between values are meaningful. NO true zero-point — ratios are NOT meaningful.
• Examples: Temperature in Celsius/Fahrenheit, calendar years, IQ scores
• Key insight: 10°C is NOT twice as warm as 5°C because 0°C does not mean 'no
temperature'
Valid measures: Mean, Median, Mode (all central tendency), plus all dispersion measures.

2.5.2 Ratio-Scaled Attributes

Numeric attribute with an inherent TRUE ZERO-POINT. Ratios between values ARE
meaningful.
• Examples: Temperature in Kelvin, height, weight, speed, age, income, years of
experience
• Key insight: $100 is exactly 100 times $1 (ratio meaningful). 200K is twice as hot as
100K.
Valid measures: All statistical measures including ratios.

2.6 Discrete vs. Continuous Attributes

This is an alternative classification scheme used especially in machine learning:
Type Definition Examples
Discrete Finite or countably infinite set of values hair_color, age (0–110), zip code,
(may or may not be integers) customer_ID
Continuous Real-valued (not discrete); represented as temperature, weight, height,
floating-point numbers income
 🧠 Memory Trick — NOIR Scale
N = Nominal: Names only (categories, no order)
O = Ordinal: Order exists but gaps unknown
I = Interval: Equal gaps, no true zero
R = Ratio: Equal gaps + true zero (most powerful)
Each level ADDS a property to the previous level.

📊 Grand Comparison: Nominal vs Ordinal vs Interval vs Ratio

Property Nominal Ordinal Interval Ratio
Has distinct categories Yes Yes Yes Yes
Has meaningful order No Yes Yes Yes
Equal intervals between No No Yes Yes
values
Has absolute/true zero No No No Yes
Ratios meaningful No No No Yes
Mode valid Yes Yes Yes Yes
Median valid No Yes Yes Yes
Mean valid No No Yes Yes
Arithmetic operations No No Addition/ All
Subtraction

📝 Quick Revision Notes — Data Types

Nominal: hair colour, nationality — mode only
Ordinal: grades, satisfaction ratings — mode + median
Interval: temperature (°C/°F), calendar years — mean/median/mode, no ratios
Ratio: height, weight, income, Kelvin — all statistics valid including ratios
Binary: special case of nominal (2 states); symmetric vs asymmetric distinction matters
Discrete: finite/countable values; Continuous: real-valued floating point
Chapter 3: Statistics of Data
Basic statistical descriptions give us an overall picture of our data. They help identify the
'centre', the 'spread', and the relationships between attributes — critical inputs to data cleaning
and preprocessing decisions.

3.1 Measures of Central Tendency

3.1.1 Mean (Arithmetic Average)
The most common measure of the centre of a numeric distribution.
x̄ = (x₁ + x₂ + ... + xₙ) / N = (1/N) Σᵢ xᵢ
Weighted Mean (when values have different importances):
x̄ _w = (w₁x₁ + w₂x₂ + ... + wₙxₙ) / (w₁ + w₂ + ... + wₙ)

Numerical Example: Salaries (in $000): 30, 36, 47, 50, 52, 52, 56, 60, 63, 70, 70, 110
x̄ = (30+36+47+50+52+52+56+60+63+70+70+110) / 12 = 696/12 =
$58,000

⚠️ Limitation of Mean
The mean is highly sensitive to outliers. A single extreme value can distort it significantly.
Example: If one executive earns $500K, the mean salary appears much higher than what most
employees earn.
Solution: Use trimmed mean (drop top and bottom k%) or switch to median for skewed data.

3.1.2 Median
The middle value in a sorted set. It separates the upper half from the lower half of the
distribution. More robust to outliers than the mean.
Rule: If N is odd → median = middle value. If N is even → median = average of two middle
values.

Example (same salaries, N=12, sorted): 30, 36, 47, 50, 52, 52 | 56, 60, 63, 70, 70, 110
Median = (52 + 56) / 2 = $54,000

For grouped/frequency data, median can be approximated by interpolation:

Median ≈ L₁ + [ (N/2 - Σfreq_l) / freq_median ] × width
where L₁ = lower boundary of median interval, freq_l = cumulative frequency below interval,
freq_median = frequency of median interval, width = interval width.

3.1.3 Mode
The value that appears most frequently in the data. Can be used for both qualitative and
quantitative attributes. Data with one mode = unimodal; two modes = bimodal; three = trimodal.
Example: In the salary data, modes are $52,000 and $70,000 (each appears twice). → Bimodal.

Empirical relation for moderately skewed unimodal data:

mean - mode ≈ 3 × (mean - median)

3.1.4 Midrange
A simple measure: average of the maximum and minimum values.
Midrange = (max + min) / 2
Example: Midrange = (30,000 + 110,000) / 2 = $70,000

3.1.5 Symmetry and Skewness

In a perfectly symmetric unimodal distribution: mean = median = mode.
• Positively Skewed (right-skewed): Long tail to the right. mode < median < mean
• Negatively Skewed (left-skewed): Long tail to the left. mean < median < mode

🧠 Memory Trick — Skewness Direction

The MEAN is 'pulled' toward the tail (the extreme values). If the tail is to the RIGHT, the mean >
median → positive skew.

3.2 Measures of Data Dispersion

3.2.1 Range
The simplest measure of spread.
Range = max(x) - min(x)
Disadvantage: Only uses two values; very sensitive to outliers.

3.2.2 Quantiles, Quartiles, and IQR

Quantiles are data points that divide a distribution into equal-size groups:
• 2-quantile (median): Splits data into 2 equal halves
 • 4-quantiles (quartiles Q₁, Q₂, Q₃): Split data into 4 equal quarters
• 100-quantiles (percentiles): Split data into 100 equal parts

Q₁ = 25th percentile (cuts lowest 25%). Q₂ = 50th percentile (median). Q₃ = 75th percentile
(cuts lowest 75%).
IQR = Q₃ - Q₁
Numerical Example (same salary data): Q₁ = $48,500, Q₃ = $66,500
IQR = 66,500 - 48,500 = $18,000

Outlier detection rule: values more than 1.5 × IQR above Q₃ or below Q₁ are suspected
outliers.
Outlier boundaries: [Q₁ - 1.5×IQR, Q₃ + 1.5×IQR]

3.2.3 Five-Number Summary and Boxplot

The five-number summary provides a concise description of a distribution:
Five-Number Summary: {Minimum, Q₁, Median (Q₂), Q₃, Maximum}
A boxplot visualises this summary: The box spans Q₁ to Q₃ (length = IQR), a line inside marks
the median, and whiskers extend to the extreme observations within 1.5×IQR of the quartiles.
Values beyond the whiskers are plotted as individual points (potential outliers).

3.2.4 Variance and Standard Deviation

Variance measures how far values are spread around the mean. Standard deviation is its
square root (same units as the data).
Population Variance: σ² = (1/N) Σᵢ (xᵢ - x̄ )²
Standard Deviation: σ = √σ²
Sample Variance (used when data is a sample from a larger population):
Sample Variance: s² = (1/(n-1)) Σᵢ (xᵢ - x̄ )²

Numerical Example: For the salary data, x̄ = 58,000:

σ² ≈ (1/12)[(30-58)² + (36-58)² + ... + (110-58)²] ≈ 379.17 (in
$000²)
σ ≈ √379.17 ≈ 19.47 (i.e., ≈ $19,470)

⚠️ Common Mistake: Population vs Sample Variance

Population variance divides by N (use when you have ALL data).
Sample variance divides by n-1 (Bessel's correction; use when data is a sample).
The n-1 corrects for the fact that the sample mean underestimates variability.
3.3 Covariance and Correlation
3.3.1 Covariance
Covariance measures how two attributes change together.
Cov(A,B) = E[(A - Ā)(B - B̄ )] = (1/n) Σᵢ (aᵢ - Ā)(bᵢ - B̄ )
Equivalently: Cov(A,B) = E(A·B) - Ā·B̄
• Positive Cov: A and B increase together
• Negative Cov: One increases as the other decreases
• Zero Cov: Likely independent (though zero covariance doesn't always mean
independence)

3.3.2 Pearson's Correlation Coefficient

Correlation normalises covariance by dividing by the product of standard deviations:
r(A,B) = Cov(A,B) / (σA × σB) = [Σaᵢbᵢ - nĀB̄ ] / (nσAσB)
Range: -1 ≤ r ≤ +1
• r > 0: Positive correlation (as A increases, B increases)
• r = 0: No linear correlation (may still be nonlinear relationship)
• r < 0: Negative correlation (as A increases, B decreases)
• r = ±1: Perfect linear correlation

⚠️ Critical Distinction
Correlation does NOT imply causation. Two attributes may correlate due to a third confounding
variable (e.g., number of hospitals and car thefts both correlate with population size, not with
each other causally).

3.3.3 Chi-Square (χ²) Test for Nominal Data

For nominal attributes A and B, correlation is assessed using the chi-square statistic:
χ² = ΣΣ (oᵢⱼ - eᵢⱼ)² / eᵢⱼ
where oᵢⱼ = observed frequency, eᵢⱼ = expected frequency = count(A=aᵢ)×count(B=bⱼ)/n
If χ² exceeds the critical value at the chosen significance level with (r-1)(c-1) degrees of
freedom, we reject the independence hypothesis → A and B are correlated.

3.4 Skewness
Skewness quantifies the asymmetry of a distribution. While the textbook doesn't give a separate
skewness formula, the following empirical relationship holds for moderately skewed data:
 Pearson's Skewness ≈ (mean - mode) / σ OR ≈ 3(mean - median) /
σ
• Positive skewness: Right-skewed; long right tail; mean > median > mode. Common in
income data.
• Negative skewness: Left-skewed; long left tail; mean < median < mode. Common in
age at retirement.
• Zero skewness: Symmetric distribution (e.g., normal distribution).

📝 Quick Revision Notes — Statistics

Central Tendency: Mean (sensitive to outliers), Median (robust), Mode (most frequent), Midrange
(simple)
Dispersion: Range (simple), IQR = Q3-Q1 (robust), Variance σ² (spread around mean), SD σ =
√variance
Outlier rule: outside Q1 - 1.5×IQR or Q3 + 1.5×IQR
Correlation: r in [-1,1]; Covariance normalised by σA×σB
For skewed data, always prefer Median over Mean
Chapter 4: Similarity and Distance Measures
In data mining tasks such as clustering, outlier detection, and nearest-neighbour classification,
we need to quantify how alike or unlike two objects are. These are called proximity measures.

Similarity: Higher value = more similar (typically 0 = completely different, 1 = identical)

Dissimilarity / Distance: Lower value = more similar (0 = identical; higher = more different)
General relationship: sim(i,j) = 1 - d(i,j) (for normalised
measures)

4.1 Euclidean Distance

Definition
The straight-line ('as the crow flies') distance between two points in p-dimensional space.
d(i,j) = √[ (xi1-xj1)² + (xi2-xj2)² + ... + (xip-xjp)² ]
For two dimensions: d = √[(x₁-x₂)² + (y₁-y₂)²]

Intuition
Imagine placing two data points as coordinates on a map. The Euclidean distance is the
shortest direct path (a straight line) between them — like flying distance.

Numerical Example
x₁ = (1, 2) and x₂ = (3, 5)
d = √[(3-1)² + (5-2)²] = √[4 + 9] = √13 ≈ 3.61

Properties (Metric Properties)

• Non-negativity: d(i,j) ≥ 0
• Identity: d(i,i) = 0
• Symmetry: d(i,j) = d(j,i)
• Triangle inequality: d(i,j) ≤ d(i,k) + d(k,j)
4.2 Manhattan Distance
Definition
Also called 'city block' or 'taxicab' distance. The distance measured as the sum of the absolute
differences along each dimension — like navigating city blocks.
d(i,j) = |xi1-xj1| + |xi2-xj2| + ... + |xip-xjp|

Intuition
Imagine you can only walk horizontally or vertically (like on a grid of city streets), not diagonally.
The Manhattan distance is the total blocks walked.

Numerical Example
x₁ = (1, 2) and x₂ = (3, 5)
d = |3-1| + |5-2| = 2 + 3 = 5

4.3 Minkowski Distance

Definition
A generalisation of both Euclidean and Manhattan distances, controlled by parameter h (also
called Lh norm or Lp norm in literature):
d(i,j) = ʰ√[ |xi1-xj1|ʰ + |xi2-xj2|ʰ + ... + |xip-xjp|ʰ ]

Value of h Distance Type Formula Equivalent

h=1 Manhattan Distance (L1 norm) Sum of absolute differences
h=2 Euclidean Distance (L2 norm) Straight-line distance
h→∞ Supremum / Chebyshev Distance Maximum difference across any
(L∞ norm) dimension

Supremum Distance
Found by identifying the attribute with the maximum difference:
d(i,j) = max_f |xif - xjf|
Example: x₁=(1,2), x₂=(3,5) → max{|3-1|, |5-2|} = max{2,3} = 3

🧠 Memory Trick — Minkowski

'ManhattAN (h=1), EuclideAN (h=2), SuPrEmum (h=∞)'. Or: 'h increases → distance focuses on
 the LARGEST single-dimension gap.'

4.4 Cosine Similarity

Definition
Measures the cosine of the angle between two vectors. Used primarily for document similarity,
text mining, and sparse high-dimensional data.
sim(x,y) = (x · y) / (||x|| × ||y||)
where x·y = dot product = Σ xᵢyᵢ, and ||x|| = Euclidean norm = √(Σ xᵢ²)

Intuition
Two documents pointing in similar directions (similar topic distributions) will have a small angle
between them → cos θ close to 1 → high similarity. Perpendicular documents (cos θ = 0) share
nothing in common.

Why Not Regular Distance for Sparse Data?

Term-frequency vectors are sparse (many zeros). Two documents may share many 'zero'
entries simply because those words don't appear in either document, but that doesn't make
them similar. Cosine similarity focuses only on words that DO appear.

Numerical Example
Doc1 = x = (5, 0, 3, 0, 2, 0, 0, 2, 0, 0), Doc2 = y = (3, 0, 2, 0, 1, 1, 0, 1, 0, 1)
x·y = 5×3 + 0×0 + 3×2 + 2×1 + 2×1 = 25
||x|| = √(25+0+9+0+4+0+0+4+0+0) = √42 ≈ 6.48
||y|| = √(9+0+4+0+1+1+0+1+0+1) = √17 ≈ 4.12
sim(x,y) = 25 / (6.48 × 4.12) ≈ 0.94 → Very similar documents

Non-metric Nature
Cosine similarity does NOT satisfy all metric properties (e.g., triangle inequality). It is a non-
metric similarity measure.
4.5 Jaccard Similarity
Definition
The Jaccard coefficient measures similarity between two sets or two asymmetric binary attribute
vectors. It focuses on positive matches (both attributes = 1) while ignoring negative matches
(both = 0).
Jaccard Similarity: sim(i,j) = q / (q + r + s)
Jaccard Dissimilarity: d(i,j) = (r+s) / (q+r+s)
where q = #attributes where both i and j = 1, r = #attributes where i=1 but j=0, s = #attributes
where i=0 but j=1

Intuition
Think of two sets A and B. Jaccard = |A ∩ B| / |A ∪ B| = shared items / all items in either set.
The larger the overlap relative to the union, the more similar.

Numerical Example
Patients (asymmetric binary: Y=1, N=0):
Jack: fever=1, cough=0, test1=1, test2=0, test3=0. Jim: fever=1, cough=1, test1=0, test2=0,
test3=0
q=1 (fever), r=1 (test1), s=1 (cough) → d(Jack,Jim) =
(1+1)/(1+1+1) = 2/3 ≈ 0.67

Application
Jaccard similarity is widely used in: document similarity, recommendation systems, set
comparison, genomics (comparing gene presence/absence across organisms), and network
analysis.

4.6 Applications of Similarity and Distance Measures

• Clustering: Objects close in distance are grouped together (e.g., K-means uses
Euclidean; hierarchical can use any metric).
• Nearest-Neighbour Classification (KNN): Classify a new object based on the class of
its k nearest neighbours (uses distance measure to find 'nearest').
• Outlier Detection: Objects that are far from all clusters (large distance to neighbours)
are flagged as outliers.
• Information Retrieval: Cosine similarity used to rank documents by relevance to a
query vector.
• Collaborative Filtering: Jaccard or cosine to find users with similar tastes
(recommendation systems).
📊 Distance and Similarity Measures Comparison
Measure Type Formula Best For Key Property
Euclidean Distance √Σ(xi-yi)² Continuous, dense Straight-line; L2
data; geometry norm
Manhattan Distance Σ|xi-yi| Grid-like data; robust City-block; L1
to outliers norm
Minkowski Distance h√(Σ|xi-yi|ʰ) Generalisation of both Lh norm; h is
tunable
Supremum Distance max|xi-yi| Worst-case difference L∞ norm; h→∞
focus
Cosine Similarity (x·y)/(||x||||y||) Text; sparse, high- Non-metric; angle-
dim vectors based
Jaccard Similarity q/(q+r+s) Binary/set data; Ignores negative
documents matches

📝 Quick Revision Notes — Distances

Euclidean: straight-line distance. For most standard numeric clustering.
Manhattan: sum of absolute differences. More robust than Euclidean to outliers.
Minkowski: generalises both (h=1 → Manhattan, h=2 → Euclidean, h=∞ → Supremum).
Cosine: angle-based. Best for text/sparse data. NOT a metric.
Jaccard: positive-matches only (asymmetric binary / sets). Used in text, genomics.
All of Euclidean, Manhattan, Minkowski satisfy the 4 metric properties.
Chapter 5: Data Quality and Data Cleaning

5.1 Data Quality

Data have quality if they satisfy the requirements of the intended use. Quality is NOT absolute
— it depends on the application. The same database might be high-quality for one task and low-
quality for another.

Six Dimensions of Data Quality

Dimension Definition Example
Accuracy Data values conform to the A customer address recorded incorrectly
actual real-world values
Completeness All required data is present; no Income attribute left blank for many customers
missing values
Consistency Data values do not contradict Date stored as 2025/12/01 in one table and
each other or other data 01-12-2025 in another
sources
Timeliness Data is up-to-date for the Month-end bonus data not updated until all
intended use submissions received
Believability Users trust the data even if Past errors erode trust even after correction
technically accurate
Interpretability Data is understandable and Accounting codes unfamiliar to sales staff
uses clear codes/formats

5.2 Data Quality Issues

5.2.1 Missing Values
Missing data occurs when an attribute value is not recorded for a tuple. Causes include:
• Attributes not considered important at time of entry
• Equipment malfunctions during data collection
• Data deleted due to inconsistency with other records
• Respondents not wishing to provide information (disguised missing data — e.g.,
choosing default 'January 1' for birthday)

5.2.2 Noise
Noise is random error or variance in measured variables. It can corrupt data values, making
them inaccurate without being completely wrong. Causes: faulty instruments, human data entry
errors, data transmission errors.
5.2.3 Inconsistency
Discrepancies in data from multiple sources or within a single source. Examples:
• Different units in different systems (metric vs imperial)
• Different formats (date as DD/MM/YYYY in one place, YYYY-MM-DD in another)
• Different naming conventions for the same entity (customer_id vs cust_number)

5.3 Data Cleaning Techniques

5.3.1 Handling Missing Values
Six standard methods for dealing with missing values:

Method Description Advantage / Disadvantage

1. Ignore the Delete tuples with missing Simple; but loses valuable information. Poor
tuple values when many attributes are missing.
2. Fill manually Human expert enters the value Accurate if done correctly; extremely time-
consuming; infeasible at scale
3. Global Replace all missing values Simple; but may create spurious patterns
constant with 'Unknown' or −∞ (mining may find 'Unknown' as interesting)
4. Mean / Median Replace with mean Simple and fast; biases the distribution; reduces
imputation (symmetric) or median variance
(skewed) of all values for that
attribute
5. Class- Replace with mean/median of Better than global mean; still biases data
conditional tuples in the same class
mean/median
6. Most probable Use regression, decision tree, Most accurate; preserves relationships between
value or Bayesian methods to predict attributes; computationally heavier
missing value from other
attributes

🧠 Best Practice
Method 6 (most probable value) is the most sophisticated and recommended approach. It uses
information from other attributes to estimate the missing value, preserving inter-attribute
relationships.

5.3.2 Noise Removal — Smoothing Techniques

Binning
Sort the data, then partition into bins (buckets). Replace each bin's values using one of three
strategies:

Example: Sorted prices: 4, 8, 15, 21, 21, 24, 25, 28, 34 → 3 equal-frequency bins of size 3
Method Bin 1 (4,8,15) Bin 2 (21,21,24) Bin 3 (25,28,34)
Smoothing by Bin Means 9, 9, 9 (mean=9) 22, 22, 22 29, 29, 29
(mean=22) (mean=29)
Smoothing by Bin Medians 8, 8, 8 (median=8) 21, 21, 21 28, 28, 28
(median=21) (median=28)
Smoothing by Bin Boundaries 4, 4, 15 21, 21, 24 25, 25, 34
(min=4,max=15) (min=21,max=24) (min=25,max=34)

Key insight: Larger bin width → greater smoothing effect. Equal-width bins have uniform
interval size. Equal-frequency bins have the same count of values per bin.

Regression
Fit a regression function to the data. Outliers or noisy values will deviate significantly from the
fitted line — they can then be identified and handled. Linear regression finds the best straight
line through two attributes.

Outlier Analysis via Clustering

Group similar values into clusters. Values that fall far outside all clusters (in low-density regions)
are potential outliers. These can be flagged for review or removal.

5.3.3 Outlier Detection

Statistical rule (for approximately Gaussian data): values more than 2 standard deviations from
the mean are potential outliers.
Outlier if: xᵢ < μ - 2σ OR xᵢ > μ + 2σ
Boxplot rule: outliers lie outside Q₁ - 1.5×IQR or Q₃ + 1.5×IQR.

5.4 Data Cleaning as a Process

Data cleaning is an iterative two-step process:

Step 1: Discrepancy Detection — use metadata, statistical descriptions, domain knowledge, and
data auditing tools to find errors, inconsistencies, and missing values
Step 2: Data Transformation — apply a series of transformations (manual corrections, code
substitutions, format harmonisation) to correct identified discrepancies
Step 3: Re-check — after transformation, re-run discrepancy detection to ensure no new errors
were introduced
Step 4: Repeat until satisfied — iterate until the data meets quality requirements

🔍 Key Concept: Discrepancy Detection Tools

Data scrubbing tools: use domain knowledge (postal codes, spelling) to detect and correct errors
Data auditing tools: discover rules/relationships in data; flag violations; uses statistical analysis
and clustering
Metadata examination: check data types, domain constraints, uniqueness rules, and null
conditions

📝 Quick Revision Notes — Data Quality & Cleaning

6 quality dimensions: accuracy, completeness, consistency, timeliness, believability,
interpretability
Missing value methods: ignore tuple, fill manually, global constant, mean/median, class
mean/median, most probable
Noise smoothing: binning (means/medians/boundaries), regression, clustering-based outlier
detection
Outlier rule: outside Q1-1.5×IQR or Q3+1.5×IQR (boxplot rule)
Cleaning is iterative: discrepancy detection → transformation → re-check
Chapter 6: Data Integration

6.1 What Is Data Integration?

Data integration is the merging of data from multiple data stores (databases, flat files, data
cubes) into a coherent, unified data store. Careful integration reduces redundancy and
inconsistencies, improving subsequent mining quality.

6.2 The Entity Identification Problem

When integrating data from multiple sources, the same real-world entity may be represented
differently:
• Attribute naming differences: customer_id in one database may be cust_number in
another
• Encoding differences: pay_type uses 'H'/'S' in one database and 1/2 in another
• Abstraction level differences: total_sales may mean branch-level sales in one system
and regional in another

Metadata (data about data) helps resolve these issues: metadata records attribute name,
meaning, data type, value range, and null rules for each attribute.

6.3 Redundancy and Correlation Analysis

An attribute is redundant if it can be derived from another attribute or set of attributes.
Redundancy wastes storage and can distort mining results.

Methods to detect redundancy:

• For nominal data: χ² (chi-square) test to check if two attributes are statistically
correlated
• For numeric data: Correlation coefficient r(A,B) and covariance Cov(A,B) measure
linear dependency

🔍 Key Concept: When to Remove a Redundant Attribute

If r(A,B) is close to +1 or -1, attribute A strongly implies B (or vice versa).
A higher correlation may indicate that one attribute should be removed to avoid redundancy.
However, causality must be considered — correlation alone doesn't justify removal in all cases.
6.4 Tuple Duplication
Duplication at the tuple level occurs when two or more rows represent the same real-world
entity. This often arises from:
• Denormalised table designs (attributes repeated for performance reasons)
• Multiple data sources recording the same event independently
• Inconsistent updates (only some occurrences of a value updated, others left stale)

Deduplication strategies include exact matching on key attributes, fuzzy matching for
approximate duplicates (e.g., slight name variations), and record linkage techniques.

6.5 Data Value Conflict Detection and Resolution

Conflicts arise when the same real-world entity has different attribute values in different sources:
Conflict Type Example Resolution Approach
Different Weight in kg vs pounds Convert to a common unit before integration
units/scales
Different currencies Price in USD vs EUR Apply exchange rate conversion
Different formats Date as DD/MM/YYYY vs Normalise to a single standard format
YYYY-MM-DD
Different Branch sales vs regional sales Aggregate or disaggregate to match level
abstraction levels totals
Genuinely Two sources give different Use trusted source priority or flag for manual
conflicting values ages for the same person review

⚠️ Common Mistake
Integration is not just about combining tables — it requires careful semantic alignment.
Combining data without resolving these conflicts introduces silent errors that are hard to detect
later.

📝 Quick Revision Notes — Data Integration

Entity identification: match same real-world entity across sources using metadata
Redundancy detection: χ² for nominal; correlation coefficient for numeric
Tuple duplication: exact or fuzzy deduplication required
Data conflicts: unit differences, format differences, abstraction level differences
Goal: coherent, consistent, non-redundant unified data store
Chapter 7: Data Transformation
Data transformation consolidates or converts data into forms appropriate for mining.
Transformation can make the mining process more efficient and the resulting patterns easier to
understand.

7.1 Normalization
Normalization scales attribute values to fall within a smaller, common range (typically [-1,1] or
[0,1]). It is essential when:
• Attributes have very different ranges (e.g., income in $10,000s vs age in years)
• Distance-based algorithms (KNN, clustering) would otherwise be dominated by large-
scale attributes
• Neural network training (backpropagation converges faster with normalised inputs)

7.1.1 Min-Max Normalization

Performs a linear transformation. Maps values to a specified [new_min, new_max] range.
v'ᵢ = [(vᵢ - minA) / (maxA - minA)] × (new_maxA - new_minA) +
new_minA

Advantage: Preserves relationships among original values.

Limitation: Out-of-bounds error if a new value falls outside the original [minA, maxA] range.

Numerical Example: income range [$12,000, $98,000], map to [0.0, 1.0]

v'(73,600) = [(73,600 - 12,000) / (98,000 - 12,000)] × (1.0-0.0)
+ 0 = 61,600/86,000 ≈ 0.716

7.1.2 Z-Score Normalization (Zero-Mean / Standard Score)

Normalises based on the mean and standard deviation. The result represents how many
standard deviations a value is from the mean.
v'ᵢ = (vᵢ - Ā) / σA

Advantage: Works well when min/max are unknown or when outliers dominate the data range.
Output range: No fixed bounds; values can be negative or > 1.

Numerical Example: mean income = $54,000, std dev = $16,000

 v'(73,600) = (73,600 - 54,000) / 16,000 = 19,600 / 16,000 = 1.225

Variation — Z-score using Mean Absolute Deviation (more robust to outliers):

sA = (1/n) Σ|vᵢ - Ā| → v'ᵢ = (vᵢ - Ā) / sA

7.1.3 Normalization by Decimal Scaling

Normalises by moving the decimal point by j places, where j is the smallest integer such that
max(|v'ᵢ|) < 1.
v'ᵢ = vᵢ / 10ʲ
Numerical Example: values range from -986 to 917, max absolute = 986. j = 3 (since 986/1000
< 1)
v'(917) = 917 / 1000 = 0.917 v'(-986) = -986/1000 = -0.986

📊 Normalization Methods Comparison

Method Formula Output Range When to Use Limitation
Min-Max [(v-min)/(max-min)] × [new_min, When you know Out-of-
range + new_min new_max]; often the bounds; well- bounds for
[0,1] scaled data future values
Z-Score (v - mean) / std_dev (-∞, +∞); typically [- Unknown bounds Not bounded;
3, 3] or outlier- doesn't work
dominated range well with
bimodal data
Z-Score (v - mean) / MAD Same as Z-score Highly skewed Less
(MAD) data; outlier-heavy standard;
datasets harder to
interpret
Decimal v / 10^j (-1, 1) Simple quick Can lose
Scaling normalisation precision;
depends on
data
magnitude

7.2 Aggregation
Aggregation involves summarising or combining data. Examples include computing totals,
averages, counts, or max/min across groups of data. Aggregation reduces data volume and can
reveal higher-level patterns not visible in the raw data.
• Example: Daily sales records → Monthly sales totals (time-based aggregation)
• Example: Individual transaction data → Customer lifetime value (customer-level
aggregation)
 • Data cubes provide pre-computed aggregations at multiple levels (daily → weekly →
monthly → yearly)

7.3 Discretization
Discretization transforms a continuous numeric attribute into a categorical (ordinal) one by
replacing raw values with interval labels or concept labels.
Example: age (0–100) → {youth: 0-18, adult: 19-64, senior: 65+}

Why Discretize?
• Some mining algorithms (decision trees, association rules) work only with categorical
data
• Reduces number of distinct values → computational efficiency
• Generates concept hierarchies that enable multi-level mining
• Can reduce noise by merging nearby values

7.3.1 Unsupervised Discretization — Binning

Top-down splitting: partition into bins without using class labels.
• Equal-width binning: Each bin covers the same value range (e.g., every $10
increment). Simple but uneven density.
• Equal-frequency (equal-depth) binning: Each bin contains the same number of data
points. Better statistical balance.

7.3.2 Unsupervised Discretization — Histogram Analysis

A histogram partitions attribute values into disjoint buckets:
• Singleton buckets: Each bucket represents one attribute-value/frequency pair. Good
for high-frequency outliers.
• Equal-width: Uniform bucket width (constant range per bucket)
• Equal-frequency: Each bucket contains the same count of data samples

The histogram algorithm can be applied recursively to generate multi-level concept hierarchies
automatically.

7.3.3 Supervised Discretization

Uses class labels to guide the discretization (e.g., entropy-based methods in decision tree
induction — see ID3/C4.5). Splits are chosen to maximise class purity within each interval.

📝 Quick Revision Notes — Data Transformation

Normalization: Scales data to a common range. Min-Max for bounded [0,1]; Z-score for unknown
bounds.
Aggregation: Combines data to higher levels (daily → monthly, row → group summary).
Discretization: Numeric → categorical by binning or histogram. Equal-width vs equal-frequency.
Always save normalization parameters (mean, std, min, max) to apply same transform to new
data!
Discretization can be supervised (class-guided) or unsupervised (no class info).
Chapter 8: Dimensionality Reduction
Dimensionality reduction is the process of reducing the number of attributes
(features/dimensions) under consideration. High dimensionality leads to the 'curse of
dimensionality' — distances become uninformative, models overfit, and computation becomes
intractable.

8.1 Why Reduce Dimensions?

• Removes irrelevant and redundant attributes that confuse mining algorithms
• Reduces noise (many features = more chances of irrelevant noise)
• Improves computational efficiency (fewer dimensions = faster algorithms)
• Reduces storage requirements
• Makes discovered patterns easier to understand and visualise
• Addresses the curse of dimensionality (in high-D space, all points appear far apart)

8.2 Feature Selection vs Feature Extraction

Approach Feature Selection Feature Extraction
Definition Selects a SUBSET of the original Creates NEW features that are
attributes; others are discarded combinations (transformations) of
original features
Original attributes Yes — selected features are No — new composite features
preserved? unchanged replace originals
Interpretability High — selected features have Lower — new features (e.g., PCA
original meaning components) may lack direct
interpretation
Example methods Stepwise forward/backward PCA, Kernel PCA, t-SNE,
selection, decision trees autoencoders
Textbook term 'Attribute Subset Selection' 'Principal Components Analysis' and
related methods

8.3 Principal Component Analysis (PCA)

What is PCA?
PCA (also called the Karhunen-Loeve, or K-L method) is a linear dimensionality reduction
method. It finds k orthonormal vectors (principal components) that best represent the data's
variance, projecting the original d-dimensional data onto a smaller k-dimensional space (k ≤ d).
Intuition
Imagine a cloud of data points in 3D space that resembles a flat pancake. Most variation in the
data lies along two directions (the pancake's flat face). PCA identifies these two directions (the
principal components) and projects all points onto that 2D plane — capturing most of the
information with one fewer dimension.

PCA Algorithm — Step by Step

Step 1: Normalise the input data so each attribute has the same range (prevents large-scale
attributes from dominating)
Step 2: Compute k orthonormal vectors (principal components) that form a basis for the data.
These are obtained from the covariance matrix's eigenvectors.
Step 3: Sort principal components in descending order of 'significance' (eigenvalue = variance
explained)
Step 4: Keep the top k components that explain most of the variance; discard the rest
Step 5: Project original data onto the k principal component axes to get the reduced
representation

Key Properties
• Each principal component: A unit vector orthogonal to all other PCs. Represents a
direction of maximum variance.
• Variance ordering: First PC explains the most variance; second PC the next most, and
so on.
• Reconstruction: Original data can be approximately reconstructed from the top-k PCs.
Quality depends on how much variance is retained.
• Linear method: Each PC is a linear combination of original attributes. Works best for
Gaussian-distributed data or linearly separable clusters.

🔍 Key Concept: What Does 'Principal' Mean?

The 'principal' directions are those along which the data varies the most. By projecting onto
these, we capture the maximum possible variance in the fewest dimensions. Directions of low
variance (little information) are discarded.

8.4 Attribute Subset Selection

Attribute subset selection (also called feature selection) reduces dimensionality by identifying
and removing irrelevant and redundant attributes. For d attributes, there are 2ᵈ possible subsets
— exhaustive search is infeasible for large d. Heuristic (greedy) methods are used instead.
Greedy Heuristic Methods
Method Starting Point Process Stops When
Stepwise Forward Empty set {} At each step, add the single No improvement or
Selection best attribute from remaining predefined size
reached
Stepwise Backward Full set {all At each step, remove the No removal
Elimination attributes} single worst attribute improves quality
Bidirectional Can start from Simultaneously add best and Convergence or
(Combined) either end remove worst at each step budget reached
Decision Tree Induction N/A (construct a Build a decision tree; attributes Tree fully grown
tree) appearing in tree are relevant;
others are irrelevant

Attribute Evaluation Measures

• Statistical tests: Assess significance of each attribute independently
• Information gain: Used in decision tree building (ID3/C4.5) — how much does this
attribute reduce entropy?
• Gini index: Used in CART decision trees
• Chi-square (χ²): For nominal attributes

⚠️ Limitation of Attribute Subset Selection

Greedy approaches may not find the globally optimal subset. Adding a seemingly poor attribute
early may prevent finding a better combination later. However, in practice, greedy methods work
well.

8.5 Nonlinear Dimensionality Reduction Methods

PCA is linear — it projects data onto a flat (hyperplane) subspace. When data lies on a
nonlinear manifold (curved surface), PCA fails to capture the structure. Nonlinear methods
address this.

General Procedure
Most nonlinear DR methods follow two steps:

Step 1: Construct a proximity matrix P: P(i,j) indicates affinity/relevance between data tuples i
and j
Step 2: Preserve proximity: learn low-dimensional representations that keep P(i,j) approximately
intact
8.5.1 Kernel PCA (KPCA)
Extends PCA to nonlinear settings by using a kernel function κ(xᵢ, xⱼ) to compute similarity in a
high-dimensional (possibly infinite) feature space:
P(i,j) = κ(xᵢ, xⱼ)
Common kernels:
• Polynomial: κ(xᵢ,xⱼ) = (1 + xᵢ·xⱼ)ᵖ
• Radial Basis Function (RBF): κ(xᵢ,xⱼ) = exp(-||xᵢ-xⱼ||²/2σ²)
• Linear (special case): κ(xᵢ,xⱼ) = xᵢ·xⱼ → degenerates to standard PCA

8.5.2 t-SNE (t-Distributed Stochastic Neighbor Embedding)

A nonlinear technique widely used for visualisation. Projects high-dimensional data to 2D or 3D
while preserving local neighbourhood structure.
SNE constructs a proximity matrix where P(i,j) is the probability that xⱼ is a neighbour of xᵢ:
P(i,j) = exp(-d²ᵢⱼ) / Σ_{l≠i} exp(-d²ᵢₗ)
The optimal low-dimensional representations minimise the KL divergence between original and
projected proximity matrices.

Key advantage: Excellent at revealing cluster structure and nonlinear manifolds that PCA
cannot capture.
Key limitation: Non-parametric (no explicit function for new data points); computationally
expensive; hyperparameter-sensitive.

📊 Comparison: PCA vs KPCA vs t-SNE

Aspect PCA Kernel PCA t-SNE
Type Linear Nonlinear (kernel trick) Nonlinear (probabilistic)
Best for Gaussian data, linear Nonlinear manifolds with Visualisation, cluster
separability kernel exploration
Proximity Covariance matrix Kernel matrix κ(xᵢ,xⱼ) Probability of being
construction (implicit linear kernel) neighbours
Proximity Minimise reconstruction Minimise ||P - P̂ ||²_fro Minimise Σ KL(Pᵢ||P̂ ᵢ)
preservation error (linear projection)
Handles new data? Yes (project onto PCs) Yes (evaluate kernel) No (must retrain)
Interpretability Moderate (PC loadings) Low (kernel space) Very low (positions only)
Output dimensions Any k ≤ d Any k ≤ d Usually 2 or 3 (for
visualisation)

📝 Quick Revision Notes — Dimensionality Reduction

Goal: Reduce attributes while preserving information. Addresses curse of dimensionality.
Feature Selection: Keep subset of ORIGINAL attributes (attribute subset selection; greedy
heuristics).
Feature Extraction: Create NEW composite attributes (PCA, KPCA, t-SNE).
PCA: Linear; finds directions of maximum variance; sorted by eigenvalue; works best for
Gaussian data.
Attribute Subset Selection: Forward/backward/combined greedy or decision tree induction.
KPCA: Uses kernel function to handle nonlinear data; polynomial/RBF kernels common.
t-SNE: Probabilistic nonlinear method; best for 2D/3D visualisation of cluster structure.
BONUS SECTION: Formula Sheet, Comparisons,
Exam & Viva Questions

📋 One-Page Formula Sheet

Central Tendency
Mean: x̄ = (1/N) Σᵢ xᵢ
Weighted Mean: x̄ _w = Σwᵢxᵢ / Σwᵢ
Median Interpolation: L₁ + [(N/2 - Σfreq_l) / freq_med] × width
Midrange: (max + min) / 2
Empirical: mean - mode ≈ 3(mean - median)

Dispersion
Range = max(x) - min(x)
IQR = Q₃ - Q₁
Outlier if: x < Q₁ - 1.5×IQR OR x > Q₃ + 1.5×IQR
Population Variance: σ² = (1/N) Σ(xᵢ - x̄ )²
Sample Variance: s² = (1/(n-1)) Σ(xᵢ - x̄ )²
Standard Deviation: σ = √σ²

Correlation & Covariance

Covariance: Cov(A,B) = E(AB) - Ā·B̄ = (1/n)Σ(aᵢ-Ā)(bᵢ-B̄ )
Pearson r: r(A,B) = Cov(A,B) / (σA × σB)
Chi-Square: χ² = ΣΣ(oᵢⱼ - eᵢⱼ)²/eᵢⱼ , eᵢⱼ =
count(A=aᵢ)×count(B=bⱼ)/n

Distance / Similarity Measures

Euclidean: d(i,j) = √Σ(xᵢf - xⱼf)²
Manhattan: d(i,j) = Σ|xᵢf - xⱼf|
Minkowski: d(i,j) = ʰ√(Σ|xᵢf - xⱼf|ʰ) [h=1→Manhattan;
h=2→Euclidean; h→∞→Supremum]
Supremum: d(i,j) = max_f |xᵢf - xⱼf|
 Cosine: sim(x,y) = (x·y)/(||x||·||y||)
Jaccard: sim(i,j) = q/(q+r+s) [q=11-matches; r=10; s=01;
ignores 00]
Nominal Dissimilarity: d(i,j) = (p-m)/p [p=total attrs;
m=matches]
Ordinal Normalisation: z_if = (r_if - 1)/(M_f - 1)

Normalization
Min-Max: v' = [(v-min)/(max-min)] × (new_max-new_min) + new_min
Z-Score: v' = (v - mean) / σ
Z-Score (MAD): v' = (v - mean) / sA , sA = (1/n)Σ|vᵢ-mean|
Decimal Scaling: v' = v / 10^j [j: min int s.t. max|v'|<1]

Missing Value: Ordinal

Ordinal rank normalisation: z_if = (r_if - 1) / (M_f - 1)
📊 Key Comparison Tables

Table 1: Data Types — Complete Comparison

Criterion Nominal Ordinal Interval Ratio
Order None Yes Yes Yes
Equal intervals No No Yes Yes
Absolute zero No No No Yes
Mode valid Yes Yes Yes Yes
Median valid No Yes Yes Yes
Mean valid No No Yes Yes
Multiplication/division No No No Yes
Examples hair colour, grades, °C, °F, calendar Kelvin, height,
nationality satisfaction year income

Table 2: Euclidean vs Manhattan vs Minkowski

Property Euclidean (L2) Manhattan (L1) Minkowski (Lh)
Definition Straight-line distance Sum of absolute Generalisation using
differences parameter h
Formula √Σ(xi-yi)² Σ|xi-yi| ʰ√(Σ|xi-yi|ʰ)
Special case of Minkowski h=2 Minkowski h=1 General form
Sensitivity to outliers More sensitive Less sensitive Depends on h
(squares diffs) (absolute diffs)
Best for Standard numeric data Robustness, city Flexible; tune h to data
navigation
Is a metric? Yes Yes Yes (h≥1)

Table 3: Normalization Methods

Method Formula Range Strength Weaknes
s
Min-Max [(v-min)/(max- [new_min, Bounded; Breaks
min)]×range+new_min new_max] intuitive with new
out-of-
range data
Z-Score (v-mean)/σ (-∞, +∞) Handles Range not
 outliers; no fixed
bound issue
Z-Score MAD (v-mean)/MAD (-∞, +∞) Most robust to Less
outliers standard
Decimal Scaling v/10^j (-1, 1) Very simple Poor if
magnitude
s vary
widely
📝 20 Important Exam Questions

Section A: Short Answer (2–5 marks)

• Q1. Define nominal, ordinal, interval, and ratio data types with one example each.
• Q2. What is the difference between symmetric and asymmetric binary attributes? Give
an example of each.
• Q3. Why is the mean not always the best measure of central tendency? When should
the median be preferred?
• Q4. Define Interquartile Range (IQR) and explain how it is used to detect outliers.
• Q5. What is the difference between population variance and sample variance?
• Q6. State the four metric properties that Euclidean and Manhattan distances satisfy.
• Q7. Why is cosine similarity preferred over Euclidean distance for text documents?
• Q8. List any four causes of missing values in real-world data.
• Q9. Explain the difference between equal-width and equal-frequency binning with
examples.
• Q10. What is data integration and what is the entity identification problem?

Section B: Medium Answer (5–10 marks)

• Q11. Compare the three normalization methods: Min-Max, Z-Score, and Decimal
Scaling. When would you prefer each?
• Q12. Explain the data cleaning process in two steps: discrepancy detection and data
transformation. Give examples of tools used.
• Q13. Given data points A=(1,2) and B=(4,6), compute: (a) Euclidean distance, (b)
Manhattan distance, (c) Minkowski distance with h=3, (d) Supremum distance.
• Q14. Explain PCA with a step-by-step procedure. What does each principal component
represent?
• Q15. Compare feature selection and feature extraction. Give two methods for each and
explain when to use each approach.

Section C: Long Answer (10–15 marks)

• Q16. A database has missing values in the 'income' attribute. Describe ALL six methods
for handling missing values, comparing their advantages and limitations. Which method
is most accurate and why?
• Q17. Explain in detail the six dimensions of data quality with real-world examples. How
does each dimension affect the outcome of data mining?
• Q18. Given the following salary data: 30, 36, 47, 50, 52, 52, 56, 60, 63, 70, 70, 110 —
Compute: (a) Mean, (b) Median, (c) Mode, (d) Q1, Q3, IQR, (e) Variance and SD, (f)
Identify any outliers.
• Q19. Explain data discretization. Describe discretization by binning and histogram
analysis in detail. What is a concept hierarchy?
• Q20. Compare linear (PCA) and nonlinear (KPCA, t-SNE) dimensionality reduction. In
which situations would each be appropriate? Describe their general two-step procedure.
🎤 Common Viva Questions

Conceptual Viva Questions

• Q1. Can nominal attributes be represented by integers? If yes, does that make them
numeric?
Answer: Yes, integers can represent nominal categories (e.g., 0=red, 1=blue), but NO
arithmetic operations are meaningful on them. The integers are just codes, not quantities.

• Q2. What does a Pearson correlation coefficient of -0.95 indicate?

Answer: Very strong negative linear relationship — as one variable increases, the other
decreases almost proportionally. The magnitude (0.95) indicates strength; the sign indicates
direction.

• Q3. Why can't we compute the mean for ordinal data?

Answer: The mean requires equal intervals between values. Ordinal data has a meaningful
order but unknown interval sizes. For example, the gap between 'good' and 'excellent' is not
necessarily the same as between 'poor' and 'good'.

• Q4. What is the difference between noise and an outlier?

Answer: Noise is random error across multiple values (measurement inaccuracy). An outlier
is a specific data point that deviates substantially from the rest — it could be a genuine
extreme value, not necessarily an error.

• Q5. When would you use z-score normalization instead of min-max?

Answer: Use z-score when: (a) min and max are unknown or may change, (b) there are
significant outliers that would compress all other values into a tiny range with min-max, or (c)
the data distribution is approximately normal.

• Q6. What is the Jaccard coefficient and why does it ignore negative matches?
Answer: Jaccard = q/(q+r+s). In asymmetric binary data, a negative match (both=0) doesn't
indicate similarity — it simply means neither object has the feature. Jaccard ignores these to
avoid inflating similarity scores.

• Q7. What is the 'curse of dimensionality'?

Answer: As dimensions increase, data becomes sparse — all points appear equidistant from
each other. Distance-based algorithms lose effectiveness. You need exponentially more
data to maintain the same density as dimensions grow.

• Q8. Can two attributes have covariance = 0 but still be dependent?

Answer: Yes. Zero covariance means NO LINEAR relationship. A nonlinear relationship
(e.g., Y = X²) would have zero covariance but clear dependence. Only under normality
assumptions does Cov=0 imply independence.

• Q9. What is the difference between data compression (lossless vs lossy) in the context
of data reduction?
Answer: Lossless compression allows perfect reconstruction of original data (e.g., run-length
encoding). Lossy compression (e.g., DWT with truncated coefficients) approximates the
original — minor information is lost but the representation is much smaller.

• Q10. What are the limitations of PCA?

Answer: PCA is a linear method — it cannot capture nonlinear relationships in data. If data
lies on a curved manifold, PCA fails. It also assumes that variance = information, which is
not always true. Additionally, principal components lack direct interpretability.
Module II — One-Page Master Summary

📊 Data Types at a Glance

NOMINAL: Categories, no order (hair colour) | Only Mode valid
ORDINAL: Ordered categories, unknown gaps (grades) | Mode + Median valid
INTERVAL: Equal gaps, no true zero (°C) | All stats except ratios
RATIO: Equal gaps + TRUE ZERO (Kelvin, weight) | All statistics valid
DISCRETE: Finite countable values | CONTINUOUS: Real-valued (float)

📊 Statistics at a Glance
MEAN: Sum/N — sensitive to outliers | MEDIAN: Middle value — robust to outliers
MODE: Most frequent | MIDRANGE: (max+min)/2
IQR = Q3-Q1 | OUTLIERS outside Q1-1.5×IQR or Q3+1.5×IQR
VARIANCE = mean squared deviations | SD = √variance
Positive skew: mean > median > mode | Negative skew: mean < median < mode

📊 Distances at a Glance
Euclidean: √Σ(xi-yi)² — straight line — L2
Manhattan: Σ|xi-yi| — city blocks — L1
Minkowski: generalises both (h=1→Manhattan, h=2→Euclidean, h→∞→Supremum)
Cosine: (x·y)/(||x||·||y||) — angle-based — for sparse/text data — NOT a metric
Jaccard: q/(q+r+s) — for asymmetric binary/sets — ignores negative matches

📊 Preprocessing Pipeline at a Glance

1. DATA CLEANING: Fill missing values (mean/median/regression), smooth noise
(binning/regression), detect outliers
2. DATA INTEGRATION: Merge sources, resolve entity matching, remove redundancy
(correlation/χ²), handle conflicts
3. DATA TRANSFORMATION: Normalise (Min-Max/Z-score/Decimal), discretise
(binning/histogram), aggregate
4. DIMENSIONALITY REDUCTION: Feature selection (subset selection) or feature extraction
(PCA, KPCA, t-SNE)

Source: Han, J., Pei, J., & Tong, H. (2023). Data Mining: Concepts and Techniques (4th ed.).
Elsevier/Morgan Kaufmann. Chapter 2.