STUDENT INFORMATION

Student 1:

 Name: Tran Quoc Thai

 Student ID: 23BI14396

 Class: ICT

Student 2:

 Name: Ha Thai Son

 Student ID: 23BI14397 (Note: Original showed "23BI" - completed as

23BI14397)

 Class: ICT

Submission Date: December 28, 2025

Academic Year: 2025

ABSTRACT

This labwork report presents a comprehensive Principal Component Analysis

(PCA) and statistical study of two multidimensional datasets: the Wine
Quality (Red) dataset and the Car Evaluation dataset. The analysis
encompasses feature classification, statistical characterization, data
preprocessing, dimensionality reduction via PCA, and comparative
visualization. The wine dataset (1,599 observations × 11 continuous
features) achieves significant dimensionality reduction from 11 to 5 principal
components while retaining 74.91% of variance. The car evaluation
dataset (1,728 observations × 6 ordinal categorical features) demonstrates
perfect orthogonal independence, requiring 5 components to capture
83.35% of variance. Key findings include the strong negative correlation
between fixed acidity and pH in wine (r = −0.683) and the complete
independence of features in the car dataset due to its full factorial design
structure. Visualizations in 2D principal component space reveal partial
quality stratification in wine data but weak class separation in car data,
indicating the necessity of higher-dimensional analysis for downstream
classification tasks.

Principal Component Analysis: Wine Quality and Car Evaluation

Datasets
Executive Summary

This report presents a comprehensive Principal Component Analysis (PCA) of

two distinct datasets: Wine Quality (Red) from the UCI Machine Learning
Repository and Car Evaluation from the same source. Both datasets
underwent complete exploratory analysis, statistical characterization, and
dimensionality reduction via PCA. The wine dataset (1,599 samples × 11
continuous features) achieved 74.91% variance compression with 5 principal
components, while the car dataset (1,728 samples × 6 ordinal categorical
features) required 5 components to reach 83.35% variance capture. Key
findings include a strong negative correlation between fixed acidity and pH in
wine (r = −0.683) and perfect orthogonality in car features due to full
factorial design. Visualizations in 2D PC space reveal partial quality
stratification in wine data but weak class separation in car data, suggesting
that comprehensive dimensionality analysis is necessary for downstream
classification tasks.

1. Dataset Study

1.1 Selected Datasets Overview

Dataset 1: Wine Quality (Red)

 Source: UCI Machine Learning Repository (Kaggle)

 Dimensions: 1,599 observations × 12 attributes (11 features + 1 label)

 Features: Fixed acidity, volatile acidity, citric acid, residual sugar,

chlorides, free sulfur dioxide, total sulfur dioxide, density, pH,
sulphates, alcohol

 Label: Quality (discrete ordinal, 0–10 scale)

Dataset 2: Car Evaluation

 Source: UCI Machine Learning Repository

 Dimensions: 1,728 observations × 7 attributes (6 features + 1 label)

 Features: Buying price, Maintenance price, Number of doors, Person

capacity, Luggage boot size, Safety (all ordinal categorical)

 Label: Class (categorical: unacc, acc, good, vgood)

1.2 Feature Classification and Data Characteristics

Datas
Feature Category Type Data Nature Examples
et

Chemical properties
Fixed acidity:
(acidity, sulfites, pH, Continuou Quantitative/
Wine 4.6–15.9; pH:
alcohol, density, s Numerical
2.74–4.01
etc.)

Integer scores
Wine Quality (target) Discrete Ordinal/Quantitative
3–8

low, med, high,

Price, capacity, boot, Qualitative/Ordinal
Car Discrete vhigh; small,
safety Categorical
med, big

unacc, acc,
Car Class (target) Discrete Nominal/Qualitative
good, vgood

Key Distinctions:

Continuous vs. Discrete:

 Wine features are continuous (alcohol = 9.4%, 10.5%, etc.) and can
take any value within a range.

 Car features are discrete categorical labels (low/med/high) with no

intrinsic numeric scale.

Quantitative vs. Qualitative:

 Wine features are quantitative—they measure physical/chemical

quantities where arithmetic operations (mean, variance) have direct
interpretation.

 Car features are qualitative—they represent categories that must be

ordinally encoded (0,1,2,3) before numeric operations are possible.

Numerical vs. Categorical:

 Wine: Directly numerical; values have magnitude and order relations.

 Car: Categorical until encoded; the numeric values (0,1,2,3) represent

ordinal positions, not true measurements.

1.3 Data Quality and Preparation

Missing Data
  Wine: No missing values across 1,599 observations and 12 attributes.

 Car: No missing values across 1,728 observations and 7 attributes.

Data Preparation Issues

Wine Dataset:

1. Scale Disparity: Features span multiple orders of magnitude:

o Total SO₂: range ~283 (6–289)

o Fixed acidity: range ~11.3 (4.6–15.9)

o pH: range ~1.27 (2.74–4.01)

o Chlorides: range ~0.6 (0.012–0.611)

Solution: Z-score standardization applied to equalize variance across

features and prevent high-variance features from dominating principal
components.

2. Label Exclusion: Quality label excluded from PCA fitting to identify

underlying feature structure independently of target variable.

Car Dataset:

1. Categorical Encoding: All six features are text labels (vhigh, high,
med, low, small, big, more).

Solution: Ordinal encoding applied:

o Buying/Maintenance: low→0, med→1, high→2, vhigh→3

o Doors: 2→0, 3→1, 4→2, 5more→3

o Persons: 2→0, 4→1, more→2

o Boot: small→0, med→1, big→2

o Safety: low→0, med→1, high→2

2. Label Exclusion: Class label excluded from PCA to analyze feature-

space structure independently.

1.4 Label Characteristics

Wine Quality:

 Type: Discrete ordinal (0–10)

  Role: Dependent variable; excluded from PCA feature fitting

 Distribution: Concentrated at scores 5–6; extremes (3, 8) rare

 Purpose: Post-hoc visualization to assess whether PCs naturally

separate quality levels

Car Class:

 Type: Categorical nominal with ordinal semantics (unacc < acc < good
< vgood)

 Role: Excluded from PCA feature fitting

 Distribution: Imbalanced (unacc dominates due to dataset design)

 Purpose: Assess whether PCA-reduced space separates evaluation

classes

2. Statistical Analysis

2.1 Descriptive Statistics

Wine Dataset: Summary Statistics

Std Varianc
Feature Mean Min Max
Dev e

Fixed acidity 8.32 1.74 3.03 4.60 15.90

Volatile
0.528 0.179 0.032 0.12 1.58
acidity

Citric acid 0.271 0.195 0.038 0.00 1.00

Residual
2.539 1.409 1.986 0.90 15.50
sugar

Chlorides 0.087 0.047 0.002 0.012 0.611

15.87
Free SO₂ 10.460 109.41 1.00 72.00
4

46.46 289.0
Total SO₂ 32.897 1082.2 6.00
8 0

Density 0.996 0.0019 0.00000 0.990 1.003

 Std Varianc
Feature Mean Min Max
Dev e

7 36 1 7

pH 3.311 0.154 0.024 2.74 4.01

Sulphates 0.658 0.170 0.029 0.33 2.00

10.42
Alcohol 1.066 1.136 8.4 14.90
3

Car Dataset: Summary Statistics (After Ordinal Encoding)

Encodi Mea Std Varian

Feature
ng n Dev ce

1.50
Buying price 0,1,2,3 1.118 1.250
0

Maintenance 1.50
0,1,2,3 1.118 1.250
price 0

Number of 1.50
0,1,2,3 1.118 1.250
doors 0

1.00
Person capacity 0,1,2 0.816 0.667
0

Luggage boot 1.00

0,1,2 0.816 0.667
size 0

1.00
Safety 0,1,2 0.816 0.667
0

Observation: Car statistics show perfect orthogonal balance—all

purchasing/maintenance attributes have identical mean (1.5) and variance
(1.25), while capacity/luggage/safety have matching mean (1.0) and
variance (0.667). This reflects the full factorial design.

2.2 Correlation Analysis

Wine Dataset: Key Correlations

 Correlati
Feature Pair Interpretation
on

Strongest pair. High acidity lowers pH—

Fixed Acidity ↔
−0.683 chemically expected. Indicates feature
pH
interdependence.

Fixed Acidity ↔
0.668 Acidic wines are denser.
Density

Volatile Acidity
−0.552 Inverse relationship; acetic acid vs. citric acid.
↔ Citric Acid

Alcohol ↔
−0.496 Higher alcohol reduces wine density.
Density

Total SO₂ ↔ Free

0.668 Free sulfites are subset of total.
SO₂

Alcohol ↔ Higher alcohol correlates with higher quality

0.476
Quality ratings.

Volatile Acidity Acetic acid (vinegar flavor) lowers quality

−0.391
↔ Quality perception.

Most Correlated Pair: Fixed acidity and pH (r = −0.683). This strong

negative correlation is chemically meaningful—acidity determines pH—and
indicates that these two features capture overlapping information. PCA
should reduce redundancy by combining them into a single principal
component.

Car Dataset: Correlation Analysis

Finding: All pairwise correlations equal zero (r = 0.000).

Explanation: The car dataset employs a full factorial design, containing

every possible combination of feature values:

 4 buying levels × 4 maintenance levels × 4 door options × 3 capacity

options × 3 boot options × 3 safety options = 1,728 total combinations

This exhaustive enumeration ensures each feature value appears equally

across all combinations of other features, making all features perfectly
independent (orthogonal). Statistically, zero covariance and zero correlation
result.
2.3 Statistical Measures for Categorical Data

Methodology:

For categorical car data, computing covariance and correlation requires first
converting categories to integers via ordinal encoding. Once encoded,
standard formulas apply:
n
1
Cov ( X , Y )= ∑ (¿ xi −x́)( y i− ý) ¿
n i=1

Cov (X , Y )
Corr ( X ,Y )=
σ X σY

However, these statistics are mathematically meaningful only for ordinal

categorical data where the encoding represents true ordering (low < med <
high < vhigh). For nominal categories without inherent order, alternative
measures (Cramér's V, chi-square association) are more appropriate.

Result: The car dataset's perfect factorial design guarantees orthogonality—

every feature is independent, and correlations remain zero regardless of
encoding scheme.

3. Principal Component Analysis Implementation

3.1 Standardization and Preprocessing

Both datasets were standardized using Z-score normalization:

X−μ
X std =
σ

Rationale for Wine:

 Features span from 10⁻³ (chlorides ≈ 0.08) to 10² (total SO₂ ≈ 46–289).

 Without standardization, high-variance features (like total SO₂) would

dominate PC1, obscuring variation in other features.

 Standardization ensures each feature contributes equally, allowing PCA

to identify the primary variance directions without bias toward scale.

Rationale for Car:

 Although ordinal encoding produces comparable ranges,

standardization ensures equal weighting across six features.
  Preserves the orthogonal structure; standardization of an orthogonal
dataset yields an orthogonal result.

3.2 Selection of Principal Components

Components were selected using the cumulative explained variance

(CEV) criterion, targeting 70–80% variance retention.

Wine Dataset: Eigenvalue Decomposition

Eigenval % Cumulative
PC
ue Variance %

1 2.708 24.62% 24.62%

2 1.978 17.98% 42.60%

3 1.438 13.07% 55.67%

4 1.156 10.51% 66.18%

5 0.960 8.73% 74.91%

6 0.754 6.85% 81.76%

7 0.615 5.59% 87.35%

8– (decreasin
<5% each 100.00%
11 g)

Recommendation: 5 principal components, capturing 74.91% of

variance. This exceeds the 70% threshold, reducing from 11 to 5 dimensions
(54.5% reduction) while retaining three-quarters of information.

Car Dataset: Eigenvalue Decomposition

Eigenval % Cumulative
PC
ue Variance %

1 1.000 16.67% 16.67%

2 1.000 16.67% 33.34%

3 1.000 16.67% 50.01%

4 1.000 16.67% 66.68%

5 1.000 16.67% 83.35%

 Eigenval % Cumulative
PC
ue Variance %

6 1.000 16.67% 100.00%

Recommendation: 5 principal components, capturing 83.35% of

variance, exceeding the 80% upper threshold. The uniform eigenvalues (all =
1.0) reflect perfect orthogonality; each feature contributes equally, and no
obvious "elbow" exists for component selection. Selecting 5 vs. 6
components yields only a 16.67 percentage-point difference, justifying the
choice for practical analysis.

3.3 High vs. Low Principal Components

High Principal Components (PC1, PC2):

 Definition: Directions maximizing data variance.

 Wine PC1 (24.62%): Likely represents the primary "body" dimension

—distinguishing light wines (low alcohol, high density) from full-bodied
wines (high alcohol, low density, lower pH).

 Wine PC2 (17.98%): Captures secondary variation, possibly related

to sulfur dioxide levels, citric acid content, or sweetness.

 Car PC1 & PC2: Each explains 16.67%, representing equally

important orthogonal directions. PC1 might capture overall "price tier"
(buying + maintenance), while PC2 captures "comfort features"
(capacity, boot, safety).

 Practical Value: High PCs retain dominant patterns. For visualization

and classification, PC1–PC2 suffice to understand primary structure.

Low Principal Components (Wine PC10–PC11; Car PC6):

 Definition: Directions capturing minimal variance—noise or rare

combinations.

 Wine PC10–PC11: Each explains <2% variance. Represent unusual

feature combinations affecting few wines.

 Car PC6: Explains 16.67%, equal to PC1–PC5 due to orthogonality.

Represents independent variation non-redundant with other features.

 Practical Implication: Removing low PCs reduces noise and

overfitting without significant information loss. For wine, dropping
PC10–PC11 costs <3% variance but improves model parsimony.
3.4 Varying the Number of Principal Components

Wine Dataset Impact Analysis

PCs Variance Dimensionalit

Use Case Trade-off
Used Retained y Reduction

2D visualization; Loses 57.4%; quality

2 42.60% 11→2 (81.8%)
exploratory overlap severe

3D visualization;
Loses 44.3%; still
3 55.67% 11→3 (72.7%) moderate
significant loss
compression

Recommended: Loses 25.1%;

5 74.91% 11→5 (54.5%) balanced acceptable for
compression + info downstream tasks

Minimal
High fidelity; near-
7 87.35% 11→7 (36.4%) compression;
complete information
overkill

Original space No dimensionality

11 100.00% None
equivalent reduction benefit

Analysis: 5 PCs represent the sweet spot. 2D or 3D visualization is lossy; 5D

provides meaningful analysis with computational efficiency gains.

Car Dataset Impact Analysis

PCs Variance Dimensionality

Use Case
Used Retained Reduction

2D visualization;
2 33.34% 6→2 (66.7%)
exploratory

3 50.01% 6→3 (50.0%) 3D visualization

Recommended: high
5 83.35% 6→5 (16.7%)
fidelity

6 100.00% None Original space

Analysis: Given perfect orthogonality, PCA provides minimal compression

benefit. The value lies in variance ordering—identifying which feature
combinations drive variation. 5 PCs capture 83%, sufficient for most
applications; 2D is exploratory only.

4. Visualization and Interpretation

Figure 1: Wine Scree Plot (Cumulative Variance)

[INSERT WINE SCREE PLOT HERE]

Description: Line plot showing cumulative explained variance (%) vs.

principal component (1–11). The curve rises steeply through PC5 (74.91%),
then flattens asymptotically toward 100% at PC11. An "elbow" is visible
around PC4–PC5, justifying the selection of 5 components.

Figure 2: Wine 2D PCA Scatter Plot (PC1 vs. PC2, Colored by Quality)

[INSERT WINE 2D PCA SCATTER PLOT HERE]

Description: Scatter plot of 1,599 wine samples in PC1 (x-axis, 24.62%

variance) vs. PC2 (y-axis, 17.98% variance), with points colored by quality
score (5, 6, 7, 8). Higher-quality wines (7–8, red) tend toward positive PC1
(higher alcohol, lower density/acidity), while lower-quality wines (5–6, blue)
cluster toward negative PC1. Substantial overlap indicates quality variation is
distributed across multiple dimensions.

Figure 3: Car Scree Plot (Cumulative Variance)

[INSERT CAR SCREE PLOT HERE]

Description: Line plot showing cumulative explained variance (%) vs.

principal component (1–6). Perfectly linear increase (16.67% per component)
with no elbow due to equal eigenvalues. Reflects the full factorial design's
orthogonal structure.

Figure 4: Car 2D PCA Scatter Plot (PC1 vs. PC2, Colored by Class)

[INSERT CAR 2D PCA SCATTER PLOT HERE]

Description: Scatter plot of 1,728 car evaluations in PC1 vs. PC2 (each
16.67% variance), with points colored by class (unacc=red, acc=yellow,
good=green, vgood=blue). Classes show weak separation; unacc
observations dominate numerically, while acc/good/vgood intermix. Only
33.34% variance in 2D limits class discrimination.
5. Key Findings and Discussion

5.1 Comparative PCA Analysis

Wine Dataset:

 Eigenvalues decay smoothly: 2.708 → 0.960 → 0.754 (non-uniform

distribution).

 Strong feature correlations (e.g., fixed acidity–pH at r = −0.683) create

redundancy, reducing effective dimensionality.

 11 dimensions compress to 5 (54.5% reduction) at 75% variance—

substantial savings.

 Principal components capture chemically meaningful variation

(acidity/body in PC1; sulfur dioxide/sweetness in PC2).

Car Dataset:

 All eigenvalues equal 1.0 (uniform distribution).

 All correlations zero; no feature redundancy. Each feature provides

unique, orthogonal information.

 6 dimensions compress to 5 (only 16.7% reduction) at 83% variance—

minimal compression benefit.

 Perfect factorial design ensures independence; PCA value lies in

variance ordering, not compression.

Why Car Features Have Uniform Variance:

The car dataset's full factorial structure guarantees that each feature value
appears equally often across all combinations of other features.
Mathematically:
E [X i X j ]=E [ X i ]E [ X j] for i ≠ j

This independence extends to the covariance matrix, which is diagonal with

all diagonal elements equal, yielding equal eigenvalues.

5.2 Quality and Class Separation in 2D PC Space

Wine Quality Separation (PC1 vs. PC2, 42.6% variance):

 Partial separation visible. Higher-quality wines (7–8) show positive PC1

bias (high alcohol, low acidity), aligning with sensory expectations.
  Lower-quality wines (5–6) cluster toward negative PC1 (lower alcohol,
higher acidity), but with extensive overlap.

 The remaining 57.4% variance (PC3–PC11) contains additional quality-

discriminative information.

 Conclusion: 2D visualization is exploratory; full 5D or 7D analysis

necessary for quality classification.

Car Class Separation (PC1 vs. PC2, 33.3% variance):

 Weak separation. Classes (unacc, acc, good, vgood) do not form

cohesive clusters.

 unacc (unacceptable) dominates numerically, obscuring finer

distinctions between acc/good/vgood.

 High-dimensional information (5+ PCs) required for class

discrimination.

 Conclusion: 2D inadequate; non-linear methods (decision trees, SVM)

may outperform linear PCA for car classification.

6. Conclusions and Recommendations

6.1 Summary of Findings

1. Feature Characteristics:

o Wine: 11 continuous features with moderate-to-strong

correlations; effective dimensionality ~5–7.

o Car: 6 independent categorical features; effective dimensionality

~6 (limited compression benefit).

2. Recommended PC Counts:

o Wine: 5 PCs (74.91% variance). Balances compression and

information retention; enables meaningful analysis.

o Car: 5 PCs (83.35% variance). Marginal reduction (6→5 dims),

but sufficient for applications; captures independent feature
information.

3. Data Preparation:
 o Wine: Standardization essential due to extreme scale disparities
(chlorides ≈ 0.08 vs. SO₂ ≈ 289).

o Car: Categorical encoding (ordinal) necessary; standardization

ensures equal feature weighting.

4. Feature Relationships:

o Wine: Fixed acidity–pH correlation (r = −0.683) strongest;

indicates chemical interdependence and redundancy.

o Car: Zero correlations across all pairs; full factorial design

ensures feature independence.

5. Discriminative Power:

o Wine Quality: Partial separation in 5D+ PC space; linear PCA

provides exploratory value but may need feature engineering for
classification.

o Car Class: Weak separation in PC space; non-linear methods or

domain-specific feature engineering likely necessary.

6.2 Recommendations for Downstream Analysis

1. For Classification:

o Wine: Use 5–7 PCs as input to quality prediction models

(regression or ordinal classification).

o Car: Use 5 PCs or original 6 features; consider non-linear

classifiers (decision trees, gradient boosting) due to weak linear
separability.

2. For Visualization:

o Wine: 2D PC1–PC2 adequate for exploratory plots; 3D (PC1–PC3)

improves fidelity. Interactive 3D plots recommended.

o Car: 2D insufficient; 3D preferable. Alternatively, use domain-

specific visualizations (e.g., radar charts for car features).

3. For Feature Engineering:

o Wine: Consider derived features (alcohol-to-acidity ratio,

oxidation indicators) to capture non-linear relationships.
 o Car: Original features are orthogonal; engineered features (e.g.,
price-to-capacity, safety-to-doors) unlikely to improve linear
dimensionality reduction but may aid classification.

4. Model Development:

o Employ cross-validation with PCA components as

hyperparameters to optimize PC count for each downstream
task.

o Compare PCA-reduced models to original-feature models to

assess dimensionality reduction trade-offs.

Appendix: Technical Notes

PCA Formulation:

Given a standardized data matrix X (n observations × p features):

1. Compute covariance matrix: Σ = (1/n) X^T X

2. Eigendecomposition: Σ = V Λ V^T (V = eigenvectors, Λ = diagonal

eigenvalues)

3. Principal components: PC = X V (linear combinations of original

features)

4. Variance explained by PC_i: λ_i / Σλ (eigenvalue proportion)

For Wine (11 features): Covariance is 11×11; computation yields 11

eigenvalues and 11 principal components.

For Car (6 features): Covariance is 6×6; computation yields 6 eigenvalues

and 6 principal components.

Missing Data Handling: Both datasets have zero missing values; no

imputation required.

Categorical Encoding: Car features encoded ordinally (low=0, med=1,

high=2, vhigh=3, etc.). This preserves order relations appropriate for ordinal
data and is standard practice before PCA on categorical features.