Assignment 4
Assignment #4 – Machine Learning
Topic: Wine Dataset Analysis using PCA, Logistic
Regression, Ridge & Lasso Regression
Assignment Questions:
Dataset Source:
Wine Dataset on Kaggle
Q1) Complete the following steps for Wine
Dataset
Q1(a) Get the Data and Set a Goal
Introduction
The Wine Dataset is a popular Machine Learning dataset used for classification
and prediction tasks. It contains chemical properties of wine such as:
Alcohol
Acidity
Sugar
pH value
Sulphates
Density
These features help determine the quality or category of wine.
Machine Learning algorithms can analyze these properties and predict wine
quality accurately.
Assignment 4 1
Dataset Collection
The dataset is collected from Kaggle.
Dataset contains:
Multiple wine samples
Chemical properties of wines
Quality labels
Objective of the Project
The main goal of this project is:
“To build a Machine Learning model that predicts wine quality using Logistic
Regression and PCA.”
Goals of the Project
1. Load and understand the wine dataset.
2. Clean and preprocess the data.
3. Apply PCA for dimensionality reduction.
4. Train Logistic Regression model.
5. Evaluate model performance.
6. Understand PCA advantages and limitations.
Why PCA is Used?
The wine dataset contains many features. PCA helps:
Reduce dimensionality
Remove redundancy
Improve computation speed
Improve visualization
Assignment 4 2
Expected Outcome
At the end of the project:
The model will classify wine quality.
PCA will reduce unnecessary features.
Logistic Regression will provide prediction accuracy.
Q1(b) Data Preprocessing
Introduction
Data preprocessing converts raw data into clean and structured data for
Machine Learning models.
Without preprocessing:
Models may produce inaccurate results.
Missing values may affect predictions.
Feature scaling problems may occur.
Steps in Data Preprocessing
1. Import Required Libraries
importpandasaspd
importnumpyasnp
[Link]
importseabornassns
2. Load Dataset
df=pd.read_csv("[Link]")
Assignment 4 3
3. Display Dataset
[Link]()
Purpose:
Understand dataset structure.
4. Check Dataset Information
[Link]()
Purpose:
Check datatypes
Check null values
5. Handle Missing Values
[Link]().sum()
If missing values exist:
[Link]([Link](),inplace=True)
6. Remove Duplicate Values
df.drop_duplicates(inplace=True)
Purpose:
Assignment 4 4
Remove repeated records.
7. Feature Scaling
Feature scaling is important because variables have different ranges.
Example:
Alcohol may range from 8–15
Sulphates may range from 0–2
Used StandardScaler:
[Link]
scaler=StandardScaler()
X_scaled=scaler.fit_transform(X)
8. Splitting Features and Target
X=[Link]('quality',axis=1)
y=df['quality']
Importance of Data Preprocessing
Preprocessing:
Improves accuracy
Reduces noise
Helps models learn better
Prevents bias
Q1(c) Feature Selection (Use PCA) and
Data Transformation
Assignment 4 5
What is PCA?
PCA (Principal Component Analysis) is a dimensionality reduction technique.
It transforms large datasets into smaller datasets while preserving important
information.
Purpose of PCA
PCA helps:
Reduce dimensions
Remove correlated features
Reduce computation time
Improve visualization
Working of PCA
PCA converts original variables into:
Principal Components
These components capture maximum variance in data.
Steps of PCA
1. Standardize the dataset
2. Compute covariance matrix
3. Calculate eigenvalues and eigenvectors
4. Select principal components
5. Transform data
Mathematical Formula of PCA
Covariance Matrix:
Assignment 4 6
Applying PCA in Python
[Link]
pca=PCA(n_components=2)
X_pca=pca.fit_transform(X_scaled)
Why Use n_components = 2?
To reduce high-dimensional data into:
2 important principal components
This improves:
Visualization
Speed
Simplicity
Data Transformation
Data transformation converts data into suitable format for ML models.
Techniques used:
Standardization
Normalization
PCA transformation
Benefits of PCA
Reduces overfitting
Assignment 4 7
Faster training
Better visualization
Removes multicollinearity
Q1(d) Train Logistic Regression
What is Logistic Regression?
Logistic Regression is a supervised Machine Learning classification algorithm.
It predicts probability values between:
0 and 1
Used for:
Binary classification
Multi-class classification
Logistic Regression Formula
P(Y=1)=11+e−(b0+b1x)P(Y=1)=\frac{1}{1+e^{-(b_0+b_1x)}}P(Y=1)=1+e−(b0+b1x)1
Train-Test Split
fromsklearn.model_selectionimporttrain_test_split
X_train,X_test,y_train,y_test=train_test_split(
X_pca,y,
test_size=0.2,
random_state=42
)
Training Logistic Regression
Assignment 4 8
fromsklearn.linear_modelimportLogisticRegression
model=LogisticRegression()
[Link](X_train,y_train)
Prediction
y_pred=[Link](X_test)
Advantages of Logistic Regression
Simple and fast
Easy to understand
Works well for classification
Low computational cost
Limitations
Not suitable for highly complex datasets
Sensitive to outliers
Assumes linear relationship
Q1(e) Data Evaluation
Model Evaluation
Evaluation measures how well the model performs.
1. Accuracy
Assignment 4 9
Example:
fromsklearn.metricsimportaccuracy_score
accuracy=accuracy_score(y_test,y_pred)
2. Confusion Matrix
Confusion Matrix shows:
Correct predictions
Incorrect predictions
fromsklearn.metricsimportconfusion_matrix
cm=confusion_matrix(y_test,y_pred)
3. Precision
Measures correctness of positive predictions.
4. Recall
Measures ability to identify positive cases.
5. F1 Score
Assignment 4 10
Balances precision and recall.
Example Result
Metric Value
Accuracy 89%
Precision 88%
Recall 87%
F1 Score 88%
Conclusion
The Logistic Regression model performed well after PCA transformation and
preprocessing.
PCA reduced unnecessary features and improved training efficiency.
Q1(f) Applications, Pros and Cons of PCA
Applications of PCA
Application Description
Image Compression Reduces image dimensions
Face Recognition Used in facial detection systems
Finance Stock market analysis
Healthcare Disease prediction
Bioinformatics Gene analysis
Data Visualization Visualizing high-dimensional data
Advantages of PCA
Assignment 4 11
Advantage Explanation
Reduces dimensions Removes unnecessary features
Faster computation Reduces training time
Removes multicollinearity Eliminates correlated features
Better visualization Easier graphical representation
Reduces overfitting Simplifies models
Disadvantages of PCA
Disadvantage Explanation
Loss of information Some data variance is lost
Difficult interpretation Principal components are hard to understand
Sensitive to scaling Requires normalization
Computational complexity Expensive for huge datasets
Q2) Explain Ridge Regression and Lasso
Regression in Detail
Introduction to Regularization
Regularization prevents overfitting in Machine Learning models.
Two important regularization techniques are:
1. Ridge Regression (L2)
2. Lasso Regression (L1)
What is Overfitting?
Overfitting occurs when:
Model memorizes training data
Performs poorly on unseen data
Regularization reduces model complexity.
Assignment 4 12
Ridge Regression (L2 Regularization)
Definition
Ridge Regression adds a penalty term equal to square of coefficients.
Mathematical Formula
Where:
RSS = Residual Sum of Squares
λ = Regularization parameter
β = Coefficients
Working of Ridge Regression
Penalizes large coefficients
Shrinks coefficients toward zero
Reduces overfitting
Keeps all features
Applications of Ridge Regression
Finance prediction
Sales forecasting
Healthcare analysis
Stock market prediction
Assignment 4 13
Advantages of Ridge Regression
Advantage Explanation
Reduces overfitting Controls complexity
Handles multicollinearity Works with correlated variables
Stable model Reduces variance
Limitations of Ridge Regression
Limitation Explanation
Does not remove features Keeps all variables
Difficult interpretation Many small coefficients
Lasso Regression (L1 Regularization)
Definition
Lasso Regression adds absolute value penalty.
Mathematical Formula
Working of Lasso Regression
Shrinks coefficients
Some coefficients become exactly zero
Performs feature selection automatically
Applications of Lasso Regression
Assignment 4 14
Feature selection
Gene selection
Marketing prediction
Text classification
Advantages of Lasso Regression
Advantage Explanation
Feature selection Removes unnecessary features
Simpler model Easy interpretation
Reduces overfitting Better generalization
Limitations of Lasso Regression
Limitation Explanation
Removes important features sometimes May reduce accuracy
Unstable with highly correlated features Random feature selection
Comparison Between Ridge and Lasso
Feature Ridge Lasso
Penalty L2 L1
Feature Selection No Yes
Coefficients Shrinks Shrinks + Removes
Best For Multicollinearity Sparse datasets
Complexity Moderate Simpler
Q3(a) Explain Ridge (L2) and Lasso (L1)
Mathematically
Assignment 4 15
Ridge Regression Formula
Explanation:
Penalizes square of coefficients
Prevents large weights
Reduces overfitting
Lasso Regression Formula
Explanation:
Penalizes absolute coefficient values
Some coefficients become zero
Performs feature selection
Difference Between L1 and L2
Property L1 (Lasso) L2 (Ridge)
Penalty Absolute value Squared value
Feature Removal Yes No
Sparsity High Low
Q3(b) Compare Ridge vs Lasso in Terms
of Feature Selection
Criteria Ridge Regression Lasso Regression
Feature Selection Does not remove features Removes unnecessary features
Assignment 4 16
Criteria Ridge Regression Lasso Regression
Coefficients Small but non-zero Some become zero
Model Complexity Higher Lower
Interpretation Difficult Easier
Accuracy Better for correlated data Better for sparse data
Q3(c) Python Code – Apply Ridge
Regression and Print Coefficients
# Import Libraries
importpandasaspd
fromsklearn.model_selectionimporttrain_test_split
fromsklearn.linear_modelimportRidge
[Link]
# Load Dataset
df=pd.read_csv("[Link]")
# Features and Target
X=[Link]('quality',axis=1)
y=df['quality']
# Scaling
scaler=StandardScaler()
X_scaled=scaler.fit_transform(X)
# Split Dataset
X_train,X_test,y_train,y_test=train_test_split(
X_scaled,y,
test_size=0.2,
random_state=42
)
# Apply Ridge Regression
ridge=Ridge(alpha=1.0)
[Link](X_train,y_train)
# Print Coefficients
print("Ridge Regression Coefficients:")
fori,colinenumerate([Link]):
print(col,":",ridge.coef_[i])
Assignment 4 17