Machine Learning Assignment
1. Explain Decision Tree Classifier in detail with an example.
A Decision Tree Classifier is a supervised learning algorithm used for both classification and regression
tasks. It splits the dataset into smaller subsets based on the most significant attributes, forming a
tree-like structure where each internal node represents a feature, each branch represents a decision
rule, and each leaf node represents an output class. Working Process:
1. Select the best attribute to split the data using measures like Information Gain or Gini Index.
2. Split the dataset into subsets based on this attribute.
3. Repeat recursively until stopping criteria are met.
Example:
Suppose we want to predict if a person will play tennis based on the weather. The tree might choose
'Outlook' as the root node since it gives the highest information gain, branching into Sunny, Overcast,
and Rain with further splits on Humidity or Wind.
2. What is Bias & Variance Tradeoff? Explain with an example.
The Bias-Variance Tradeoff helps balance a model’s simplicity and complexity. Bias refers to the error
due to simplifying assumptions in the model (high bias leads to underfitting). Variance measures how
much the model's predictions change with different training data (high variance leads to overfitting).
Example:
A linear regression model on non-linear data → high bias, low variance.
A deep neural network on small data → low bias, high variance.
The goal is to minimize both for good generalization.
3. Explain KNN Algorithm with an example.
The K-Nearest Neighbors (KNN) algorithm is a simple, non-parametric method used for classification
and regression.
Steps:
1. Choose K (number of neighbors).
2. Calculate distance (Euclidean) from test point to all training points.
3. Select K closest neighbors.
4. For classification, assign the majority class among them.
Example: To classify a fruit as apple or orange based on color and weight, if K=3 and 2 of 3 nearest
neighbors are apples, the new fruit is classified as an apple.
4. Why categorical variables are encoded? Explain One-Hot Encoding and Two-Hot
Encoding with an example.
Models require numerical inputs, so categorical (text-based) data must be converted into numeric form.
One-Hot Encoding:
Creates binary columns for each category.
Example (Color: Red, Blue, Green):
Red→[1,0,0], Blue→[0,1,0], Green→[0,0,1]
Two-Hot Encoding:
Used for multi-label data where more than one category applies.
Example (Movie genres: Action, Comedy, Drama): Movie A→[1,0,1], Movie B→[0,1,0].
5. Explain various feature extraction techniques.
Feature extraction transforms raw data into meaningful features.
1. PCA – Reduces dimensionality capturing max variance.
2. LDA – Maximizes class separability for classification.
3. ICA – Separates mixed signals into independent components.
4. Bag of Words – Converts text to word frequency vectors.
5. TF-IDF – Gives higher weight to rare but important words.
6. CNN Feature Extraction – Learns spatial features from images.
6. Why is Sampling Distribution needed? Give an example.
A Sampling Distribution represents the distribution of a statistic (like mean) from repeated random
samples.
It is needed to estimate population parameters, measure variability, and perform hypothesis testing.
Example: Taking multiple samples of students’ heights and plotting their mean values gives a sampling
distribution of the mean, helping infer about the population.
7. List out types of Sampling Distribution. Explain Central Limit Theorem.
Types:
1. Sampling distribution of the mean
2. Sampling distribution of the proportion
3. Sampling distribution of the difference between means
4. Sampling distribution of the variance
Central Limit Theorem (CLT):
When independent samples are drawn from any population, the sampling distribution of the sample
mean approaches normality as the sample size increases (n > 30), regardless of the population shape.
8. What are some of the feature transformation techniques? Give example for all.
Feature transformation makes data suitable for ML models.
1. Normalization: Scales data to [0,1]. Example: (X−Xmin)/(Xmax−Xmin).
2. Standardization: Mean=0, SD=1. Example: (X−µ)/σ.
3. Log Transformation: Reduces skewness. Example: log(X).
4. Power Transformation: Stabilizes variance (Box-Cox).
5. Encoding: Converts categories to numbers (One-Hot, Label).
6. Polynomial Features: Adds powers of features for non-linear models.