Explain Bagging and Boosting process.
What are the differences
between thosetwo? (8)
Below highlighted explanations and differences are must (4 marks each). Diagrams
are optional.
• Bagging
• A parallel process
• Creating multiple datasets using the bootstrap using sampling with
replacement
• Each new dataset is of the same size of the original dataset
• Fitting a separate model/decision tree to each copy
• Combining/aggregation all of the models/trees in order to create a single
predictive model.
• Each tree is built independently of the other trees.
• Boosting
• A Serial process
• The trees are grown sequentially
• Each tree is grown using information from previously grown trees
• Each tree works on a modified version of the training set
Q.2 Explain the steps in k-means clustering algorithm. What are the different ways to
choose the most appropriate k value? Explain with the help of an appropriate
diagram or plot. (4+4)
Steps in k-means clustering (4 marks)
Objective: Minimize Sum of Squared distances
Step 0: Create random k centroids
Repeat iteratively till convergence (no further improvement/movement)
Step 1: Assign points to the closest centroid.
Step 2: Re-calculate and move centroids.
Choosing right k (2 marks)
1. Using domain knowledge
2. Using elbow method: plot a line chart of SSD/Inertia/Cost function on y-axis
Vs number of clusters on x-axis. The point which appears like an elbow is the
most appropriate number of clusters. In the below diagram, 4 is the optimum
number of clusters.
Correct Diagram (2 marks)
Q.3 Explain the Coefficient of Determination with an appropriate diagram. (8)
Correct Definition (1 mark)
Coefficient of Determination (r-squared) is the proportion of variance in a dependent
variable that can be explained by an independent variable.
All of the below formulas and explanations (5 marks)
Correct Diagram (2 marks)
Q.4 A company has a small dataset of its customers’ age, salary and whether they
bought a new product under a promotion (Class 1: will buy the product, Class 0:
will not buy the product). The company decides to build a 3-NN model to predict
the behaviour of their future customers. Predict the behaviour of a new customer
having age 30 years and salary 10 LPA, using a 2-dimensional plot. Use 2
different colours to represent 2 different classes and a cross (x) sign to represent
the test example. Feature scaling and distance calculations are NOT required. (8)
Age Salary Bought the
(years) (LPA) product?
46 11 0
26 14 0
35 20 1
28 7 1
33 13 0
49 20 1
20 7 1
38 18 1
Below plot with correct 9 points marked at approximately correct positions - 5
class 1 dots in one colour (or symbol), 3 class 0 dots in another colour (or symbol),
and a correctly placed cross (X) sign (6 marks)
Behaviour of the new customer with age 30 years and salary 10 LPA (2 marks)
As 2 of the 3 nearest neighbours are from class 0, the majority vote goes to class 0
and the prediction is class 0.
Q.5 Consider the below decision tree. Each node has observations from two different
classes namely, Y and N. Calculate the gini impurity for each node. (8)
Gini index (G) formula (0.5 mark)
1.5 marks for each correct calculation (step + answer)
Level 0:
P(Y) = ¾
P(N) = ¼
G = ¾ * ¼ + ¼ * ¾ = 3/8 or 0.375
Level 1 Left:
P(Y) = 1
P(N) = 0
G=1*0+0*1=0
Level 1 Right:
P(Y) = 2/3
P(N) = 1/3
G = 2/3 * 1/3 + 1/3 * 2/3 = 4/9 or 0.44
Level 2 Left:
P(Y) = 9/10
P(N) = 1/10
G = 9/10 * 1/10 + 1/10 * 9/10 = 0.18
Level 2 Right:
P(Y) = 1/5
P(N) = 4/5
G = 1/5 * 4/5 + 4/5 * 1/5 = 8/25 or 0.32
********