Machine Learning for Business
Homework 7
Random Forests
In this homework exercise, we will apply random forests to the same eBay auction data
"[Link]" used in Homework 6. Complete the tasks outlined below.
Tasks
1. Import the data into R. Convert “Competitive”, “Category” and “Weekend” into factor.
Note: the outcome variable “Competitive” MUST be converted into factor in order for the
randomForest package to recognize it as a classification problem.
2. Separate the full dataset into a training set and a test set. Set the seed to 100, which is the
same as what we used in Homework 6, so the separation of the training and test sets will
remain the same as previously. Randomly select 1,000 observations from the full dataset to
save as the training set; save the rest 292 observations as the test set (you can use sample()
function).
3. Set the seed to 1000. Run the randomForest() function in the randomForest package to
create random forests. Designate the training and test sets properly; set the number of
trees to 1000, and set importance=TRUE. First set mtry=5, so it reduces to the basic bagging
approach. Then set mtry=2, so a random sample of 2 predictors is selected at each split and
we have a random forest. Save the respective outcomes to two variables.
4. Retrieve and display the confusion matrix for the test set from the random forest’s (i.e.,
with mtry=2) outcome. Compute the overall accuracy rate. Compare it with the overall
accuracy rate from the best pruned tree we obtained from Homework 6. Does random
forests improve on out-of-sample prediction accuracy as compared to an individual decision
tree?
Note: Because we set the seed to the same value in both parts of this homework, the
results are thus comparable. This shows the importance to ensure replicability.
5. Create a matrix by combining 4 vectors of prediction error rates: (1) the OOB error rates of
the bagging outcome, (2) the OOB error rates of the random forest outcome, (3) the Test
1
error rates of the bagging outcome, and (4) the Test error rates of the random forest
outcome. Each vector of error rates is a function of the number of trees (from 1 to 1000), so
the error matrix is 1000x4. Plot the four series of error rates as line curves against the
number of trees. You can use the matplot() function as demonstrated in class. Use different
colors/line types to differentiate the four series on the plot. Add legend properly and
clearly.
6. Based on the plot generated in Step 5, answer the following questions:
• What role does the number of trees (𝐵𝐵) play in a random forest algorithm? Will a large
𝐵𝐵 lead to overfitting? What value of 𝐵𝐵 would you say is enough for this application?
• What is the difference between OOB errors and Test errors?
• How do bagging and random forests differ? Which approach has the better prediction
performance in this application? Why do random forests outperform bagging in
general?
7. Show the variable importance from the random forest outcome using the importance()
function. Focus on “MeanDecreaseGini” (i.e., type=2). What does it measure? Plot this
variable importance measure using varImpPlot(). Which predictor variable is the most
influential on the outcomes in this application?