Introduction to Machine Learning
TP 7 - Random Forest, Bagging and Boosting
Ayoub Asri
2023-11-17
Contents
Starting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Models 2
Bagging Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Random Forests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Gradient Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Xgboost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Final Evaluation 2
Starting
The consumer credit department of a bank wants to automate the decisionmaking process for approval of
home equity lines of credit. To do this, they will follow the recommendations of the Equal Credit Opportunity
Act to create an empirically derived and statistically sound credit scoring model. The model will be based on
data collected from recent applicants granted credit through the current process of loan underwriting. The
model will be built from predictive modeling tools, but the created model must be sufficiently interpretable
to provide a reason for any adverse actions (rejections).
Q1. Read the data set associated with this file : “[Link]” that contains these variables :
BAD 1 = client defaulted on loan 0 = loan repaid
LOAN Amount of the loan request
MORTDUE Amount due on existing mortgage
VALUE Value of current property
REASON Reason for credit DebtCon = debt consolidation HomeImp = home improvement
JOB Six occupational categories
YOJ Years at present job
DEROG Number of major derogatory reports
DELINQ Number of delinquent credit lines
CLAGE Age of oldest trade line in months
NINQ Number of recent credit lines
CLNO Number of credit lines
DEBTINC Debt-to-income ratio
1
Q2. explore the data set by precising the number of rows and columns associated with it. Precise also the
number of categorical and numerical variables.
Q3. use the following code before splitting the data (70%/30%)
[Link](2023)
Q4. define an appropriate recipe.
Models
In this part, you have to answer the following questions :
• give random values to hyperparameters and fit the model
• tune 2 hyperparameters for each model.
Bagging Trees
Q1. define a bagging trees model
Q2. fit the model
Q3. evaluate the model by producing the performance metrics and confusion matrix
Q4. plot the ROC curve and the variable importance plot
Random Forests
Q1. define a bagging trees model
Q2. fit the model
Q3. evaluate the model by producing the performance metrics and confusion matrix
Q4. plot the ROC curve and the variable importance plot
Gradient Boosting
Q1. define a bagging trees model
Q2. fit the model
Q3. evaluate the model by producing the performance metrics and confusion matrix
Q4. plot the ROC curve and the variable importance plot
Xgboost
Q1. define a bagging trees model
Q2. fit the model
Q3. evaluate the model by producing the performance metrics and confusion matrix
Q4. plot the ROC curve and the variable importance plot
Final Evaluation
Q1. What is the best model between the 4 models used ? Justify
Q2. Can you provide one plot that contains all the ROC curve to facilitate comparaisons.