Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Loan Default Risk Assessment using

Supervised Learning
Anushi Jain1, Shivangi Gupta2, Mandeep Singh Narula3
1,2
Student, 3Assistant Professor, Dept. of ECE,
Jaypee Institute of Information Technology, Noida, India

Abstract:- The goal of this research is to develop a provided, each borrower must be classified as a defaulter or
model for forecasting loan defaults. This type of not.
strategy is unavoidable since bad loans are a critical
problem in the financial sector. To address this issue, Observing the previous methods used in various research
a literature analysis has been conducted to study the papers we have concluded that currently to solve the situation
significant factors that lead up to and solve this of Loan Risk Assessment using predictive methods based on
problem. Dense Neural Network with Dropout (ANN machine learning algorithms. The main approach has been to
with Deep Learning), XGBoost, Random Forest, assign client probability to an individual based on their
Logistics Regression, and Support Vector Classifier payment history and profile features. Summary of the past
are the approaches utilized. We have compared the work done on this problem statement is as follows.
models' accuracies, performance, and confusion
matrix measures during the experimental phase. The In [1], Bagherpour’s paper was based on a dataset of
best approach has been chosen, described, and loans issued between 2001-2016, at quarter frequency. To
suggested based on these factors. Our final results are estimate the loan defaults, he relied on K-Nearest Neighbours,
based on the number of defaulters predicted and SVMs. Factorization Machines, Random Forest classifiers. He
actualized, while we have also suggested a model if we came to the conclusion that nonlinear and non-parametric
models/algorithms provided better results than traditional
prefer institutional research that prioritized accuracy,
logistic regression models. It was also observed that
performance, and speed.
Factorization Machines predicted AUC values between 88-
Keywords:- Credit Score, Logistic Regression, XGBoost. 91% which was the highest amongst other classifiers.

I. INTRODUCTION One year later, in 2018 Xiaojun, M. et al. employed two

relatively new machine learning algorithms namely
Since knowing clients and their behavior has LightGBM and XGBoost for predicting loan default of
become crucial in today's financial fields, we are working customers based on physical peer-to-peer transactions from
on Loan Default Risk Assessment. When there were a Lending Club.[2] Also, various studies show that their
significant number of loan applications, manual application showed a significant reduction in overfitting. With
techniques were usually effective, but they were an error rate of 19.9% and an accuracy of 80.1%, LightGBM
insufficient and took a long time. As a result, the machine was found to give better results than XGBoost.
learning model for loan prediction can be used to assess a
customer's loan condition and develop plans. To that end, Kvamme, H. et al. in 2018 also proposed a new approach
multiple data analyses have been conducted on databases for default mortgage prediction by taking into consideration
to predict the ability of a consumer to repay what they the time series data related to customer transactions within
owe. In doing so companies detect patterns in their data to current accounts, savings accounts, and credit cards. This
prevent income loss. algorithm was implemented via Convolutional Neural
Networks. The AUC for Neural Networks was 0.918, while
Lending Club, located in San Francisco, California, the AUC for a combination of Neural Networks and Random
was a peer-to-peer lending platform. It used to be the Forest Classifiers was 0.926.[3]
world's largest peer-to-peer lending marketplace. The
Lending Club allowed borrowers to establish unsecured Koutanei, F. N. et al. in 2015 had conducted a study and
personal loans ranging from $1,000 to $40,000. Three proposed a new model for a hybrid credit scoring system. This
years was essentially the normal borrowing term. On the works on testing four feature selection algorithms and using
Lending Club website, investors could search and browse ensemble learning classifiers. Amongst the features, Principal
loan listings and choose loans to invest in based on the Component Analysis (PCA) was regarded as the best choice
information provided about the borrower, loan size, loan while for the classification part, an ANN adaptive boosting
grade, and loan purpose. The interest determined how algorithm- Artificial Neural Network-AdaBoost was
investors gained money. Borrowers pay an origination chosen.[4]
fee, and investors pay a service fee, which was how
Lending Club made money. This was changing the However, rather than using a binary categorization of
banking system to make credit more accessible and excellent or bad payers, Kruppa, J. et al. (2013) employs
investing more profitable. However, there still existed a machine learning methods to assess the chance of default.
substantial chance of debtors defaulting on their debts. As They argue that these algorithms' probability estimation is
a result, using the data obtained when the loan was based on nonparametric regression and it compares several

IJISRT22MAY1359 [Link] 1105

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
approaches based on random forests (RF), k-nearest Bad Credit borrowers. Financial organizations can enhance
neighbors (KNN), and bagged k-nearest neighbors (b- the amount of credit they issue while lowering possible losses
kNN). Finally, they discovered that the random forests by accurately judging applicants' credit qualifying.[12] Then,
model beats the other three approaches on the test data in talk about how to use decision trees for credit risk analysis in
terms of AUC scores.[5] different ways.

Khandani, A.E. et al. (2010) had proposed to use a In [13], the authors do research about behaviors of
set of features consisting of standard credit scoring, debt- default prediction models based on credit scoring methods
to-income ratio, and consumer banking transactions to be with machine learning algorithms. The authors compare the
utilized as input for the model. He corroborated that the prediction performance of different models with the data of
transactional features increase the predictive power of the the "My Home, My Life" program, and the results indicate
model greatly.[6] that : the accuracy of models improves with the number of
days overdue increasing, traditional ensemble techniques,
While in 2011, Khashman A. recommended a new bagging, random forest, and boosting.
approach for predicting the credit risk by employing an
emotional neural network which would account for the II. PROBLEM STATEMENT
negative and positive confidence during the learning
process and then the results were to be compared to a Developing a solution to assess loan default risk, using
traditional neural network.[7] The author resolved that the machine learning models and relevant datasets, and
emotional neural network had a better performance index comparing their performance. So that banks can predict the
than other neural network models in terms of speed, capability of each candidate for paying back the loan
accuracy, and clarity. beforehand. This can save a lot of time for both the bank and
the applicant.
Beque, A., Lessmann, S. worked with a new type of
feed-forward neural network which compared its III. PROPOSED SOLUTION
performance to that of traditional methods such as
artificial neural networks, decision trees, regularized
logistic regression, forests, and support vector machines
called Extreme Learning Machine(ELM)[8]. They
claimed that this new method marks a substantial step
forward.

Harris, T. (2013) conducted a study on credit risk

prediction using a support vector machine algorithm
applied for two definitions of default: on one hand, a
broader rule was considered for up to 90 days payment
overdue; on the other hand, with more than 90 days late
payment. He believes that the model employed for the
broader definition is more accurate than the other one and
that it is a more dependable.[9]

Zhang, T. et al. (2018) present a new methodology

for developing a credit score model that includes socio-
demographic, loan application data and applicant's
transaction history data. This approach allows for the
extraction of dynamic features from transactional data,
Fig. 1: Proposed solution flowchart
and the results indicated that all classifiers used with
newly added data had an impactful boost in accuracy.[10] A. Preprocessing Data
We started data preprocessing by deleting columns based
Papouskova, M., and Hajek, P. (2019) present a
on how useful they were to us such as emp_length, emp_title,
novel two-stage credit risk model: the first stage consists
title, grade, issue_date.
of a model that predicts the probability of default (PD)
using ensemble classifiers; the second stage performs an For Null value imputation:
in-depth analysis on customers with a predicted  For mortgage_acc, we found the highest correlated column
probability of default and uses a regression ensemble to total_acc and used it to fill null values.
determine the exposure at default (EAD). The two models  Since revol_util & pub_rec_bankruptcies had very few
are then merged to predict the projected loss in Predictive missing values we simply dropped those rows
Models for Loan Default Risk Assessment (EL). [11]

A Study of Classification Based Credit Risk Our dataset consists of 27 columns after cleaning, of
Analysis Algorithm by Ketaki Chopde, et. al. From this, which it is the loan_status column that classifies loans as
they discuss credit score modeling, which divides loan either charged-off or fully paid. Following data preprocessing,
applicants into two groups: Good Credit borrowers and we performed EDA where we used Pearson correlation to find

IJISRT22MAY1359 [Link] 1106

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
which attributes were highly related to one another. We and 30% for testing for the bias-variance tradeoff, based on
further explored the correlated attributes such as loan_amt positive and negative values.
and installment, grade and sub_grade, term,
home_ownership, verification_status and purpose, B. Building A Model and Evaluation
emp_title, and emp_length, int_rate and annual income, We have created four different classification prediction
isse_d and earliest_cr_line, etcetera for more inferences. models: Artificial Neural Networks (dense neural network
We found that loans with subgrades F, G have lower with dropout), XGBoost Classifier, Random Forest Classifier,
chances of being repaid: Logistic Regression, and Support Vector Machine. Compared
performance of deep learning and machine learning.

In most circumstances, ANN is used when something

that happened in the past is repeated in a similar way. It is a
type of machine learning technology with a large memory
capacity. Based on the patterns in our dataset, we can utilize
this particular feature offered by ANNs.

Fig. 2: Graphical Representation of Grade vs Count

We noticed that the higher the interest rate the lower

the repayment chances, i.e they tend to be charged off.

Fig. 4: ANN model from [14]

In ANN we adjusted the default batch size from 32 to

65536. Our model has 59413 parameters, of which 58357 are
trainable. We implemented four drop-out layers within the
neural network, which has 15 layers. We confined the model
to 30 epochs because it was overfitting 150 epochs, giving us
an accuracy of 22%. After reducing the number of epochs, we
were able to achieve an accuracy of 88.74%. The validation
Fig. 3: Graphical Representation of Interest rate vs Count accuracy is rather good, and the losses are also quite low.
In addition, we observed that the smaller the dti (
debt to income ratio) the more likely it is that the loan
will not be paid. On concluding the analysis we found the
variables to be of two categories:
 Based on loan characteristics (int_rate, loan_amt, etc)
 Based on individual characteristics (annual_income,
employment details, etc.)

We have performed feature engineering to remove

redundant data, and extract information. We have created
a new column called ‘zip_code’ from the address in the
dataset. We have converted strings to categorical integers
for normalization in ‘earliest_cr_line’. We've extracted
the year and converted this to a numeric feature. We have Fig. 5: Loss Evolution during training in ANN.
split the testing and training data with 70% for training

IJISRT22MAY1359 [Link] 1107

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
For Random Forest Classifier, we normalized our data
fairly well, then used the default 100 trees in the forest to
model the forecast, which gave us an accuracy of 88.86%.In
terms of accuracy, it is comparable to XG Boost, but it is
slower than XGBoost and ANN. Since we have a large dataset
and ease of understanding isn't a major concern, Random
Forest seems like an appropriate choice as it is an ensemble
method.

Fig. 6: AUC score during training in ANN.

Fig. 9: Flowchart of logistic regression from [17]

For prediction of categorical dependent variables, we can

also use Logistic regression. It is also valuable for predicting
the likelihood of an event which is something we need for
loan approval prediction. We only reached a 70% accuracy in
Logistic Regression owing to overfitting when utilizing 500
Fig. 7: General architecture of XGBoost from [18] max iterations. As a result, we reduced the number of
iterations to 200, resulting in an accuracy of 88.89 percent.
XGBoost is a self-contained and principled tool for
However, Logistic Regression was the most expensive in
boosting trees. It was created with careful consideration
terms of time complexity, as it required the longest to run,
of both system optimization and machine learning
analyze, and train the datasets. In addition, a convergence
techniques. The purpose of this library is to push systems
error was discovered where the maximum iterations were
to their boundaries in terms of computing in order to
done before convergence.
create a scalable, portable, and accurate library. It has a
better confusion matrix than ANN.

Fig. 10: Flow of SVM algorithm from [16]

SVC is more effective in high-dimensional spaces,

especially when they are greater than the number of samples.
Fig. 8: Logical architecture view of random forest In the Support Vector Classifier, the same issues as the
algorithm from [15] Logistic regression model were observed but when the max
number of iterations was changed from 500 to 200, only 2%

IJISRT22MAY1359 [Link] 1108

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
accuracy change was observed. This means that there was
no overfitting due to the margin difference in the
accuracy. The Support Vector Classifier also did not
converge which was causing a bottleneck.

Both the Logistic Regression Classifier and the

Support Vector Classifier are Euclidean models, which
means they must attain convergence. However, they are
approaching their maximum iterations before reaching
convergence, which is nearly impossible given the data
has 27 dimensions and almost 400k samples.

C. Synthetic Data Generation

Synthetic data is data generated by various algorithms
that matches the statistical properties of the actual data
but does not reveal any information about real people. We
generated synthetic data with the same format and
statistical properties by using the GaussianCopula class
from SDV. We split our processed dataset into defaulter
and paid dataset and then split these dataset into train and
test [Link] concatenated the sample synthetic data to our
processed data and applied XGBoost to test the precision
of our model with the synthetic data. The model gave us
an accuracy of 88.45 percent along with a precision of
0.88. Fig 12: Accuracy and Confusion Matrix of ML Models on
Testing Data
Though the change in precision was insignificant ,
the accuracy of the model did not decrease after using In ANN, the notable points are that there are zero False
data generation technique. Negatives i.e. no one has been mislabeled as a defaulter who
had paid their debts. We should also acknowledge that 13,662
IV. EXPERIMENTAL RESULTS
people were expected to repay their loans but ended up
defaulting. While 9595 persons were expected to default, they
For evaluation, we will consider the confusion
really did.
matrix metric.
We observe that XGBoost has a superior confusion
True Positive (TP) False Negative (FN) matrix metric than ANN, since the False Positive of XGBoost
is fewer than ANN's, with a value of 12158. Similarly, the
True Negative, with a value of 11099, is greater than ANN.
Comparatively, XGBoost is preferred if the concerned output
is prediction and actuality of the number of defaulters.
False Positive (FP) True Negative (TN) The False Positives in Random Forest are 12837, while
the True Negatives are 10420. Although Random Forest is not
as excellent as XGBoost Classifier with respect to confusion
matrix metrics. Also, it is clearly superior to Artificial Neural
Fig. 11: Confusion Matrix for Loan Default Prediction Network in terms of overall accuracy. Random Forest
Classifier also takes longer to compute than ANN during
Where, execution.
TP= Paid loan debt and was predicted to pay off.
FN= Paid loan debt but the prediction defaulter. We achieve an accuracy of 88.89 percent with Logistic
FP= Did not pay the debt but was predicted to pay Regression, but the performance of this classification model is
off. 0.722 at all possible thresholds as calculated by AUC-ROC,
TN= Did not pay the debt and was predicted, which is very low when compared to ANN, XGBoost, and
defaulter. Random Classifier, making it a less preferred model.

In light of our current situation, we are concerned Support Vector Classifiers, like Logistic Regression,
about defaulters if our models are used commercially. As have an AUC-ROC performance score of 0.501, which is
a result, we must reduce False Positives, or circumstances quite volatile. Both these models suffer because of the
in which the defaulter was expected to have paid off his convergence problem .
obligations while maximizing True Negatives, or cases in The GaussianCopula class from sdv was used to generate
which the defaulter was anticipated to have not paid off data similar to our processed data (with similar statistical
his debts. properties).

IJISRT22MAY1359 [Link] 1109

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
REFERENCES
We observe no significant change in accuracy and
precision of XGBoost on synthetic data compared to that [1.] Bagherpour, A. (2017), Predicting Mortgage Loan
of XGBoost on our processed dataset. Default with Machine Learning Methods; University of
California, Riverside;
V. CONCLUSIONS
[2.] Xiaojun, [Link] al. (2018), Study on a Prediction Of P2P
Consequently, it can be concluded that the best Network Loan Default Based on the Machine Learning
model in terms of overall accuracy is Artificial Neural Lightgbm and Xgboost Algorithms According to
Network, more precisely Dense Neural Network with Different High Dimensional Data Cleaning; Electronic
Dropout (Deep Learning). While in terms of speed and Commerce Research and Applications, 31, pp.24-39;
confusion matrix metrics XGBoost is superior. After the [3.] Kvamme, H. et al. (2018), Predicting Mortgage Default
observations and execution of our models through the Using Convolutional Neural Networks; Expert Systems
process we inferred from our data that most people return With Applications, 102, pp.207-217;
their debts, our data is skewed toward completely paid. [4.] Koutanaei, F.N. et al. (2015), A Hybrid Data Mining
And it's the incidents when folks don't pay in whole that Model of Feature Selection Algorithm and Ensemble
are our subject of interest. As a result, rather than the Learning Classifiers for Credit Scoring; Journal of
more typical situation, we must forecast the rare Retailing and Consumer Services, 27, pp.11-23;
occurrence of defaulters. This is known as outlier [5.] Kruppa, J. et al. (2013), Consumer Credit Risk:
detection or anomaly detection. The better overall Individual Probability Estimates Using Machine
accuracy of ANN is due to the fact that it predicts positive Learning; Expert Systems with Applications, 40,
outcomes with greater precision. The ANN model has one pp.5125-5131;
of the parameters, False Negative valued as 0, which is [6.] Khandani, A.E. et al. (2010), Consumer Credit-Risk
the strength for this model as no one is mislabeled a Models via MachineLearning Algorithms; Journal of
defaulter if they have paid off their debts. Banking & Finance, 34, pp.2767-2787;
[7.] Khashman, A. (2011), Credit Risk Evaluation Using
Neural Networks: Emotional versus Conventional
Models; Applied Soft Computing, 11, pp.5477-5484;
[8.] Beque, A., Lessmann, S. (2017), Extreme Learning
Machines for Credit Scoring: An Empirical Evaluation;
Expert Systems with Applications, 86, pp.42-53;
[9.] Harris, T. (2013), Quantitative Credit Risk Assessment
Using Support Vector Machines: Broad versus Narrow
Default Definitions; Expert Systems with Applications,
40, pp.4404-4413;
[10.] Zhang, T. et al. (2018), Multiple Instance Learning for
Credit Risk Assessment with Transaction Data;
Knowledge-Based Systems, 161, pp.65- 77;
Fig.13: Graphical Comparison of results acquired via all [11.] Papouskova, M., Hajek, P. (2019), Two-stage Consumer
the models employed. Credit Risk Modeling Using Heterogeneous Ensemble
Learning; Decision Support Systems, 118, pp.33-45;
Using the Area Under the ROC curve, the [12.] KetakiChopde, Pratik Gosar, ParasKapadia,
accompanying graph Fig. 13 compares the performance NiharikaMaheshwari, Pramila M. Chawan, “A Study of
characteristics of the models we used. We find that ANNs Classification Based Credit Risk Analysis Algorithm”,
have the highest accuracy for training data, which they International Journal of Engineering and Advanced
maintain for testing datasets as well, indicating that there Technology (IJEAT) ISSN: 2249 – 8958, Volume- 1,
is no overfitting. The AUC-ROC for Random Forest, on Issue-4, April 2012.
the other hand, is much lower. Also, while Logistic [13.] J R de Castro Vieira, F Barboza, V A Sobreiro et al.,
Regression achieved the maximum accuracy of 88.89 "Machine learning models for credit analysis
percent, its performance measure is considerably lower, improvements: Predicting low-income families’
therefore ANN, XGBoost, and Random Tree would still default[J]", Applied Soft Computing, vol. 83, pp. 105640,
be favored over it. In comparison, the Support Vector 2019.
Classifier still underperforms. [14.] Haykin, S. (1994) Neural Networks: A Comprehensive
Foundation. Macmillan Publishing, New York.
Even if the overall accuracy of Artificial Neural [15.] Research on machine learning framework based on
Networks is better, XGBoost is somewhat superior since random forest algorithm, AIP Conference Proceedings
ANN is a generalized model rather than a decision-based 1820, 080020 (2017)
model. Even while False Negatives in confusion matrix [16.] Chaudhari, Mohini & Govilkar, Sharvari. (2015). A
measures are incredibly low in ANN, the minimization of Survey of Machine Learning Techniques for Sentiment
False Positives, which is the statistic we genuinely care Classification. International Journal on Computational
about as a corporation. Science & Applications. 5. 13-23.
10.5121/ijcsa.2015.5302.

IJISRT22MAY1359 [Link] 1110

Volume 7, Issue 5, May – 2022 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
[17.] Khan, Mohammad & Masud, Mehedi & Aljahdali,
Sultan & Kaur, Manjit & Singh, Parminder. (2021).
A Comparative Analysis of Machine Learning
Algorithms to Predict Alzheimer’s Disease. Journal
of Healthcare Engineering. 2021.
10.1155/2021/9917919.
[18.] Wang, Yuanchao & Pan, Z. & Zheng, J. & Qian, L.
& Mingtao, Li. (2019). A hybrid ensemble method
for pulsar candidate classification. Astrophysics and
Space Science. 364. 10.1007/s10509-019-3602-4.

IJISRT22MAY1359 [Link] 1111