CREDIT CARD FRAUD DETECTION USING

MACHINE LEARNING METHOD

MD. ALAMGIR HOSEN

MD. RABIUL ISLAM
SUJIT GOMASTA

A thesis in fulfilment of the requirements for the degree

of
Bachelor of Science in Computer Science and Engineering

Department of Computer Science and Engineering

Dhaka University of Engineering & Technology, Gazipur
Gazipur, Bangladesh

June 2022
CREDIT CARD FRAUD DETECTION USING
MACHINE LEARNING METHOD

MD. ALAMGIR HOSEN

Student No.164012
Reg.9669, Session:2019-2020

MD. RABIUL ISLAM SUJIT GOMASTA

Student No. 164023 Student No.164049
Reg.9680, Session:2019-2020 Reg.9706 , Session:2019-2020

Supervisor Name: Dr. Mohammod Abul Kashem

Proofessor
CSE,DUET

A thesis submitted for the degree of

Bachelor of Science in Computer Science and Engineering
Department of Computer Science and Engineering
Dhaka University of Engineering & Technology, Gazipur
Gazipur, Bangladesh

June 2022
Declaration
It is hereby declared that the work presented in this thesis or any part of this has
not been submitted elsewhere for the award of any degree or diploma
Signatures:

....................................................
(MD. ALAMGIR HOSEN)

.....................................................
([Link] ISLAM)

......................................................
(SUJIT GOMASTA)

Signature of the Thesis Supervisor

......................................................
[Link] Abul Kashem
Professor
Department of Computer Science and Engineering
Dhaka University of Engineering & Technology, Gazipur.
Acknowledgements

All the praises are due to almighty Allah for giving us the ability to complete
this thesis and project. We would like to acknowledge the supports provided by
different people in different aspects of our [Link] of all, we express our
sincere gratitude to our thesis supervisor Dr. Mohammod Abul Kashem,
Professor, Department of Computer Sci- ence and Engineering, DUET,Gazipur,
for providing us time to time advice and instructions in the progression of the re-
search. We are also thankful to him for his insightful review during the editing
and writing of this dissertation. We wish to all the teachers who have directly
or indirectly contributed towards the completion of the thesis work. Last but
not least, we are grateful to our parents and our families for their pa- tience and
support during our studies.

ii
Abstract

Now a days credit card fraud is very serious problem because,use of online bank-
ing and credit card is increasing day by day over the world. As the use of credit
card is increasing, the fraudulent activity also increasing. There are many type of
fraud in credit card. Fraudulent transaction of credit card cause a huge amount
of loss. There are various type of fraud detection technique based on machine
learning algorithm like Neural Network, Logistic Regression, Multi-layer percep-
tron, Random Forest, Nave Bayes, Support Vector Machine etc. Supervised and
unsupervised algorithm apply for credit card fraud detection gives approximate
accurate [Link] this research work we use Logistic Regression ,SGD, K-Nearest
Neighbor, Random Forest, Support Vector Machine, and Naive Bayes Machine
Learning algorithm and we get Random Forest give better perfomance among
them .Then we use fine tune in Random Forest and also use re-sampling tech-
nique on dataset and get result Precision 97%, Recall 97%, F1 score 97% Accuracy
99.98%.

iii
Contents

Declaration i

Acknowledgements ii

Abstract iii

List of Figures vi

List of Tables x

List of Symbols xii

List of Abbreviations xiii

1 Introduction 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Outlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Literature Review 3
2.1 Background Study . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.1 Credit Card Fraud and Types . . . . . . . . . . . . . . . . 4
2.1.2 Machine learning . . . . . . . . . . . . . . . . . . . . . . . 5

iv
 [Link] Unsupervised learning . . . . . . . . . . . . . . . 6
[Link] Reinforcement learning . . . . . . . . . . . . . . 6
[Link] Supervised learning . . . . . . . . . . . . . . . . . 6
2.1.3 Used Algorithm and Technique . . . . . . . . . . . . . . . 7
[Link] Random Forest . . . . . . . . . . . . . . . . . . . 7
[Link] Stochastic Gradient Descent (SGD) . . . . . . . . 9
[Link] Support Vector Machine . . . . . . . . . . . . . . 10
[Link] Naive Bayes . . . . . . . . . . . . . . . . . . . . . 11
[Link] Logistic Regression . . . . . . . . . . . . . . . . 12
[Link] K-nearest Neighbor . . . . . . . . . . . . . . . . 13
[Link] Fine tune parameter . . . . . . . . . . . . . . . . 14
[Link] Re-sampling . . . . . . . . . . . . . . . . . . . . . 15
2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Methodology 21
3.1 Short description of Methodology . . . . . . . . . . . . . . . . . . 21
3.2 Brief description of Methodology . . . . . . . . . . . . . . . . . . 22
3.2.1 Data-set collection . . . . . . . . . . . . . . . . . . . . . . 22
3.2.2 Exploratory Data Analysis . . . . . . . . . . . . . . . . . 23
3.2.3 Classification Dataset . . . . . . . . . . . . . . . . . . . . . 30
3.2.4 Dataset Prepossessing . . . . . . . . . . . . . . . . . . . . 30
3.2.5 Applying Machine learning algorithm . . . . . . . . . . . 34
3.2.6 Performance evaluation. . . . . . . . . . . . . . . . . . . . 34
3.2.7 Update selected best model . . . . . . . . . . . . . . . . . 35

4 Experimental Result and Discussion 36

4.1 Experimental Result of various model . . . . . . . . . . . . . . . 37
 4.1.1 Logistic Regression(LR) . . . . . . . . . . . . . . . . . . . 37
4.1.2 Stochastic Gradient Descent (SGD) . . . . . . . . . . . . . 38
4.1.3 K-nearest Neighbor(K-NN) . . . . . . . . . . . . . . . . . . 40
4.1.4 Random Forest(RF) . . . . . . . . . . . . . . . . . . . . . 41
4.1.5 Support Vector Machine (SVM) . . . . . . . . . . . . . . . 43
4.1.6 Naive Bayes(NB) . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.7 Output of Random Forest(RF) using different methods . . 47
[Link] Output of Random Forest(RF) with Fine-Tune
and Resampled dataset. . . . . . . . . . . . . . . 47
[Link] Output of Random Forest(RF) with Fine-Tune
and Class Weight. . . . . . . . . . . . . . . . . . 49
4.2 Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Conclusions and Future Work 52

5.0.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.0.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 53

References 54

Appendix 57
List of Figures

2.1 Working process of Random Forest. . . . . . . . . . . . . . . . . . 8

2.2 SGD Classifier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3 Working process of SVM classifier. . . . . . . . . . . . . . . . . . 10

2.4 logistic Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.5 Before apply K-NN. . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.6 After apply K-NN. . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1 Proposed Methodology. . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Part of Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3 Dataset Information. . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.4 Visual Representation of Credit Card Fraud dataset. . . . . . . . 24

3.5 Representation of fraud and non-fraud data . . . . . . . . . . . . 25

3.6 Value for Skewness. . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.7 Value for Skewness. . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.8 Probability distribution for Time. . . . . . . . . . . . . . . . . . . 26

vii
3.9 Probability distribution for V1. . . . . . . . . . . . . . . . . . . . 26

3.10 Probability distribution for V2. . . . . . . . . . . . . . . . . . . . 27

3.11 Probability distribution for V3. . . . . . . . . . . . . . . . . . . . 27

3.12 Probability distribution for V4. . . . . . . . . . . . . . . . . . . . 27

3.13 Probability distribution for V5. . . . . . . . . . . . . . . . . . . . 27

3.14 Probability distribution for V6 . . . . . . . . . . . . . . . . . . . . 27

3.15 Probability distribution for V28. . . . . . . . . . . . . . . . . . . . 27

3.16 Probability distribution for Amount. . . . . . . . . . . . . . . . . 28

3.17 Probability distribution for Class. . . . . . . . . . . . . . . . . . . 28

3.18 Part of Correlation value . . . . . . . . . . . . . . . . . . . . . . . 29

3.19 Heatmap of Correlation value . . . . . . . . . . . . . . . . . . . . 29

3.20 Description of data set . . . . . . . . . . . . . . . . . . . . . . . . 31

3.21 Null Value Check. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.22 Null Value Check. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.23 Detect for Outliers. . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.24 State before filtering Outlier from Amount. . . . . . . . . . . . . . 33

3.25 State after filtering Outlier from Amount. . . . . . . . . . . . . . 33

4.1 Confusion matrix for LR. . . . . . . . . . . . . . . . . . . . . . . . 37

4.2 Precision, Recall, F1 score and Accuracy for LR. . . . . . . . . . . 37

4.3 Precision Recall curve for LR. . . . . . . . . . . . . . . . . . . . . 38

4.4 Confusion matrix for SGD. . . . . . . . . . . . . . . . . . . . . . . 38

4.5 Precision, Recall, F1 score and Accuracy for SGD. . . . . . . . . . 39

4.6 Precision Recall curve for SGD. . . . . . . . . . . . . . . . . . . . 39

4.7 Confusion matrix for K-NN. . . . . . . . . . . . . . . . . . . . . . 40

4.8 Precision, Recall, F1 score and Accuracy for K-NN. . . . . . . . . 40

4.9 Precision Recall curve for K-NN. . . . . . . . . . . . . . . . . . . 41

4.10 Confusion matrix for RF. . . . . . . . . . . . . . . . . . . . . . . . 41

4.11 Precision, Recall, F1 score and Accuracy for RF. . . . . . . . . . . 42

4.12 Precision Recall curve for RF. . . . . . . . . . . . . . . . . . . . . 42

4.13 Confusion matrix for SVM. . . . . . . . . . . . . . . . . . . . . . 43

4.14 Precision, Recall, F1 score and Accuracy for SVM. . . . . . . . . 43

4.15 Precision Recall curve for SVM. . . . . . . . . . . . . . . . . . . . 44

4.16 Confusion matrix for NB. . . . . . . . . . . . . . . . . . . . . . . 44

4.17 Precision, Recall, F1 score and Accuracy for NB. . . . . . . . . . 45

4.18 Precision Recall curve for LR. . . . . . . . . . . . . . . . . . . . . 45

4.19 Graph of comparative output of various method. . . . . . . . . . 46

4.20 Confusion matrix for RF with Fine-Tune and Resampled dataset. 47

4.21 Precision, Recall, F1 score and Accuracy for RF with Fine-Tune

and Resampled dataset. . . . . . . . . . . . . . . . . . . . . . . . 48

4.22 Precision Recall curve for RF with Fine-Tune and Resampled dataset. 48
4.23 Confusion matrix for RF with Fine-Tune and Class Weight. . . . 49

4.24 Precision, Recall, F1 score and Accuracy for RF with Fine-Tune

and Class Weight. . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.25 Precision Recall curve for RF with Fine-Tune and Class Weight. . 50

4.26 Comparison Graph . . . . . . . . . . . . . . . . . . . . . . . . . . 51

List of Tables

2.1 Result[2] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 Result[7] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3 Accuracy, Precision and MCC values before applying SMOTE[9] . 19

2.4 Accuracy, Precision and MCC values after applying SMOTE[9] . . 20

3.1 Correlation coefficient. . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1 Summarized output of all methods. . . . . . . . . . . . . . . . . . 46

4.2 Summarized output of all Random Forest model. . . . . . . . . . 50

xi
List of Symbols

P
Summation
σ Standard Daviation

xii
List of Abbreviations

RF Random Forest
SVM Support Vector Machine
NB Naive Bayes
LR Logistic Regression
KNN K-Nearest Neighbour
DL Deep Learning
XGBT Extended Gradient Boosted Tree
NN Neural Networks
QDA Quadratic Discriminated Analysis
HSA Hybrid Supervised Approaches
SGD Stochastic Gradient Descent
ML Machine Learn
SMOTE Synthetic Minority Oversampling Technique
MCC Matthews Correlation
TP True Positive
FP False Positive
TN True Negative
FN False Negative

xiii
Chapter 1

Introduction

1.1 Introduction

In the age of digitization, with the increasing use of e-commerce, use of credit
cards for purchasing goods also exponentially increased. Unfortunately, fraudu-
lent use of credit cards has also become a source of crime. When one or more than
one person directly or indirectly use the money of a victim by fake transactions
and without let the owner know about transaction then this activity is called a
fraud activity. There are two level of fraud:
1. Management level fraud
2. Customer level fraud

[Link] level fraud

When fraud activity is done by management level of a company of organization
then it called as management level fraud.
2 1.2. Objective

2. Customer level fraud

If fraud is done by a person to a person then it called customer level [Link]
mainly happened in Credit/Debit card.

1.2 Objective

The fraud detection methodology already have, they use various machine learn-
ing algorithm give approximate accurate [Link] this thesis, the objective is to
analysis previous methods and develop a algorithm with better performance and
accuracy.

1.3 Outlines

In chapter 1, we discuss about why fraud detection is necessary is necessary,

objective, contribution, outlines.
In chapter 2, we do literature review. Where we include background study about
some previous research which is done in credit card fraud and related work.
In chapter 3, we describe the methodology which has been used for this thesis.
In chapter 4, we discuss about result do comparison among several model.
In Chapter 5, we conclude everything.
Chapter 2

Literature Review

2.1 Background Study

At present the prevention of fraud is very important because day by day trans-
action through credit card are [Link] prevention is best way to
reduce fraud, but fraud detection methodologies are therefore necessary if once
prevention is failed [1].In any exchange theft and miss utilization of a credit card
is known as credit card fraud [2].The fraudulent activities impact the business
negatively and the company can loose trust of their customers[a3]. The revenue
and turnover of the company can also be affected [3].The user of credit card can
face financial loss. That’s why credit card transactions need to be analyzed in a
variety of ways to provide more security.
4 2.1. Background Study

2.1.1 Credit Card Fraud and Types

Credit card fraud is a broad term basically used to define those fraud which is
committed by using a payment card. It’s occurred when fraud steal a credit card
or illegally obtains a victim’s card details.
There are different types of credit card fraud
1. Lost and Stolen Card Fraud
2. Card not Received Fraud
3. Counterfeit Card Fraud
4. Cardholder not Present Fraud
5. Card ID Theft Fraud

[Link] and Stolen Card Fraud:

When fraudsters get the card by stolen or by finding it and can use it to purchases
goods then it called lost and stolen fraud. In this case cash withdraw will not
possible without PIN.

[Link] not Received Fraud:

In this fraud the card is stolen in the way before the recipient receive it, like if it
steals from the mail.

[Link] Card Fraud:

When the fraudster makes an exact copy of victim’s credit card then it called
counterfeit card fraud.

[Link] not Present Fraud:

5 2.1. Background Study

When fraud is committed, but cardholder have not to be present physically to

complete successful transaction using the credit card. Like as if credit card details
are stolen from online marketplaces.

[Link] ID Theft Fraud:

This kind of fraud is occurred when fraudster gains significant knowledge about
victim which is necessary to make a third-party application for issuing a credit
card or take control on an existing one [4].

2.1.2 Machine learning

Machine learning approach is very different from the conventional computation

approaches. In machine learning, system get program explicitly to calculate or
solve a problem. In general, machine learning can define as a field of AI (Ar-
tificial Intelligent). Machine learning provide capability to a system, to learn
automatically from the experience and try to predict future outcome using var-
ious algorithmic model as accurate as possible. In machine learning data-set is
use to train a model. In training, different pattern in input data is learn by the
model and that knowledge is use to predict unknown result. Typically, there are
three categories in machine learning.
1. Supervised
2. Unsupervised
3. Reinforcement
We used supervised learning in this thesis so supervised learning is briefly de-
scribed in next section.
6 2.1. Background Study

[Link] Unsupervised learning

In unsupervised learning, model is trained with unlabeled data that is data-set

only has input labels, from which the model learns various structure and patterns.
Some unsupervised methods are bellow: -
1. Self-organizing maps
2. K- means
3. Isolation Forest
4. Local Outlier Factor.

[Link] Reinforcement learning

Reinforcement learning deals with learning how to obtain a complex goal by

maximizing along a specific dimension step by step.

[Link] Supervised learning

Supervised learning is a machine learning approach where for training the model
use data-set, which is input and output labeled and extract various pattern from
input data. Those patterns are used to solve the unknown [Link] credit card
fraud detection supervised machine learning is a technique that apply algorithm
on both legitimate and fraudulent instance to construct model that can assign
new observations into legitimate or fraudulent class [5]. Supervised learning can
be represented as follows:
Y = f(x)
where x represents the input variables, Y denotes an output variable and f(X) is
a mapping function.
7 2.1. Background Study

Supervised learning is mainly two types:

1. Classification
2. Regression [6]

Some supervised methods are bellow: -

1. Random Forest (RF)
2. Support Vector Machine (SVM)
3. Naive Bayes (NB)
4. Logistic Regression (LR)
5. The K-Nearest Neighbor (KNN)
6. Deep Learning (DL)
7. Extended Gradient Boosted Tree (XGBT)
8. Neural Networks (NN)
9. Quadratic Discriminant Analysis (QDA)
10. Hybrid Supervised Approaches
[Link] Gradient Descent (SGD) [7]

2.1.3 Used Algorithm and Technique

[Link] Random Forest

Random Forest is one of the most popular supervised machine learning algo-
rithms. In machine learning Random Forest can used for both regression and
classification. To improve the performance and solve complex problem it uses
combination of multiple classifier which is called ensemble learning. From given
dataset Random Forest classifier, takes a number of decision trees on various
8 2.1. Background Study

subset and then it takes the average for improving the predictive accuracy of the
dataset. The greatest number of trees in the forest leads higher accuracy which
prevent overfitting problem.

Figure 2.1 Working process of Random Forest.

Main benefits of Random Forest are below: -

1. As compare to other algorithm it takes less training time.
2. It run efficiently even for the large dataset and give high accuracy in output
prediction.
3. It can maintain the level of accuracy even when a large part of data is missing.
Algorithm:
Step 1: From the training set select K random data point.
Step 2: From the selected points build associate decision trees.
Step 3: For decision trees chose the number (N) that you wish to build.
Step 4: Repeat step 1 & 2.
9 2.1. Background Study

Step 5: Find prediction of every decision tree for new data and which category
wins majority votes assign new data point to that [8].

[Link] Stochastic Gradient Descent (SGD)

SGD is a one kind of classification and regression technique. Its apply for classifi-
cation issue. Therefore its method applies regularized for build an estimator. Its
work well for large scale datatsets and it is an efficient and easy to implements
methods.
Advantage of SGD:
1. Very efficient.
2. Easy to implements and lots of opportunities of code tunning.
Disadvantage of SGD:
1. Requires several hypermeters.
2. Very sensitive for feature scaling.

Figure 2.2 SGD Classifier.

10 2.1. Background Study

[Link] Support Vector Machine

Support vector machine is another popular supervised machine learning algo-

rithm. SVM can work for both classification and regression but suitable for
classification. SVM create best decision boundary or line which can separate n
dimensional space into classes. Which make easy to put new data in right category
in the future. The decision boundary, which is best is called hyperplane. SVM
select the extreme points/vectors which help to create the hyperplane. These
extreme cases are called support vectors, that’s why algorithm called Support
Vector Machine.

Figure 2.3 Working process of SVM classifier.

Advantages of SVM are bellow: -

1. Effective for high dimensional cases.
2. In the decision function called support vector, SVM use subset of training
point, so it is memory efficient [9].
11 2.1. Background Study

[Link] Naive Bayes

Naı̈ve Bayes is a machine learning algorithm which fall in supervised technique

and work based on Bayes theorem. It work for classification [Link], it
predicts class based on the probability of an object, Naı̈ve Bayes called proba-
bilistic classifier.

Bayes’ theorem
Bayes Rule or Bayes law is another name Bayes theorem. To find the probability
of a hypothesis it use prior knowledge.
Formula for Bayes rule as below: -

P (B|A) ∗ P (A)
P (A|B) = (2.1)
P (B)

Where,
P(A—B) It is called Posterior probability, the Probability of hypothesis A on the
observed event B.
P(B—A) It is called Likelihood probability, the Probability of the evidence when
given that, the probability of a hypothesis is true.
P(A) is called Prior [Link] is the Probability of hypothesis before observ-
ing the evidence.
P(B) Probability of Evidence is called Marginal Probability.

Steps for Naı̈ve Bayes

Step1: Covert dataset which is given into frequency table.
Step2: By finding probabilities of given feature generate a likelihood table.
Step3: Then calculate posterior probability using bayes theorem.
12 2.1. Background Study

Naı̈ve Bayes model are three types: -

1. Gaussian
2. Multinomial
3. Bernoulli [10].

[Link] Logistic Regression

Logistic Regression is another supervised machine learning algorithm. Logistic

regression use a given set of independent variables for finding the output of cat-
egorical dependent variable. So, the output must be categorical or discrete and
it can be true or false, yes or no,0 or 1 etc. Instead of giving the value direct as
0 or 1, LR gives a probabilistic value between 0 and [Link](Logistic Regression) is
used for classification whereas, Linear Regression is used for regression problem.
In LR we fit a ”S” shaped logistic function ,instead of fitting a regression line.
Logistic Regression one of the significant ML algorithms since it has capability
to give probabilities and classify new data using discrete and continuous dataset.
Sigmoid Function
A mathematical function used to map the predicted real values to probabilities
is called sigmoid function, which has a range between 0 and 1.

1
g(z) = (2.2)
1 + ez

The value of LR cannot go beyond the range 0 and 1, that’s why it forms a “S”
shape curve which is called the Sigmoid function or logistic function. It uses
concept of threshold value, the value above 1 tends to 1 and below 0 tends to 0.
13 2.1. Background Study

Figure 2.4 logistic Function.

There are three types of logistic regression: -

1. Binomial
2. Multinomial
3. Ordinal [11].

[Link] K-nearest Neighbor

K-Nearest Neighbor is one of the simplest supervised machine learning algorithms.

K-NN look for similarity between new data and available data and categorize new
data into the most similar category from available categories. That means K-NN
can easily classified a new data into best category. K-NN mostly used for clas-
sification problem although it is suitable for both classification and regression
problem. On underlying data K-NN does not make any assumption that’s why
it’s called non-parametric algorithm. It also known as lazy learner algorithm
since, instead of learning immediately from training set it stores the dataset and
it performs an operation on dataset to classify new data.
14 2.1. Background Study

Figure 2.5 Before apply K-NN. Figure 2.6 After apply K-NN.

Algorithm of K-NN:
Step 1: Select K the number of neighbors.
Step 2: For K number of neighbors calculate the Euclidian distance.
Step 3: As per distance take K nearest neighbors.
Step 4: From these K nearest neighbors, count the data points number in each
category.
Step 5: Which category has the maximum number of data point, assign the new
data points to that category.
Step 6: End [12].

[Link] Fine tune parameter

Fine tune is a widely used model reuse technique, it is complementary to feature

extraction technique. For feature extraction it unfreezes some layer of top layer
of frozen model library. And jointly trained the newly added parts of the model
15 2.1. Background Study

and top layers. Fine tune slightly adjusts abstract representation of model going
to be reuse, in order to make the model more relevant to the problem [13].
Hyperparameter is a fine-tuning process for Random Forest algorithm. A param-
eter that is set before starting the learning process of a model is called hyperpa-
rameter. They can be manually adjusted. The most used hyperparameters are
1. Number of trees
2. Maximum depth of each tree
3. Bootstrap method
4. The minimum data point needs to split at node
The two most commonly used for choose which hyperparameter should adjust
is exhaustive grid search and validation curve. Validation curve search potential
values of hyperparameter visually, which can optimize. Exhaustive grid searches
every single possible combination of hyperparameter. Advantage of adjusting hy-
perparameter:
1. It improves accuracy of the model.
2. On some dataset, not adjusting the size of trees can be very large since it is
fully grown and unpruned. Adjusting the size and complexity of trees help to
reduce memory consumptions [14].

[Link] Re-sampling

A very common problem in Machine Learning and Data Science is imbalanced

class distribution. It is a state, where observation of one class is very than any
other classes. Since ML algorithm try to increase accuracy by reducing error,
it doesn’t consider class distribution. This is very common problem with Fraud
detection, Facial recognition, Anomaly detection etc. Two common methods for
16 2.2. Related Work

Resampling are: – 1. Cross Validation 2. Bootstrapping

Standard machine learning technique has a bias towards majority class and
they had tendency to ignore minority class. Hence it can have major misclassifi-
cation in minority class than majority class. It can handle by technique below: -
1. Random Over-sampling
2. SMOTE (Synthetic Minority Oversampling Technique)
3. Random Under-Sampling
4. Cluster-based Over Sampling etc.

Our thesis we use SMOTE and Random Under-Sampling.

SMOTE: SMOTE stands for Synthetic Minority Oversampling [Link]
synthesizes new minority instances between minority instances which already ex-
ist. It randomly takes a point from minority class and calculate the K-Nearest
Neighbor for that point. Then the synthetic point is added between the neighbor
and chosen spot.
Random Under-Sampling: To balance class distribution, Random Under-
Sampling eliminate majority class example randomly [15].

2.2 Related Work

Dejan Varmedja, Mirjana Karanovic, Srdjan Sladojevic, Marko Arsenovic, An-

dras Anderla [16],working with some steps for complete the work . this steps are
data-set collection ,Preprocessing ,Experiment .
In Preprocessing steps they used some techniques and tools, this are Visualization
techniques, Synthetic Minority Oversampling Technique (SMOTE),feature selec-
17 2.2. Related Work

tor tools. The Visualization techniques used for Carefully choosing appropriate
features and removing the less important one can reduce over-fitting, improve
accuracy and reduce training time. SMOTE is a popular oversampling method
that used when the imbalanced dataset are used .The feature selector tools used
for determined which features are the most important.
In Experiment steps they used some machine learning algorithm. this are Lo-
gistic regression, Naive Bayes,Random forest, Multi-layer perceptron. using this
algorithm they get different type of result.

Methods Precision Recall Accuracy

LR 58.82% 91.84% 97.46%

NB 16.17% 82.65% 99.23%

RF 96.38% 81.63% 99.96%

MLP 79.21% 81.63% 99.93%

Table 2.1 Result[2]

Naresh Kumar Trivedi, Sarita Simaiya,*Umesh Kumar Lilhore,Sanjeev Ku-

mar Sharma [17] ,working with different methods includes random forest method,
tree classifiers method, artificial neural networks method, supporting vector ma-
chine method, Naı̈ve Baiyes method, logistic regression method and gradient
boosting classifier methods, on even a skewed credit card fraud dataset. tech-
nique contributes to enhancing its classifier’s detection rate as well as efficiency.
the multiple performances evaluating parameters have been calculated such as
precision, recall, accuracy percentage
18 2.2. Related Work

Methods Precision Recall Accuracy

Random Forest 95.9887% 95.1023% 94.9991%

Naı̈ve Baiyes 91.201% 91.0021% 91.8887%

Logistic Regression 92.8956% 91.5456% 90.448%

SVM 93.228% 92.789% 93.963%

kNN 94.5891% 91.998% 94.999%

Decision Trees 90.998% 91.7752% 90.998%

GBM 93.998% 93.556% 94.001%

Table 2.2 Result[7]

Sangeeta Mittal,Shivani Tyagi [18], working with popular supervised and un-
supervised machine learning algorithms have been applied to detect credit card
frauds in a highly imbalanced dataset. they found that unsupervised algorithms
handle the dataset skewness in better ways and hence perform well over all met-
rics absolutely and relatively to other techniques.

Vaishnavi Nath Dornadula, Geetha S [19] ,Used popular supervised and un-
supervised machine learning [Link] are Local Outlier factor ,Isolation
forest,Support vector machine(SVM), Logistic regression, Decision tree, Ran-
dom [Link] used one-class SVM which can be best used for binary class
19 2.2. Related Work

[Link] customers are grouped based on their transactions and extract be-
havioural patterns to develop a [Link] cardholders are clustered into different
groups based on their transaction amount. Then using sliding window strategy to
aggregate the transaction made by the cardholders from different groups so that
the behavioural pattern of the groups can be extracted [Link] observed
that the Matthews Correlation Coefficient was the better parameter to deal with
imbalance data-set. By applying the SMOTE, we balanced the [Link] pro-
duced Accuracy, Precision and MCC values before and after applying SMOTE
that show the below:

Methods Accuracy Precision MCC

Local Outlier factor 0.8990 0.0038 0.0172

Isolation forest 0.9011 0.0147 0.1047

Support vector machine 0.9987 0.7681 0.5257

Logistic regression 0.9990 0.875 0.6766

Decision tree 0.9994 0.8854 0.8356

Random forest 0.9994 0.9310 0.8268

Table 2.3 Accuracy, Precision and MCC values before applying SMOTE[9]
20 2.2. Related Work

Methods Accuracy Precision MCC

Local Outlier factor 0.4582 0.2941 0.1376

Isolation forest 0.5883 0.9447 0.2961

Logistic regression 0.9718 0.9831 0.9438

Decision tree 0.9708 0.9814 0.9420

Random forest 0.9998 0.9996 0.9996

Table 2.4 Accuracy, Precision and MCC values after applying SMOTE[9]

Here from related work we can see that, for detecting fraud in credit card
they only apply machine learning model like Logistic Regression, Random For-
est, Support Vector Machine,Naive Bayes classifier, k Nearest Neighbor, Decision
Tree etc. But they don,t apply any further process to increase its performance.
Major challenges for credit card fraud detection are:
[Link] information: It means majority of transactions (98.9%) are non-
fraud, which make it impossible to identify fraudulent activity.
2. Enormous information: has been stored on even a daily basis as well as the
design construction should be quick enough yet to react properly to a fraud.
[Link] information: It is another major issue because not all fraudulent
activity is captured or recorded.
[Link] methods: Fraudsters also can use Adaptive methods against the sys-
tem.
Chapter 3

Methodology

3.1 Short description of Methodology

Credit card payments are one of the most popular forms of electronic payments.
A credit car allows users to purchase goods without having to pay cash. So that
credit card security must be strong and prevent from fraudulent and rightly pro-
vide security must be [Link] procedure start with collecting data set on
credit card fraud. In this research the credit card fraud detection data-set will
be [Link] data preprocessing is done, which can refer to manipulation or
dropping of data before it is used in order to ensure or enhance [Link]
apply various machine learning algorithm. To determine which algorithm is most
suitable for the problem of detecting fraud transaction, different criteria for algo-
rithm comparisons have been used. Most used metrics for determining the results
of machine learning algorithm are accuracy, recall and precision. Lastly, we will
chose method which method providing good and effective result on it.
22 3.2. Brief description of Methodology

3.2 Brief description of Methodology

For this thesis we chose Some popular machine learning algorithms in supervised
categories to solve the underlying [Link] we will try to solve credit fraud
problem by using a data set called ”[Link]” and various machine learning
algorithm and [Link] Figure 3.1 show the overall proposed methodology.

Figure 3.1 Proposed Methodology.

3.2.1 Data-set collection

Our used dataset, ”[Link]” has 284807 rows × 31 columns and we collect
it from [Link] some of the input variables contains financial information,
the PCA transformation of these input variables were performed in order to keep
23 3.2. Brief description of Methodology

these data anonymous. Some portion of ”[Link]” dataset is shown in

figure: 3.2

Figure 3.2 Part of Dataset

3.2.2 Exploratory Data Analysis

The dataset has three of the given features where the PCA transformation is
not apply. Feature ”Time” shows the time between first transaction and the
every other transaction in the dataset. Feature ”Amount” is the amount of the
transactions made by credit card. Feature ”Class” represents the label, and takes
only 2 values: value 1 in case of fraud transaction and 0 [Link] figure:
3.3 show the information about [Link].4 show the visualization of used
dataset using plot.
24 3.2. Brief description of Methodology

Figure 3.4 Visual Representation of

Figure 3.3 Dataset Information.
Credit Card Fraud dataset.

In our ”[Link]” dataset there are 284315 non-fraud data and 492 fraud
[Link].5 represent this value as a graph.
25 3.2. Brief description of Methodology

Figure 3.5 Representation of fraud and non-fraud data

Next we see Skewness,the term “Skewness” refers to the statistical metric

that is used to measure the asymmetry of a probability distribution of random
variables about its own mean, and its value can be positive, negative, or unde-
fined. The calculation of the Skewness equation is done on the basis of the mean
of the distribution, the number of variables, and the standard deviation of the
distribution.
Mathematically, the Skewness formula is represented as:

N i(Xi − X)3
P
Skewness = (3.1)
(N − 1) ∗ σ 3

where,Xi is the ith Random Variable. X is the Mean of the Distribution,N is the
Number of Variables in the Distribution, sigma is the Standard Daviation. From
it we get the value in figure: 3.6 and figure: 3.7 : -
26 3.2. Brief description of Methodology

Figure 3.6 Value for Skewness. Figure 3.7 Value for Skewness.

We can represent the probability distribution regarding mean value of at-

tribute,which we get from Skewness. The figure: 3.8 to figure:3.17 show some
probability distribution regarding mean value.

Figure 3.8 Probability distribution for Figure 3.9 Probability distribution for
Time. V1.
27 3.2. Brief description of Methodology

Figure 3.10 Probability distribution Figure 3.11 Probability distribution

for V2. for V3.

Figure 3.12 Probability distribution Figure 3.13 Probability distribution

for V4. for V5.

Figure 3.14 Probability distribution Figure 3.15 Probability distribution

for V6 for V28.
28 3.2. Brief description of Methodology

Figure 3.16 Probability distribution Figure 3.17 Probability distribution

for Amount. for Class.

After Skewness we find correlation coefficient,the correlation coefficient is a

number between -1 and 1 that tells you the strength and direction of a relationship
between variables.A correlation coefficient is also an effect size measure, which
tells you the practical significance of a result.

Correlation coefficient value Correlation type Meaning

-.7 to -1 very strong Negative
-.5 to -.7 strong Negative
-.3 to -.5 Moderate Negative
0 to -.3 Weak Negative
0 None Zero
0 to .3 Weak Positive
.3 to .5 Moderate Positive
.5 to .7 strong Positive
.7 to 1 very strong Positive

Table 3.1 Correlation coefficient.

29 3.2. Brief description of Methodology

Correlation coefficients are unit-free, which makes it possible to directly com-

pare coefficients between studies. Figure:3.18 show a part of correlation value.

Figure 3.18 Part of Correlation value

Using the value Correlation we draw heatmap.

Figure 3.19 Heatmap of Correlation value

30 3.2. Brief description of Methodology

3.2.3 Classification Dataset

Split data-set is used to estimate the performance of machine learning algorithms

that are applicable for prediction-based Algorithms/[Link] data set
was divided into two categories of 80% for training and 20% for testing.

3.2.4 Dataset Prepossessing

Prepossessing refer to manipulation or dropping of data before it is used in order

to ensure or enhance [Link] selection is a fundamental technique,
which selects the variables that are most relevant in the given data-set. Carefully
choosing appropriate features and removing the less important one can reduce
over-fitting, improve accuracy and reduce training [Link] first we describe the
total Data-set in 8 rowsx 31 column. Here we describe the dataset give based on
criteria count,mean, mean, max etc. Where we get 8 row and 31 column which
is showed below.
31 3.2. Brief description of Methodology

Figure 3.20 Description of data set

Then we check for null value in dataset and get zero null value there, so we
don’t have to do further [Link] of null value check is shown in figure:
3.21 and figure: 3.22
32 3.2. Brief description of Methodology

Figure 3.21 Null Value Check. Figure 3.22 Null Value Check.

After null value check we detect and filter outlier, Box plots Graphing features
is the easiest way to detect outliers. This can be done using box plots.A box plot
is a standardized graphical representation of a data distribution. It displays five
summary statistics: minimum value, first quartile (Q1), median (Q2), third quar-
tile (Q3) and maximum value.A boxplot is a simple yet effective visualization for
identifying outliers of a variable. Outliers are plotted as points at the endpoints
of the plot. These are the values that are either below (Q1-1.5IQR) or above
(Q3+1.5IQR).
33 3.2. Brief description of Methodology

Figure 3.23 Detect for Outliers.

We check outliers in attribute ’Amount’ in training dataset using box [Link]

get outliers in ’Amount’, so we filter outliers from ’Amount’.Figure 4.24 show the
state before filtering outliers and Figure 4.25 show state after outliers.

Figure 3.24 State before filtering Out- Figure 3.25 State after filtering Outlier
lier from Amount. from Amount.
34 3.2. Brief description of Methodology

3.2.5 Applying Machine learning algorithm

After preprocessing training dataset we apply various machine learning algo-

[Link] this credit fraud detection thesis, we use the machine learning algorithm
below: -
[Link] Regression(LR)
[Link] Gradient Descent (SGD)
3.K-nearest Neighbor(K-NN)
[Link] Forest(RF)
[Link] Vector Machine (SVM)
[Link] Bayes(NB)

3.2.6 Performance evaluation.

Since the dataset label is extremely imbalanced, only 0.18% of instances in the
dataset is fraud. Selecting an appropriate evaluation metrics is crucial as standard
metrics work well on balanced dataset. For example, using the accuracy as metrics
for this dataset. The model can just predict all the instances in the dataset as
non-fraud and accuracy of the model still be 99.82%. As 99.82% of the instances
in the dataset is non-fraud. Hence, we need to select ’recall’ and ’f1-score’ as the
metrics to evaluate the model.

TP
Recall = (3.2)
TP + FN

TP + TN
Accuracy = (3.3)
TP + TN + FP + FN
TP
P recision = (3.4)
TP + FP
35 3.2. Brief description of Methodology

2 ∗ P recision ∗ Recall
F 1Score = (3.5)
P recision + Recall

Where,
TP=True Positive
FP=False Positive
TN= True Negative
FN=False Negative

3.2.7 Update selected best model

After performance evaluation we select the Best machine learning model from ap-
plied [Link] the perfomance evaluation we get the value recall and f1-score
for every [Link] model which has highest recall and f1-score is considered as
best model.
After selecting best we apply Fine-Tuning Model on the parameter. After fine-
tune, the model has a slightly increase on recall than default parameter.
Then we used the resampling technique on the dataset to make the label bal-
[Link], use the SMOTE to oversample of the minority class. Next, we use
random undersampling to reduce the number of instances of majority class.
We also apply class weight to the [Link] we use fine tuned model along
with resampled dataset and class weighted dataset one after another.
Chapter 4

Experimental Result and

Discussion

In our credit card fraud detection thesis, we used the Machine learning model
below: -
[Link] Regression(LR)
[Link] Gradient Descent (SGD)
3.K-nearest Neighbor(K-NN)
[Link] Forest(RF)
[Link] Vector Machine (SVM)
[Link] Bayes(NB)
We also used: -
[Link] Tune
[Link] on Dataset
[Link] weight on Dataset
37 4.1. Experimental Result of various model

4.1 Experimental Result of various model

4.1.1 Logistic Regression(LR)

At first we use logistic regression and get the output below:-

Figure 4.1 Confusion matrix for LR.

Figure 4.2 Precision, Recall, F1 score and Accuracy for LR.

38 4.1. Experimental Result of various model

Figure 4.3 Precision Recall curve for LR.

4.1.2 Stochastic Gradient Descent (SGD)

For Stochastic Gradient Descent (SGD) we get the output below:-

Figure 4.4 Confusion matrix for SGD.

39 4.1. Experimental Result of various model

Figure 4.5 Precision, Recall, F1 score and Accuracy for SGD.

Figure 4.6 Precision Recall curve for SGD.

40 4.1. Experimental Result of various model

4.1.3 K-nearest Neighbor(K-NN)

For K-nearest Neighbor(K-NN) we get the output below:-

Figure 4.7 Confusion matrix for K-NN.

Figure 4.8 Precision, Recall, F1 score and Accuracy for K-NN.

41 4.1. Experimental Result of various model

Figure 4.9 Precision Recall curve for K-NN.

4.1.4 Random Forest(RF)

For Random Forest(RF),we get the output below:-

Figure 4.10 Confusion matrix for RF.

42 4.1. Experimental Result of various model

Figure 4.11 Precision, Recall, F1 score and Accuracy for RF.

Figure 4.12 Precision Recall curve for RF.

43 4.1. Experimental Result of various model

4.1.5 Support Vector Machine (SVM)

For Support Vector Machine (SVM),we get the output below:-

Figure 4.13 Confusion matrix for SVM.

Figure 4.14 Precision, Recall, F1 score and Accuracy for SVM.

44 4.1. Experimental Result of various model

Figure 4.15 Precision Recall curve for SVM.

4.1.6 Naive Bayes(NB)

For Naive Bayes(NB), we get the output below:-

Figure 4.16 Confusion matrix for NB.

45 4.1. Experimental Result of various model

Figure 4.17 Precision, Recall, F1 score and Accuracy for NB.

Figure 4.18 Precision Recall curve for LR.

The summary of Output of all applied Machine Learning algorithm is repre-

sented in table below: -
46 4.1. Experimental Result of various model

Methods Precision Recall F1 Score Accuracy

Logistic Regression 91% 78% 83% 99.9034%

SGD 89% 74% 80% 99.8894%

kNN 90% 87% 89% 99.9245%

Random Forest 99% 96% 98% 99.986%

SVM 99% 84% 90% 99.9421%

Naı̈ve Baiyes 52% 90% 53% 96.1202%

Table 4.1 Summarized output of all methods.

The data of summary table can be represented as graph below.

Figure 4.19 Graph of comparative output of various method.

47 4.1. Experimental Result of various model

From graph of Figure 4.19, we can see that Random Forest model has the
highest Recall And F1 score so we select Random Forest for further process.

4.1.7 Output of Random Forest(RF) using different meth-

ods

For Update Random Forest(RF), at first We apply Fine-Tune to the parameter of

Random Forest [Link] we resample the dataset and apply fine tuned Ran-
dom [Link] also use class weight and apply fine tuned Random [Link]
of this two type is below:-

[Link] Output of Random Forest(RF) with Fine-Tune and Resam-

pled dataset.

Figure 4.20 Confusion matrix for RF with Fine-Tune and Resampled dataset.
48 4.1. Experimental Result of various model

Figure 4.21 Precision, Recall, F1 score and Accuracy for RF with Fine-Tune and
Resampled dataset.

Figure 4.22 Precision Recall curve for RF with Fine-Tune and Resampled dataset.
49 4.1. Experimental Result of various model

[Link] Output of Random Forest(RF) with Fine-Tune and Class Weight.

Figure 4.23 Confusion matrix for RF with Fine-Tune and Class Weight.

Figure 4.24 Precision, Recall, F1 score and Accuracy for RF with Fine-Tune and
Class Weight.
50 4.2. Comparison

Figure 4.25 Precision Recall curve for RF with Fine-Tune and Class Weight.

4.2 Comparison

Summarising the Random Forest output of previous research work with highest
value and our default parameter Random Forest and Fine-tuned Random Forest
with resampled dataset and dataset with class weight, we get the Table:4.2

Methods Precision Recall F1 Score Accuracy

Reference RF output 96.38% 81.63% 88.39% 99.96%

Default parameter RF 99% 96% 98% 99.986%

RF with Fine-Tune and

97% 97% 97% 99.9807%
Resampled dataset

RF with Fine-Tune and

98% 96% 97% 99.9807%
Class Weight

Table 4.2 Summarized output of all Random Forest model.

From above table we draw the comparison graph below: -

51 4.2. Comparison

Figure 4.26 Comparison Graph

In above graph the Random Forest Classifier with Fine-Tuned parameter and
resampled dataset give the best recall so it is the best model. From previous
work done on credit card fraud we can see Random Forest give best performance
with default parameter,which has Precision 96.38%, Recall 81.63%, Accuracy
99.96%.In Preprocessing Dataset we use outlier detection and filtering and feature
[Link] our default parameter Random Forest give Precision 99% Recall
96% F1 score 98% Accuracy 99.986% which is better than previous. Here after
using fine tune to parameter of Random Forest and using re-sampling technique
on dataset get Precision 97%, Recall 97%, F1 score 97% Accuracy 99.98%, where
Recall increase 15.37 % and accuracy increase 0.02%. Since we focus on recall
Random Forest with fine tune and resample dataset is best.
Chapter 5

Conclusions and Future Work

5.0.1 Conclusion

Credit card frauds describe the very serious business problem. These frauds can
administrate to huge losses in business and personal. For this reason, companies
invest more and more money in developing new ideas and ways that will help to
detect and prevent [Link] that we research this topic to improve the Credit
Card Frauds [Link] previous work done on credit card fraud we can see
Random Forest give best performance with default parameter,which is Precision
96.38%, Recall 81.63%, Accuracy 99.96% .They don,t apply any further process to
increase its [Link] we use fine tune in Random Forest and also use re-
sampling technique on dataset and get result Precision 97%, Recall 97%,F1 score
97%, Accuracy 99.98% where we can see Recall increase 15.37 % and accuracy
increase 0.02%.
53

5.0.2 Future Work

Using Hybrid Model, performance maybe improve.

References

[1] Josef Vonasek. Fraud Detection in Unlabeled Payment Card Transactions,May

25, 2018.

[2] Lawrence Borah, Saleena B, Prakash B. CREDIT CARD FRAUD DETEC-

TION USING DATA MINING TECHNIQUES,Journal of Seybold Report,ISSN
NO: 1533-9211,September 2020 .

[3] Megasari Gusandra Saragih, Jacky Chin, Rianti Setyawasih, Phong Thanh
Nguyen, K. [Link] Learning Methods for Analysis Fraud Credit
Card [Link] Journal of Engineering and Advanced Tech-
nology (IJEAT),Volume-8,ISSN: 2249 – 8958, Issue-6S August 2019

[4] Nicholas Nicolaou.A System for Detecting Credit Card

[Link],05 June 2019.

[5] Joseph King-Fung [Link] Credit Card Fraud Detection using a Meta-
Learning Strategy,2011.

[6] Ronish [Link] of Machine Learning T Application of Machine

Learning Techniques in Cr echniques in Credit Car edit Card Fraud Detec-
[Link] Sciences Commons ,December 2018
55 References

[7] Sangeeta Mittal, Shivani Tyagi. Performance Evaluation of Machine Learning

Algorithms for Credit Card Fraud Detection.9th International Conference on
Cloud Computing, Data Science & Engineering (Confluence), 2019 .

[8] [Link]

[9] [Link]
algorithm

[10] [Link]

[11] [Link]

[12] [Link]
learning

[13] [Link]

[14] [Link]
and-hyper-parameter-tuning

[15] [Link]

[16] Dejan Varmedja , Mirjana Karanovic , Srdjan Sladojevic , Marko Arsen-

ovic, Andras [Link] Fraud Detection - Machine Learning meth-
ods.18th International Symposium INFOTEH-JAHORINA,volume-18,13 Oc-
tober 2019.

[17] Naresh Kumar Trivedi, Sarita Simaiya ,Umesh Kumar Lilhore, Sanjeev
Kumar [Link] Efficient Credit Card Fraud Detection Model Based on
Machine Learning Methods. International Journal of Advanced Science and
Technology,colume-29, page-3414 - 3424,13 June 2020.
56 References

[18] Vaishnavi Nath Dornadula, Geetha [Link] Card Fraud Detection using Ma-
chine Learning [Link] Computer Science, pages 631–641, 2019.

[19] Rekha [Link] Mining Techniques in Fraud DetectionJournal of Dig-

ital Forensics, Security and Law,Volume-3(2),2008 .
Appendix

Python Code:
58
59
60
61
62
63
64
65