Confusion Matrix
Dr. S. N. Ahsan
Confusion Matrix
A confusion matrix is a table that is used to define the performance of a
classification algorithm. A confusion matrix visualizes and summarizes the
performance of a classification algorithm.
A Confusion matrix is an N x N matrix used for evaluating the performance of a
classification model, where N is the number of target classes. The matrix
compares the actual target values with those predicted by the machine learning
model.
Understanding Confusion Matrix
The following 4 are the basic terminology which will help us in
determining the metrics we are looking for.
1. True Positives (TP): when the actual value is Positive and predicted
is also Positive.
2. True negatives (TN): when the actual value is Negative and
prediction is also Negative.
3. False positives (FP): When the actual is negative but prediction is
Positive. Also known as the Type 1 error
4. False negatives (FN): When the actual is Positive but the prediction
is Negative. Also known as the Type 2 error
Understanding Confusion Matrix
Classification Measure
Basically, it is an extended version of the confusion matrix. There are
measures other than the confusion matrix which can help achieve
better understanding and analysis of our model and its performance.
1. Accuracy
2. Precision
3. Recall (TPR, Sensitivity)
4. F1-Score
5. FPR (Type I Error)
6. FNR (Type II Error)
Accuracy
• Accuracy simply measures how often the classifier makes the correct
prediction. It’s the ratio between the number of correct predictions
and the total number of predictions.
• The accuracy metric is not suited for imbalanced
classes. Accuracy has its own disadvantages, for imbalanced data,
when the model predicts that each point belongs to the majority class
label, the accuracy will be high. But, the model is not accurate.
Accuracy
Precision
It is a measure of correctness that is achieved in true prediction. In
simple words, it tells us how many predictions are actually positive out
of all the total positive predicted.
Recall
It is a measure of actual observations which are predicted correctly,
i.e. how many observations of positive class are actually predicted as
positive. It is also known as Sensitivity. Recall is a valid choice of
evaluation metric when we want to capture as many positives as
possible.
F-measure / F1-Score
• The F1 score is a number between 0 and 1 and is the harmonic mean of
precision and recall. We use harmonic mean because it is not sensitive
to extremely large values, unlike simple averages.
• F1 score sort of maintains a balance between the precision and
recall for your classifier. If your precision is low, the F1 is low and if
the recall is low again your F1 score is low.
Analysis of Confusion Matrix
After creating the matrix, the scientists analyze their sample data.
Assume the scientists predict that 350 test samples contain the genetic
variant and 150 samples don't. If they determine the actual number of
samples containing the variant is 305, the actual number of samples
without the variant is 195. These values become the "true" values in
the matrix and the scientists enter the data in the table:
Confusion Matrix Example