Image Classification Using Convolutional
Neural Network (CNN)
Face Mask Detection Using Faster R-CNN
Submitted By:
Vaishal Shah (249538050)
Samikshya Dhakal (259642980)
Deepshikha (259641760)
Submitted To:
Dr. Arghavan Asad
Introduction
Object detection is a computer vision algorithm which not only finds the type of objects in an
image but also their location by bounding boxes around the objects. Unlike in image
classification, where only one label is allowed in one image, object detection gives the ability to
identify and characterize the number of objects in the same frame.
It is important because it enables real-world applications where both recognition and localization
are essential. Examples of these are autonomous vehicles and obstacle detection (of pedestrians
and traffic signs), surveillance in preventing suspicious activity, quality control in
manufacturing, and public health care (recommending appropriate use of masks in densely
populated places). Object detection allows many AI-based decision-making systems since it
gives the “what” and the “where.”
In this project we develop and train an object detection model that will detect whether a person
is correctly wearing a mask, inaccurately wearing a mask, or does not use a mask at all. The
model identifies more than one face in an image, and it makes a bounding box with a class-label
and confidence score. We selected Faster R-CNN because of its relative performance between
accuracy and depends on and thus a likely contending approach to small to medium sized
datasets.
Objective
The main goals of the project were as follows:
a. Acquire and preprocess the Face Mask Detection dataset from Kaggle.
b. Implement the Faster R-CNN object detection architecture using a ResNet-50 backbone.
c. Train the model to detect and classify faces into three categories: with_mask,
without_mask, and mask_weared_incorrect.
d. Evaluate the trained model’s performance both quantitatively and qualitatively.
e. Visualize the predictions with bounding boxes, labels, and confidence scores.
Methodology
Technologies, Libraries, and Dataset
The project was implemented in Python using PyTorch and Torchvision for model training and
inference. Additional libraries such as NumPy, Pillow (PIL), and Matplotlib were used for data
processing and visualization. The training was performed in Google Colab to leverage GPU
acceleration. The Kaggle data was downloaded directly through the API.
Dataset Overview
The Face Mask Detection dataset, created by Andrew Mvd and hosted on Kaggle, contains 853
annotated images in Pascal VOC format. The images are labeled into three classes:
1. mask_weared_incorrect
2. with_mask
3. without_mask
The dataset was divided into training (80%), validation (10%), and testing (10%) subsets to
ensure unbiased evaluation. Images vary in resolution and contain faces in different poses,
lighting conditions, and backgrounds.
Data Preprocessing
The preprocessing stage involved unzipping the dataset, parsing XML annotations to extract
bounding box coordinates, and splitting the data into train, val, and test sets. Data augmentation
techniques such as random horizontal flipping were applied to the training set to improve
generalization. All images were normalized and converted into PyTorch tensors for model
compatibility.
The directory structure was organized into separate folders for images and annotations for each
split. This format was compatible with the custom PyTorch Dataset class used to load data
during training and evaluation.
Model Architecture
The Faster R-CNN model used in this project was based on fasterrcnn_resnet50_fpn pretrained
on the COCO dataset. The architecture consists of:
• A ResNet-50 backbone with a Feature Pyramid Network (FPN) for multi-scale feature
extraction.
• A Region Proposal Network (RPN) for generating candidate object regions.
• A classification head to predict object classes and a regression head to refine bounding
box coordinates.
The final classification layer was modified to output four classes: three object classes and one
background class.
The optimizer used was Stochastic Gradient Descent (SGD) with a learning rate of 0.005,
momentum of 0.9, and weight decay of 0.0005. A StepLR scheduler reduced the learning rate by
a factor of 0.1 every 5 epochs. The model was trained for 8 epochs with a batch size of 4.
Model Training
Training was conducted in Google Colab with GPU acceleration. During each epoch, images and
their corresponding bounding boxes were passed to the model, and a combination of
classification and bounding box regression losses was computed. The optimizer updated the
model parameters after each iteration, and the learning rate scheduler adjusted the learning rate at
specified intervals.
Loss values were monitored to ensure the model was learning effectively. After training, the
model weights were saved for later evaluation and inference.
Evaluation
The model was evaluated on the test dataset, comprising 10% of the total images. Evaluation was
performed using both quantitative and qualitative methods. For the quantitative assessment,
Intersection-over-Union (IoU) with a threshold of 0.5 was applied to determine correct
detections (True Positives). The Faster R-CNN model achieved high IoU matches for the
with_mask and without_mask categories, while the performance for the mask_weared_incorrect class
was lower due to its smaller representation in the training set.
Qualitative evaluation was conducted by visually inspecting detection outputs. The model
consistently produced accurate bounding boxes with high confidence scores, even in images with
multiple faces and partial occlusion. Occasional misclassifications occurred, typically involving
confusion between mask_weared_incorrect and without_mask.
Results
The trained Faster R-CNN model was evaluated on the held-out test dataset, and figure below
presents a representative output image. In this example, the model successfully detected multiple
faces within the same frame and classified them into all three target categories:
• with_mask — correctly identified with a confidence score of 0.92.
• without_mask — correctly identified with a confidence score of 0.85.
• mask_weared_incorrect — correctly identified with a confidence score of 0.88.
The bounding boxes in the output are tightly aligned with the detected faces, and the assigned
labels match the ground truth annotations for this image. This single example highlights the
model’s capability to detect and distinguish between different mask-wearing conditions
simultaneously, even when multiple individuals appear in the same scene.
The qualitative results confirm that Faster R-CNN is effective for the face mask detection task,
providing accurate localization and classification across all three classes. Further improvements,
such as adding more examples of the mask_weared_incorrect category during training, could
help the model handle edge cases even more reliably.
Conclusion
In this project, Faster R-CNN was successfully implemented to detect and classify face mask
usage. The model achieved reliable detection for with_mask and without_mask categories, while
performance for mask_weared_incorrect could be improved by collecting more training data and
applying additional augmentation techniques. This work demonstrates the practical applicability
of deep learning-based object detection in public health and safety monitoring.
Analysis Questions
1. Why did you choose this object detection algorithm (YOLO, Faster R-CNN, SSD, etc.)
over others?
We chose Faster R-CNN because it offers a strong balance between detection accuracy and
localization precision, especially on smaller datasets. Compared to single-stage detectors like
YOLO or SSD, Faster R-CNN generally achieves higher accuracy for detecting small or partially
occluded objects, which is important for face mask detection.
2. What dataset did you use for training and testing your model, and how many
images/classes did it contain?
We used the Face Mask Detection dataset from Kaggle, which contains 853 annotated images.
The dataset includes three classes: mask_weared_incorrect, with_mask, and without_mask. The
dataset was split into 80% training, 10% validation, and 10% testing.
3. What framework and tools did you use to build and train your model? Why did you
choose them?
We used PyTorch and Torchvision for building and training the Faster R-CNN model, along
with Google Colab for GPU acceleration. PyTorch was chosen for its flexibility, strong support
for object detection models, and straightforward integration with pretrained weights. Google
Colab was used because it provides free GPU access and easy integration with the Kaggle API
for downloading datasets.
4. What were the main hyperparameters you selected for training (learning rate, batch size,
number of epochs, etc.)? Why?
The main hyperparameters were:
• Learning rate: 0.005 — provided a balance between convergence speed and stability.
• Batch size: 4 — chosen due to GPU memory limitations.
• Number of epochs: 8 — enough to achieve reasonable accuracy without overfitting.
• Momentum: 0.9 and weight decay: 0.0005 — to stabilize training and prevent overfitting.
5. How did your model perform on the test images? What metrics did you use to evaluate
it?
The model performed well for the with_mask and without_mask classes, achieving high-
confidence detections in most test images. Intersection-over-Union (IoU) with a threshold of 0.5
was used to evaluate detection accuracy. The mask_weared_incorrect class had lower
performance due to fewer examples in the training set.
6. Display sample outputs. Were there any incorrect classifications? Explain why you think
these errors happened.
Yes, there were occasional misclassifications. For example, some mask_weared_incorrect cases
were predicted as without_mask. This likely occurred because of the visual similarity between
the two classes and the limited number of training samples for mask_weared_incorrect. Figures
7–10 in the Results section display both correct and incorrect predictions.
7. What challenges did you encounter during dataset preparation, model training, or
testing? How did you address them?
Challenges included:
• Limited data for the rare class mask_weared_incorrect.
• GPU memory restrictions that required a small batch size.
• Time limits in Colab sessions.
These were addressed by applying data augmentation, adjusting batch size, and
optimizing code execution to fit within Colab’s runtime.
8. What are the advantages and limitations of your chosen object detection algorithm based
on your results?
Advantages:
• High detection accuracy.
• Effective at detecting multiple small objects in an image.
Limitations:
• Slower inference speed compared to single-stage detectors like YOLO.
• Requires a GPU for efficient training.
9. If you were to improve this project, what modifications would you make to either the
model, dataset, or training process?
We would:
• Collect more samples for the mask_weared_incorrect class.
• Apply additional data augmentation to improve generalization.
• Fine-tune the model for more epochs and possibly experiment with YOLOv8 for faster
inference.
10. What did you learn from this project about object detection and deep learning in
general?
We have understood the entire end to end pipeline of applying object detection model, including
preparing the datasets, to validate its performance. We also obtained real-life experience on how
difficult deep learning training can be, how it is critical to have balanced data, and the speed-
accuracy trade-offs in object detection algorithms.