Deepfake Detection: A Tutorial

Md Shohel Rana∗ Andrew H. Sung

Stephen F. Austin State University The University of Southern Mississippi
Nacogdoches, Texas, USA Hattiesburg, Mississippi, USA
[Link]@[Link] [Link]@[Link]

ABSTRACT In spite of the growing concern over the spread of Deepfakes,

This tutorial presents developments on the detection of Deepfakes, DL researchers have continued to make significant advances in gen-
which are realistic images, audios and videos created using deep erative modeling. For example, computer vision researchers have
learning techniques. Deepfakes can be readily used for malicious proposed methods for facial re-enactment [9], image and video
purposes and pose a serious threat to privacy and security. The transformation into different styles [10], and lip movement syn-
tutorial summarizes recent Deepfake detection techniques and eval- chronization [8]. Deepfakes have been widely used in pornography
uates their effectiveness with respect to several benchmark datasets. early on, with multiple platforms posting thousands of Deepfake
Our study finds that no single method can reliably detect all Deep- videos; more recently Deepfakes have been exploited for financial
fakes and, therefore, combining multiple methods is often necessary fraud and extortion [1]. Recognizing the emerging threat of Deep-
to achieve high detection rates. The study also suggests that more fakes, the field of Deepfake detection has attracted considerable
extensive and diverse datasets are needed to improve the accuracy attention from academic researchers, financial services, cyberse-
of detection algorithms. A taxonomy of Deepfake detection tech- curity practitioners, and law enforcement alike, and hundreds of
niques is introduced to aid future research and development in the papers have been published in various venues in just recent years.
field. We conclude by calling for the development of more effective In a recent IEEE Access paper [6], we presented a systematic
Deepfake detection methods and countermeasures to combat this literature review on Deepfake detection research regarding cur-
evolving and spreading threat. rent tools, techniques, and datasets. We classify these studies into
four categories: deep learning-based techniques, classical machine
CCS CONCEPTS learning-based methods, statistical techniques, and blockchain-
based techniques. To evaluate the performance of these techniques,
· Security and privacy → Social aspects of security and pri-
we use several benchmark datasets, including FaceForensics++
vacy.
(FF++) [7], Deepfake Detection Challenge (DFDC) [5], and Celeb-DF
KEYWORDS [4]. We compare the detection capability of different techniques on
these datasets using standard evaluation metrics such as precision,
Deepfakes, Deep learning, Taxonomy, Security and Privacy recall, and F1-score. The best-performing techniques reported in
ACM Reference Format: the study achieved a detection accuracy of over 99% on the FF++
Md Shohel Rana and Andrew H. Sung. 2023. Deepfake Detection: A Tutorial. and over 93% on the DFDC.
In Proceedings of the 9th ACM International Workshop on Security and Privacy
Analytics (IWSPA ’23), April 26, 2023, Charlotte, NC, USA. ACM, New York,
NY, USA, 2 pages. [Link]

1 INTRODUCTION
The rapid advancement of artificial neural network (ANN) based
technologies has made it easier than ever to manipulate multimedia
content. AI-enabled software tools like FaceApp [2] and FakeApp
[3] have been used for realistic-looking face swapping in videos,
leading to a surge of concern and anxiety over the propagation
of fake videos. This phenomenon, known as Deepfake, refers to
specific photo-realistic video or image content created with deep
learning (DL) techniques. Deepfakes have numerous malicious uses,
including spreading misinformation, creating political discord, or
perpetrating various cybercrimes.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@[Link]. Figure 1: Taxonomy of Deepfake detection techniques. This
IWSPA ’23, April 26, 2023, Charlotte, NC, USA taxonomy classifies the detection algorithms according to
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM. the media, the features used, the detection methods, and the
ACM ISBN 979-8-4007-0099-6/23/04. . . $15.00
[Link] clue for the detection.

55
IWSPA ’23, April 26, 2023, Charlotte, NC, USA Md Shohel Rana & Andrew H. Sung

We also introduced a taxonomy (see Figure 1) that classifies detection methods are mostly inadequate due to their limited size
Deepfake detection techniques into four categories based on their and lack of diversity, so more extensive and diverse datasets are
features. Our taxonomy provides an overview of the different cat- needed for training, testing and validation in building new models
egories and their related features, which will be useful for future to improve the accuracy of detection.
research and development of Deepfake detection methods and aims Multimedia forensics concerns the analysis of manipulated mul-
to describe and analyze common grounds and the diversity of ap- timedia contents and has been an important subject of digital foren-
proaches in current practices. sics; currently, detection methods and countermeasures for Deep-
Our taxonomy is based on Detection (i.e., Digital Media Foren- fakes are active research topics of multimedia forensics. The re-
sics, and Face Manipulation), Methods indicate the algorithmic cat- cently introduced generative AI tools (for answering questions,
egory (ML: Machine Learning, DL: Deep Learning, STAT: Statistical composing essays, etc.) have brought forth interesting challenges
method), Models represent types of model (see Figure 2) where (DL: of how to detect this kind of AI-generated material. We hope that
(CNN: Convolutional Neural Network, RNN: Recurrent Neural Net- the tutorial will be useful for researchers and practitioners of mul-
work, RCNN: Regional Convolutional Neural Network), ML: (SVM: timedia forensics and motivate the development of more effective
Support Vector Machine, RF: Random Forest, MLP: Multilayer Per- methods to combat the spreading threats to security and privacy
ceptron Neural Network, LR: Logistic Regression, k-MN: K means posed by manipulated and/or AI-synthesized multimedia.
clustering, XGB: XGBoost, ADB: AdaBoost, DT: Decision Tree,
NB: Naive Bayes, KNN: K-Nearest Neighbour, DA: Discriminant REFERENCES
Analysis), STAT: (EM: Expectation Maximization, CRA: Co-relation [1] Jon Bateman. 2020. Fakeapp. Webpage. (Aug. 2020). Retrieved March 11, 2023
from [Link]
Analysis)), Features (Special Artifacts, Visual Artifacts, Biological o-be-used-in-financial-scams-pub-82469.
Artifacts, Face Landmarks, Spatio-temporal Consistency, Texture, [2] 2017. Faceapp. Application. (2017). Retrieved March 11, 2023 from [Link]
Frequency Domain Analysis, Latent Feature, Generative Adver- [Link]/.
[3] 2018. Fakeapp. Application. (2018). Retrieved March 11, 2023 from [Link]
sarial Network based feature, Mesoscopic features, Intra-frame [Link]/en/soft/fakeapp/.
inconsistency, Constrastive and Photoresponsive PRNU pattern, [4] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df:
Image Metadata, Augmentation & Steganalysis, Others), Datasets a large-scale challenging dataset for deepfake forensics. In 2020 IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), 3204ś3213. doi:
(FaceForensics++ (FF++), Deepfake Detection: DeepFake Forensics 10.1109/CVPR42600.2020.00327.
V1 (CELEB-A), DeepFake Forensics V2 (CELEB-DF), Deepfake De- [5] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning
face attributes in the wild. In 2015 IEEE International Conference on Computer
tection Challenge (DFDC), Deepfake-TIMIT, DeeperForensics-1.0 Vision (ICCV), 3730ś3738. doi: 10.1109/ICCV.2015.425.
(DF-1.0), Deep Fakes (DFS), Fake Faces in the Wild (FFD), FakeET, [6] Md Shohel Rana, Mohammad Nur Nobi, Beddhu Murali, and Andrew H. Sung.
Face Shifter, Deepfake (DF), Inconsistent Head Poses (UADFV), 2022. Deepfake detection: a systematic literature review. IEEE Access, 10, 25494ś
25513. doi: 10.1109/ACCESS.2022.3154404.
Tampered Face (MANFA), and others). [7] Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus
Thies, and Matthias Niessner. 2019. Faceforensics++: learning to detect manip-
ulated facial images. In 2019 IEEE/CVF International Conference on Computer
Vision (ICCV), 1ś11. doi: 10.1109/ICCV.2019.00009.
[8] Supasorn Suwajanakorn, Steven M. Seitz, and Ira Kemelmacher-Shlizerman.
2017. Synthesizing obama: learning lip sync from audio. ACM Trans. Graph.,
36, 4, Article 95, 13 pages. doi: 10.1145/3072959.3073640.
[9] Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and
Matthias Niessner. 2018. Face2face: real-time face capture and reenactment of
rgb videos. Commun. ACM, 62, 1, 96ś104. doi: 10.1145/3292039.
[10] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired
image-to-image translation using cycle-consistent adversarial networks. In
2017 IEEE International Conference on Computer Vision (ICCV), 2242ś2251. doi:
10.1109/ICCV.2017.244.

Figure 2: The allocation of categories of detection models. (a)

CNN (b) RNN (c) RCN (d) Statistical, and (e) ML.

2 CONCLUSION
This tutorial provides a comprehensive overview of the current
state of Deepfake detection research, including available datasets
and performance analysis, with the literature covered in the study
mostly focused on Deepfake videos.
Our study reveals that there is no single method that can reliably
detect all Deepfakes, and that combining multiple methods would
seem essential to achieving high detection accuracy. Additionally,
we found that current datasets for training and evaluating Deepfake

56