International Journal of

Molecular Sciences

Article
DeepPGD: A Deep Learning Model for DNA Methylation
Prediction Using Temporal Convolution, BiLSTM, and
Attention Mechanism
Shoryu Teragawa 1, *, Lei Wang 1 and Yi Liu 2

1 School of Software, Dalian University of Technology, Dalian 116024, China; [Link]@[Link]

2 School of Engineering, University of Southern Queensland, 487-535 West Street,
Toowoomba, QLD 4350, Australia; [Link]@[Link]
* Correspondence: frozen@[Link]

Abstract: As part of the field of DNA methylation identification, this study tackles the challenge of
enhancing recognition performance by introducing a specialized deep learning framework called
DeepPGD. DNA methylation, a crucial biological modification, plays a vital role in gene expression
analyses, cellular differentiation, and the study of disease progression. However, accurately and effi-
ciently identifying DNA methylation sites remains a pivotal concern in the field of bioinformatics. The
issue addressed in this paper is the presence of methylation in DNA, which is a binary classification
problem. To address this, our research aimed to develop a deep learning algorithm capable of more
precisely identifying these sites. The DeepPGD framework combined a dual residual structure involv-
ing Temporal convolutional networks (TCNs) and bidirectional long short-term memory (BiLSTM)
networks to effectively extract intricate DNA structural and sequence features. Additionally, to meet
the practical requirements of DNA methylation identification, extensive experiments were conducted
across a variety of biological species. The experimental results highlighted DeepPGD’s exceptional
performance across multiple evaluation metrics, including accuracy, Matthews’ correlation coefficient
(MCC), and the area under the curve (AUC). In comparison to other algorithms in the same domain,
DeepPGD demonstrated superior classification and predictive capabilities across various biological
Citation: Teragawa, S.; Wang, L.; Liu,
species datasets. This significant advancement in algorithmic prowess not only offers substantial
Y. DeepPGD: A Deep Learning Model
technical support, but also holds potential for research and practical implementation within the
for DNA Methylation Prediction
Using Temporal Convolution,
DNA methylation identification domain. Moreover, the DeepPGD framework shows potential for
BiLSTM, and Attention Mechanism. application in genomics research, biomedicine, and disease diagnostics, among other fields.
Int. J. Mol. Sci. 2024, 25, 8146.
[Link] Keywords: RNA methylation; deep learning; gene expression
ijms25158146

Academic Editor: Abdelkrim

Hmadcha
1. Introduction
Received: 3 May 2024 In 1942, the concept of epigenetics was introduced for the first time, defined as the
Revised: 7 June 2024 heritable changes in gene expression without altering the DNA nucleotide sequence [1,2].
Accepted: 25 June 2024
In the 1950s, MacLintock and Brink experimentally confirmed the validity of epigenetics
Published: 26 July 2024
and were awarded the Nobel Prize in 1983.
With the advancement of life sciences, various epigenetic mechanisms have gradually
been discovered. Numerous epigenetic processes such as acetylation and the methylation
Copyright: © 2024 by the authors.
of proteins have been identified [3]. A systematic study of epigenetic mechanisms holds
Licensee MDPI, Basel, Switzerland.
crucial significance in deepening the understanding of mysterious biological phenomena
This article is an open access article and their underlying mechanisms. Since the 1980s, countries worldwide have established
distributed under the terms and layouts and plans for epigenetics research. In 2003, the Human Epigenome Project was
conditions of the Creative Commons launched with the aim of mapping the variable methylation sites of the human genome [4,5].
Attribution (CC BY) license (https:// In 2006, researchers from China, Japan, South Korea, and Singapore convened the first Asian
[Link]/licenses/by/ Epigenome Alliance meeting, which has since become a vital platform for communication
4.0/). and collaboration in the development of Asian epigenetic research. Hence, as researchers

Int. J. Mol. Sci. 2024, 25, 8146. [Link] [Link]

Int. J. Mol. Sci. 2024, 25, 8146 2 of 15

in the field of bioinformatics, it is imperative to focus on epigenetics research, explore

advanced technologies, and strive for productive outcomes in the realm of epigenetic
scientific investigation [6].
DNA methylation stands as a significant area of epigenetic study [7]. It has been
discovered that DNA methylation commonly occurs at the fifth carbon atom of cytosine
in the CpG dinucleotides of DNA sequences. The methyl group (-CH3) from S-adenosyl
methionine (SAM) is transferred by DNA methyltransferase (DNMT) to the fifth carbon
atom of cytosine, forming 5-methylcytosine [8]. Simultaneously, SAM becomes S-adenosyl
homocysteine (SAH) after demethylation. According to incomplete statistical data, there are
approximately 28 million CpG sites in the human genome, with the majority of them being
methylated, accounting for about 60–80% of the total count [9]. The remaining sites are
primarily found in the promoter and exon regions of genes, often defined as CpG islands
(CGIs) with GC contents exceeding 50% and lengths ranging from 500 to 1000 bp [10]. Pre-
vious research indicates that the impact of DNA methylation frequently results in adverse
effects on gene expression [11]. The mechanisms encompass the following aspects: 1. the
process of DNA sequence binding with transcription factors is influenced by DNA methy-
lation; 2. methylated CpG sites are likely to bind with CpG-binding proteins, resulting in a
heterochromatin structure under the influence of histone-modifying enzymes; 3. The inter-
action between histone modifiers and DNA methyltransferases promotes the formation of
fixed chromatin structures. DNA methylation demonstrates distinct characteristics in differ-
ent cells, a contrast that becomes pronounced between normal and cancer cells [12]. Studies
demonstrate that DNA methylation is implicated in the development of almost all cancers.
Furthermore, DNA methylation inhibits the deactivation of transposons, which can be
inserted into host DNA and lead to gene inactivation, ensuring the normal function of the
body [13,14]. Currently, the three extensively studied types of DNA methylation include
N6-Methyladenine (6 mA), 5-Hydroxymethylcytosine (5 hmC), and N4-Methylcytosine
(4 mC). Each type has distinct biological functions. N4-Methylcytosine (4 mC) plays diverse
roles in regulating DNA replication, distinguishing itself from DNA, maintaining the cell
cycle, correcting replication errors, and modulating gene expression. Importantly, 4 mC pro-
tects host DNA from degradation by restriction enzymes [15]. 5-Hydroxymethylcytosine
(5 hmC), a product of 5-Methylcytosine (5 mC) demethylation, is implicated in neural de-
velopment and tumorigenesis [16]. N6-Methyladenine (6 mA), a methylated base prevalent
in prokaryotes, is primarily involved in host defense mechanisms [17].
In conclusion, as a prevalent epigenetic phenomenon, DNA methylation plays a vital
role in maintaining the stability of genetic information concerning gene expression reg-
ulation, chromatin structural variations, and transposon inactivation. Consequently, the
establishment of efficient DNA methylation prediction methods not only advances biologi-
cal research, but also holds significant implications for disease diagnosis, classification, and
personalized treatment development [18,19].
Although detecting the DNA methylation status using experimental means can yield
more accurate predictive results, the high economic cost limits the advancement of DNA
methylation research to some extent [20]. The widespread use of computational models
supplements DNA methylation research. Predicting DNA methylation using machine
learning algorithms has become a focal point in the field of bioinformatics, offering a
convenient approach to exploring whole-genome DNA methylation patterns across mul-
ticellular and multitype tissues. Although many traditional machine learning methods
have been employed for DNA methylation prediction [21,22], these methods struggle
to effectively extract feature information, necessitating enhancements in the predictive
accuracy. For example, 4 mCPred [15] uses an SVM to predict 4 mC DNA methylation,
and MM-6 mAPred [17] employs the Markov algorithm to predict 6 mA DNA methyla-
tion. Compared with these traditional algorithms, the algorithm presented in this paper
could extract greater feature information such as multidimensional structural information.
Additionally, DeepPGD could autonomously learn feature representations from the input,
thereby reducing the reliance on manual feature [Link] learning models can
Int. J. Mol. Sci. 2024, 25, 8146 3 of 15

automatically extract complex features from data without the need for manual feature
engineering. Additionally, these models can efficiently and accurately process and analyze
large volumes of data [18–20].
In this paper, the model incorporated a CNN block with an attention mechanism and
a BiLSTM block with an attention mechanism. The CNN block used convolutional kernels
of different sizes to increase the dimensionality of the structural feature extraction. The
BiLSTM block extracted sequential information, enhancing the diversity of the feature
extraction by the network. The DeepPGD model could effectively classify methylated
and nonmethylated DNA. This motivation propelled us to further investigate this topic
using deep learning technology. Residual network deep learning models exhibit robust
feature learning capabilities, effectively extracting highly complex and crucial nonlinear
features, thereby improving the model’s predictive performance for the DNA methylation
status. To enhance the accuracy of DNA methylation prediction, we applied our deep
learning model to a large methylation dataset and compared it with existing deep learning
prediction methods. The results indicated that our approach enhanced the accuracy of
DNA methylation prediction, thus exhibiting significant implications for the advancement
of deep learning technology research in DNA methylation prediction.
Predictive algorithms for DNA methylation may bring about a series of biological
implications. These include:
Understanding epigenetic regulatory mechanisms: DNA methylation is a crucial
epigenetic modification that plays a key role in gene expression and cell differentiation.
Predictive algorithms can help reveal patterns of DNA methylation across different cell
types and physiological states, thereby enhancing our understanding of epigenetic regula-
tory mechanisms.
Discovery of biomarkers: DNA methylation plays an important role in the onset
and progression of diseases such as cancer and neurodegenerative disorders. Predictive
algorithms can assist in the identification of DNA methylation patterns associated with
specific diseases, aiding the discovery of new biomarkers and improving the early diagnosis
and treatment of diseases.
Guidance for genome editing and therapy: DNA methylation plays a significant role
in genome stability and functional gene regulation. Predictive algorithms can help identify
and interpret the location and function of DNA methylation in the genome, providing
guidance and a reference for genome editing and therapy.

2. Results
In order to investigate the practical performance of the model proposed in this study,
publicly available datasets comprising 10 distinct DNA methylation datasets were em-
ployed as experimental materials. The experimental results were comprehensively com-
pared with those of benchmark algorithms, revealing substantial advantages. This chapter
provides a visual demonstration of the comprehensive outstanding performance of the
proposed algorithm, emphasizing its commendable predictive capabilities across multiple
evaluation metrics.

2.1. Experimental Results

Through experimentation, DeepPGD exhibited several advantages across diverse
datasets, particularly in terms of Matthews’ correlation coefficient (MCC) and the area
under the curve (AUC). The advantages of DeepPGD are highlighted below (Tables 1–3).
Higher Matthews’ correlation coefficient (MCC): DeepPGD demonstrated relatively
higher MCC values across most biological species. Through experiments, it was found that
the DeepPGD algorithm outperformed the comparative algorithms in 9 out of 10 datasets,
except for the 5 hmC H. sapiens dataset. Particularly significant advantages were observed
in the experimental results for the 4 mc C. equisetifolia and S. cerevisiae datasets. The iDNA-
ABT [23] algorithm performed best on the 5 hmC H. sapiens dataset. However, the iDNA-
MS [24] algorithm exhibited a relatively mediocre performance across all datasets. The MCC
Int. J. Mol. Sci. 2024, 25, 8146 4 of 15

metric evaluates the predictive performance of a classification model, being particularly

useful for imbalanced datasets. Elevated MCC values signified that DeepPGD maintained
a well-balanced performance in classifying positive and negative instances, potentially
conferring advantages in reliability and robustness for DNA methylation prediction tasks.

Table 1. ACC comparison of each method.

4 mc 4 mc 4 mc 5 hmC 6 mA 6 mA 6 mA 6 mA 6 mA 6 mA
C. S. D. T. Xoc
F. vesca H. sapiens A. thaliana C. elegans H. sapiens
equisetifolia cerevisiae melanogaster thermophile BLS256
iDNA-MS 0.7262 0.8217 0.6962 0.9475 0.834 0.8549 0.9027 0.8799 0.8602 0.8449
iDNA-ABT 0.8251 0.842 0.7027 0.9492 0.8538 0.8903 0.9122 0.898 0.874 0.8694
DeepPGD 0.8579 0.8554 0.7179 0.948 0.8636 0.9127 0.9255 0.9045 0.8802 0.8824
The best value in each column is in bold.

Table 2. MCC comparison of each method.

4 mc 4 mc 4 mc 5 hmC 6 mA 6 mA 6 mA 6 mA 6 mA 6 mA
C. S. D. T. Xoc
F. vesca H. sapiens A. thaliana C. elegans H. sapiens
equisetifolia cerevisiae melanogaster thermophile BLS256
iDNA-MS 0.452 0.6433 0.395 0.8966 0.6697 0.7099 0.805 0.7623 0.7342 0.693
iDNA-ABT 0.6517 0.6842 0.4064 0.9009 0.7088 0.7808 0.8244 0.796 0.754 0.7394
DeepPGD 0.716 0.711 0.436 0.8976 0.7273 0.8121 0.8512 0.8091 0.7688 0.7649
The best value in each column is in bold.

Table 3. AUC comparison of each method.

4 mc 4 mc 4 mc 5 hmC 6 mA 6 mA 6 mA 6 mA 6 mA 6 mA
C. S. D. T. Xoc
F. vesca H. sapiens A. thaliana C. elegans H. sapiens
equisetifolia cerevisiae melanogaster thermophile BLS256
iDNA-MS 0.79 0.8991 0.7612 0.962 0.9093 0.9311 0.962 0.9507 0.926 0.9251
iDNA-ABT 0.8555 0.907 0.7537 0.9553 0.9184 0.9433 0.9544 0.951 0.931 0.9261
DeepPGD 0.9181 0.9285 0.7763 0.9655 0.9354 0.9662 0.9709 0.9664 0.943 0.9497
The best value in each column is in bold.

Higher area under the curve (AUC): DeepPGD presented higher AUC values across
most biological species. The AUC measures the overall performance of a classification
model across different thresholds and is often employed to assess the classifier’s dis-
criminative ability. Increased AUC values suggested that DeepPGD possessed a strong
discrimination ability between positive and negative instances, potentially enhancing its
capacity to identify DNA methylation sites.
Comprehensive performance superiority: DeepPGD demonstrated superiority across
various metrics using multiple biological species, indicating a favorable performance across
multiple aspects. Its comprehensive performance superiority implied that DeepPGD could
offer stable and proficient performance across diverse data contexts.
Adaptability to biological species: DeepPGD excelled across different biological
species, implying a certain degree of adaptability to various organisms. The ability to
adapt to different biological species is crucial when analyzing diverse types of DNA methy-
lation datasets, and DeepPGD’s performance in this aspect enhanced its appeal.
Overall, based on these datasets, the DeepPGD algorithm consistently exhibited
superior performance, particularly in terms of Matthews’ correlation coefficient and the area
under the curve. The iDNA-ABT algorithm also demonstrated competitive performance
across most biological species, while iDNA-MS appeared to exhibit a comparatively lower
performance in most performance metrics.

2.2. Box-Plot Analysis

Based on the box plots derived from the experimental data, the following analytical
conclusions were drawn. In terms of accuracy (ACC), the box plots demonstrated that the
DeepPGD algorithm exhibited higher accuracy across various biological species datasets,
 2.2. Box-Plot Analysis
2.2. Box-Plot
Based onAnalysis
the box plots derived from the experimental data, the following analytical
conclusions
Based on werethe drawn.
box plots In terms
derived of from
accuracy (ACC), the box
the experimental plots
data, thedemonstrated
following analyticalthat the
DeepPGD algorithm exhibited higher accuracy across various biological
conclusions were drawn. In terms of accuracy (ACC), the box plots demonstrated that the species datasets,
Int. J. Mol. Sci. 2024, 25, 8146 markedly algorithm
DeepPGD surpassingexhibited
the iDNA-ABT and iDNA-MS
higher accuracy algorithms.
across various Although
biological species iDNA-ABT
datasets,
5 of 15
and iDNA-MS
markedly displayed
surpassing the certain
iDNA-ABT fluctuations
and iDNA-MSin accuracy, their overall
algorithms. Although levelsiDNA-ABT
remained
comparatively
and iDNA-MS lower. Concerning
displayed Matthews’ correlation
certain fluctuations in accuracy,coefficient (MCC),
their overall the DeepPGD
levels remained
algorithm
comparativelysignificantly
lower. outperformed
Concerning the
Matthews’ other two algorithms
correlation
markedly surpassing the iDNA-ABT and iDNA-MS algorithms. Although iDNA-ABT coefficientin terms
(MCC), of MCC
the values
DeepPGD
for most
algorithm
and iDNA-MSdatasets,
significantlyindicating
displayed its
outperformed superior predictive
the other in
certain fluctuations two performance
algorithms
accuracy, in
theirinoverallclassification
terms of MCC
levels tasks.
values
remained
Meanwhile,
for
comparatively the distribution
most datasets, indicating
lower. ranges of the MCC
its superior
Concerning Matthews’ valuesperformance
predictive for iDNA-ABT
correlation and
coefficientin(MCC), iDNA-MS
classification were
tasks.
the DeepPGD
wider, suggesting
Meanwhile,
algorithm potential
the distribution
significantly instability.
ranges
outperformed of theIn
the the
MCC context
other two of
values forthe area under
iDNA-ABT
algorithms and
in terms the curve
iDNA-MS
of MCC(AUC),were
values
the
forDeepPGD
wider, suggesting
most algorithm
datasets, demonstrated
potential
indicatinginstability. elevated
In the
its superior AUC of
context
predictive values for most
the area
performance underindatasets,
the curve indicating
classification (AUC),
tasks.
larger
the areas under
DeepPGD
Meanwhile, the the ROC
algorithm curves
rangesand
demonstrated
distribution stronger
elevated
of the MCC AUCclassification
values values capabilities.
for
for iDNA-ABT andIn
most datasets, contrast,
iDNA-MSindicatingthe
were
AUC
larger values
wider,areas for
suggesting iDNA-ABT
under the and
ROC curves
potential iDNA-MS
and stronger
instability. were generally relatively
classification
In the context lower.
capabilities.
of the area under the In contrast,
curve (AUC), the
AUC In consideration
the DeepPGD
values foralgorithm
iDNA-ABTof theseandmetrics,
demonstrated
iDNA-MS the following
elevated
were AUC conclusions
values
generally for most
relatively could be drawn.
datasets,
lower. The
indicating
DeepPGD
larger
In areas algorithm
under thepossessed
consideration ofROC
thesecurves a distinct
and the
metrics, advantage
stronger in DNAcapabilities.
classification
following conclusions methylation
could beIndrawn. recognition
contrast, the
The
AUC excelling
tasks,
DeepPGD values forin
algorithm iDNA-ABT
accurate
possessedand aiDNA-MS
classification
distinctand were generally
prediction.
advantage in DNArelatively
It [Link]
consistently
methylation out-
recognition
tasks, In
standing consideration
performance
excelling of these
across
in accurate themetrics,
accuracy,
classification the following
MCC,
and and AUC conclusions
prediction. evaluation could
It consistently metricsbe drawn.
(Figures
exhibited The
1–
out-
DeepPGD
standing
3). algorithm possessed a distinct advantage in DNA methylation
performance across the accuracy, MCC, and AUC evaluation metrics (Figures 1– recognition tasks,
excelling in accurate classification and prediction. It consistently exhibited outstanding
3).
performance across the accuracy, MCC, and AUC evaluation metrics (Figures 1–3).

Figure 1. Box-plot analysis based on accuracy.

Figure 1. Box-plot analysis based on accuracy.
Figure 1. Box-plot analysis based on accuracy.

Figure 2. Box-plot analysis based on MCC.

Figure 2. Box-plot analysis based on MCC.
Figure 2. Box-plot
2.3. Ablation analysis based on MCC.
Experiments
To gain a deeper understanding of the algorithm’s performance, we conducted ab-
lation experiments on one representative dataset from each of the 4 mC, 5 hmC, and
6 mA categories. Based on the ablation experiment results, several analyses were derived
(Figure 4).
Int. J. Mol. Sci. 2024, 25, x FOR PEER REVIEW 6 of 15
Int. J. Mol. Sci. 2024, 25, 8146 6 of 15

Figure 3. Box-plot analysis based on the AUC.

2.3. Ablation Experiments

To gain a deeper understanding of the algorithm’s performance, we conducted abla-
tion experiments on one representative dataset from each of the 4 mC, 5 hmC, and 6 mA
categories. Based on the ablation experiment results, several analyses were derived (Fig-
ure 4).3. Box-plot analysis based on the AUC.
Figure
Figure 3. Box-plot analysis based on the AUC.

2.3. Ablation Experiments

To gain a deeper understanding of the algorithm’s performance, we conducted abla-
tion experiments on one representative dataset from each of the 4 mC, 5 hmC, and 6 mA
categories. Based on the ablation experiment results, several analyses were derived (Fig-
ure 4).

Figure 4.
Figure 4. The
The introduction
introduction of
of models
models in
in the
the ablation
ablation experiments.
experiments. The
The left
left part
part of
of the
the figure
figure shows
shows
the original
the originalmodel
modelwith
withthe
the LSTM
LSTMmodule
moduleremoved.
removed. The
The middle
middlepart
part of
of the
the figure
figure shows
shows thethe original
original
model. The right part of the figure shows the original model with the TCN module removed.
model. The right part of the figure shows the original model with the TCN module removed.

Impactononaccuracy
Impact accuracy (ACC):
(ACC): The ablation
The ablation experiment
experiment results revealed
results revealed that the
that the DeepPGD
DeepPGD
model model consistently
consistently achieved higherachieved higher
accuracy foraccuracy
all threefor all threespecies
biological biological species
datasets whenda-
tasets when both BiLSTM and a TCN were used. This underscored
both BiLSTM and a TCN were used. This underscored the crucial role of BiLSTM and the the crucial role of
Figure
BiLSTM 4. The
andintroduction
the TCN of
in models in
enhancing the ablation
accuracy experiments.
within the
TCN in enhancing accuracy within the DeepPGD model. Conversely, removing either The left
DeepPGD part of the
model. figure shows
Conversely,the
Int. J. Mol. Sci. 2024, 25, x FOR PEER the
REVIEW
originalor model 7 of 15
removing
BiLSTM TCNwith
either thethe LSTMfrom
BiLSTM
component module
or TCNtheremoved.
component
model, The
as middle
from the
indicated part of theexperimental
bymodel,
the figure shows the
as indicated byoriginal
the led
results, ex-
model. The right
perimental part ofled
results, theto
figure shows the
decreased originalfor
accuracy model
thesewith the TCN
datasets module
(Figure 5).removed.
to decreased accuracy for these datasets (Figure 5).
Impact on Matthews’ correlation coefficient (MCC): Similarly, the experimental re-
Impact
sults on accuracy
demonstrated (ACC): The
that DeepPGD ablation
achieved experiment
the highest MCCresults
values forrevealed that
all three the
biologi-
DeepPGD model consistently achieved higher accuracy for all three
cal species datasets. The ablation experiments highlighted that the removal of BiLSTM or biological species da-
tasets
the TCNwhenled both BiLSTMMCC
to reduced and avalues.
TCN were used. Thisthe
This indicated underscored
pivotal rolethe crucial role
of BiLSTM and ofthe
BiLSTM and the TCN in enhancing accuracy within the DeepPGD
TCN in enhancing the robustness and predictive performance of the classification model model. Conversely,
removing
(Figure 6). either the BiLSTM or TCN component from the model, as indicated by the ex-
perimental
Impact results,
on arealedunder
to decreased
the curve accuracy
(AUC):for Thethese
AUCdatasets (Figure
is a crucial 5). for assessing the
metric
Impact on Matthews’ correlation coefficient (MCC):
predictive performance of classification models. The experimental results Similarly, the experimental
indicated re-that
sults
DeepPGD attained the highest AUC values for two of the three biological speciesbiologi-
demonstrated that DeepPGD achieved the highest MCC values for all three datasets
cal species
when both datasets.
BiLSTMThe andablation
TCN were experiments highlighted
used. However, that
for one the removal
dataset, omittingof BiLSTM
the TCNorre-
the TCNinled
sulted to reduced
higher MCC values.
AUC values, This indicatedimproved
which consequently the pivotal therole of BiLSTMofand
performance the the
clas-
TCN in enhancing the robustness and predictive
sification model for that specific dataset (Figure 7). performance of the classification model
(Figure 6).
Impact on area under the curve (AUC): The AUC is a crucial metric for assessing the
predictive performance of classification models. The experimental results indicated that
DeepPGD attained the highest AUC values for two of the three biological species datasets
when both BiLSTM and TCN were used. However, for one dataset, omitting the TCN re-
sulted
Figurein
Figure higher
[Link],
Comparison accuracy
accuracy which consequently
performance
performance improved
ininablation
ablation the performance of the clas-
experiments.
experiments.
sification model for that specific dataset (Figure 7).
Int. J. Mol. Sci. 2024, 25, 8146 7 of 15

Impact on Matthews’ correlation coefficient (MCC): Similarly, the experimental results

demonstrated that DeepPGD achieved the highest MCC values for all three biological
species datasets. The ablation experiments highlighted that the removal of BiLSTM or the
TCN led to reduced MCC values. This indicated the pivotal role of BiLSTM and the TCN in
Figure 5. Comparison
enhancing of accuracy
the robustness performance
and predictive in ablation experiments.
performance of the classification model (Figure 6).

Figure 5. Comparison of accuracy performance in ablation experiments.

ComparisonofofMCC
[Link]
Figure MCCperformance
performancein
inablation
ablationexperiments.
experiments.

Impact on area under the curve (AUC): The AUC is a crucial metric for assessing
the predictive performance of classification models. The experimental results indicated
that DeepPGD attained the highest AUC values for two of the three biological species
datasets when both BiLSTM and TCN were used. However, for one dataset, omitting the
TCN resulted in higher AUC values, which consequently improved the performance of the
Figure 6. Comparison of MCC performance in ablation experiments.
classification model for that specific dataset (Figure 7).

Figure 7. Comparison of AUC performance in ablation experiments.

Figure7.
Figure Comparisonof
[Link] ofAUC
AUCperformance
performancein
inablation
ablationexperiments.
experiments.

Overall, the ablation experiment results underscored the significant roles of BiLSTM
and a TCN within the DeepPGD model in improving key metrics such as accuracy, MCC,
and the AUC. This further validated the superior performance of the DeepPGD algorithm in
DNA methylation recognition tasks and highlighted the interdependence and importance
of different components within deep learning models. Consequently, the integration of
BiLSTM and a TCN into the deep learning framework enhanced the accurate identification
capability for DNA methylation.
 in DNA methylation recognition tasks and highlighted the interdependence and im-
portance of different components within deep learning models. Consequently, the inte-
gration of BiLSTM and a TCN into the deep learning framework enhanced the accurate
identification capability for DNA methylation.
Int. J. Mol. Sci. 2024, 25, 8146 8 of 15

2.4. Cross-Species Prediction of DNA Methylation Correlation

In order to Prediction
2.4. Cross-Species understand the correlation
of DNA Methylation between
CorrelationDNA methylation among different
species, this study conducted cross-testing using DNA methylation datasets from various
In order to understand the correlation between DNA methylation among different
species. Thestudy
species, this experiments
conductedrevealed that DNA
cross-testing usingmethylation amongdatasets
DNA methylation differentfromspecies may be
various
correlated
species. The experiments revealed that DNA methylation among different species may alt-
and could be predicted using DNA methylation data from other species,
hough the effectiveness
be correlated and could was limited. Among
be predicted all test
using DNA results, training
methylation data fromand testing using data
other species,
from the same
although species yielded
the effectiveness the bestAmong
was limited. predictive performance.
all test However,
results, training in cross-predic-
and testing using
tion
data scenarios,
from the same the results
species were
yielded notthealways positive. performance.
best predictive For instance, However,
the 6 mA_A. thaliana
in cross-
training
prediction setscenarios,
could predict the 6 mA_D.
the results were notmelanogaster test set relatively
always positive. For instance,well,the
but6the 6 mA_T.
mA_A.
thaliana training
thermophile set could
training predict the
set performed poorly whenmelanogaster
6 mA_D. predicting thetest 6set relatively
mA_Xoc well, test
BLS256 but set.
the 6 mA_T. thermophile
Considering training set
these experimental performed
results, poorlythe
predicting when
DNA predicting
methylationthe 6 of
mA_Xoc
unknown
BLS256 using
species test set. Considering
existing species’these
DNA experimental
methylation results, predicting
data may require the DNA methylation
complex decision-mak-
of unknown
ing. (Figure 8) species using existing species’ DNA methylation data may require complex
decision-making (Figure 8).

Theresults
Figure 8. The resultsof
ofcross-prediction
cross-prediction of
of DNA
DNA methylation
methylation among
among different
different species, with the ver-
tical axisaxis
vertical representing
representingthe training
the trainingset
setand
andthe
thehorizontal
horizontal axis representing
representingthe
thetest
testset
set(ACC).
(ACC).

3. Discussion
3. Discussion
Deep learning has rapidly advanced and made significant strides across various fields,
Deep learning has rapidly advanced and made significant strides across various
such as natural language processing and computer vision [25–27]. The exploration of
fields, such as natural language processing and computer vision [25–27]. The exploration
machine learning and deep learning methods for DNA methylation classification has
of machine
been learning
a prominent and deep
avenue learning
of research, methods for their
demonstrating DNAeffectiveness
methylationover
classification
standard has
been
biological and statistical approaches. A predominant focus of these studies has standard
a prominent avenue of research, demonstrating their effectiveness over been on bi-
ological and statistical
binary classification, approaches.
determining A predominant
whether focus ofbelong
specific sequences theseto
studies has been on bi-
DNA methylation
nary classification, determining whether specific sequences belong to DNA methylation
sites [23,24,28,29].
sites Deep-4mCGP
[23,24,28,29]. [30], a model approached from a 4 mC classification perspective, em-
ploys gradient-boosting decision tree for the feature selection. Subsequently, it feeds the
combination of sequences and features into a convolutional neural network (CNN) for 4 mC
classification. Mouse4mC-BGRU [31] employs adaptive embedding to extract features from
sequences and utilizes bidirectional gated recurrent units (BiLSTMs) for the encoding, ad-
hering to a conventional neural 4 mC classification network. DNC4mC-Deep [32] attempts
embedding using a cross-species dataset and applies various encoding methods, including
the dinucleotide composition (DNC) and the trinucleotide combination (TNC) position.
Finally, it uses a modified neural network for 4 mC classification. Prior to applying a CNN
Int. J. Mol. Sci. 2024, 25, 8146 9 of 15

model, Deep-4 mCW2V encodes DNA sequences using word embeddings to learn and
accurately identify 4 mC sites. Concerning 5 hmC, iRNA5hmC [33] employs position nu-
cleotide binary carriers as features, utilizes a two-stage optimization method for the feature
selection, and then employs support vector machines (SVMs) for the classification. Regard-
ing typical RNA methylation, a well-described study [34] summarizes several classification
methods, including binary encoding (e.g., specific-position k-mers) and combination encod-
ing (e.g., k-spacing nucleotide pair frequency (KSNPF)). Researchers have employed SVMs,
random forests (RFs), CNNs, and other machine learning models for classification. In 6 mA
prediction research, Le and Ho [28] designed a complex predictor structure comprising
pretrained transformers and CNNs. i6mA-Vote [35] employs one-hot encoding and five
selected basic machine learning models for voting. MGF6mARice [36] transforms original
sequences into a simplified SMILES format and employs graph convolutional networks
(GCNs) for the encoding and classification.
However, whether traditional machine learning methods or deep learning methods,
most still heavily rely on manual approaches to train classifiers for model inputs, neces-
sitating researchers to possess prior knowledge. Furthermore, these methods struggle to
be universally applicable across all species. Additionally, the aforementioned methods
are only designed for a specific methylation type, or some methods are only tailored to a
particular species. Therefore, there is an urgent need for a universal method to identify
cross-species DNA methylation sites. iDNA-MS [24] initially configured samples through
three sequences to encode features and then employed a RF to identify DNA methyla-
tion sites for 5 hmC, 6 mA, and 4 mC. However, the experimental results showed that its
algorithm performance was suboptimal, leaving room for improvement.
Given the aforementioned backdrop, this paper introduces an innovative deep learning
framework named DeepPGD, aiming to address the issue of DNA methylation identifi-
cation. To overcome the complexity of DNA sequences and the importance of sequence
features, this framework adopted a dual residual structure that combined convolutional
networks (CNNs) and bidirectional long short-term memory (BiLSTM) networks, thus
wielding formidable feature extraction capabilities. The introduction of a dual residual
structure further enhanced the model’s depth, aiding the model to learn more abstract
and higher-level features from raw DNA sequences. This structure gradually constructed
multilayered feature representations, allowing the model to better comprehend the hierar-
chical structure of the data. A CNN itself excels at capturing features of different scales,
while a BiLSTM can capture long-term dependencies within sequences. By combining
them, DeepPGD proved capable of multiscale feature learning, thereby providing a more
comprehensive understanding of DNA sequence characteristics.

4. Materials and Methods

4.1. Dataset
The datasets utilized in this study were the same as those in iDNA-MS [24]. This study
employed a total of 10 datasets, representing the following three types of DNA methylation
site: 4 mC, 5 hmC, and 6 mA. The 4 mC category consisted of the following three datasets:
4 mC_C. equisetifolia, 4 mC_F. vesca, and S. cerevisiae. The 5 hmC category included the
dataset 5 hmC_H. sapiens. The 6 mA category comprised the following six datasets: 6 mA_A.
thaliana, 6 mA_C. elegans, 6 mA_D. melanogaster, 6 mA_H. sapiens, 6 mA_Tolypocladium,
6 mA_T. thermophile, and 6 mA_Xoc, BLS256. All datasets shared a common DNA sequence
length of 41 nt.

4.2. Evaluation Criteria

In this research, the investigators utilized the AUC (area under the curve), ACC (accu-
racy), and MCC (Matthews’ correlation coefficient) as the primary evaluation criteria to
assess the performance of predictive models based on DNA methylation data. DNA methy-
lation prediction is a significant binary classification problem that involves predicting the
Int. J. Mol. Sci. 2024, 25, 8146 10 of 15

presence or absence of methylation sites within DNA molecules. The following elaboration
and explanation provide further insights into these evaluation criteria.
AUC (area under the curve): In DNA methylation prediction, the AUC is used to
measure the balance between the true-positive rate and false-positive rate of a model
at various classification thresholds. A high AUC value indicates that the model can
accurately distinguish between positive and negative samples, thereby demonstrating a
good predictive performance.
ACC (accuracy): ACC in DNA methylation prediction measures the overall classifica-
tion accuracy of a model. The accurate classification of positive and negative samples is
crucial for the task of DNA methylation prediction as the predictive outcomes can have
significant implications in biological research and medical diagnosis.

ACC = ( TP + TN )/( TP + TN + FP + FN ) (1)

MCC (Matthews’ correlation coefficient): MCC is a comprehensive metric that takes

into account true positives, true negatives, false positives, and false negatives. In DNA
methylation prediction, MCC offers a more holistic performance evaluation. In particular,
when dealing with imbalanced datasets, MCC better reflects the model’s performance,
aiding researchers to accurately assess the model’s performance across different classifica-
tion scenarios.
TP × TN − FP × FN
MCC = p (2)
( TP + FP)( TP + FN )( TN + FP)( TN + FN )

The combined application of these evaluation criteria helps researchers to gain a

comprehensive understanding of the performance of DNA methylation prediction models,
enabling more accurate model selections and optimization decisions. In the field of bioinfor-
matics, these evaluation metrics hold significant significance in verifying model reliability,
applicability, and their role in uncovering gene regulation and disease mechanisms.

4.3. RNA Representation Method

This study encompassed a total of ten datasets, each containing DNA sequences of
a fixed length. During the data preprocessing phase, the 3-mer technique was employed
to process the original DNA sequences, resulting in a reduction in the sequence length.
Subsequently, to achieve a uniform sequence length, a zero-padding strategy was applied
at the end of the DNA sequences, extending the length to 48 base pairs.
In the subsequent stages of processing, the word embedding technique was employed
to convert DNA sequences into corresponding matrix representations. In this process,
each DNA motif was mapped to a sixteen-dimensional embedding space, facilitating the
conversion from sequence data to continuous vector representations. The resultant matrix
possessed dimensions of 1 × 48 × 16, providing a rich informational foundation for the
subsequent in-depth analysis and model construction (Figure 9).

4.4. Residual MLP Block in DeepPGD

In the DeepPGD model, the multilayer perceptron (MLP) layer could be viewed as an
operation for feature compression and mapping. The role of the MLP layer lay in abstraction
and encoding. Through the introduction of nonlinear activation functions, the MLP could
map original features to a higher-level abstract representation space, aiding the model to
comprehend and distinguish patterns and relationships within the data. Furthermore, in
this context, an attention mechanism was introduced, enabling the model to learn feature
disparities between the MLP layer and the original DNA sequences (Figure 10).
 Int. J.J. Mol.
Int. Mol. Sci.
Sci. 2024,
2024, 25,
25, 8146
x FOR PEER REVIEW 11
11 of 15
of 15

Figure 9. The structure of DeepPGD.

4.4. Residual MLP Block in DeepPGD

In the DeepPGD model, the multilayer perceptron (MLP) layer could be viewed as
an operation for feature compression and mapping. The role of the MLP layer lay in ab-
straction and encoding. Through the introduction of nonlinear activation functions, the
MLP could map original features to a higher-level abstract representation space, aiding
the model to comprehend and distinguish patterns and relationships within the data. Fur-
thermore, in this context, an attention mechanism was introduced, enabling the model to
learn feature disparities between the MLP layer and the original DNA sequences. (Figure
10) 9.9. The
Figure
Figure The structure
structure of
ofDeepPGD.
DeepPGD.

4.4. Residual MLP Block in DeepPGD

In the DeepPGD model, the multilayer perceptron (MLP) layer could be viewed as
an operation for feature compression and mapping. The role of the MLP layer lay in ab-
straction and encoding. Through the introduction of nonlinear activation functions, the
MLP could map original features to a higher-level abstract representation space, aiding
the model to comprehend and distinguish patterns and relationships within the data. Fur-
thermore, in this context, an attention mechanism was introduced, enabling the model to
learn feature disparities between the MLP layer and the original DNA sequences. (Figure
Figure
10) 10.
Figure 10. The
The structure
structure of
ofthe
theMLP
MLPblock.
block.

4.5.
4.5. TCN
TCN Block
Block inin DeepPGD
DeepPGD
AA temporal convolutional network
temporal convolutional network (TCN)
(TCN) is is aa neural
neural network
network architecture
architecture designed
designed
specifically
specifically for sequence data, adept at capturing relationships and patterns within
for sequence data, adept at capturing relationships and patterns within DNA
DNA
sequences.
sequences. ThisThis is is crucial
crucial for
for methylation
methylation detection
detection tasks,
tasks, as
as the
the methylation
methylation state
state in
in DNA
DNA
sequences
sequences is is often
often influenced
influenced bybyadjacent
adjacent base
basepairs.
pairs. AA TCN
TCN utilizes
utilizes causal
causal convolutions
convolutions
to
to ensure
ensure that
that predictions
predictionsat atany
anygiven
given point
point in
in the
the sequence
sequence depend
depend only
only on
on current
current andand
past information, preserving the sequence order and avoiding information
past information, preserving the sequence order and avoiding information leakage from leakage from
future
futurepoints.
points.
FigureIn this
In10.
this
Thestudy,
study, aaTCN
structure TCN model
model
of the employing
employing causal
MLP block. causal convolutions
convolutions was was used
used to
to extract
extract se-
se-
quence
quence features from DNA sequences. Note that while TCN typically involves various
features from DNA sequences. Note that while TCN typically involves various
components
4.5. TCN Block such as dilated convolutions, this study specifically utilizes causal convolu-
in DeepPGD
tions. Causal convolutions ensure that each convolution operation only depends on current
A temporal convolutional network (TCN) is a neural network architecture designed
and past information, which is crucial for maintaining the sequence order of the data. The
specifically for sequence data, adept at capturing relationships and patterns within DNA
model incorporated multiple causal convolutional kernels of varying sizes within each
sequences. This is crucial for methylation detection tasks, as the methylation state in DNA
block to capture structural information across different spans of the sequence. This design
sequences is often influenced by adjacent base pairs. A TCN utilizes causal convolutions
enables the TCN to effectively handle long-range dependencies by preserving the sequence
to ensure that predictions at any given point in the sequence depend only on current and
while considering distant base pairs (Figures 11 and 12).
past information, preserving the sequence order and avoiding information leakage from
future points.
In this study, a TCN model employing causal convolutions was used to extract se-
quence features from DNA sequences. Note that while TCN typically involves various
 components such as dilated convolutions, this study specifically utilizes causal convolu-
tions. Causal convolutions ensure that each convolution operation only depends on cur-
tions. Causal convolutions ensure that each convolution operation only depends on cur-
rent and past information, which is crucial for maintaining the sequence order of the data.
rent and past information, which is crucial for maintaining the sequence order of the data.
The model incorporated multiple causal convolutional kernels of varying sizes within
The model incorporated multiple causal convolutional kernels of varying sizes within
each block to capture structural information across different spans of the sequence. This
Int. J. Mol. Sci. 2024, 25, 8146 each block to capture structural information across different spans of the sequence.
12 ofThis
15
design enables the TCN to effectively handle long-range dependencies by preserving the
design enables the TCN to effectively handle long-range dependencies by preserving the
sequence while considering distant base pairs (Figures 11 and 12).
sequence while considering distant base pairs (Figures 11 and 12).

Figure 11.
11. The basic
basic structure of
of the CNN
CNN block.
Figure [Link]
Figure The basicstructure
structure ofthe
the CNNblock.
block.

Figure 12. The structure of the CNN block.

Figure12.
Figure [Link]
Thestructure
structureofofthe
theCNN
CNNblock.
block.
MHA (multihead attention) is an important mechanism used to calculate the
MHA(multihead
MHA (multihead attention)
attention) anisimportant
an important mechanism used to the
calculate the
weighted sum of the key vector is
according to the mechanism
query vector used where to calculate
the weightweighted
is deter-
weighted
sum sumvector
of the key of theaccording
key vectortoaccording
the query to the query
vector wherevector
the weight where is the weight isbydeter-
determined the
mined by the similarity between the query vector and the key vector. The formulas are as
mined bybetween
similarity the similarity between
the query vectorthe query
and vector
the key and the
vector. Thekey [Link]
formulas The
asformulas
follows: are as
follows:
follows:
Q Q K K V
Qi =QiQW= QW , KQ = KW
i, K = KWi , V, iV==VW VWi V, ,ii== 1,1,.. .. . , h (3)
Qi = iQWi i , Ki i = KWi iK , Vi i = VWi iV , i = 1, . . . , h (3)
headi = Attention(Qi , Ki , Vi ), i = 1, . . . , h (4)
headi = Attention(Qi , K i , Vi ), i = 1, . . . , h (4)
head = Attention(Q , K i , Vi ), i = 1, . . . , h O (4)
MultiHead(Q, iK, V) = Concati(head 1 , . . . , headh )W (5)
O
where Q, K, V represent MultiHead(Q,
the queryK,K,V) = Concat(head
matrix, key matrix, 1 , .and
. . , head h )W
value matrix,
O (5)
respectively;
MultiHead(Q, V) = Concat(head 1 , . . . , headh )W (5)
Q K V
W i , Wi Q,
where , and
K, VWrepresent
i represent thethe weight
query matrices
matrix, of the query
key matrix, and value matrix, key respectively;
matrix, matrix, and
where
value K Q, K, V V represent the WOquery matrix,the
key matrix, and value matrix, respectively;
Wi Q, Wmatrix,
, and Wrespectively; represents output weight matrix; h represents the
Q
i K i V represent the weight matrices of the query matrix, key matrix, and value
W
number
i , W , and W
i of heads; i represent
head i the weight
represents
O the matrices
output ofof the
the query
i-th head; matrix,
and key matrix,
Concat and value
represents the
matrix, respectively; W O represents the output weight matrix; h represents the number
matrix, respectively;
concatenation operation.W represents the output weight matrix; h represents the number
of heads; headi represents the output of the i-th head; and Concat represents the concat-
of heads; headi represents the output of the i-th head; and Concat represents the concat-
enation operation.
4.6. BiLSTM
enation Block in DeepPGD
operation.
Bidirectional long short-term memory (BiLSTM) represents a variant of a recurrent
neural network (RNN) that exhibits bidirectional properties, enabling the simultaneous
consideration of forward and backward contextual information. In the context of methyla-
tion detection tasks, each DNA base pair is regarded as a point in the sequence, allowing
BiLSTM to effectively capture sequence dependencies. The BiLSTM model performs com-
putations in both the forward and backward directions. In the forward computation, the
model starts with the first base pair in the sequence and sequentially calculates the hidden
state for each subsequent base pair. In the backward computation, the model starts from
the last base pair and calculates the hidden states in reverse order. The hidden states from
 consideration of forward and backward contextual information. In the context of methyl-
ation detection tasks, each DNA base pair is regarded as a point in the sequence, allowing
BiLSTM to effectively capture sequence dependencies. The BiLSTM model performs com-
putations in both the forward and backward directions. In the forward computation, the
Int. J. Mol. Sci. 2024, 25, 8146 model starts with the first base pair in the sequence and sequentially calculates the hidden
13 of 15
state for each subsequent base pair. In the backward computation, the model starts from
the last base pair and calculates the hidden states in reverse order. The hidden states from
both directions
both directions are
arecombined
combined through
throughconcatenation
concatenationor or addition,
addition, yielding
yielding comprehensive
comprehensive
contextual information. These hidden states can be interpreted as feature representations
contextual information. These hidden states can be interpreted as feature representations
of each
of each base
base pair
pairininthe
thesequence,
sequence,capturing
capturing patterns and
patterns relationships.
and These
relationships. feature
These rep-
feature
resentations serve as the foundation for the methylation state prediction in subsequent
representations serve as the foundation for the methylation state prediction in subsequent
layers.(Figure
layers (Figure13).
13)

Figure13.
Figure 13. The
The structure
structure of
ofthe
theBiLSTM
BiLSTMblock.
block.

In
In this
this study,
study,aaBiLSTM
BiLSTMresidual
residualmodel
modelaugmented
augmentedwithwithaamultihead
multiheadcross-attention
cross-attention
mechanism
mechanism was was employed
employed to to extract
extract sequence
sequence features.
features. Specifically,
Specifically, three
three BiLSTM
BiLSTM units
units
were
wereemployed
employedto toextract
extract sequence
sequence features
featuresfrom
fromDNA
DNA sequences,
sequences,enabling
enabling thethe extraction
extraction
of
of complex
complexhierarchical
hierarchicalfeatures.
[Link]
Theformulas
formulasfor
forthe
theLSTM
LSTMcalculation
calculationareareasasfollows:
follows:

ft f= σ(W
t = f ⋅ f[h
σ(W · t−1
[ht−, x1t,]x+
t] b
+f )bf ) (6)

it 𝑖= = Wi · [h⋅ [ℎ
σ (𝜎(𝑊 , xt ] + b ) (7)
𝑡 𝑖 t −1 𝑡−1 , 𝑥𝑡 ] +i 𝑏𝑖 ) (7)
∼
C t = tanh(WC · [ht−1 , xt ] + bC ) (8)
𝐶̃𝑡 = 𝑡𝑎𝑛ℎ (𝑊𝐶 ⋅ [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝐶 ) (8)
∼
Ct = f t ∗ Ct−1 + it ∗ C t (9)
̃
o = 𝐶σ𝑡(=
t W𝑓𝑡· ∗[h𝐶𝑡−1, x+ ]𝑖𝑡+∗b𝐶𝑡)
o t −1 t o
(9)
(10)

𝑜𝑡 h=t = ot ∗⋅ tanh
𝜎(𝑊 (Ct ) (11)
𝑜 [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝑜 ) (10)
5. Conclusions
ℎ𝑡 = 𝑜𝑡 ∗ 𝑡𝑎𝑛ℎ (𝐶𝑡 ) (11)
In this paper, we introduced DeepPGD, an innovative deep learning model designed
to address the challenge of DNA methylation recognition. By integrating the attention
mechanism of a transformer with MLP layers and by incorporating the feature extraction of
5. Conclusions
a TCN and BiLSTM, DeepPGD demonstrated remarkable performance using DNA methy-
In this paper, we introduced DeepPGD, an innovative deep learning model designed
lation datasets from multiple biological species. Through its adept ability to efficiently
to address the challenge of DNA methylation recognition. By integrating the attention
capture both structural and sequence features of DNA sequences, DeepPGD surpassed con-
mechanism of a transformer with MLP layers and by incorporating the feature extraction
ventional methods in metrics such as accuracy, MCC, and AUC. Based on the experimental
of a TCN and BiLSTM, DeepPGD demonstrated remarkable performance using DNA
findings of this study, diversity in feature extraction using models may contribute to an
methylation datasets
improvement in modelfrom multiple biological
performance. Therefore,species. Through
we believe its adept
that further ability to into
exploration effi-
ciently capture both structural and sequence features of DNA sequences, DeepPGD
the diversity of effective feature extraction using models could be a future direction in this sur-
passed
field conventional
of research. This methods in metrics
accomplishment such as
provides accuracy,
a robust toolMCC, and AUC.
to advance DNABased on the
methylation
experimental
research. findings
Despite of this
certain study, diversity
challenges in feature
that remain, extraction
this study providesusing models
a new may con-
direction for
tribute to an improvement in model performance. Therefore,
the application of deep learning within the field of bioinformatics. we believe that further

Author Contributions: Conceptualization, L.W.; Methodology, S.T. Formal analysis, Y.L. All authors
have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data availability statements are available at [Link]
FROZEN160/DeepPGD (accessed on 24 June 2024).
Int. J. Mol. Sci. 2024, 25, 8146 14 of 15

Conflicts of Interest: The authors declare no conflict of interest.

References
1. Waddington, C.H. The epigenotype. Int. J. Epidemiol. 2012, 41, 10–13. [CrossRef] [PubMed]
2. Robertson, K.D. Epigenetic Mechanisms of Gene Regulation. Epigenetics 1996, 3, 21.
3. Feng, Z.; Xu, W.; Ying, L.Q.; Rong, L.Q.; Yu, H.D.; Li, L.Z.; Dai, M.H.; Yuan, Z.H. Epigenetics and its Research Methods. Adv. Mod.
Biomed. 2017, 17, 2371–2376.
4. Goldberg, A.D.; Allis, C.D.; Bernstein, E. Epigenetics: A Landscape Takes Shape. Cell 2007, 128, 635–638. [CrossRef]
5. Shakya, K.O.; O’Connell, M.; Ruskin, H.J. The Landscape for Epigenetic/Epigenomic Biomedical Resources. Epigenetics Off. J.
DNA Methylation Soc. 2012, 7, 982–986. [CrossRef]
6. Lee, C.Z. Design and Implementation of an Online Prediction Platform for DNA Methylation Status. Master’s Thesis, University
of Electronic Science and Technology of China, Chengdu, China, 2016.
7. Holliday, R.; Pugh, J. DNA Modification Mechanisms and Gene Activity during Development. Science 1975, 187, 226–232.
[CrossRef] [PubMed]
8. Hendrich, B.; Bird, A. Identification and Characterization of a Family of Mammalian Methyl-CpG Binding Proteins. Mol. Cell.
Biol. 1999, 18, 6538–6547. [CrossRef]
9. Smith, Z.D.; Meissner, A. DNA Methylation: Roles in Mammalian Development. Nat. Rev. Genet. 2013, 14, 204–220. [CrossRef]
10. Liu, Y.X. Prediction of Whole-Genome DNA Methylation Using Abel Complexity Method. Master’s Thesis, Huazhong Agricul-
tural University, Wuhan, China, 2018.
11. Bird, A. DNA Methylation Patterns and Epigenetic Memory. Genes Dev. 2002, 16, 6–21. [CrossRef]
12. Xie, W.; Kagiampakis, I.; Pan, L.; Zhang, Y.W.; Murphy, L.; Tao, Y.; Kong, X.; Kang, B.; Xia, L.; Carvalho, F.L.; et al. DNA
Methylation Patterns Separate Senescence from Transformation Potential and Indicate Cancer Risk. Cancer Cell 2018, 33, 309–321.
[CrossRef]
13. Wang, B.P. Study on Somatic Cell Cloning of FGF5 Gene Targeted Goats. Ph.D. Dissertation, Inner Mongolia Agricultural
University, Hohhot, China, 2014.
14. Kato, M.; Miura, A.; Bender, J.; Jacobsen, S.E.; Kakutani, T. Role of CG and Non-CG Methylation in Immobilization of Transposons
in Arabidopsis. Curr. Biol. 2003, 13, 421–426. [CrossRef] [PubMed]
15. He, W.; Jia, C.; Zou, Q. 4mCPred: Machine learning methods for DNA N4-methylcytosine sites prediction. Bioinformatics 2019, 35,
593–601. [CrossRef] [PubMed]
16. Thomson, J.P.; Meehan, R.R. The application of genome-wide 5-hydroxymethylcytosine studies in cancer research. Epigenomics
2017, 9, 77–91. [CrossRef] [PubMed]
17. Pian, C.; Zhang, G.; Li, F.; Fan, X. MM-6mAPred: Identifying DNA N6-methyladenine sites based on Markov model. Bioinformatics
2020, 36, 388–392. [CrossRef] [PubMed]
18. Li, Y.T.; Tollefsbol, T.O. DNA Methylation Detection: Bisulfite Genomic Sequencing Analysis. Methods Mol. Biol. 2011, 791, 11–21.
[PubMed]
19. Liu, Z.; Xiao, X.; Qiu, W.R.; Chou, K.C. iDNA-Methyl: Identifying DNA Methylation Sites via Pseudo Trinucleotide Composition.
Anal. Biochem. 2015, 474, 69–77. [CrossRef] [PubMed]
20. Liu, G.H. Research on Whole-Genome DNA Methylation Prediction Based on Neural Network. Master’s Thesis, Dalian Maritime
University, Dalian, China, 2017.
21. Fan, S.; Zhang, M.Q.; Zhang, X. Histone methylation marks play important roles in predicting the methylation status of CpG
islands. Biochem. Biophys. Res. Commun. 2008, 374, 559–564. [CrossRef] [PubMed]
22. Zhou, X.; Li, Z.; Dai, Z.; Zou, X. Prediction of methylation CpGs and their methylation degrees in human DNA sequences.
Comput. Biol. Med. 2012, 42, 408–413. [CrossRef] [PubMed]
23. Yu, Y.; He, W.; Jin, J.; Xiao, G.; Cui, L.; Zeng, R.; Wei, L. iDNA-ABT: Advanced deep learning model for detecting DNA methylation
with adaptive features and transductive information maximization. Bioinformatics 2021, 37, 4603–4610. [CrossRef]
24. Lv, H.; Dao, F.-Y.; Zhang, D.; Guan, Z.-X.; Yang, H.; Su, W.; Liu, M.-L.; Ding, H.; Chen, W.; Lin, H. iDNA-MS: An integrated
computational tool for detecting DNA modification sites in multiple genomes. IScience 2020, 23, 100991. [CrossRef]
25. Yu, F.; Koltun, V. Multi-Scale Context Aggregation by Dilated Convolutions. In Proceedings of the International Conference on
Learning Representations (ICLR), San Juan, Puerto Rico, 2–4 May 2016.
26. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. In
Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 4–9
December 2017.
27. Bai, S.; Kolter, J.Z.; Koltun, V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling.
arXiv 2018, arXiv:1803.01271.
28. Le, N.Q.K.; Ho, Q.-T. Deep transformers and convolutional neural network in identifying DNA N6-methyladenine sites in
cross-species genomes. Methods 2022, 204, 199–206. [CrossRef] [PubMed]
29. Zheng, Z.; Le, N.Q.K.; Chua, M.C.H. MaskDNA-PGD: An innovative deep learning model for detecting DNA methylation by
integrating mask sequences and adversarial PGD training as a data augmentation method. Chemom. Intell. Lab. Syst. 2023,
232, 104715. [CrossRef]
Int. J. Mol. Sci. 2024, 25, 8146 15 of 15

30. Zulfiqar, H.; Huang, Q.-L.; Lv, H.; Sun, Z.-J.; Dao, F.-Y.; Lin, H. Deep-4mCGP: A deep learning approach to predict 4mC sites in
geobacter pickeringii by using correlation-based feature selection technique. Int. J. Mol. Sci. 2022, 23, 1251. [CrossRef] [PubMed]
31. Jin, J.; Yu, Y.; Wei, L. Mouse4mc-BGRU: Deep learning for predicting DNA N4-methylcytosine sites in mouse genome. Methods
2022, 204, 258–262. [CrossRef] [PubMed]
32. Wahab, A.; Mahmoudi, O.; Kim, J.; Chong, K.T. DNC4mC-deep: Identification and analysis of DNA N4-methylcytosine sites
based on different encoding schemes by using deep learning. Cells 2020, 9, 1756. [CrossRef] [PubMed]
33. Liu, Y.; Chen, D.; Su, R.; Chen, W.; Wei, L. iRNA5hmC: The first predictor to identify RNA 5-hydroxymethylcytosine modifications
using machine learning. Front. Bioeng. Biotechnol. 2020, 8, 227. [CrossRef] [PubMed]
34. Wang, H.; Wang, S.; Zhang, Y.; Bi, S.; Zhu, X. A brief review of machine learning methods for RNA methylation sites prediction.
Methods 2022, 203, 399–421. [CrossRef]
35. Teng, Z.; Zhao, Z.; Li, Y.; Tian, Z.; Guo, M.; Lu, Q.; Wang, G. i6mA-Vote: Cross- species identification of DNA N6-methyladenine
sites in plant genomes based on ensemble learning with voting. Front. Plant Sci. 2022, 13, 845835. [CrossRef]
36. Liu, M.; Sun, Z.-L.; Zeng, Z.; Lam, K.-M. MGF6mARice: Prediction of DNA N6- methyladenine sites in rice by exploiting
molecular graph feature and residual block. Brief. Bioinform. 2022, 23, bbac082. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.