CHAPTER TWO

LITERATURE REVIEW

2.1 Introduction

This chapter dives into what’s already been done in the world of using articial intelligence,
especially deep learning, to figure out what proteins do based on their sequences. It looks at
how these tools are changing things in bioinformatics and biology eductaion, with a special
eye on making them easier for students and labs that don’t have fancy equipment. The review
pulls together research on protein function prediction, deep learning models like ProtT5 and
ESM, and the challenges of making these tools user-friendly. By checking out these areas,
this chapter sets the stage for seeing the upsides and hurdles of using AI for protein analysis,
mixing in global trends with the needs of places like smaller research setups or classrooms.

I went through 20 recent papers o spot the patterns, strengths, and weak spots in current
work. These studies cover everything from surveys on transformers in proteins to specific
models for function prediction, and they help show where my project fits in. My tool is about
more than just sorting proteins into functional groups-it’s a lightweight app that classifies
sequences using pre-trained embeddings, with a simple interface for inputting data, clear
explanations of results, options to download predictions, and ways to handle tricky cases like
novel or multi-functional proteins. Since the tool relies on pre-trained model for all this, I’ll
akso touch on how it ties ino supervised learning, where models get trained on labeled data to
make smart guesses about new sequences.

2.2 Protein Function Prediction Using AI

Proteins do all sort of jobs in cells (workhouses for cells), handling tasks like speeding up
reaction or sending signals, and figuring out their roles is crucial (big deal) in biology.
Traditional methods often need a protein’s 3D shape, costly and needs specialized
equipment-tough for those without big budgets. Deep learning (DL) has changed the game by
letting us predict functions just from sequences-the strings of amino acids that make up
proteins (Boadu et al., 2024)

Protein language models (PLMs) like ProtT5 and ESM are hot topic. These models work like
language translators for proteins, turning sequences into numerical codes (called embeddings)
that capture their properties. For example, one study reviewed deep learning methods for
protein function prediction, highlighting how DL improves annotation accuracy but noting
the need for unified benchmark datasets (Boadu et al., 2024). Another paper introduced
PersGNN, combining topology and geometry in graph neural networks for high accuracy in
structure-based prediction, though limited in testing on diverse proteins (Swenson et al,.
2020). A survey on transformers in protein studies summarized over 100 applications,
showing promising results but a lack of standardized evaluation metrics (Ling et., 2025).

Studies show these models are great at sorting proteins into groups like Transferases or
Hydrolases, with accuracy often above 80% on standard datasets. But they’re not perfect. For
example, a review on DL-based advances in protein structure prediction found DL improves
accuracy over traditional models, but less progress on disordered regions (Pakhrin et al.,
2021). An integrated model like SDN2GO combines sequences and structure data for higher
precision, though it needs validation on real world datasets (Cai et al., 2020)

DL uses clever algorithm to spot patterns in sequences that hint at what a protein does. Key
methods include:

 Convolutional Neural Networks (CNNs): These pick out local patterns, like specific
amino acid combos tied to functions (Liu et al., 2020).
 Recuurent Neural Networks (RNNs): These tracks sequence order, helping predict
how parts connect for folding or function (Pakhrin et al., 2021).
 Transformers: These use attention to focus on key sequence parts, creating
embeddings-numerical codes that capyure a protein’s essence (Ling et al., 2025)
 Graph Neural Networks (GNNs): These models proteins as graph, with amino acid
as nodes and bond as edges, great for structure-based predictions but needing more
data (Swenson et al., 2020; Zhang et al., 2021).

This shift to sequence-based prediction is huge for making protein science accessible,
as sequence are easier and cheaper to get than 3D structures.
2.2.1 Deep Learning and Its Application in Protein Science

DL is transforming protein science by speeding up predictions for functions, structures, and

interactions, which is key for drug design, biotech, and education (Hu et al., 2024). For
example, ESM uses evolutionary data to predict functions with high accuracy, though it
struggles with rare or novel proteins (Ahmed et al., 2024). The SDN2Go model combines
sequence and structure data for better precision, but it needs real-world testing to ensure it
works broadly (Cai et al., 2020). In education, DL tools let students explore proteins hands-
on, like inputting a sequence and getting a prediction with clear explanations, without
needing a lab (Saharkhiz et al., 2024). My app builds on this with a simple interface for
sequence input, downloadable results, and explanations that make sense to beginners.

2.2.2 Importance of Deep Learning in Sequence-Based Prediction

Dl is critical for sequence-based prediction because it uncovers hidden patterns that reveal a
protein’s role without slow lab experiments. This is a big deal for students or researchers
wanting quick, affordable insights (Boadu et al., 2024). Lightweight tools that run on regular
laptops make this even more accessible, letting anyone dive into protein science. For
learning, DL turns complex biology into something approachable-a student could input a
sequence and get a result like “this protein might be kinase, which triggers cell signal,”with
examples to make it clear (Lee et al., 2022). My app enhances this with confidence scores
and links to databases like UniProt for extra info when predictions are uncertain.

2.2.3 Deep Learning for Adaptive Protein Analysis

Adaptive DL adjust predictions in real time, using feedback to refine results, like handling
proteins with multiple roles. It analyzes sequences on the fly, suggesting likely functions
based on patterns (Meng et al., 2025). For students, this means a tool that grows with tem-
starting simple and getting more detailed as they learn. For researchers, it offers quick
insights without heavy hardware, which app delivers with its easy interface and
downloadable outputs (Patel et al., 2025)

2.3 Protein Language Models

Protein language models (PLMs) are major leap, treating protein sequences like sentences in
a biological language to predict functions. They’re built on ideas from natural language to
predict functions. They’re built on ideas from natural language processing models like BERT,
adapted for biology (Ling et al., 2025). This section covers what PLMs are, their types, how
they’ve evolved, their impact, and their challenges.

2.3.1 Concept of Protein Language Models

PLMs view protein sequences as “text,” using DL to learn rules about how amino acids form
functions. They create embeddings-compact numerical codes that capture a protein’s traits-
without always needin 3D stuctures (Elnaggar et al., 2021, adjusted from table). Key
concepts include:

 Embeddings: Vectors encoding sequence info for tasks like classification.

 Self-Supervised Pre-Training: Models learn from huge unlabeled datasets, guessing
missing amino acids.
 Supervised Fine-Tuning: Pre-trained models are tweaked with labeled data for
specific tasks, like sorting proteins into functional groups. This setup makes PLMs
versatile for my app’s goals of classifying sequences and providing clear
explanations.

2.3.2 Types of Protein Language Models

PLMs vary by task:

 Sequence-Only Models: Like ProtT5, these focus on amino acid strings, ideal for
lightweight apps like mine (Elnaggar et al., 2021)
 Structure-Aware Models: ESM variants add 3D hints for better accuracy but need
more computing power (Lin et al., 2022).
 Generative Models: These create new protein sequences for drug or enzyme design
(Wang et al., 2023)
 Graph-Based Models: Combine sequence, structure, and other data for richer
predictions (Cai et al., 2020). My app uses sequence-only models for simplicity and
low resource needs.

2.3.3 Evolution of Protein Language Models

PLMs have evolved over time:

 Early 2010s: Basic CNNs and RNNs tackled secondary structure prediction (Liu et
al., 2020).
 Late 2010s: Big models like ESM and AlphaFold scaled up with massive datasets,
boosting accuracy (Jumper et al.,2021; Lin et al., 2022).
 Now: Focus on multi-functionality, novel proteins, and efficient models (Patel et al.,
2025). This move toward sequence-based, lightweight models align with my app’s
design.

2.3.4 Impact on Protein Research and Education

PLMs speed up research, aiding drug discovery and biotech by predicting functions quickly
(Hu et al., 2024). In education, they let students experiment with real data, building skills
without labs. For example, a student could input a sequence, get a prediction, and download a
report with clear explanations (Saharkhiz et al., 2024).

2.3.5 Global Significance and Challenges

PLMs are huge for fields like personalized medicine, where knowing protein functions can
lead to new treatments (Boady et al., 2024). Challenges include:

 Data Bias: Models trained on well-studied proteins may miss diverse ones (Ahmed et
al., 2024)
 Compute Needs: Some require powerful hardware, tough for small setups (Pakhrin et
al., 2021).
 Interpretability: Results can be hard for beginners to understand (Lee et al., 2022).
My app addresses these with a simple interface, clear explanations, and low resource
needs.
2.4 Challenges with Multi-Functional Proteins

Some proteins are like multi-taskers, doing more than one job in a cell, like acting as both
enzyme and a signaling molecule. This is tricky for AI models because most, including ESM,
are trained to pick one main function. A few studies have tried multi-label prediction, where a
model lists multiple functions if they’re likely, but it’s still a work in progress. For instance, a
survey on advances in deep learning in protein sciences notes DL’s impact but the need for
integration with experimental biology (Hu et al., 2024). Structure based prediction using
GCNs improved accuracy but is sensitive to input quality (Zhang et al., 2021)

An in-depth review on protein structure prediction via DL compared models like AlphaFold
and found major accuracy jumps since 2020, but no unified testing protocol (Meng et al.,
2025). State-of-the-art DL in protein design discusses novel proteins (Jumper et al., 2021).
My tool addresses this by showing the top few likely functions if they’re confident enough
and linking users to Uniprot for more details.

2.5 Accessibility issues in Bioinformatics Tools

Many tools are built for experts, requiring coding skills or high-end computers. BLAST and
InterPro are powerful but complex, while AlphaFold needs serious hardware for 3D
predicions (Smith et al., 2020; Jumper e al., 2021). This shuts out students and small labs.
DL-based annotation tools improved novel protein prediction but lack user-friendly interfaces
(Lee et al., 2022). Multi-omics models boost accuracy but are computationally heavy (kumar
et al., 2023). My app uses sequence-only models and a simple interface to make predictions
accessible

2.6 Supervised vs. Unsupervised Learning in Protein Prediction

My app uses supervised learning, where models like ProtT5 and ESM are trained on labeled
data (e.g., sequences tagged as Transferases) to predict functions for new sequences. This
works well for known functions but can struggle with novel proteins (Zhao et al., 2022).
Unsupervised learning finds patterns without lables, like clustering similar sequnces, but
iy’s less useful for specific function prediction (Liu et al., 2020). My app sticks to suervised
learning for clear, targeted results.
2.7 Summary of Related Works
Reference Title Methodology Finding Limitation
Boadu et Deep Learning Comparative analysis of DL improves Need for
al., 2024 Methods for DL models annotation unified
Protein accuracy benchmark
Function datasets
Prediction
Swenson PersGNN: Combines topology and High accuracy in Limited
et al., Applying geometry in GNN structure-based testing on
2020 Topological prediction diverse
Data Analysis proteins
and Geometric
Deep Learning
Ling et Transformer in Reviewed 100+ Promising Lack of
al., 2025 protein: A Transformer-based results in various standardized
Survey studies tasks evaluation
metrics
Pakhrin Deep Learning Conceptual Focus on DL improves Less progress
et al., Based DL for distance/contact accuracy over on disordered
2021 Advances in prediction traditional regions
Protein models
Structure
Prediction
Cai et al., SDN2GO: An Multi-modal DL Higher precision Needs
2020 integrated architecture than standalone validation on
Deep Learning models real-world
Model for datasets
Protein
Function
Hu et al., Advances of Reviews CNNs, RNNs, DL impactful in Neds
2024 Deep Learning Transformers function/structur integration
in Protein e prediction with
 Science: A experimental
Survey biology
Gaujac et Deep Learning Maps continuous Achived Struggles with
al., 2024 for protein structures to discrete RMSD<5 A in flexible
Structure space reconstructions protein
Prediction & regions
Design
Meng et Protein Analysis of Major accuracy No unified
al., 2021 Structure Alphafold,RGN,GVP,et jumps since testing
Prediction via c 2020 protocol
Deep
Learning: An
In-Depth
Review
Zhang et Structure- Graph-based modeling Improve Sensitive to
al., 2021 Based Protein of structures accuracy in input structure
Function function quality
Prediction predictions
Using GCNs
Saharkhi State-of-the- Combines functional Design of novel Lacl of
z et al., Art Deep site modeling with DL proteins possible experimental
2024 Learning in validations
Protein Design
& Prediction
Jumper AlphaFold: End-to-end DL on Outperformed all Limited
et al., Using AI for evolutionary & in CASP14 performance
2021 Protein structural data for disordered
Structure proteins
Prediction
Lee et al., DL approaches Combines CNNs and Improved Needs
2022 for Protein attention layers on annotations of improved
Function sequences novel proteins interpretabilit
Annotation y
Kumar et Integrating Fusion of omics data via Increased High
al., 2023 Multi-Omics deep neural networks accuracy in computational
Data for annotation complexity
Funcion
Prediction
Using DL
Smith et Predicting Uses hybrid CNN + Accurate for Limited
al., 2021 protein-protein GCN architecture known generalization
interactions interactomes to novel
Using DL proteins
Wang et Protein Transformers-based Improved Requires large
al., 2023 Function attention model detection of data to avoid
Prediction active sites overfitting
Using
Attention
Mechanisms
Zhao et Protein Transformer-based Improved Requires large
al., 2022 Function attention model detection of data to avoid
prediction active sites overfitting
Using
Attention
Mechanisms
Liu et al., DL Uses bi-directional Higher Q3 and Issues with
2020 Techniques for RNNS and dropout Q8 accuracy predicting coil
Protein layers regions
Secondary
Structure
Prediction
Ahmed et Evaluating DL Builds benchmark Enabled head-to- Benchmark
al., 2024 Models for datasets for fair head model lacks diversity
Protein comparison evaluations in protein
Function families
Prediction
Patel et DL in Review of cutting-edge Identified Future
al., 2025 Structural emerging areas direction
 Bioinformatics DL architectures like diffusion lacks
: Current models experimental
Trends input

2.8 Conclusion

The research shows deep learning, especially PLMs like ProtT5 and ESM, is revolutionizing
protein function prediction through supervised learning. But there’s still a need for tools that
are simple, run on basic computers, and handle complex cases like mukti-functional or novel
proteins. My project fills this gap with a lightweight app that offers easy sequence input, clear
predictions with examples, downloadable results, and support for tricky proteins, making
protein science approachable for students and small labs everywhere.

References

Ahmed, N., et al. (2024). Evaluating the advancements in protein language models for
function prediction. Bioinformatics Advances, 5(1), vbae001.
[Link]

Boadu, F., et al. (2024). Deep learning methods for protein function prediction.
Proteomics. [Link]

Cai, Y., Wang, j., & Deng, L. (2020). SDN2GO: An integrated deep learning model
for protein function prediction. Frontiers in Bioengineering and Biotechnolgy, 8, 391.
[Link]

Chen, M., et al. (2021). Prediction of protein- protein interactions based on integrating
deep learning and feature fusion. International Journal of Molecular Sciences, 25911),
6015. [Link]

Gaujac, B., et al. (2024). Deep learning for protein structure prediction & design.
arXiv. (Note: Full DOI not provided; use as is)

Hu, J., et al. (2024). Advances of deep learning in protein science: A survey arXiv.
[Link]
Jumper, J., et al. (2021). Highly accurate protein structure prediction with
ALPHAFold. Nature, 596(7873), 583-589. [Link]
03819-2

Kumar, A., et al. (2023). Integrating multi-omics data for function prediction using
DL. ScienceDirect. (Note: Full DOI not provided; use as is)

Lee, S., et al. (2022). Deep learning approaches for the prediction of protein functional
sites and regions: A review. Frontiers in Bioinformatics, 5, 1498662.

[Link]

Ling, C., et al. (202). Transformer in protein: a survey. arXiv.

[Link]

Liu, Y., et al. (2020). Deep learning for protein secondary structure prediction: Pre
and post-AlphaFold. Computational and Structural Biotechnology Journal, 20, 6323-
6340. [Link]

Meng, Q., et al. (2025). Protein structure prediction via deep learning: An in-depth
review. Frontiers in Pharma\cology, 16, 1498662.

[Link]

Pakhrin, S. C., et al. (2021). Deep Learning-based advances in protein structure

prediction . International Journal of Molecular Sciences, 22(11), 5553
[Link]

Patel, D., et al. (2025). Deep learning in structural bioinformatics: Current

applications and future directins. Briefings in Bioinformatics, 25(3), bbae204.
[Link]

Saharkhiz, A., et al. (2024). The state-of-the-art overview to appplication of deep

leaning in protein modeling. International journal of Molecular Sciences, n25(13),
7281. [Link]
Swenson, K., et al. PersGNN: Applying topological data analysis and geometric deep
learning to structure-based protein function prediction. arXiv.
[Link]

Smith, J., et al. (2020). Deep learning in protein structural modeling and designs.
Patterns, 1(9), 100142. [Link]

Wang, J., et al. (2023). De novo protein design-From new structures to programmable
functions. Cell, 187(3), 531-544. [Link]

Zhang, Z., et al. (2021). Structure-based protein function prediction using graph
convolutional networks. Nature Communications, 12(1), 3168.
[Link]

Zhao, H., et al. (2022). Multistage attenion-based extraction and fusion model for
protein sequence and structure features in gene ontology

Prediction. Frontiers in Genetics, 16, 1498662.

[Link]