DEEP LEARNING

SEMINAR PAPER

PRESENTED BY

SUNDAY, NSEMEKE ISAIAH

AKP/ASC/CSC/HND2020/1320

TO

THE DEPARTMENT OF COMPUTER SCIENCE AKWA

IBOM STATE POLYTECHNIC, IKOT OSURUA

IN PARTIAL FULLFILMENT OF
THE REQUIREMENTS FOR THE AWARED OF
HIGHER NATIONAL DIPLOMA (HND) IN
COMPUTER SCIENCE

AUGUST, 2022

1
 APPROVAL PAGE
This seminar paper is the original work carried out by SUNDAY,
NSEMEKE ISAIAH with Registration Number
AKP/ASC/CSC/HND2020/1320, of the Department of Computer Science,
Akwa Ibom State Polytechnic, Ikot Osurua, Ikot Ekpene, under the
supervision of:

MR. UBOKOBONG, BENJAMIN

…………………………..
Supervisor Signature & Date

2
 ABSTRACT
Deep learning is a class of machine learning algorithms that uses multiple
layers to progressively extract higher-level features from the raw input. In
the last few years, the deep learning (DL) computing paradigm has been
deemed the Gold Standard in the machine learning (ML) community.
Moreover, it has gradually become the most widely used computational
approach in the field of ML, thus achieving outstanding results on several
complex cognitive tasks, matching or even beating those provided by human
performance. One of the benefits of DL is the ability to learn massive
amounts of data. The DL field has grown fast in the last few years and it has
been extensively used to successfully address a wide range of traditional
applications. More importantly, DL has outperformed well-known ML
techniques in many domains, e.g., cyber security, natural language
processing, bioinformatics, robotics and control, and medical information
processing, among many others. In this paper, the basic concept of deep
learning is considered. In this paper, an overview of DL is presented that
adopts various perspectives such as the main concepts, performance feature,
classification and applications.

3
Introduction

Deep learning (also known as deep structured learning) is part of a broader

family of machine learning methods based on artificial neural
networks with representation learning. Learning can be supervised, semi-
supervised or unsupervised (Bengioet al, 2015)

Deep-learning architectures such as deep neural networks, deep belief

networks, deep reinforcement learning, recurrent neural
networks, convolutional neural networks and Transformers have been
applied to fields including computer vision, speech recognition, natural
language processing, machine translation, bioinformatics, drug
design, medical image analysis, climate science, material inspection
and board game programs, where they have produced results comparable to
and in some cases surpassing human expert performance (Ciresan et al,
2012).

Concept of Deep Learning

Deep learning attempts to mimic the human brain—albeit far from

matching its ability—enabling systems to cluster data and make predictions
with incredible accuracy.

Deep learning is a subset of machine learning, which is essentially a

neural network with three or more layers. These neural networks attempt to
simulate the behavior of the human brain—albeit far from matching its
ability—allowing it to “learn” from large amounts of data. While a neural
network with a single layer can still make approximate predictions,
additional hidden layers can help to optimize and refine for accuracy (Adeel,
2020).

4
 Deep learning drives many artificial intelligence (AI) applications and
services that improve automation, performing analytical and physical tasks
without human intervention. Deep learning technology lies behind everyday
products and services (such as digital assistants, voice-enabled TV remotes,
and credit card fraud detection) as well as emerging technologies (such as
self-driving cars).

Fig.1: Deep Learning Family

Deep Learning (DL) is a subset of Machine Learning (ML) as seen in

Fig. 1, Deep Learning is inspired by the information processing patterns
found in the human brain. DL does not require any human-designed rules to

5
operate; rather, it uses a large amount of data to map the given input to
specific labels. DL is designed using numerous layers of algorithms
(artificial neural networks, or ANNs), each of which provides a different
interpretation of the data that has been fed to them (Zhang et al, 2020).

Deep learning neural networks, or artificial neural networks, attempts

to mimic the human brain through a combination of data inputs, weights,
and bias. These elements work together to accurately recognize, classify, and
describe objects within the data.

Deep neural networks consist of multiple layers of interconnected

nodes, each building upon the previous layer to refine and optimize the
prediction or categorization. This progression of computations through the
network is called forward propagation. The input and output layers of a deep
neural network are called visible layers. The input layer is where the deep
learning model ingests the data for processing, and the output layer is where
the final prediction or classification is made.
Another process called back propagation uses algorithms,
like gradient descent, to calculate errors in predictions and then adjusts the
weights and biases of the function by moving backwards through the layers
in an effort to train the model. Together, forward propagation and back
propagation allow a neural network to make predictions and correction for
any errors accordingly. Over time, the algorithm becomes gradually more
accurate.
Deep Learning is useful in many situations which is equal or better than
human experts in some cases (Topol, 2019), meaning that DL can be a
solution to the following problems:

6
 - Cases where human experts are not available.
- Cases where humans are unable to explain decisions made using their
expertise (language understanding, medical decisions, and speech
recognition).
- Cases where the problem solution updates over time (price prediction,
stock preference, weather prediction, and tracking).
- Cases where solutions require adaptation based on specific cases
(personalization, biometrics).
- Cases where size of the problem is extremely large and exceeds our
inadequate reasoning abilities (sentiment analysis, matching ads to
Facebook, calculation webpage ranks).

Performance Feature of Deep Learning

The following are the several performance feature of deep learning:

Universal Learning Approach: Because DL has the ability to perform in

approximately all application domains, it is sometimes referred to as
universal learning.

Robustness: In general, precisely designed features are not required in DL

techniques. Instead, the optimized features are learned in an automated
fashion related to the task under consideration. Thus, robustness to the usual
changes of the input data is attained.

Generalization: Different data types or different applications can use the

same DL technique, an approach frequently referred to as transfer learning
(TL) which explained in the latter section. Furthermore, it is a useful
approach in problems where data is insufficient.

7
Scalability: DL is highly scalable. ResNet (He et al, 2016), which was
invented by Microsoft, comprises 1202 layers and is frequently applied at a
supercomputing scale. Lawrence Livermore National Laboratory (LLNL), a
large enterprise working on evolving frameworks for networks, adopted a
similar approach, where thousands of nodes can be implemented (Van, et al,
2015).

Classification of Deep Learning Approach

DL techniques are classified into three major categories: unsupervised,

partially supervised (semi-supervised) and supervised. Furthermore, deep
reinforcement learning (DRL), also known as RL, is another type of learning
technique, which is mostly considered to fall into the category of partially
supervised (and occasionally unsupervised) learning techniques.

Deep Supervised Learning: This technique deals with labeled data. When
considering such a technique, the environs have a collection of inputs and
resultant outputs. For DL, there are several supervised learning techniques,
such as recurrent neural networks (RNNs), convolution neural networks
(CNNs), and deep neural networks (DNNs). In addition, the RNN category
includes gated recurrent units (GRUs) and long short-term memory (LSTM)
approaches

The main advantage of this technique is the ability to collect data or generate
a data output from the prior knowledge. However, the disadvantage of this
technique is that decision boundary might be overstrained when training set
doesn’t own samples that should be in a class. Overall, this technique is
simpler than other techniques in the way of learning with high performance.

8
Deep Semi-Supervised Learning: In this technique, the learning process is
based on semi-labeled datasets. Occasionally, generative adversarial
networks (GANs) and DRL are employed in the same way as this technique.
In addition, RNNs, which include GRUs and LSTMs, are also employed for
partially supervised learning. One of the advantages of this technique is to
minimize the amount of labeled data needed. On other the hand, one of the
disadvantages of this technique is irrelevant input feature present training
data could furnish incorrect decisions. Text document classifier is one of the
most popular example of an application of semi-supervised learning. Due to
difficulty of obtaining a large amount of labeled text documents, semi-
supervised learning is ideal for text document classification task.

Deep Unsupervised Learning: This technique makes it possible to

implement the learning process in the absence of available labeled data (i.e.
no labels are required). Here, the agent learns the significant features or
interior representation required to discover the unidentified structure or
relationships in the input data. Techniques of generative networks,
dimensionality reduction and clustering are frequently counted within the
category of unsupervised learning. Several members of the DL family have
performed well on non-linear dimensionality reduction and clustering tasks;
these include restricted Boltzmann machines, auto-encoders and GANs as
the most recently developed techniques. Moreover, RNNs, which include
GRUs and LSTM approaches, have also been employed for unsupervised
learning in a wide range of applications. The main disadvantages of
unsupervised learning are unable to provide accurate information concerning

9
data sorting and computationally complex. One of the most popular
unsupervised learning approaches is clustering (Saeed, et al, 2020).

Deep Reinforcement Learning: Reinforcement Learning operates on

interacting with the environment, while supervised learning operates on
provided sample data. This technique was developed in 2013 with Google
Deep Mind (Mnih, et al, 2015). Subsequently, many enhanced techniques
dependent on reinforcement learning were constructed.

Based on this concept, several supervised and unsupervised

techniques were developed. In comparison with traditional supervised
techniques, performing this learning is much more difficult, as no
straightforward loss function is available in the reinforcement learning
technique. In addition, there are two essential differences between
supervised learning and reinforcement learning: first, there is no complete
access to the function, which requires optimization, meaning that it should
be queried via interaction; second, the state being interacted with is founded
on an environment, where the input is based on the preceding actions
(Arulkumaran, 2017).

For solving a task, the selection of the type of reinforcement learning

that needs to be performed is based on the space or the scope of the problem.
Some of the applications of reinforcement learning are business strategy
planning and robotics for industrial automation. The main drawback of
Reinforcement Learning is that parameters may influence the speed of
learning. Here are the main motivations for utilizing Reinforcement
Learning:

10
 - It assists you to identify which action produces the highest reward
over a longer period.
- It assists you to discover which situation requires action.
- It also enables it to figure out the best approach for reaching large
rewards.
- Reinforcement Learning also gives the learning agent a reward
function.

Deep Learning and Machine Learning

Deep learning is a subset of machine learning, they differ in various

ways. Deep learning distinguishes itself from classical machine learning by
the type of data that it works with and the methods in which it learns.

Machine learning algorithms leverage structured, labeled data to make

predictions—meaning that specific features are defined from the input data
for the model and organized into tables. This doesn’t necessarily mean that it
doesn’t use unstructured data; it just means that if it does, it generally goes
through some pre-processing to organize it into a structured format.

Deep learning eliminates some of data pre-processing that is typically

involved with machine learning. These algorithms can ingest and process
unstructured data, like text and images, and it automates feature extraction,
removing some of the dependency on human experts. For example, let’s say
that we had a set of photos of different pets, and we wanted to categorize by
“cat”, “dog”, “hamster”, et cetera. Deep learning algorithms can determine
which features (e.g. ears) are most important to distinguish each animal from

11
another. In machine learning, this hierarchy of features is established
manually by a human expert.

Then, through the processes of gradient descent and back propagation,

the deep learning algorithm adjusts and fits itself for accuracy, allowing it to
make predictions about a new photo of an animal with increased precision.

Machine learning and deep learning models are capable of different

types of learning as well, which are usually categorized as supervised
learning, unsupervised learning, and reinforcement learning. Supervised
learning utilizes labeled datasets to categorize or make predictions; this
requires some kind of human intervention to label input data correctly. In
contrast, unsupervised learning doesn’t require labeled datasets, and instead,
it detects patterns in the data, clustering them by any distinguishing
characteristics. Reinforcement learning is a process in which a model learns
to become more accurate for performing an action in an environment based
on feedback in order to maximize the reward.

Deep Learning Applications

Real-world deep learning applications are a part of our daily lives, but
in most cases, they are so well-integrated into products and services that
users are unaware of the complex data processing that is taking place in the
background. Some of these examples include the following:

Law Enforcement: Deep learning algorithms can analyze and learn from
transactional data to identify dangerous patterns that indicate possible
fraudulent or criminal activity. Speech recognition, computer vision, and
12
other deep learning applications can improve the efficiency and
effectiveness of investigative analysis by extracting patterns and evidence
from sound and video recordings, images, and documents, which helps law
enforcement analyze large amounts of data more quickly and accurately.

Financial Services: Financial institutions regularly use predictive analytics

to drive algorithmic trading of stocks, assess business risks for loan
approvals, detect fraud, and help manage credit and investment portfolios for
clients.

Customer Service: Many organizations incorporate deep learning

technology into their customer service processes. Chatbots—used in a
variety of applications, services, and customer service portals—are a
straightforward form of AI. Traditional chatbots use natural language and
even visual recognition, commonly found in call center-like menus.
However, more sophisticated chatbot solutions attempt to determine,
through learning, if there are multiple responses to ambiguous questions.
Based on the responses it receives, the chatbot then tries to answer these
questions directly or route the conversation to a human user.

13
Virtual assistants like Apple's Siri, Amazon Alexa, or Google Assistant
extends the idea of a chatbot by enabling speech recognition functionality.
This creates a new method to engage users in a personalized way.

Healthcare: The healthcare industry has benefited greatly from deep

learning capabilities ever since the digitization of hospital records and
images. Image recognition applications can support medical imaging
specialists and radiologists, helping them analyze and assess more images in
less time.

14
Conclusion

Deep learning is a type of machine learning and artificial intelligence (AI)

that imitates the way humans gain certain types of knowledge. Deep
Learning was seen to be applicable in areas such as Law Enforcement,
Financial Services, Customer Services and Healthcare.

15
Recommendations

The researcher therefore recommended that deep learning concepts should

be adopted by relevance bodies as it can replace human expert to cover up
lapses.

16
References

Adeel A, Gogate M, Hussain A. (2021).Contextual deep learning-based

audio-visual switching for speech enhancement in real-world
environments. Inf Fusion; 59:163–70

Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017). Deep

reinforcement learning: a brief survey. IEEE Signal Process Mag.;
34(6):26–38

Bengio, Yoshua; LeCun, Yann; Hinton, Geoffrey (2015). "Deep

Learning". Nature. 521 (7553): 436 444. Bibcode:
2015Natur.521. .436L. doi:10.
1038/nature14539. PMID 26017442. S2CID 3074096.

Ciresan, D.; Meier, U.; Schmidhuber, J. (2012). "Multi-column deep neural

networks for image classification". 2012 IEEE Conference on
Computer Vision and Pattern Recognition. pp. 3642–3649. arXiv:
1202.2745. doi:10.1109/c vpr. 2012.6248110. ISBN 978-1-4673-
1228-8. S2CID 2161592.

He K, Zhang X, Ren S, Sun J. (2016). Deep residual learning for image

recognition. In: Proceedings of the IEEE conference on computer vision
and pattern recognition; p. 770–8

Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. (2017) A survey of deep
neural network architectures and their applica- tions. Neurocomputing;
234:11–26.

Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG,

Graves A, Riedmiller M, Fidjeland AK, Ostrovski G. (2015). Human-

17
 level control through deep reinforcement learning. Nature.
2015;518(7540):529–33.

Saeed MM, Al Aghbari Z, Alsharidah M. (2020). Big data clustering

techniques based on spark: a literature review. PeerJ Comput
Sci.;6:321.

Shorten C, Khoshgoftaar TM, Furht B. (2021)Deep learning applications for

COVID-19. J Big Data. ;8(1):1–54

Topol EJ (2019). High-performance medicine: the convergence of human

and artifcial intelligence. Nat Med;25(1):44–56.

Van Essen B, Kim H, Pearce R, Boakye K, Chen B. (2015). Lbann:

livermore big artifcial neural network HPC toolkit. In: Proceedings of
the workshop on machine learning in high-performance computing
environments; p. 1–6.

Zhang Z, Cui P, Zhu W. (2020). Deep learning on graphs: a survey. IEEE

Trans Knowl Data Eng. 2020. [Link]
1109/TKDE.2020.2981333.

18