Deep Neural Networks
1. The extension of conventional artificial neural networks is deep neural networks.
Compared to conventional neural networks, there are two main differences that deep
neural networks have.
2. It is shallow for conventional neural networks to have one or two hidden layers. On the
other hand, there are many hidden layers in deep neural networks. For instance,
a neural network of millions of neurons was used by the Google brain project.
3. There is a wide range of models for deep neural networks, ranging from DNNs, CNNs,
RNNs, and LSTMs.
4. Recent studies have even brought us attention-based networks that focus on specific
parts of a deep neural network.
5. The larger the network and the more layers it has, the more complex the network
becomes and the more resources and more time it needs to train. Deep neural networks
work best with GPU-based architectures that take less time to train than classical CPUs,
while recent developments have shortened training times considerably (Sarkar
et al., 2018).
6. Deep is a word we have all been reading for a long time. Let us understand why it is
called deep neural network. The past rules developed for networks such as perceptron,
HNN, etc., worked on the concept of one input layer and one output layer.
7. But, if it's more than 3 layers, including the output and input layers, then it is called
a deep neural network. Hence, deep in its rawest form is more than just one hidden
layer. In DNNs, every layer of nodes trains on a different set of features based on the
previous layer's output.
8. These nets allow complex nonlinear relationships. The major advantage which DNNs
hold is the ability of dealing with unstructured and unlabeled data which is actually most
of the world's data. Especially in the medical field, most of the data is unstructured and
unlabeled. So DNNs are apt for this large amount of data and its processing. See Fig. 8.3.
A deep neural network (DNN) is an ANN with multiple hidden layers between the input and output
layers. Similar to shallow ANNs, DNNs can model complex non-linear relationships.
The main purpose of a neural network is to receive a set of inputs, perform progressively complex
calculations on them, and give output to solve real world problems like classification. We restrict
ourselves to feed forward neural networks.
We have an input, an output, and a flow of sequential data in a deep network.
Neural networks are widely used in supervised learning and reinforcement learning problems. These
networks are based on a set of layers connected to each other.
In deep learning, the number of hidden layers, mostly non-linear, can be large; say about 1000 layers.
DL models produce much better results than normal ML networks.
We mostly use the gradient descent method for optimizing the network and minimising the loss
function.
We can use the Imagenet, a repository of millions of digital images to classify a dataset into
categories like cats and dogs. DL nets are increasingly used for dynamic images apart from static
ones and for time series and text analysis.
Training the data sets forms an important part of Deep Learning models. In addition, Back
propagation is the main algorithm in training DL models.
DL deals with training large neural networks with complex input output transformations.
One example of DL is the mapping of a photo to the name of the person(s) in photo as they do
on social networks and describing a picture with a phrase is another recent application of DL.
Neural networks are functions that have inputs like x1,x2,x3that are transformed to outputs like z1,z2,z3
and so on in two (shallow networks) or several intermediate operations also called layers (deep
networks).
The weights and biases change from layer to layer. w and v are the weights or synapses of layers of the
neural networks.
The best use case of deep learning is the supervised learning [Link],we have large set of data
inputs with a desired set of outputs.
Here we apply back propagation algorithm to get correct output prediction.
The most basic data set of deep learning is the MNIST, a dataset of handwritten digits.
We can train deep a Convolutional Neural Network with Keras to classify images of handwritten digits
from this dataset.
The firing or activation of a neural net classifier produces a score. For example,to classify patients as sick
and healthy,we consider parameters such as height, weight and body temperature, blood pressure etc.
A high score means patient is sick and a low score means he is healthy.
Each node in output and hidden layers has its own classifiers. The input layer takes inputs and passes on
its scores to the next hidden layer for further activation and this goes on till the output is reached.
This progress from input to output from left to right in the forward direction is called forward
propagation.
Credit assignment path (CAP) in a neural network is the series of transformations starting from the input
to the output. CAPs elaborate probable causal connections between the input and the output.
CAP depth for a given feed forward neural network or the CAP depth is the number of hidden layers plus
one as the output layer is included. For recurrent neural networks, where a signal may propagate
through a layer several times, the CAP depth can be potentially limitless.