SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Network (RNN)
BCA-424
Semester -III
Dr. SHIV KUMAR VERMA
Professor
Department of Computer Science and Applications
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
What ML Can Do:
• Traditional machine learning models, such as linear regression, decision trees, support vector
machines (SVM), and k-nearest neighbors (KNN), can perform well on structured data where features
are independent of one another and the relationships between inputs and outputs are clear.
• Supervised ML: These models excel at supervised learning tasks like classification and regression.
• Unsupervised ML: ML techniques such as k-means clustering and PCA can also handle
unsupervised tasks.
Limitations of Traditional ML:
• Feature Engineering: Traditional ML methods heavily rely on manual feature extraction and domain
knowledge to transform raw data into features the algorithm can process.
• Handling Complex Data: Traditional ML models struggle with unstructured data (e.g., images, video,
or natural language) because these methods do not natively learn hierarchical feature representations.
• Dependence on Independence Assumption: Many algorithms (e.g., linear regression) assume
independence between features, which doesn’t hold true for complex data structures.
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Artificial Neural Networks (ANN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Artificial Neural Networks (ANN)
What ANN Can Do:
• ANNs are inspired by the human brain and are designed to handle more complex patterns and
non-linear relationships in the data, which traditional ML models struggle with.
• ANNs are particularly good at automatic feature learning. They can learn from raw data without
requiring manual feature extraction, which is critical for tasks involving images, audio, or text.
• Feedforward ANNs: These neural networks can approximate any continuous function, making them
more flexible than traditional ML models for classification, regression, and other tasks.
What ANN Struggles With:
• Static Input: ANNs are not designed to handle sequential or temporal data (e.g., time series data or
language processing). They treat inputs independently and do not consider the order of inputs.
• High Dimensional Data: While ANNs can handle complex data, they don't perform well when
dealing with high-dimensional data such as images, as they are not optimized for spatial
hierarchies.
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Backpropagation ANN
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNNs)
What CNN Can Do:
• CNNs are specifically designed for handling spatial data, making them the ideal choice for tasks like
image classification, object detection, and video analysis.
• CNNs can automatically learn spatial hierarchies of features through convolution operations, which
enables them to capture local dependencies in images (e.g., edges, shapes, textures).
• CNNs use layers like convolutional layers, pooling layers, and fully connected layers to
efficiently handle high-dimensional data such as images and video.
What CNN Struggles With:
• Sequential Dependencies: CNNs are limited to processing spatial relationships and struggle with
temporal sequences (e.g., time series, language processing, or any data where order matters).
• CNNs cannot remember long sequences or understand the order of events (i.e., they lack temporal
memory).
Solution: Recurrent Neural Networks (RNNs) are designed to handle
sequential data and temporal dependencies.
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Convolutional Neural Networks (CNN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
Overview:
Recurrent Neural Networks (RNNs) are designed for sequential data or time-
series data, where the order of the data points is important. Unlike ANNs and
CNNs, which assume independence between inputs, RNNs have connections
that form cycles, allowing them to "remember" previous information.
RNNs utilize a hidden state to retain information about previous inputs, making
them suitable for tasks where context matters (e.g., language modeling).
Key Components:
Hidden State: RNNs maintain a hidden state (memory) that captures
information from previous inputs. This hidden state is updated at each time
step.
Recurrent Connections: The output of each neuron is fed back into the
network, influencing the next step's input and maintaining temporal
dependencies.
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
What RNN Can Do:
• RNNs are designed to work with sequential data, such as time series, text, speech, or video. They
maintain a memory of previous inputs by using hidden states that pass information from one time step to the
next.
• RNNs can capture dependencies over time by sharing parameters across different parts of the sequence,
making them suitable for tasks like language modeling, machine translation, speech recognition, and
time series forecasting.
What RNN Struggles With:
• Vanishing Gradient Problem: RNNs struggle with capturing long-term dependencies because of the
vanishing gradient problem during backpropagation through time. This means that RNNs can forget early
parts of a long sequence as they propagate information forward.
• Limited Memory: While RNNs can retain some information over short sequences, their ability to remember
long-term information is limited.
Solution: Long Short-Term Memory (LSTM) networks, a type of
RNN, address the vanishing gradient problem and can handle
long-term dependencies.
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Recurrent Neural Networks (CNN)
Real-World Example:
Language Translation:
o Input: A sequence of words in one language (e.g., English).
o Output: The translated sequence of words in another language
(e.g., French).
o The RNN processes each word in the input sequence, remembers
the context, and generates the translation sequentially.
Applications of RNN:
Natural Language Processing (NLP): Text generation, language modeling, sentiment analysis, machine translation.
Speech Recognition: Converting spoken words into text, where temporal context is key.
Time Series Forecasting: Stock market predictions, weather forecasting, or predicting energy consumption over time.
– Music Generation: Generating sequences of notes or melodies
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Comparison of ANN, CNN, RNN
ANN (Artificial Neural CNN (Convolutional Neural RNN (Recurrent Neural
Feature
Networks) Networks) Networks)
General-purpose, structured Grid-like data (e.g., images, Sequential data (e.g., time
Data Type
or unstructured data 2D data) series, text, speech)
Convolutional layers Recurrent connections with
Architecture Fully connected layers
followed by pooling layers hidden state
No memory of previous Has memory through hidden
Memory No memory of previous inputs
inputs state (remembers past inputs)
Excellent at recognizing Good for tasks requiring
Versatile, can be applied to a
Strengths spatial hierarchies and sequence processing, such as
variety of tasks
patterns in images time-series or text
Struggles with long-term
Inefficient for spatial or Cannot handle sequential
Limitations dependencies (unless using
sequential data data efficiently
LSTM/GRU)
Language translation, speech
Common Classification, regression, Image classification, object
recognition, time-series
Applications medical diagnosis detection, medical imaging
forecasting
More complex, needs more
Slower due to sequential data
Training Complexity Simple and fast computation for large
processing
images
Handwritten digit recognition Language translation, stock
Real-World Example Predicting house prices
(MNIST), object detection price forecasting
SCHOOL OF ENGINEERING &TECHNOLOGY
D E P A R T M E N T O F C O M P U T E R S C I E N C E A N D APPLICATIONS
Happy
Learning