Scribd
Upload
8 views14 pages

RNN and LSTM: Applications and Examples

The document discusses Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, highlighting their ability to handle temporal data for applications such as speech recognition, machine translation, and stock price prediction. It addresses the vanishing gradient problem and outlines the components of LSTM, including input, forget, and output gates. Additionally, it provides practical examples and applications of LSTM in various fields, including speech synthesis and sentiment analysis.

Uploaded by

潘建蒼
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views14 pages

RNN and LSTM: Applications and Examples

The document discusses Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, highlighting their ability to handle temporal data for applications such as speech recognition, machine translation, and stock price prediction. It addresses the vanishing gradient problem and outlines the components of LSTM, including input, forget, and output gates. Additionally, it provides practical examples and applications of LSTM in various fields, including speech synthesis and sentiment analysis.

Uploaded by

潘建蒼
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Recurrent Neural Networks

& Long Short-Termed


Memory
Prof. Kuan-Ting Lai
TA Alan Tian
2020/07/27
Recurrent Neural Network (RNN)
• Feedforward networks output
don’t consider temporal
states
• RNN has a loop to Recurrent
“memorize” information Connectio
n
RN
N

input

2
Unroll the RNN Loop
• Effective for speech recognition, language modeling,
translation

[Link] 3
Unroll RNN

4
Vanishing Gradient Problem
• Hochreiter (1991) [German] and Bengio, et al. (1994)

5
Long Short-Term Memory (LSTM)
• Input gate: control when to let new input in
• Forget gate: delete the trivial information
• Output gate: let the info impact the output at the current
time step

Hochreiter &
Schmidhuber
(1997)

6
Long Short-Term Memory (LSTM)

7
Long Short-Term Memory (LSTM)

8
LSTM 數學式 ( 李宏毅老師投影片)
Various types of LSTM

Ref 9
LSTM application
• Speech Recognition (Input is audio and output is text)
– Google Assistant
– Apple Siri
• Speech Synthesis (Input is text and output is audio)
• Machine Translation (Input is text and output is also text)
– Google Translation
• Image Captioning (Input is image and output is text)
• Sentiment Analysis (Input is text and output is rating)
• Music Generation/Synthesis ( input music notes and output is music)
• Video Activity Recognition (input is video and output is type of activity)

10
LSTM example - MNIST
• Build a LSTM model to
recognize hand write digit
• Example code on colab

11
Using LSTM to Predict Stock Price
12
Prac4
- using Keras LSTM to predict Stock Trends

• Dataset :TATA stock


price from 2010 to
2018
• Test our model with
the price data from
2018 to 2019

13
Prac4
- using Keras LSTM to predict Stock Trends
• Use sequence to sequence
model to predict future price
• Practice code on colab

14

You might also like