Scribd
Upload
2 views8 pages

Encoder and Decoder

The document explains the encoder-decoder model in natural language processing, which consists of three main components: the encoder, hidden vector, and decoder. The encoder converts input sequences into a hidden vector, while the decoder generates output sequences based on this hidden vector. The output layer utilizes a Softmax activation function to create a probability distribution for the predicted output.

Uploaded by

Loga baalan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views8 pages

Encoder and Decoder

The document explains the encoder-decoder model in natural language processing, which consists of three main components: the encoder, hidden vector, and decoder. The encoder converts input sequences into a hidden vector, while the decoder generates output sequences based on this hidden vector. The output layer utilizes a Softmax activation function to create a probability distribution for the predicted output.

Uploaded by

Loga baalan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

NLP-Natural Language

Processing
Prepared by
V. Sangavi AP/AI&DS
6
ENCODER-DECODER MODEL
There are three main blocks in the encoder-decoder model,
Encoder
Hidden Vector
Decoder
The Encoder will convert the input sequence into a single-
dimensional vector (hidden vector).
The decoder will convert the hidden vector into the output
sequence.
6
ENCODER-DECODER MODEL

Encoder-decoder sequence to sequence model


6
ENCODER
Multiple RNN cells can be stacked together to form the encoder.
RNN reads each inputs sequentially.
For every timestep (each input) t, the hidden state (hidden vector)
h is updated according to the input at that timestep X[i].
After all the inputs are read by encoder model, the final hidden
state of the model represents the context/summary of the whole
input sequence.
6
DECODER
The Decoder generates the output sequence by predicting the
next output Yt given the hidden state ht.
The input for the decoder is the final hidden vector obtained at
the end of encoder model.
Each layer will have three inputs, hidden vector from previous
layer ht-1 and the previous layer output yt-1, original hidden
vector h.
A stack of several recurrent units where each predicts an output
y_t at a time step t.
Each recurrent unit accepts a hidden state from the previous unit
and produces an output as well as its own hidden state
6
DECODER

ENCODER DECODER
6
OUTPUT LAYER
We use Softmax activation function at the output layer.
It is used to produce the probability distribution from a vector of
values with the target class of high probability.
The output y_t at time step t is computed using the formula:
THANK YOU

You might also like