Text Data Padding
[00:00:00.00] [MUSIC PLAYING]
[00:00:07.92] RYAN AHMED: Hello, everyone, and welcome to this lesson on text data
padding. Padding is a key step before training machine learning models, and it refers to the
process of adding characters to a text input to make all inputs have the same length.
[00:00:24.30] Here are the key learning objectives of this lesson. Understand the intuition behind
padding. Perform text data padding using pad sequence. Understand the concept of tensors. So
let's head over to our Jupyter Notebook and get started.
[00:00:40.85] [MUSIC PLAYING]
[00:00:48.06] All right, so let's go ahead and perform text data padding. So padding refers to the
process of adding characters to a text input to make all inputs have the same length. Let me show
you an example.
[00:01:02.23] If you recall what we have done before in the previous lesson, we learned how to
tokenize our three text data sets. If you recall, if we go up here. We had three news data. We had
the Hong Kong data. We also have the Novo Nordisk news. We also had the Amazon news as
well.
[00:01:21.66] What we have done is we have been able to simply tokenize all these different
news texts. But what you see here is, well, you see that these tokenized version of the news text
data, they have different lengths.
[00:01:35.88] For example, the first one here has 14 tokens. The second one, the second use data,
had 16 tokens, and the third one had 19 tokens. Basically all what the padding operation does, it
just adds an additional characters here to the right.
[00:01:55.36] And if we perform specifically zero padding, that is going to add zeros here to
make all the different news data here have the same length. That's all it is.
[00:02:07.45] So let me show you how we're going to perform that encode. So what I'm going to
do first here is I'm going to view and encoded version of sample data set one, and I'm going to
also obtain the length.
[00:02:19.48] So I'm going to say please print the encoded news underscore one, and please also
obtain the length, how many tokens are included in the encoded news underscore one. You press
Shift and Enter, here we go.
[00:02:31.36] This is, again, the encoded version of my first news data. And here I simply have
14 tokens. If you go ahead and do the same thing as well for the encoded news two, so if I say
please print encoded news two, and also print out the length as well, here we go.
[00:02:49.11] You will see that right now I have 16 tokens. And let's go ahead and run the next
one. If you press Shift and Enter, here we go. You would see that right now I have 19 tokens.
And basically because all of these data sets right now, they have different lengths, so now I
would like to simply perform padding to increase the 16 here by adding three zeros to it to make
it 19.
[00:03:14.25] And then you grab that 14 here, you add five zeros to it, you make it 19, and that's
what you see here. I'm going to add zero padding to the second one, and I'm going to add zero
padding to the first one as well to make them all have the same length as the third news data set.
[00:03:32.20] So please note that data need to be converted to tensors before we perform
padding. So the tensors are the main data structure used in PyTorch to represent multi-
dimensional arrays such as images, text, and audio.
[00:03:48.61] And PyTorch is a widely used framework that has been developed by Meta AI,
which is formerly Facebook, and offers an easy-to-use interface for building, training, and testing
artificial neural networks.
[00:04:03.13] Tensors can be created from NumPy arrays or from Python lists. So basically in
order for us to do padding using PyTorch, well, I need to first install PyTorch.
[00:04:13.69] So this line of code is going to install PyTorch. So I'm going to say pip install
torch. If you press Shift and Enter, that is going to run or execute this cell for you. And then I'm
going to import Torch. And then I'm going to import pad sequence, which is going to perform
padding for us.
[00:04:31.93] And then all you need to do is to say pad sequence. And then you grab the encoded
news one. I'm going to convert it to a tensor using PyTorch. I'm going to do the same as well for
encoded news two, so I'm going to convert it as well to tensors. Do the same as well for news
three.
[00:04:49.72] And if you scroll to the right, you will see that right now I'm going to convert that
data into NumPy arrays. And I'm also going to set the padding value to be 0, because I would
like to perform zero padding.
[00:05:01.40] So if you press Shift and Enter, here we go. Well, you see right now is I ended up
with simply three here arrays. And what you see is that basically added zeros here to the first
one, five zeros, and I added three zeros so the second one. And I didn't add any zeros here
because that was the list that has the maximum number of tokens, which is 19 tokens.
[00:05:24.97] And that's it. That's simply all I have for this lesson. I hope you enjoyed it. Please
go ahead give the next practice opportunity a try. And then I'm going to show you a detailed
video explanation so you can compare your answers to mine. Best of luck, and I will see you in
the next lesson.
[00:05:38.56] [MUSIC PLAYING]