Perform Data Visualization
[00:00:00.00] [AUDIO LOGO]
[00:00:08.01] RYAN AHMED: Hello, everyone and welcome to this lesson on performing data
visualization. Python offers several data visualization libraries such as Matplotlib, Seaborn, and
Plotly. Matplotlib is the grandfather of data visualization libraries and can create static and
interactive visualization. Seaborn is Matplotlib on steroids. Seaborn offers enhanced features and
generates visually-appealing statistical plots. Plotly creates powerful, interactive charts using a
few simple lines of code.
[00:00:43.54] Here are the key learning objectives of this lesson-- visualize financial data using
key libraries, such as Matplotlib, Seaborn, and Plotly Express. Calculate the correlation matrix
and plot heatmaps using Seaborn library. Use Plotly to plot interactive histogram. So let's head
over to our Jupyter notebook and get started.
[00:01:14.80] All right, so in this lesson, we will perform data visualization. And Python offers
many libraries for data visualization, such as Matplotlib, Seaborn, and Plotly. Matplotlib is the
grandfather of data visualization libraries, and I've included a link here to the documentation.
And I've included the link here as well in a markdown segment. So if you just click on it, it
should simply open the Matplotlib documentation for you.
[00:01:44.56] And what I really like about Matplotlib, Seaborn, and Plotly documentation is that
it includes a ton of examples and tutorials with sample code. So if you click on Examples here,
you will find many examples if you would like to create bar charts, you'd like to do, for example,
line charts, and many additional examples available here for you. Let's pick maybe a simple one.
Maybe let's say bar chart demo. If you click on it, and you should see a sample graph, and if you
scroll down, you should see the code that generated that plot, which is pretty powerful.
[00:02:23.41] The next data visualization library is known as Seaborn, and Seaborn is based on
Matplotlib, and it offers enhanced features compared to Matplotlib. And I included the link here
as well for you. So this is the link to Seaborn documentation. And I have it open. And similar to
Matplotlib, Seaborn documentation includes a lot of examples. So for example here, if you click
on Gallery, you will see that you can plot scatterplots, you can do line plots, can also do hist
plots, the join plots, and you can see that, here, you can generate heatmaps, and that's actually the
visualization that we are going to create for us today. But we are going to use the last library, and
that is going to be Plotly library, to generate an example of a heat map.
[00:03:16.33] And, of course, if you go ahead and click on scatterplot, that is going to show you
a sample here of the plot. And you can see, also, the sample code as well, so you can leverage
some of these code scripts to create or generate your own charts on your own data sets. The last
example that I got here is from Plotly library, and Plotly creates interactive charts using a few
simple lines of code. And Plotly the offers over 40 plus chart types, including statistical,
financial, and scientific.
[00:03:48.16] And here is the data to Plotly documentation. If you just click on it, that is going to
open the documentation here for you. And what you can see here is that you can generate basic
charts, statistical charts, and scientific charts. And if you maybe click on, let's say, basic one, the
scatter plots, you just click on it, you will see a sample here of the codes and the sample of figure
that is generated. And again, one key differentiating feature of Plotly is that it generates
interactive charts. So if you hover on top of these data points, you will see the x and y labels.
And that's, of course, very different compared to the static plots that we're going to generate
using Seaborn.
[00:04:32.23] So let's move on to the code, and let's get started with our data visualization part.
So what I'm going to do first is I would like to install all the three different libraries. These are
Plotly, Seaborn, and Matplotlib. So to install these libraries, we're going to say exclamation mark
pip install plot. And once you install that library, now we need to import the library in our
current environment. That's what I'm doing here-- going to say import [Link] spx.
[00:05:02.59] Please note that we are going to use Plotly Express library, which is a built-in part
of the Plotly library. And think of it as more of a recommended starting point to create really
powerful visualizations with a very few simple lines of code. The learning curve for the original
Plotly or the full Plotly library is quite steep, and that's why we are going to just use Plotly
Express, which is going to make our life extremely easy to generate powerful plots using a few
simple lines of code.
[00:05:35.76] And then we're going to install a Seaborn library, and I'm going to import Seaborn
as an sns. It's going to be the alias name for Seaborn. And then I'm going to import Matplotlib,
and install it here as well. If you press Shift and Enter on your keyboard, that is going to install
all these different libraries for us and import them as well in our current environment so I can go
ahead and use them.
[00:06:03.83] What you see here on the left-hand side-- there is a number indicator that this cell
ran successfully. Everything looks good. What I'm going to do first is I'm going to focus our
analysis on predicting the percentage change in quarterly EPS. If you recall, we had two outputs
in our data. We had the percentage change in quarterly revenue, and we had our percentage
change in quarterly EPS, or earnings per share.
[00:06:30.73] So what I'm going to do first is I'm going to drop the change in quarterly revenue
column and, basically, I'm going to focus my analysis on the EPS first. And by the end of this
project, I'm going to shift to the quarterly revenue. Then I'm going to ask you to simply develop
machine learning and deep learning models to predict the quarterly revenue instead.
[00:06:54.67] Let's go ahead and drop that. I'm going to say apple_df.drop. Columns is going to
be percentage change in quarterly revenue. So if you press Shift and Enter on your keyboard, that
is going to drop our percentage change in quarterly revenue column. You press Shift-Enter-- here
we go-- first, you will see that here, right now, I have 201 columns instead of 202 that I had
before.
[00:07:19.23] And if you scroll to the right, you will see that, right now, I ended up with the
percentage change in quarterly EPS. This is simply the only output that I'm going to predict right
now. And as I mentioned, by the end of the notebook, we are going to predict the change in
quarterly revenue instead.
[00:07:38.30] The first visualization that I would like to show you is using Plotly Express library
to plot the histogram. So I'm going to say [Link], and I'm going to plot the histogram
specifically for the percentage change in quarterly EPS. And that would be the target output
column. So I'm going to plot that, and I multiply that times 100, and I'm also going to specify the
number of bins for my histogram to be equal to 30. And I'm going to update the layout to simply
set the background color to my plot to simply set it to white color.
[00:08:14.04] If you press Shift and Enter, it should take some time, and here we go. What you
will see is I can simply see a histogram for my percentage change in quarterly EPS. And you will
see that, basically here, for example, when the percentage change in quarterly EPS was between
0 and around 19.9%, I had 18 samples out of my data set that formed within that bin or within
that category. If you scroll to the right, you will see that the moment the percentage change was
between 20 and around 39.9%.
[00:08:51.97] In EPS, you will find that I have 11 samples out of my data-- the 11 quarters where
I got the data that falls within that range between 20 and around 39.9. You will see that as you go
above, let's say, 200, very few samples-- only one sample where I was able to get a percentage
change in quarterly EPS, kind of between 260 and 280, which is quite a large number. And you
can see, if you go to the left-hand side, you'll see between minus 240 and 260. The count was
only one-- I only got one sample that falls within that range.
[00:09:32.15] The next visualization that I would like to do is using Seaborn heat map, and what
I'm going to do first is I'm going to calculate the correlation matrix for my entire Pandas
dataframe. I'm going to grab apple_df, and then I'm going to say the .corr, which stands for
correlation, and that is going to calculate the correlation matrix for me. You press just Shift and
Enter in your keyboard-- here we go. Now I simply got the correlations between all the different
features that I have in my data.
[00:10:02.09] Well, it's a little bit hard to visualize because first, I have many samples. I have
192 columns here and rows. And you will also see that, basically, it's not color-coded. That's
why we are going to grab that correlation matrix and simply use Seaborn to plot a heat map for
us, so I can see all the colors for every different cell here that I have in my heat map.
[00:10:28.15] So what I'm going to do next is I would like to view the correlation coefficients for
every feature with the target output. You recall the target output with the percentage change in
quarterly EPS. That's what I'm doing here. I'm going to say correlation matrix, open square
bracket, another square brackets, and then specify percentage change and quarterly EPS, which is
my target output. You press Shift and Enter-- here we go. This is simply the correlation
coefficients for every feature that we have in my data with the output.
[00:11:00.11] If I take that Pandas dataframe and I simply sort it in a descending order-- that's
what I'm going to do next-- I'm going to grab that, simply, Pandas dataframe. Then I'm going to
say .sort_values.. I'm going to sort it based on the percentage change in quarterly EPS. If I set the
ascending to be equals to false, meaning that I would like to sort it in a descending order, and
only select the 10 samples, that is going to return back to me the top positive correlation samples.
[00:11:33.56] So if you press Shift and Enter-- here we go-- these are simply the top 10
positively-correlated features with the outputs. And you'll notice that, of course, the percentage
change in quarterly EPS, the target output, is perfectly correlated with itself. That's why we
obtained the perfect one.
[00:11:53.03] The next one is dividend payout ratio at around 0.27, and then, here, I have the net
debt, 0.27, and so on. I'm going to do the same print operation but I'm going to select the top 10
negatively-correlated features with the output. I'm going to grab, again, my Pandas dataframe,
but I'm going to say .sort_values with percentage change in quarter EPS. And then I'm going to
select the last 10 samples. If you press Shift and Enter-- here we go-- here, you will simply get
that the sales per share change for the last three quarters has minus 0.46 correlation with the
output with the percentage change in quarterly EPS. You will also see that there is return on
equity change, there is return on invested capital change, and so on.
[00:12:40.01] Next, what I'm going to do is I'm going to simply create one Pandas dataframe that
contains only these top 10 and least 10 features. So, in total, I'm going to have 20 features. I'm
just going to combine them together. So I'm going to say grab my apple_df. Here, I'm going to
pick the top negative correlation. You recall here, I selected or created two Pandas dataframes,
one of the top_positive_corr. Second one is top_negative_corr. I'm going to simply convert it to a
list. So I'm going to grab the index, convert it to a list-- same deal as well, I'm going to do it to
the top_positive_corr.
[00:13:20.06] So if you press Shift and Enter, simply what you get here is that you get a Pandas
dataframe that only contains the 20 columns of interest. These are kind of the key important ones
because they are the most correlated with the output. So I have the top 10 positively-correlated
and the top negatively-correlated features.
[00:13:41.16] So I'm going to go ahead and recalculate the correlation matrix for me. So I'm
going to say please grab the exact same data here that I obtained. I'm going to grab that. And I'm
going to say please take that data, which is the exact same one, apply or obtain the .corr. Obtain
the correlation matrix for Press shift and enter. Press Shift-Enter-- here we go. Here, I got the
correlation matrix.
[00:14:02.66] And now I'm going to use Seaborn heat map to simply plot a heat map for that
correlation matrix. So I'm going to say [Link]. I'm going to set the fixed size to be 15 by 9.
Then I'm going to use Seaborn-- if you recall sns was the alias name for Seaborn. Then I'm going
to see .heatmap, pass along my correlation matrix, and enable or activate the annotations. We're
going to set the annotations to be equal to true.
[00:14:30.25] If you press Shift and Enter right now, it might take some time for it to plot, but
here we go. What you see is simply here I have my output. This is my change in quarterly EPS,
and you would notice that there is, of course, with itself, I have perfect positive correlation of
one. You can also see the dividend payout ratio with the change in quarterly EPS around 0.27
and so on. So simply by just plotting the heat map you can visually see if I have a feature that has
strong positive correlation with the output or if it has strong negative correlation with the output.
And you can just indicate that using that color map that you can see here on the right-hand side.
[00:15:13.80] And that's it. That's simply all I have for this lesson. I hope you enjoyed it. In the
next lesson, we can have our practice opportunity. Please go ahead, give it a try. And then I'm
going to walk you through the solution afterwards. Best of luck, and I will see you in the next
lesson.