0% found this document useful (0 votes)
4 views50 pages

Data Science Session Overview and Tools

The session led by Rahul and Vignesh focuses on data science, specifically on the steps to solve business problems using data science, including problem definition, data collection, preparation, solution building, and evaluation. The discussion also covers the importance of understanding data types and formats, as well as the use of Python packages like Numpy and Pandas for data processing and feature engineering. Today's session aims to provide hands-on experience with these packages while recapping previous lessons and preparing for future topics in data visualization and advanced statistics.

Uploaded by

rudra_mitra2002
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views50 pages

Data Science Session Overview and Tools

The session led by Rahul and Vignesh focuses on data science, specifically on the steps to solve business problems using data science, including problem definition, data collection, preparation, solution building, and evaluation. The discussion also covers the importance of understanding data types and formats, as well as the use of Python packages like Numpy and Pandas for data processing and feature engineering. Today's session aims to provide hands-on experience with these packages while recapping previous lessons and preparing for future topics in data visualization and advanced statistics.

Uploaded by

rudra_mitra2002
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

Hello, everyone! Welcome to today's session.

My name is Rahul, and I'm an associate


manager for today.
So before we go ahead with the session. Please give me a quick confirmation. If my
boss is loud and clear.
you can confirm it in the chat box.
Thank you. Thank you, folks, for your confirmation. So before we go ahead, please
let me do a quick audio checkup with our trainer. Hi Vignesh.
Vigneshwar Veluchamy
Vigneshwar Veluchamy
00:31:16
Yeah.
Hi, Rahul, good evening. Yeah. Hello, Dunkley. From Europe. Yeah.
faculty 58
faculty 58
00:31:20
Yeah, yeah, you are also loud and clear. I made you the post. You can start in the
session.
Thank you.
Vigneshwar Veluchamy
Vigneshwar Veluchamy
00:31:33
Yes. Hi! All a very good evening to everyone. We'll give another 2 more minutes for
the people to join, and then we'll start the session. Thank you.
Okay, we'll get started for the day once again. A very good evening to everyone,
just a quick confirmation if my voice is loud and clear for you, you can give me a
yes in the chat window for that.
So subhashish is asking for yesterday's class notepad so subhashish. I have
uploaded all the artifacts in the session materials. Section
so I hope everyone is able to access that and
uploaded materials section. Can I get a quick confirmation on that?
If not, I can share my screen, and I can let you know where we can access that.
So let me share my screen
so subhashish, saying that the notepad was not there.
but ideally, when you go to the session materials we'll be having a Zip folder
created day? 5.
Can someone confirm if the notepad files was there or not in that? Because I
remember keeping it? Let me also do a quick check in my system.
Downloads. Defi.
Yeah, we do have it right? Python packages for data science, one HTML document,
which is nothing but the the notebook file.
and the 3 notepad files will be like the text documents.
So vashish. Please do check it. So we'll have 3
note text files in it, and one HTML file, so that HTML file is the notebook.
Jupiter notebook that we practiced yesterday, and the 3 text files would be the
notepad files.
Okay? If not, I can share that file here also, just in case if you are not able to
download.
I'll go ahead and upload that here in the chat
you can take it from here as well. So just keep this for your reference. Whatever
topics that has been discussed yesterday will be available in this Zip folder.
Okay.
cool. Let's get started with today's session before we start. Just a quick recap of
what we did yesterday. So yesterday. It was all about trying to get ourselves
introduced to. What is data science, a proper introduction to data science. So
trying to understand what is data, science. What are the applications of it? And
what is the process? The main takeaway from yesterday's session was.
what are the steps that needs to be followed
when we are trying to solve a business problem using data science. So step number
one, we define the problem step number 2, we collect the
data set step number 3, we prepare the data. That is nothing but doing the
exploratory data analysis step number 4, we build the solution
and step number 5. We evaluate it and step number 6, we are going to be
deploying it so only after we
make sure that all the evaluation metrics are satisfied, at least it should not be
like always, like 1995. But whatever we could get the best
for the data that we have and the algorithms that we implemented, we'll be
evaluating for it. And if we feel that that's the best one, we go for the final
deployment, that is nothing but making sure that everyone uses the solution that we
prepared for the
requirement.
Right? So that was the steps that needs to be followed. And then after that, we
also saw some concepts on what are the different types of data?
Right? So by format, we'll be trying to classify this as structured, unstructured,
and semi-structured. So that was the 1st part. The second one is by data types. We
can
broadly classify into 2 categories, quantitative and qualitative. In quantitative.
We'll have discrete continuous in qualitative, we'll have binary, nominal and
ordinal.
So why are we actually focusing? This is that whenever a data set is given to you
for a requirement
do not directly jump into the solution.
You just need to make sure that you understand the format of the data, whether it's
a structured, unstructured, or semi-structured, and then understand what are the
types of variables that has been given to you? How many numerical, how many
categorical.
and then we go forward to see how to clean up the data pre-process, the data. And
then only you should start building your solution.
Okay, so there are a couple of steps that needs to be done before we build the
actual solution.
So that was the whole context. 1, st 1st part of the session was focused on that.
And then we started talking about, what are all the tools that are going to be
helping us to
process those data for the requirement? Right? So we'll be leveraging python
programming language here.
So we'll be talking about what packages are going to be used for the individual
steps inside the requirement. Right? So whenever we want to use a numerical
computation. So whenever you want to calculate something or trying to do a
mathematical computation.
you go for a package called Numpy.
So in numpy. What is the concept behind numpy is that whatever be the data.
it will be converted into an n-dimensional array nd array, we call it.
and when it's converted into nd array you will be able to apply the methods. What
is there as a part of the numpy package
and get the results?
So that is going to be so. We saw some examples on numpy as well, and we also just
listed out the packages right? So instead of trying to learn every package
individually, if you get to know what that package is actually for. So if you are
going to do something on data processing, there will be some couple of packages for
that. And if you're going to be doing some data visualization, there will be some
packages for that. So like that if you try to bucket it under the respective
you know requirements.
The the packages count, will not be that much overwhelming for you, because if you
just take it individually. There are close to 4 50 packages in python alone for the
data. Science related activities. Okay, so we should not like, try to see that it's
4 50 packages. Rather, it will be across this 5 categories
where it's across computations, data processing data, visualization and feature
engineering and model building part, we'll talk about model building and machine
learning module. So when you try to
categorize it, it'll be easier for us to learn about that package. So that's why we
just wanted to list it out.
So today we'll be more focusing on the packages. We'll start the discussion more on
the hands on where?
how each package is? What is the capability of each package in order to make sure
that we are properly trying to process the data for the requirement.
Okay?
So we had listed out numpy pandas.
And for the visualization we have something called matplot 11 c. Bond.
And when we try to take the steps forward. You'll also be hearing about a term
called feature engineering.
So just to give you a quick idea of what is feature engineering is that whatever
data you take. So you collected the data.
let's say you have got a data and excel spreadsheet with 100 columns for the
requirement.
But the question will be like, Do we need all the 100 columns to process for the
solution?
Sometimes it will be a yes, but most of the time it will be a no, because out of
the 100 columns only certain columns only will be playing a very vital role in
trying to solve for the requirement.
Right. So how do you identify those columns? So that is what feature engineering
will be all about. So trying to take only what is needed, and then trying to move
forward to the solution right? So that process the techniques that are going to be
implemented there will be called as feature engineering.
So we'll have a package which we will try to apply it on the data
where it will try to tell us which column is important for the requirement.
and take only those columns and prepare the solution rather than taking all the
data points that is provided for you.
So that is what we'll be trying to learn in feature. Engineering part Scikit learn
will be a package which will be helping us to do that.
Okay, so in today's session, we'll be more focusing on trying to give a proper
closure for the the numpy and the pandas thing. I know it's already been covered
for you, but I wanted to keep today's session for a proper closure on the numpy and
pandas capability. So that after today's session, whatever things that has been
given on numpy and pandas, you should be able to give it a try.
What are the capabilities you need to have an understanding? So those are some of
the things what I would like to have as a takeaway for today's session.
Okay? I hope, like everyone is there with me, with the agenda for today's session.
So we'll be giving. We'll be trying to do both numpy and pandas with exercises and
some additional capabilities. I'll try to explain it in today's session.
and with that a proper closure on the numpy pandas part. And then from next week
we'll start talking on the data visualization and and the feature engineering part
as we progress forward.
Okay, any questions, guys, any questions related to the topic that was discussed
yesterday?
I hope everyone is clear with the agenda. What we have for today's session.
Can I get a confirmation in the chat for that?
So how about the theory of it? So, as I speak about the capability part? Right,
Vijay.
so we won't be having much theory when we talk about the packages, because packages
of python will be more about application where you apply pandas. And how is it
trying to give that answer for us. Right? So that will be the theory part. But for
today's thing, I don't think so. We'll have much of theory. But maybe from next
week, right? When we start talking about that statistics, advanced statistics,
hypothesis, testing. And all right. So that is where we'll again pick up more of
theory.
Okay? But as of today.
I think yesterday was more of more of theory. That's what I could find, because I
was trying to talk more today, it will be more of hands on where we take some
packages. We apply it on the data and see what kind of results we get for it.
Okay, cool. So now let's get started. So yesterday we we started with some
idea on the numpy part, right? So we 1st created a a list, and
it was not able to do. We were not able to do any mathematical computations on
that. So then we imported numpy. We converted that into an nd array.
and we were trying to do all sorts of mathematical computations on that, so you can
add it. You can subtract it, multiply it.
and you can also try to do the inbuilt
array functions. You will call this sas universal array functions, taking the mean,
taking the the variance or standard deviations, and all those things right will
start coming into picture.
So let's try to work on the same file what we did yesterday. Let me just
run all the cells.
Okay? So now, as we speak about numpy, the reason why people are preferring numpy
is that the computational time? Right? So even when it is a list. Yesterday we were
talking about that arithmetic operation is not at all possible in a native
collection type.
So when I say native collection type. It is your list, Tuples or Dictionary. Okay,
but there is a possibility to do that. But the only thing is we need to iterate. I
hope everyone would have learned about iterations right for loop
while loop in your programming fundamentals. Yeah? So when you apply that iteration
condition, it will, you will still be able to do the mathematical operations on
native collection type. But you need to take it one by one.
So the computational time taken will be very high.
and it will also be a little complex right? Because you need to take one element
from the list, one and one element from List 2. Then we add it, and then we come up
with the final result. So to avoid all those what people python developers will
always be like. They always have this concept of lazy programming. Okay, so they
tend to make us lazy because all the things will be integrated in the package. So
all you need to do is to call out that package, apply the method and get the
results for that.
Okay, so here in this case, if I wanted to take the average of the complete matrix.
So they integrated everything into this method called dot mean.
and when I call out the numpy package, it automatically computes the mean value for
me
the same thing. If I wanted to calculate the
the variance right yesterday we were talking about, I was asking you to do all the
basic walkthrough on the mathematical fundamentals that we learned right? So when
we talk about variance, how do we do that so directly? There is an inbuilt method
called Np. Dot, where and when I pass on this new variable inside it.
So you'll be getting this value called 666 dot. 6. So what does this mean? If I am?
If I may ask this question.
what do you mean by a variance of the data?
Yesterday we had the discussion on what is variance? Right?
So you got a value here. So what does this value actually signify for the variable
that we have
difference from the mean?
Yes. So how a data is spread from the mean?
Okay?
So since you have said this answer, now, the next question will be like, we also
have something called standard deviation.
Right? So we also have something called standard deviation. We have something
called variance.
Now then, what is standard deviation? And what is variance?
That will be my next question.
So square of standard deviation is variance. Okay?
But what does it signify? That is what I am trying to understand. Okay, so you are
trying to compute standard deviation, you are trying to compute variance.
So you're saying that standard deviation, it is going to be the distance of every
data point from the mean
that is standard deviation.
Then what do you call as variance when you square up all the distances? What is
going to be there from the mean, you will be trying to arrive at this number called
variance. Right? So variance is nothing. But it is going to be the just to keep it
very simple for everyone to understand it is the spread of the data.
Okay. So if I have 2 examples here.
let me take a notepad for this.
So let's take. Let's take an example
where you will be having 2 scenarios.
Okay? So where the data will be like this.
this, let's call this as A,
and there is going to be another scenario where the data is going to be like this.
this is going to be B, okay.
So now, out of A and B,
where do you think the variance will be higher?
Is it going to be higher in a, or is it going to be higher in B Marion spata?
Is it going to be higher in a, or is it going to be higher in B variance is higher
in A or variance is higher in B,
so like I said from the definition, variance means it is the spread of the data,
right? So spread of the data is like something where the ranges are there. So you
take the range from the both the tails.
Okay, so you take that this is going to be the variance here of my
scenario. A. And this is going to be the variance where it is going to be in
scenario. B.
So what we'll try to say is that B is going to be having higher variance than A,
because the range of the data, what we have taken is little higher.
So when I compute the variance for A and compute the variance for B,
the variance, what we have in B will be higher.
I hope everyone is able to understand the concept of variance. Right? Why is this
important?
The importance of variance in a data? So do you think a high variance data is good
or a low variance data is good.
If I wanted to ask this question right? For a data science problem.
do you think a data with high variance is good or low, variance is good.
high variance, low variance.
I wanted to see the answers, let me go back to the chat.
Low variance is good.
High variance is good. Low variance is good. Okay? I see
all the ants is getting mixed up here.
Low variance. Okay, think about what I what I talked about variance here.
what do you mean by variance? It's the spread of the data, right?
So when I say, spread of the data.
more data we give to the model. To learn better will be the results. Right? So if I
have a scenario
where I feed only a certain section of data.
right? Because the reason why I'm telling you let me be very clear, because most of
you are saying this as low.
But what happens? Let's take a scenario where I have a data where I'm just
specifying only this particular range. Okay? And there is a data
where I'm going to be specifying this range.
So which data will be good for building a model to learn all the scenarios
perfectly
is that whenever we are going to be having a low variance. We told that this is a
low variance, right? But what we are actually doing here, we are feeding only a.
a kind of a biased data there only whatever is there within that particular section
we are feeding that so a low variance data will lead to a high biased scenario. We
know this concept called biasness. Right? So when you have, when you are feeding a
data with low variance, you are actually feeding only a certain scenarios of data
to it. But whereas when you feed a data with high variance, it is covering a larger
scenarios compared to your scenario, a right?
So that is why we tend to prefer a data with high variance
for your training requirements. Whenever you want to train a model or build a
solution in data, science, right? Your data should have high variance because it is
covering different scenarios.
What we have collected in that requirement, and that will be a good learning
material for the model to understand the ins and outs of the process.
So am I making myself clear why we need a high variance data for building a
machine, a data science solution here.
right? And that is the reason, like why we need to make sure that at least we, we
might not always get a high variance data.
because it's very difficult to cover every aspects in the process, right? But the
ideal scenario is, if it is a high variant data.
the model. What we are going to prepare will be unbiased. It will not be biased for
a particular scenario. What we are going to be building it for
is everyone there. With me, high variance, low variance scenario.
Yes.
So
as the variance gets higher, what we try to have in the problem is as the high
variance gets higher, high variance
will lead to.
If I just want to write this out, high variance will lead to low biasness.
but as we increase the variance very much higher
as we increase the variance very much higher. What happens is that the randomness
in your data also will increase
right? You won't be able to detect a pattern
if we increase the variance very high.
And apparently what is also going to happen. The randomness in your data is going
to happen. So when the randomness is increasing, you won't be able to detect a
particular pattern in your data, right? So that is also going to be a problem. So
when you deep dive into the machine learning module right that is going to be,
you'll be learning something called as a bias variance trade off.
You'll be hearing this term very frequently bias variance trade-off.
So we need to make a trade-off
at one place, that the biasness should be low
and the variance should be higher.
That type of a data should be taken for building the machine learning model.
Okay, so just to give you a heads up right? These terminologies will be something.
What you'll be frequently hearing in your next set of modules. So variance is
nothing. But it's the spread of the data.
whether it's a high spread or a low spread, so there is no unit for it.
Only when you compare between 2 things you will know whether it is higher or lower,
so I compute the variance for a separately.
and I compute the variance for B separately. So if my variance of B is higher, then
I tend to say that B has a higher variance compared to A,
so there is no specific unit for variance.
So that is going to be the takeaway what you would like to know for variance. But
variance. The definition is the measure of the
spread of the data. So how spread your data is, how much values it is covering
that is going to be called as your variance part.
Am I making myself clear guys with the topic of what is variance here.
Yes.
So with that, how do you find that variance in your data?
Let me show you that.
So this means for single data, there will be less so, depending on what we are
going to be using right?
Always make sure that it is all about the scenario again
for a machine learning scenario. When you're trying to make a system to learn the
patterns of your data.
If you are trying to make a system to learn the patterns of your data.
we need to have a data with high variance.
Okay?
So that's going to be there. We are increasing the boundaries of the data, the
scenarios, right? So that is what we are trying to do. And we increase the variance
pass coverage. Yes, that's right.
Okay, so this value 666.
Okay, it means that the value is what we have has a good spread.
if one, if new, variable one is there.
and I also do the variance on new variable. 2,
and new variable has 666 new, variable. One has 666, and, let's say, new, variable.
2 has 400.
So new, variable, one has more variance than new, variable? 2. So that is the
conclusion. What we'll try to make.
So when it is low.
what does it mean? It means that the spread is not there. It's not as good as what
we expected. So maybe we can increase the spread and look at the variance, how it
is increasing.
Okay?
So that is on the variance part. And when we look at the standard deviation, it is
nothing. But it's the square root of this value. Right. So if I do, an Np dot
standard deviation, Std is the inbuilt function.
I can take this new where one. So this will be the square root of your variance.
That's nothing but your standard deviation.
So all these measures, whatever we are calculating.
it is all going to be talking about the data properties.
whether the data spread is high or low, or how much it is varying from the mean.
So all these statistical measures will give me some information on that.
Okay? So it doesn't mean only for the statistics part so numpy like I said, it is
going to be applicable for
your arithmetic and mathematical engineering related computations or logarithmic
calculations. So if I wanted to know the log of the matrix, the log of the
individual elements in my matrix right, I can directly go for Np dot log.
and I can go for passing that variable here.
So within a fraction of time.
you will be getting the logarithmic values of this matrix
of the 1020, 30, 40, we had a 3 cross 3 matrix, right? So it will be able to
compute the logarithmic values for it.
So like that all these functions are inbuilt within numpy. First, st
okay, same thing. If you wanted to take the square root. So these are some of the
functions that we regularly use.
So if I just wanted to take Np. Dot square root, it's called Sqrt.
And you pass on this variable inside it.
So you'll now be able to see the square root of the individual elements of the
matrix that we have.
So whatever arithmetic, mathematical operations you name it. So that is all going
to be available like an inbuilt function in numpy for us.
Okay.
so till we compare the 666 value to something, how can we find? Yeah, we need to
compare and give that idea here, Shalab. So we should not stop at one portion of
data.
Okay, you need to take multiple samples, and in which sample we have the maximum
variance, right?
So that is going to be the best sample to be taken for the requirement.
so people may stop at one and go forward just for the sake of time. They might just
take a random sample, and they go forward with that requirement.
But ideally, if you are a good data scientist or a machine learning engineer,
right? You need to take multiple samples, look at the variance and then carry
forward to the next step.
So maximum variance.
No hallucination is a different concept. Again, don't try to bring in the
generative. I you know engineering concepts into this part. Okay, so the thing here
is.
are we giving the data points? What is there in the process or not for the machine
to learn.
Okay, because variance, we are not giving a completely new data to it.
Whatever is there within the range? We are just trying to give all the scenarios in
that range?
Are we giving all the scenarios in that range, or
just trying to keep only a certain portion of the data and feed it to the system.
So when you are trying to give, let's not take it from the maximum variance part,
take it from the minimum variance. But if you feed a data with minimum variance.
You are going to make the system more biased for only those section of data.
Let's, I'll tell you with an example. Let's say you are trying to build a machine
learning model
to predict whether a person will get a loan or not from a bank.
Okay, the data, what we need to feed for the system is, it should be
an equal proportion of
your people who have got the loan from the bank, and people who have not got the
loan from the bank. It should be like a good variance between people who have got
the loan and not got the loan. Both the data points should be given to it. But if
you feed a data with a low variance only on certain portions only, if people who
have not got the loan.
and you feed it to the system right? Then what happens? Your machine model will be
more biased in always predicting that the person will not get the loan.
and that's that's that type. One type, 2 error will increase over there.
So that is the reason, like why we need to make sure that we have a good amount of
variance to take it forward.
Okay.
I hope, like everyone is there with me. Why, this variance becomes an important
here factor here, a very, very important factor, especially when it comes to the
machine learning model. Right? It will be like trying to make sure that we get the
best model with a good variance
at the end of the day.
Okay.
okay, Nive is asking, can we round off the values to 3 decimal values. Yes, we can
do that so ideally, there is a separate inbuilt method for doing the rounding off.
So in numpy you will be having this method called Np dot round.
Okay, so this Np dot round. What you can do is that you can apply this method on
top of this. Let's say that you're storing this in a variable.
I'll store this in variable called result. Let's take whatever square root value
you have got. I'm storing it in this variable called result.
So I am executing that now, if I wanted to round off that result values to 3
decimal values.
I'll call out this method called Np. Dot round
and bring in this variable, so it'll it'll ask me 2 parameters. One is, what should
I round off
so I need to round off this variable called result. So that's my 1st parameter, and
then the second one is to how many places of decimals should I
round it off?
Okay, but I know this.
But there is a way by which you can also know
what are all the parameters that particular function is expecting.
So how do you know, that is, especially when you use Jupyter notebook. That's 1
good thing. Why we need to use Jupiter notebook is that
there are some helping materials already available to us.
Okay, so when you type out that method, Np, dot round you, keep the cursor open and
close the round brackets. Keep the cursor in between your brackets and do a shift
and the tab key in your keyboard. That's a keyboard shortcut.
Keep the cursor in between your brackets and do a shift and a tab key, so it will
tell you the method signature.
what is needed for that particular method to execute.
So Np dot round, it needs a variable called a is nothing but that array, n
dimensional array. What we have, and the decimals will be to how many decimals, so
by default it will round it off to the whole number. Decimal value will be rounded
off to 0, and this output like, whether you want it in a particular output format.
This is an optional parameter. No need to worry about this, but these 2 will be the
most important one.
So you need to provide a array, and the decimals to which how it has to be rounded
off.
So what I'll do I'll try to give the variable as result.
This is what I need to round it off.
and I need to round it off to decimal value.
Let's say 2 for an example.
Okay, so and I go ahead and execute it.
Now you'll be able to see that whatever I got in my Np. Dot square root right
separately.
I'll just try to show you that before rounding off. How was the result?
So before rounding off.
you will be having like 7 decimals, you can just round it off using this method
called Np, dot route.
Yeah.
I'm just trying to break it down and show Saudjan. Yeah, you can always apply that
directly on the Np dot Squirotwell, instead of result. Storing this in a variable
right, you can directly take this function, replace it here, and it will still work
for you.
Okay, I know that people have already got some experience in coding, but I'm also
talking about the people who are coming in for the 1st time, so it's always good to
break it down and then combine it so that you'll be able to understand it much
better.
Okay, so and I'm also seeing a question like, Why do we need standard deviation.
So standard deviation is how far it is from the mean value. Shalab. So I'll tell
you an example. Okay, where? So we can take Shalab's question for an example,
right? So where do we actually need this standard deviation for the data
is that we need to know whether the data follows a normal distribution pattern or
not.
Okay. So whenever you are trying to build a machine learning model, most of the
machine learning model will expect the data to follow a normal distribution
pattern, I hope, like, everyone is comfortable with the term called normal
distribution.
Yeah. So what do you mean by normal distribution? When do you say that the data is
normally distributed?
If I may ask the question, since you completed your fundamentals
when your mean value is equal to one? Is it always mean
or anything else? All the central measures of your tendencies? Right will be same,
your mean Median mode.
Everything will be the same right
collide at 1 point. That's right. And what is the other property?
So if I just draw that normal distribution right? So the 1st thing is that it will
be a bell-shaped curve.
Okay? And it is going to be symmetrical.
So if I split the chart into 2,
you'll be having equal halves on both these sites.
Yes, and your mean Median mode will be at the center here.
right? And the other one is your standard. Deviation is equal to what
the standard deviation of your value will be equal to one.
So when I try to compute the distance of the data points from the mean, you should
arrive at
the standard deviation is one. So when your standard deviation is not one.
what does it mean? So it is not going to exhibit a proper normal distribution.
it will tend to execute something like a skewness here.
either on the left hand side or on the right hand side.
So this is the place where the standard deviation will not be equal to one. So when
we find out the standard deviation.
we'll actually come to know whether there is skewness in the data or not. So if
there is skewness, there are outliers present. So we need to be very careful and
cautious in processing that data.
So for that purpose, we actually compute the standard deviation and use it for the
requirement.
I hope, like, I'm making myself clear all the statistical measures that we are
going to be computing right will try to tell some information about the data that
we are actually going to be using for the requirement.
Am I making myself clear. Let let it so. Variance talks about the spread right
standard. Deviation talks about this normal distribution, whether it follows the
normal distribution or not.
Right? And we'll also have something called the percentage.
25th percentile, 75th percentile. So everything talks about one or the other way of
the property that the data is going to exhibit for us.
So that is the reason we need to be very much comfortable with the statistical
terminologies that we have.
Okay, am I making myself clear guys on? How are we going to use these statistical
measures?
Yep, okay.
wonderful. So now, going on to the the next thing. So here
all these things, whatever we are executing right? It's all what we call as a
universal
array function, right? In fact, like there are also other capabilities in in numpy.
where you can try to not always make one variable as n dimensional array. You can
also generate n dimensional arrays.
Have you given a practice on generating n-dimensional arrays rather than converting
one list into another n-dimensional array.
Did you get a chance to generate n-dimensional arrays in your exercises with numpy.
Can I get a confirmation on that? If you have done that exercise, maybe the
commands like Np dot
a range function, a range. I hope everyone has given a try on that. So let's do a
quick
thing, because the reason why this is needed is that this is going to be also very
much helpful. Whenever you want to quickly practice something.
you can go ahead and try out these inbuilt functions. Right? So Np, Dot, arrange
what it will do.
It will try to generate the n dimensional array format
when you set the starting and the ending values. So when I say, Np, dot a range 0
comma 20.
So starting value, 0
ending value will be 20, but 20 will not be considered, since it's a range
function, right? So the values will be generated from 0 to 19.
So if I just go ahead and execute it, you'll get the values from 0 to 90.
So that is what is going to happen automatically. An n dimensional array will be
generated for you when we use this function called
a range. So you need to give a starting step, ending step, and by default
the the difference is going to be one unit. So with one units, you will be able to
see the
n-dimensional array generator.
Okay.
Shalab's question is all this variance standard deviation? Yes, it is going to be
used. We we need to understand about the data and then only go for the processing.
Right? Shalob.
So all this variance standard deviation will try to tell something about our data
and based on that, we need to pick the strategy
and go for the processing. We should not blindly go with the standard steps
according to what the variance standard deviation. You will pick the strategy and
go for it.
Okay.
Saudjani's question is, this is a 1 dimensional array how to create 2. Yeah, we'll
talk about that. We'll come to that. Saudjan. Yeah, very shortly.
but I hope, like everyone, is able to remember this command, a range one
where it's trying to create a set of variables for us.
So by default. It is going to be with one.
But let's say I don't want it with one units. I want it with a defined difference
between the units. So I want it with 4 difference. So 0 4, 8, 12 like that. So if
that is going to be my requirement.
I can add my 3rd parameter
np, dot a range. I give my starting step. I give my ending step. And what is the
step size?
What is going to be the difference between the 2 units right if I mention it as 4.
So what is going to happen in this particular command is.
I've given 0 to 20. But I'm saying that the step is 4, so how it will generate this
it will, it will be starting with 0.
Then it goes to 4, then 8, then 12, then 16,
then, after 16. Ideally, it should go to 20, but in the A range 20 will not be
considered right. So that is why, it stops at 16 and gives you the value till 16.
Okay, so this is where you give it a step and try to generate your arrays.
Yes, so I hope, like everyone, is comfortable with the arrangement.
So this is a place where you can generate, arrays, and play around with that.
Now there is also a very similar comment, what is going to be
converging with this? A range function?
But that command will be called as Lin space.
Have you given a try with? A lint space command, lint space.
No. Yes. I see both of it coming. Okay? No problem. So yeah. So lint space. What is
the difference between
lint space and a range come in is that
in a range the last value will not be considered.
Okay. But whereas in your lymph space.
the last value will be considered.
So that's the ideal difference. Okay? So if I go ahead and give Np dot lint space
is the command.
Okay, and you're going to be giving the same 0 to 10 for an example.
Now, what is the difference between a range and and Lin? Space
is that I always take this analogy. So Link, space is like, just think of
having a 10 cm scale or a ruler with you. So where 0 is at one end.
10 is at the other end. Just imagine that you're holding a ruler, so how that 0 to
10 is equally divided
with the steps. Right? That is what lint space is all about. So by default.
if you look at the method signature, I told you how we can look at the method
signature right? So keep the cursor within the brackets and do a shift and a tab
key.
so it will give you the method signature. You see that there is a start value.
There is a stop value.
and you will be seeing that there is a parameter called Nu MS. 50.
So what it is trying to do is it is trying to equally divide
the values from 0 to 10 into 50 equal divisions.
Just imagine that small small lines. What you see on the ruler. Right? Zeros at one
end.
10 is at the other end, and you see 50 small lines between them. So what does that
individual lines represent?
That is what is going to be the output, what you can expect from lint space. If I
go ahead and execute this.
so you'll be seeing a value of 50 divisions between 0 to 10.
So in this case the value 10 will also be included in your output.
I hope, like everyone is there with me in understanding what is the output you can
expect with lint space.
If you want to include your final value.
you can go ahead and mention that with the link space and generate that one.
So by default, it generates 50.
So if you want to generate only 5, you can
add that whatever you would like to have.
I just say that between 0 to 10 I want only 5 values, 5 equally split values
between 0 to 10, so I'll give my step s. 5.
So now this 0 to 10 will be divided into 5 equal halves.
So if I go ahead and execute this. Now you see 0
2.5 5 7.5 and 10. So you see 5 divisions between 0 to 10.
The only difference is that the final value? Whatever you give us the final step
size, right? That is also going to be included
in this n-dimensional array.
I hope, like everyone is there with me, with the Lin Space Command.
Yes, wonderful. So now, so these are all going to be certain commands where you can
generate your arrays.
and you can use it for some requirements. So there was a there was a question.
where can I? Can I go ahead? And
so here in this case. Okay, even before that, right? We talked about generating
arrays between a certain set of values.
So if I give it 0 to 10, it will generate from 0 to 10. If I give it from 0 to 20,
it will try to generate between 0 to 20. But what if I want to make a random set of
variables
every time and execute it right.
It should be giving me different values
sometimes for iterations. You might need that random variables to be generated to
try it out with different testing data. Right? So how do we do that?
So in numpy, we'll have a we'll have a method called Np
dot random. So random is like a a sub package inside numpy.
And inside that random, you will have multiple inbuilt methods.
Okay, so how will the hierarchy in your python project will look. Is that so?
First, st you will have a
so first, st you'll have a package.
Okay? And inside the package you will have lots of modules inside it.
and inside the modules you will have lot of methods.
and inside the methods only you will be like having all the instructions being
configured.
Okay, so from the bottom to top methods, a collection of methods will become
modules. A collection of module is going to become a package. So here in our case.
Nampa, is the package.
So in Nampai
we have a module called Random. So inside the random we are going to pick one
method. So np.
dot random dot, you'll be having one method called rant.
Okay. So np, dot random dot trend. If I go ahead and execute this
I can mention. How many variables do I want to have in my output?
So if I say 5 here, np, dot random, dot Rand
and I mentioned. 5 inside this.
What is going to happen?
It is going to be generating me a a set of 5 random variables in my output.
so it will be in the E powers. That's fine.
but every time when I run it right so now it is showing 1.5 5 8.7 3 6.6 4.
Next time, when I execute it.
you'll be seeing a different set of values, 0 point 9 5 0 point 1 3 0 point 5 8 0
point 7.
I hope, like everyone is able to see that difference when you apply this random dot
Rand function.
So this is going to be something like where you want to generate a random variables
in your requirement.
You will be like trying to use this Np, dot random dot, Rand function.
Yeah, you can go for it. Yeah, I'll say I'll tell you that how to set the random
seed and all.
But before that, let's try out all the comments [Link]
So it is going to be wearing. So what does this rant function will do?
Ran function will actually be trying to generate. I hope, like everyone is before I
talk about it. Right? I hope, like everyone is comfortable with the the types of
distribution that is there for a data.
What are the different types of distribution that you can expect in a data? One is
normal distribution.
What are the other distribution patterns that you can have for a data.
Can I get some answers for that?
So you have something called as a uniform distribution? Yes, you have something
called as a Bernoulli distribution.
normal distribution, poison distribution. I hope everyone is
comfortable to be aware of this terminologies.
You might have heard this in your fundamentals, right?
The different types of distribution patterns. Yes, no.
I know that only few people are answering, but I just wanted to hear it from
others.
Are you got? Are you all comfortable with these terms that we have not yet covered.
Okay, but all graphs look the same.
No, ideally, it's not. There are properties. What we are going to be having for
each of it.
Okay? So no problem. Anyways, we are going to talk about in the advanced statistics
module, right? That is the next lesson after we cover this numpy pandas and all,
I'll try to give you some idea on what are the
the different distribution patterns that we have.
So there is a pattern just to give you an idea for whatever is the pattern? Right?
So there is a pattern which is going to be looking like this.
So just now, what we saw this. So this is going to be a distribution pattern. So we
call this as
normal distribution pattern
right? And there is also going to be a pattern where the data will be having a
pattern something like this
between 0 to one.
So this is going to be an example, where it is going to be called as a uniform
distribution pattern.
Okay, so this is a normal distribution. This is a uniform distribution like that.
You'll be having different types of distributions like, you'll have
an S shaped curve something like this.
sigmoid distribution. So all those things, every
shape. What you have for a data is going to be a distribution inside the data. And
there is depending on what size of the height we have in the peak, right? So
depending on that, you will also be having. This is a sine wave
sinusoidal distribution. We call it something like this.
right? So all these things are different patterns in which we can see the data
being distributed for us in a scenario. So for everything, there is going to be
some properties associated with that.
Okay? So I'll maybe, like, once we start looking into the hands under, these are
all again. Theory part
what I would like you to. Maybe like, go through. I can share the reference
materials, but I can also talk about it. But if I start talking only about this in
the session, right? It might take like the complete session. It might go like a
mathematics session for us.
but for us, the major takeaways that how do you interpret this distribution and use
it for the requirement.
Okay? So maybe, like, I'll try to see how much ever I can explain during my
session. But what I would like you to do is that it's not going to immediately
happen tomorrow.
But I think this is going to be there
2 or 3 weeks from now. Towards the end of the this particular model. Only we'll
talk about this distributions in detail, so you will be having an ample time of 2
weeks
to go for this to understand what are the types of distribution?
What are the properties of the distribution? Just list it out, keep it handy like a
note.
and when you come back to that discussion right.
we'll quickly walk through what are the different distributions and understand the
properties of it.
Will you be able to do that. I'm not asking you to do it tomorrow, but just take
the time for one week
and look at what are the different distribution patterns that are there. Write down
the properties of it and keep it handy like a notes for you.
Okay?
Because even without knowing this right, you can still go forward and do data
syncs.
Okay, I always keep. I always try to give this kind of an explanation for everyone.
The way how data science modules are now, solutions are now configured is that you
need not even worry about what is the mathematics that is happening at the back end
of it. You can just go ahead and import the package, apply it on the data, get the
solution, and
give the output to whoever is asking it.
Okay, but there is a difference between
a data scientist and a good data scientist.
Okay, so a data scientist is a person who applies everything whatever is being
taught and gives the result
a good data scientist will get into the depth of it. Understand why the results are
given like that, and then explain that solution to the stakeholders and then give
it.
Okay. Now, it is our time to know whether we want to be just a data scientist or a
good data scientist.
So if you want to be a good data scientist.
then it is it is going to be very essential to know the mathematics foundational
knowledge for that.
Okay? And once you get that
be. And the reason again why this is being iterated is that
you can be sustainable in this field for a very long time, because nowadays you
might be seeing
co-pilots and and lot of AI solutions trying to do all these data science
activities in a in a very short span of time.
Right? All the complex operations are done in a very short span of time. So
whatever, if you just go ahead and just apply it on the data and get the results.
then you will no longer be adding value to your team.
But when you explain, why is it? Why is that happening and then give your results.
You'll be adding a lot of value to your team. So, being sustainable in this field
is also one factor.
why we need to know all these foundational knowledges at the back end of it.
Okay, so that is the reason, like, please spend some effort on trying to
make the foundational aspects also very clear mathematics, statistics, whatever we
discussed yesterday in the prerequisites, right statistics, linear algebra,
probability and distributions, probability and distributions will come together as
single subject.
So these 3 concepts are going to be very important for us to know the essentials,
the basics of any data science algorithm.
Okay.
yes. So now, coming back to our output here. So np, dot random dot Rand, right?
What it is going to do is it is going to generate a set of variables
which is going to follow a uniform distribution for me. So every time when I run
it, the values are going to be between 0 to one. You see that every time the moment
I run it, every time you will be always seeing that the values are going to be
between 0 to one. So when I use this Rand function.
it is going to be generating a variables for me which will follow a uniform
distribution
from the values 0 to one.
Am I making myself clear guys, what this rant function is doing for me?
It will generate values that follows uniform distribution in my data.
Yes, no.
The same function. I also have a very similar function called Np, dot random dot
rant end. No.
Okay. So what do you mean by Np, dot random dot Rand, N. Now
again, the name itself will give you a hint. So this is going to be generating some
values
which will follow normal distribution for me.
So if I give Rand, N. Of 5
Np. Dot random dot random of 5 and go ahead and execute this.
So now you will see the value that it is going to be between minus
3, 2 plus 3, depending on the variance. What we have right? So by default
it. There is a 3 Sigma rule. You have 6 sigma, 3 sigma, and all right depending on
the variance. So this is going to be on the 3 sigma part, so the value will be
generally between minus
3, 2 plus 3.
So every time when I execute it, right? So you'll be seeing the values coming up on
the negatives
as well as the positives.
So this is going to be a good example where you can generate values which follows
a.
a a normal distribution brand. N is for normal.
Rand is for uniform.
Okay?
No. I'm talking about so generally, your normal distribution, right? It will have a
deviation.
right? Deviation in the sense that how much it is deviating from the central value,
right?
Because there are going to be different varieties of
normal distribution. So if I go ahead and take like this, right?
So I'm going to be having my. So here this is minus 3. Sigma.
Sigma is nothing but the standard deviation of the data.
Okay? So the values will be like this.
Either it will be a 3 Sigma value, or
it is going to be a 1 sigma value
one sigma will be like this is sigma minus sigma plus sigma.
so the value will be like this.
So either so by default. This is configured. So that is why you will always get the
values from minus 3 sigma to plus 3 sigma value. Sigma is nothing but the standard
deviation of how it is varying from the central mean.
okay.
Rand, whereas when you take Rand, it is always going to be between 0 and one. So
the values from 0 to one right, you will be having the values from
0 to one getting generated. So when I use Rand right.
it'll pick any values in this range and gives me the result.
Okay.
look, can we give some range to it? So here, in this case, right? Yes, you can give
it some range, the reason why
this cannot be configured. These are all inbuilt functions.
So if you really want to customize it with values from minus one to one or give it
some range. Right? You need to write a conditional statement for that writing,
basically a programming instruction saying that generate within minus one to one.
So you need to write a separate statement for it and then go for it. We'll do that
exercise. But before that we need to know what are the inbuilt functions right?
That is why I wanted to cover this topic.
So by default, it is giving me with some values. But if you want to customize it.
you need to add a programming instruction to it and then go forward and generate
that.
Okay.
we can multiply values. Yes, that can be done. But you can write. So basically, the
the idea is that you need to write a separate instruction
to generate it with the values. What you are expecting.
but what we are seeing as an example now is whatever numpy has its own right, the
inbuilt ones. We are seeing some examples on that.
Okay, so what is 5 Internet? We are asking it to generate 5 random values
subhashish.
So if I mention 3 here, it is going to be
trying to generate 3 random values for me.
Okay?
So sushma is asking. Rand gives only, yeah. Rand is going to be uniform
distribution, right? Sushma. So the values will always be ranging from 0 to one.
ran this uniform. Maybe I'll write it as a comment here. I hope everyone knows how
to write comments right? This will generate
values that will follow uniform distribution.
Okay? And whereas if you put Rand in.
this is going to be following
generating values that will follow normal distribution.
Is everyone there with me, with Rand and Rand and function.
and there is also again a very similar one, which is called as rant int.
what if we want to generate random integers?
Okay, so that is, np, dot random dot Rand, int, we call it
okay. So np, dot random dot Rand. Int.
If I just mentioned 5 here so randomly, it will be generating 5 integers for me
between. So if I just say
between 0 to 5, right? You will be getting different values here. So if I put 10,
it will be like trying to give me a 1 integer. If I want multiple integers. Right?
Let's say I want 4 integers now
between 10. Right? So it will generate.
I'm sorry.
So it is going to. So if I go ahead and give Np. Dot random.
So whatever be the value, it is going to be generating it between that. So if I
just say
5 Kamarten, okay, now, what I'm trying to instruct here is generate some random
integers between 5 and 10,
so you will be getting values like 6, 5, 7, 9
different random integers between the tens. But if I just say, if I want 3
integers.
3 integers between 5 and 10 randomly generated.
so you know you will be going ahead and seeing 3 integers getting generated, it
might get repeated. Because, like I said, it is a
a customized algorithm, right? So you just need to make sure that you give a bigger
range.
So if I just say between 5 and 100,
you're going to be having 3 set of values right?
So every time when you do it you will get some different set of values.
3 random integers between 5 and 100 getting generated.
So I hope, like everyone is clear with Rand Randon and Randent.
It will include it will include the rant starting and the ending part. But just try
to make sure that it is going to be.
bacon.
Okay, low will be included.
The lowest value will be included, but the ending value will not be included in
random.
If that is the question you are asking right, 5 will be included.
but 100 will not be included in your output.
Okay?
Is everyone there with me with Rand Randen and Randint
just wanted to take a quick confirmation on that. So all these things right, are
all going to be the inbuilt functionalities.
What we have with numpy
lot of capabilities here. But the only thing is where to apply what right? So that
is going to be the
interesting thing that always happens.
Okay. Now, coming to the question, what we have now, we have seen all one
dimensional arrays.
Is it possible to make 2 dimensional arrays like a matrix.
That is what will be the next question. Right?
So let's take this as an example. I'm going to be generating 20 numbers
between 5 and 100. So I'll take this random integer as a function
between 5 to 100, I'll take 20 values.
So what it means. It means that between 5 to 100
generate 20 random integers for me.
That is what is happening right in this command. So if I go ahead and execute it,
you'll be seeing that
20 integers have been generated.
But what is the dimension?
The dimension here is, it's going to be
one dimension right in one dimension. We have 20 values.
So how do I make this as a 2 dimensional value? Then we can have this command
called reshape. I hope everyone has given a practice on reshape when you did your
numpy exercise.
Yeah. So whatever you have it in one dimension, you can always reshape it into 2
dimensions.
Okay, so let's store this variable. Let's call this a sample.
I'm I'm taking this sample
of 20 integers generated. So when I how do you look at the shape of the matrix or
the n dimensional array. I want to know what type of a dimension it has.
So you'll have this command called dot shape.
Right? So if I do dot shape. It has 20 elements
in a single dimension, basically in one dimensions. We have 20 elements. Now, how
can I reshape it? What are the
the different combinations in which it can be reshaped.
So I can reshape it into a 4, cross, 5. Matrix, a Phi cross, 4. Matrix.
a 10, cross 2 matrix, a 2 cross, 10 matrix like that. These are the different
combinations that I have. Right?
So how do I do? The reshape is.
I can apply that on the sample itself, sample, dot reshape.
I can mention the rows and columns dimensions if I give Phi Comma 4.
So 5 rows and 4 columns.
So that is the reshape I'm doing on the sample.
So you'll be getting that as a matrix now.
So it's been reshaped as a Phi cross 4 matrix.
right? Yeah, inside that size. Also, you can do it. Okay. But I don't want to do it
just like that, but just trying to break it down and do it. Okay? So if you are
comfortable, adding that parameter inside the random, please go ahead and do that.
But
let's try to 1st understand the basics, and and then go for adding it inside the
parameters and then trying to add that complexity part. Right?
You can go for that.
Now, what I'm trying to say here is Shalab
500, right between 5 and 100. I'm generating 20
in pieces. I hope you are clear with this part.
how it is generating the 20 integers between 5 and 100. Yeah. Now, instead of
making this as a 1 dimensional array, because this is a 1 dimension right in 20
values are there. I can reshape it into 2 dimensions.
As a matrix. So 20 entries are there inside an array. I can have the option to
reshape it as a Phi cross 4. Matrix.
So how to do that I can apply this dot reshape and tell that
it should be like a 5 row and a 4 column matrix.
or if you want, you can make it as a 2 row
and 10 column metrics something like this.
so it'll be like 2 rows and 10 columns will be there
so that matrix what the array that got generated can be reshaped into
your dimensions, what you are specifying in the reshape command.
Okay, no shape is nothing, but it is the dimension. So
rows and columns. Is there right?
So here in this case, if it is not specifying any dimensions for your columns.
that means that it is in a single dimension.
Okay, so it is 20 entries in. It's like a, it's like a vector a vector having 20
values in a single direction.
That is what is called as this 20 comma. But whereas when I reshape it, and then
look at the shape right after reshaping. When I look at the shape.
Okay, I need to store this in a variable. I'll store this sample result
after reshaping. I'm storing this in a result right
now, when I look at my sample result shape.
Now, it will be 2 Kamartin. Now it's a matrix where it has 2 rows and 10 columns.
That is how we'll be seeing the result.
Okay. So when I gave only Rand it gave 520.
No, Rudra, I'm not understanding. So you're saying that. Did you try? Np, dot
random dot Rand in 10
you got that result
ideally, it should not work right? So when I say, random, dot Rand, okay. So when I
just say Rand here
and say, if I come a hundred 20.
Ideally, this should be your result
right between 5 and 120 divisions will be made.
So don't try this out with Rand. I'm saying that. Try this out with random integer.
Understand the difference between what is going to happen between Rant Randent and
Randent.
Okay, so if you want integers, go for random. If you want uniform distribution, go
for Rand.
If you want normal distribution, go for random. That is going to be the difference
between the 3 commands.
I hope, like, everyone is able to now make 2 dimensional.
I raise. Now.
yeah, Rand, dot uniform is also because the package keeps on Updating the data.
Okay? So the reason is that whatever I'm saying right now, it's based on whatever
we are practicing. But maybe because of the package upgrades, right? They might
have brought in a very specific command with Dot uniform.
But my suggestion is, instead of dot, inform. You can also go with the Rand
function, which will be generating uniform for you.
Okay.
they might have come up with a very specific method for uniform, but at the end of
the day all you get is the same result.
Okay.
again. Guys, just a heads up here is that when we start talking about a package,
right? So as we speak, the package will always be upgraded. Today there might be
some new releases that might be happening on the package. But that doesn't mean
that it will completely change the functionality.
Let's 1st get the basics done, the foundational aspects done, and then, as you
practice more and more, that because what I what I'm trying to say, here is, we
have a dedicated
numpi documentation
package. Now, documentation portal for us, where whatever we are trying right will
already be here as one of the examples that is kept for reference. Okay? So
whenever we start talking about a package, please make sure that you bookmark this
link also. So I'm sharing this link in the chat.
So when you start talking about a package, there is a a dedicated documentation
link for that.
So whatever we are discussing is not the 100% capability of the package because of
the time constraint, we might be doing 30 to 40%.
But the other things are going to be more like a self-explanatory one. When you get
into this documentation page.
am I making myself clear guys with all these things? Whatever I'm trying to
explain.
Yep.
So I'll try to give you the documentation link when we start discussing on a
package you please, bookmark that in your browser and then try out the basic
exercises. What we do in the session get comfortable with that, and then try to
explore more in that documentation link and come back with more questions.
Okay.
how to check my package versions in the command line. You can do numpy dot. Okay,
if you want to know that right
again, like we are still in the basic right. I wanted to take baby steps towards my
end goal. Right? So I'll always make sure that everyone gets the fundamentals
right? And then I go for the next step. But since you asked, let me go ahead and
tell it so whenever you import a package, if you just wanted to look at the python
version.
So when I just say python dot, you'll just
or underscore version.
Okay, I think this might be there in the terminal, I think from the terminal it
will be better to look for it. I'll open up a new one.
So when you want to look for a version. Right? Python, hyphen hyphen version is for
your python version.
Okay? And if you want a particular package, right? So you need to close your
notebook because we need the terminal for that. But you can just put the package,
name
and dot
double underscore version prefix and suffix with a double underscore. You'll be
getting the package version for that.
Okay? In fact, like those things right? Will be there in your installation pattern
somewhere here.
So you see, import numpy Np, dot double underscore prefixed and double underscore.
After the version, you'll get this one if you just wanted to really see that.
let's go for it and
see that portion. So, my by default, it will be like one dot, 2, 4 dot, 3.
Because this package right, we have not installed it separately.
Whatever comes with the anaconda framework. Right? We are directly leveraging that
package.
And that package version is automatically going to be upgraded as we upgrade the
anacondas version.
Okay.
yes. Cool.
Now, going forward so these are going to be some of the comments that I have on my
numpy generating part we have seen.
and you would also be knowing how to do the
finding out the maximum value minimum value.
Did you get that exercise done in your numpy part
how to find out the maximum value, how to find out the minimum value in the array.
Np, dot, Max, np, dot, min, np, dot argmax, argmin.
Yes, no ringing a bell, maybe a quick practice on that. Okay, so let's take this
sample result. Now.
okay, or else take the sample one.
So now in this array sample result.
I want to find out what is my maximum value in this n-dimensional array.
How do we do that?
So there is a direct command which you can apply that on the sampled result to get
the maximum value. What is there in that array?
So you'll be having a method called Np. Dot, Max.
So you apply that on your sample result.
So what will happen here in this command is, it will look for the maximum value in
that array and displace that as an output.
If I go ahead and execute this, so in my
sample, result 97 would be the highest value. Maybe, since I did a random integer,
my values and your values will be little different.
But this command will help you fetch the largest value in your array.
I hope, like everyone is able to give this a try. Np, dot Max, trying to find out
the maximum value in the array.
Can I get some quick confirmations in the chat for that
largest value in each row. Again, you are putting a condition there. Maybe so you
need to write a iteration statement, iterating through each row and finding it out.
So not right now, but towards, as we progress in the exercises. Right, we'll do
that.
So whatever we are seeing now are the inbuilt methods.
It will search through all the arrays and gives the largest value.
Okay, now.
just the next step, what we generally do finding out the minimum value. You go for
Np, dot, min.
and you go ahead and pass that array value inside it.
So the minimum value in my array is 11.
So finding out the maximum value, finding out the minimum value.
using the Np, dot, Max and Np dot, min function.
Yes, I hope everyone is able to do this, finding out the maximum and minimum value
in the Nampay array straightforward command
questions. Please do let me know if there are any questions, please do post it in
the chat.
I might be able to pick it from there and answer the question.
Okay, now we found out, what is the maximum value and minimum value in my array.
Okay, but sometimes we need to know where my maximum and minimum value are located.
The position of my maximum and minimum value.
So when I say position here, it is the index position.
So what is the index position of my largest value?
And what is the index position of my minimum value. So if I wanted to find that
there are again commands for it, so you'll be having a command called Np, dot.
Argmax.
Okay, so Np, dot, Max will give me the value.
Np, dot. Argmax will give me the position of the largest value. So where is 97 kept
considering it as a single dimensional array. Where is 97 here.
So 97 will be starting from 0, right? 90, 0 0 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, and 11
at the 11th index position we have 97, the largest value, right? So that will be my
output.
So if I go ahead and apply that on my sampled result, I'll be getting the output.
As 1111 is not the value
11 is nothing but the index position of my largest value
that is called Argmax for me.
Okay? And similarly, if you wanted to find out the index position of minimum value.
It is going to be called Arg. Min.
finding out the index position of the minimum value. So what is the index position
of minimum value?
Starting from 0 1, 2, 3, 4, 5, 6, 7, 8, 9, 1011, 1213, 1415, right at 15, th in the
expression, we have 11.
So the answer. What we'll get for argument will be 50.
Yes, I hope, like everyone is able to be there with me
in finding out what is the index position here.
Yeah, I know that it's a 2 dimensional array, zitendra, since it's an inbuilt
function, right? It is going to be considering it's a reshaped one. So, considering
it as a 1 dimensional array, what is the index position?
That is what we have got as an answer here.
We won't be able to get the row column representation of it.
Rather the index position when it is going to be in a 1 dimensional array format.
A 2 dimensional array can always be flattened to one dimension. Right? There is a
there is an option called flatten. We'll talk about that later.
Even when it is in 2 dimension, we can flatten it and make it as a 1 dimensional
array, and then we'll find out whatever needs to be calculated on that.
Okay, so the difference between Max arg, Max, min. Arg min.
So these are some of the questions what might come right. So whenever you say that
you are comfortable with numpy package
right they might be asking you the questions on this, just like how they have the
classic question on what is the difference between while and do while right.
So when we talk about numpy, they might be asking, What is the difference between
Max and Arc Max? So one
retrieves the value, another one retrieves the index position of that value.
Okay.
all good. Till this point. Just wanted to take a confirmation. If everyone is there
with me
till this point.
Yes, wonderful!
No.
There is also one more property I would like to talk about right before we move on
to the next topic. A very interesting property. What generally happens in your end,
dimensional array. Okay, so this property is called broadcasting.
So what do you mean by broadcasting of variables
is that when we are trying to prepare a
an array. Right? Let's take the sample as an example. Whatever we created, right?
So the sample has some random integers that are
generated between 5, 200. Right
now, let's take a you know. Yesterday we we did some examples on indexing and
selections. Right? So if I wanted to retrieve
37. Now, how do I retrieve it?
If I wanted to retrieve 37 from my sample, so I'll take sample, open up my square
bracket. So where is 37 here, 0, 1, 2, 3, and 4.
So at my 4th index position, I have my 37,
and I'll get that value. So this is how we'll do the
indexing. We call this as indexing and selection.
Okay, now, if I wanted to, let's say, retrieve values from 20 to 68.
I want the values from my 5th value till my 10th value.
How will I do that? I need to give a range selection now?
Right? So I need the values from 5th value. That's here. In my case, it is going to
be from 37,
2, 53, right? I want the values from 37 to 53. What I'll do
I'll take the sample and I'll give the index position
from which one it will be like 0 1, 2, 3, 4, right? So 4. It should start from 4,
colon
11, so it should pick 4,
37, 5, 6, 7, 8, 9, and 10, so till 25 it will be like.
take it, or I'll take it till 10 itself, so that there'll be 5 values here.
So if I go ahead and give 4, Colon 10. So you'll be getting 6 values, (372) 031-
9653, and 68.
So you can either choose a specific index and select it, or you can put a range
and you can collect the values from that range and select it. So this is another
way of doing your selection, or you can pick the respective index position and
select it. So let's say I want 90,
70, and 37 in my example.
I want 90, 70, and 37.
Now, that's not a range for me.
But that's a multiple selection.
So what I can do is I can do sample.
I can pick that respective index positions. So I want the Zeroth index position
comma, second index position
and 4th index position.
So when I mention it as 0 comma 2 comma. 4.
It will pick the Zeroth position.
second position, and the 4th position in my index.
and when I go ahead and execute it.
So I'm sorry what has happened.
Oh, yeah, I think since I'm doing a multi selection right? I need to mention that
in a separate array.
So I need to mention this inside one more square bracket.
Okay, something like this. So if I am trying to do multi selection, this will
itself become an array.
and that array I'm going to be indexing and taking it out. So if I go ahead and
execute it now, it will be working. So 90,
70 and 37 will be taken.
Okay.
So now we, I hope, like, everyone is comfortable to know these types of indexing
and selections. Right.
Either you can give
that particular index position and select it one index, and you can select it, or
you can give a range of index and select it.
or you can mention the specific index positions and select it.
Is everyone comfortable to do it, because this is an important takeaway. You'll be
doing this very frequently in your upcoming activities.
I wanted to make sure that everyone is comfortable in the indexing part.
Can I get a quick confirmation on that?
With this indexing part? Is everyone comfortable with a single index, my range of
index.
So this is applicable only for N, yeah, number, yeah, number is the basic here,
like I told you yesterday, right? Numpy, n-dimensional array is the basic unit.
And only when you modularize it and collect it. That's where your Panda series data
frame. So even if it is a pandas data frame.
when you break down the pandas data frame, the individual value in a pandas data
frame is actually an n-dimensional array.
So n-dimensional array is the basic building block.
And when you collect it and organize it, that's where you get the different
formats, like Data Frames Series.
All those things will be meant only for the processing steps.
But the property will remain the same whatever we are discussing now.
Okay, yeah, now we got this indexing part clear.
Now, what I would like to talk about here is, let's take the sample, and I'm I'm
going to be taking. Let's say, the 1st 4 values.
90, 1270, and 28.
Okay, so I'm going to take a subset now.
a subset of the 1st 4 values.
So what I'll do sample, I'll take 0 Colon 4.
So what it will do. 0 1, 2, 3,
those values will be taken right? So this is the subset
now a question for all of you
numpy arrays. Is it mutable or immutable?
We all talked about mutability and immutability in your fundamentals? Right?
Is it a mutable one or an immutable one
mutable? I hope everyone is comfortable with saying that it's mutable.
because that is why we need it, or else. If it is immutable, right? All the
mathematical computations cannot happen.
Right? So it is going to be having a mutable property. It's not immutable. Okay,
immutable is Tuples only
all the other things are mutable for us.
What does that mean? I can change my value. What is there in my n dimensional
array, instead of creating a new, variable, and changing there whatever is already
there in my
n-dimensional array, I can change it.
How can I change it?
Let's say I want to change this 90, 1270, 20 to some other value I can directly
give it. Let's say I want all these values to change it as 100.
So when I say, sample 0 Colon 4,
and I'm changing this to 100, what does it mean? It means that I'm changing all the
values, whatever is there in this range as 100.
So if I go ahead and do this.
and now take a look at my sample.
what is going to happen. My 1st 4 values
has now got muted to 100.
That is why I'm saying that my ndimensional array is mutable.
Yes.
Is everyone there with me?
Why am I saying that this n dimensional array is mutable?
Yep, now, coming to our property right broadcasting property here? So whatever we
do on the subset
that is going to be replicated on the original data frame also. So that's the
property. But let me explain. You
let me take this sample result that we have got.
So that's a that's a 2 dimensional array. Right?
I'm I'm going to be like trying to. Or maybe let's let's not complicate this one.
I'll make it very simple.
I'll try to generate a a sample array itself for this example, just to show this
property right?
I'm I'm going to be taking one sample, are they
where I'll use my Np. Dot a range function.
and I'll give 10 comma 21. So between 10 to 20 it will generate one
array for me sample array. It will have values from 10 to 20.
Now, what I'm going to do. I'm I'm going to take a subset a portion of this array,
and keep it separately.
Let me call this with a different variable. Let me call this as subset sample
array.
So in my subset sample array
what I'm going to do. I'm just going to take, let's say, the 1st 5 entries in my
array.
So in my sample array I'm going to be taking 0, Colon 6.
So 0 1, 2, 3, 4, 5,
right? So till 10 to 15 will be selected
and stored in this variable, called subset sampler.
Let's see that.
So the subset sample array you're having the values from 10 to 50.
Okay, so just taking a small pause here.
So what I have done. I have generated a
a sample array values from 10 to 20,
and I created a variable called subset sample array, and I picked the 5 value 6
values. What is there.
and stored it in a variable called subset sample array, and kept. It?
Is everyone there with me till this point.
Yes, creating an array and creating that subset of that array.
just giving it a minute for everyone to catch up here. So create one array.
Np. Dot a range and take a subset
of that array by taking the 1st 5 elements of it.
Okay.
yes. So now we have a variable
which is called subset sample array.
and it has 5 elements in it.
Okay, now, the next step, I'm going to take this subset sample array.
Okay?
And what I'm going to do. I'm going to be changing everything. All the elements
in my subset sample array to a particular value.
Okay? So what I'm going to do. I'm going to use my colon here, which means that
change. Whatever is there in the subset sample array?
Let me change that to 100.
So what does this command do? It will pick all the elements in my subset sample
array
and change that value to 100.
Okay, so let's see that I'm going to change it. Let's execute this command.
And now, when I take a look at my subset sample array.
now, instead of 1011, 1213, 1415,
all my values will be 100
again. Taking a small pause here, everyone there with me.
What has happened? I have taken that subset sample array.
and I have changed all the values to 100
right now in my, the interesting property here is that actually, I made the
modification
in the subset sample array variable right. I've changed the variable subset sample
array to keep it as 100.
But now, since it is in the n-dimensional array format, it also follows a property
called broadcasting.
What does that mean? Whatever changes I make in the subset.
it is also going to reflect in the original data also. So what do I mean by that? I
actually took the subset from the sample array. Right
now, if I take a look at my sample array
ideally, which was supposed to be having values from 10 to 20. Now I took a subset,
and I changed that right. It not only changed in the subset, but it will also
change in the master data. Also.
you see that the 1st 6 values are changed to 100, then (161) 718-1920. Still.
you're all able to observe this property right ideally, we changed it in a
different variable called subset array.
But it is actually reflecting that in the main array, because what we are doing.
We are actually not trying to
split that and take only those variables in the subset one.
We are taking a a portion of that array and using it for our requirement.
So the original arrays. Dependence is still there in the data
it's going to be. It's it's going to be happening every time to an array, because
what we are doing in the subset here
when we take this range right?
It is not basically creating a copy of it. It is just referencing whatever we have
given and storing it inside that variable.
So how to avoid this scenario
I don't want the change to happen in my original array.
so that might be the something what everyone needs to know. Right? So there is a
way to make a copy of it.
So keep a copy of the original data, store it in a variable and use the other copy
for all your processing activities so that your original data will stay intact.
Okay, so how do we do that? Let's maybe bring back that array, sample, array right?
I'll try to use that Np, dot arrange function.
You bring back that sample array.
Now, what I'm going to do is
I'm I'm going to be, or
maybe like, okay, let me just do that exercise one more time.
Okay, so now, this is a, this is a problem for us. How can we overcome this?
So I will go ahead and create
the sample one more time. Np, dot.
okay, 8 inch of 10. Comma, 21.
Okay. So now, my sample arrays here. Sorry this is coming up again. And
so my sample arrays here now before I start making any modifications or processing
changes in it.
What I'll do is I'll take a copy of this array, so I'll just say copy array is my
function.
I'm sorry. It's a variable copy array is my variable, and I'll take the sample
array, and there is a method, a default method in numpy, which is called dot copy.
Okay, so dot copy is going to be a function. What happens here in this command is
it makes a copy of the sample array and puts it into this variable called copy
array.
Okay? So now you'll see that whatever is there in copy array is just a
the same values what you have in the sample array.
So now we made a copy of it, and then now we can start working on the copy array so
that the sample array will still be intact with all those things. So now, if I take
a subset now.
I'll take the subset sample array.
So this subset sample array will be a a subset of our copy. Are, you know, 0 Colon
5.
Okay. And then we go ahead. And
look at the subset sample array and we change it.
I change everything to 100.
Okay? And then you see that now in my copy array. Also the chains will be
reflecting.
But in my
sample array, which was supposed to be a copyright, the original data that will be
intact with the original data.
I hope, like everyone is able to be there with me in the scenario, right? So we
made a copy, and that copied array was the one what we are using for the processing
and the original data will be staying intact in that.
No, we need to have a complete copy of the entire information shellab. So always a
good practice.
So whenever original data is given to you, right.
don't try to do any type of processing on the original data. You make a copy of the
original data
and work on that copy.
Okay.
so Rudra is asking if the memory location is same for the copy and the subset. Yes,
for the copy and the subset. The memory location is the same.
But for the sample array it is a different memory allocation, right? So that is why
it is staying intact for us.
And this property, what you're observing is called broadcasting.
Do you think this is a good property or a bad property to have for any type of a
data broadcasting property. Do you think it's good or bad?
Bad? Why bad?
And if you say good, why, it's good.
bad in the sense that it is going to be
modifying the original data every time.
But ideally, this is going to be a good property. The reason for that is your
memory space.
When you are going to be computing data, right?
You need not always create a new, variable
right. It always tries to just like a calculator. Right? You remember you you know
how you're working with calculator one plus 2 plus 3.
It's not going to store one plus 2 in a separate variable, and then 2 plus 3 in
another variable right? 3 plus. Then you go for the next 1, 3 plus 3,
one plus 2 plus 3 together. At the end of the day we get 6 as the result.
and that computation time is much faster in intermediate variables will not be
created right?
So it saves a lot of time. And the memory when computations are happening in numpy,
so broadcasting is actually a good property. But how do we make it for our benefit?
We make a copy
and then work on the copy for all sorts of processing techniques.
I hope, like everyone is there with me in trying to understand. What is this
broadcasting property?
Yes.
no application of this is what I'm trying to tell you now shall up. So the
computational speed right? If you are going to be storing everything in individual
intermediate output tree, say, say that you are adding something. Then, after
addition you are dividing something. Then, after division, you are multiplying
something, and after multiplication you are again dividing something.
So if you are going to be, if this property is not there. Broadcasting is not
there.
After every operation there will be an intermediate output getting generated which
will be added to the memory.
but instead of that, it will process everything in a single go.
There will be no intermediate memory getting stored for us at the end of it. You
will be getting a quicker result within a short span of time.
So that is going to be the result of our broadcasting property here.
Okay.
Varu has one error. Let's see, what is that error, Scott.
It has to be a double underscore Baru version. Prefix right before version and
after version. You have put a single underscore.
You need to put a double underscore and check it.
Okay.
why is it called broadcasting? Okay, so broadcasting. What do you mean by
broadcasting? You say something, and it goes to multiple people. Right?
So you do your change in one variable. It is going to be reflected in other
variables also. Right? So that is why this property name came as broadcast.
Okay?
So there's something called time, which is yeah.
space complexity, as when you go in depth to the calculations pattern. How quickly
you want to make the calculations to happen
so numpy python like I said, python follows a
a very lazy type of programming, just to make your work a little bit faster.
it will try to have its own properties. Are we going to play with that? Or are we
going to make our own conditions and do the computations. That's where we need to
take a call. Do you like to spend time in your computations and then get the
results?
Or would you like to make everything on the fly and get the results at the end of
it.
saving the time and the memory utilization.
People prefer python for that.

very simple command, Np, dot, save will save it. Np, dot load will load that file.
Np, dot save will save that array in an Npy file in your file section.
Okay, that is, Np, dot saves.
Come in.
If you want to load that content. What is there from that Npy file
we'll use this command called Np. Dot load. Put that into a variable, so that
whatever is in that Npy file
will get transferred into this variable, and you can use it for your processing. So
all I have to do now is share this Npy. File with Nilesh
and Nilesh will be executing this command. Np, dot load.
Take that output. What I have got
and carry on with the variable what he has created.
Okay.
I hope everyone is there with me
in trying to save the n-dimensional array
into a file and loading that file.
Yes, yes, we can save multiple arrays in our system.
we'll be trying to do something called as a zip. We zip multiple arrays together.
and we provide indexes for that. Okay, let's let's take that also as an example.
But before that
I would like to know if everyone is able to save the n dimensional array as a file
and load that file into a variable. Please give a confirmation on that.
Okay.
now, the second exercise. What if I want to store multiple arrays? Let's take an
example, I want to store 2 different arrays.
So I have my sample array, which is 1011
to 20. And I have my sample result.
Which is this setting.
Okay? So sample array, I have this value sample result. I have some values there, a
2 dimensional array for that. I want to store these 2 values.
and then put that into a file and share it with another person.
How do I do that? I need to zip this to arrays, give it some index.
and then based on that, I need to share it to the other person.
So how will I do that is using my numpy function? Np.
Instead of save, I have something called save z
zipping. Okay, zipping the 2 arrays and sharing it so save Z. Is going to be my
command.

So in that I'll give my function name. Sorry the file name, so I'll say I'm storing
2 arrays right. I'll call this as 2 arrays.
Dot. It will not be like a Npy. It will be called as an Npz file.
Okay, I'll give this extension. It's a Zip file, just to tell the person that there
are 2 arrays in it. So 2 arrays dot Npz is the file extension.
and in that
I need to tell what are those 2 arrays? Right. So how do you access it? A means
sample array should be called okay. This is my thing, and B means sample results
should be called.
So I'm zipping it in a way that A will signify sample array
and B will signify sample result. And then when I zip it right.
you'll get that 2 arrays dot Npz file getting created. You can check that in your
workspace.
Let's refresh this.
So now you see that it has got to erase dot Npz file in your folder getting
created.
So this is the file. What you will be sharing with your team member.
and how he or she will load that the load command will be the same.
so I'll I'll show you after. Let's say you've shared this Npz. File with your team
member
how he or she will load it is.
Let's say I'm I'm just calling this as my archived arrays, so whatever array you've
got right.
I'll call this Np. Dot load.
I'll call out my 2 arrays dot. Npz.
okay, so [Link], dot load
command same. So a 2 arrays or Npz file will get loaded and stored in this variable
called archived results archive Race.
Now the Zip file is loaded into this variable. Now, how do I access this? So in my
archived arrays.
If I call out A, it will retrieve me. The sample results value.
If I call out B,
it is going to be retrieving me. The sample result.
Variables, sample array, sample results called out by A and B separately.
So how much ever zipping you are doing right. Just make sure that you're signifying
it with that appropriate index. And with that index they'll be able to call out
that variable.
storing it like a dictionary. Exactly.
You have to tell it. Okay, what to give it as A and what to give it as B right, you
will be storing it, giving this definitions, telling it to your stake team members
that okay, I've stored this value in A and B, so he or she will pick it from there
and use it for the processing.
This thing you need to tell. They won't be able to know that the key value pair,
what you're signifying right? You need to mention that.
So the team members have to know what is there. So once you send out this file.
you have to tell them that how you have kept the values inside your dictionary.
So A is the place where I kept this variable. B is the place where I've kept this
variable, so you can access it when you are using it in your account. It need not
be always A and B,
it can be any key value pair. This is basically a key, right? Even I can keep it as
value one and value 2,
it will still work.
Okay, so this is just for sharing the results.
What we have got between the team members to work for the next steps.
Rarely this happens, but still it's good to know that how we can transfer the
results in numpy operations, right?
So that you'll have a complete closure on the numpy package. So whatever you do on
numpy package, you save it. You can share that file and ask the team members to
load it and take it for the next steps.
Is everyone there with me? And how do we save and export and
access the files that we have for the numpy operations, questions for me.
any questions with respect to what I'm just taking a small pause here
with all the things what we have.
So you want to print the Arcade address. Yeah. So if we print the Arcade address,
both, the things will come up.
It will be just coming like an Npz file.
Okay? So that that's why we need this accessing part. So if I look at that, it's a
library
Npy library here and in that Npy library it's stored in this memory address. So
that's why we need to access it with this particular index that we have.
So when you access it, that particular memory address will be accessed and the
output will be given.
Okay.
wonderful. Now, I think this is going to be a place where we bring a a proper
closure to the numpy package.
So many things we I don't know how much was covered in your previous session, but I
hope. Now, after all this discussion.
I believe everyone would be now getting more comfortable on how to use your numpy
package for your operations.
Can I get a quick confirmation on that, knowing what are all the things that needs
to be done on your numpy package.
Yes.
wonderful. Now I think it's a good time. I don't want to continue it further with
the next package. So we it's a good time to take a break now for that 20 min.
So after the 20 min we'll come back
and we'll start with the next package, which is called Pandas. Okay, just maybe an
introduction and just doing a quick recap of what's been done already, and all the
capabilities, because I feel pandas will be more important. I don't mind taking it
to the next session also next week. But today I just want to at least, maybe walk
through all the basic capabilities of pandas. So that next week, when we come back
right, we can take the advanced capabilities of pandas as the next discussion.
Okay, but before that, let's go ahead and take the break for
20 min, and after the big, we'll come back and continue the discussion. And before
that I think I just need to hear the feedback as well. So I'll I'll start the
polling activity for the day.
So please do let me know how you have felt so far in the session with the numpy
discussion that has happened so far.
and we'll take the break and we'll continue the next session. Okay.
day, 5 materials on the chat. Yeah, I can do that. Just give me a minute
and I should have that in downloads. Day 5.
Yes, I've shared it subhashish in the chat, and I'll open the timer as well for 20
min.
So once this timer is done, we'll come back and continue the discussion. Okay,
thanks a lot, guys.
Hi, all. I hope everyone is back from the break.
Just wanted to take a quick confirmation in the chat for that
wonderful. Let me take a look at the feedback.
Oh, wow! That's great. Okay, all the 100% of the badge are there with me. That's
really good.
This is something. What keeps me going.
Thanks a lot for the feedback.
Okay, let's get started. We will be like trying to let me end the poll.
Okay.
yes. So before the break, we were trying to talk about the capabilities of the
number package? Right? So we we had a clear walkthrough of what number is capable
of. What are the inbuilt functions in it. How do we operate with the end?
Dimensional arrays once it's created right
with the functions that is available in numpy.
So with even though numpy has so much capability and trying to do everything
whatever is possible in the numerical computations. Part. So one drawback, what
people were facing with it was the accessing part of the data, especially when it's
in the n dimensional array. When we talked about indexing right? Especially when
your data is having 10,000 elements or 20,000 elements in it.
We still need to remember the numerical index for it.
Right?
Guys, just give me 1 min. I'll be back. Just give me 1 min.
Yeah, sorry. I just got a call so before the break, yeah, we're talking about the
numpy right? So numpy. What was the drawback people faced when they were using
numpy package was when they tried to index the data points from the n-dimensional
array. It was always meant to call for the numerical indexes, let's say, every
time, even if it's a 10,000 element array. If I wanted to call the 9,000th element,
I need to type out the number 9,000 and then get that element
right? So writing those numbers was becoming complex, and people didn't really want
that. So can we do our own custom, indexing
rather than the default numerical index which is already there. Can we create our
index and then go for calling out to the variables is something what people
explored for, so that the data processing and the data accessing part becomes
easier. So that is where the next package came into picture, which is called pandas
panel data. Right? So they tried to organize the data as panels
and using the custom indexing option that is available in that package data
retrieval became more easier.
So that was the overall thought process behind that package. So that package name
pandas is what we are going to be discussing in our next
part. So let me open up a new notebook for this.
I'll call it as Pandas Basic. So that one notebook you have completely on numpy.
And another notebook. We'll call this as pandas.
Okay? So just like how you imported your numpy package. You'll also be doing the
same thing for your pandas package. Now, okay. So pandas is panel data.
I think yesterday I was trying to tell you that right? So
panel data, right? The blended word for that became like pandas.
Okay. So in pandas, just like how we had n dimensional array in numpy Wes Mckinney.
He was the person who created this package. So he came up with object types called
series and data frame.
So now it's time for us to understand what is a series
and what is a a data frame.
so series is nothing. But it is also an n-dimensional array. But you'll be having
an index what we can assign to it rather than the default ones. You still can
access with the default ones.
but it also has an additional capability where we can configure our own index for
the values that we have.
Okay. So let's see a quick example for that. Let me open up the Jupyter notebook.
So it's always a good practice to keep the 1st block of your Jupyter notebook for
all the import statements. So whatever packages you are importing right always have
the practice of keeping the 1st block
for all the import statements, so I'll I'll do the import like import, numpy.
I'll give it with the alternate name as Np.
And I'll give import pandas.
So these are the 2 packages, what I'm importing for this particular exercise
activity. Now, once these packages are imported, we'll be now able to use all the
inbuilt methods of it. So just to show you an example, what is a a series.
right? A series is going to be something which is available in pandas. But before
that I'll try to create one list variable. Now, okay, I'll just call this as a
sample list.
and I'll put some values inside it.
Let me put the values like 1020, 30, and 40. Okay? And
I will try to create a variable called labels
and put some values inside that. So I'll create one
another variable called labels. And inside that I will put some values like A be C
and D,
okay.
so I've created a sample list with the values 1020, 30, 40, and labels is also a
list which is having text values like A, BC, and T,
okay, so now let's create these 2.
Now, how do I make this list? As an n dimensional array?
I can go ahead and call out my numpy package. Np, dot array function. Apply that on
my sample list.
It becomes an n-dimensional array for me. Right?
So let's go ahead and do that.
I'll call out my
n dimensional array that is, we'll we'll call it as sample array.
And in that Np dot array
I'll call out my sample list.
So if I have my sample array.
If I wanted to access any values in it, I need to call out the
index position of it. So if I wanted to retrieve 30,
I will give the numerical index value for it, and retrieve the value. So this is
what we have seen so far.
Now, when we come to
Pandas right? So in pandas. What is the capability that we are going to have is
apart from this numerical indexing? Now I can assign these labels, what I've
configured, or what I've created to the values, what I have in my list or my
arrays. Okay, so how to do that in pandas. I've given the alternate name as Pd. So
Pd. Is the industry standard practice
to keep the alternate or alias name for pandas package. So in Pd.
We'll have one method called series.
Okay, so pd, dot series. So that's the method name in pandas.
What it will do. You can always look at the method signature.
Please be, you know, handy with all the keyboard shortcuts. What I'm trying to tell
now it's the method signature. If you really don't know what that function is doing
for that package right? Type out that command.
keep the cursor in between the brackets, and do a shift and a tab key in your
keyboard, so it will tell you the signature of the method.
So you see what are all the attributes it is expecting.
So it is expecting the data.
Which data would you like to convert that into a series form?
And what is the index position? Right? So this is the very interesting comment.
Here you can set the index what you would like to refer the variables for
okay and data type. Here, we need not make much importance, because it's all going
to be numbers. But just to show you an example, the data. What I would like to
change is is my sample array.
Okay? And
the index. What I would like to keep for it is not the default index. I would like
to keep the labels. What I created in my
previous step right that I would like to keep it as my index now.
So when I execute this command.
so you'll now be, see that it looks more like a table form, right? A panel form
panel data form. This is what we call as a panel data form, where we have a row
with a index to it, and a value associated with that.
So this is going to be what we call as a series object. So if I put this into a
variable.
let me call this a spar one
and put this into a variable right. So this is no more going to be called as an n-
dimensional array
if I go ahead and look at the type of this of where one
you'll now be seeing the data type of it being called as Pandas Core Series object.
which means that it is having a custom index along with the value. So if I go ahead
and display that for you.
this is how it is going to be displayed for you now.
Previously, when I wanted to call out for the value 30,
I was calling out with my numerical index 2, and getting that value. But now, since
I created it as a series, now.
all I can do now is, I can call it, with my custom index. What I created. Now I can
call it with C.
Let me show you how it's done. So wire one
instead of the numerical index they call out with the custom index C.
And now I'll be getting the same result. What I got with my n-dimensional array.
So the advantage here is here we can set our index, and according to that, we can
call out the values. What you would like to call
is everyone there with me with this property? What pandas is meant for
pandas is more focused on this particular aspect of accessing and retrieving the
data part.
Right?
Yeah. Use case. I'll come to it with Jacob very shortly. Okay, but before that I
hope everyone is comfortable with the logic concept that I have explained here.
Now, how is it going to be applied in the examples is that
whatever data you collect, let's say, you're going to be having our
a collection of data. I'm going to be taking it as a very simple example. Where you
go ahead and collect the people
who are interested to play a particular sports, you. Let's assume that you are
going to be a physical education
instructor in a classroom, and you go to a classroom and ask people
for nominations on a particular sport.
Okay, so for Class One. Let's imagine that you're going to class one.
How do you store that data? Is Class One.
I'll go ahead and collect the data
you can. You can either take it as a list
or you can make it as a dictionary also. Okay, so here in this case, I'm I'm going
to be
asking people to nominate themselves for
cricket, football, basketball like that. Okay, so for each sport, I'll be getting a
value
like 1 1 person interested, 2 person interested. So how will my series now look?
So the number of students I get this, I'll be getting 1%
for cricket, 2% for football, 3% for
basketball, 4% for tennis like that number of students is a list now.
and the sports is going to be my index. How do I make sense of the data. How does
this make sense? Because if I, if I don't
give the custom index and directly convert this into a an n-dimensional array.
Okay, number of students. Sorry
now, when I take a look at my list.
the way the variable that I created here.
it doesn't make sense for me right. How many people are interested to play that
individual sport from this result?
I might not be knowing how many people are interested to play cricket.
football, basketball like that. I hope everyone is there with me in this scenario.
This is something a challenge, what we will be having in our.
But if I wanted to make sense on how many people are actually interested to play
cricket, football, basketball.
I'll be giving the respective indexes for that indices. We call it right. So how do
I do that? I prepare a label for it. So label one
is going to be for me. So sports I create
as a list. So I'm going to be taking cricket.
So whatever is my 1st value is cricket.
Second value is football.
3rd value is tennis, basketball.
and 4th value is tennis.
Okay? Now, if I go ahead and make a series for this.
so if I say, Pd dot series
and pass the data, data is the number of students.
Okay? And the index is going to be the sports
that I have to get the nominations for. So now, if I go ahead and execute it.
And then. Now, when you take a look at this data, no.
you are all able to observe that right? This data gives me a lot more sense than
what I had created with my number is
right. So this is the reason why we need the
index to be customized so that we can get a lot of information on what the data is
all about. And why was it collected? All those things we'll be able to get? The
information.
Is everyone there with me now, if I wanted to retrieve.
how many people are interested to play basketball.
all I need to do is to just call out this, let's say I call this a sports one
from classroom one. Whoever is interested to play basketball. So classroom one.
I say, sports. One is my classroom one data. So in that if I call out my basketball
index.
it's going to tell me that 3 people are interested to play basketball.
I hope everyone is able to be there with me with this example. Why, this custom
indexing, will make a lot of sense in terms of the data analysis. Part.
Yes.
yeah, it's basically we are referring it right? So cricket is supposed to be
referred with the number of entries that we get for it. Same thing. This is what
I've explained is a very simple example. Imagine that you are collecting the data
for each
day of the week.
Monday. You have so much data. Tuesday. You have so much data, unless or until you
index it and refer it.
you won't be able to understand what that information is going to tell for you.
Right?
So that is the reason why we need that custom index to be prepared whenever you
have a data to be analyzed.
So this is going to be a good example of series here, so you need not always have
it like a list and
keep it inside your series. Imagine that you go to Classroom 2, I'll I'll go to
sports
2, which is nothing. But I go to Classroom 2 and ask for the nominations.
I'll I'll keep this as my number of students in.
Or I just say, Class 2.
Okay, in that. I'm now going to be making as a dictionary. Now, not like a list. So
dictionary, key value pair, you remember. Right? You did your fundamentals.
So key would be cricket and the value for it will be fight.
Okay, key will be football, and the value for it would be 4.
Key will be tennis.
The value for it would be like that. You can
even make it as a dictionary.
But the dictionary is going to be like having a key value pair.
But why can't we just keep it as a dictionary and go for it. Why do we need to
explicitly create that as a series? Now.
we discussed this yesterday? Right? Why can't we just keep it as a dictionary and
move forward for all the operations
any thoughts based on whatever we discussed yesterday, why do we need to actually
convert it into series? Or
why do we need to use numpy pandas package? That's my question.
Why can't we just keep it as dictionary and move forward?
The only challenge when we are going to be having the data in dictionary is that we
won't be able to do the
mathematical operations right? That was the only thing. Even the data set is huge.
Also, you can still store it in list Tuples and Dictionary.
but when you store it in list, tuples and dictionaries, will you be able to do the
mathematical operations on that?
Let's take for an example. In this case I want to know how much people are. What is
the average of the people interested in a particular sport?
Will I be able to do it
if it is stored in a dictionary variable?
I hope you are there with me in the question right?
Unless or until you convert this into a numpy or a pandas format, you won't be able
to do any mathematical operations on that
right? So that is the major reason why we are leaning towards numpy and pandas
package.
Okay, this is not the index here, Shalab. Okay, I'm talking about the values of the
data.
When I go to classroom 2, 3 people are interested to play cricket.
4 people are interested to pay football, and 2 people are interested to play tennis
index will start with 0 1, 2 only.
But how do you organize that data and keep it? So let's say, this is the
dictionary.
How do I convert this into a Panda series? Object test.
I'll take Sports 2 as my final output. Pd. Dot series.
I can directly pass on that dictionary variable that I collected
into this Pd dot series method.
So what happens here?
Since it's a key value pair, the key will be index.
and the value will be the data.
So 3, 4, 2 will be the data.
Cricket, football tennis will be the index. So I. All I need to do is to pass on
this variable
the moment I do it, and take a look at my sports, too.
you'll see that it automatically becomes a series object for me.
I hope, like everyone is there with me
on how we are creating the series. Object.
Maybe I'm okay. Maybe has got some error. Let's see that
series. Yeah, it should. It's a case sensitive language, right? Python is a very
case. Sensitive language. So please make sure that you put the capital case and
smaller case correctly. Your series object is
going to be with a capitalist, not with the smallest.
Okay.
okay, in 32 in 64, I think in all the cases it will always be in in this case.
Right? 1020, 30, 40 depends on what is the bit size that we have for the variables
that are created South India. Okay, no need to worry about it. Python automatically
does a dynamic memory allocation part for you. Okay, so 32 bit or a 64 bit will not
make a much difference in terms of the storage capacity there.
So when it is going to be doing some when it is done with. The Pd. Dot series as a
dictionary. Int. 64 will be there when it is converted from list to a series.
Object in 32 will be assigned. Python will automatically take care of that.
Okay.
And one thing about python is that python always assigns the maximum memory
capacity so that it won't go wrong in the memory utilization part. So whatever is
the highest capacity you have in that operation.
that bit size will be automatically assigned for the variable in most cases
64 bit. If your OS is a 64 bit OS most of the time. Your 64 bit storage is only
going to be processed for your
variables.
Okay, I hope everyone is now able to create the series. Object in pandas.
Yes.
Now, once you create it as a series object. Right
now, the interesting things are going to happen. Now we can start looking at the
data summary of it.
So let's say, if I wanted to know how many people are interested to play cricket
between Class One and Class 2.
So what I need to do. I need to add my sports, one variable and sports 2 variable.
so that I'll get the collective count of the individual sports right?
I hope everyone is there with me but the example that I'm trying to tell. If I
wanted to know the total number of people who are interested to play cricket
total number of people who are interested to pay football.
How do I get it?
So all I need to do is.
I just go ahead and add that sports one plus sports, too.
So you'll be getting the answer for the individual sports.
So if I go ahead and execute that.
so one interesting thing that happened here is, even though we have basketball's
number as 3 in the first, st variable.
in the second variable. It's not there, right? So what it will try to do is that it
will not throw us an error. It will just say that it's a null value. We don't have
the enough information to tell the collective count for it for basketball alone.
whereas in both the cases, if cricket is there, so what it has done 2 people, and
then. So one people there, and 3 people here in sports, too. So the total count of
it is 4
for football. It is 6 for tennis. It is 6,
so like that it will do the collective count. If both the values are there in the
variables. If it is not there, it will not stop the operation or throw any error,
it will just say that the value is missing. So if you give me the value, I will
give you the answer, or else I'm going to say that
it's not. It's null value. Nan means it's a null value. It's a missing data. It's
not 0. It's not infinity.
It's something like for which we do not know the value. For
so we we have that representation automatically, intuitively done by pandas
package. First, st okay?
Questions.
So why it is float. Again. Good question, good observation. When you do a
mathematical operation. Right? Zendra. Okay? So integer integer combined whenever
an arithmetic operation or mathematical operations is done.
python will automatically convert that into a float variable to maintain the
precision.
Okay? So precision, I hope you're comfortable with the term precision. Right. So
just to maintain that precision, it's better to keep that data type as float.
So that's why it's automatically getting assigned there. If you want to explicitly
change that to integer.
You can still do that.
No, I don't want this to be seen as 4 dot 0. I want to see this as 4, 6, 6.
You can do what we call as typecasting.
We'll talk about that later in the sea later in the session. Maybe not today. Maybe
next week.
If you don't want this to be in the float data type, you want it to explicitly be
in a particular data type.
You can do a typecasting and bring it to that data type.
If not, python, will automatically tell you what is the best one, and gives you
that data.
I hope everyone is there with me. Why do we see this data type as float now?
That's what typecasting is there? So, Jania, I'll talk about that little. Let's not
bring too many things on the plate right now.
Okay, so that's exactly what you'll be doing when you explicitly wanted to
typecast. But we'll see that with a separate example.
Okay.
no. What I'm trying to say here is, if you take sports one individually. The
question was
sports one individually is int. 64. It is an integer sports 2 separately.
It is an in 64.
But when I add sports, one plus sports, 2.
Why do I get a float? That was the question.
So when you add 2 numbers together, just to maintain the precision of it, whenever
you do any type of an arithmetic or a mathematical computations.
you will be trying to get that value automatically assigned as a float. Variable
float means it will have a decimal.
So instead of 4, it will be 4.0. So Python will automatically do that to maintain
the precision for
the calculations.
Is everyone there with me, guys? Why, it is called float. Now.
just because of the mathematical computation that has happened on top of that.
no integer and integer cannot. Okay. I know that integer, when it is added, the
integer will never be float. Okay, but
this is a mathematical computation that is happening automatically by the packages.
Right?
So Python assigns this as a float.
If you are not comfortable to have it as a float.
Okay, then you can typecast it to an integer and keep that value.
So by default. It is a float.
but if you don't want to have it as a float, you can typecast it and do the
operation. That is what we are trying to explain here.
Okay? Because now, in this particular instance, it is a integer value.
Okay?
Whereas when you add more data, I'll tell you what is the thought process behind
this? Now, every value here in sports, oneness?
Indiesel.
Okay, let's assume a hypothetical scenario where you will be getting a value which
is a decimal in this data.
Then if you get that decimal value in the data, what you should do
if it is not going to be defined properly. Python does a dynamically typed
language. Python is a dynamically typed language. I hope everyone is comfortable
with this term called dynamically typed language.
Yes, no. Yesterday, we talked about this dynamically typed language, right?
Depending on what data it has.
it will automatically assign that value to it.
The data type to it.
Okay, let's take for an example. Imagine I, I got a
I got a a value which is decimal. I'll show this with an example. Now wait.
I'll take this class 2,
imagine that hypothetically, I got some value in basketball.
S.
4.5.
Now, if you have this data, let's take the scenario where you have class 2 with
these values? If you have this data, what should the data type be for this?
Should this data type be an integer? Or should I? Should the data type be a float
float?
Why, why did you say that it is float?
Because float can accept integer values also?
Yes, 4 dot 0 is going to be called as a float. Integers can be expressed as float
right? So that is the reason Python has intuitively understood that. So just
because if someone enters something in decimal values at that time, no need to
change that as a float, let's keep the complete value as a float, and if at all,
you explicitly want that to be integer, you can bring that to integer.
Am I making myself clear, jitendra? I know that that's a very valid question you're
asking in Tezer. Intzer will never become a float.
But at the same time, how quickly and how easily we can do the mathematical
processing without changing the data types multiple times.
So that is what python is helping us to do here through these packages.
Okay, are we good till this point
now, depending on what value do we have an Nan in there? That is the question.
Right? Like, let's say you have kept the complete values.
Nan is something we do not know the value for right. Just think about like it's all
about. Python is very intuitive. That's what I'm saying. It's going to be like not
assuming anything, saying that it's always going to get integers. So it's always
going to put the maximum thing where it can always be correct and give us the
answer.
Okay.
NAN means we do not know the value. It's not integer, it's not float. It's
something we do not know the value for so it keeps it as a float. And then does the
computation.
Are we clear till this point guys series computation part?
Not a number. That's the question. That's the thing. It's not a number. We don't
know whether it's a number or not.
We can still get a text value. Also, we don't know that. So that is why it has kept
that representation as Nan.
it's a missing data point.
Okay, wonderful. So now, coming back to our next exercise. So now we understood
that
the takeaway from this activity is that we can do custom indexing.
I hope everyone is clear with that we can give our own index.
and we can access and retrieve the values for that. So that was the activity that
we did now with the help of
the series, method.
Okay.
Now, coming to the next part, let's talk about data frames. Now. So what do you
mean by data frames?
Okay? So I I think Rajiv has a question. I think I missed out. So one. If we don't
specify the label, then it will generate the default index.
Yeah, that's the whole idea here. Right, Rajiv. It's
if we need to access it with custom index. Only we go for pandas.
or else we could still go with Numpy and do all the operations. Right? So it is.
It is highly essential for us to specify the index. If we don't specify.
it will not throw us an error, but it will create it with the default. Numerical
index. I'll show you an example. Let's say
I am not giving my index position in this
in this particular scenario where one right, I'm removing this index. Now.
I just give Pd dot series and data as sample array.
So if I go ahead and take a look at it. Still, it will be a series.
But what will happen? It will be with the default. Numerical index here, so I can.
Still, I know if I can't call it the custom Index. If I call it like this, it will
throw me an error.
saying that it is not able to identify this key part.
So it's always good to give this index whatever you have created, and then
call out with that index variable.
Okay.
so not a number is, it's not a number. It's like a representation for a missing
data point. Chakravarti.
Okay, not a number, even though the name signifies that it's something like for
which we do not know the value, for so we can say so null. So you, it's like trying
to differentiate between what is the difference between a null value and a missing
value.
So null value missing value
context, wise. Both are same because we do not know the value for it. Right? That's
why it's given the notation as Nan for us.
Okay.
so Janaki has a question. So cricket football, I think you have tried to create a
data frame. There, Janaki, did you try using Pd, dot series?
I think you might have tried out a different command called Pd dot dataframe if I'm
not wrong.
Yeah, is it something like period or series you are using? I just wanted to confirm
that
no Pd dot series should not give you the can. I know the command. What you typed
for this
series will generally give you the output something like this
the moment it is shown with the column index. That's where data frame will come
into picture.
Can I know the command? What you typed for this Janaki
Pd, dot series of class 2, you are giving. Okay?
Hmm, anyways. Okay, I think, are you using it in Jupyter colab by any chance?
I just wanted to make sure that in which editor you are using it.
Yeah, that's the reason. Okay, so colab is trying to give you an advanced view of
what is happening on that series part so ideally that 0 what you see on that output
is a column index. So that's what I'm trying to come to the next discussion here.
So 0 is supposed to be that you can give you a
a column index also here.
So let's take an example.
I am going to be trying to make sure that
I will give you the next command, so that we are all in the same line.
So the next thing what I would like to talk about is something called Data Frame
series is
a single column.
Data frames are going to be a collection of series
can be called as a data frame.
So imagine that a vector a vector is in one direction.
right? A magnitude in one direction is a vector a, vector it's a series.
Okay, a matrix is in 2 dimensions, rows and columns. Right? So a matrix will be a
data frame.
So if you take vectors and matrices, definition part, right? So vector is.
I'll have a vector representation like 1, 2, 3, 4 like that. So this is a vector.
in one dimension.
Okay? And I'll have one more. Vector let's call this as 6, 7, 8, and 9.
This is my second. Vector the collection of these 2 vectors will be called as a
matrix.
Right?
Is everyone there with me with the example. What I'm trying to say, a single row
will be a vector.
a collection of those rows and columns will be called as a a matrix.
Right? So the same case series is going to be one direction.
Matrix will be a 2 dimensional array. One dimensional array is called a series.
Two-dimensional array is going to be called as data frame.
That's the difference between series and data framework.
So the only difference what you'll be seeing in a data frame. Is that along with
your row index?
Okay, you're going to be giving a column index also for your reference.
Okay. So Devdata is asking, what is okay. Tenza is altogether a different world.
Devdata. Let's not bring here to this conversation. Let's keep it simple. And when
you get into the deep learning modules. Right? You'll get to know what is Tenza's
in that particular scenario. Okay, let's not complicate again. One thing, what I
always prefer is that never try to bring too much on your plate at the same time.
Okay, first, st let's get the basic foundations right.
And once you get the basic foundations right, then you move to the intermediate.
and then once you move to the intermediate, then you go for the advanced
conversations.
Okay, so what you have called as tensors, right? Those are all in the advanced
levels. Unless or until you get comfortable with the foundational part.
it's going to be very difficult to interpret what is a sensor, even if I tell it.
No.
okay. So that's why I'm trying to avoid those questions. So just try to stick with
the basics. But again.
I'm going to be considering each and every one of you as a 1st timer who's going to
learn about data, science and machine learning. I'm always going to make sure that
that's always the mindset. Okay? Because
if I make a concept very clear for a person who comes in for the 1st time.
then that person can always be the most advanced person in the group can also be
there in the same level. Right? So that's how the conversation needs to be built.
But don't worry. These. All these things right will slowly build up for you. So now
you learned about series and data frame, then you learn about what is called as
neurons artificial neural networks. And then in that artificial neural networks for
processing you'll have bring. You'll be bringing that conversation called tensors
so slowly, step by step, it will come into your conversation.
Okay?
So yes. Now, data frame part. So now, data frame is going to be more about having a
row reference
as well as a column's custom reference.
So one good example is, let me show you with an exercise what we have already done.
We'll try to generate some random numbers.
Okay, and try to make a 2 dimensional array for it and give it a a row index
as well as a column index and see how we can reference it. Okay, so for that.
let's go ahead and try to
have that random function getting added here.
So I'll say
people are asking about. I think now it's a good time, so that to keep whatever
values we have, the same right
here is going to be a command to set the seed value. So the combinations, what I
get
will be the same what you get when you set the seed value. Okay, so how do you set
that? It will be np.
dot random dot. C,
okay, so np, dot random dot, C, let's keep it as one. So whatever is happening in
the 1st combination.
it is going to be the same.
Okay for me as well as for you.
Okay, now, let's go ahead and try to create a a random
integer or random. Maybe, like I'll keep it as a normalized value between 0 and one
right? So what is the command to generate values in dimensional arrays that follows
normal distribution.
We just did a practice before the break right.
the command to generate values that follows normal distribution.
Randon.
Yes. So np, dot random dot random.
So what does this one represent? One is going to be the combination.
So when you apply that random function, you get different different variables
getting generated right? So every generation is one iteration, one combination.
So if I fix the combination as one.
whatever combination that happens at the 1st iteration.
That is what is going to happen for everyone. Equally. At the same time, the values
will be equal.
Seed is nothing, but
every time when you generate values right? It keeps on changing right again. I'm
repeating this point.
So to avoid that, just to keep the common values between my screen and your screen.
We are going to set some seed value for it.
just to maintain the combinations of the values that we get.
So if I keep it as 2, and you also keep it as 2.
So whatever happens in the second combination for me will be the same value
as what you get in your second combination as well.
So that is what is going to be done through this seed function.
Okay.
so let's keep any value. Let's keep it as 2, maybe just in case if people are very
much interested in know the difference. So let's keep it as Np. Dot random dot
seed.
And now it's time for us like like, let's say, if I go ahead and apply my random
function. Now.
I say, fight.
So now
I hope, like everyone is. Let's not take Rand a random now. I hope everyone would
be getting the same combination. Everyone's 1st value is minus point 4 1.
Can I get the confirmation after you set the seed value to 2.
Yes, after setting the seed value to 2
value what you get in your combination, 2.1. I use rant function.
So if I use Rand function, the 1st value is 0 point 2 0 4.
Second value is point 6 1
3rd value is 0 point 2 9. 4th value is 0 point 2 6. 5th value is 0 point 6 2.
So can I get a confirmation. If everyone has the same set of values. Now.
whatever I have on my screen. So and that is the reason why we are setting this
seed value
in our code.
Okay.
Now, it's time for us to prepare what is called as a data frame. Now.
So what I'm going to do.
I'm going to be generating some rant uniform distribution values.
Okay, so let me just show you how I'm doing that. So I'm going to be generating 10
numbers.
So
across. Phi, so 10 comma phi. So what what is going to happen if I mention this as
Np dot random dot rand of 10 comma 5 means it is going to generate a matrix for me
with 10 rows and 5 columns.
which will follow a normal distribution. So if I go ahead and execute this, so this
is what I get.
and all the values will be the same like. What you see on your screen
point 5 2 is the 1st value. Can I get a confirmation.
If you go for the Np. Dot random dot Rand.
and give 10 comma 5. The 1st value. What you get is 0 point 5 2 9.
Can someone confirm me for that?
No.
okay, let me just see. I think maybe I have to set that value. Let me set the value
and do it. Okay, I'm sorry this is going to be the value point 4 3 5.
I'm sorry. Point 4 3 5 9 is the 1st value.
followed by point 3 3.6 2.
Yes, I forgot to.
Is everyone getting the same value? What you are having on what? What you're having
based on what you see on my screen.
Can I get a confirmation quickly on that? No.
I hope like, yeah, you need to execute that command, guys. One more time.
Np, dot random dot seed, execute that command.
and then go for this rant function as 10. Comma phi.
Okay, execute that np, dot random function
seed function and then go for trying out the command.
Okay, so I'm doing it again. So np, dot random.
So every time when I keep the seed as 2, right? It should be the same value.
if not, no issues. See? It is just maintaining the combination. Okay, that's
absolutely fine. But whatever is there in the second combination it should be
constant. It should not
change.
That should be the only thing that needs to be taken care of.
Okay. So every time when you execute that np, dot random, it should be the same
thing.
Okay.
no. Someone has put a screenshot there. I'm sorry. I okay.
I hope, like everyone has got this numbers generated. Guys just wanted to
take a quick confirmation on that.
So now, what what is happening here is that this is going to be
a matrix right? What type of a matrix is this?
This is going to be a 10 cross, 5 matrix, right? 10 rows and 5 columns are there.
Now let's assume again a scenario. Here, let's assume that
these are all going to be the marks scored by a candidate in an exam.
So let's assume that these are like 10 candidates marks across 5 different
subjects.
Okay, so 10 candidates marks across 5 different subjects is what is the scenario
now, but I need to index it properly. Right? I need to give the row index with the
student names and the column index with the scores. Right? That is where your data
frame will come into picture.
So how to prepare that data frame is that
let's make this one. I'll I'll give this a student's marks as my variable.
Okay?
And what I'm going to do now is I'm going to call out this function called data
frame. So please make sure that you get this function correctly.
So in the data frame, your D and F will be in capital. It's what we call as Pascal
casing. So data frame is an inbuilt method which is there in pandas for you.
And when you, when you try to utilize this command on the data, you can always look
at the method signature.
So the method signature is, it will ask for the data what needs to be converted
into a data frame.
And what is the index that we need to give.
The additional thing. What you see from the series is that it will also ask for
setting your column indexes also.
So that's the difference between series and data frame in series. You can keep only
a row index
customized. But when it is in a data frame
you can keep both your row as well as your columns, also with a custom index.
So that is going to be the advantage of using data frames over series. So now, what
I'm going to do is so this is my data, whatever I collected as my random variables.
Right?
So the data, let's call this function. So data will be
the the function, the random 10 cross, 5 matrix that I got right with the seed
value. So that's my data that I need to convert it into a data frame.
Now, what is my index? I need to give row indexes for each of it, right? So instead
of typing it out.
I'll give it in A in in one command itself.
I'll try to give it like a space. BC. Just type this along with me. I'll tell you
what it does.
You have a space between each characters.
EFGH. INJ.
So this is our index.
So A, BCD. Till J.
I've given it to the space
in your fundamentals. You might have seen one command in Python, which is called
Dot splitter.
So whenever you are having some text data.
you can split it with a delimiter. So whenever there is a delimiter like a white
space between the characters, so now a will become separate. When I apply this dot
split method.
a will become separate. B separate C separate d, like that individually, it starts
printing so that I am setting it as my index row index of my data.
Is everyone there with me with this dot split comment.
So by default, it is going to be with the app
dot split command. We can take it. Okay, no need to. Okay, okay. You tried it.
Let's keep it as it is. Now, whatever you have given it a try. But for others.
the data part you, you must be taking the random ones and the index ones.
I'm going to be keeping everything in a variable and using this dot split command
for that.
Okay?
And then the 3rd parameter is your columns.
So columns is going to be your
3rd parameter. Right, the column index. So for that also, right, we can keep one
variable for it. Score score, one space score, 2 space, core, 3 score 4 and score
5.
So just like how I gave
A, BCD. Till J. Right, I'm going to be giving my score
titles also right score one. The column headers basically.
So score one is 1st column header, space score, 2, score, 3, score, 4 and score 5.
Then I use my dot split command at the end.
Is everyone there with me with these 3 parameters that have configured for data
frame? First, st is the data. What you would like to convert that as a as a data
frame. Second, one is the index.
the row index. What you would like to configure.
and the 3rd one is the columns, the column headers, what you want to configure
just giving it a minute for everyone to try this out. So 1st is the data. Second,
one is your index.
3rd one is your columns.
So when I go ahead and execute this.
and now take a look at my student marks.
It will be now looking like a perfect table for us.
So where this A. To J will be my student ids
and score one score, 2 score, 3, score, 4 and score 5 will be my columns for that.
So, just taking a small pause here.
I would like each and every one of you to try this command
and make sure that you get this table.
and once you get this table, maybe you can give me a yes in the chat window, so
that I can understand that you have
created this table and kept it ready
all good till this point, creating a data frame.
So now
it looks very simple. Now, right? It looks more organized compared to what we have
in our n dimensional arrays. And in our series, right
data frame looks more organized than what we have.
Right?
So we need to look at it.
Okay, I think someone has got an error. Who's that satyam
satyam? Says, student Pd, dot data frame data index.
Let me take the command. What! You have typed, satyam, and see what is error. It's
very difficult to see it from the chat.
So what is happening here.
So this is satyam's command.
Student marks period or data frame indexes here. Columns.
Okay? Looks good for me.
Let's see that some value error is coming for this.
Okay? So you see that 10 comma, 5 values is the shape, but only 9 indices are there
because you have missed out an eye here. That is a problem.
Okay? So I is not there in this. Efgh, I and J, right? So then maybe I think your
issue will be sorted.
That's fine.
And for Vijay I think the split brackets
right. I think it's not there. Jignesh has found that
again. Guys don't worry. I think you're just writing it right. This might happen
missing out a bracket, but Python is very cruel there, even if you miss a single
bracket or a single quotation right? It will throw you an error right away.
So just make sure that you write the comment.
Whatever you have.
Okay?
And it's a practice no worries. If you are getting errors now, it's absolutely
fine. The only thing is you just need to have more practice in writing these codes.
So try creating new data frames and then doing some activities on that, these
issues can be sorted.
Okay, all good. Can we go to the next step? So now we got a table
where you will be having some values represented there right
now, accessing part becomes more interesting now more and more interesting. So
let's say, I want to know
what are the marks scored in the 3rd subject, score 3.
So what I need to do, I can just call out this data frame
and just call out that column index alone. Score 3,
and I'll be getting this as a
a single column output. I it will extract only the score 3 column and gives me the
output.
Now, a question for all of you. What type of an data type is this going to be
when I extract a single column and get you the output.
So the if I, if I do a type, function on this output, what would be my
data type for it float, or what is it? It's a
it's a 1 dimensional array that we have got right a dimension across the score 3.
So when I look, when I put my type function on that, it's not a float.
It's actually like, when you break down the data frame as individual columns, it
becomes a series.
So now you understand the hierarchy. Right now, when I break down the series, the
individual values will become a n dimensional array, and the
data type of the 10 dimensional array is what we call as float.
Okay? So n dimensional array collection of n-dimensional arrays, a series and
collection of the series will become a
a data frame.
Is everyone there with me, with the hierarchy bottom to top, n-dimensional array.
Collection of n-dimensional array is a series
and collection of series is going to be a a data frame.
Okay?
Now, I got score 3 retrieved. Now let's say I want score 3 and
score 5 together in one frame.
How can I do that
again? You can call it out with the column index here. But the only thing is, since
you're calling multiple
indices. Now it'll be like
I need to put that in one more square bracket. I'll just say score 3 and score 5.
So if I go ahead and do it. So now it will be like retrieving me only the score 3
and score 5 of all my
data points.
So you know, you can extract single columns as well as multiple columns without
writing any.
So this is more like, if you relate it to your SQL. I don't know how many of you
have gotten experience working in database management systems. Right? So all the
data will be in table. You need to write queries to retrieve the data from the
table.
But here, once, once it comes as a pandas data frame. Right?
You need not write that special SQL queries. You just need to call out the index,
and it will be much faster than that.
That's why Pandas is considered more powerful when it comes to the data retrieval
part.
Yeah. So you need to write. Select star
right, select star, and you need to give the table name and then give what is the
filtering condition. All these things will now become like a single line commands
for us.
So why double braces? Because we are calling multiple index right?
So score 3 and score 5 together itself will become a data frame. So we are taking a
subset of that data frame, Sushma.
If it is a single column, it is a series. But if it is 2 columns together. If I
just take the type function on this output. Now.
what will be my output if I apply my type command on this.
Now, it's a collection of 2 series, right?
So the output. What you will be expecting here is it's going to be a a data frame,
right?
You see that now it is called as a data frame.
No, nothing like, okay, if it's 3 means not not 3 brackets and all
whatever various. So let's say you want to call out Score 2. Now, for an example.
you just need to add that inside that
brackets. That's it. So now it will call score 3, score 5, and score 2. No order
also.
in whatever order you call it. It is going to be bringing that output for you. So
if I go ahead and call it here.
let's go ahead and call this now. So now it will be score 3, score, 5, and score 2
in the final output for me.
Okay, so how much ever values you would like to call you just make sure that you
put that inside the inner square brackets.
and it will automatically retrieve that value for you.
A list of indices, a column indices that we have.
Yeah, I'll come to the row part very shortly. Rudra, I hope, like everyone, is
comfortable with the columns. Part. Now, how to retrieve the columns. What you have
in a data frame.
Okay, I have a question, or we can retrieve data from database load. And
yes, absolutely fine. In fact, like that will be a much more easier way to see how
we can get the data to be extracted. A more better customized view can be created
in pandas
rather than keeping it in your database management system, so you can connect it to
your database system.
make, take it as a data frame.
and then you can go for it
to show what are all the view that you can create with your data much quickly.
That's a use case.
Okay?
So now, when it comes to the columns, you can extract single column multiple
columns, and there is no order need to be maintained. So like I said, here, right
score 2 can be called after score. Phi also. So it's not going to do a directional
way of retrieving.
It is more like the referencing.
It will look at score 3, 1st score, 5, 1st score 2 1st puts all the values together
and gives you the output.
Okay, so that is going to be one more advantage where we have when we collect
variables with pandas.
Okay.
Now, okay. So now, this is all on the columns. Maybe one more thing. What I could
have in your column thing. Interesting thing, what I could show in columns is that
when you want to create a new column, you need not write a specific command for
that. Since Python does a dynamic memory allocation.
It will automatically create that column for you when you just mention that
variable name. So what I'm trying to tell you is, let's say that in this example
I wanted to create a column called Score 6.
Okay, just a new column called Score 6, which is supposed to be the addition of
score 2 and score 4,
so when I add score 2 and score 4,
the values of it should be populated in a column called Score 6.
So within just one line of command. You can make that happen.
because, since it's all referenced now, what I'm going to do now is in this frame
in this data frame
I'm going to be creating.
Let's call out the data frame student marks.
So the new column name is Core 6.
Okay, so how that score 6 should be created is.
I'll take my student marks score 2 plus student marks.
Score 4.
Okay? So when you add, it's a very simple command. Right. So take score 2. Add that
with score 4 the resultant one should be stored in a variable called Score 6.
That's it. So when I go ahead and execute this command.
A new column would be created, called Score 6, with all the added values from score
2 and score 4.
So you'll be seeing a new column getting generated, which is nothing but the
addition of score 2 and score 4. The values will be populated in this
within just a fraction of time you can create a new column, add the values, and
prepare that column for additional insights.
So that is also going to be one more. Very good use case or insight that we can get
from the Pandas package.
Is everyone there with me in trying to create this column and making this addition
operation on the data frame.
Yeah, I'll come to that. How to delete a column, how to filter the rows step by
step. We'll go there. Okay.
but I just wanted before that. I hope, like everyone is there with me with this
point
of generating a column.
You all are getting very inquisitive. That's very nice to see just going on and on
and on. But let's take for the others also, right slowly to catch up with whatever
they are writing.
You can go ahead and do that.
Yeah. So Rudra wants me to scroll up a little. Okay, can do that. You want the
generation of the column. Rudra.
Okay?
No problem.
Yeah. The data retrieval speed. Right? I think Chakrapani has a good point in the
chat. So the data retrieval speed between python and a typical database management
system. The moment you connect your python package a python to your database
management system.
python will have its own capability added on top of the database performance, so
that the retrieval speed will be more optimized compared to directly writing your
queries in database management system.
Okay, so that's why people prefer it to have it added with their python package,
and then scroll through the data from there.
Even now, right? If I just wanted to see something from my postgressql for an
example. It's a database management system. I don't want to go to Pg admin and log
in there. I just write a block of code where it where it's logged in, and I just
retrieve the queries directly I write my queries in Python itself, and I can
retrieve the data, and I can give it in a more formatted output way, like a data
frame. And I can add values to it.
Add some more text data to it. Those things
will help me to understand the data much better.
Okay.
cool. So now, coming to your other questions, right? Okay. Now, I added, this
column called Score 6. Now, how do I delete this?
No more. It's needed for my processing requirement.
Right? So how do we do that?
I take this data frame called students marks.
So there is going to be a command in pandas. Now we are going to be looking at the
inbuilt functions of pandas now, so you'll be having this command called Dot drop.
Okay? So dot drop is going to be the command
which will help us to remove the columns or the rows from our frame.
So, depending on what we add in the parameter, we need to do that.
So they have that asked a question, how you are getting that function help.
So let's say.
let's take this itself as an example right there. Where I want to know what this
drop function does for me. So I
open and close my brackets. It's a function.
So I keep my cursor in between the brackets and do a shift and a tab key.
Yeah, Google Collab will not work there. That's the very big disadvantage of using
Google Collab. You won't be getting this
method signature very clearly defined, not always in Google Collab. So Jupiter
Notebook will be more friendly. In that case.
that's not a problem. But still just that's why I said
Jupiter notebook. Then Google Collab, Google Collab will be much easier in terms of
not setting up too many things.
But when you want to learn something right, it's better to keep this in Jupiter
notebook, and then take it forward. Because if you
not to work well in Jupyter notebook.
Google Collab will be much more easier for you.
So that's that's the standards. How we take it.
Okay.
great. So now, in this drop function, right? What we are going to be doing is we
are going to be making sure that
what we have are
signatures. Right? So let me just show you that one more time. So we need to know
which label do we need to drop?
So basically, the column index right? Which column index do we need to drop? That
is the labels. And there is a very interesting parameter called axis here.
So what do you mean by access?
Is that in axis you are giving instructions to python by which
you need to delete that. Do we need to delete it by row wise?
Or do we need to delete it? Column wise? Okay. So when you mention the axis as 0.
It means that you are giving instructions to this method that do the operation row
wise, remove that particular row for me.
If I mention the axis as one.
Then you are giving instruction to the method, saying that do it, column wise. So
now, if I wanted to remove the column.
score 6.
The access value. What I should give is one
if I wanted to remove something from my row. Let's say I want to remove the row H.
From my data.
so I'll give my axis as 0. So axis 0 is for row
axis, oneness for columns. So now I want to remove the row. Right sorry column,
right? So I'll just go for dot drop.
I'll give the column index, so I need to remove this column called Score 6.
Okay, and I'll give the parameter axis as not 0 by default, it is 0. But now, since
I want to do this operation column wise.
I'll give the access as one.
Okay, so score 6 is the column that I want to delete and access one. So if I go
ahead and execute it.
it will show me the view
where my score 6 column will be deleted.
Is everyone there with me with the dot? Drop? Comment.
Yep.
but one thing, what you will also be observing right when you do this, the next
time when you call out, student marks
right ideally. What we might be thinking is
in this view, okay, score 6 is removed, so we'll be like getting that column
completely. Not but the moment I call out, my student marks one more time
you will see that the score 6 will come up again.
Okay, the reason for this is that what we have actually done here is it's the view
what has been created when the score 6 column is dropped, how will the outlook
output look like?
So this is not a permanent modification done on the data frame. So if you want to
make it as a permanent modification on the data frame.
to completely remove score 6 from all our operations, you need to add one more
parameter.
which is called in place.
So when I mention in place equal to true, so by default, this parameter will be
false.
Okay. But if I mention in place equal to true.
it means that I'm telling this function that permanently remove the score 6 column
from all my operations.
Okay, so that is what I'm trying to tell you this in place equal to true. So now,
when I execute it
every time when I call it
now, it will never, ever try to give me the score 6 back because
of this parameter that is being added in my drop function.
So please be very cautious in
adding this parameter, unless or until you do not want that column completely in
your analysis, you can.
You don't want to do it. You can just store it in another variable, and use that
variable so that your original variable will stay intact, but only when you are
sure that you don't want that variable used in your analysis. Then you go, for in
place equal to true and completely remove that column.
Is everyone there with me, with the drop comment that we have
the same way. If you would like to, maybe remove a row from my data frame.
let's assume that a particular student has moved out of the school
right? Maybe column. This candidate I has moved out of the school. I need to remove
this record from my data frame.
Right? So how do I do that now? So the same data frame dot drop.
So now I want to remove the row. Id right row index. I now. So I'll give that row
index, ie. Now what should be my access.
since I am trying to remove it? Row wise. My access for this particular function
will be
0. Right? I told that axis 0 is for row
axis, oneness for my column, so I'll mention my axis as 0
by default. It is 0, but it's good to mention that that's not a problem, and
if I do that.
so drop, keep the I value index. The axis is 0. If I go ahead and execute it.
Now you will see that the I
row will be removed from my record.
No axe. How? Okay we know. Score 6 is a column. So, Jania, how will Python know
that score 6 is a column?
That's a question. Right? I am giving a drop function. How will the drop function
know that score 6 is a column. So for that only we are specifying access as 0 and
one.
okay.
so Chakrati is asking. But the score 6 itself is not an original column in the ray.
No score 6 is what we created. It has become a column inside our frame. Chakrati
here, right
when I did that score 6 python does a dynamic memory allocation so automatically it
will create a column called Score 6. Add that to my frame by adding the values from
score 2 and score 6, score 2 and score 4.
Okay, that's why we are specifying the access as one to remove it.
Access 0 for rows, access one for columns only 0 and one only 2 values are accepted
for access.
Either it'll be 0 or it'll be one.
Okay, I think someone has given it a try. I give I. And
what happens if I give access as one.
some people will give that a try
right? What will happen in this case is, we are actually misleading or misguiding
this particular function, that search for the value, ie. In the column.
So by this drop function will never find out the value I in the column, so it will
throw us an error. So we are guiding it to say that
search it in the row.
Look for this value, I, and drop it.
That is why this access parameter is very important, and if you want to make it as
a permanent modification.
What is the parameter? You need to add in place equal to?
Yeah, you can also do multiple
deletions also. Okay, you're giving it a try. But yes, I'm coming to there, but
just wanted to take step by step towards that. Okay, you can do multiple deletions
also
like, give it within a list of values here, and it will do that for you.
and you can also, if you don't want to use in place equal to True. So after
removing I you can
store it in a variable. So let's say that this is a new student list.
So whatever you drop without adding that in place equal to true, you can transfer
it into another variable.
So this variable will now be having the data without the I student.
whereas the student marks will still be with the original data, you can either
transfer it into a variable and move forward, or you can do in place equal to true
and permanently modify that. So those are the 2 options that will be available for
you. So if I look at the new student list.
it'll be with with without that candidate, ie.
But if I look at the student marks, which is my original data.
so the original data will still stay intact for me. I data will still be there.
So you can transfer this into a new variable, and carry forward with the operation,
if at all. You're not very confident on using in place equal to 2
here we need, because whatever is there, it's just a transferring the part right.
But if you make any modifications.
let's say you, you modify some values it is going to impact.
Okay. But we will not modify anything in the original data unless or until in a
very rare instances.
So it's better to transfer that
and use it in the upcoming operations
again. Whatever data you have got in real time scenarios, I'm saying.
whatever data you have got, you will never tamper with the original data.
I showed an example for broadcasting just to explain the process to you.
but in real time, scenario no one will do broadcasting.
It will definitely not be a good practice to take a subset and change it there.
But just to show you the property of broadcasting. I I gave that example for you.
Okay, but in real time scenario.
when a original data is given to you, you can always take a copy and then do it.
After processing
you can always transfer it. The original data will always be a copy, but the
processed information you can keep
it towards the end of whatever processing steps you do, it can change whatever ways
it wants to change.
Okay.
I hope, like everyone is there with me, with the drop command, dropping the rows
and dropping the columns.
Yes.
I'm just looking at the time. So maybe one final exercise before we
call it a day accessing the values right? So we saw how we can access it with
column index right now, if I wanted to access it with the the row index, let's say
I want to explicitly
know how much candidate C. Has got an score 3.
Okay, just stay with me on the requirement. I want to know how much candidate C.
Has gotten? Score 3.
The 3rd subject, how do I do that? So you can't like. Just go ahead and ask, like
student marks
and type C and comma, your score 3. You can just go ahead and
call it like this. It will give you a key error.
Ideally, this is what you would have tried out just by looking at how we have seen
so far from numpy.
You give a row index. You give a column index.
and you should be expecting that output to be seen. But in pandas it is little
different. There is an function that will take care of this.
So the function name is called dot loc location.
So locating this index C with the score 3. So when I apply this dot loc function.
it will look at C in the row. Wise
score 3 in the column base, and extract that converging value, and the output of it
will be seen here. So 0 point 7 6 4 9 5.
So this is going to be the value. What you'll be seeing here.
Okay? So you'll be like trying to get that location accessing the values with row
column representations. Right? So using this loc command, you'll be able to do
that. So if I wanted to retrieve candidate D's score. 5 column.
How will I do that?
Candidate deeds score 5 column student marks dot loc row index will be D
column index will be score 5,
so can we. A. D score 5 is what I'll be retrieving. So that's the output. What I
get.
So this is how you will be accessing the
rows and columns inside your data frames.
I hope, like everyone is there with me. I think this was already done. I saw the
recording, but still I wanted to do a quick recap of whatever has already been
covered. Right?
So everyone is comfortable with the loc commands to retrieve the values from the
frame.
I think this could be the last exercise that we can do for today.
and maybe next week, when we come back right, we'll talk more on how we can do some
additional capabilities
of pandas. Right? How do we aggregate the data, grouping, summarizing.
getting more and more insightful informations about the data that we already have.
Right? So those parts we'll we'll talk in the in the next session.
Okay? But I think what we did very good today was the complete closure on the numpy
package.
So please do practice on the numpy package, on different things.
And then a basic introduction to the Pandas
trying to access create data frame series and accessing the values inside it
so that could be the exercise. What you can do for the week.
And next week, when you come back we'll we'll extensively talk on pandas package.
What are the different capabilities of it?
And we'll see how the conversation goes from there. Okay.
yeah. Loc Iol. Okay, if you want that, okay, time has gone up. So that's why I
stopped there. So next week, when we come back right, I'll talk about this? Ils, it
doesn't stop here. Okay, so we'll always have more discussions on. I just don't
want to, you know.
Hang it on, you know, in the middle and then go for it. So that's the reason.
Okay?
So Rudra is asking. Sometimes I get confused with square brackets and round
brackets. Okay, that's always a problem. So that comes with practice. Guys, okay, I
think I also face the same thing. So the more you type codes right
the better you will be like finding the comfort to know where to use square
brackets and round brackets.
Okay.
so that is just to know how we can dynamically increase the size of an array one by
one. Does it require any dynamically? Read something? Yeah, you can append it.
We'll talk about appending data points when you have it in a data frame, right? So
you have some inbuilt methods in in pandas for that
to do it.
So when you're going to be reading something it automatically, you can open that
and have it as a part of your data frame and do go for the accessing.
Okay.
so loc, C, okay, this command gives the output as multiple rows.
No, it's not okay. It's just a view. What you get here. Chakrapani, so ideally when
you give Dot loc as C, right, I'll show you that.
So this is ideally the row. Only the view only shown us multiple rows. But it's
ideally a single column. Only
when you store it in a variable right?
This is just a view.
What do you get?
Okay? But what you have actually got here is just a single row of the candidate C
across the different marks.
Okay, the browser view is helping you to show it in the multiple row way.
Okay, Santosh is asking, can't you give some time for python? Oh, Python, I don't
think so. It is going to be possible now, Santosh, but let's see, like as we are
trying to talk about this numpy pandas. Right? I'll also talk about the python's
best practices.
maybe that can help you out.
But it's very difficult to go back and teach the complete python, and then proceed.
That's the reason.
faculty 58
faculty 58
04:33:57
Thank you, Vignesha, for the wonderful session.
faculty 58
faculty 58
04:34:00
And go ahead.
Vigneshwar Veluchamy
Vigneshwar Veluchamy
04:34:01
Yeah, yeah.

You might also like