Python Pandas - II
Statistical methods
• Statistical/Descriptive statistics methods help
in the understanding and analyzing the
behavior of data. We will now learn a few
statistical functions, which we can apply on
Pandas objects.
Data Aggregation
It is the process of turning the values of a
dataset or a subset of it, into one single value.
It is also referred as multi-valued function,
which requires multiple value and returns a
single value as a result.
Aggregate Functions
max()
It is used for finding the maximum value from a
given set of values or columns of a DataFrame or
a Series.
min()
It is used for finding the minimum value from a
given set of values or columns of a DataFrame or
a Series.
count()
• This function is used to get the number of
values present in the column.
sum()
• This function is used to add all the values in a
particular column of a DataFrame or Series.
• By default it skips the missing value i.e. Nan
Statistical Function(s)
mean()
• It is used to calculate the arithmetic mean
(average) of a given set of values/numbers in a
DataFrame.
• Axis=0 , returns the mean of every single
column.
• Axis=1, returns the mean of every single row.
median()
• It is used to calculate the median or middle
value of a given set of numbers.
• Axis=0 calculates the column-wise median.
• Axis=1 calculates the row-wise median.
mode()
It is used to calculate the mode or most repeated
or occurrence of each element or value of a given
set of numbers.
std()
• Returns the Bressel standard deviation of the
numerical columns.
var()
It is used to calculate variance of a give set of
numbers.
It is the expectation of squared deviation of a
random variable from its mean.
A variance of zero indicate that all of the data
values are identical.
It describe how much a random variable
differs from its expected value.
Covariance
• Covariance is applied on series data. The
Series object has a method cov() to compute
covariance between series objects. NA will be
excluded automatically.
Correlation
• Correlation shows the linear relationship
between any two array of values (series). There
are multiple methods to compute the correlation
like pearson(default), spearman and kendall.
• Cor(X, Y) = 0
– implies no relationship [Link] correlation.
quantiles
• Quantiles are the set of values/points that
divide the dataset into groups of equal size.
• If a figure, there are 10 values that splits the
dataset.
• These 10 values are quantiles.
quartile()
• It is used to find the values that split the data
into four groups of equal size.
Summarizing Data
• The describe() function computes a summary
of statistics pertaining to the DataFrame
columns.
• print [Link]()
Data Ranking
• Data Ranking produces ranking for each
element in the array of elements. In case of
ties, assigns the mean rank.
• import pandas as pd
• import numpy as np
• s = [Link]([Link](5),
index=list('abcde'))
• s['d'] = s['b'] # so there's a tie
• print [Link]()
• Rank optionally takes a parameter ascending
which by default is true; when false, data is
reverse-ranked, with larger values assigned a
smaller rank.
• Rank supports different tie-breaking methods,
specified with the method parameter −
• average − average rank of ed group
• min − lowest rank in the group
• max − highest rank in the group
• first − ranks assigned in the order they appear
in the array
Descriptive Statistics
• large number of methods collectively compute
descriptive statistics and other related operations
on DataFrame. Most of these are aggregations
like sum(), mean(), but some of them,
like sumsum(), produce an object of the same
size. Generally speaking, these methods take
an axis argument, just like ndarray.{sum, std,
...}, but the axis can be specified by name or
integer
• DataFrame − “index” (axis=0, default), “columns”
(axis=1)
NumPy - Arithmetic Operations
• Input arrays for performing arithmetic
operations such as add(), subtract(),
multiply(), and divide() must be either of the
same shape or should conform to array
broadcasting rules.
• import numpy as np
• a = [Link](9, dtype = np.float_).reshape(3,3)
• print 'First array:'
• print a
• print '\n'
• print 'Second array:'
• b = [Link]([10,10,10])
• print b
• print '\n'
• print 'Add the two arrays:'
• print [Link](a,b)
• print '\n'
• print 'Subtract the two arrays:'
• print [Link](a,b)
• print '\n'
• print 'Multiply the two arrays:'
• print [Link](a,b)
• print '\n'
• print 'Divide the two arrays:'
• print [Link](a,b)
• It will produce the following output −
• First array: [[ 0. 1. 2.] [ 3. 4. 5.] [ 6. 7. 8.]] Second array: [10 10 10] Add the two arrays: [[ 10. 11. 12.]
[ 13. 14. 15.] [ 16. 17. 18.]] Subtract the two arrays: [[-10. -9. -8.] [ -7. -6. -5.] [ -4. -3. -2.]] Multiply
the two arrays: [[ 0. 10. 20.] [ 30. 40. 50.] [ 60. 70. 80.]] Divide the two arrays: [[ 0. 0.1 0.2] [ 0.3 0.4
0.5] [ 0.6 0.7 0.8]]
[Link]()
• This function treats elements in the first input
array as base and returns it raised to the
power of the corresponding element in the
second input array.
• import numpy as np
• a = [Link]([10,100,1000])
• print 'Our array is:'
• print a
• print '\n'
• print 'Applying power function:'
• print [Link](a,2)
• print '\n'
• print 'Second array:'
• b = [Link]([1,2,3])
• print b
• print '\n'
• print 'Applying power function again:'
• print [Link](a,b)
[Link]()
• This function returns the remainder of division
of the corresponding elements in the input
array. The function [Link]() also
produces the same result.
•
• import numpy as np
• a = [Link]([10,20,30])
• b = [Link]([3,5,7])
• print 'First array:'
• print a
• print '\n'
• print 'Second array:'
• print b
• print '\n'
• print 'Applying mod() function:'
• print [Link](a,b)
• print '\n'
• print 'Applying remainder() function:'
• print [Link](a,b)