Lesson 14 - NumPy Array Statistical Operations
NumPy provides a comprehensive suite of functions for performing statistical
analysis on arrays, which are faster and more memory-efficient than standard
Python lists for numerical computing.
Key NumPy Statistical Functions
The primary functions fall into categories for order statistics, averages/variances,
and correlation.
Order Statistics (Min/Max/Range)
[Link]() or [Link](): Returns the minimum value of an array or along
a specified axis.
[Link]() or [Link](): Returns the maximum value of an array or
along a specified axis.
[Link](): Computes the "peak-to-peak" range of values (maximum -
minimum) along a specified axis.
[Link](a, q): Computes the q-th percentile of the data along the
specified axis (e.g., q=50 for the median).
[Link](a, q): Similar to percentile, computes the q-th quantile
(where q is between 0 and 1).
Averages and Variances
[Link](): Calculates the arithmetic mean (average) of the elements in
the array.
[Link](): Computes the median (middle value) of the data along the
specified axis.
[Link](): Computes the weighted average along a specified axis,
allowing custom weights to be applied.
[Link](): Calculates the standard deviation (a measure of data spread
from the mean).
[Link](): Calculates the variance (the average of squared deviations from
the mean).
Correlation and Histograms
[Link](): Returns the Pearson product-moment correlation
coefficients, useful for determining the linear relationship between two
variables.
[Link](): Estimates a covariance matrix.
1
[Link](): Computes the frequency histogram of a dataset.
NumPy +3
Handling Missing Data (NaNs)
NumPy also provides functions that ignore NaN (Not a Number) values, which is
crucial for real-world data analysis:
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
Example
import numpy as np
# Create a sample 2D array
data = [Link]([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
# Calculate statistics over the entire array
mean_total = [Link](data)
max_total = [Link](data)
# Calculate statistics along a specific axis (e.g., axis=0 for columns,
axis=1 for rows)
mean_columns = [Link](data, axis=0)
median_rows = [Link](data, axis=1)
print(f"Overall Mean: {mean_total}")
print(f"Overall Maximum: {max_total}")
print(f"Mean of Columns: {mean_columns}")
2
print(f"Median of Rows: {median_rows}")
Finding maximum and minimum of array in NumPy
NumPy [Link]()and [Link]()functions are useful to determine the minimum
and maximum value of array elements along a specified axis.
import numpy as np
arr= [Link]([[1,23,78],[98,60,75],[79,25,48]])
print(arr)
#Minimum Function
print([Link](arr))
#Maximum Function
print([Link](arr))
Output
[[ 1 23 78]
[98 60 75]
[79 25 48]]
Finding Mean, Median, Standard Deviation and Variance in NumPy
Mean
Mean is the sum of the elements divided by its sum and given by the following
formula:
Mean in NumPy
It calculates the mean by adding all the items of the arrays and then divides it by
the number of elements. We can also mention the axis along which the mean
can be calculated.
import numpy as np
a = [Link]([5,6,7])
print(a)
print([Link](a))
Output
3
[5 6 7]
6.0
Median
Median is the middle element of the array. The formula differs for odd and even
sets.
Median in NumPy
It can calculate the median for both one-dimensional and multi-dimensional
arrays. Median separates the higher and lower range of data values.
import numpy as np
a = [Link]([5,6,7])
print(a)
print([Link](a))
Output
[5 6 7]
6.0
Standard Deviation
Standard deviation is the square root of the average of square deviations from
mean. The formula for standard deviation is:
Standard Deviation Equation in NumPy
import numpy as np
a = [Link]([5,6,7])
print(a)
print([Link](a))
Output
[5 6 7]
0.816496580927726
Variance
Variance is the average of the square deviations. Following is the formula for the
same:
4
Variance in NumPy
import numpy as np
a = [Link]([5,6,7])
print(a)
print([Link](a))
Output
[5 6 7]
0.6666666666666666
NumPy Average Function
NumPy [Link]() function determines the weighted average along with the
multi-dimensional arrays. The weighted average is calculated by multiplying the
component by its weight, the weights are specified separately. If weights are not
specified it produces the same output as mean.
import numpy as np
a = [Link]([5,6,7])
print(a)
#without weight same as mean
print([Link](a))
#with weight gives weighted average
wt = [Link]([8,2,3])
print([Link](a, weights=wt))
Output
[5 6 7]
6.0
5.615384615384615
NumPy Percentile Function
It has the following syntax:
[Link](input, q, axis)
The accepted parameters are:
5
input: it is the input array.
q: it is the percentile which it calculates of the array elements between 0-100.
axis: it specifies the axis along which calculation is performed.
a = [Link]([2,10,20])
print(a)
print([Link](a,10,0))
Output
[ 2 10 20]
3.6
NumPy Peak-to-Peak Function
NumPy [Link]() function is useful to determine the range of values along an axis.
a = [Link]([[2,10,20],[6,10,60]])
print([Link](a,0))
Output
[4 0 40]
Summary:
These functions are useful for performing statistical calculations on the array
elements. NumPy statistical functions further increase the scope of the use of the
NumPy library. The objective of statistical functions is to eliminate the need to
remember lengthy formulas. It makes processing more user-friendly.
NumPy provides a suite of efficient aggregate functions for computing summary
statistics on arrays. These functions operate much faster than Python's built-in
equivalents on large datasets because they are implemented in compiled code
NumPy is equipped with the following statistical functions:
1. [Link]()- This function determines the minimum value of the element along
a specified axis.
2. [Link]()- This function determines the maximum value of the element
along a specified axis.
3. [Link]()- It determines the mean value of the data set.
4. [Link]()- It determines the median value of the data set.
5. [Link]()- It determines the standard deviation
6. [Link] – It determines the variance.
7. [Link]()- It returns a range of values along an axis.
8. [Link]()- It determines the weighted average
6
9. [Link]()- It determines the nth percentile of data along the specified
axis.
Key NumPy Aggregate Functions
The primary NumPy aggregate functions include:
[Link](): Computes the sum of all elements in the array.
[Link](): Computes the product of all elements in the array.
[Link]() / [Link](): Finds the minimum value.
[Link]() / [Link](): Finds the maximum value.
[Link](): Computes the arithmetic mean (average).
[Link](): Computes the standard deviation.
[Link](): Computes the variance.
[Link](): Computes the median (middle value).
[Link]() / [Link](): Finds the index of the minimum or
maximum value, respectively.
[Link](): Checks if all elements evaluate to True.
[Link](): Checks if any element evaluates to True.
For most of these functions, a corresponding NaN-safe version exists
(e.g., [Link](), [Link]()) that ignores missing or invalid data.
Examples
Here are examples using the primary functions:
python
import numpy as np
# Create a sample 1D array
arr = [Link]([1, 2, 3, 4, 5, 6, 7, 8])
# Create a sample 2D array (matrix)
M = [Link]([[1, 2, 3, 4],
7
[5, 6, 7, 8],
[9, 10, 11, 12]])
print(f"Original 1D array: {arr}")
print(f"Original 2D array:\n{M}\n")
1. Sum and Product
[Link](): Calculates the total of all elements.
[Link](): Calculates the product of all elements.
total_sum = [Link](arr)
# Or using the array method syntax:
total_sum_method = [Link]()
print(f"Sum of 1D array: {total_sum}") # Output: 36
total_prod = [Link](arr)
print(f"Product of 1D array: {total_prod}\n") # Output: 40320
2. Minimum and Maximum
[Link](): Finds the smallest value.
[Link](): Finds the largest value.
minimum_val = [Link](arr)
maximum_val = [Link](arr)
print(f"Minimum value: {minimum_val}") # Output: 1
print(f"Maximum value: {maximum_val}\n") # Output: 8
3. Mean and Standard Deviation
[Link](): Calculates the average value.
[Link](): Calculates the standard deviation (measure of spread).
average_val = [Link](arr)
std_dev_val = [Link](arr)
print(f"Mean value: {average_val}") # Output: 4.5
8
print(f"Standard deviation: {std_dev_val}\n") # Output: ~2.29
4. Aggregation along a specific axis in 2D arrays
The axis parameter specifies the dimension that will be collapsed during the
operation.
axis=0: Aggregates down the columns.
axis=1: Aggregates across the rows.
# Sum of each column (axis=0)
col_sums = [Link](M, axis=0)
print(f"Sum of columns in 2D array: {col_sums}") # Output: [15 18 21 24]
# Mean of each row (axis=1)
row_means = [Link](M, axis=1)
print(f"Mean of rows in 2D array: {row_means}") # Output: [ 2.5 6.5 10.5]