Data visualizations in python

Histogram
a histogram represents a frequency distribution by means of rectangles whose
widths represent class intervals and whose areas are proportional to the corresponding
frequencies: the height of each is the average frequency density for the interval.
import [Link] as plt
import numpy as np
import pandas as pdx = [Link].random_integers(1, 100, 5)
[Link](x, bins=20)
[Link]('No of times')
[Link]()
Subplotting
With multi dimensional subplots
import [Link] as plt
import numpy as np

fig, ax = [Link](2, 2)
x = [Link](0, 8, 1000)

ax[0, 0].plot(x, [Link](x), 'g') #row=0, col=0

ax[1, 0].plot(x, [Link](x), 'k') #row=1, col=0
ax[0, 1].plot(range(100), 'b') #row=0, col=1
ax[1, 1].plot(x, [Link](x), 'r') #row=1, col=1
[Link]()
Using single for loop to print images
nrows = 2
ncols = 5
image_row_pos = 0
image_col_pos = 0
fig, ax = [Link](nrows, ncols, figsize=(10,7))
for X_test, y_test in sample_generator:
print(X_test.shape)
output = model(X_test.to(device))
output_probs = [Link](dim=1)(output)
predicted_labels = [Link](output_probs,dim=1)
images_list = X_test.cpu().numpy()
y_test_list = y_test.cpu().numpy()
# iterate through the images to plot them
for image_array, actual_label, predicted_label, probs in zip(images_list,y_test_list,predicted_labels,
output_probs):
print(image_row_pos, image_col_pos)
ax[image_row_pos,image_col_pos].imshow(image_array.T)
ax[image_row_pos,image_col_pos].set_title('Actual:{}\nPredicted:{}\nConf:{}'.format(actual_label,
predicted_label.item(),
round([Link](probs).item(),3)
)
)
image_col_pos += 1
if image_col_pos == ncols:
# reset the image_col_pos
image_col_pos = 0
# increase the row pos
image_row_pos += 1

[Link]()

With single dimensional subplots

fig, ax = [Link](1,2,figsize=(10,5))
ax[0].hist(cross_tab['Total'], bins=50)
ax[0].set_xlabel('Number of patches')
ax[0].set_ylabel('Frequency')
ax[0].set_title('Distribution of Number of patches per patient')

ax[1].hist(cross_tab['%_1'], bins=50)
ax[1].set_xlabel('% of IDC sample')
ax[1].set_ylabel('Frequency')
ax[1].set_title('Distribution of IDC sample patches per patient')

Using seaborn
fig, ax = [Link](1,2,figsize=(10,5))
[Link](cm, annot=True, ax=ax[0])
[Link](x=fpr,
y=tpr,
ci=None, markers=thresholds, ax=ax[1])

Dynamically add subplots

fig = [Link](1)
for k in range(Tot):

# add every single subplot to the figure with a for loop

ax = fig.add_subplot(Rows,Cols,Position[k])
[Link](x,y) # Or whatever you want in the subplot

[Link]()

Adding annotations
for x, y, t in zip(x_list, y_list, text_list):
t = '{:.2f}'.format(t)
ax[1].text(x, y, t, rotation=45)

Add legend
x = [Link](0, 10, 1000)
fig, ax = [Link]()
[Link](x, [Link](x), '-b', label='Sine')
[Link](x, [Link](x), '--r', label='Cosine')
[Link]('equal')
leg = [Link]();
Using ggplot in python
Jitter plot
from plotnine import *
For the following dataset

(ggplot(data=details)
+ aes(x='X', y='Y', color='class')
+ geom_jitter()
+ labs(x='X_coordinate', y='Y_coordinate')
+ facet_wrap('patient_id')
+ ggtitle('Distribution of IDC vs. Non. IDC Samples for {} patients < benchmark of Total
slices'.format(sample_size))
+ theme(figure_size=(10,5))
)
Histogram
(ggplot(data=df)
+ aes(x='original_price',fill='search_criteria')
+ geom_histogram()
+ facet_wrap('search_criteria',scales='free_x')
+ ggtitle('Original Price distribution of courses')
+ theme(figure_size=(10,7), panel_spacing_y=0.3, panel_spacing_x=0.5)
)

Common options
1. Define graph colors. aes(x=<X axis column name>,y=<Y axis column name>,fill='search_criteria')
2. Define type of graph: geom_jitter(), geom_histogram()
3. Facet wrapping: facet_wrap(<discrete column name>,scales='free')
4. Giving chart title: ggtitle('Relationship of original price with duration')
5. Adjust figure size and spacing between each facets: theme(figure_size=(10,7),
panel_spacing_y=0.3, panel_spacing_x=0.5)
6. Rotate axis text: theme(axis_text_x=element_text(size=7, rotation=90))
7. Giving labels other than default: labs(x='X_coordinate', y='Y_coordinate')
8. Put text in bar charts: geom_text(aes(label='stat(count)'), stat='count', position='fill')
9. Put text in column charts: geom_text(aes(label='y_column'), position='identity',angle=90)

Data Visualization in R
ggplot
Data:

Code:
ggplot(visualizations, aes(Item_Visibility, Item_MRP, color = Item_Type)) +
geom_point() +
scale_x_continuous("Item Visibility", breaks = seq(0,0.35,0.05))+
 scale_y_continuous("Item MRP", breaks = seq(0,270,by = 30))+
theme_bw() + labs(title="Scatterplot")

Output:
Facet wrapping
ggplot(visualizations, aes(Item_Visibility, Item_MRP)) + geom_point(aes(color = Item_Type)) +
scale_x_continuous("Item Visibility", breaks = seq(0,0.35,0.05))+
scale_y_continuous("Item MRP", breaks = seq(0,270,by = 30))+
theme_bw() + labs(title="Scatterplot") + facet_wrap( ~ Item_Fat_Content, scales=”free”)

Multiple plots using ggplot

library(ggpubr)
# plot the curves
g1 <- ggplot(data=df_pdf, aes(X, Y)) +
geom_point()
print(g1)

g2 <- ggplot(data=df_cdf, aes(X, Fx)) +

geom_point()
ggarrange(g1,g2)
Rotate X axis labels
g2 = ggplot(data=df) +
aes(platform,fill="red") +
geom_bar() +
scale_x_discrete() +
theme([Link].x = element_text(angle = 90, hjust = 1)) +
labs(title="Distribution of users from different 'Platform' who visit the website")

Boxplots

Plot area under the curve

Following are the features we can learn about:
1. Set graph sizes
2. Plot area under the curve with upper and lower x and y limits
## Given attributes
u = 77.0
sigma = 3.4
## Visualize the distribution
# Create a sequence of numbers between 50 and 100 incrementing by 0.01.
x <- seq(50, 100, by = .01)
# Choose the mean and standard deviation as sigma
#to give probability distribution at each point of x
y <- dnorm(x, mean = u, sd = sigma)

## probability that random variable is (c) between 81 and84

## Let the random variable is in range [n1,n2]
n1 = 81; n2= 84
options([Link]=4, [Link]=3)
plot(x,y, type='l',
main='random variable is between 81 and 84 ',
ylab='frequency')
polygon(x=c(n1,x[n1<=x & x<=n2],n2), y=c(0,y[n1<=x & x<=n2],0), col='red')
Draw straight lines in a plot
no_of_trials = 15; no_of_success = 5
# Create a sequence of 15 numbers which denotes our maximum number of trials and each denotes
number of successes
#which are incremented by 1.
x <- seq(0,no_of_trials,by = 1)
# Create the binomial [Link] is the number of trials, prob= Probability of success
y <- dbinom(x=x, size=no_of_trials, prob=0.5)
# set the figure size
options([Link]=4, [Link]=4)
plot(x,y, type='o',
main='Binomial distribution',
xlab='No. of successes',
ylab='Probability of getting x number of successes')

Draw multiple lines in same graph with legends

options([Link]=5, [Link]=5)
plot(x,y1,type="l",col="red", xlab='Amount in INR of claims filed', ylab='Probability of each agent')
lines(x,y2,col="blue")
legend(x=x[400000], y=y1[1], legend=c("Agent 1", "Agent 2"),
col=c("red", "blue"), lty=1:1, cex=0.8)
polygon(x=c(0,x[y2<=y1],x[y2>=y1][1]), y=c(0,y2[y2<=y1],0), col='blue')
Numpy tricks
Create matrix
a = [Link]([[1,2,3],
[4,5,6]
])
Matrix transpose
a.T
array([[1, 4],
[2, 5],
[3, 6]])

Multiply elementwise
b = [Link]([[1,2,3],
[4,5,6]
]
)
a*b
array([[ 1, 4, 9],
[16, 25, 36]])

Multiply vector with all the columns element wise

a = [Link]([[1,2,3],
[4,5,6]
])
b = [Link]([[2],
[4]
])
a*b
array([[ 2, 4, 6],
[16, 20, 24]])

Dot product
a.T
array([[1, 4],
[2, 5],
[3, 6]])

b = [Link]([[2],
[4]
])
[Link](a.T, b)
array([[18],
[24],
[30]])
Matrix multiplication
[Link](a.T, b)
array([[18],
[24],
[30]])

Initialize matrix of zeroes and ones

[Link](shape=(3,2))
array([[0., 0.],
[0., 0.],
[0., 0.]])

[Link](shape=(3,2))
array([[1., 1.],
[1., 1.],
[1., 1.]])
Working with 3d arrays
Think of 3d array of dimension as (n, dim1, dim2) as n 2D arrays of shape (dim1, dim2). Let’s see below:
import numpy as np
a = [Link]([
[[1,2,3],
[4,5,6]
],
[[7,8,9],
[10,11,12]
],
[[13,14,15],
[16,17,18]
]
])
This creates a 3 2x3 matrices as below:

Multiplying a single matrix with all these three

Now we want to multiply a matrix of shape (3x4) with all these 3 matrices.
1. Create the constant matrix
2. result = [Link](a,c) whose shape will be (3, 2, 4)
Element wise cosine similarity
Numpy arrays if done properly can give us element wise cosine similarities map as below without using
native python loops.
1. Consider this matrix of 4 3-dimensional vectors as below
a = [Link]([[1,2,3],
[-1,-2,-3],
[0,1,0],
[4,5,6]
])
Shape of a = (4, 3)
⃑⃑⃑⃑1 = [1,2,3] with ⃑⃑⃑⃑
2. Target is to get cosine similarity of 𝑣 𝑣2 = [−1, −2, −3], ⃑⃑⃑⃑
𝑣3 = [0,1,0], ⃑⃑⃑
𝑣4 =
[4,5,6] and so on.
3. First create the vector norms of all 𝑣 ⃑⃑⃑⃑𝑛 𝑠 = |𝑣
⃑⃑⃑⃑𝑛 |
# create the vector norm
vector_norm = [Link](a,axis=1,ord=2).reshape(-1,1) #
shape = (4,1)

4. Now we have a vector of vector norms denoted by

|𝑣1 |
|𝑣 |
𝑛4,1 = ⌊ 2 ⌋
|𝑣3 |
|𝑣4 |
We want to convert it to following:
|𝑣1 ||𝑣1 | |𝑣1 ||𝑣2 | |𝑣1 ||𝑣3 | |𝑣1 ||𝑣4 |
|𝑣2 ||𝑣1 | |𝑣2 ||𝑣2 | |𝑣2 ||𝑣3 | |𝑣2 ||𝑣4 |
𝑁4,4 = = 𝑛. 𝑛𝑇
|𝑣3 ||𝑣1 | |𝑣3 ||𝑣2 | |𝑣3 ||𝑣3 | |𝑣3 ||𝑣4 |
[|𝑣4 ||𝑣1 | |𝑣4 ||𝑣2 | |𝑣4 ||𝑣3 | |𝑣4 ||𝑣4 |]
N = [Link](vector_norm, vector_norm.T)
5. Now we need to create the element wise dot products.
𝑣
⃑⃑⃑⃑1
𝑣
⃑⃑⃑⃑
𝑎4,3 = 2
𝑣3
⃑⃑⃑⃑
[⃑⃑⃑
𝑣4 ]
We have to convert this to:
𝑣
⃑⃑⃑⃑1 . 𝑣
⃑⃑⃑⃑1 𝑣
⃑⃑⃑⃑1 . ⃑⃑⃑⃑
𝑣2 𝑣 ⃑⃑⃑⃑1 . ⃑⃑⃑⃑
𝑣3 𝑣 ⃑⃑⃑⃑1 . ⃑⃑⃑
𝑣4
𝑣2 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑⃑1 ⃑⃑⃑⃑
𝑣2 . ⃑⃑⃑⃑
𝑣2 ⃑⃑⃑⃑
𝑣2 . ⃑⃑⃑⃑
𝑣3 ⃑⃑⃑⃑
𝑣2 . ⃑⃑⃑𝑣4
𝑑4,4 = = 𝑎. 𝑎𝑇
𝑣3 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑⃑1 ⃑⃑⃑⃑
𝑣3 . ⃑⃑⃑⃑
𝑣2 ⃑⃑⃑⃑
𝑣3 . ⃑⃑⃑⃑
𝑣3 ⃑⃑⃑⃑
𝑣3 . 𝑣 ⃑⃑⃑4
[⃑⃑⃑
𝑣4 . 𝑣 ⃑⃑⃑⃑1 ⃑⃑⃑
𝑣4 . ⃑⃑⃑⃑
𝑣2 ⃑⃑⃑
𝑣4 . ⃑⃑⃑⃑
𝑣3 ⃑⃑⃑
𝑣4 . ⃑⃑⃑𝑣4 ]

dot_product = [Link](a, a.T)

6. Final cosine similarity is:

𝑣
⃑⃑⃑⃑1 . 𝑣
⃑⃑⃑⃑1 𝑣
⃑⃑⃑⃑1 . ⃑⃑⃑⃑
𝑣2 𝑣
⃑⃑⃑⃑1 . ⃑⃑⃑⃑
𝑣3 𝑣
⃑⃑⃑⃑1 . ⃑⃑⃑
𝑣4
|𝑣1 ||𝑣1 | |𝑣1 ||𝑣2 | |𝑣1 ||𝑣3 | |𝑣1 ||𝑣4 |
𝑣2 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑⃑1 𝑣2 . ⃑⃑⃑⃑
⃑⃑⃑⃑ 𝑣2 𝑣2 . ⃑⃑⃑⃑
⃑⃑⃑⃑ 𝑣3 𝑣2 . ⃑⃑⃑
⃑⃑⃑⃑ 𝑣4
|𝑣2 ||𝑣1 | |𝑣2 ||𝑣2 | |𝑣2 ||𝑣3 | |𝑣2 ||𝑣4 | 𝑑
=
𝑣3 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑⃑1 𝑣3 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑⃑2 𝑣3 . ⃑⃑⃑⃑
⃑⃑⃑⃑ 𝑣3 𝑣3 . 𝑣
⃑⃑⃑⃑ ⃑⃑⃑4 𝑁
|𝑣3 ||𝑣1 | |𝑣3 ||𝑣2 | |𝑣3 ||𝑣3 | |𝑣3 ||𝑣4 |
𝑣4 . 𝑣
⃑⃑⃑ ⃑⃑⃑⃑1 𝑣4 . ⃑⃑⃑⃑
⃑⃑⃑ 𝑣2 𝑣4 . ⃑⃑⃑⃑
⃑⃑⃑ 𝑣3 𝑣4 . ⃑⃑⃑
⃑⃑⃑ 𝑣4
[|𝑣4 ||𝑣1 | |𝑣4 ||𝑣2 | |𝑣4 ||𝑣3 | |𝑣4 ||𝑣4 |]

cosine_similarity = dot_product/N
Creating Paths
Suppose we want python to create paths and folders automatically for a given path as follows:
'''
[Link]
'''

from pathlib import Path

path_to_create = './folder1/folder2/folder3'
Path(path_to_create).mkdir(parents=True,exist_ok=True)