Data Science
Python cheat sheet
numpy | panda | matplotlib | scipy | seaborn | tensorflow
Numpy:
Numpy data types:
i - integer
b - boolean
u - unsigned integer
f - float
c - complex float
m - timedelta
M - datetime
O - object
S - string
U - unicode string
V - fixed chunk of memory for other type ( void )
[Link]() : to make array
[Link]([[1, 2, 3], [4, 5, 6]]) : 2d array
[Link]([1, 2, 3, 4], ndmin=<n>) : create array of n dimensions
[Link]((3,3),4,dtype=int) : to make an array full of same value
[Link](5,5) : identity matrix
[Link](n,m) and [Link](n,m) : array of zeroes and ones
[Link]() : to make array for ap
[Link]() : to make array of equal space
[Link] : check no. of dimensions
[Link] : check data type
[Link]('i') : change data type
[Link](arr) : get mean
[Link](arr) : get median
[Link](arr) : get mode
[Link](arr) : get standard deviation
[Link](arr) : get variance
[Link](arr,percentage) : get max value as per percentile
[Link](arr) : get sum to all elements
[Link]() : to make a copy (non updatable)
[Link]() : to make a view (updatable)
[Link] : to get the dimensions
[Link]() : to reshape the array
[Link](arr) : transpose of matrix
reshape(n,m,-1) : -1 for unknown value
reshape(-1) : to convert any array to 1D
for x in [Link](arr): to iterate array in numpy
for idx, x in [Link](arr): to iterate array with index
[Link]((arr1, arr2), axis=1) : to join two array
[Link]((arr1, arr2), axis=1) : to stack two array (also check hstack, vstack and dstack)
np.array_split(arr, n, axis=1) : to divide array in n no. of array (also check hsplit, vsplit and dsplit)
[Link](arr == n) : to find all indices if n
[Link](arr, 7, side='right') : to return the index where a value should be placed to keep the
array sorted
[Link](arr) : to sort array
[Link](arr, dec) : to round up the array elements to the specified decimal places (also check
[Link](arr) and [Link](arr))
[Link](a, b, n) : give n no. of uniform values between a-b
[Link](a, b, n) : give n no. of normal values between a-b
[Link](n) : to print a random number between 0 to n
[Link](n) : random n no. of floats between 0 to 1
[Link](n, size=(m)) : to give m no. of int between 0 to n
[Link]([3, 5, 7, 9], size=(3, 5)) : one out of the choice of give size
[Link]([3, 5, 7, 9], p=[0.1, 0.3, 0.6, 0.0], size=(100)) : probability
Matplotlib
[Link] as plt
[Link](x,y,label=“linename”, marker = ‘’, color = ‘’, linestyle = ‘’, lw=) : plots a graph using arrays
x and y
[Link](x1,y1,x2,y2) : plots a graph of 2 lines
[Link](“labname”) and [Link](“labname”) : to put labels for axis
[Link](“title”) : give title
[Link](n,m) and [Link](n,m) : to set upper and lower limit of graph
[Link](y0,yn,x0,xn) : to set upper and lower limit
[Link]() : show graph
[Link]() : show grid lines in graph (use grid(axis=‘x|y’))
[Link](x,y,z) : plots many graphs (x:no of rows, y: no of columns, z: graph no.)
[Link](title=“title”,loc=“”) : to show legend in the graph
[Link](figsize=(n,m)) : to set graph size
[Link](x, n,color= ‘’) : plot histogram with n no. of bars
[Link](x,y,color= “”, size=,alpha=,cmap=) : plot points on coordinates x,y
[Link](x, y,color=,width=,height=,label= ‘’) : plot bar graph of x values and y no. (use barh() for
horizontal)
[Link](x,y,rotation) : to change the value of array x on x axis with array y
[Link](x,labels=arr,startangle=,explode=,shadow=,colors=, autopct= ‘%.2f%%’) : to plot pie chart
[Link](arr) : to get a boxplot of the data
Seaborn
[Link](tab[‘col’],kde= true, hist= true, bin= n) : to get a displot with n bins
[Link](arr,shade=true) : to display a kernel density estimation plot
[Link](y= ‘colname’, data= tab) : to get a boxplot
[Link](y= ‘colname’, data= tab) : to get a violin plot (plot to check symmetry)
[Link](x= ‘colname1’, y= ‘colname2’, data= tab, kind= ‘’, size= n, color= ‘’) : to get a joint plot
[Link](‘colname1’, ‘colname2’, data= tab, hue= ‘colname3’, col= ‘colname4’, row= ‘colname5’) : to get
a scatter plot between 2 columns
[Link](x= ‘colname1’, data= tab, hue= ‘colname2’) : to get a count bar graph between 2 columns
[Link](x= ‘colname1’, y= ‘colname2’, data= tab) : to get a boxplot between 2 columns
[Link]([Link]()) : to get a heat map of all correlations in the table
[Link](tab) : to get a scatter plot of all pairs of columns in the table
Pandas
[Link]([values], index = [values], columns = [values]) : to create a series like [Link](), but also
like dictionary
[Link] : to display index
[Link] : to display value
arr.value_counts(normalize=False, sort=True, ascending=False, bins=None, dropna=True) : to count each
value
[Link]([Link]) : to apply functions of numpy to each value of panda array
to use with datetime (import datetime as dt)
[Link] : returns the year of the date time.
[Link] : returns the month of the date time.
[Link] : returns the day of the date time.
[Link] : returns the quarter of the date time.
[Link] : returns the day of the week.
[Link].weekday_name : returns the name of the day of the week.
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
pd.to_datetime(‘date and time’) : to give date and time in a proper format
pd.date_range(start= ‘date1’, end= ‘date2’, freq= ‘D’) : to display dates between two dates
[Link]() : to drop empty values
[Link](value) : to fill empty values with given value
[Link]({‘value’: ‘newvalue’}) : to replace/map values
[Link](r“addr”, sep=“seperator”,names= <array of col names>,usecols=[name or index],
dtype={“colname” : type}) : to read a table in given address
[Link](n) and [Link](n) : to give first or last n number of columns
tab[‘colname’] : to access the give column (can be used to create a new column, like tab[‘newcol’] =
tab.col1 + ’,’ + tab.col2)
[Link] : to access the give column
[Link] : no. of rows and columns
[Link]() : names of unique columns
[Link]() : number of unique columns
[Link] : all columns
[Link](include= ‘all’) : describe data in table (use describe(include= ‘all’) to get all data)
[Link]() : describe table
[Link] : data type of each column
[Link].value_counts(normalize=True, dropna=True) : to count number of rows of the column
[Link](kind= ‘’,x=“namexplot”,y=“nameyplot”) : to plot on graph (use with matplotlib)
tab[colname].mean() : to get mean { also look for mode(),median(),max() }
[Link]() : correlation of each column with themselves
[Link]([‘functions’]) : to run functions aggregately on a column (like mean, median, count)
[Link](0.25|0.75) : to get a quantile
[Link](‘colname’) : to group entries by column name
[Link](tab.col1,tab.col2,margin=True) : to get crosstable between 2 columns
[Link](tab2, on= ‘colname’, how= ‘’) : to merge to tables on common columns (how= left, right, inner,
outer)
[Link]([tab1, tab2], axis=0) : to concatenate 2 tables
tab.set_index(‘colname’) : to set a column as index
[Link](dict, ignore_index=True, sort=False) : to append data in table using a dictionary.
[Link][len([Link])]=list(dict[0].values()) : to add data in tab
[Link][n]=list(dict[0].values()) : to replace data using a dictionary
[Link](**{‘newcol’ : arr) : to add a new column using an array
[Link](columns={“oldname”: “newname”},inplace = True) : change column names
[Link](type) : to change type of values
[Link](‘colname’,axis = 1, inplace= True) : to drop a column
tab.drop_duplicates(keep= ‘first’, inplace= True) : to drop duplicate value rows
[Link](index ,axis = 0, inplace= True) : to drop a row at the given index
[Link](subset= [“col”], how=“value”) : to drop all rows with any empty data {values: all, any}
tab[‘col’].interpolate(method=‘linear’) : to fill none value with estimated values
[Link]() : to check for duplicate values
[Link]() : to check for null values (also check [Link](), [Link]() and [Link]())
tab[‘colname’].fillna(value, inplace=true) : to replace null value to given value
[Link](to_replace= “val1”,replace= “val2” ) : replace val1 with val2 anywhere in the table
tab[‘col’].replace({‘val’: ‘valnew’}) : to replace a value in a row
[Link].sort_values(ascending= “true”).head() : to sort values
tab.sort_values(‘colname’,ascending= “true”).head() : to sort by a column
tab[[Link]>=200].colname : conditional statement
[Link][index or col conditional, ‘colname’] : to show selected column
[Link][[index or rows],[index of columns]] : to show selected rows and columns are
[Link]([array of values]) : to select rows with specific values
[Link]() : to use a string method on rows
pd.get_dummies([Link], prefix=“value” prefix_step= ‘_’, drop_first=False) : to show k values with k-1
entries
tab = pd.get_dummies(tab,columns=[‘col’]) : same as one hot encoder
Scipy
from [Link] import bernoulli
[Link](p=0.5, size=n) : to print a Bernoulli series of size n and p
Sklearn
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(tab,arr, test_size = 0.3, random_state = 0): to get
data for training and testing
[Link](x_train, y_train): to get score of accuracy
from sklearn.feature_selection import VarianceThreshold
sel = VarianceThreshold(threshold=0.01)
[Link](tab) : to get data with the given threshold
from sklearn.linear_model import LinearRegression
lr = LinearRegression()
[Link](x, [Link]) : to get a linear regression (x can be multiple columns of the table)
[Link](x, [Link]) : to get r2 score (x can be multiple columns of the table)
y_predict = [Link](x) : to predict values
lr.coef_ , lr.intercept_ = m, c #y = mx + c (m is array if multiple regression)
from sklearn.linear_model import LogisticRegression
classifier = LogisticRegression()
[Link](x_train, y_train) : to get logistic regression
y_pred = [Link](x_test)
from [Link] import SVC
classifier = SVC(kernel = 'rbf' | ‘linear’ | ‘poly’, c = [0.01,0.1,1,10], gamma = [0.01,0.1,1])
[Link](x_train, y_train) : to get svm
y_pred = [Link](x_test)
from [Link] import KNeighborsClassifier
classifier = KNeighborsClassifier(n_neighbors=n) : to get with n neighbors
[Link](x_train, y_train) : to get K neighbor Classification
y_pred = [Link](x_test)
from [Link] import DecisionTreeClassifier
classifier = DecisionTreeClassifier(criterion = ‘gini’ | ‘entropy’, max_depth = [2,3,4],
min_samples_split = [int])
[Link](x_train, y_train) : to get decision tree classification
y_pred = [Link](x_test)
from [Link] import RandomForestClassifier
clf_rf = RandomForestClassifier(criterion = ‘gini’ | ‘entropy’, n_estimators = [int], max_depth = [3,4],
min_samples_split = [5,7], random_state=43)
clr_rf = clf_rf.fit(x_train,y_train) : to get random forest
y_pred = clr_rf.predict(x_test)
from xgboost import XGBClassifier
classifier = XGBClassifier()
[Link](x_train, y_train) : to get XGB classification
y_pred = [Link](x_test)
from [Link] import SVR
clf = SVR()
[Link](X_train, y_train) : to run svr
predicted = [Link](X_test)
import xgboost as xgb
clf = [Link]()
[Link](X_train, y_train) : to get XGB regression
predicted = [Link](X_test)
from [Link] import DecisionTreeRegressor
clf = DecisionTreeRegressor()
[Link](X_train, y_train) : to get decision tree regression
predicted = [Link](X_test)
from [Link] import RandomForestRegressor
clf = RandomForestRegressor()
[Link](X_train, y_train) : to get random forest regression
predicted = [Link](X_test)
from [Link] import mean_absolute_error, mean_squared_error, r2_score
mean_absolute_error(y,y_predict) : give mean absolute error between actual and predicted values
mean_squared_error (y,y_predict) : give mean squared error between actual and predicted values
[Link](mean_squared_error(y,y_predict)) : give root mean absolute error between actual and predicted
values
r2_score(y,y_predict) : give R2 score between actual and predicted values (same ar [Link](x,[Link]))
from [Link] import confusion_matrix, classification_report
cm = confusion_matrix(y_true,y_predict) : gives confusion matrix
[TP FP]
[FN TN]
cm = classification_report(y_true,y_predict) : gives report of classification
from [Link] import f1_score, accuracy_score
ac = accuracy_score(y_test,clf_rf.predict(x_test)) : to get accuracy score
from sklearn.feature_selection import SelectKBest, chi2
# find best scored 5 features
select_feature = SelectKBest(chi2, k=5).fit(x_train, y_train)
print('Score list:', select_feature.scores_) : to get best scores
from sklearn.feature_selection import RFE
# Create the RFE object and rank each pixel
clf_rf_3 = RandomForestClassifier()
rfe = RFE(estimator=clf_rf_3, n_features_to_select=5, step=1)
rfe = [Link](x_train, y_train) : to get RFE ranks
from sklearn.feature_selection import RFECV
# The "accuracy" scoring is proportional to the number of correct classifications
clf_rf_4 = RandomForestClassifier()
rfecv = RFECV(estimator=clf_rf_4, step=1, cv=5,scoring='accuracy') #5-fold cross-validation
rfecv = [Link](x_train, y_train)
print('Optimal number of features :', rfecv.n_features_)
print('Best features :', x_train.columns[rfecv.support_]) : to get rfecv
from [Link] import LabelEncoder
lb = LabelEncoder()
tab[‘col’] = lb.fit_transform(tab[‘col’])
for i in [Link]:
tab[i] = lb.fit_transform(tab[i]) : to convert any entry of values to int form
from [Link] import OneHotEncoder
ob = OneHotEncoder()
for i in [Link]:
tab[i] = ob.fit_transform(tab[i]) : to convert any entry of values to int form columns
from [Link] import TSNE
tn = TSNE(n_components=2, random_state=0)
xn = tn.fit_transform(tab)
from sklearn.model_selection import GridSearchCV
gs = GridSearchCV(model, parameters, cv=n, scoring= ‘f1_macro’)
[Link](X_train,Y_train)
from [Link] import KMeans
import numpy as np
X = [Link]([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])
kmeans = KMeans(n_clusters=2, random_state=0).fit(X) : to get kmean clusters
kmeans.labels_ : labels for n clusters
ypredict = [Link](x)
from [Link] import PCA
pca = PCA(n_components=n)
tabpca = pca.fit_transform(tab) : to standardize a table into an array of n components
from [Link] import StandardScaler
sc = StandardScaler(n_components=n)
tabsc = sc.fit_transform(tab) : to standardize a table into an array
(scaled value = actual-mean/standard deviation)
from [Link] import MinMaxScaler
mns = MinMaxScaler((n,m) : to scale values between n and m
tabmns = mns.fit_transform(tab)
from [Link] import normalize
tabnorm = normalize(tab, norm= ‘l1’) : to normalize a table into an array
ss=[]
k=range(1,20)
for i in k:
km = KMeans(n_clusters=i)
[Link](x)
[Link](km.inertia_)
[Link](k,wss)
[Link]() : Elbow method
from [Link] import silhouette_score
silhouette_score(tabpca, KMeans(n_clusters=n).fit_predict(tabpca)) : to calculate silhouette score
(higher the score, better the clusters)
from [Link] import MeanShift
ms = MeanShift(bandwidth = 2, bin_seeding = True)
[Link](x) : to get Mean Shift clusters
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
gscv = GridSearchCV(classifier, param_grid = dict)
[Link](x_train,y_train)
print(gscv.best_params_) : to get the best parameters for a classifier
print(gscv.best_estimator_) : to get the best estimators for a classifier
[Link](x_test)
rmcv = RandomizedSearchCV (classifier, param_distributions = dict)
[Link](x_train,y_train)
print(gscv.best_params_) : to get the best parameters for a classifier
print(gscv.best_estimator_) : to get the best estimators for a classifier
[Link](x_test)
from sklearn.model_selection import LeaveOneOut, KFold, StratifiedKFold
n = [Link]([1, 2, 6, 3, 2, 6, 8, 7, 3, 5, 8, 4, 3, 7, 3, 7])
km = KFold(n_splits=4, shuffle=False)
for train, test in [Link](n):
print("Train: ", n[train], " Test: ", n[test])
n = [Link]([1, 2, 6, 3, 2, 6, 8, 7, 3, 5, 8, 4, 3, 7, 3, 7])
y = [Link]([1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0])
sk = StratifiedKFold(n_splits=4, shuffle=False)
for train, test in [Link](n, y):
print("Train: ", n[train], " Test: ", n[test])
lt = LeaveOneOut()
for train, test in [Link](n):
print("Train: ", n[train], " Test: ", n[test])