1

Use freely, cite me

Python intro notes (hopefully helpful for people coming from R )

Notes from openHPI pythonjunior2020 , Python for Digital Health and personal learning. RefCards search
By Berry Boessenkool, berry-b@[Link], [Link], Jan-Mar 2021 RefCard example
!!! means this will fry your brain if you're used to R. Especially subsetting with positions will be horrible.
Most python interpreters don't print unless excplicitely stated. print() is mostly left away here for brevity.
Download & install, hints for Windows users. tutorial, standard libraries, language reference, documentation.
IDEs
PyCharm Good for scientific development, but slow in startup
VScode (Visual studio code) increasingly popular, supports multiple languages, e.g. R
IDLE Installed by default, not suitable for large projects
More: [Link]/python-programming/ide , [Link], Jupyter Manual

Syntax
function(arg, "txt", 'single quotes', 77.86) # comment
""" multi-line comment
with line breaks """
7*6 ; 21+21 # semicolon possible, but not good practice. here for effective space use
9 // 2 ; 20 % 7 ; 3 ** 2 # ≈ 9 %/% 2 ; 20 %% 7 ; 3^2 in R ([Link], modulo)
a = 5 ; a += 1 # short for a = a + 1 ; a *= a+2 # short for a = a * (a+2)
variable_name = "value" # naming convention: lowercase, underscore
NameError: non-existing objects - List of errors
Variable names cannot start with numbers, Python is case sensitive
Reserved statements (keywords) like else cannot be used as variable name
SyntaxError: often forgotten brackets or colons (e.g. in loops)
Method = function for an object class, e.g. [Link]
Linter: program to analyze code style and determine structural problems
(pointless lines of code, potentially overwriting variable names, etc)

Collections (Arrays)
type example changeable ordered indexed notes
list [1,3] yes yes yes -
tuple (1,2) no yes yes -
set {1,4} no, but add no no no duplicates
dictionary {"a":7, "b":3} yes no* by key no duplicates
Data types
integer, float, string, boolean (True/False), complex (2+1j) !!!
isinstance(7.5, int) # check for class
isinstance("Hello", (float, int, str, list, dict, tuple)) # one of types
value = input("Enter number: ") # interactive input ≈ readline("Enter num: ")
print("Type is: ", type(value)) ; type(int(value)) ; type(float(value))
value + 7 # ValueError if keyboard input was charstring
value = float(input("Give a number: ")) # read keyboard input and convert

Lists
list = [7, -4, 9, 1, 2, 3, 9] ; len(list)
list[0] # first element !!! ; list[1] # second element !!!
list[-1] # last element
list[-2] # second-to-last element list[5:-2] # range from left + right
list[2] = "newvalue" # overwrite third element, mixed data types possible !!!
list[2:5] # elements 3,4,5 exclusive at the right end !!!
list[4:] ; list[:6] # slicing: fifth till last element ; first till sixth
: is not an operator outside subsetting !!! list = [] # empty list
bar = list
[Link](66) # mutable object: changed even without re-assigning !!!
id(list) == id(bar) # both with 7,-4, "new value",1,2,3,9,66 !!!
 2

[Link]() # remove last element (+ return it invisibly)

[Link](index) # remove (+ return) selected element
del(list[index]) # only remove element at given index
[Link](5, "new_val") # insert at given position
[Link](9) # location of 9 ; [Link](9) # remove first instance of 9
9 not in list # check for non-presence, returns a boolean ≈ ! 9 %in% vec in R
[Link]() ; [Link]() ; [Link](reverse=True)
list = [1,2,3,[31,32,33],4] # Nesting possible
list_with_charstrings[3][7] # eighth letter in fourth element
one_list.extend(another_list) # ≈ one_list <- c(one_list, another_list) in R
one_list + another_list # ≈ c(one_list, another_list) in R
one_list * 2 # ≈ rep(one_list, 2) in R

Dictionaries
*: since python version 3.6/3.7, dictionaries are ordered
dict = {'name': "Berry", 'age': 31} # keys (name, age) must be unique
len(dict)
dict ['name'] = "new_value" # key + value = 'pair' dict['new_key'] = 42
f"Hi, {dict['name']}" # fstring double and single quotes cannot be mixed
print("Hello, {name}" .format(name=dict['name']) )
[Link]('NAME', "value_if_key_not_present")
del(dict['age']) # delete pair (entire entry). del d['k'] brackets optional!!
[Link]() ; [Link]() ; [Link]()
list([Link]()) # -> list with tuples -> very high memory usage!
the_age = [Link]('age') # KeyError: key no longer in dict
other_dict = [Link]() ; [Link]() ; [Link](another_dictionary)

Sets
s1 = {1,2,3,4,5} ; s2 = {3,4,5,6,7,8} ; {} empty dict ; set() empty set
s1 | s2 ; s1 & s2 ; s1 - s2 # union ; intersection ; difference

Charstrings
"Hey" + "You there" # + operator to concatenate (chain) strings
3*"Hi" # -> "HiHiHi" # * operator to repeat strings
len("char string") # ≈ nchar in R. Not the same as len(some_list) !!!
print("Hey", "You there", sep=" ", end="--\n—")
charstring = "Hi this is a text. with words"
"this" in charstring # ≈ grepl("this", charstring) in R
charstring[0] # ≈ substr(cs, 1,1) in R. Not the same as some_list[0] !!!
charstring[1] = "b" # not possible: unlike lists, strings are immutable
charstring[5:-2] # subset region
charstring[300] # IndexError: subsetting outside of existing range
[Link]() # split at spaces. immutable - does not change object !!!
[Link](".") # split at periods. The default includes \n as space.
"_".join(["list", "of", "words"]) # ≈ paste(wordvec, collapse="_") in R
" char string ".strip() # strip white space (or given symbols) on both sides
"CharString".lower() # ≈ tolower("CharString") in R
"CharString".startswith("Ch") # ≈ startsWith("CharString", "Ch") in R
"CharString".count("r") ; max("CharString")
"CharString".find("tri") # gregexpr("tri", "CharString") in R
"Chars".replace("Ch", "K") # ≈ gsub("Ch", "K", "Chars", fixed=TRUE) in R
re module for regular expressions aka. wildcards (see section Packages):
import re ; [Link]('[xyz]', 'K', "abycd") # ≈ gsub("[xyz]", "K", "abycd")
F-string placeholder (since Python 3.6). Inline arithmetics posible:
person="Berry" ; f"{person} is a nice guy with {5+5} fingers"
print("%d %s cost $%.2f" % (6, "bananas", 1.74)) # -> 6 bananas cost $1.74
print("{0} {1} cost ${2}" .format(6, "bananas", 1.74))
 3

Packages
pip to install packages e.g from [Link] (PYthon Package Index) ≈ CRAN for R. pip install pandas
Anaconda to install binary packages (also R) from their cloud. Anaconda Prompt: conda install pandas ; conda list
Popular packages: Data science: pandas, numpy, Machine learning: tensorflow, pytorch, Statistical analysis: scipy,
Web application: django, Plotting: matplotlib, seaborn , package version management: virtualenv
ImportError: wrong library/module/script name, non-existing objects
from library import * # all functions -> bad practice: object origin unclear
from library import function1, function2 # specific function(s)
import library then you can use [Link](…)
import library as lib then you can use [Link](…)
from random import random, randint # random,math,etc come with python
random() # float between 0 (inclusive) and 1 (exclusive!!!), ≈ runif() in R
randint(1, 6) # int between start and end, including these
import os # os is a module in the standard library, no installation needed
print([Link]()) # ≈ getwd() in base R ; [Link]() # ≈ setwd()
from math import pi

Read files
If at [Link](), there is [Link] with age = 45, we can use:
from mydataset import age # to then use age + 2
from mydataset import * # to import all ≈ source("[Link]") in R
import mydataset # to then use [Link] + 2
print(dir(mydataset)) # list the objects in the module
import os, sys ; fname = [Link]([Link][0],"[Link]") # for wd
with open(fname) as f: # with closes the connection (even in case of error)
content = [Link]()#.splitlines() # ≈ readLines("[Link]") in R

Logicals
< ; <= ; > ; >= ; == ; != ; and ; or ; not # comparison / logical operators
7 < 8 ; "9" < "A" ; "A" < "B" ; "A" < "a" ; "a" < "b" !!order in R: "a" < "A"
7>1 & 6>1 in R, Python needs: (7>1) & (6>1), Py reads 7 > 1&6 > 1 and 1&6=0

Conditional code execution Loops

IndentationError: wrong number of spaces at the beginning of a line
if cond: for number in (0,1,2,3): # or in range(4)
do(1) print(number) # range(8, 0, -2)
do(2) # range stop exclusive!!!
else: # convention for unused index variable:
do(3) for _ in range(8): # or _var
print("stuff")
if cond1: for a,b in ( (1,4), (5,7), (6,9) ):
do(1)
print(f"a={a}, b={b}, a+b={a+b}")
elif cond2:
do(2) while cond:
else: run_things()
do(3) if(cond2):
break
if cond1 and (cond2 or cond3): # continue ≈ next in R
print("stuff")
enumerate("hello") ; iter # iterators
result = []
for item in item_list:
new_item = do_something_with(item)
[Link](new_item)
result = [do_something_with(item) for item in item_list] # list comprehension
out = [] for word in charstring_list if word[0] == "B": [Link](word)
out = [x for x in charstring_list if x[0] == "B"]
≈ char_vec[substr(char_vec,1,1)=="B"] in R # not vectorizable in Python !!!
 4

Write custom functions

def greet(name, time="morning"): # name+time are parameters
return f"Hello {name}! Good {time}." # return exits function execution
# explicit return is needed !!! else a function returns None (≈ NULL in R)
greet("Berry") # Berry+evening are arguments
greet("Berry", "evening") # parameter=argument ≈ argument=value in R

def change_object():
global ab
ab = 2
ab = 1 ; ab ; change_object() ; ab # is now 2

multiply = lambda x,y: x*y # single expression function on one line of code
multiply(7, 3)

Multiple assignment
def myfun(x, y): # related: swap two variables: a, b = b, a
return x*2, y*2
a, b = myfun(3, 4) # two int objects, each with a single value
c = myfun(3, 4) # tuple object with (6, 8)

list(map(len, ["abcdef","ab","abc"])) # sapply(c("abcdef","ab","abc"), nchar)

Error management
import traceback
try:
7 + "2" # code that might fail. int("seven") would give ValueError
except TypeError: # TypeError: wrong data type for operator or function
print("That mixed charstrings and numbers")
except Exception: # print instead of error
print("another error occured: ", traceback.format_exc() )
else:
do("stuff")

Write custom class

class Person:
pass # Placeholder for future code. A class body may not be empty.
p1 = Person() # create object instance
[Link] = "Berry" ; [Link] = 31 # add attributes
class Person: # class attributes, generate w/ constructor
def __init__(self, name, age): # initialize (assign values) to data members
[Link] = name # of the object when Person() is called
[Link] = age
if name=="forbidden":
raise Exception("Name cannot be 'forbidden'") # ≈ stop("msg") in R)
def can_watch_movie(self): # class methods
if [Link] >= 18:
return "Sure, watch it" # self represents object of class Person,
else: # always first arg to __init__
return "Too young, sorry"
p2 = Person("John", 25) ; [Link] ; p2.can_watch_movie() # instantiation
p2.__dict__ # dictionary of all given parameters and arguments
p2 = Person("forbidden", 25) ;

turtle
package to draw figures on plot range -200:200
forward(nsteps), right(degrees), goto(x,y), penup(), pendown(),
shape("turtle"), register_shape(), pencolor("yellow"), bgcolor(),
fillcolor(), begin_fill(), end_fill()
 5

Count table
colors = ['red', 'blue', 'blue', 'yellow', 'blue', 'red', 'green']
import collections
[Link](colors).most_common(6) ≈ sort(table(colors))[1:6] in R

Numpy: computationally efficient numerical arrays. pip install numpy

import numpy as np ;
[Link]([1,2,3,4,5,6]) # 1D array. type: [Link]
ar = [Link]([[1,2,3,4], [5,6,7,8]]) # 2D array from list of lists
[Link](10, size=(3,4)) # random integers 0-9, 3 rows, 4 columns
Attributes: accessed without brackets (methods with brackets):
[Link] ; [Link] ; [Link] # ≈ length(dim(ar)); dim(ar); length(ar) in R
[Link] # numpy-internal data type [Link] ; [Link] # in bytes

ar1 = [Link](10) # sequential 1D array

ar1[4] ; ar1[-1] # fifth and last element of 1D array
ar1[start:stop:step] # general subsetting of 1D arrays
ar1[:5] ; ar1[4:] ; ar1[4:7] ; ar1[::2] / ar1[1::3] # every other element
ar1[::-1] # all elements, reversed. Works for lists & charstrings as well

ar[:2, :3] ; ar[:, 5] # 2 rows, 3 columns. all values in sixth column

ar[0] # first row, not first column !!!# as in R, not recommended!
ar[2,0] = 3.1415 # change single element at row three, column 1 of 2D array
# If array is integer, float is silently truncated to 3: Downcasting !!!
ar_sub = ar[:2, :2] ; ar_sub[0,0] = 99 # changes both ar_sub and ar !!!
ar_sub = ar[:2, :2].copy() ; ar_sub[0,0] = 42 # does not change ar
ar = [Link](1,10) ; grid = [Link]((3,3)) # 1D array to 2D array
ar[[Link], :] # 2D, 1 row. Both do not change ar.
[Link]((ar1, [67,68,69]))
[Link]([grid, grid]) # ≈ rbind(grid, grid) in R
[Link]([grid, grid], axis=1) # ≈ cbind(grid, grid) in R
[Link](); [Link](); [Link]() # the same, d for depth (3rd dimension)
s1,s2,s3 = [Link]([1,2,3,4,5,6,7,8,9], [3,5]) ; [Link]; [Link] for 2D

Ufunc (Universal functions operating on full array)

%timeit compute_long_thing(big_array). # %timeit by Ipython
Numpy enables very fast vectorized operations: 1.0/ar ; ar1/ar2 ; ar>=3.
ar + 5 # element-wise operation: broadcasting (≈ recycling in R)
[Link]((3,4)); [Link]() # arrays full of 1 (or 0)
ar3x3 + ar3 -> ar3x3 # ar3 (1D)repeated for each row of ar3x3 (2D).
ar3 + ar3x1 -> ar3x3.
angles = [Link](0, [Link], 3) # ≈ seq(0, pi, len=3) in R
[Link](); [Link](); np.log10(); [Link]()

np.count_nonzero(ar<6) or [Link](ar<6) # ≈ sum(ar<6) in R

[Link](ar<6, axis=1) # ≈ rowSums(ar<6) or apply(ar, 1, sum) in R
[Link]() ; [Link]() # can be called without np. as well
[Link](ar<6, axis=0) # columns ≈ apply(ar, 2, any) in R -> axis != MARGIN !!!
[Link](ar<6, axis=1) # rows ≈ apply(ar, 1, any) in R
ar[ar < 6] # reduces dimension (e.g. 2D to 1D)
[Link](); [Link](axis=0);[Link](axis=1) #≈ mean(ar); colMeans(ar);rowMeans
np. corrcoef(ar1, ar2) ; [Link]() ; [Link]() ; [Link]() ;
[Link](ar) # ≈ sd(ar, [Link]=TRUE) in R ; [Link](ar) ; [Link](ar)
[Link](ar)
[Link](5, 100) # 100 random numbes from poisson distribution
[Link](mu, sigma, 100)
 6

Pandas: panel data analysis, builds on numpy. API docs. pip install pandas
Series (column) with axis labels and DataFrame of Series
import pandas as pd ;
[Link](data=list_of_vals, index=list_of_strings) # data can be numpy array
s1 = [Link](dictionary) ; s1.to_list() ; s1.to_dict() ; [Link] ;
s1 = [Link]([1,2,3,4], index=["A","C","D","E"])
s2 = [Link]([1,2,5,4], index=["A","B","C","E"]) ; s1["A"] # subset by name
s1 + s2 # returns: A:2, B:NaN, C:7, D:NaN, E:8 # Operations per index

df = [Link](randn(5,4), index='A B C D E'.split(), # ≈ rownames in R

columns='W X Y Z'.split()) # ≈ colnames in R
[Link]; [Link] ; [Link] ; [Link] ; [Link] ;
[Link]() ≈ str(df) in R; [Link]
df.select_dtypes(include='number') # see dtypes # does not change df
Select columns Select rows Select elements
df["colname"] [Link]["rowname"] [Link][2, 5]
[Link] [Link][2] [Link]["rname", "cname"]
df[["col1","col2"]] [Link][0:3] # first 3 r [Link][3:5]
[Link][:, -1] # last C [Link][-1] # last row
[Link][:, 1:3] # C 2+3 df[ [Link] < 15 ] iloc for index location
Missing values
df1 = [Link]({'A':[1,2,[Link]],
'B':[5,None,[Link]],
'C':[1,2,3]})
[Link]() # ≈ [Link](df) in R
[Link]().sum() # ≈ apply(df, 2, [Link]) in R # number of Nas per column
[Link]().sum(axis=1) # number of Nas per row
df1[df1["B"].notna()] # ≈ df[, ] in R
[Link]() # ≈ [Link](df) in R # see also [Link](axis=1)
[Link](thresh=2) # at least 2 finite numbers needed to be kept
[Link](value='missing') # replace NA with "missing" # value=0 possible
[Link](value=[Link]()) # Replace with mean value of column
df1[[Link]()].[Link]() # ≈ rownames(df)[[Link](df$A)] in R
rows_with_nan = [index for index,row in [Link]() if [Link]().any()]
[Link][[Link]().sum(axis=1) > 0].tolist() # the same, more readable
Combining dataframes
[Link]('Age_group').mean() # .min() ; .count() # mean only for numerics
[Link](df1, df2, on='key_column', how="outer") # on=['key1','key2']
how: outer, inner, left, right # ≈ merge( all=T) all=F, all.x=T, all.y=T in R
[Link](df2, how="outer") # cbind by rownames
[Link]([df1, df2], axis=0) # outer by default. ≈ rbind in R, but expands
[Link]([df1, df2], axis=1) # inner by default. ≈ cbind in R, but expands
Pandas misc
[Link]() # .nunique() ; [Link].value_counts() # ≈ table(df$col) in R
df = [Link](new_col = lambda x: ([Link]*1000)) # df["new_col"]= [Link]*1000
[Link](lambda x : x/100, axis=1) ≈ apply(df, 1, function(x) x/100) in R
([Link] > 6) & [Link] # &, |, !=, ==, ~ (not), >, <, >=, <=
df = df.sort_values(by='colname') # ≈ df = df[order(df$colname)] in R
df.sort_values(by='colname', inplace=True) # modify df directly
dfcp = df ; dfcp[5,2] = 42 # changes df as well, dfcp is only a pointer to df
dfcp = [Link]() # as usual :)
df.pivot_table(index=['c1','c2'], columns=['c3'])
[Link](df.c1, df.c2); [Link](index=x, columns="Count") # ≈ table(x)
pd.read_csv ; pd.read_excel ; pd.read_html ; df.to_csv() ; pd.to_exel()
[Link]() ≈ summary(df) in R ; [Link]() ; [Link]()
 7

Statistics
import pandas as pd ; import numpy as np ; import scipy ; import statistics
[Link](x)
[Link]() # [Link] excludes nan by default ; [Link]()
[Link]() safer than [Link]() with nans
[Link](x, n=4) # in Python >3.8
[Link]([0, 0.05, .25, .5, .75, .95, 1]) ; [Link](x)
[Link](x) ; [Link]() ; [Link](x, ddof=1) ; [Link]()
[Link](x) ; [Link](x) # Py>3.8 ; [Link](x)
[Link](x) ; [Link](axis=0, skipna=True)
[Link](x, 'norm') # Kolmogorov-Smirnov test for normality
[Link](x)[1] # Shapiro-Wilk test for normality
corcoef,pvalue = [Link](x, y) ; [Link](method="pearson")
[Link].ttest_1samp(x, popmean=182) # "is mean of x = 182?"
[Link].ks_2samp(x,y).pvalue # to answer "is x different from y?"
[Link].ttest_ind(x,y) # independent T-test "is x diff from y?"
[Link].ttest_rel(x,y) # paired T-test, when x and y related
[Link](x,y, alternative="greater") # one-sided Mann-
Whitney-U Wilcoxon Rank test (≈ T-test for non-normal distribution shape)
[Link].chi2_contingency([x,y]) # categories. can take [Link] output
[Link](f_obs=observed, f_exp=expected) # Goodness of fit test
[Link].f_oneway(x,y,z) # ANOVA "are x, y and z the same?"
mod=[Link]('y ~ C(x1)+C(x2)+C(x1):C(x2)', data=df).fit()
[Link].anova_lm(mod, typ=2) # kind of ≈ lm(y~x1+x2+x1:x2) in R

Data visualisation with matplotlib

import [Link] as plt
%matplotlib inline # in notebook ; [Link]() in last line in other editors
[Link](x,y) ; [Link](x) ; [Link](data, vert=True)
[Link](x, y, 'r--') # 'r--': red dashed line ; 'g*-': green stars + line
[Link]('X axis title') ; [Link]('Plot title') ;
[Link]("[Link]", dpi=200) # save to disc as png, pdf, svg, etc
fig = [Link]() ; ax = fig.add_subplot(1,1,1) # object-oriented API
[Link](x, x**3, label="x**3", linewidth=3, color="blue", alpha=0.5)
[Link](x, x**2, label="x**2", linestyle="-.", marker="s")
[Link](loc='lower right') (ax is an axes, i.e. a figure window)
Multipanel plots
[Link](1,2,1) # 1 is figure number. ≈ par(mfrow=c(1,2)) in R
[Link](x, y) ; [Link](1,2,2) ; [Link](y, x)
fig = [Link](figsize=(8,4), dpi=100) # nested plots
window = fig.add_axes([0.1,0.1, 0.8,0.8]) # bottomleft + proportion of canvas
[Link](x, y) ; window.set_ylabel("ylab") ; window.set_xlim([0,20])
fig,ax = [Link](1,2) # ≈ par(mfrow=1:2, mar=c(3,2,1,0.5)) in R
ax[0].plot(x,y) ; ax[1].plot(x,y);ax[1].set_ylabel('y') ; plt.tight_layout()

Data visualisation with seaborn (builds on matplotlib, nice with pandas df)
import seaborn as sns # histogram with kernel density estimate:
[Link](data=df, x='column', kde=True, bins=30) # distributional summary
[Link](data=df, kind="swarm", x="catcol", y="numcol", hue="catcolumn")
kind="box"; kind="bar" # categorical data; swarmplot, boxplot, barplot
[Link](…, palette="Set2") # mypal={cat1:"g", cat2:"b"} color palettes
[Link](data=df, kind="kde", diag_kind="hist", hue="category",
corner=True, diag_kws=dict(fill=False), …)
sns.set_theme() # ≈ par(…) in R
[Link]() # relationship scatterplots
 8

Machine learning - classification, regression, clustering, dimensionality reduction, model selection

pip install scikit-learn ; import sklearn # note different names for install / import
Data prep
y = [Link] ; x = [Link]('target', axis=1)
x_train, x_test, y_train, y_test = sklearn.model_selection.train_test_split(
x, y, test_size=0.3, random_state=12) # 70% for training, seed for shuffle
scaler = [Link]() # see also: minmax_scale
x_train_norm = scaler.fit_transform(x_train.values)
x_test_norm = [Link](x_test.values)
Multivariate linear regression
logreg = sklearn.linear_model.LogisticRegression(max_iter=1000)
[Link](x_train, y_train) ; logreg_pred = [Link](x_test)
[Link](y_test, logreg_pred) ; [Link](x_test, y_test) # accuracy
logreg.predict_proba(x_test) ; logreg.coef_ ;
k-Nearest-Neighbors Classification: predict outcome by majority at k most similar data points
knn_5 = [Link](n_neighbors=5) # set the model
knn_5.fit(x_train_norm, y_train) # train the model
knn_5.predict(x_test_norm) ; knn_5.score(x_test_norm, y_test)
Decision trees & Random forests
Hyperparameter: how high can tree depth be? (too high -> overfittting)
dt = [Link](random_state=2, max_depth=3)
[Link](x_train, y_train) ; [Link](x_test, y_test)
[Link].plot_tree(dt, feature_names=x_train.columns, filled=True)
rf = [Link](random_state=2)
[Link](x_train, y_train) ; [Link](x_test, y_test)
Evaluation
y_pred = [Link](x_test)
[Link].confusion_matrix(y_test, y_pred) ;
[Link].plot_confusion_matrix(rf, x_test, y_test) # normalize='true'
[Link].recall_score(y_test, y_pred) # TP/(TP+FN)
[Link].precision_score(y_test, y_pred) # TP/(TP+FP), WikiLink
[Link].plot_precision_recall_curve(rf, x_test, y_test) # doc
[Link](x_test, y_test) # 'regular' accuracy, good when labels are balanced
[Link].balanced_accuracy_score(y_test, y_pred)
Unsupervised learning: cluster analysis & PCA - No target variable, goal is not to predict something
kmeans = [Link](n_clusters=2, init='random', random_state=3)
[Link](x_train_norm) ; kmeans.cluster_centers_ # n dimensions = n columns
pred_k_means_test = [Link](x_test_norm)
[Link].accuracy_score(y_test, pred_k_means_test)
pca = [Link](n_components=2) # number of target dimensions
[Link]([Link]([x_train.columns, # component contributions
pca.components_.round(2)]).transpose()) # (feature effects)
pc = pca.fit_transform(x_test_norm)
[Link](x=pc[:,0], y=pc[:,1], hue=pred_k_means_test, palette="Blues")
hierarc_clust = [Link]. [Link](x_test_norm, method='ward')
[Link]. [Link](hierarc_clust)
agg_clustering = [Link]. AgglomerativeClustering(n_clusters=2,
affinity='euclidean', linkage='ward')
pred_agg_test = agg_clustering.fit_predict(x_test_norm)
[Link].accuracy_score(y_test, pred_agg_test) # and sns pred_agg_test
X_agg_values = scaler.inverse_transform(x_test_norm)
X_agg = [Link](X_agg_values, index=x_test.index,
columns=x_test.columns) ; X_agg['clust'] = pred_agg_test
[Link](X_agg, hue='clust', palette='Blues')