15/10/2024, 16:35 DSCI 100 function reference sheet for Python
DSCI 100 function reference sheet for Python
This reference sheet contains the key objects that we use in DSCI 100, and a brief syntax example for
each of the main packages. During the closed book exams, you will still have access to this page, so
get familiar with it already now. There is no guarantee that every function or parameter in the
textbook is covered here, but if you think there is something missing, please let us know and we can
consider adding it.
Note that we have only described use cases relevant to DSCI 100. For example, the function
reference for a pandas data frame is [Link](data_as_dict) , because in DSCI 100 we show
you how to create data frames from a dictionary in this manner. Although there are many other
ways of creating data frames and many parameters that you could use inside [Link] , we
have opted to only include what we use in DSCI 100 to make this guide more useful for you.
Sometimes we have included the exact parameter name of a function, e.g. drop(columns) , other
times we have opted to included a more descriptive name, e.g. agg(list_of_aggregations) .
Data wrangling ( pandas )
A typical data frame operation would look something like this:
import pandas as pd
[Link][
df['column2'] < 10,
['column2', 'column3']
].mean()
pandas functions, prefix with pd.
Name Description
A two-dimensional, heterogeneous tabular data structure with
DataFrame(data_as_dict)
labeled axes
A one-dimensional labeled array capable of holding data of
Series(data_as_list)
various types
[Link] 1/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
Compute a cross-tabulation of two (or more) factors, e.g. a
crosstab(df)
confusion matrix.
concat(dfs_or_series) Concatenate pandas objects (dfs, series) along a particular axis
read_csv(filepath, sep) Read data from a CSV file
read_excel(filepath,
Read data from an Excel file
sheet)
Parse HTML tables from a web page and return a list of
read_html(filepath)
DataFrames
to_csv(filepath, index) Write a DataFrame to a CSV file
Data frame methods, prefix with the name of the data frame (e.g. df. )
Name Description
abs() Convert to absolute value
agg(list_of_aggregations) Perform multiple aggregations (‘mean’, ‘median’, etc)
apply(func) Apply a function to every column or row in the df
assign(newcol=oldcol*2) Create a new column
astype(dtype) Convert the data type of a column
describe() Calculate common descriptive statistics
drop(columns) Remove the specified row(s) or column(s)
dropna() Remove the rows that contains NULL/NA values
groupby(column_as_str) Group rows together if share value in the specified column
contains(string) For searching for a str within the values of a df column
For each value in a column, check if it is present in the
isin(values)
specified values
iloc[] Select rows and columns by their integer positions
info() Print a concise summary of a df
loc[] Filter rows and select columns at the same time
[Link] 2/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
max() Return the maximum value of the columns
mean() Return the mean value of the columns
melt(id_vars, var_name,
Unpivot a df from wide to long format
value_name)
merge(df, on) Merge data frames or series
min() Return the minimum value of the columns
Return top n rows after sorting the df by a column’s
nlargest()
highest value
Return top n rows after sorting the df by a column’s
nsmallest()
smallest value
pivot(index, columns, values) Turn df from long to wide
quantile() Return the value at the specified quantile
Filter a data frame based on row condition query, similar to
query()
[].
rename(columns) Rename columns in a df
replace(to_replace, value) Replace specific values to desired new values
reset_index() Reset the index of the df to a numerical range
round() Round values in a df
sample(n, frac, replace) Return a random sample of n rows from a df
shape Find the number of rows and cols of the df
sort_values() Order the rows of a df by the values of certain col
Test if the start of each str element in a column matches a
[Link]()
pattern
sum() Return the sum of the values over the requested axis
tail() Return a specified number of rows from the end of the df
unique() See all unique values in a column
[Link] 3/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
Return a Series containing the frequency of each distinct
value_counts()
value in the column
Data base methods, perform on a database connection or a table
Name Description
[Link](database, host, port,
Connect to a database
user, password)
list_tables() List all tables in the database
table() Connect to a specific table in a database
order_by() The same as sort_values for a data frame
Execute a database table operation to yield a
execute()
data frame from the database
Visualization ( altair )
A typical chart syntax would look something like this:
import altair as alt
[Link](df).mark_point().encode(
x='column1',
y=alt.Y('column2').title('Column 2')
)
Name Description
Chart(df) Specity the data used for the chart
Helpers for modifying the corresponding encoding, eg. adding
Color, X, Y, …
a title
axis(tickCount, format) Modify the axis format of a single axis
configure_axis(titleFontSize) Modify the axis format of all axes
[Link] 4/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
mark_point(color, opacity,
Represent the data with the corresponding graphical mark
size)
bin(maxbins) Group the values of a column into bins (buckets)
Special encoding strings to compute the count, mean, etc of a
‘count()’, ‘mean()’, etc
column
A special encoding that creates data points directly instead of
datum(value)
using a data frame
facet(column_as_str, Create multiple views of a dataset where each panel contains a
columns) different subset
Disable the default maximum row limit for displaying data,
disable_max_rows()
which simplifies working with large data frames
Specify how data columns are encoded as visual channels (x, y,
encode(x, y, color, …)
color, etc)
& (ampersand) Concatenating charts vertically
| (pipe) Concatenating charts horizontally
+ (plus) Layer multiple charts on top of each other
legend(orient) Modify the legend format and properties for the chart
properties(width, height) Set various properties and configurations for the chart
resolve_scale(x,
scale y) Control if scale
scales are shared or independent between charts
save(‘filename’) Save the chart to a file
scale(zero, type)
scale Modify scale properties for encoding channels in the chart
Control if marks (e.g. bars of areas) should stack on top of each
stack()
other
title() Add a title to the axis
Machine learning ( scikit-learn )
Setting up a typical scikit-learn model would look something like this:
[Link] 5/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
knn = KNeighborsClassifier(n_neighbors=3)
[Link](X_train, y_train)
[Link](X_test, y_test)
Name Description
Perform an exhaustive search over a
GridSearchCV(estimator, param_grid, cv) hyperparameter grid to find the best
combination using cross-validation.
Initialize a K-Means clustering algorithm for
KMeans(n_clusters) grouping data points into clusters based on
similarity.
Initialize a k-Nearest Neighbors classifier for
KNeighborsClassifier(n_neighbors)
classification tasks.
Initialize a k-Nearest Neighbors regressor
KNeighborsRegressor(n_neighbors)
for regression tasks.
Initialize a Linear regression model for
LinearRegression() predicting continuous target values from
input features.
Initialize a Imputation transformer for
SimpleImputer() handling missing data using simple
strategies.
Initialize a Scaler
Scale for standardizing features
StandardScaler()
Scale by subtracting the mean and scaling to unit
variance.
Attribute in GridSearchCV containing the
best_params_ best hyperparameters found during the grid
search.
Attribute in linear models (e.g.,
coef_ LinearRegression) containing the estimated
coefficients of features.
Function for evaluating a model’s
cross_validate() performance using cross-validation and
returning multiple scores.
[Link] 6/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
Attribute in GridSearchCV containing
cv_results_ detailed results from cross-validation grid
search.
Compute pairwise Euclidean distances
euclidean_distances(x, y)
between points in two datasets.
fit(X, y) Fit/train the model
Method to retrieve the hyperparameters of
get_params()
an estimator.
Attribute in KMeans indicating the sum of
inertia_ squared distances from samples to their
cluster centers.
Attribute in linear models (e.g.,
intercept_ LinearRegression) containing the intercept
term of the model.
Function for creating a column selector for
make_column_selector(dtype_include)
transformers in a ColumnTransformer.
make_column_transformer((transformer, Function for creating a composite
list_of_columns), remainder) transformer for feature preprocessing.
Function for creating a composite estimator
make_pipeline(preprocessor, model) (pipeline) with specified preprocessing and
model steps.
Function to calculate the mean squared
mean_squared_error() error between true and predicted values for
regression tasks.
Attribute in a Pipeline object providing
named_steps
access to individual steps by name.
Method used to make predictions on new
predict(X)
data samples for various estimators.
Method for calculating the accuracy or
score(X, y) performance metric of a classifier or
regressor.
[Link] 7/8
15/10/2024, 16:35 DSCI 100 function reference sheet for Python
Name Description
Function to configure global scikit-learn
set_config()
settings and behavior.
Function for splitting a dataset into training
train_test_split(df, train_size, stratify)
and testing sets for model evaluation.
Other
Name Description
[print(num**2) for num in A list comprehension to loop through the given range and
range(10)] perform an operation (here printing the square of the numbers)
append() Add elements to the end of a list or array
enumerate(iterable) Generate indices and values from an iterable
print() Display text or variable values to the console
range(start, stop, step) Create a range with regularly spaced values
[Link](start, stop, step) Create an array with regularly spaced values
[Link](list) Create a NumPy array from a list or other iterable
[Link](seed) Seed the random number generator for reproducibility
[Link](x) Calculate the square root of a numeric value or array
This site is open source. Improve this page.
[Link] 8/8