MEI Data Science Taught Course

Python commands used in the course

Lesson 1: Preparing data and exploratory data analysis

Importing libraries and data

Import libraries

# import pandas for data analysis

import pandas as pd
# import seaborn for visualisations
import seaborn as sns

Import a csv file

# import the csv file to a data set called weather_data

weather_data = pd.read_csv('[Link]')

Display a data set

# display the data to verify it has imported

weather_data

Preparing data

Get information about the data types

# display the data types of the features

weather_data.info()

TB v2.11 © MEI
1 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Take a slice: create a new data set from a subset of an existing one – NB it is good practice to add .copy() at the end

# create a new data set called heathrow_data where the station value is 'heathrow'
heathrow_data = weather_data[weather_data['station'] == 'heathrow'].copy()

Exploratory data analysis

Get a summary of a numerical feature

# use describe() to output a summary of the tmax column

weather_data['tmax'].describe()

Get a summary of a numerical feature rounded to 2 decimal places

# use describe to get the statistics for the feature rounded to 2 dp

weather_data['tmax'].describe().round(2)

Get a summary of a numerical feature grouped by a categorical feature

# get the statistics for the feature grouped by station

weather_data.groupby('station')['tmax'].describe().round(3)

Draw a boxplot

# display a boxplot for the tmax column using catplot from Seaborn
[Link](kind='box', x='tmax', data=heathrow_data, aspect=2);

Draw a boxplot grouped by a categorical feature

# display a boxplot for the tmax feature grouped by decade using aspect to get a wider plot
[Link](kind='box', x='tmax', y='decade', data=heathrow_data, aspect=2);

TB v2.11 © MEI
2 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Lesson 2: More preparing data and exploratory data analysis

Preparing data

Create a new numerical feature

# create a new numerical feature from a function of existing features

bike_data['total'] = bike_data['casual']+bike_data['registered']

Create a new categorical feature from a numerical feature (e.g. when categories have been stored as numbers

# replace numbers with categories

bike_data['weather'] = bike_data['weathersit'].replace({1: 'Clear', 2: 'Cloud', 3:'Rain', 4:'Heavy rain' })

Create a slice of a data set based on multiple conditions (using & for “and”)

# create a slice of the data based on the condition

daytime_data = bike_data[(bike_data['hr'] >6) & (bike_data['hr'] <21)].copy()

Exploratory data analysis

Create a two-way table of frequencies split across two categorical features (using crosstab)

# create a two-way table for two categories

[Link](daytime_data['season'],daytime_data['weather'])

Create a two-way table with the means of a third, numerical, feature in the cells (using crosstab)

# create a two-way table for two categories using crosstab with the mean of a numerical feature in each cell
[Link](daytime_data['season'],daytime_data['weather'],values=daytime_data['total'],aggfunc='mean').round(2)

TB v2.11 © MEI
3 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Lesson 3: Visualisations in Seaborn

Plots for categorical features

Bar chart split by categories showing frequencies

# bar chart split by a category (frequencies)

[Link](data=wdi_data, x='Region', hue='High Income Economy', multiple='stack', aspect=2);

Bar chart split by categories showing proportions

# bar chart split by a category (proportions)

[Link](data=wdi_data, x='Region', hue='High Income Economy', stat='proportion', multiple='fill', aspect=2);

1D plots for a single numerical feature

Box plots grouped by a categorical feature (on the y-axis)

# box plot for female life expectancy categorised by High Income Economy
[Link](data=wdi_data, kind='box', x='Life expectancy, female', y='High Income Economy', aspect=2);

Violin plot

# create a violin plot with a continuous variable on x, a categorical on y and a binary variable for hue
[Link](data=wdi_data, kind='violin', x='Life expectancy, female', y='Region', hue='High Income Economy', aspect=2);

Split violin plot – NB the hue must be a categorical feature with exactly two values.

# create a violin plot with a continuous variable on x, a categorical on y and a binary variable for hue with split violins
[Link](data=wdi_data, kind='violin', x='Internet use', y='Region', hue='High Income Economy', split=True, aspect=2);

TB v2.11 © MEI
4 of 10 2025-06-12

(x
 MEI Data Science Taught Course

Histogram

# create a basic distribution plot

[Link](data=wdi_data, x='Life expectancy, female');

Histograms for each of the values in a categorical column

# col='Region' - one column per region

[Link](data=wdi_data, x='Life expectancy, female', col='Region');

Normalised ‘histograms’ for each of the values in a categorical column

# plot histograms using a category for columns - stat='density', common_norm=False normalises each one
[Link](data=wdi_data, x='Life expectancy, female', col='Region', stat='density', common_norm=False);

KDE (kernel density estimate) to display the distribution

# kind='kde' - estimate the shape of a continuous distribution

[Link](data=wdi_data, kind='kde', x='Life expectancy, female', hue='High Income Economy', aspect=2 );

2D plots for a pair of numerical features

Scatter plot

# scatter plot
[Link](data=wdi_data, x='Internet use', y='Emissions per capita', aspect=2);

Scatter plot with a category for the colour (or hue)

# hue='Region' - colour by region

[Link](data=wdi_data, x='Internet use', y='Emissions per capita', hue='Region', aspect=2);

TB v2.11 © MEI
5 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Separate scatter plots grouped by additional categorical features

# create scatter plots split by Region and High Income Economy

[Link](data=wdi_data, x='Internet use', y='Emissions per capita', col='Region', row='High Income Economy');

Create a rectangular grid showing density with histogram on each axis

# create a jointplot with histograms on each side

[Link](data=wdi_data, kind='hist', x='Internet use', y='Emissions per capita');

Hexagonal grid showing density with histogram on each axis

# create a hexagonal jointplot with histograms on each side

[Link](data=wdi_data, kind='hex', x='Internet use', y='Emissions per capita');

Create all the possible scatter plots for two lists of numerical features

# create all possible scatter plots

[Link](data=wdi_data, x_vars=['Internet use', 'Population', 'Physicians'], y_vars=['Emissions per capita']);

Setting titles and axes

# plot the visualisation

plot = [Link](data=wdi_data, x='Life expectancy, female', aspect=2);

# Label the axes and show the plot

[Link](xlim=(50,90), xlabel="Female life expectancy (years)", ylabel="Frequency", title="Average female life expectancy")
plot;

TB v2.11 © MEI
6 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Lesson 4: Introduction to practical task

Create a table comparing the means for a range of numerical features grouped by a categorical feature

features=['lead_time', 'stays_in_weekend_nights', 'stays_in_week_nights', 'adults', 'children', 'babies',

'previous_cancellations', 'previous_bookings_not_canceled', 'booking_changes', 'days_in_waiting_list',
'required_car_parking_spaces', 'total_of_special_requests', 'adr']

# use group by to get the means of these for low and high income
hotel_data.groupby('cancelled')[features].mean().round(2)

Create a two-way table of frequencies split across two categorical features (using crosstab)

# create a two-way table for two categories

[Link](hotel_data['hotel'],hotel_data['cancelled'])

Create a two-way table to compare proportions of each row (using crosstab)

# create a two-way table for two categories showing the proportions of each row
[Link](hotel_data['hotel'],hotel_data['cancelled'],normalize='index').round(3)

TB v2.11 © MEI
7 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Lessons 5-8: Binary classification

Import functions for binary classification

# import modelling and metrics for decision trees from sklearn

from [Link] import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from [Link] import precision_score, recall_score

Import functions for plotting a decision tree and define a plot_decision_tree() command

# import pyplot and plot_tree for displaying decision trees

import [Link] as plt
from [Link] import plot_tree

def plot_decision_tree(model):
fig, ax = [Link](figsize=(18,8))
plot_tree(model, filled=True, impurity=False, feature_names=input_features, proportion=True, class_names=["No","Yes"], ax=ax)
[Link]()

Compare the means for a list of numerical features grouped by a categorical feature

# create a list of the features to be explored

features=['GP', 'MIN', 'PTS', 'FGS', 'FGA', '3PM', '3PA', 'FTM', 'FTA', 'OREB', 'DREB', 'AST', 'STL', 'BLK', 'TOV']

# use groupby to get the means of these for 'no' and 'yes'
nba_data.groupby('Successful')[features].mean().round(2)

TB v2.11 © MEI
8 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Define the input/target features, perform the training/testing split, create a classification model and measure the model with precision and recall

# define the input features create the input table (X) and define the target feature (y), perform the training-testing split
input_features = ['PTS', 'DREB']
X = nba_data[input_features]
y = nba_data['5Yrs']
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8, random_state=1)

# create the model and display the decision tree

tree_model = DecisionTreeClassifier(max_depth=2).fit(X_train, y_train)
plot_decision_tree(tree_model)

# create a list of the predictions and calculate the metrics

y_pred = tree_model.predict(X_test)
print("Precision: ",round(100*precision_score(y_test, y_pred, zero_division=0),1),"%")
print("Recall: ",round(100*recall_score(y_test, y_pred),1),"%")

Display a confusion matrix (two-way table) for the predicted and actual values

# Display a two-way table of predictions and actual values for the testing data
print([Link](y_test, y_pred, rownames=['Actual'], colnames=['Predicted'], margins=True))

TB v2.11 © MEI
9 of 10 2025-06-12

(x
 MEI Data Science Taught Course
Lesson 9-10: Linear regression

Import functions for linear regression

# import functions from sklearn for modelling

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from [Link] import r2_score, mean_squared_error

Define the input/output features, perform the training/testing split, create a linear model and measure the model with 𝑅𝑀𝑆𝐸 and 𝑅 2

# define the input features create the input table (X) and define the target feature (y), perform the training-testing split
input_features = ['Maximum temperature', 'MSL pressure']
X = solar_data_clean[input_features]
y = solar_data_clean['solar_value']
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8, random_state=1)

# create the model and display the parameters - output the coefficients and y-intercept
linear_model = LinearRegression().fit(X_train,y_train)
print('Coefficients: ', linear_model.coef_.round(3))
print('Intercept: ', linear_model.intercept_.round(3))

# create a list of the predictions and calculate the metrics (RMSE and R² score) by comparing to the target values in y_test
y_pred = linear_model.predict(X_test)
print('RMSE: ',mean_squared_error(y_test, y_pred, squared=False).round(3))
print('R²: ',(100*r2_score(y_test, y_pred)).round(3),"%")

TB v2.11 © MEI
10 of 10 2025-06-12

(x