Computer Programming & Data Science

– An introduction with Python

Chapter 05: Data Science Analysis & Machine Learning

[Link]. Kevin Riehl, Dr. Milos Balac

Institut für Verkehrsplanung und Transportsysteme
03. & 10. April 2024, Zürich
What you will learn
Structure Of Course / Syllabus
Lecture Date Presentation Content Interactive Content
L01 20.02.2025 Chapter 01: Setup Computer, IDE (Spyder), Anaconda Environment, Git
Motivation, Introduction & Kick-Off Presentation of Tools (IDE, Git) and how to run a programme
Run Demo Video Game (Space Invaders, from Crash Course Python)
L02 27.02.2025 Chapter 02: Write custom Chatbot, Practice Hackerrank
Foundations of Programming with Python
L03 06.03.2025 Write custom Chatbot, Practice Hackerrank
L04 13.03.2025 Chapter 03: Work with Climate DataSet (Prof. Peter Molnar), Practice Hackerrank
Processing Tabular Data with Pandas
L05 20.03.2025 Work with Climate DataSet (Prof. Peter Molnar), Practice Hackerrank
L06 27.03.2025 Chapter 04: Visualization of Climate DataSet (Prof. Peter Molnar)
Data Visualization With MatplotLib & Seaborn Time Series, Maps, etc.
L07 03.04.2025 Chapter 05: Descriptive Statistics & Tests with Climate DataSet (Prof. Peter Molnar)
Data Science Analysis & Machine Learning
L08 10.04.2025 Time Series Prediction With Climate DataSet (Prof. Peter Molnar)
L09 17.04.2025 Chapter 06: In-Class Experiment About Communication, Practice Hackerrank
Programming Theory & Clean Software Engineering API Development Exercise & Coding Rotation Exercise
Kick-off student projects
L10 08.05.2025 Guest Lecture TBD
L11 15.05.2025 (Buffer) Work on student projects
L12 22.05.2025 Final Presentation of Coding Projects –

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 2
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 3
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 4
Chapter 05: Data Science Analysis & Machine Learning
Descriptive Statistics For Distributional Analysis

▪ Before you can analyse data with any tool (machine learning, regression models, tests)
often these tools require certain properties from the data used
▪ For example:
▪ the t-test assumes normally distributed random variables
▪ machine learning models can achieve higher performances if data is normalized, etc.
▪ Descriptive statistics allow you to get an initial overview of numerical data and about its
distribution

Mean Standard Deviation Skewness Kurtosis

Minimum 25% percentile Median 75% percentile Maximum

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 5
Chapter 05: Data Science Analysis & Machine Learning
Descriptive Statistics For Distributional Analysis

Minimum 25% percentile Median 75% percentile Maximum

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 6
Chapter 05: Data Science Analysis & Machine Learning
Descriptive Statistics For Distributional Analysis

Mean Standard Deviation Skewness Kurtosis

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 7
Chapter 05: Data Science Analysis & Machine Learning
Different types of distributions

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 8
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 9
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / What is a hypothesis test?

▪ A hypothesis test is a statistical tool, which can falsify a hypothesis about statistical
properties of one or multiple random variables is false or not based on samples

▪ Example
▪ A factory filling 250ml bottles with a drink. The machines will sometimes fill more,
and sometimes fill less content into the bottles. Is the amount in the bottles on
average really 250ml? (or does the machine fill in too much or too little?)
▪ You are creating a new medicine (drug). You treat 100 patients with the drug, and
100 patients you treat with a placebo. Is the effect of the new medicine significant to
treat a disease / create immunity?
▪ You are wondering if you should change the ingredients of your product to reduce
costs. 50% of the users you sell the old product, 50% of the users you sell the new
product. Are changes significantly impacting (damaging) customer satisfaction?

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 10
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / Single-sided vs two-sided tests

▪ Single-sided tests (one-tailed tests)

▪ examine the possibility of an effect in one direction
▪ e.g. the bottles consistently contain less than 250ml of the drink

▪ Two-sided tests (two-tailed tests)

▪ Examine difference between groups, no specific prediction about direction
▪ e.g. a new medicine, will it affect the blood pressure, probability for heart attack, etc.

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 11
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / Parametric vs. Non-Parametric tests

▪ Parametric tests (one-tailed tests)

▪ assume that data follows a specific distribution (mostly the normal distribution)
▪ most precise when assumptions are met
▪ sensitive to outliers

▪ Non-parametric tests (two-tailed tests)

▪ do not assume anything about the data distribution
▪ can be used with a wide variety of different data
▪ robust to outliers
▪ can work with small samples as well
▪ might not be that precise

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 12
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / How to do a test? (Example t-test)

1. Define hypothesis to be tested

You can only falsify,
▪ H0 (null-hypothesis): The bottles are filled with less than 250 ml.
but not verify, choose
▪ H1 (alternative hyp.): The opposite of H0 (equal or greater). hypothesis therefore
2. Collect Data like this
▪ In a factory, take n samples of filled bottles
▪ Do descriptive statistic analysis, and validate that test is suitable
▪ T-test is parametric test, assuming the variables are normally distributed
3. Calculate Test statistic 4. Compare Test statistic with Test distribution
determine p-value

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 13
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / How to do a test? (Example t-test)

5. Accept or reject hypothesis based on p-value and significance threshold (alpha)

▪ P-value smaller than 5% - weak significant
▪ P-value smaller than 2% - significant
▪ P-value smaller than 1% - highly significant

(see code example on next page)

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 14
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / How to do a test? (Example t-test)
# Imports
import numpy as np
from scipy import stats

# Generate 100 samples of bottle content (in ml) (because we don’t have measurement data here)
# Assuming the actual mean is slightly below 250ml with some variation
n = 100
bottle_content = [Link](loc=245, scale=5, size=n)

# Define the null hypothesis value & significance threshold alpha

h0_value = 250
alpha = 0.02

# Perform one-sided t-test

t_statistic, p_value = stats.ttest_1samp(bottle_content, popmean=h0_value, alternative='greater')

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 15
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / How to do a test? (Example t-test)
# Perform one-sided t-test
t_statistic, p_value = stats.ttest_1samp(bottle_content, popmean=h0_value, alternative='greater')

# Print results
print(f"Sample mean: {[Link](bottle_content):.2f} ml")
print(f"t-statistic: {t_statistic:.4f}")
print(f"p-value: {p_value:.4f}")

# Interpret results
if p_value < alpha:
print("Reject the null hypothesis.")
print("There is significant evidence that bottles do not have less than 250ml content.")
else:
print("Fail to reject the null hypothesis.")
print("There is not enough evidence to conclude that bottles do not have less than 250ml content.")

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 16
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / Types of Tests

Normality Tests
▪ Kolmogorov-Smirnov
▪ Shapiro-Wilk
▪ Anderson-Darling

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 17
Chapter 05: Data Science Analysis & Machine Learning
Test Theory & Statistics / Contingency Matrix, Precision of Test, Errors

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 18
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 19
Chapter 05: Data Science Analysis & Machine Learning
Data preprocessing, cleansing & outlier detection

▪ Before doing any analysis: Check your data !!!

▪ Go through every column of your dataset
▪ For numerical data
▪ Check the value range
▪ Are there missing (nan or null) values?
▪ Are there continuous values > 0.0 but then some -1 values?
▪ Consider replacing (imputing) data, or to remove records from dataset
▪ For string / categorial data
▪ Check the unique values of your dataset
▪ Are there empty rows?
▪ Are there spelling mistakes and several categories should belong together?
(e.g. column "fruit" has values "apples", "Apples", "apple", "Apple")
▪ Consider replacing wrongly spelled categories, or to remove records from dataset

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 20
Chapter 05: Data Science Analysis & Machine Learning
Data preprocessing, cleansing & outlier detection

▪ Outlier Detection For Numerical Data

▪ Sometimes it can make sense to exclude certain numerical values from your sample
to improve the accuracy of your analysis
▪ Example: Linear regression through a point cloud with outliers

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 21
Chapter 05: Data Science Analysis & Machine Learning
Data preprocessing, cleansing & outlier detection

▪ Outlier Detection For Numerical Data (Anomaly Detection)

▪ Outlier detection can be conducted with statistical tests for outlier detection
▪ For example:
▪ Z-score
▪ Tukey's range test
▪ Grubb's test

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 22
Chapter 05: Data Science Analysis & Machine Learning
Outlier Detection With Z-Scores (example)

# Imports
import numpy as np
import [Link] as plt
from scipy import stats

# Generate data for linear regression with outliers

def generate_data(n_samples=100, n_outliers=10):
X = [Link](0, 10, n_samples)
y = 2 * X + 1 + [Link](0, 1, n_samples)

# Add outliers
outlier_indices = [Link](n_samples, n_outliers, replace=False)
y[outlier_indices] += [Link](0, 15, n_outliers)

return X, y

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 23
Chapter 05: Data Science Analysis & Machine Learning
Outlier Detection With Z-Scores (example)

# Function to remove outliers using z-score

def remove_outliers(X, y, threshold=3):
z_scores = [Link]([Link](y))
mask = z_scores < threshold
return X[mask], y[mask]

X, y = generate_data()
X_clean, y_clean = remove_outliers(X, y)

# Do further analysis with X_clean, and Y_clean instead of X and y

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 24
Chapter 05: Data Science Analysis & Machine Learning
Data preprocessing: Linearization & Normalization

▪ Linearization
▪ Sometimes, if non-linear relationships are analyzed, it can make sense
to apply functions such as logarithm or exponential function,
to "linearize" the non-linear variable, and make it thus accessible for linear tools
▪ Linearization can also contribute to transforming not-normally distributed variables
to normally distributed variables

▪ Normalization & Standardization

▪ In machine learning, normalization
(remove mean, divide by standard deviation)
can significantly improve model performance
▪ Normalization can also contribute to
transforming not-normally distributed
variables to normally distributed variables

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 25
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 26
 Chapter 05: Data Science Analysis & Machine Learning
Regression Models

▪ Regression models try to analyze the relationship between variables

and they can also be used for interpolation and prediction

▪ Examples
▪ What is the potential price for a real estate given the square meters, location in the city?
▪ What is the chance of a person to fail paying back debt? (for banks)
▪ What is the chance of a risk to occur? (for insurances)
▪ What stability will my concrete have based on the ingredients used?
▪ What is the amount of rain for rain given temperature, air pressure, wind, and time of the year?
▪ …

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 27
Chapter 05: Data Science Analysis & Machine Learning
Regression Models

▪ Regression models try to analyze the relationship between variables

and they can also be used for interpolation and prediction

▪ Important terms
▪ Y dependent variable(s)
▪ X independent variable(s) (explaining variable)
▪ A model parameters
▪ F model function
▪ R residuals (unexplainable errors)

▪ Optimization
▪ Process of determining A (for F) based on given observations X,Y

▪ Any regression model be like

Y = F(X, A) + R
Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 28
Chapter 05: Data Science Analysis & Machine Learning
Regression Models – Example Linear Regression Model

Y = F(X, A) + R Y = X*A + R

Y is a vector (could also be matrix)

X is a matrix and includes… interc. X_1 Y

n samples
1 532.23 353.32
▪ intercept (just ones) 1 334.12 642.23
▪ explanatory variable 1 1 325.23 753.64
▪ explanatory variable 2 1 126.23 752.36

▪ … … … …

A_1 A_2
How to determine A (matrix)? → Optimization ??? ???

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 29
Chapter 05: Data Science Analysis & Machine Learning
Regression Models For Non-Linear Data

▪ Regression models by their design assume that the data follows a specific function
e.g. a linear trend, a logistic curve, polynomial, logarithm, etc.
▪ You could use non-linear models and find reasonable parameters to fit the data
▪ but the optimization can become complex and numeric
▪ You could also linearize data (i.e., transforming Y or X)
▪ e.g. if the relationship is logarithmic,
you could do a linear regression on exp(y) instead of y

Linear Regression Logisitc Regression Polynomial Regression Logarithmic Regression

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 30
Chapter 05: Data Science Analysis & Machine Learning
Advantages of Linear Regression

▪ The optimization (fitting the curve to data) is computationally much less expensive
▪ The optimization can (often) be conducted analytically instead of numerically
▪ There are many tests and tools for linear regression models
▪ You can easily interpret linearized non-linear relationships

Linear Regression Logisitc Regression Polynomial Regression Logarithmic Regression

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 31
Chapter 05: Data Science Analysis & Machine Learning
Regression Models For Specific Data

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 32
Chapter 05: Data Science Analysis & Machine Learning
Optimization – How to optimize?

▪ Variety of different optimization algorithms

▪ Most important: definition of goal function (cost function) that should be minimized

▪ Any Regression Model: Y = F(X, A) + R

▪ Goal function should try to make the errors R (residuals) as small as possible,
then the model F can perfectly explain Y if data X is available

▪ L^2-Norm is a very popular norm

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 33
Chapter 05: Data Science Analysis & Machine Learning
Optimization – Analytic Solution: Ordinary Least Squares for Linear Regression

▪ Computationally simple approach → analytic (was important when we did

not have lots of computation power)

▪ BUT: imposes various assumptions on y and X, issues with inversion of XTX, etc.

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 34
Chapter 05: Data Science Analysis & Machine Learning
Optimization – Numerical Solution (Common Today)

▪ Numerical solutions use iterative algorithms

▪ They find different parameters by trial and error
▪ At some point they stop (converge)
▪ No guarantee to find optimal solution, but often good is good enough

▪ Examples
▪ Gradient Descent
▪ BFGS (Broyden-Fletcher-Goldfarb-Shanno)
▪ Nelder-Mead Simplex
▪ Heuristics (such as Evolutionary Algorithms)
▪ …

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 35
Chapter 05: Data Science Analysis & Machine Learning
Optimization – Easy Example With Python

# Imports
import numpy as np
import [Link] as plt
from [Link] import mean_squared_error

# Step 1: Generate random data with noise

x = [Link](-10, 10, 100) # 100 random points [-10;10]
noise = [Link](0, 10, [Link]) # Gaussian noise
y = 5 + x * 3 + 2 * x**2 + noise
sorted_indices = [Link](x)
x = x[sorted_indices]
y = y[sorted_indices]

# Step 2: Fit a polynomial regression model

model1 = np.poly1d([Link](x, y, 1))
model2 = np.poly1d([Link](x, y, 2))
model3 = np.poly1d([Link](x, y, 3))
y_pred1 = model1(x)
y_pred2 = model2(x)
y_pred3 = model3(x)

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 36
Chapter 05: Data Science Analysis & Machine Learning
Optimization – Easy Example With Python

# ...

# Step 2: Fit a polynomial regression model

model1 = np.poly1d([Link](x, y, 1))
model2 = np.poly1d([Link](x, y, 2))
model3 = np.poly1d([Link](x, y, 3))
y_pred1 = model1(x)
y_pred2 = model2(x)
y_pred3 = model3(x)

# # Step 3: Calculate and print L2 norm of residuals

residuals1 = y - y_pred1
residuals2 = y - y_pred2
residuals3 = y - y_pred3

l2_norm1 = [Link](mean_squared_error(y, y_pred1)) * len(y)

l2_norm2 = [Link](mean_squared_error(y, y_pred2)) * len(y)
l2_norm3 = [Link](mean_squared_error(y, y_pred3)) * len(y)

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 37
Chapter 05: Data Science Analysis & Machine Learning
Optimization – Easy Example With Python

# ...

# Step 4: Plot scatter and fitted curve

[Link](x, y, label='Data with noise')
[Link](x, y_pred1, "--", label='(1 Parameter) L2='+"{:.2f}".format(l2_norm1))
[Link](x, y_pred2, "--", label='(2 Parameters) L2='+"{:.2f}".format(l2_norm2))
[Link](x, y_pred3, "--", label='(3 Parameters) L2='+"{:.2f}".format(l2_norm3))
[Link]('x')
[Link]('y')
[Link]('Polynomial Regression Fit ')
[Link]()
[Link]()

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 38
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis

Useful to analyse if (a) certain independent

variable(s) provide explanatory power for
dependent variable

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 39
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis
intercept

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 40
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis
explanatory variables

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 41
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis regression model and
dependent variable

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 42
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis coefficient (model parameter)
and t-test statistic below

* Stars indicate if variable contributes

statistically significant power

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 43
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis Fit of the model (R-squared)
measures similar to L2 norm

models (4) to (11) are much better

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 44
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis F-test assesses overall quality
and significance of the model with
all parameters

T-test only assesses significance

of single parameters

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 45
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis The explanatory variable was non-
linearly distributed, so
linearization improved the model

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 46
Chapter 05: Data Science Analysis & Machine Learning
Interpretation of Regression Analysis

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 47
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 48
Chapter 05: Data Science Analysis & Machine Learning
What is Machine Learning?

▪ Machine Learning is a discipline of artificial intelligence focusing on the development of

algorithms without explicit programming, based on provided data the computer can learn from.

▪ Currently:
▪ You define a problem.
▪ You think about how to solve a problem and define an algorithm
▪ You implement the algorithm (e.g., in Python)
▪ You run the software, observe it, evaluate it, and refine it
▪ Outcome: working programme that solves a problem

▪ With Machine Learning:

▪ You define a problem.
▪ You let the machine come up with an algorithm (and sometimes even implementation).

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 49
Chapter 05: Data Science Analysis & Machine Learning
What is Machine Learning?

▪ Example 1: Software to predict real-estate price

▪ You could based on your knowledge and experience craft a formula (the algorithm)
▪ You could also use machine learning to learn from data, for example
you could optimize the parameters of a regression model (the algorithm)
▪ Example 2: Software to detect and annotate faces on a picture
▪ Example 3: Software to control a robot
▪ Example 4: Software to drive a car

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 50
Chapter 05: Data Science Analysis & Machine Learning
What is Machine Learning?

▪ Besides (linear) regression, machine learning has a multitude of models that

can be used, with higher degrees of non-linearity or higher number of
parameters

▪ Example Random Forest:

▪ Non linear model
▪ Example Neural Network in Computer Vision:
▪ YOLOv8x (68.7 million parameters)
▪ Example Chat-GPT:
▪ ChatGPT-3.5 (175 billion parameters)
▪ ChatGPT-4.0 (100 trillion parameters)

▪ More non-linearity, more data, and more parameters does not necessarily
mean better outcomes!

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 51
Chapter 05: Data Science Analysis & Machine Learning
What is Machine Learning?

▪ Besides (linear) regression, machine learning has a multitude of models that

can be used, with higher degrees of non-linearity or higher number of
parameters

▪ Example Random Forest:

▪ Non linear model
▪ Example Neural Network in Computer Vision:
▪ YOLOv8x (68.7 million parameters)
▪ Example Chat-GPT:
▪ ChatGPT-3.5 (175 billion parameters)
▪ ChatGPT-4.0 (100 trillion parameters)

▪ More non-linearity, more data, and more parameters (complexity)

does not necessarily mean better outcomes!

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 52
Chapter 05: Data Science Analysis & Machine Learning
Supervised Machine Learning

▪ Machine Learning:
▪ with labelled training data
▪ X – training data
▪ Y – label data

▪ Problem Categories:
▪ Y (continuous variable) – Regression Problems (also time series prediction)
▪ Y (discrete variable) – Classification Problems

▪ Algorithms / Models:
▪ Support-vector machines - K-nearest neighbor algorithm
▪ Linear & Logistic regression - Neural networks (Multilayer perceptrons)
▪ Naïve Bayes - Similarity learning
▪ Linear discriminant analysis
▪ Decision trees

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 53
Chapter 05: Data Science Analysis & Machine Learning
Unsupervised Machine Learning

▪ Machine Learning:
▪ with unlabeled training data
▪ X – training data
▪ Y – label data

▪ Problem Categories & Algorithms/Models:

▪ Clustering (group similar objects into clusters based on variables/features)
▪ K-Means, DBSCAN, Hierarchical Clustering
▪ Association (identify relationships between variables/features in large datasets)
▪ Apriori, Eclat, FP-Growth
▪ Dimensionality Reduction (reduce number of variables/features)
▪ Principal Component Analysis (PCA), ICA, t-SNE

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 54
Chapter 05: Data Science Analysis & Machine Learning
Limitations of Machine Learning: Bias-Variance-Tradeoff

▪ More non-linearity, more data, and more parameters (complexity)

does not necessarily mean better outcomes!

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 55
Chapter 05: Data Science Analysis & Machine Learning
Reinforcement Learning

▪ Machine Learning:
▪ with labelled training data
▪ X – training data
▪ Y – label data
▪ But the labelled data are created automatically during the learning process
by interactions with the environment (e.g. simulator, video game)
▪ Major goal: train an agent to learn to interact with environment

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 56
Chapter 05: Data Science Analysis & Machine Learning
What is Generative AI?

▪ Can be considered primarily as unsupervised machine learning.

▪ Creates original content such as text, images, audio, videos, or code
in response to prompts by users.

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 57
Chapter 05: Data Science Analysis & Machine Learning
▪ Descriptive Statistics For Distributional Analysis
▪ Test Theory & Statistics
▪ What is a hypothesis test?
▪ Single-sided vs two-sided tests
▪ Parametric vs non-parametric tests
▪ Data preprocessing, cleansing & outlier detection
▪ Regression Models
▪ Machine Learning
▪ What is Machine Learning?
▪ Supervised Learning
▪ Unsupervised Learning
▪ Reinforcement Learning

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 58
Chapter 05: Data Science Analysis & Machine Learning
Recommended Readings

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 59
Contact Information

[Link]. Kevin Riehl ▪ Email communication through Moodle

▪ If you do not plan to enroll, but still want to receive
Dr. Milos Balac course-related emails, please send me your email
kriehl@[Link] address at kriehl@[Link]
(include 101-0720-00L in the title)
[Link]@[Link] ▪ Refer to the Syllabus for content and contact information

ETH Zürich
Institut für Verkehrsplanung und Transportsysteme (IVT)
HIL F 41.2
Stefano-Franscini-Platz 5
8093 Zürich, Switzerland

[Link]
[Link]/

Office hours: Thursday, 17:30-18:00 (right after class)

KICK OFF STUDENT PROJECTS
Today: Kick-Off Student Projects (5 minutes)
▪ Form groups of one to two students (one student recommended)

Until Next Time:

▪ Write us an email, with your group name and members
▪ Write us idea and scope of your project, we will reply to you and give you
guidance to make sure its not too small and not too big

Deliverables of Student Project:

▪ Code on GitHub (you create your own repository) and share a link
▪ Runnable & well documented and commented code
▪ ReadMe (description, installation instructions, exemplary usage, …)
▪ Python Code
▪ Figures & Data (if necessary in your project)
▪ Presentation of 15 minutes (22.05.2025)
▪ Written report of 5 pages
▪ Scope / Examples:
▪ Types / Examples:
▪ A tool (give an input, provide a useful output, to a problem)
▪ An analysis (given a dataset, you analyse it, create visualizations, etc.)
▪ A video game (you find templates online, could refine them, PacMan,
BrickBreaker, Dilbert, Space Invaders…)
▪ Minimum Requirement:
▪ You work with at least one Python package that was not covered in class
(numpy, pandas, matplotlib, scikit-learn)
▪ You being creative (work beyond the scope of this lecture and excercises)

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 61
Chapter 05: Interactive Part
Meteo Swiss Dataset → Time Series Analysis & Prediction

Computer Programming & Data Science – Introduction with Python, Kevin Riehl, Milos Balac, IVT Institute 10.04.2025 62