11/14/2025 Dr.

Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 289
Chapter Outline

 Overview of Machine Learning

 Introduction to scikit-learn package
 Supervised Learning Algorithms
 Unsupervised Learning Algorithms
 Hands-On Practices

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 290
What is Machine Learning?
“The field of study that gives computers the ability to
learn without being explicitly programmed“

Arthur Samuel
(1901 – 1990)

Founder of ML at
IBM in 1959

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 291
Machine Learning: What & Why?

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 292
Machine Learning: An Overview
Machine Learning

Supervised Unsupervised
Learning Learning

Classification Clustering

Regression Dimensionality
Reduction

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 293
Overview of Supervised Learning

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 294
Overview of Unsupervised Learning

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 295
Supervised vs Unsupervised Learning

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 296
Introduction to scikit-learn

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 297
Supervised Learning Algorithms

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 298
Unsupervised Learning Algorithms

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 299
Reinforcement Learning

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 300
Reinforcement Learning: Key Concepts

• Agent: The learner or decision-maker.

• Environment: Everything the agent interacts with.

• State: A specific situation in which the agent finds itself.

• Action: All possible moves the agent can make.

• Reward: Feedback from the environment based on the

action taken.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 301
Reinforcement Learning
Principle: Learning through Trial and Error
• Agent: The robot

• Environment: The maze

• State: Current location

• Action: Move

• Reward: Fire (negative)

or Diamond (positive) or
Safe Location (positive:
good move)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 302
Reinforcement Learning

Agent

Reward

Action

State

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 303
Reinforcement Learning: Flappy Bird Game

The Fundamentals of Reinforcement Learning and How to Apply It |

Intel® Tiber AI Studio

GitHub - anthonyli358/FlapPyBird-Reinforcement-Learning: Exploration

implementing reinforcement learning using Q-learning in Flappy Bird.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 304
Regression Algorithms

 Introduction
 Linear Regression
 Polynomial Regression
 Logistic Regression
 Hands-On Practices

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 305
Linear Regression - Example
Linear regression example:
import numpy as np
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error
import [Link] as plt

# Sample data
X = [Link]([[1], [2], [3], [4], [5]]) # Input features
y = [Link]([2, 4, 5, 4, 5]) #Target variable

# Create a linear regression model

regression = LinearRegression()
#Train the model
[Link](X, y)
# Make predictions on the training data
y_pred = [Link](X)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 306
Linear Regression - Example
Linear regression example (cont’d):
# Calculate the mean squared error
mse = mean_squared_error(y, y_pred)
print("Mean Squared Error:", mse)

# Plot the data points and the regression line

[Link](X, y, color='blue', label='Actual')
[Link](X, y_pred, color='red', label='Predicted')
[Link]('x')
[Link]('y')
[Link]()
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 307
Linear Regression - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 308
Linear Regression - Quiz
Quiz: Regression regression
Input:
 X is an array of 100 random values between 0 and 1.5.
 y is generated by the formula 𝑦 = 2 + 1.5X + noise, where
the noise is added using [Link](100,1)
Task Requirements:
a. Train a Linear Regression model using the provided
dataset.
b. Predict the values of y for X = 0 and X = 2 using the
trained model.
c. Plot the following: the original dataset, the predicted
values from the model
d. Output the model's intercept and coefficient.
e. Calculate the MSE between the true target variable y and
the predicted
11/14/2025 Dr. Maivalues
Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 309
Linear Regression - Solution
Linear regression quiz (solution)

import numpy as np # import linear algebra library

import [Link] as plt # import matplotlib library
from sklearn.linear_model import LinearRegression # import
Linear Regression model from sklearn
from [Link] import mean_squared_error # import
mean_squared_error from [Link]

# Generate 100 random numbers that are distributed in [0,1.5]

X = 1.5 * [Link](100, 1)

# Generate y to make X and y have a linear-like relationship

y = 2 + 1.5 * X + [Link](100, 1)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 310
Linear Regression - Solution
Linear regression quiz (solution) – cont’d

# Plot dataset (X, y)

[Link](X, y, 'b.')
[Link]([0, 2, 0, 8]) # specify axis range
[Link]('X') # x-axis label
[Link]('y') # y-axis label
[Link]('Linear-like dataset')
[Link](True)

# Train the Linear Regression model

lin_reg = LinearRegression()
lin_reg.fit(X, y)

# Make predictions for X = 0 and X = 2

X_new = [Link]([[0], [2]]) # Generate new X data points for
prediction
y_predict = lin_reg.predict(X_new) # Predict y values for X =
0 and X = 2

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 311
Linear Regression - Solution
Linear regression quiz (solution) – cont’d

# Print the predictions

print("Predicted y for X = 0:", y_predict[0])
print("Predicted y for X = 2:", y_predict[1])

# Plot the data points and the predictions

fig, ax = [Link]()
[Link](X, y, 'b.', label='Data samples')
[Link](X_new, y_predict, 'g-', label='Predictions')
[Link]([0, 2, 0, 8])
ax.set_xlabel('X')
ax.set_ylabel('y')
[Link]()
[Link](True)
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 312
Linear Regression - Solution

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 313
Linear Regression - Solution
Linear regression quiz (solution) – cont’d
# Print the model's intercept and coefficient
print("Intercept:", lin_reg.intercept_)
print("Coefficient:", lin_reg.coef_)

# Calculate the Mean Squared Error (MSE)

y_pred_all = lin_reg.predict(X) # Predict y for all input X
values
mse = mean_squared_error(y, y_pred_all) # Compute MSE
print("Mean Squared Error (MSE) for the model:", mse)

>>
Intercept: [2.51794653]Coefficient: [[1.49144611]]Mean Squared
Error (MSE) for the model: 0.08105176427547395

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 314
Multiple Linear Regression
Example:
Dataset: A synthetic dataset will be generated to predict
car prices based on two features:
 Engine size (in liters)
 Horsepower
⇒ The target variable will be the car's Price.
The price of a car is calculated using the following
formula:
Price = (Engine Size×5000) + (Horsepower×200) + ϵ
Where, ϵ represents random noise, modeled as a normal
distribution with a mean of 0 and standard deviation of
1000
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 315
Multiple Linear Regression
Instruction:
1. Dataset Creation: A synthetic dataset with the specified
features will be generated.
2. Data Preprocessing: The dataset will be split into training and
testing sets.
3. Model Building: A Multiple Linear Regression model will be
trained using the training data.
4. Model Evaluation: The model's performance will be assessed
using R-squared and Mean Squared Error (MSE) on the test
data.
5. Prediction: The trained model will be used to predict prices
for new data.
6. Results visualization: A plot comparing predicted prices to
actual prices will be created.
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 316
Multiple Linear Regression
Multiple Linear Regression example – Task 1: Create the Dataset
import pandas as pd
import numpy as np

# Set a random seed for reproducibility

[Link](42)

# Generate a simple synthetic dataset

engine_size = [Link](3.0, 0.5, 100) # Engine size in
liters
horsepower = [Link](150, 30, 100) # Horsepower
price = (engine_size * 5000) + (horsepower * 200) +
[Link](0, 1000, 100) # Price calculation
(simplified)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 317
Multiple Linear Regression
Multiple Linear Regression example – Task 1: Create the Dataset
# Create a DataFrame
df = [Link]({
'Engine Size': engine_size,
'Horsepower': horsepower,
'Price': price
})

# Print the first few rows of the dataset

print([Link]())
>>
Engine Size Horsepower Price
0 3.248357 107.538878 38107.348291
1 2.930868 137.380640 42691.251837
2 3.323844 139.718565 45645.985489
3 3.761515 125.931682 45047.713078
4 2.882923 145.161429 42069.232925

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 318
Multiple Linear Regression
Multiple Linear Regression example – Task 2: Preprocess the
Data
from sklearn.model_selection import train_test_split

# Separate features and target

X = df[['Engine Size', 'Horsepower']]
y = df['Price']

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=42)

# Print the shapes of the training and testing sets

print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
>>
(70, 2) (30, 2) (70,) (30,)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 319
Multiple Linear Regression
Multiple Linear Regression example – Task 3: Build a Multiple
Linear Regression Model
from sklearn.linear_model import LinearRegression

# Create a multiple linear regression model

model = LinearRegression()

# Train the model using the training data

[Link](X_train, y_train)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 320
Multiple Linear Regression
Multiple Linear Regression example – Task 4: Evaluate the Model
from [Link] import mean_squared_error, r2_score

# Make predictions on the test data

y_pred = [Link](X_test)

# Calculate the R-squared and Mean Squared Error (MSE)

r2 = r2_score(y_test, y_pred)
mse = mean_squared_error(y_test, y_pred)

# Print the evaluation metrics

print(f'R-squared: {r2:.2f}')
print(f'Mean Squared Error: {mse:.2f}')

>>
R-squared: 0.97
Mean Squared Error: 992085.17

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 321
Multiple Linear Regression
Multiple Linear Regression example – Task 5. Predict

# Example new data for prediction

new_data = [Link]({
'Engine Size': [3.5],
'Horsepower': [200]
})

# Make a prediction
prediction = [Link](new_data)

# Print the prediction

print(f'Predicted price: ${prediction[0]:,.2f}')

>>
Predicted price: $57,807.80

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 322
Multiple Linear Regression
Multiple Linear Regression example – Task 6: Plot the Results

import [Link] as plt

# Plot predicted vs actual prices

[Link](y_test, y_pred, color='blue')
[Link]([[Link](), [Link]()], [[Link](), [Link]()], 'k--', lw=2,
label='Perfect Prediction')
[Link]('Actual Price')
[Link]('Predicted Price')
[Link]('Predicted vs Actual Price')
[Link]()
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 323
Multiple Linear Regression

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 324
Polynomial Regression

Where:
 𝑐 , 𝑐 , …, 𝑐 are the coefficients of the polynomial,
 𝑥 is the value of the independent variable (input
feature).

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 325
Polynomial Regression

𝑦 is the actual data point at 𝑥 ,

𝑦 is the predicted value (model output) for 𝑥 ,
𝑛 is the number of data points.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 326
Polynomial Regression

To minimize the SSE, set =0

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 327
Polynomial Regression

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 328
 Quiz x y
-5 -228.75
-4.5 -179.81
Given the following dataset, determine the -4 -141
coefficients of the cubic polynomial: -3.5 -108.31
y = c0 + c1*x + c2*x^2 + c3*x^3 that best fits -3 -81
-2.5 -58.06
the data using Excel. -2 -38
-1.5 -20.75
-1 -6
-0.5 5.81
0 9
0.5 9.81
1 8
1.5 4.75
2 3
2.5 4.06
3 7
3.5 11.31
4 16
4.5 20.81
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 329
Quiz

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 330
Polynomial Regression workflow with
Scikit-learn
 Step 1: Import required libraries
 Step 2: Generate the dataset
 Step 3: Split data into training and testing sets
 Step 4: Polynomial feature transformation
 Step 5: Train the linear regression model
 Step 6: Make predictions and evaluate the model
 Step 7: Visualize the results

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 331
Polynomial Regression workflow with
Scikit-learn
 Step 1: Import required libraries
 Step 2: Generate the dataset
 Step 3: Split data into training and testing sets
 Step 4: Polynomial feature transformation
 Step 5: Train the linear regression model
 Step 6: Make predictions and evaluate the model
 Step 7: Visualize the results

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 332
Polynomial Regression – Quiz 1
 Create a dataset with 100 data points for x ranging from -3
to 3.
 Add random noise to the y-values, where the relationship
between x and y is governed by the equation: y = 2*x^3 -
3*x^2 + 5*x -1 + noise
 Use the seed value 42 for reproducibility.
 Split the dataset into training and testing sets with an 80-
20% split ratio, ensuring the results are reproducible by
using a fixed random state.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 333
Polynomial Regression – Quiz 1
(solution)
Polynomial regression
import numpy as np quiz 1:
import [Link] as plt
from [Link] import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split

# Step 1: Generate the dataset

[Link](42) # For reproducibility
x = [Link](-3, 3, 100).reshape(-1, 1) # 100 points
between -3 and 3
noise = [Link](0, 2, size=[Link]) # Add some random
noise
y = 2 * x**3 - 3 * x**2 + 5 * x - 1 + noise # Generate y
values

# Step 2: Train-test split

x_train, x_test, y_train, y_test = train_test_split(x, y,
test_size=0.2, random_state=42)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 334
Polynomial Regression – Quiz 1
(solution)
Polynomial regression quiz 1:
# Step 3: Polynomial features transformation (degree 3)
poly = PolynomialFeatures(degree=3)
x_train_poly = poly.fit_transform(x_train)
x_test_poly = [Link](x_test)

# Step 4: Fit the model

model = LinearRegression()
[Link](x_train_poly, y_train)

# Step 5: Make predictions

y_pred_train = [Link](x_train_poly)
y_pred_test = [Link](x_test_poly)

# Step 6: Evaluate the model

mse_train = mean_squared_error(y_train, y_pred_train)
mse_test = mean_squared_error(y_test, y_pred_test)
r2_train = r2_score(y_train, y_pred_train)
r2_test = r2_score(y_test, y_pred_test)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 335
Polynomial Regression – Quiz 1
(solution)
Polynomial regression
print("Training MSE:",quiz 1:
mse_train)
print("Testing MSE:", mse_test)
print("Training R^2:", r2_train)
print("Testing R^2:", r2_test)

# Step 7: Visualization
[Link](figsize=(10, 6))

# Plot original data

[Link](x, y, color='blue', label='Original Data')

# Plot training data fit

[Link](x_train, y_train, color='green', label='Training
Data')
[Link]([Link](x_train, axis=0),
[Link]([Link]([Link](x_train,
axis=0))),
color='red', label='Model Fit (Training)')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 336
Polynomial Regression – Quiz 1
(solution)
Polynomial regression quiz 1:

# Plot test data fit

[Link](x_test, y_test, color='orange', label='Testing
Data')
[Link]([Link](x_test, axis=0),
[Link]([Link]([Link](x_test,
axis=0))),
color='purple', linestyle='dashed', label='Model Fit
(Testing)')

[Link]("Polynomial Regression (Degree 3)")

[Link]("X")
[Link]("Y")
[Link]()
[Link](True)
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 337
Polynomial Regression – Quiz 1
(solution)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 338
Polynomial Regression
Polynomial regression example:
from sklearn.linear_model import LinearRegression # liner
regression model
from [Link] import PolynomialFeatures #
polynommial features(extended features)
import numpy as np
import [Link] as plt

n = 100 # 100 data points

X = 6 * [Link](n,1)-4
y = X**2 + 2 * X + 3 + [Link](n,1)

# plotting the dataset

[Link](X,y,'b.')
[Link]('X')
[Link]('y')
[Link]('Non-linear Dataset')
[Link]([-3,3,0.5,12])
[Link](True)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 339
Polynomial Regression

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 340
Polynomial Regression
Example: Polynomial regression

Sample: X: Generates 100 random numbers between -4 and

2
y: Defines a quadratic relationship with some noise
added: 𝑦 = 𝑋 + 2𝑋 + 3

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 341
Polynomial Regression
Polynomial regression example (cont’d):
poly_features = PolynomialFeatures(degree=2) # decide the
maximal degree of the polynomial feature
X_poly = poly_features.fit_transform(X) # convert the original
feature to polynomial feature
# check the extened polynomial features of the first data point
print('original feature:', X[0])
print('polynomial features',X_poly[0])

lin_reg = LinearRegression()
lin_reg.fit(X_poly,y)
lin_reg.intercept_, lin_reg.coef_ # check the bais term and
feature weights of the trained model

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 342
Polynomial Regression
Polynomial regression example (cont’d):
X_new = [Link](X,axis = 0) # in order to plot the line of the
model, we need to sort the the value of x-axis
X_new_poly = poly_features.fit_transform(X_new) # compute the
polynomial features
y_predict = lin_reg.predict(X_new_poly) # make predictions
using trained Linear Regression model

# plot the original dataset and the prediction results

fig,ax = [Link]()
[Link](X,y,'b.', label = 'Training date samples')
[Link](X_new,y_predict,'g-',linewidth=2, label =
'Predictions')
[Link]([-3,4,0.5,12])
ax.set_xlabel('X')
ax.set_ylabel('y')
[Link]()
[Link](True)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 343
Polynomial Regression

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 344
Polynomial Regression
Quiz: Polynomial regression
Input:
A dataset consisting of n = 100 data points:
 X: A 1D array of random values between -3 and 3.
 y: A target variable, generated using the formula: y = X^3
+ 2*X^2 + 3*X + noise, where noise is a Gaussian noise
added to the data points.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 345
Polynomial Regression
Quiz: Polynomial regression
Task Requirements:
a. Polynomial Regression (Degree 2 and 3):
 Implement linear regression for the dataset.
 Implement polynomial regression with a degree of 2
and 3.
 Fit a linear model and two polynomial models (degree
2 and degree 3) to the dataset.
b. Predictionpredict the target:
For each model (linear, quadratic, cubic), variable y for a new
set of X values (X_new) in the range from -3 to 3.
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 346
Polynomial Regression
Quiz: Polynomial regression
Task Requirements:
c. Visualization:
 Plot the original dataset
 Plot the predicted values for the three model
d. Calculate the MSE for each model (linear regression,
quadratic regression, cubic regression) using the true target
variable y and the predicted values from each model.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 347
Polynomial Regression
Polynomial regression quiz (solution)
import numpy as np
import [Link] as plt
from sklearn.linear_model import LinearRegression
from [Link] import PolynomialFeatures
from [Link] import mean_squared_error

# Step 1: Create the dataset

n = 100 # Number of data points
X = 6 * [Link](n, 1) - 3 # Random X values between -3
and 3
y = X**3 + 2 * X**2 + 3 * X + [Link](n, 1) # y = x^3
+ 2x^2 + 3x + noise

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 348
Polynomial Regression
Polynomial regression quiz (solution) – cont’d

# Step 2: Add polynomial features for degree 2 and 3

poly_features_2 = PolynomialFeatures(degree=2) # Degree 2 for
quadratic regression
poly_features_3 = PolynomialFeatures(degree=3) # Degree 3 for
cubic regression

X_poly_2 = poly_features_2.fit_transform(X) # Polynomial

features of degree 2
X_poly_3 = poly_features_3.fit_transform(X) # Polynomial
features of degree 3

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 349
Polynomial Regression
Polynomial regression quiz (solution) – cont’d
# Step 3: Create and fit the models
lin_reg = LinearRegression() # Linear Regression model
poly_reg_2 = LinearRegression() # Polynomial Regression of
degree 2
poly_reg_3 = LinearRegression() # Polynomial Regression of
degree 3

# Fit the models

lin_reg.fit(X, y)
poly_reg_2.fit(X_poly_2, y)
poly_reg_3.fit(X_poly_3, y)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 350
Polynomial Regression
Polynomial regression quiz (solution) – cont’d
# Step 4: Make predictions
X_new = [Link](-3, 3, 100).reshape(100, 1) # New X values
for prediction
X_new_poly_2 = poly_features_2.transform(X_new) # Polynomial
features of degree 2
X_new_poly_3 = poly_features_3.transform(X_new) # Polynomial
features of degree 3

y_pred_lin = lin_reg.predict(X_new) # Linear regression

predictions
y_pred_poly_2 = poly_reg_2.predict(X_new_poly_2) # Polynomial
regression predictions of degree 2
y_pred_poly_3 = poly_reg_3.predict(X_new_poly_3) # Polynomial
regression predictions of degree 3

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 351
Polynomial Regression
Polynomial regression quiz (solution) – cont’d
# Step 5: Plot the results
[Link](figsize=(10, 6))
[Link](X, y, color='blue', label='Original Dataset') #
Plot original dataset points
[Link](X_new, y_pred_lin, color='red', label='Linear
regression (Degree 1)')
[Link](X_new, y_pred_poly_2, color='green', label='Polynomial
regression (Degree 2)')
[Link](X_new, y_pred_poly_3, color='orange',
label='Polynomial regression (Degree 3)')
[Link]('x')
[Link]('y')
[Link]('Polynomial Regression Comparisons')
[Link]()
[Link](True)
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 352
Polynomial Regression

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 353
Polynomial Regression
Polynomial regression quiz (solution) – cont’d
# Step 6: Calculate and print Mean Squared Error for each model
mse_lin = mean_squared_error(y, lin_reg.predict(X))
mse_poly_2 = mean_squared_error(y,
poly_reg_2.predict(X_poly_2))
mse_poly_3 = mean_squared_error(y,
poly_reg_3.predict(X_poly_3))

print("Mean Squared Error (Linear Regression):", mse_lin)

print("Mean Squared Error (Polynomial Regression Degree 2):",
mse_poly_2)
print("Mean Squared Error (Polynomial Regression Degree 3):",
mse_poly_3)
>>
Mean Squared Error (Linear Regression): 55.66933039720356
Mean Squared Error (Polynomial Regression Degree 2):
20.60787767157929
Mean Squared Error (Polynomial Regression Degree 3):
0.8800978720459568
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 354
Logistic Regression - Example
Dataset Description: This synthetic dataset predicts student
performance based on two features:
 study_hours: Number of hours spent studying (continuous).
 sleep_hours: Number of hours spent sleeping (continuous).
 passed: Binary target variable (1 if the student passed, 0 if the
student failed).

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 355
Logistic Regression - Example
Objective of the prediction:
The goal is to predict whether a student will pass a course using
logistic regression, focusing on the following:
 Training the model: Fit the logistic regression model to the
data (study and sleep hours) to learn the pass/fail
relationship.
 Evaluating the model: Assess the model's accuracy in
predicting pass/fail on test data.
 Making predictions: Use the trained model to predict the
probability of passing for new students based on their study
and sleep hours.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 356
Logistic Regression - Example
Logistic regression
import pandas as pd example: Task 1. Create the Dataset
import numpy as np

# Generate a simple dataset

[Link](42)
study_hours = [Link](5, 2, 100)
sleep_hours = [Link](7, 1, 100)
passed = (study_hours + sleep_hours + [Link](0, 1,
100)) > 11

# Create a DataFrame
df = [Link]({
'study_hours': study_hours,
'sleep_hours': sleep_hours,
'passed': [Link](int)
})

# Print the first few rows of the dataset

print([Link]())
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 357
Logistic Regression - Example
Logistic regression example: Task 2. Preprocess the Data
from sklearn.model_selection import train_test_split

# Separate features and target

X = df[['study_hours', 'sleep_hours']]
y = df['passed']

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=42)

# Print the shapes of the training and testing sets

print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 358
Logistic Regression - Example
Logistic regression example: Task 3. Build a Logistic
Regression Model
from sklearn.linear_model import LogisticRegression

# Create a logistic regression model

model = LogisticRegression()

# Train the model using the training data

[Link](X_train, y_train)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 359
Logistic Regression - Example
Logistic regression example: Task 4. Evaluate the Model
from [Link] import accuracy_score

# Make predictions on the test data

y_pred = [Link](X_test)

# Calculate the accuracy of the model

accuracy = accuracy_score(y_test, y_pred)

# Print the accuracy

print(f'Accuracy: {accuracy:.2f}')
>>
Accuracy: 0.90

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 360
Logistic Regression - Example
Logistic regression example: Task 5. Predict

# Example new data for prediction

new_data = [[6, 8]]

# Make a prediction
prediction = [Link](new_data)

# Print the prediction

print(f'Predicted class: {prediction[0]}')

>>
Predicted class: 1

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 361
 Logistic Regression - Example
Logistic regression example: Task 6. Plot the Results
import [Link] as plt
from [Link] import ListedColormap

# Define the function to plot decision boundaries

def plot_decision_boundary(X, y, model):
x_min, x_max = [Link][:, 0].min() - 1, [Link][:, 0].max() + 1
y_min, y_max = [Link][:, 1].min() - 1, [Link][:, 1].max() + 1
xx, yy = [Link]([Link](x_min, x_max, 0.01),
[Link](y_min, y_max, 0.01))
Z = [Link](np.c_[[Link](), [Link]()])
Z = [Link]([Link])

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 362
 Logistic Regression - Example
Logistic regression example: Task 6. Plot the Results – cont’d

[Link](xx, yy, Z, alpha=0.3, cmap=ListedColormap(('red',

'blue')))
[Link]([Link][:, 0], [Link][:, 1], c=y, edgecolors='k',
marker='o', cmap=ListedColormap(('red', 'blue')))
[Link]('Study Hours')
[Link]('Sleep Hours')
[Link]('Decision Boundary')
[Link]()

# Plot the decision boundary

plot_decision_boundary(X, y, model)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 363
Logistic Regression - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 364
Navie Bayes - Example
Dataset Description: A synthetic dataset is created to predict a
patient's medical condition based on two features:
 Age (in years)
 Blood Pressure (in mmHg)
⇒ The target variable, Condition: Binary (1 for Positive
condition, 0 for Negative condition).
Condition: 'Positive' if (Age > 50) and (Blood Pressure > 120);
else 'Negative'.

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 365
Navie Bayes - Example
Instructions::
1. Step 1: Generate synthetic data for age, blood pressure, and
medical condition.
2. Step 2: Separate features (X) and target (y), then split into
training and testing sets..
3. Step 3: Train a Naive Bayes model using the training data..
4. Step 4: Predict on test data and calculate accuracy..
5. Step 5: Plot results to visualize predictions versus actual
data.
6. Step 6: Predict the condition of a new patient based on age
and blood pressure..

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 366
Navie Bayes - Example
Navie Bayes example: Step 1. Data Generation
import pandas as pd
import numpy as np

# Set a random seed for reproducibility

[Link](42)

# Generate synthetic data

age = [Link](20, 80, 100) # Age between 20 and 80
years
blood_pressure = [Link](80, 180, 100) # Blood
pressure between 80 and 180 mmHg

# Create a simple rule for medical condition:

# If age > 50 and blood pressure > 120, mark as "Positive"
condition, else "Negative"
condition = [Link]([1 if age[i] > 50 and blood_pressure[i] >
120 else 0 for i in range(100)])

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 367
Navie Bayes - Example
Navie Bayes example: Step 1. Data Generation
# Create a DataFrame
df = [Link]({
'Age': age,
'Blood Pressure': blood_pressure,
'Condition': condition
})

# Print first few rows of the dataset

print([Link]())

>>
Age Blood Pressure Condition
0 58 157 1
1 71 166 1
2 48 141 0
3 34 119 0
4 62 164 1

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 368
Navie Bayes - Example
Navie Bayes example: Step 3. Data Splitting

# Split the data into features (X) and target (y)

X = df[['Age', 'Blood Pressure']]
y = df['Condition']

from sklearn.model_selection import train_test_split

# Split the data into training and testing sets (80% training,
20% testing)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)

# Print the shape of the splits to verify

print(f'Training set size: {X_train.shape}, Testing set size:
{X_test.shape}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 369
Navie Bayes - Example
Navie Bayes example: Step 4. Model Training
from sklearn.naive_bayes import GaussianNB

# Create and train the Gaussian Naive Bayes model

model = GaussianNB()
[Link](X_train, y_train)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 370
Navie Bayes - Example
Navie Bayes example: Step 5. Model Evaluation
from [Link] import accuracy_score

# Make predictions on the test data

y_pred = [Link](X_test)

# Calculate the accuracy of the model

accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy:.2f}')
>>
Accuracy: 0.95

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 371
Navie Bayes - Example
Navie Bayes example: Step 6. Visualization

import [Link] as plt

# Visualizing the results: Plot training data and predictions

[Link](X_train['Age'], X_train['Blood Pressure'],
c=y_train, cmap='coolwarm', label='Training Data')
[Link](X_test['Age'], X_test['Blood Pressure'], c=y_pred,
cmap='winter', marker='x', label='Test Predictions')
[Link]('Age (years)')
[Link]('Blood Pressure (mmHg)')
[Link]('Patient Condition Prediction: Naive Bayes')
[Link]()
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 372
Navie Bayes - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 373
Navie Bayes - Example
Navie Bayes example: Step 7. New Prediction

# Example new data for prediction (patient with age 55 and BP

130)
new_data = [Link]({
'Age': [55],
'Blood Pressure': [130]
})

# Make a prediction for the new data

prediction = [Link](new_data)
print(f'Predicted condition: {"Positive" if prediction[0] == 1
else "Negative"}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 374
Decision tree - Example
Dataset Description: Iris Dataset:
Features (Input variables):
 Sepal length
 Sepal width
 Petal length
 Petal width
⇒ Target variable (Output variable): Species (setosa, versicolor,
virginica)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 375
Decision tree - Example
Decision tree - Example
import [Link] as plt
from [Link] import plot_tree
from sklearn.model_selection import train_test_split
from [Link] import DecisionTreeClassifier
from sklearn import metrics
from [Link] import load_iris

# Load the iris dataset

iris = load_iris()
X = [Link]
y = [Link]

# Split the dataset into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1)

# Create a Decision Tree Classifier

clf = DecisionTreeClassifier()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 376
Decision tree - Example
Decision tree - Example
# Train the model
clf = [Link](X_train,y_train)

# Predict the response for test dataset

y_pred = [Link](X_test)

# Model Accuracy
print("Accuracy:",metrics.accuracy_score(y_test, y_pred))

# Visualize the decision tree

[Link](figsize=(12,8))
plot_tree(clf, filled=True, feature_names=iris.feature_names,
class_names=iris.target_names)
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 377
Decision tree - Example
Decision tree - Example
# Plot the confusion matrix
from [Link] import confusion_matrix
import seaborn as sns

# Create a confusion matrix

cm = confusion_matrix(y_test, y_pred)

# Visualize the confusion matrix

[Link](cm, annot=True, cmap='Blues')
[Link]('Predicted labels')
[Link]('True labels')
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 378
Decision tree - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 379
Decision tree - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 380
K-nearest neighbors (KNN)- Example
K-nearest neighbors (KNN)- Example

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from [Link] import KNeighborsClassifier
from [Link] import accuracy_score
import [Link] as plt

# Set a random seed for reproducibility

[Link](42)

# Generate synthetic data

hours_of_study = [Link](1, 10, 200) # Hours of
study between 1 and 10 hours
attendance_rate = [Link](50, 100, 200) # Attendance
rate between 50% and 100%

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 381
 K-nearest neighbors (KNN)- Example
K-nearest neighbors (KNN)- Example

# Define a simple rule for the final grade (pass or fail):

# Pass if hours_of_study > 5 and attendance_rate > 75, else fail
final_grade = [Link]([1 if hours_of_study[i] > 5 and
attendance_rate[i] > 75 else 0 for i in range(200)])

# Create a DataFrame
df = [Link]({
'Hours of Study': hours_of_study,
'Attendance Rate': attendance_rate,
'Final Grade': final_grade
})

# Split the data into features (X) and target (y)

X = df[['Hours of Study', 'Attendance Rate']]
y = df['Final Grade']

# Split the data into training and testing sets (80% training, 20%
testing)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2,
11/14/2025 random_state=42)
Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 382
K-nearest neighbors (KNN)- Example
K-nearest neighbors (KNN)- Example

# Create and train the KNN model

k = 3 # Using 3 nearest neighbors
model = KNeighborsClassifier(n_neighbors=k)
[Link](X_train, y_train)

# Make predictions on the test data

y_pred = [Link](X_test)

# Calculate the accuracy of the model

accuracy = accuracy_score(y_test, y_pred)
print(f"Model Accuracy on Test Data: {accuracy:.2f}")

# Example new data for prediction (student with 7 hours of study

and 85% attendance)
new_data = [Link]({
'Hours of Study': [7],
'Attendance Rate': [85]
})
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 383
K-nearest neighbors (KNN)- Example
K-nearest neighbors (KNN)- Example

# Make a prediction for the new data

new_prediction = [Link](new_data)
print(f'Predicted Final Grade: {"Pass" if new_prediction[0] == 1
else "Fail"}’)

# Visualize the KNN decision boundary with distinct colors

[Link](figsize=(8, 6))

# Plot training data (Pass: Green, Fail: Red)

train_pass = [Link](X_train['Hours of Study'][y_train==1],
X_train['Attendance Rate'][y_train==1], color='green',
edgecolors='k', s=50, label='Train Pass')
train_fail = [Link](X_train['Hours of Study'][y_train==0],
X_train['Attendance Rate'][y_train==0], color='red',
edgecolors='k', s=50, label='Train Fail')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 384
 K-nearest neighbors (KNN)- Example
K-nearest neighbors (KNN)- Example
# Plot test data predictions (Pass: Blue, Fail: Yellow)
test_pass = [Link](X_test['Hours of Study'][y_pred==1],
X_test['Attendance Rate'][y_pred==1], color='blue',
edgecolors='k', marker='x', s=80, label='Test Pass')
test_fail = [Link](X_test['Hours of Study'][y_pred==0],
X_test['Attendance Rate'][y_pred==0], color='yellow',
edgecolors='k', marker='x', s=80, label='Test Fail')

# Create legend
[Link](loc='best')

[Link]('Hours of Study')
[Link]('Attendance Rate')
[Link]('KNN for Student Final Grade Prediction')
[Link]()
>>
Model Accuracy on Test Data: 0.97
Predicted Final Grade: Pass
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 385
K-nearest neighbors (KNN)- Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 386
 Support Vector Machine (SVM) - Example
Support Vector Machine (SVM) - Example
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from [Link] import SVC
from [Link] import accuracy_score
import [Link] as plt

# Set a random seed for reproducibility

[Link](42)

# Generate synthetic data

age = [Link](20, 70, 200) # Age between 20 and 70 years
bmi = [Link](18, 35, 200) # BMI between 18 and 35 kg/m²

# Define a simple rule for disease status (1 = has disease, 0 = no

disease):
# Patient has disease if age > 40 and BMI > 25, otherwise no disease
disease_status = [Link]([1 if age[i] > 40 and bmi[i] > 25 else 0 for
i in range(200)])
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 387
 Support Vector Machine (SVM) - Example
Support Vector Machine (SVM) - Example
# Create a DataFrame
df = [Link]({
'Age': age,
'BMI': bmi,
'Disease Status': disease_status
})

# Split the data into features (X) and target (y)

X = df[['Age', 'BMI']]
y = df['Disease Status']

# Split the data into training and testing sets (80% training, 20%
testing)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)

# Create and train the SVM model with a linear kernel

model = SVC(kernel='linear')
[Link](X_train, y_train)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 388
 Support Vector Machine (SVM) - Example
Support Vector Machine (SVM) - Example

# Make predictions on the test data

y_pred = [Link](X_test)

# Calculate the accuracy of the model

accuracy = accuracy_score(y_test, y_pred)
print(f"Model Accuracy on Test Data: {accuracy:.2f}")

# Example new data for prediction (patient with age 50 and BMI 28)
new_data = [Link]({
'Age': [50],
'BMI': [28]
})

# Make a prediction for the new data

new_prediction = [Link](new_data)
print(f'Predicted Disease Status: {"Has Disease" if new_prediction[0]
== 1 else "No Disease"}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 389
 Support Vector Machine (SVM) - Example
Support Vector Machine (SVM) - Example
# Optional: Visualizing the SVM decision boundary
[Link](figsize=(8, 6))
[Link](X_train['Age'], X_train['BMI'], c=y_train,
cmap='coolwarm', label='Training Data', edgecolors='k', s=50)
[Link](X_test['Age'], X_test['BMI'], c=y_pred, cmap='winter',
marker='x', label='Test Predictions', s=80)
# Create grid for decision boundary visualization
x_min, x_max = X['Age'].min() - 1, X['Age'].max() + 1
y_min, y_max = X['BMI'].min() - 1, X['BMI'].max() + 1
xx, yy = [Link]([Link](x_min, x_max, 0.1), [Link](y_min,
y_max, 0.1))
# Plot decision boundary
Z = [Link](np.c_[[Link](), [Link]()])
Z = [Link]([Link])
[Link](xx, yy, Z, alpha=0.3, cmap='coolwarm')
[Link]('Age')
[Link]('BMI')
[Link]('SVM for Disease Prediction')
[Link]()
[Link]()
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 390
Support Vector Machine (SVM) - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 391
 Neural Network (NN) Model- Example
Neural
import Network (NN)
pandas as pd Model - Example
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from [Link] import accuracy_score
import [Link] as plt
from [Link] import StandardScaler

# Set a random seed for reproducibility

[Link](42)

# Generate synthetic data

age = [Link](18, 65, 200) # Age between 18 and 65 years
income = [Link](20, 150, 200) # Income between 20K and
150K dollars

# Define a simple rule for purchase status (1 = Purchase, 0 = No

Purchase):
# Customer will purchase if age > 30 and income > 60K, otherwise no
purchase
purchase_status
11/14/2025
= [Link]([1 if age[i] > 30 and income[i] > 60 else
Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 392
0
for i in range(200)])
 Neural Network (NN) Model- Example
Neural Network (NN) Model - Example
# Create a DataFrame
df = [Link]({
'Age': age,
'Income': income,
'Purchase Status': purchase_status
})

# Split the data into features (X) and target (y)

X = df[['Age', 'Income']]
y = df['Purchase Status']

# Preprocess the data: Split the data into training and testing sets
(80% training, 20% testing)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)

# Standardize the data (Neural Networks perform better with

standardized data)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 393
X_test_scaled = [Link](X_test)
 Neural Network (NN) Model- Example
Neural Network (NN) Model - Example

# Build a Neural Network model (Multi-layer Perceptron Classifier)

model = MLPClassifier(hidden_layer_sizes=(10,), max_iter=1000,
random_state=42)
[Link](X_train_scaled, y_train)

# Make predictions on the test data

y_pred = [Link](X_test_scaled)

# Calculate the accuracy of the model

accuracy = accuracy_score(y_test, y_pred)
print(f"Model Accuracy on Test Data: {accuracy:.2f}")

# Example new data for prediction (customer with age 35 and income
80K)
new_data = [Link]([[35, 80]])

# Standardize the new data using the same scaler

new_data_scaled = [Link](new_data)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 394
 Neural Network (NN) Model- Example

Neural Network (NN) Model - Example

# Make a prediction for the new data
new_prediction = [Link](new_data_scaled)
print(f'Predicted Purchase Status: {"Purchase" if new_prediction[0] ==
1 else "No Purchase"}')

# Optional: Visualizing the decision boundary (2D plot)

[Link](figsize=(8, 6))

# Scatter plot for training data (X_train_scaled is now a numpy array)

[Link](X_train_scaled[:, 0], X_train_scaled[:, 1], c=y_train,
cmap='coolwarm', label='Training Data', edgecolors='k', s=50)
[Link](X_test_scaled[:, 0], X_test_scaled[:, 1], c=y_pred,
cmap='winter', marker='x', label='Test Predictions', s=80)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 395
 Neural Network (NN) Model- Example
Neural Network (NN) Model - Example
# Create grid for decision boundary visualization
x_min, x_max = X_train_scaled[:, 0].min() - 1, X_train_scaled[:,
0].max() + 1
y_min, y_max = X_train_scaled[:, 1].min() - 1, X_train_scaled[:,
1].max() + 1
xx, yy = [Link]([Link](x_min, x_max, 0.1), [Link](y_min,
y_max, 0.1))

# Plot decision boundary

Z = [Link](np.c_[[Link](), [Link]()])
Z = [Link]([Link])
[Link](xx, yy, Z, alpha=0.3, cmap='coolwarm')

[Link]('Age')
[Link]('Income')
[Link]('Neural Network for Customer Purchase Prediction')
[Link]()
[Link]()
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 396
Neural Network (NN) Model- Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 397
 K-Means Clustering - Example
K-Means Clustering - Example
import pandas as pd
import numpy as np
from [Link] import StandardScaler
from [Link] import KMeans
import [Link] as plt

# Set a random seed for reproducibility

[Link](42)

# Generate a simple synthetic dataset

annual_income = [Link](15, 135, 200) # Annual income in
thousands of dollars
spending_score = [Link](1, 100, 200) # Spending score
between 1 and 100

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 398
 K-Means Clustering - Example
K-Means Clustering - Example
# Create a DataFrame
df = [Link]({
'Annual Income': annual_income,
'Spending Score': spending_score
})

# Print the first few rows of the dataset

print([Link]())

# Standardize the features

scaler = StandardScaler()
X = scaler.fit_transform(df)

# Print the first few rows of the standardized data

print(X[:5])

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 399
 K-Means Clustering - Example
K-Means Clustering - Example
# Determine the optimal number of clusters using the Elbow Method
inertia = []
K = range(1, 11)
for k in K:
kmeans = KMeans(n_clusters=k, random_state=42)
[Link](X)
[Link](kmeans.inertia_)

# Plot the Elbow Method graph

[Link](figsize=(8, 5))
[Link](K, inertia, 'bx-')
[Link]('Number of Clusters')
[Link]('Inertia')
[Link]('Elbow Method to Determine Optimal Number of Clusters')
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 400
 K-Means Clustering - Example
K-Means Clustering - Example
# Based on the Elbow Method, choose the optimal number of clusters
(e.g., k=4)
optimal_k = 4
kmeans = KMeans(n_clusters=optimal_k, random_state=42)
[Link](X)

# Add the cluster labels to the original DataFrame

df['Cluster'] = kmeans.labels_

# Visualize the clusters

[Link](figsize=(8, 6))
[Link](df['Annual Income'], df['Spending Score'],
c=df['Cluster'], cmap='viridis', s=50, edgecolors='k')
[Link]('Annual Income (k$)')
[Link]('Spending Score')
[Link]('K-means Clustering of Customers')
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 401
 K-Means Clustering - Example
K-Means Clustering - Example

# Print cluster centers

print('Cluster Centers:')
print(scaler.inverse_transform(kmeans.cluster_centers_))

# Example new data for clustering (customers with new income and
spending scores)
new_data = [Link]([[50, 75], [85, 20]])
new_data_scaled = [Link](new_data)

# Predict the cluster for the new data

new_predictions = [Link](new_data_scaled)
print(f'Predicted Clusters for New Data: {new_predictions}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 402
K-Means Clustering - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 403
K-Means Clustering - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 404
 Hierarchical Clustering - Example
Hierarchical Clustering - Example
import pandas as pd
import numpy as np
from [Link] import StandardScaler
from [Link] import dendrogram, linkage, fcluster
import [Link] as plt

# Set a random seed for reproducibility

[Link](42)

# Generate a simple synthetic dataset

study_hours = [Link](1, 10, 200) # Study hours per week
exam_scores = [Link](50, 100, 200) # Exam scores out of
100

# Create a DataFrame
df = [Link]({
'Study Hours': study_hours,
'Exam Scores': exam_scores
})
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 405
 Hierarchical Clustering - Example
Hierarchical Clustering - Example
# Print the first few rows of the dataset
print([Link]())

# Standardize the features

scaler = StandardScaler()
X = scaler.fit_transform(df)

# Print the first few rows of the standardized data

print(X[:5])

# Perform hierarchical clustering using the 'ward' linkage method

Z = linkage(X, method='ward')

# Create a dendrogram
[Link](figsize=(10, 7))
dendrogram(Z, truncate_mode='level', p=5)
[Link]('Dendrogram for Hierarchical Clustering')
[Link]('Data Points')
[Link]('Euclidean Distance')
[Link]()
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 406
 Hierarchical Clustering - Example
Hierarchical Clustering - Example
# Based on the dendrogram, choose the optimal number of clusters
(e.g., 3)
optimal_clusters = 3
clusters = fcluster(Z, optimal_clusters, criterion='maxclust')

# Add the cluster labels to the original DataFrame

df['Cluster'] = clusters

# Visualize the clusters

[Link](figsize=(8, 6))
[Link](df['Study Hours'], df['Exam Scores'], c=df['Cluster'],
cmap='viridis', s=50, edgecolors='k')
[Link]('Study Hours per Week')
[Link]('Exam Scores')
[Link]('Hierarchical Clustering of Student Performance')
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 407
 Hierarchical Clustering - Example
Hierarchical Clustering - Example

# Example new data for clustering (students with new study hours and
exam scores)
new_data = [Link]([[5, 80], [9, 95]])
new_data_scaled = [Link](new_data)

# Calculate cluster centers for existing clusters

cluster_centers = [Link]([X[clusters == k].mean(axis=0) for k in
range(1, optimal_clusters + 1)])
print('Cluster Centers:')
print(scaler.inverse_transform(cluster_centers))

# Assign new data to the closest cluster center

def assign_clusters(new_data_scaled, cluster_centers):
distances = [Link](new_data_scaled[:, [Link]] -
cluster_centers, axis=2)
return [Link](distances, axis=1) + 1

new_predictions = assign_clusters(new_data_scaled, cluster_centers)

print(f'Predicted
11/14/2025 Clusters
Dr. Mai Cao Lanfor New
- Faculty Data:
of Geology {new_predictions}')
& Petroleum Engineering, HCMUT 408
Hierarchical Clustering - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 409
Hierarchical Clustering - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 410
 Gaussian Mixture Model- Example
Gaussian Mixture Model - Example
import pandas as pd
import numpy as np
from [Link] import StandardScaler
from [Link] import GaussianMixture
import [Link] as plt

# Set a random seed for reproducibility

[Link](42)

# Generate a simple synthetic dataset

annual_spending = [Link](1, 100, 200) # Annual spending in
thousands of dollars
frequency_of_visits = [Link](1, 50, 200) # Frequency of
visits per year

# Create a DataFrame
df = [Link]({
'Annual Spending': annual_spending,
'Frequency of Visits': frequency_of_visits
}) 11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 411
 Gaussian Mixture Model- Example
Gaussian Mixture Model - Example

# Print the first few rows of the dataset

print([Link]())

# Standardize the features

scaler = StandardScaler()
X = scaler.fit_transform(df)

# Print the first few rows of the standardized data

print(X[:5])

# Determine the optimal number of clusters using the Bayesian

Information Criterion (BIC)
bic = []
n_components = range(1, 11)
for n in n_components:
gmm = GaussianMixture(n_components=n, random_state=42)
[Link](X)
[Link]([Link](X))
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 412
 Gaussian Mixture Model- Example
Gaussian Mixture Model - Example
# Plot the BIC scores
[Link](figsize=(8, 5))
[Link](n_components, bic, 'bx-')
[Link]('Number of Clusters')
[Link]('BIC')
[Link]('BIC to Determine Optimal Number of Clusters')
[Link]()

# Based on the BIC, choose the optimal number of clusters (e.g., n=3)
optimal_n = 3
gmm = GaussianMixture(n_components=optimal_n, random_state=42)
[Link](X)

# Add the cluster labels to the original DataFrame

df['Cluster'] = [Link](X)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 413
 Gaussian Mixture Model- Example
Gaussian Mixture Model - Example

# Visualize the clusters

[Link](figsize=(8, 6))
[Link](df['Annual Spending'], df['Frequency of Visits'],
c=df['Cluster'], cmap='viridis', s=50, edgecolors='k')
[Link]('Annual Spending (k$)')
[Link]('Frequency of Visits')
[Link]('Gaussian Mixture Model Clustering of Customers')
[Link]()

# Print cluster means and covariances

print('Cluster Means:')
print(scaler.inverse_transform(gmm.means_))
print('Cluster Covariances:')
print(gmm.covariances_)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 414
 Gaussian Mixture Model- Example
Gaussian Mixture Model - Example
# Example new data for clustering (customers with new annual spending
and visit frequencies)
new_data = [Link]([[30, 10], [70, 40]])
new_data_scaled = [Link](new_data)

# Predict the cluster for the new data

new_predictions = [Link](new_data_scaled)
print(f'Predicted Clusters for New Data: {new_predictions}')

>>
Predicted Clusters for New Data: [2 0]

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 415
Gaussian Mixture Model- Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 416
Gaussian Mixture Model- Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 417
 DBSCAN Clustering - Example
DBSCAN Clustering- Example

import numpy as np
from [Link] import DBSCAN
import [Link] as plt

# Generate sample data

# Replace this with your actual data
[Link](42)
centers = [[1, 1], [5, 5], [3, 8]]
X, _ = make_blobs(n_samples=1000, centers=centers, cluster_std=0.6,
random_state=0)

# Compute DBSCAN
db = DBSCAN(eps=0.3, min_samples=10).fit(X)
core_samples_mask = np.zeros_like(db.labels_, dtype=bool)
core_samples_mask[db.core_sample_indices_] = True
labels = db.labels_

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 418
 DBSCAN Clustering - Example
DBSCAN Clustering - Example

# Number of clusters in labels, ignoring noise if present.

n_clusters_ = len(set(labels)) - (1 if -1 in labels else 0)
n_noise_ = list(labels).count(-1)

print('Estimated number of clusters: %d' % n_clusters_)

print('Estimated number of noise points: %d' % n_noise_)

# Plot result
unique_labels = set(labels)
colors = [[Link](each)
for each in [Link](0, 1, len(unique_labels))]

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 419
 DBSCAN Clustering - Example
DBSCAN Clustering - Example
for k, col in zip(unique_labels, colors):
if k == -1:
# Black used for noise.
col = [0, 0, 0, 1]

class_member_mask = (labels == k)

xy = X[class_member_mask & core_samples_mask]

[Link](xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
markeredgecolor='k', markersize=14)

xy = X[class_member_mask & ~core_samples_mask]

[Link](xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
markeredgecolor='k', markersize=6)

[Link]('Estimated number of clusters: %d' % n_clusters_)

[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 420
DBSCAN Clustering - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 421
 DBSCAN Clustering - Example
DBSCAN Clustering - Example
for k, col in zip(unique_labels, colors):
if k == -1:
# Black used for noise.
col = [0, 0, 0, 1]

class_member_mask = (labels == k)

xy = X[class_member_mask & core_samples_mask]

[Link](xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
markeredgecolor='k', markersize=14)

xy = X[class_member_mask & ~core_samples_mask]

[Link](xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
markeredgecolor='k', markersize=6)

[Link]('Estimated number of clusters: %d' % n_clusters_)

[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 422
 Principal components analysis - Example
Principal components analysis - Example
import pandas as pd
import numpy as np
from [Link] import load_iris
from [Link] import StandardScaler
from [Link] import PCA
import [Link] as plt

# Load the Iris dataset

iris = load_iris()
data = [Link](data= np.c_[iris['data'], iris['target']],
columns= iris['feature_names'] + ['target'])

# Separate features and target variable

X = [Link]('target', axis=1)
y = data['target']

# Standardize the data

scaler = StandardScaler()
scaled_data = scaler.fit_transform(X)
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 423
 Principal components analysis - Example

Principal components analysis - Example

# Create a PCA object with 2 components
pca = PCA(n_components=2)

# Fit PCA to the data

principalComponents = pca.fit_transform(scaled_data)

# Create a DataFrame of the principal components

principalDf = [Link](data = principalComponents
, columns = ['principal component 1', 'principal
component 2'])

# Concatenate with target variable

finalDf = [Link]([principalDf, y], axis = 1)

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 424
 Principal components analysis - Example
Principal components analysis - Example
# Visualize the results
[Link](figsize=(10,8))
[Link](x=finalDf['principal component 1'],
y=finalDf['principal component 2'],
c=finalDf['target'],
cmap='viridis')
[Link]('Principal Component 1')
[Link]('Principal Component 2')
[Link]('2D PCA of Iris Dataset')
[Link]()
[Link]()

# Explained variance ratio

print('Explained variance ratio:', pca.explained_variance_ratio_)

# Cumulative explained variance ratio

print('Cumulative explained variance ratio:',
[Link](pca.explained_variance_ratio_))
11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 425
Principal components analysis - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 426
 Factor analysis - Example
Factor analysis - Example

import pandas as pd
from [Link] import StandardScaler
from [Link] import FactorAnalysis
import [Link] as plt
import numpy as np

# Load the Wine Quality dataset from a CSV file

url = '[Link]
quality/[Link]'
df = pd.read_csv(url, delimiter=';')

# Print the first few rows of the dataset

print([Link]())

# Handle missing values (if any)

df = [Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 427
 Factor analysis - Example
Factor analysis - Example

# Standardize the features

scaler = StandardScaler()
X = scaler.fit_transform(df)

# Print the first few rows of the standardized data

print(X[:5])

# Perform Factor Analysis

n_components = 5 # Number of factors to extract
fa = FactorAnalysis(n_components=n_components, random_state=42)
X_fa = fa.fit_transform(X)

# Print the factors

print(f'Factors:\n{X_fa[:5]}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 428
 Factor analysis - Example
Factor analysis - Example
# Visualize the factor loadings
[Link](figsize=(10, 8))
loadings = fa.components_.T
[Link](loadings, cmap='viridis', aspect='auto')
[Link]()
[Link](range(n_components), [f'Factor {i+1}' for i in
range(n_components)])
[Link](range([Link][1]), [Link])
[Link]('Factor Loadings')
[Link]('Factors')
[Link]('Features')
[Link]()

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 429
 Factor analysis - Example
Factor analysis - Example

# Print the factor loadings

print('Factor Loadings:')
print(loadings)

# Example new data for transformation (new wine samples)

new_data = [Link][:2, :] # First two samples from the dataset for
demonstration
new_data_scaled = [Link](new_data)

# Transform the new data using the FA model

new_data_fa = [Link](new_data_scaled)
print(f'Transformed New Data:\n{new_data_fa}')

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 430
Factor analysis - Example

11/14/2025 Dr. Mai Cao Lan - Faculty of Geology & Petroleum Engineering, HCMUT 431