DDA3020 Machine Learning:

Lecture 1 Introduction

JIA, Kui
School of Data Science, CUHK-SZ

January 07/09, 2026

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 1 / 46
 Outline

1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 2 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 3 / 46
 Instructors and teaching assistants (Session 1)
Time and venue:
Lecture: Monday/Wednesday 9:00–10:20 am, Administration Bldg E101
Tutorial: TBD
Instructor:
Baoyuan Wu (wubaoyuan@[Link])
Personal homepage: [Link]
Office hour (OH): 10:30–11:30 am, Monday in the even weeks (2/4/6/8/10/12), TA
412a
TAs:
Meixi Zheng: 223040254@[Link]
OH: Mon 11:00-12:00, TXA 101
Yuan Xie: 224040374@[Link]
OH: Tue 16:00-17:00, DaoYuan 318
Jun’an Chen: 225040494@[Link]
OH: Tue 16:00-17:00, DaoYuan 318
Yueci Deng: 225040502@[Link]
OH: Tue 14:00-15:00, TXB 609
Dekun Chen: dekunchen@[Link]
OH: Wed 14:00-15:00, TXC 604
Fanzeng Xia: 223040232@[Link]
OH: Mon 9:00-10:00, TXB 611
JIA, Kui School of Data Science, CUHK-SZ
DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 4 / 46
 Instructors and teaching assistants (Session 2)
Time and venue:
Lecture: Wednesday/Friday 10:30–11:50 am, Administration Bldg E101
Tutorial: TBD
Kui Jia (kuijia@[Link])
Personal homepage: [Link]
Office hour (OH): 14:00–15:00 pm, Wednesday in even weeks (2/4/6/8/10/12), TxC
607
TAs:
Meixi Zheng: 223040254@[Link]
OH: Mon 11:00-12:00, TXA 101
Yuan Xie: 224040374@[Link]
OH: Tue 16:00-17:00, DaoYuan 318
Jun’an Chen: 225040494@[Link]
OH: Tue 16:00-17:00, DaoYuan 318
Yueci Deng: 225040502@[Link]
OH: Tue 14:00-15:00, TXB 609
Dekun Chen: dekunchen@[Link]
OH: Wed 14:00-15:00, TXC 604
Fanzeng Xia: 223040232@[Link]
OH: Mon 9:00-10:00, TXB 611
JIA, Kui School of Data Science, CUHK-SZ
DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 5 / 46
 Agenda with tutorials

Week Content Homework Tutorial

W1 Introduction None
W2 Review of Probability & Linear Algebra etc. python and sklearn
W3 Linear Regression I Linear regression coding
W4 Linear Regression II & Logistic Regression H1 release Logistic regression coding
W5 Support Vector Machines SVM coding
W6 Decision Tree and Random Forest H1
W7 Neural Networks I (MLP & CNN ) Pytorch coding
W8 Neural Networks II (RNN & Transformer) H2 release Pytorch coding
W9 Over-fitting, Bias-Variance Trade-off Demo
W10 Performance Evaluation H2
W11 Intro. to Unsupervised Learning, K-Means K-means coding
W12 Mixture Models, EM algorithm H3 release GMM coding
W13 PCA PCA coding
W14 Review

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 6 / 46
 Learning materials

Textbooks:
Required: Andriy Burkov, “The Hundred-Page Machine Learning Book”,
2019. (read first, buy later: Link)
Required: K. Murphy. Machine Learning: A Probabilistic Perspective.
MIT Press, 2012. (Link)
Recommended: Andreas C. Muller and Sarah Guido, “Introduction to Ma-
chine Learning with Python: A Guide for Data Scientists”, O’Reilly Media,
Inc., 2017. (The PDF will be uploaded to the BB system, only used for
learning)
Recommended: Jeff Leek, “The Elements of Data Analytic Style: A guide
for people who want to analyze data”, Lean Publishing, 2015. (Link)
Recommended: C. Bishop. Pattern Recognition and Machine Learning.
Springer, 2011. (Link)

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 7 / 46
 Grading policy
Grading:
25% Written homework assignments (3 times)
25% Programming homework assignments (Python/Scikit-learn) (3 times)
50% Final exam
Submission: You have 2 weeks to independently complete each assignment
(written + programming). Late submission will get discounted score: (0, 48]
hours → 50%; (48, ∞) hours → 0%.
Requirements to finish the homework smoothly: basic knowledge of probability
and linear algebra, familiarity with Python programming, and correct under-
standing of the teaching content.
Plagiarism: Zero mark is given for the whole assignment (including written
and programming) in the first plagiarism case. Students will FAIL the whole
course for repeated plagiarism. Note: If there are heavy overlaps between
two answers, then both will be identified as plagiarism (we don’t have time
to distinguish). Thus, discussions are encouraged, but you must finish the
assignment by yourself, and don’t share your answer with others.
JIA, Kui School of Data Science, CUHK-SZ
DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 8 / 46
 LLM policy

The use of LLMs or AI agents to assist your learning, homework, or program-

ming is permitted.
However, you must clearly declare such usage. Any work identified as AI-
generated without a proper declaration will be treated as plagiarism.

Be the master of AI, not its slave!

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 9 / 46
 Students’ background
The students come from diverse years and majors, thus, may have
different expectations and experiences.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 10 / 46
 Students’ background

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 11 / 46
 Course difficulty

Machine learning is a complex and evolving discipline. Our course will

cover the fundamental concepts and algorithms in machine learning.
Fundamental 6= SIMPLE. Machine learning is built on several other
foundation subjects, including probability, linear algebra, optimiza-
tion, computer science, etc.
If you find it difficult to understand the teaching content, please don’t
hesitate to tell me or TAs, and we will help you overcome the challenges. If
you feel the content is too easy, no worry, we can provide further reading
materials on advanced and emerging topics (e.g., deep learning, adversarial
machine learning). Wish each of you an enjoyable learning experience!

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 12 / 46
 Communications

Smooth communications between you and us are very important to build a

successful course. If any questions/difficulties/suggestions, you are welcome to
Talk to me directly, after class or during office hours
Email or talk to your TAs, questions will be collected and sent to me weekly
for responses
Questions about programming are handled by TAs
You may also send me emails if you need my help

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 13 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 14 / 46
 Definition of machine learning

Arthur Samuel: “the field of study that gives computers the ability to learn
without being explicitly programmed.”

Tom Mitchell: “A computer program is said to learn from experience E

with respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with experience E.”

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 15 / 46
 Definition of machine learning

Tom Mitchell: “A computer program is said to learn from experience E with

respect to some class of tasks T and performance measure P, if its performance
at tasks in T, as measured by P, improves with experience E.”

Suppose your email program watches which emails you do or do not mark as
spam, and based on that learns how to better filter spam. What is the task T
in this setting?
• Classifying emails as spam or not spam.
• Watching you label emails as spam or not spam.
• The number (or fraction) of emails correctly classified as spam/not spam.
• None of the above – this is not a machine learning problem.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 16 / 46
 ML is a branch of artificial intelligence

Artificial intelligence (AI) is intelligence demonstrated by machines (e.g.,

computer, robots), unlike the natural intelligence displayed by humans and
animals, which involves consciousness and emotionality.
AI covers many topics, such as machine learning (ML), computer vision
(CV), natural language processing (NLP), and speech processing.

Figure: Machine learning is one of the most important branches of artificial intelligence.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 17 / 46
 Connection with other disciplines

Figure: Machine learning is interdisciplinary.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 18 / 46
 Applications of machine learning

Figure: Machine learning has been widely used in many mission-critical tasks, such as
Game (AlphaGo), protein structure prediction (AlphaFold2), EEG signal processing,
medical image diagnosis, face recognition, etc.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 19 / 46
 Two basic paradigms of ML

Given some data x1 , x2 , x3 , . . ., you can do

Supervised learning: you are also provided some human-labeled outputs
y1 , y2 , y3 , . . ., and your task is to learn a mapping function from one input
data xi to one output yi . Learning from teacher
Unsupervised learning: your task is to build/learn a good model of
x, such that some characteristics of the data could be revealed, such as
clustering, dimensionality reduction, etc. Learning by oneself

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 20 / 46
 Other learning paradigms

Reinforcement learning: you can make some actions ai to change the

data xi (the state of the environment), and you will receive some re-
wards/punishments ri . Gradually, you will automatically learn to make
suitable actions for different data to get more rewards. It is one of three
basic machine learning paradigms, alongside supervised learning and un-
supervised learning. Learning from rewards/punishments

In this course, we will focus on supervised learning and unsupervised learning.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 21 / 46
 Supervised vs. unsupervised learning

Refer to:
[Link]

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 22 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 23 / 46
 Supervised learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 24 / 46
 Supervised learning

In supervised learning, the dataset is the collection of labeled examples,

denoted as {(xi , yi )}N
i=1 ,

(1) (j) (D) >

xi = [xi , . . . , xi , . . . , xi ] , i = 1, . . . , N

Each element xi is called a feature vector: it is a vector in which each

dimension j = 1, . . . , D contains a value that describes the example some-
how.
The label yi can be either an element belonging to a finite set of classes
{1, 2, . . . , C}, or a real number.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 25 / 46
 How supervised learning works

General procedure of supervised learning:

Data collection: {(xi , yi )}N
i=1
Training: conducting model training on the training data to learn the
model parameters
Inference (test): Using the trained model to predict the output of unseen
data x

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 26 / 46
 Regression example

Task: To predict the price of diamond as a function of mass

Performance: Accuracy of predicted values
Experience: Historical data

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 27 / 46
 Classification example

Task: To classify the input data into two categories

Performance: Classification accuracy
Experience: Historical data
JIA, Kui School of Data Science, CUHK-SZ
DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 28 / 46
 Regression vs. classification

Suppose that you are running a company, and you want to develop learning
algorithms to address the following two problems.

Problem 1: You have a large inventory of identical items. You want to predict
how many of these items will sell over the next 3 months.

Problem 2: You’d like a program to examine individual customer accounts, and

for each account, decide if it has been hacked or compromised.

Should you treat these as classification or regression problems?

Treat both as classification problems.
Treat problem 1 as classification, and problem 2 as regression.
Treat problem 1 as regression, and problem 2 as classification.
Treat both as regression problems.
Regression with continuous output vs. Classification with finite and discrete or
categorical outputs

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 29 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 30 / 46
 Unsupervised learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 31 / 46
 Unsupervised learning

In unsupervised learning, the dataset is a collection of unlabeled ex-

amples, i.e., {xi }N
i=1 .
Again, x is a feature vector, and the goal of an unsupervised learning
algorithm is to create a model that takes a feature vector x as input and
either transforms it into another vector or into a value that can be used to
solve a practical problem.
Its main task is to analyze the structure of data for future inference.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 32 / 46
 How unsupervised learning works

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 33 / 46
 Clustering example

Task: to partition a set of unlabeled points to clusters.

Performance (for test data):
points within the same cluster are close to each other
points from different clusters are far from each other
the clusters have an appropriate coverage of all data
Experience: available data

The key question is how to define and measure close or far, depending on what
features are adopted (e.g., global Euclidean distance, local distance, shape ... )
JIA, Kui School of Data Science, CUHK-SZ
DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 34 / 46
 Dimensionality reduction example

The purposes of dimensionality reduction:

Data simplification: non-linear → linear
Data visualization: high dimensional → low-dimensional
Reduce noise: some dimensions of the input data may be noises
Variable selection for prediction: learn a sparse model, if there are
redundancies among different dimensions

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 35 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 36 / 46
 Basic concepts in supervised learning

Data for supervised learning:

Training set: Dtrain = {(xi , yi )}n i , with xi ∈ X being the feature pre-
sentation (could be scalar, image/video, text sequence, graph, cloud point,
etc.), yi ∈ Y being the supervision/ground-truth value (could be a discrete
label, continuous value, vector, sequence, tensor, etc.).
Testing set: Dtest = {(xi , yi )}m
i is used to evaluate the performance of the
trained model.
Data for unsupervised learning
Training set: Dtrain = {xi }ni , with xi ∈ X being the feature presentation
(could be scalar, image/video, text sequence, graph, cloud point, etc.).
Testing set: Dtest = {xi }m i is used to evaluate the performance of the
trained model.
Independent and identically distributed (i.i.d.) assumption: In
standard machine learning, it assumes that all samples are observations (or
realizations) of independent and identical random variables, and training
and testing sets follow the same distribution.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 37 / 46
 Basic concepts in supervised learning
Target function t : X → Y: the ground-truth mapping function from the
input to the output behind the training/testing data. It is unknown, and
our goal is to find it.
Hypothesis h: A hypothesis is a candidate function that describes the
unknown target function, for example
 
x
h(x) = 1 × x1 + 2 × x2 = [1, 2] · 1
x2
Hypothesis space H: Hypothesis space is the set of all the possible legal
hypotheses, for example
Hw (x) = w1 × x1 + w2 × x2 = w> x

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 38 / 46
 Basic concepts in supervised learning
Cost function is a measure of how good/bad the hypothesis is in terms of
its ability to estimate the relationship between x and y, such as the square
loss (h(x) − y)2 .
Objective function is the function that we want to optimize (minimize,
maximize, or minimax). When we are minimizing it, we may also call it
the cost function, loss function, or error function. In this course, we use
these terms interchangeably.
Training/learning is the process of searching a good hypothesis h in the
hypothesis space H, through optimizing the objective function on a training
set Dtrain using the optimization method, such as
1 X
h∗ = arg min (h(xi ) − yi )2
h∈H n
(xi ,yi )∈Dtrain

Testing/evaluation: evaluating the performance of the learned model h∗

on a test set Dtest , such as
1 X
(h∗ (xi ) − yi )2
m
(xi ,yi )∈Dtest

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 39 / 46
 A general machine learning workflow

1 Collecting data: Be it the raw data from Excel, access, text files, etc.,
this step (gathering past data) forms the foundation of future learning.
The better the variety, density, and volume of relevant data, the better the
learning prospects for the machine become.
2 Preprocessing data: One needs to spend time determining the quality
of data and then taking steps for fixing issues such as missing data and
treatment of outliers.
3 Determining the hypothesis space, objective function, optimiza-
tion method.
4 Training: learning the parameters of the hypothesis function through op-
timizing the objective function.
5 Testing: evaluating the learned model on testing data.
6 Improving the performance: This step might involve choosing a dif-
ferent model altogether or introducing more variables to augment the effi-
ciency. That’s why a significant amount of time needs to be spent on data
collection and preparation.

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 40 / 46
 Review of this lecture

Machine learning definition: “A computer program is said to learn from

experience E with respect to some class of tasks T and performance
measure P, if its performance at tasks in T, as measured by P, improves
with experience E.”
Three main paradigms of machine learning:
Supervised learning: learning from teacher
Unsupervised learning: learning by oneself
Reinforcement learning: learning from rewards/punishments
Many other learning paradigms . . .
Supervised learning: regression and classification.
Unsupervised learning: clustering and dimensionality reduction.
Some basic concepts

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 41 / 46
 Further reading

Other learning paradigms:

Reinforcement learning: an area of machine learning concerned with
how intelligent agents ought to take actions in an environment in order to
maximize the notion of cumulative reward. (Link)
Semi-supervised learning: labeled data + unlabeled data (Link)
Ensemble learning: learning with multiple ML models, and the final pre-
diction is obtained by combining these models. Improving the performance
of individual models 三个臭皮匠，顶个诸葛亮 (Link)
Transfer learning: source domain data + target domain data, useful
especially when the target data is insufficient (Link)
Federated learning: learning the model at local servers using local data,
then uploading locally updated parameters to the central server to obtain
the unified parameters. Protecting users’ privacy (Link)
Machine unlearning: erasing the effect of some particular training sam-
ples from a trained model. Protecting users’ privacy (Link)

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 42 / 46
 1 About this course

2 What is machine learning

3 Supervised learning

4 Unsupervised learning

5 Some basic concepts in machine learning

6 Practice for machine learning

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 43 / 46
 Practice: Scikit-Learn
[Link]

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 44 / 46
 Practice: UCI Data Repository

[Link]

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 45 / 46
 Practice: Kaggle
[Link]

JIA, Kui School of Data Science, CUHK-SZ

DDA3020 Machine Learning: Lecture 1 Introduction
January 07/09, 2026 46 / 46