Introduction
Welcome
Machine Learning
Introduction
What is machine
learning
Machine Learning
Andrew Ng
Machine Learning definition
• Arthur Samuel (1959). Machine Learning: Field of
study that gives computers the ability to learn
without being explicitly programmed.
• Tom Mitchell (1998) Well-posed Learning
Problem: A computer program is said to learn
from experience E with respect to some task T
and some performance measure P, if its
performance on T, as measured by P, improves
with experience E.
Andrew Ng
“A computer program is said to learn from experience E with respect to
some task T and some performance measure P, if its performance on T,
as measured by P, improves with experience E.”
Suppose your email program watches which emails you do or do
not mark as spam, and based on that learns how to better filter
spam. What is the task T in this setting?
Classifying emails as spam or not spam.
Watching you label emails as spam or not spam.
The number (or fraction) of emails correctly classified as spam/not spam.
None of the above—this is not a machine learning problem.
A computer program is said to learn from experience E with
respect to some task T and some performance measure P if its
performance on T, as measured by P, improves with experience E.
Suppose we feed a learning algorithm a lot of historical weather
data, and have it learn to predict weather. In this setting, what is
E?
The weather prediction task.
The probability of it correctly predicting a future date’s weather.
The process of the algorithm examining a large amount of historical weather data.
None of the above.
Machine learning algorithms:
- Supervised learning
- Unsupervised learning
Others: Reinforcement learning, recommender
systems.
Andrew Ng
Introduction
Supervised
Learning
Machine Learning
Andrew Ng
Housing price prediction.
Price ($)
in 1000’s
Size in feet2
Supervised Learning Regression: Predict continuous
“right answers” given valued output (price)
Andrew Ng
Cancer (malignant, benign)
1(Y)
Classification
Discrete valued
Malignant?
output (0 or 1)
0(N)
Tumor Size
Tumor Size
Andrew Ng
- Clump Thickness
- Uniformity of Cell Size
Age - Uniformity of Cell Shape
…
Tumor Size
Andrew Ng
You’re running a company, and you want to develop learning algorithms to address
each of two problems.
Problem 1: You have a large inventory of identical items. You want to predict how
many of these items will sell over the next 3 months.
Problem 2: You’d like software to examine individual customer accounts, and for each
account decide if it has been hacked/compromised.
Should you treat these as classification or as regression problems?
Treat both as classification problems.
Treat problem 1 as a classification problem, problem 2 as a regression problem.
Treat problem 1 as a regression problem, problem 2 as a classification problem.
Treat both as regression problems.
• Suppose you are working on stock market prediction, and you would like to predict
the price of a particular stock tomorrow (measured in dollars). You want to use a
learning algorithm for this.
Would you treat this as a classification or a regression problem?
• Suppose you are working on stock market prediction. You would like to predict
whether or not a certain company will declare bankruptcy within the next 7 days
(by training on data of similar companies that had previously been at risk of
bankruptcy).
Would you treat this as a classification or a regression problem?
• Suppose you are working on stock market prediction, Typically tens of millions of
shares of Microsoft stock are traded (i.e., bought/sold) each day. You would like to
predict the number of Microsoft shares that will be traded tomorrow.
Would you treat this as a classification or a regression problem?
Andrew Ng
Introduction
Unsupervised
Learning
Machine Learning
Andrew Ng
Supervised Learning
x2
x1
Andrew Ng
Unsupervised Learning
x2
x1
Andrew Ng
Andrew Ng
Andrew Ng
Organize computing clusters Social network analysis
Market segmentation
Andrew Ng
Cocktail party problem
Speaker #1 Microphone #1
Speaker #2 Microphone #2
Andrew Ng
Microphone #1: Output #1:
Microphone #2: Output #2:
Microphone #1: Output #1:
Microphone #2: Output #2:
[Audio clips courtesy of Te-Won Lee.] Andrew Ng
Of the following examples, which would you address using an
unsupervised learning algorithm? (Check all that apply.)
Given email labeled as spam/not spam, learn a spam filter.
Given a set of news articles found on the web, group them into
set of articles about the same story.
Given a database of customer data, automatically discover market
segments and group customers into different market segments.
Given a dataset of patients diagnosed as either having diabetes or
not, learn to classify new patients as having diabetes or not.
• Some of the problems below are best addressed using a supervised learning
algorithm, and the others with an unsupervised learning algorithm. Which of
the following would you apply supervised learning to? (Select all that apply.)
In each case, assume some appropriate dataset is available for your
algorithm to learn from.
– Take a collection of 1000 essays written on the US Economy, and find a way to
automatically group these essays into a small number of groups of essays that
are somehow “similar” or “related”.
– Given genetic (DNA) data from a person, predict the odds of him/her developing
diabetes over the next 10 years.
– Examine a large collection of emails that are known to be spam email, to
discover if there are sub-types of spam mail.
– Examine the statistics of two football teams, and predict which team will win
tomorrow’s match (given historical data of teams’ wins/losses to learn from).
Andrew Ng