Agenda
• Introduction to supervised learning
• A case study with simple classification
approach
• Evaluation the classification
• Make it practical: un upgraded version
Supervised vs. Unsupervised
Learning
Classification: Overview
• Classification
Training Data Classification Model Make Predictions on
unseen Data
• Example :
• Play goft
• Neuron Network
Process 1: Model Construction
Classification
Algorithms
Training
Data
Classifier
(Model)
IF rank = ‘professor’
OR years > 6
THEN tenured = ‘yes’
9
Process 2: Using the Model in
Prediction
Classifier
Testing
Data Unseen Data
(Jeff, Professor, 4)
Tenured
?
Classification Algorithms
• Classification Algorithms:
• Support Vector Machines
• Neural Network (multi-layer perceptron)
• Decision Tree
• K-Nearest Neighbor
• Naive Bayes Classifier…
K-Nearest Neighbor
• Classifying objects based on closest training
examples in the feature space
• Approximated locally
• All computation is deferred until
classification
• Classified by a majority vote of its neighbors
Data
• The training examples are vectors in a
multidimensional feature space, each with a
class label
• The training phase of the algorithm consists
only of storing the feature vectors and class
labels of the training samples
Algorithm
- k is a user-defined constant
- Choose k nearest neighbors (NNS)
- Label is label which is most frequent among
the k training samples nearest.
K-Nearest Neighbor
• Classifying objects based on closest training
examples in the feature space
• Approximated locally
• All computation is deferred until
classification
• Classified by a majority vote of its neighbors
Similarity and Representation
• How do we define similarity?
• How do we represent the objects whose
similarities we wish to measure?