0% found this document useful (0 votes)
12 views5 pages

Classification and Model Evaluation Guide

The document discusses classification in machine learning, focusing on predicting whether the number of rooms in a property exceeds three based on area and price. It outlines the training data format, the loss function for evaluating model performance, and the importance of using separate test and validation datasets for model evaluation and selection. Additionally, it emphasizes the need for a linear separator in classification tasks.

Uploaded by

Godspell
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
12 views5 pages

Classification and Model Evaluation Guide

The document discusses classification in machine learning, focusing on predicting whether the number of rooms in a property exceeds three based on area and price. It outlines the training data format, the loss function for evaluating model performance, and the importance of using separate test and validation datasets for model evaluation and selection. Additionally, it emphasizes the need for a linear separator in classification tasks.

Uploaded by

Godspell
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Classification

• E.g. Predict if rooms>3 from area and price.


<latexit sha1_base64="xwKQEvWNytkNjvhzlQVdAtsFjT8=">AAAL/3icbZbdTuQ2FMcz26/dmba7tJe9scoggZaOJlOprVYCLSBY0LJ0+GbBM9RJnMTCsYPtAEOUi970VXpX9bZP0GfoQ/S2va0zDNRjk6sz5/f/22dyTpwEOSVSdbt/NZ588OFHH3/y9Fmz9elnnz9/MfPFkeSFCPFhyCkXJwGSmBKGDxVRFJ/kAqMsoPg4uFir+fEVFpJwdqBGOR5kKGEkJiFSOnU+0/gJBjghrCQKZ+QWV80mrEOw3kk6oC9wREIFSAwE55lc/hbEgmcA6S0AYhHIBQlx595zIBBhhCUgQgq9Am1YzsObob84GvoLi6COezru6RjSiCt5l2OLYDRkC7Bqg/uF2jpNACQMwL1hVPPJr/Klvwi+8bX2XrpCEy6ISjPAC5UXSgIEMh5hCtrxq9oNlx9xbXMpQXsJwFigsPSrklVQFtl5SZb8aqg38sF8XBdHFgBk+LKuYOHB3a7RAtB2SRI2D6+HUPEcwBvwEgRaBjGLHm7o+YvZbqc7voAb+JNg1ptc/fOZZ7/AiIdFhpkKKZLyzO/malAioUhIxx0qJM5ReIESfCZDRHE0KFNMr7CqmlBghq9DnmW6PzBGGaGjCMeooKqEMp6EFZha5MAflDFnCrPQWl6HDGVYDsrxrFVgTmciEHMBagMYZ01HiTIpR1mglRlSqbRZnXyMnRUq/mFQEqZ7WJcx3iguKFAc1IMLIiJwqOhIByjUHSchCFOk26f0eE//7RKmSFUlvCYRriObKkIjPOHjGDiS5aosoSBJqpAQ/LpyBEvaL5WuXWCqVxQEsUQ3p1yqpdOl0FhrFb5RUo0oLh9mrlfZ2kSgSGsZCiiyEBJJhm405DkWSHFRt6WcZB/REvaYVmctbcjZla0c5yxdzunI1o1zlk4gdmHrxjlLVz85tm6cs3VFnnOhHOkkXTXnpvV7dePq6QoCsGcvdmrAUxvuGHDHbXj/Acegb3vXDbhuwyMk/sdl/ctWbBn2LRv6BvRt2DVg170bKwZecVr/QPUZ4uJVk6465jUTr9l4zdjYgRumdcOmwYbhdegb0/vGedg2TbzpFL1lYudOvzXpW6eubaOubZvSbdPs4HcmfefUtWPiHQf3TewMX7BnFObM/K4Bd92x3jeX3nf5qmF3h+DAdB/Y+NikxzY9MemJTd+b9L2z8amJ60d5Duyn+kAICwXWUn0W42kD/7E23GKF3IckQMZ/RM7dDQwaODQ0aGjT2ICxc9gbMHHWTTVNq2ruXmHhzDBnjjk3aO7QS4NeOtQ4q4BwqDSodGhh0MKhVwZ13i83BnReaCMDOm+cWwPe2vDagPVb3O48zdNx9wNOo/qzhVMA75JVU3+5+fZ3mhsc9Tr+d53ebm/29erkG+6p95X3tTfv+d733mtv0+t7h17Y+LPxd+Ofxr+tn1u/tn5r/X4nfdKYeL70pq7WH/8BFr11wQ==</latexit>

• Training data: {(x1 , y 1 ), (x2 , y 2 ), . . . , (xn , y n )}

• xi 2 Rd , y i 2 {+1, 1}

• Algorithm outputs a model f : Rd !{+1, 1}


Pn
• Lossas
= 1
n i=1 1(f (x i
) 6
= y i
) Fraction of trainingdata
classified wrongly by f
• f (x) = sign(w> x + b)

Linear separator
Classification Illustration 1
x y f g k f I
I ti t f
0,0 4 x x
I I I
0,1
X
I ti t l z
1 0
I
4 4 I I I y
2 ti
3,43 l l I g
l l l
4,3
I i J
X X l
K
O I 2 3 4
f X X O
2 Loss
f
e
sign o

g x sign x 2K
Loss g
tch
Classification Illustration 2
Area Price Rooms

9 5.0 -1 Rooms=1 or 2 or 3 is encoded as -1.


7 Rooms > 3 is encoded as +1.
3.1 -1
f h
12 6.9 +1 g
16 9.7 +1
I I I
15 8.5 +1 t

11 7.1 +1

was t
H l
Glx sign price f

h X sign Price 9
Evaluating Learned Models : Test Data

• Learning algorithm uses training data (x1 , y 1 ), . . . , (xn , y n ) to get model f .


<latexit sha1_base64="/iLRyvDz0RMcDti32weNoHGMIaI=">AAALz3icbZbNbtw2EMe16Vey2zZxe+yFqG0gAQxj5UPbS4pkF3FsxHH9bcfWxqAkSiJMkQpJ7XotqOipQJ+gT9Nr+wx9m1KK7M6S1mk0v/8MR+SIZFgwqvRw+G/vwSeffvb5Fw8f9QdffvX14ydL35woUcqIHEeCCXkWYkUY5eRYU83IWSEJzkNGTsOrccNPp0QqKviRnhdkkuOU04RGWBvX5VJvGIQkpbyimuT0htT9ftCYaIdgySlPEWapkFRnOSoVUUhLTFt/jDVGK0+v3/tr8/f+szUUsFhotYaMixsXf7aCtEAp0SgXMWFoJVlZv80+KjUiU8xKU4bJpTOCWDMgiTtxXiqNAk2udZhUXOgahQTFghMkeCtfrINqMwXJXfpjZQTEZGihzrBGVC2mo/elSYTsxr8tTvAmKeHx3QRdPlkerg/bB7mG3xnLXvfsXS49+j2IRVTmhOuIYaUu/GGhJxWWmkasnXEzswWOrnBKLlSEGYknVUbYlOi6H0jCySwSeY55HCQ4p2wekwSXTFeBSjqzRgtJjvxJlQiuCY+s9MbkOCdqUrW9U6NV44nbD28CUOuFERXOlZrnoVHmWGfKZo3zPnZR6uSnSUV5UbZltAMlJWt6omlEFFNJIs3mxsCRaTAaoSjDEkfatOviZ1eBWcG6CmY0Jo1lU01ZTDre2siR/FxXVSBpmmkspZjVjuC5iVfa1C4JMxklxTw1i1M9b6SLpbDEaJteUnrOSBUkpujKr6uN2tamEsdGy3HIsIWwTHN8baAoiMRayGZZqs57j5by+7TGa2kjwae2svVZukKwua1rfZZOYn5l61qfpVM0dSpsfbauLAohtSPt3HV/dVF/0Cxc011hiA7sZOcAnttwF8Bdd8H37nCC9uzYVwC+suEJlv/jqnmzFdsgfNuGPoC+DYcADt3ZeAnwS2fp76jZQ1w8gnTkBI8hHtt4DAZ24CYM3bRpuAliHfoaxr52frYtiLecorchdmb6DaRvnLp2QF07NmU7MNjBbyF969S1C/Gug/cgdpovPACFOT2/D+C+29aHMPWhy0cg3G2CIxh9ZONTSE9tegbpmU3fQfrOGfgc4uZXXkWHmdkQInNRGGdmLyaLAeKXJuCGaOz+JCEG34id2Q0BDR0aARrZNAEwcTZ7AFMnb2ZoVtertwoL5yA4d4ILQAuHfgD0g0PBXoWkQxWgyqEloKVDp4A658s1gM6BNgfQOXFuALyx4QzA5hS3V54VWbv6oWBxc20RDAUfnXXf3Nx8+57mGicb6/4P6xv7G8svRt0d7qH3nfe999TzvR+9F96Wt+cde1Hvz95fvb97/wz2B7PBr4PfPkof9LqYb72FZ/DHf6rUbo0=</latexit>

• But evaluating the learned model must not be done on the training data itself.

• Use test data that is not in the training data for model evaluation.

a fix Ii if f
4 x x 1
if X
x
9
3 1 other w
L

IX

I i J
X X l
X
O I 2 3 4
Model Selection : Validation Data

• Learning algorithms just find the “best” model in the


collection of models given by the human.

• How to find the right collection of models?

• This is called model selection, and it is done by using


another subset of data called validation data that is
distinct from train and test data.

Price W rooms area t W distanc


wa
b

You might also like