Classification

Machine Learning Techniques

Karthik Thiagarajan
References and Credits

• The content presented in these slides is derived from professor

Arun Rajkumar's
Rajkumar's lectures and slides in the MLT course.
course. This is the "ground
truth" for almost all the content in these slides. As a result, these slides
should be viewed as a presentation of the same content in the professor's
lectures using a different medium. At the same time, these slides are not
meant to be a replacement for the lectures.
• The method of incrementally displaying content on slides is borrowed
from professor Mitesh Khapra.
Khapra.
• These slides were prepared using the tool [Link]
[Link]..
Classification

D = {( x y
1, 1 ), ⋯ ,( xn yn
, )}
Classification

D = {( x y
1, 1 ), ⋯ ,( xn yn
, )}

yi ∈ { 0, 1 } OR yi ∈ { - 1, 1 }
Classification

D = {( x y 1, 1 ), ⋯ ,( xn yn
, )}

yi ∈ { 0, 1 } OR yi ∈ { - 1, 1 }

h Rd
: → { 0, 1 }

hx y ( ) =
Classification

D = {( x y 1, 1 ), ⋯ ,( xn yn
, )}
0-1 Loss

yi ∈ { 0, 1 } OR yi ∈ { - 1, 1 }

h : Rd → { 0, 1 } Loss h
( ) =
1
∑
n i I
⋅ h
n
x
=1
i [ ( ) ≠ yi ]

hx y( ) =
Linear Classifiers
class-1
class-0
x 2

x 1
Linear Classifiers
class-1
class-0
x 2

w
x 1
Linear Classifiers
class-1
class-0
x 2

x wTx| ⩾ 0

w
x 1

x wTx
| < 0 halfplanes
Linear Classifiers hw x
( ) = sign w T x
class-1
class-0 wTT x
wx
1, ⩾ 0
=
0, < 0

x 2

x wTx| ⩾ 0

w
x 1

x wTx
| < 0 halfplanes
Linear Classifiers hw x
( ) = sign w T x
class-1
class-0 wTT x
wx
1, ⩾ 0
=
0, < 0

x
H hw hw x sign w T x
2

x wTx| ⩾ 0
linear
= | ( ) =

w
Linear Classifiers
x 1

x wTx
| < 0 halfplanes
Linear Classifiers hw x( ) = sign w T x
class-1
class-0 wTT x
wx
1, ⩾ 0
=
0, < 0

x
H hw hw x sign w T x
2

x wTx| ⩾ 0
linear
= | ( ) =

w
Linear Classifiers
x
∑
1

n
min
1

n i I
⋅ h x i [ ( ) ≠ yi ]

x | wTx < 0 halfplanes hH

∈
linear =1
Classification as Regression

x 2

w
x 1
Classification as Regression
w *

w
= arg min
1
∑
n i w
⋅
n
T xi - yi
=1
2

x 2
hw x * ( ) = sign w x *
T

w
x 1
Classification as Regression
w *

w
= arg min
1
∑
n i w
⋅
n
T xi - yi
=1
2

x 2
hw x * ( ) = sign w x *
T

w • Sensitive to outliers

x 1
Classification as Regression
w *

w
= arg min
1
∑
n i w
⋅
n
T xi - yi
=1
2

x 2
hw x * ( ) = sign w x *
T

w • Sensitive to outliers
• Natural ordering is absent
x 1
– example: spam, work, family