INTRODUCTION
What
is
Image
Processing?
• To
convert
an
image
into
digital
form
and
perform
some
opera:ons
on
it,
in
order
to
get
an
enhanced
image
or
to
extract
some
useful
informa:on
from
it.
What
is
Image
Processing?
• Computer
manipula:on
of
pictures,
or
images,
that
have
been
converted
into
numeric
form.
• Typical
opera:ons
include:
o low-‐level:
inputs
and
outputs
are
images
o mid-‐level:
outputs
are
acributes
extracted
from
input
images
o high-‐level:
an
ensemble
of
recogni:on
of
individual
objects
The
purpose
of
Image
Processing
• Visualiza:on
-‐
Observe
the
objects
that
are
not
visible.
• Image
sharpening
and
restora:on
-‐
To
create
a
becer
image.
• Image
retrieval
-‐
Seek
for
the
image
of
interest.
• Measurement
of
pacern
–
Measures
various
objects
in
an
image.
• Image
Recogni:on
–
Dis:nguish
the
objects
in
an
image.
A
hard
task
Vision
is
an
amazing
feat
of
natural
intelligence.
• Visual
cortex
occupies
about
50%
of
a
Macaque’s
brain
• More
human
brain
devoted
to
vision
than
anything
else
Human
Vision
• Recognising
objects
and
people
• Locate
objects
in
space
• Track
objects
in
mo:on
• Recognise
ac:ons
• But
humans
can
tell
a
lot
about
a
scene
from
a
licle
informa:on…
Source:
“80
million
:ny
images”
by
Torralba,
et
al.
Human
Vision
– We
see
effortlessly
and
immediately.
– We
perceive
the
visual
world
as
external
to
ourselves,
but
it
is
a
reconstruc:on
within
our
brains.
– We
regard
how
we
see
as
reflec:ng
the
world
“as
it
is;”
BUT…
human
vision
is…
• subject
to
illusions
• quan:ta:vely
imprecise
• limited
to
a
narrow
range
of
frequencies
of
radia:on
• passive
Illusion
Examples
Photometry
Overview
Light
source
emits
photon
Sensor
Response
IlluminaJon
Some
reaches
the
eye/camera
and
is
measured.
Photons
travel
in
a
straight
line
They
hit
an
object.
Some
gets
absorbed
while
some
is
reflected
in
a
new
direc:on
Surface
ReflecJon
Fundamental
Challenges
• 3D
to
2D
implies
informa:on
loss
graphics
vision
• Noise
is
introduced
• Ambiguity
–
different
things
may
look
similar
Fundamental
Challenges
• Appearance
changes
with
viewpoints
• Geometric
changes:
surface
slant
depends
on
viewpoint
• Photometric
changes:
surface
brightness
and
colour
depend
on
viewpoint
• Occlusions:
what
is
hidden
depends
on
viewpoint
Fundamental
Challenges
• We
are
trying
to
infer
things
about
the
world
from
an
array
of
numbers
Why
study
Image
Processing?
• Images
and
videos
are
everywhere
• And
they
are
diverse
Why
Study
Image
Processing?
• Fast-‐growing
collec:on
of
useful
applica:ons
Safety
Surveillance
Medical
…
etc.
Entertainment
Environment
ApplicaJons
–
Medical
DiagnosJcs
ApplicaJon
–
Face
DetecJon
and
RecogniJon
• Many
new
digital
cameras
now
can
detect
faces
• Sony’s
Smile
Shucer
Mode
ApplicaJon
–
Image
based
Biometrics
“How the Afghan Girl was
Identified?”
Image
Processing
System
input
device
AcquisiJon
image
Pre-‐
processing
image
Problem
SegmentaJon
SoluJon
image
regions
DescripJon
and
feature
extracJon
RepresentaJon
features
PaWern
learning
models
ClassificaJon
output
Image
Processing
System
input
device
AcquisiJon
image
Pre-‐
processing
Post-‐code
RecogniJon
image
LN6
38PF
Problem
SegmentaJon
SoluJon
image
regions
DescripJon
and
feature
extracJon
RepresentaJon
11
black
pixels
features
PaWern
learning
models
ClassificaJon
three!
output
Image
AcquisiJon
Input:
Light,
Output:
Digital
Image
Courtesy
of
[Link]
Image
AcquisiJon
Image
source
• Light
-‐
sensi:ve
camera,
scanner,
web-‐cam,
etc.
o usually
CCD
or
CMOS
sensor
• Range
sensors,
tomography
devices,
radar,
ultra-‐sonic
cameras,
etc.
• Output
of
most
sensors
is
a
con:nuous
value
• Need
to
convert
it
into
digital
form
Sampling
and
quan:sa:on
• Digi:sing
the
coordinate
values
is
called
sampling
• Digi:sing
the
amplitude
values
is
called
quan:sa:on
Digital
Image
The
image
data
contains
informa:on
of
the
gray
level
at
each
posi:on
in
the
image.
A
digital
image
of
a
rat.
The
magnifica:on
of
the
rat’s
nose.
Digital
Image
A
digital
image
can
be
thought
of
as
a
matrix
(2-‐d
array,
raster
map)
of
intensity
values
94 100 104 119 125 136 143 153 157 158
103 104 106 98 103 119 141 155 159 160
109 136 136 123 95 78 117 149 155 160
110 130 144 149 129 78 97 151 161 158
109 137 178 167 119 78 101 185 188 161
100 143 167 134 87 85 134 216 209 172
104 123 166 161 155 160 205 229 218 181
125 131 172 179 180 208 238 237 228 200
131 148 172 175 188 228 239 238 228 206
161 169 162 163 193 228 230 237 220 199
The
magnifica:on
of
the
rat’s
nose.
Intensity
values
of
the
rat’s
nose.
(common
to
use
one
byte
per
value:
0
=
black,
255
=
white)
Pre-‐processing
Input:
Image,
Output:
Image
Aim:
an
improvement
of
the
image
data
that
suppresses
unwanted
distor:on
or
enhance
some
image
features
important
for
further
"Lena"
with
noise
processing.
Image
enhancement
• adjust
contrast,
remove
noise,
sharpen
details
Format
conversion
median
filtering
• e.g.
from
RGB
to
HSV
colour
space
Filtering
• e.g.
remove
small
objects,
fill
in
holes
edge
detec:on
SegmentaJon
Input:
Image,
Output:
Image
+
Regions
Segment
(poten:ally)
relevant
image
regions
• e.g.
foreground/background
detec:on
• e.g.
skin
colour
filtering
Based
on
1
or
2
basic
proper:es
of
intensity
values:
discon:nuity
&
similarity
• Such
as
thresholding,
region
growing,
region
spliwng
&
merging
ROI
thresholded
RepresentaJon
&
DescripJon
Input:
Image
+
Regions,
Output:
Features
Representa:on:
two
choices
• In
terms
of
its
external
characteris:cs
(boundary)
• In
terms
of
its
internal
characteris:cs
(pixels
comprising
the
region)
Descrip:on:
based
on
the
representa:on
• size,
shape
• colour,
texture
• manual
and
automa:c
extrac:on
Examples
• area
(all
black
pixels)
• height
and
width
16
black
pixels
31
black
pixels
PaWern
ClassificaJon
Input:
Features,
Output:
Class/Category
Requires
learning
of
what
a
typical
class
‘looks’
like
(generalisa:on,
model)
–
based
(usually)
on
the
extracted
features
• object
recogni:on:
which
object
is
that?
Classifier
• training
–
learning
of
the
model
• tes:ng
–
comparing
new
examples
to
the
learned
model
PaWern
ClassificaJon
Example
Task:
is
it
class
a
or
class
b?
Training
• A
data
set,
i.e.
a
large
number
of
example
images
(both
‘a’
and
‘b’)
already
classified
by
a
human
• Extract
features
for
each
example
xn
=
black
pixel
count
x1
=
black
pixel
count
• Construct
a
histogram
for
each
class
–
count
the
number
of
examples
with
a
specific
feature
value
a
frequency
b
feature
value
x
PaWern
ClassificaJon
Example
Task:
is
it
class
a
or
class
b?
Training
(cont.)
• Find
value
T
(discriminant)
that
will
best
separate
two
classes
(e.g.
minimise
the
number
of
misclassifica:ons)
a
frequency
b
feature
value
x
Tes:ng
T
• extract
feature
x
from
a
given/new
example
• if
x
<
T
then
the
new
example
is
a,
else
b
input
device
AcquisiJon
image
Pre-‐
processing
image
SegmentaJon
image
regions
DescripJon
and
RepresentaJon
features
PaWern
ClassificaJon
output