Spatial Filtering

CS474/674 - Prof. Bebis

Sections 3.4, 3.5, 3.6, 3.7, 3.8
 Spatial Filtering Methods

Area/Mask processing methods.

output image
 Spatial Filtering (cont’d)
• Spatial filters are defined by:
(1) Neighborhood (i.e., which pixels to process)
(2) Operation (i.e., how to process the pixels)

output image
 Spatial Filtering – Neighborhood

Typically, it has a square shape K x K center

(we call it a “window”).

We prefer an odd size neighborhood

(e.g., 3x3 or 5x5) so we can easily
define its center.
 Spatial filtering - Operation
• Operation depends on the task, for example:
z’5 = 5z1 -3z2+z3-z4-2z5-3z6+z8-z9-9z7
• The result is stored at the corresponding center location
in the output image.

output image
 Linear vs Non-Linear filters
• Depending on the operator used, a filter can be of two
types: linear or non-linear, for example:
z’5 = 5z1 -3z2+z3-z4-2z5-3z6+z8-z9-9z7 linear
z’5 = max(z1,z2,z3,z4,z5,z6,z7,z8,z9) non-linear

output image
 Linear Operators

• Most common linear operators:

– Correlation
– Convolution

• The output is always a linear combination of the inputs!

 Correlation (linear operator)
• The output of correlation is a weighted sum of the input pixels.

The weights are defined by

z5 w1 z1  w2 z2  w3 z3  w4 z4  w5 z5  w6 z6  w7 z7  w8 z8  w9 z9
a K x K mask (has the same
size as the window):

W1 W2 W3
output image

W4 W5 W6
W7 W8 W9
 Correlation (cont’d)
w(s,t)
g(i,j)
Output
Image

f(i,j)
The output image is
generated by moving the
center of the mask at
each location in the input
image.

K /2 K /2
i,j=0,1,…,N-1
g (i, j ) w( s, t )  f (i, j )   
s  K /2 t  K /2
w( s, t ) f (i  s, j  t )
Handling Locations Close to Boundaries
• Pad with zeroes.

• Alternatively, skip the first/last few rows/columns.

0 0 0 ……………………….0
0 0 0 ……………………….0
Correlation (cont’d)

Useful in applications where we

need to measure the similarity
between an image and a pattern
(e.g., template matching).

Simple template matching

does not work in most
practical cases.
 Convolution (linear operator)

• Similar to correlation except that the mask is first flipped

both horizontally and vertically (same as rotating by 180o).

we can flip the

image instead

K /2 K /2
g (i, j ) w( s, t )  f (i, j )   
s  K /2 t  K /2
w( s, t ) f (i  s, j  t )
i,j=0,1,…,N-1
• If w(i, j) is symmetric (i.e., w(i, j)=w(-i,-j)), then
convolution is equivalent to correlation!
 Example

Correlation:

Convolution:
 Filter Categories

• We will focus on two types of filters:

– Smoothing (also called “low-pass”) filters

• Useful for removing details

– Sharpening (also called “high-pass”) filters

• Useful for enhancing details
 Smoothing Filters (low-pass)

• Useful for removing fine details and noise.

– The elements of the mask must be positive.
– Mask elements sum to 1 assuming normalized weights (i.e.,
divide each weight by the sum of weights).

x 1/(sum of weights)
Smoothing filters – Example

input image smoothed image

 Sharpening Filters (high-pass)

• Useful for enhancing fine details but be aware that it

enhances noise too!
– The elements of the mask contain both positive and
negative weights.
– Mask elements sum to 0.
 Sharpening Filters - Example

input image sharpened image

Note: for better visualization, the

original image has been added to the
result of sharpening.
 Common Smoothing Filters

• Averaging (linear)

• Gaussian (linear)

• Median filtering (non-linear)

 Smoothing Filters: Averaging
• The mask weights are all equal to 1
Smoothing Filters: Averaging (cont’d)
• Mask size determines degree of smoothing (i.e., loss of detail).

original 3x3 5x5 7x7

15x15 25x25
Smoothing Filters: Averaging (cont’d)
Example: extract largest, brightest objects

15 x 15 averaging After image thresholding

 Smoothing filters: Gaussian
• The mask weights are sampled values from a 2D Gaussian:
 Smoothing filters: Gaussian (cont’d)
• Mask size depends on σ, - usually, we use the formula:
 Smoothing filters: Gaussian (cont’d)
• σ controls the amount of smoothing (since mask
size depends on σ)

σ=3
σ = 1.4
Smoothing filters: Gaussian (cont’d)

Example
Averaging vs Gaussian Smoothing
Averaging

Gaussian
 Smoothing Filters: Median Filtering
(non-linear)
• Very effective for removing “salt and pepper” noise (i.e.,
random occurrences of black and white pixels).

median
averaging filtering
 Smoothing Filters: Median Filtering (cont’d)
• Idea: replace each pixel by the median in a neighborhood
around the pixel.
• The size of the neighborhood controls the amount of
smoothing.
 Sharpening Filters

• Common sharpening filters

– Unsharp masking
– High Boost filtering
– Gradient (1st derivative)
– Laplacian (2nd derivative)
 Sharpening Filters: Unsharp Masking
• First, obtain a sharp image by subtracting a smoothed
image (i.e., low-passed (LP)) from the original image:

g mask ( x, y )  f ( x, y )  f LP ( x, y )

f ( x, y ) f LP ( x, y ) g mask ( x, y )

- =
Note: contrast enhancement
has been applied for
better visualization
 Sharpening Filters: Unsharp Masking (cont’d)
• Image sharpening emphasizes edges but other details are lost.
• Idea: add gmask back to the original image.

g ( x, y )  f ( x, y )  g mask ( x, y )

f ( x, y ) g mask ( x, y ) g ( x, y )

+ =
 High Boost Filtering

• Idea: add a weighted portion of gmask back to the original

image
– When k=1, this is the same as unsharp masking.
– When k>1, this process is known as high boost filtering.

g ( x, y )  f ( x, y )  kg mask ( x, y ), k 0

f ( x, y ) g mask ( x, y )

+k
 High Boost Filtering - Example

original

Gaussian
smoothed

2D Example: gmask

Unsharp
Masking
(k=1)
Highboost
Filtering
(k>1)
 Sharpening Filters: Derivatives
• The derivative of an image results in a sharpened image.

• Image derivatives can be computed using the gradient:

 Gradient

• The gradient is a vector which has magnitude and direction:

 Gradient (cont’d)
• Gradient magnitude: provides information about edge strength.
• Gradient direction: provides information about edge direction -
perpendicular to the direction of the edge. (useful for tracing
object boundaries).
 How do we compute the Gradient of an image?

• Approximate partial derivatives using finite differences:

Δx

Convention:
x

 232 177 82 7 
y
 241 18 152 140 
 
 156 221 67 3 
 
 100 45 1 103
Example: Image Gradient

sensitive to horizontal edges

sensitive to vertical edges

 Example: visualize partial derivatives
• Image derivatives can be visualized as an image
by mapping their values to the interval [0, 255]

f
x

f
y
 Implement Gradient Using Masks

• We can implement and using masks:

(x+1/2,y)
good approximation
at (x+1/2,y) (x,y+1/2)
*
*
good approximation
at (x,y+1/2)

Problem: derivatives are not

computed at the same location!
Implement Gradient Using Masks (cont’d)
• Consider a different approximation of the gradient:
good approximation
(x+1/2,y+1/2)

Derivatives are
*
computed at the
same location!

• We can implement and using the following masks:

 Implement Gradient Using Masks (cont’d)
• Other approximations

Prewitt

Sobel
 Example: Visualize Gradient Magnitude
• The gradient magnitude can be visualized
as an image by mapping the values to [0, 255]

f Gradient Magnitude

x

f
y
(isotropic, i.e., detects
edges in all directions)
 Second Derivative – 1D case
f ( x)  f ( x  1)  f ( x)

f ( x)  f ( x  1)  f ( x)  f ( x  1)  f ( x  1)  2 f ( x)

max

less clear sign change

or zero-crossing
 Second Derivative – 1D case (cont’d)
“step” edges
• Typicaly, points that lie on an edge
can be detected by:

(1) Local maxima or

minima of the first derivative.

1st derivative
(2) Zero-crossings
of the second derivative (i.e., where
2nd derivative
the second derivative changes sign).
Second Derivative – 1D case (cont’d)

Example:
 Second Derivative – 2D case: Laplacian
The Laplacian is defined as:

(dot product)

Approximate
2nd partial
derivatives:
Second Derivative – 2D case: Laplacian
(cont’d)
Laplacian Mask

input image output image

5 5 5 Edges can
5 -10 -5 -5
be found
5 -10
by detecting
10
the zero-
-10
crossings
Second Derivative – 2D case: Laplacian
(cont’d)
• Other realizations of the Laplacian mask found in practice.
 How to best visualize the results
of the Laplacian?
• Typically, we add/subtract the results of the Laplacian
to/from the original image for better visualization!

g ( x, y )  f ( x, y )  c  2 f ( x, y ) 
f ( x, y ) is the input image, g ( x, y ) is the sharpenend images

c -1 if 2 f ( x, y ) has a negative center value

or
c 1 if 2 f ( x, y ) has a positive center value.
 How to best visualize the results
of the Laplacian? (cont’d)
Example:
input image

f ( x, y )

Laplacian, negative Laplacian, values

values clipped to zero scaled to [0, 255]

g ( x, y )  2 f ( x, y ) 

g ( x, y )  f ( x, y )  c  2 f ( x, y ) 
 Laplacian vs Gradient
Laplacian Sobel

• Laplacian localizes edges better (zero-crossings).

• Higher order derivatives are typically more sensitive to noise.

• Laplacian is less computational expensive (i.e., one mask only).

• Laplacian can provide edge magnitude information (how?)

but no information about edge direction.
Example
Example
Example
 Quiz #2
• When: Wednesday, October 2nd at 1:00pm
• What: Arithmetic, Logical and Geometric Transformations,
Spatial Transformations