Digital Image Processing

Module - 5

Restoration

Dr Suresh D, Professor, RNSIT

 Syllabus
Restoration:
A model of the Image Degradation/Restoration
Process, Noise models, Restoration in the Presence
of Noise Only using Spatial Filtering and
Frequency Domain Filtering, Inverse Filtering,
Minimum Mean Square Error (Wiener) Filtering.

[Text: Chapter 5: Sections 5.1 to 5.4.3, 5.7, 5.8]

L1, L2, L3

Dr Suresh D, Professor, RNSIT

 Introduction
• Restoration technique is to improve an image in
some predefined sense.
• Restoration is an objective process and it
attempts to recover an image that has been
degraded by using a priori knowledge of the
degradation phenomenon.
• Thus, restoration techniques are oriented toward
the modeling the degradation and applying the
inverse process in order to recover the original
image.
Dr Suresh D, Professor, RNSIT
 Image Restoration
• Image restoration vs. image enhancement
▪ Enhancement:
►largely a subjective process.
►Priori knowledge about the degradation is not a
must (sometimes no degradation is involved).
►Procedures are heuristic and take advantage of the
psychophysical aspects of human visual system.
▪ Restoration:
►more an objective process.
►Images are degraded.
►Tries to recover the images by using the knowledge
about the degradation.
Dr Suresh D, Professor, RNSIT
 Why Image Restoration?
► Imagerestoration is to recover the original image
by removing noise and blur from image.

► Image blur is difficult to avoid in many situations

like photography, to remove motion blur caused by
camera shake, radar imaging to remove the effect
of image system response, etc.

► Image noise is unwanted signal which comes in

image from sensor such as thermal or electrical
signal and Environmental condition such as rain,
snow etc.
Dr Suresh D, Professor, RNSIT
Dr Suresh D, Professor, RNSIT
What is Image Restoration?

Dr Suresh D, Professor, RNSIT

 Image Degradation/Restoration Process
► As fig. shows, degradation process is modeled as
degradation function that, together with the
additive noise term, operates on an input image
f(x, y) to produce a degraded image g(x, y).

► Given g(x,y), some knowledge about the

degradation function H, and additive noise term
η(x,y), the objective of restoration is to obtain an
estimate of the original image .

► Estimation of the output image should be as close as

possible to the original input image.
Dr Suresh D, Professor, RNSIT
A Model of Image Degradation / Restoration
Process

► Degradation
 ( x, H
▪ Degradation function y)
▪ Additive noise
Dr Suresh D, Professor, RNSIT
 A Model of Image Degradation/Restoration
Process contd..

If H is a linear, position-invariant process, then

the degraded image is given in the spatial domain by
g ( x , y ) = h ( x , y ) f ( x , y ) +  ( x, y )

► Where h(x, y) is the spatial representation

of the degradation function and symbol *
indicates convolution.
► Convolution in the spatial domain is analogous
to multiplication in the frequency domain.

Dr Suresh D, Professor, RNSIT

 A Model of Image Degradation/Restoration
Process contd..

The model of the degraded image is given in

the frequency domain by

G (u , v) = H (u , v) F (u , v) + N (u , v)

➢ where the terms in capital letters are the Fourier

transforms of the Corresponding terms of
previous spatial domain expression.

Dr Suresh D, Professor, RNSIT

 Noise Models
► The principal sources of noise in digital images arise during
image acquisition and/or transmission.

▪ Image acquisition
e.g., light levels, sensor temperature, etc.
Sensor performance is affected by environmental
conditions during image acquisition, and by the quality of
sensing elements.

▪ Transmission
Interference in the channel used for transmission.
e.g., lightning or other atmospheric disturbance in wireless
network.
Dr Suresh D, Professor, RNSIT
 Spatial and frequency properties of noise

❑ White noise
➢The Fourier spectrum of noise is constant

❑ With the exception of spatially periodic noise, we

assume
➢Noise is independent of spatial coordinates
➢Noise is uncorrelated with respect to the image
itself

Dr Suresh D, Professor, RNSIT

 Important noise probability density functions
Gaussian Noise
• Noise (image) can be classified according the distribution
of the values of pixels (of the noise image) or its
(normalized) histogram
• Gaussian noise is characterized by two parameters, 
(mean) and σ2 (variance), by
1 − ( z −  ) 2 / 2 2
p( z ) = e
2 
• 70% values of z fall in the range [(-σ),(+σ)]
• 95% values of z fall in the range [(-2σ),(+2σ)]

Dr Suresh D, Professor, RNSIT

Gaussian Noise

Dr Suresh D, Professor, RNSIT

• Rayleigh noise

▪ The PDF of Rayleigh noise is given by

2
 ( z − a )e −( z − a ) 2 / b
for z  a
p( z ) =  b
0 for z  a

▪ The mean and variance of this

density are given by
b( 4 −  )
 = a + b / 4 and  = 2

4
▪ a and b can be obtained through
mean and variance
Dr Suresh D, Professor, RNSIT
Erlang (Gamma) noise

▪ The PDF of Erlang (Gamma)

noise is given by
 a b z b −1 −az
 e for z  0
p( z ) =  (b − 1)!
0 for z  0
▪ The mean and variance of this
density are given by
b
 = b / a and  = 2 2

a
▪ a and b can be obtained
through mean and variance
Dr Suresh D, Professor, RNSIT
Exponential noise

▪ The PDF of Exponential

noise is given by
ae− az for z  0
p( z ) = 
0 for z  0
▪ The mean and variance of
this density are given by
1
 = 1 / a and  = 2
2

a
▪ Special case of Erlang PDF
with b=1
Dr Suresh D, Professor, RNSIT
Uniform noise

▪ The PDF of Uniform noise

is given by
 1
 if a  z  b
p( z ) =  b − a

 0 otherwise
▪ The mean and variance of
this density are given by

(b − a ) 2
 = (a + b) / 2 and  =
2

12

Dr Suresh D, Professor, RNSIT

Impulse (Salt-and-Pepper) Noise

The PDF of (bipolar) impulse noise is given by

 Pa for z = a

p( z ) =  Pb for z = b
0
 otherwise

if b  a, gray-level b will appear as a light dot,

while level a will appear like a dark dot.
If either Pa or Pb is zero, the impulse noise is called
unipolar
Dr Suresh D, Professor, RNSIT
Dr Suresh D, Professor, RNSIT
Above PDFs provide useful tools for modeling a broad range of
noise corruption situations as given below.
• Gaussian noise arises in an image due to factors such as
Electronic circuit noise, sensor noise due to poor illumination
and/or high temperature
• The Rayleigh density is helpful in characterizing noise
phenomenon in Range imaging.
• The exponential and gamma densities find application in
laser imaging.
• Impulse noise is found in situations where quick transients,
such as faulty switching take place during imaging.
• Uniform density is useful as the basis for numerous
random number generators that are used in simulations.

Dr Suresh D, Professor, RNSIT

Examples of Noise: Original Image

Dr Suresh D, Professor, RNSIT

Examples of Noise: Noisy Images

Dr Suresh D, Professor, RNSIT

Examples of Noise: Noisy Images

Dr Suresh D, Professor, RNSIT

 Periodic Noise
• Periodic noise in an image arises typically from
electrical or electromechanical interference
during image acquisition.

• It can be observed by visual inspection both in the

spatial domain and frequency domain.

• It is a type of spatially dependent noise

• Periodic noise can be reduced significantly via

frequency domain filtering
Dr Suresh D, Professor, RNSIT
An Example of Periodic Noise

Dr Suresh D, Professor, RNSIT

 Estimation of Noise Parameters
• Periodic noise
▪ Parameters can be estimated by inspection of the
Fourier spectrum of the image.
▪ Periodic noise tends to produce frequency spikes that
can be detected by visual analysis.
• Noise PDFs
▪ From sensor specifications
▪ If imaging sensors are available, capture a set of images
of plain environments
▪ If only noisy images are available, parameters of the PDF
involved can be estimated from small patches of constant
regions of the noisy images.

Dr Suresh D, Professor, RNSIT

 Estimation of Noise Parameters contd..

The shape of the histogram identifies the closest PDF match

Dr Suresh D, Professor, RNSIT

 Estimation of Noise Parameters contd..

Consider a subimage denoted by S , and let ps ( zi ), i = 0, 1, ..., L -1,

denote the probability estimates of the intensities of the pixels in S .
The mean and variance of the pixels in S:
L −1
z =  zi ps ( zi )
i =0
L −1
and  =  ( zi − z )2 ps ( zi )
2

i =0

Dr Suresh D, Professor, RNSIT

 Restoration in the Presence of Noise Only
̶ Spatial Filtering
When the only degradation present in an image is noise,
equations are

N(u,v) can be subtracted from G(u,v) to obtain an estimate

of the original image. Spatial filtering is a choice when only
Additive noise is present.

Dr Suresh D, Professor, RNSIT

 Mean Filters
Noise reduction capabilities of spatial filters are
introduced
Arithmetic mean filter:
Let S xy represent the set of coordinates in a rectangle
subimage window of size m  n, centered at ( x, y ).
This filter computes the average value of the corrupted
image g(x,y) in the area defined by Sx,y. The value of the
Restored image is shown below is at point (x,y) is the
arithmetic mean Computed using the pixels in the region
defined by Sx,y.
ˆf ( x, y ) = 1
 g ( s, t )
mn ( s ,t )S x , y

Dr Suresh D, Professor, RNSIT

Geometric mean filters

An image is restored using this filter is given by the

expression
1
  mn
ˆf ( x, y ) = 
 g ( s, t ) 
( s ,t )S x , y 

Generally, a geometric mean filter achieves smoothing

comparable to the arithmetic mean filter, but it tends to
lose less image detail in the process

Dr Suresh D, Professor, RNSIT

Harmonic mean filter

The Harmonic mean filter operation is given by the

expression
mn
fˆ ( x, y ) =
1

( s ,t )S x , y g ( s, t )

It works well for salt noise, but fails for pepper noise.
It does well also with other types of noise like Gaussian
noise.

Dr Suresh D, Professor, RNSIT

Contraharmonic mean filter

This filter yields a restored image based on the expression

 g ( s , t
( s ,t )S x , y
) Q +1

fˆ ( x, y ) =
 g ( s , t
( s ,t )S x , y
) Q

Q: order of the filter

Positive Q works for pepper noise
Negative Q works for salt noise
Q=0➔arithmetic mean filter
Q=-1➔harmonic mean filter
It is well suited for reducing the effects of salt-and-
pepper noise. Q>0 for pepper noise and Q<0 for salt
noise.
Dr Suresh D, Professor, RNSIT
Spatial Filtering

Dr Suresh D, Professor, RNSIT

Spatial Filtering

Dr Suresh D, Professor, RNSIT

Spatial Filtering

Dr Suresh D, Professor, RNSIT

 Order statistics filters

• Order statistics filters are spatial filters whose

response is based on ordering (ranking) the values of
the pixels contained in the image area encompassed
by the filter.

• The ranking result determines the response of the

filter.

Dr Suresh D, Professor, RNSIT

 Order Statistics filters contd…
Median filter
• In this method value of a pixel is replaced by the median of
the intensity levels in the neighborhood of that pixel.
• Median represents the 50th percentile of a ranked set of
numbers
fˆ ( x, y ) = median{g ( s, t )}
(s,t)S x , y

• It has excellent noise reduction capabilities for certain types

of random noise.
• Less blurring than linear smoothing filters of similar size.
• It is effective in the presence of both bipolar and unipolar
noise.
Dr Suresh D, Professor, RNSIT
Order Statistic Filters contd..

• Max and min filter

▪ Max filter uses the 100th percentile of a ranked set of
numbers Max. filter
fˆ ( x, y ) = max {g ( s, t )}
(s,t)S x , y

►Good for removing pepper noise

▪ Min filter uses the 0th percentile of a ranked set of

numbers
Min. filter
fˆ ( x, y ) = min {g ( s, t )}
(s,t)S x , y

►Good for removing salt noise

Dr Suresh D, Professor, RNSIT
Midpoint filter:

• The midpoint filter computes between the maximum and

Minimum Values in the area encompassed by the filter.

ˆf ( x, y ) = 1  max {g ( s, t )} + min {g ( s, t )}

2 ( s ,t )S xy ( s ,t )S xy 

• This filter combines order statistics and averaging filter.

• It works best for randomly distributed noise, like
Gaussian or uniform noise.

Dr Suresh D, Professor, RNSIT

 Alpha-Trimmed Mean Filter
We delete the d / 2 lowest and the d / 2 highest intensity values of
g ( s, t ) in the neighborhood S xy . Let g r ( s, t ) represent the remaining
mn - d pixels.
• A filter formed by averaging these remaining pixels is called an
Alpha-trimmed mean filter.
1
fˆ ( x, y ) =
mn − d
g
( s ,t )S xy
r ( s, t )

▪ Where d is ranging from 0 to mn-1. when d = 0, it becomes

arithmetic mean filter. When d = mn – 1, this filter becomes a
median filter.
▪ gr(s,t) represent the remaining mn-d pixels
▪ It is useful in situations involving multiple types of noise like
a combination of salt-and-pepper and Gaussian.
Dr Suresh D, Professor, RNSIT
Dr Suresh D, Professor, RNSIT
Dr Suresh D, Professor, RNSIT
Dr Suresh D, Professor, RNSIT
Numerical Example:
Consider the following list of elements:
A={22,43,65,65,87,80,80,92,100}. Find the pixel value to replace
the central pixel in case of median, maximum, minimum, midpoint
and alpha-trimmed(with d=2) filter.

Solution :

Median = 80
Max -100
Min = 22
Midpoint = (100+22)/2 = 61
Alpha trimmed = 1/7(43+ 65+65+80 +80 +87+92 ) = 73
Dr Suresh D, Professor, RNSIT
 Spatial Filtering: Adaptive Filters
Adaptive filters

• The behavior of adaptive filter changes based

on statistical characteristics of the image
inside the filter region defined by the mхn
rectangular window.

• The performance is superior to that of the

mean filters and order statistics filters.

• Filter complexity is increased when the filter

is designed for improved filtering power.
Dr Suresh D, Professor, RNSIT
 Adaptive, Local Noise Reduction Filters

• The simplest statistical measures of random are

its mean and variance.

• The mean gives a measure of average intensity in

the region over which the mean is computed.

• The variance gives a measure of contrast in that

region.

Dr Suresh D, Professor, RNSIT

 Adaptive, Local Noise Reduction Filters contd..
Filter is to operate on local region Sxy
S xy : local region
The response of the filter at the center point (x,y) of S xy
is based on four quantities:
(a) g ( x, y ), the value of the noisy image at (x, y );
(b)  2 , the variance of the noise corrupting f ( x, y )
to form g ( x, y );
(c) mL , the local mean of the pixels in S xy ;
(d)  L2 , the local variance of the pixels in S xy .

Dr Suresh D, Professor, RNSIT

 Adaptive Filters:
Adaptive, Local Noise Reduction Filters contd..
The behavior of the filter:
(a) if  2 is zero, the filter should return simply the value
of g ( x, y ).
(b) if the local variance is high relative to  2 , the filter
should return a value close to g ( x, y );
(c) if the two variances are equal, the filter returns the
arithmetic mean value of the pixels in S xy .

Dr Suresh D, Professor, RNSIT

 Adaptive Filters:
Adaptive, Local Noise Reduction Filters contd..

Dr Suresh D, Professor, RNSIT

Dr Suresh D, Professor, RNSIT
 Adaptive Filters:
Adaptive Median Filters
• Median filter is effective for removing salt-
and-pepper noise
•The density of the impulse noise can not be too large.

The notation:
zmin = minimum intensity value in S xy
zmax = maximum intensity value in S xy
zmed = median intensity value in S xy
z xy = intensity value at coordinates ( x, y )
S max = maximum allowed size of S xy
Dr Suresh D, Professor, RNSIT
 Adaptive Filters:
Adaptive Median Filters contd..
The adaptive median-filtering works in two stages:
Stage A:
A1 = zmed − zmin ; A2 = zmed − zmax
if A1>0 and A2<0, go to stage B
Else increase the window size
if window size  S max , repeat stage A; Else output zmed
Stage B:
B1 = z xy − zmin ; B2 = z xy − zmax
if B1>0 and B2<0, output z xy ; Else output zmed

Dr Suresh D, Professor, RNSIT

 Adaptive Filters:
Adaptive Median Filters contd..
❑ This filter is designed by considering the
following three main purposes:

▪ To remove salt and pepper (impulse) noise.

▪ To provide smoothing of other noise that
may not impulsive, and
▪ To reduce distortion such as excessive
thinning or thickening of object boundaries.

Dr Suresh D, Professor, RNSIT

 Example:
Adaptive Median Filters

Dr Suresh D, Professor, RNSIT

Periodic Noise Reduction by Frequency Domain Filtering
The basic idea
Periodic noise can be analyzed and filtered effectively
using frequency domain techniques.
The basic idea is that the Periodic noise appears as
concentrated bursts of energy in the Fourier
transform, at locations corresponding to the frequencies
of the periodic interference.
Approach
• A selective filter is used to isolate the noise.
• Bandreject, bandpass, and notch filters are used as tools
for periodic noise reduction.
• Bandreject filters remove or attenuate a band of
frequencies about the origin of the Fourier transform.
Dr Suresh D, Professor, RNSIT
 Perspective Plots of Band reject Filters

❑Band reject filtering is for noise removal in applications

where the general locations of noise components in
frequency domain is approximately known.

Dr Suresh D, Professor, RNSIT

❑ Example is an image corrupted by additive
periodic noise that can be approximated as 2 D
sinusoidal functions.
❑ Fourier transform of sine consists of two
impulses that are mirror images of each other
about the origin of the transform.
❑ The sinusoidal noise components appear as
symmetric pairs of bright dots in the Fourier
Transform shown in figure 5.16.

Dr Suresh D, Professor, RNSIT

 Band reject Filters
1) Ideal band reject filter
Band reject filters remove or attenuate a band of frequencies about
the origin of the Fourier Transform
• The transfer function of ideal band reject filter is given by
 W
1 if D(u, v)  D0 − 2
 W W
H (u, v) = 0 if D0 −  D(u, v)  D0 +
 2 2
1 if D(u, v)  D0 + W
 2

• Where D(u,v) is the distance from the origin of the centred

frequency rectangle, W is the width of the band of frequency and
D0 is the radial centre.
Dr Suresh D, Professor, RNSIT
Band reject Filters contd..

• Butterworth bandreject filter

1
H (u, v) = 2n
 D(u, v)W 
1+  2 2
 D (u , v ) − D0 

n =1

Dr Suresh D, Professor, RNSIT

Band reject Filters contd..

• Gaussian bandreject filter

1  D 2 ( u ,v ) − D02 
−  
2  D ( u ,v )W 
H (u, v) =1 − e

Dr Suresh D, Professor, RNSIT

 A Butterworth bandreject filter
of order 4, with the appropriate
radius and width to enclose
completely the noise impulses

Dr Suresh D, Professor, RNSIT

 Band-pass Filters
❑ Band-pass filter performs the opposite of a band-pass filter.
❑ The transfer function of Band-pass filter is obtained from
corresponding transfer function of Band – reject filter.
H bp (u , v ) = 1 − H br (u , v )
❑ Generally It removes too much image details. It is quite
useful in isolating the effects on an image caused by
selected frequency bands.

Dr Suresh D, Professor, RNSIT

 Notch Filters
• Notch filter rejects (or passes) frequencies in
predefined neighborhoods about a center frequency.

• It appears in symmetric pairs about the origin because the

Fourier transform of a real valued image is symmetric.

The transfer function of Notch pass filter is

HNP(u,v) = 1 – HNR(u,v)

• where HNP(u,v) is the transfer function of the notch pass

Filter Corresponding to the notch reject filter with transfer
function HNR(u,v)
• This filter reduces the noise in the image with out
introducing appreciable blurring.
Dr Suresh D, Professor, RNSIT
 Notch Filters contd..

• The transfer functions of ideal, Butterworth and

Gaussian notch (reject) filters are as follows.

Dr Suresh D, Professor, RNSIT

Perspective Plots of Notch Filters

Dr Suresh D, Professor, RNSIT

Dr Suresh D, Professor, RNSIT
 Inverse Filtering
• The simplest approach to restoration is direct inverse filtering.
Estimation of the transformation of the original image is
obtained by dividing the transform of the degraded image,
G(u, v), by the degradation function as given below
ˆ G (u, v)
F (u, v) =
H (u, v)
G(u,v) = H(u,v)F(u,v) + N(u,v)
ˆ N (u, v)
F (u, v) = F (u, v) +
H (u, v)
• If the degradation has zero or very small values, then the ratio N/H
could easily dominate our estimation of F. Noise is enhanced when
H(u, v) is very small.
• One approach to get around the zero or small-value problem is to
limit the filter frequencies to value near the origin.
Dr Suresh D, Professor, RNSIT
 Inverse Filtering contd..

ˆ N (u, v)
F (u, v) = F (u, v) +
H (u, v)

1. We can't exactly recover the undegraded image

because N (u, v) is not known.
2. If the degradation function has zero or very
small values, then the ratio N (u, v) / H (u, v) could
easily dominate the estimate F (u, v).

Dr Suresh D, Professor, RNSIT

 Inverse Filtering

EXAMPLE

The image in Fig. 5.25(b) was inverse filtered using the

exact inverse of the degradation function that generated
that image. That is, the degradation function is
5/6
− k ( u − M /2 ) + ( v − N /2)2 
2

H (u , v) = e  
, k = 0.0025

Dr Suresh D, Professor, RNSIT

 Inverse Filtering
One approach is to limit the filter frequencies to values near the origin.

EXAMPLE

The image in Fig. 5.25(b) was inverse filtered using the

exact inverse of the degradation function that generated
that image. That is, the degradation function is
5/6
− k ( u − M / 2 ) + ( v − N / 2) 2 
2

H (u , v) = e  

k = 0.0025, M = N = 480.

Dr Suresh D, Professor, RNSIT

 The poor performance
of direct inverse
filtering in general

A Butterworth
lowpass
function of
order 10

Dr Suresh D, Professor, RNSIT

Minimum Mean Square Error (Wiener) Filtering
This approach incorporate both the degradation function
and statistical characteristic of noise into the restoration
process.
➢ Objective
Objective is to find an estimate of the uncorrupted
image such that the mean square error between them is
minimized. This error measure is given by
e2 = E[( f − fˆ )2 ]
where E {.} is the expected value of the argument.
It is assumed that the noise and the image are
uncorrelated.
Dr Suresh D, Professor, RNSIT
 Minimum Mean Square Error Filtering (Wiener
Filtering) contd..
The minimum of the error function is given in the frequency
domain by the expression.
 H *
(u , v ) S (u , v ) 
Fˆ (u , v) =  G (u , v)
f

 S f (u , v) H (u , v) + S (u , v) 
2

If the noise is zero,

 H * (u , v)  then the Wiener Filter
= G (u , v) reduces to the inverse
 H (u , v) + S (u , v) / S f (u , v) 
2
filter.

 1 H (u , v)
2

= G (u , v)
 H (u , v) H (u , v) + S (u , v) / S f (u , v) 
2

S (u , v) = N (u , v) = power spectrum of the noise

2

S f (u , v) = F (u , v) = power spectrum of the undegraded image

2

Dr Suresh D, Professor, RNSIT

Minimum Mean Square Error (Wiener) Filtering contd..
• Product of complex quantity with its Conjugate is equal to the
magnitude of the complex quantity squared. This result is known
as the Weiner filter.

G(u, v) is the transform of the degraded image. The restored

image in the Spatial domain is obtained by the inverse Fourier
transform of the frequency domain estimate.
If the noise is zero, noise power spectrum vanishes and the
Wiener filter reduces to inverse filter.
Dr Suresh D, Professor, RNSIT
Minimum Mean Square Error (Wiener) Filtering contd..

Dr Suresh D, Professor, RNSIT

Minimum Mean Square Error (Wiener) Filtering SNR and MSE:
►A number of useful measures are based on the power
spectra of noise and the un-degraded image. These are as
follows.

This is an important metric used in characterizing the performance

of restoration algorithm.
Dr Suresh D, Professor, RNSIT
• Mean Square error (MSE) given in statistical form
can be approximated in terms of summation involving
the original and restored images.

Dr Suresh D, Professor, RNSIT

Minimum Mean Square Error (Wiener) Filtering contd..

 1 H (u , v )
2

Fˆ (u , v) =  G (u , v)
 H (u , v) H (u , v) + S (u , v) / S f (u , v) 
2

Constant Unknown

 1 H ( u , v )
2

Fˆ (u , v) =  G (u , v)
 H (u , v) H (u , v) + K 
2

Dr Suresh D, Professor, RNSIT

Left:
degradated
image

Middle:
inverse
filtering

Right:
Wiener
filtering

Dr Suresh D, Professor, RNSIT

Thank you

Dr Suresh D, Professor, RNSIT