CSET344

Image and Video Processing

29th Sep. to 03rd Oct. 2025
(Module 3 )

Overall Course Coordinator-

Dr. Shiv Shivhare Prepared By -
[Link]@[Link] Dr. Gaurav Kumar Dashondhi

Note : Any query related to course then first connect with overall course
1
coordinator.
Module 3 Content

1. Image Enhancement in the Frequency Domain

1.1 1-D and 2-D Fourier transform and its inverse.

1.2 Low pass filter, High pass filter, Butterworth and Gaussian filters.

1.3 Homomorphic Filtering.

2. Image Compression Fundamentals: Lossless compression, Lossy compression.

3. Text Recognition.

2
Image Enhancement in the Frequency Domain
1. Frequency domain Fundamentals.

2. 1-D and 2-D Fourier Transform and its inverse.

3. Frequency domain filtering fundamentals.

4. Ideal Low Pass, Butterworth and Gaussian Filters.

5. Ideal High Pass, Butterworth and Gaussian Filters.

6. Homomorphic Filtering.

7. Correspondence between filtering in the spatial domain and frequency domain.

What is the requirement of frequency domain image enhancement -

1. Fourier transform is a global transform. Every pixel in the spatial domain contributes to every point in the frequency
domain.

2. Convolution in the time domain equivalent to the multiplication in the frequency domain. It directly reduces the
computational complexity. 3
Frequency Domain Fundamentals: Fourier series
1. Fourier Series: Any periodic function can be
expressed as a sum of sines and/or cosines of
different frequencies, each multiplied by a
different co-efficient.

Where –
A function f(t) of a continuous variable, t that
is periodic with period T.

This function is the sum of all the four

above functions
Note: Here, Sine and cosines are the basis functions. 4
Frequency Domain Fundamentals: Fourier
Transform
2. Fourier Transform : Functions those are not periodic (But whose area under the curve is finite) can be
expressed as the integrals of sines and/or cosines of different frequencies each multiplied by a weighting
function.

Fourier Transform Formula Inverse Fourier Transform

Formula

f(t) is a continuous function.

It’s a Fourier transform pair which indicate forward and Inverse Fourier transform is possible. .

Note: Here, Sine and cosines are the basis functions. 5

Frequency Domain Fourier Transform Working
Example
Fundamentals

(Signum function) Note: A Functions that is

expressed by either
Fourier series or Fourier
Transform could be
reconstructed or
recovered completely via
an inverse process.

(a) A Box function (b) Fourier Transform (c) Spectrum

6
Frequency Domain Fundamentals: Sampling
Sampling: A continuous functions has to be converted into the sequence of discrete values before they
processed in the computers.

(a) Continuous Function

(b) Train of Impulses used to model sampling.

(c) Sampled function formed as product of a

and b.

7
Frequency Domain Fundamentals: Sampling theorem, Nyquist
rate
Sampling Theorem: A continuous bandlimited signal can be
Example -
recovered completely from the set of its samples if the samples
are acquired at a rate exceeding at least twice the highest m(t) = sin2Πt + sin3Πt + sin4Πt
frequency content of the function.
Fs > 2fm (Oversampled) W = 2Π

Fs = 2fm (Perfect sampling or Nyquist rate) 2Πf = 2Π

A sampling rate exactly equal to twice the highest frequency is f=1

called the Nyquist rate.
Similarly f = 1.5 and 2 for other
Fs < 2fm (Under sampling or aliasing effect) two cases respectively.

Where,
Fs = sampling frequency
fm = highest frequency component present in the signal

Nyquist Frequency = Nyquist rate / 2

8
Frequency Domain Fundamentals: Sampling theorem, Nyquist
rate
FT of band limited function

Oversampling
Fs > 2fm (Oversampled)

Critical sampling Fs = 2fm (Perfect sampling or Nyquist rate)

A sampling rate exactly equal to twice the

highest frequency is called the Nyquist rate.

Under sampling Fs < 2fm (Under sampling or aliasing effect)

Where,
Fs = sampling frequency, fm = highest
frequency component present in the signal9
Frequency Domain Fundamentals: Aliaising
What happened if during sampling of continuous function, someone Solution
is not following the sampling theorem ? Anti Aliasing :
Answer: Aliasing or false identity Aliasing can be reduced by smoothening (Low
Pass filter) input function to attenuate the higher
Aliasing: It’s a phenomena where different signals are indistinguishable
frequencies.
from one another after sampling.
This process has to be done before the function
Fs < 2fm (Under sampling or aliasing effect)
is sampled because aliasing is an sampling issue
that cannot be “undone after the fact” using
computational techniques.

From here, it is very difficult to recover

back the original signal because digitization of
Two different signal is same.

10
Two different function but their digitization is same.
 Interpretation image in the Frequency Domain: Fourier Transform
Fourier Transform: It's an image enhancement tool, which is used to decompose an image into its sine and cosine compone
Low frequencies : It represent smooth transition in the image.
High frequencies : It capture the rapid changes in the image. Like edges and texture.

Center shows low

Frequency component

High Frequency
component

Input Image F.T. of input Image with heatmap

Keep only low frequency components: image is blurred Keep only high frequency components

11
Frequency Domain Filters
Frequency Domain
Filters

Low Pass Filters Homomorphic

High Pass Filters (HPF) Selective Filtering
(LPF) Filtering

Band Pass
Ideal LPF Ideal HPF Filtering

Butterworth Butterworth Band Reject

LPF HPF Filtering

Gaussian LPF Gaussian HPF Notch Filtering

12
Frequency Domain Filters: Ideal Low Pass Filter
(ILPF)
A 2D filter which passes all the frequencies within a circle of radius from the origin and cut off or attenuate all the
frequencies which are outside to this circle.

It is specified by the transfer function H(u,v)

Where - Do = Positive Constant

D(u,v) = distance between a point (u , v) in the frequency domain and the center.

13
Frequency Domain Filters: Ideal Low Pass Filter
(ILPF)
Cut off Frequency

(a) Ideal LPF Transfer function Plot (b) Function displayed as an image (c) Radial Cross
Section

14
Test Pattern image Circle with radi 10,30,60,160,460
Frequency Domain Filters: Ideal Low Pass Filter
(ILPF) a b c

a – Original Image
b – ILPF with cut off Frequency set at
radii value - 10
c – ILPF with cut off Frequency set at
radii value - 30
d – ILPF with cut off Frequency set at
radii value - 60
e – ILPF with cut off Frequency set at
radii value - 160
f – ILPF with cut off Frequency set at
radii value - 460

d e f
15
Frequency Domain Filters: Gaussian Low Pass Filter
(GLPF)
It is specified by the transfer function H(u,v)

(a) Gaussian LPF Transfer function Plot (b) Function displayed as an image (c) Radial Cross Section with various value of D0

16
Frequency Domain Filters: Gaussian Low Pass Filter
(GLPF) a b c

a – Original Image
b – GLPF with cut off Frequency set at
radii value - 10
c – GLPF with cut off Frequency set at
radii value - 30
d – GLPF with cut off Frequency set at
radii value - 60
e – GLPF with cut off Frequency set at
radii value - 160
f – GLPF with cut off Frequency set at
radii value - 460

d e f
17
Frequency Domain Filters: Butterworth Low Pass Filter
(BLPF)
It is specified by the transfer function H(u,v)

(a) Gaussian LPF Transfer function Plot (b) Function displayed as an image (c) Radial Cross Section with orders 1 to 4.

18
Frequency Domain Filters: Butterworth Low Pass Filter
(BLPF) a b c

a – Original Image
b – BLPF with cut off Frequency set at
radii value - 10
c – BLPF with cut off Frequency set at
radii value - 30
d – BLPF with cut off Frequency set at
radii value - 60
e – BLPF with cut off Frequency set at
radii value - 160
f – BLPF with cut off Frequency set at
radii value - 460

d e f
19
Comparative Analysis Between ILPF, GLBF and BLPF

The shape of the Butterworth filter is controlled by the filter order.

If n is large then Butterworth filter approaches to ILPF.

if n is small then Butterworth filter approaches to GLPF.
20
Frequency Domain Filters: Image Sharpening using High Pass
Filter
High Pass Filter

Image Sharpening can be achieved in the frequency domain by passing the high frequency components (i.e
Edges or other sharp transitions) and attenuate the low frequency components.

Where

D0 = Cut off frequency

n = order
21
Frequency Domain Filters: Image Sharpening using High
PF

Transfer Function (a) HPF Transfer function Plot (b) Function displayed as an image (c) Radial Cross Section
22
Frequency Domain Filters: Image Sharpening using High
PF a b c

Filtered with (a) IHPF, (b) GHPF, (c) BHPF with D0 =60

Filtered with (d) IHPF, (e) GHPF, (f) BHPF with D0 =160

d e f
23
Frequency Domain Filters: Homomorphic Filtering
Objective –
Overall objective is to separate illumination and reflectance components to manipulate them independently.

It used to correct uneven illumination and simultaneously enhance the contrast of the image.
Illumination and reflectance model –
An image is modeled as illumination function and reflectance function at every point.

I (x,y) = L(x,y) x R(x,y)

For an images,
illumination is the primary contributor to the dynamic range of an image and it varies slowly. (related to low frequencies)

Reflectance component refers to the details of object edges and it varies rapidly. (related to the high frequencies)

Homomorphic filtering: The idea is to separate these two components and apply separate transfer function on these
two components to get more control over it.

But Fourier transform can not define for the product of two functions, it means we can not apply it.

Solution is : Apply logarithmic function that will separate the multiplicative term into additive. 24
Frequency Domain Filters: Homomorphic Filtering.
Flow diagram of homomorphic
filtering

Apply log to separate out the

Multiply the output of DFT Apply exponential operation
illumination and reflectance
with transfer function to convert back the output
Component.
into the original form.

Note: Convolution in the spatial domain is equivalent to the multiplication in the frequency
domain. 25
Frequency Domain Filters: Homomorphic Filtering
Radial Cross Section of a homographic filter
transfer function

Here
consider Gamma(L) <1:
it attenuate the contribution made by
low frequencies (illumination)

and Gamma(H) =>1: Original Image Image Enhanced using homomorphic filtering
it amplify the contribution made by
high frequencies. (reflectance.)
The net results is simultaneously dynamic range compression
Note: D(u,v) is the distance from the center. and contrast enhancement. 26
Frequency Domain Filters: Working Examples
1-D Discrete Fourier Transform (DFT) and Inverse Discrete Fourier Transform (IDFT)

DFT

IDFT

27
1-D DFT and Inverse DFT working
example

DFT

IDFT

28
1-D DFT and Inverse DFT working
example

29
Frequency domain filtering : Flow Chart

3 4 5

Filter Function
Fourier Transform Inverse Fourier Transform
H(u,v)
F(u,v)

Pre-Processing 2 F(u,v) H(u,v) 6 Post-Processing

F(x,y) G(x,y)
Input Image Enhanced Image
1 7

30
Frequency Domain Filters: Working Examples
Question
Input Image

For the spatial domain image, perform the frequency domain 1 0 1 0

filtering using Ideal High pass filter with cut off frequency 0.5. 1 0 1 0
1 0 1 0
1 0 1 0

Step 1: Multiply the input image by (-1)^x+y to shift the Centre from (0,0) to
(2,2).

0,0 0,1 0,2 0,3 0,0 0,1 0,2 0,3

1,0 1,1 1,2 1,3 Note:
1,0 1,1 1,2 1,3
Input image is 4x4. so the centre is
2,0 2,1 2,2 2,3 2,0 2,1 2,2 2,3 row/2, column/2 i.e 4/2 = 2, 4/2 = 2.
3,0 3,1 3,2 3,3 3,0 3,1 3,2 3,3

31
Frequency Domain Filters: Working Examples
(-1)^0,0 = 1, (-1)^0,1 = -1 (-1)^0,2 = 1 (-1)^0,3 = -1
0,0 0,1 0,2 0,3
(-1)^1,0 = -1, (-1)^1,1 = 1 (-1)^1,2 = -1 (-1)^1,3 = 1
1,0 1,1 1,2 1,3
2,0 2,1 2,2 2,3 (-1)^2,0 = 1, (-1)^2,1 = -1 (-1)^2,2 = 1 (-1)^2,3 = -1
3,0 3,1 3,2 3,3
(-1)^3,0 = -1, (-1)^3,1 = 1 (-1)^3,2 = -1 (-1)^3,3 = 1

Input Image (-1)^(x+y)

1 0 1 0 1 -1 1 -1 1 0 1 0
1 0 1 0 -1 1 -1 1 -1 0 -1 0
1 0 1 0 X 1 -1 1 -1 = 1 0 1 0
1 0 1 0 -1 1 -1 1 -1 0 -1 0

Note: Its pixel-to-pixel multiplication NOT a matrix

32
multiplication.
Frequency Domain Filters: Working Examples
Step 2: Compute the DFT of the image.

F(U,V) DFT = Kernel x f(x,y) x KernelT

Kernel Input Image Kernel

Transpose
1 1 1 1 1 0 1 0 1 1 1 1
1 -j -1 j -1 0 -1 0 1 -j -1 j
X X =
1 -1 1 -1 1 0 1 0 1 -1 1 -1
1 j -1 -j -1 0 -1 0 1 j -1 -j

0 0 0 0
0 0 0 0
F(U,V) DFT =
16 0 16 0
0 0 0 0

Note: Matrix multiplication. Consider only real values. 33

Frequency Domain Filters: Working Examples
Step 2: Compute the distance between each value and the centre.

(𝑥 − 𝑢)2 + (𝑦 − 𝑣)2 u, v = 2,2

0,0 0,1 0,2 0,3 2.82 2.23 2 2.23

1,0 1,1 1,2 1,3 D(u,v) = 2.23 1.14 1 1.41
2,0 2,1 2,2 2,3 2 1 0 1
3,0 3,1 3,2 3,3 2.23 1.41 1 1.41

Filter Function H(u,v)

1 1 1 1
H(u,v) = 1 1 1 1 Note: IHPF – Any value greater than 0.5 will be 1
else is 0.
D0 = 0.5 1 1 0 1
1 1 1 1 34
Frequency Domain Filters: Working Examples
Step 3,4: G(u,v) = F(u,v) x H(u,v)

0 0 0 0 1 1 1 1 0 0 0 0
0 0 0 0 1 1 1 1 0 0 0 0
X =
16 0 16 0 1 1 0 1 16 0 0 0
0 0 0 0 1 1 1 1 0 0 0 0

Note: Its pixel-to-pixel multiplication not a matrix multiplication.

35
Frequency Domain Filters: Working Examples
Step 5: Compute the IDFT of the image.

DFT = 1/4 (Kernel) x f(x,y) x 1/4(KernelT )

1 1 1 1 0 0 0 0 1 1 1 1
1 j -1 -j 0 0 0 0 1 j -1 -j
X X =
1 -1 1 -1 16 0 0 0 1 -1 1 -1
1 -j -1 j 0 0 0 0 1 -j -1 j

16 16 16 16 1 1 1 1
-16 -16 -16 -16 = -1 -1 -1 -1
1/16 16 16 16 16 1 1 1 1
-16 -16 -16 -16 -1 -1 -1 -1

Note: Matrix multiplication. 36

Frequency Domain Filters: Working Examples
Step 6: Multiply the output image by (-1)^x+y to shift the Centre from (2,2) to (0,0)

(-1)^(x+y)

1 1 1 1 1 -1 1 -1
-1 -1 -1 -1 -1 1 -1 1
X
1 1 1 1 1 -1 1 -1
-1 -1 -1 -1 -1 1 -1 1 =

1 -1 1 -1
1 -1 1 -1
Final Output =
1 -1 1 -1
1 -1 1 -1

Note: Its pixel-to-pixel multiplication NOT a matrix 37

multiplication.
Spatial domain and Frequency Domain Filters Analogy

Spatial Domain Frequency Domain

Transfer Function Transfer Function

F(x,y) G(x,y) F(x,y) G(x,y)
H(x,y) H(x,y)

Convolution Multiplication

G(x,y) = F(x,y) convolution H(x,y) G(x,y) = F(x,y) Multiplication H(x,y)

38
Difference between spatial domain and frequency domain.
Feature Spatial Domain Enhancement Frequency Domain Enhancement

Frequency components (Fourier

Domain of Operation Image plane (pixels)
transform)

Direct pixel manipulation,

Primary Operations Filtering in the frequency spectrum
neighborhood filtering

Low-pass filtering (blurring), high-

Histogram equalization, spatial
Common Techniques pass filtering (sharpening),
filtering (smoothing, sharpening)
homomorphic filtering

Localized tasks (e.g., edge Global tasks (e.g., removing

Best For
detection) periodic noise)

Fast for small kernels, slow for large Slower due to transforms for small
Computational Speed
kernels kernels, faster for large kernels

Low (abstract frequency

Intuition High (direct pixel manipulation)
components)

Frequency domain: The process requires two computationally intensive steps: a forward Fourier transform and
an inverse Fourier transform. 39
 CSET344
Image and Video Processing
27th Oct. to 31st Oct. 2025
(Module 3 )

Overall Course Coordinator-

Dr. Shiv Shivhare Prepared By -
[Link]@[Link] Dr. Gaurav Kumar Dashondhi

Note : Any query related to course then first connect with overall course
40
coordinator.
Image Compression

0. Image Compression: Motivation.

1. Image Compression Fundamentals.

2. Lossless Compression.

3. Run Length Encoding.

4. Huffman Coding.

5. Lossy Compression.

6. Discrete Cosine Transform, Quantizer and Zigzag coding and DCT application.

7. Color Image Compression.

41
Image Compression : Motivation

As the amount of data in daily life increases,

there is a constant need to compress it using
efficient techniques.

Image Compression Key Points:

1. Compression and Decompression are virtually invisible to the user.
2. Data compression is a process of reducing the amount of data required to represent given quantity of information.
Here, the data and information both are different.
Data are the means by which information is conveyed. Like Grade 9 or 10 convey information ?
Data may be redundant.
3. Redundant data – If some amount of data contains the same information or repeated information.
42
Image Compression Fundamentals
Lossless Compression - In this type of compression, recovered image is exactly same as it was before applying the
compression technique. (No Loss of Information)

Lossy Compression – In this type of compression, after performing the inverse transformation we cant get exactly
the same image or data as the older one. (Loss of information). Overall the quality of the image get significantly
reduced.
Compression Ratio:
b

Relative Data Redundancy R = 1 – 1/C

C = Compression Ratio
Same information is
C = b / b′
represented by two different
representation If C =10 (C = 10:1 , b = 10 & b’ = 1 )
then it means, larger representation has 10 bits
of data for every 1 bit of data in the smaller representation.
b′ then,
Relative data redundancy is 0.9 (1-1/10 = 9/10)
43
i.e 90% of data is redundant.
Image Compression Fundamentals

Redundancy

Coding Redundancy Spatial and Temporal Irrelevant Information

Redundancy

It refers to, the presence

of unnecessary bits used Spatial Redundancy – Pixel those are
close to each other often have similar Data that can be discarded without
to represent the image significantly affecting the perceive
data. values.
(understanding ) quality of the image
Temporal Redundancy (data).
For example -
There is very less difference
between to successive video frames.

44
Image Compression Models: Compression and Decompression flow diagram
Image

For videos –
Where the discrete
parameter t specify
the time.

Overall Objective – Input image is fed to the encoder which creates a compressed representation of it. Now
this compressed data is fed to the decoder which reconstruct the original data or image.
45
Image Compression Fundamentals
Encoding or Compression
Process

Mapper Quantizer Symbol

Coder

Mapper function is generally

used to reduce the spatial and
The objective of this step is It generate a fixed or variable length
temporal redundancy.
to keep out the irrelevant code to represent quantizer output.
information present in the
In video application, mapper
compressed
function uses previous video
representation.
frames to facilitate the
removal of temporal
redundancy.

NOTE: Operation performed by NOTE: Operation performed by NOTE: Operation performed by

mapper function is reversible. quantizer is not reversible. symbol coder is reversible.

46
Image Compression Fundamentals
Decoding or Decompression
Process
It contains mainly two components - First component is symbol decoder and second component is an inverse
mapper. These two components perform exactly inverse operation that is performed by encoder.

Image Compression formats, Containers and Compression Standards

Compression Container Compression

Formats s Standards
It is a standard way to It is similar to the file It define the process of
organize or store the format but handles compressing and decompressing
data. multiple types of image the images.
or any other data.
These standards are widespread
accepted by image compression
technology.

47
Image Compression Fundamentals

Black Color Formats

Not Sanctioned by International
Blue color Formats Standard Organization (ISO)
Sanctioned by International
Standard Organization (ISO) 48
Loss Less Compression
Huffman Encoding
Loss Less Compression
Run Length Encoding

Huffman Encoding : it's a measure, which helps to reduce the coding

redundancy.
Steps followed in the Huffman encoding.
Calculate the
Perform Source Reduction
probability and arrange Assign the symbols in
until only 2 probabilities
them in the descending terms of binary.
are left.
order.

Calculate different
parameters

Average Length of the Total bits to be Entrop How much space saved
code transmitted y
49
Lossless Compression: Huffman Coding working
example
Problem Statement –

1. Consider an image of size 10 by 10 (5 bit image). Consider some symbols with different
frequencies.
a2 = 40 , a6 = 30 , a1 = 10 , a4 = 10 , a3 = 6 , a5 = 4 (Probability – a2 40/100, a6 30/100, a1 =
10/100…..etc)
Source Reduction

Symbol Probability 1 2 3 4

a2 0.4 0.4 0.4 0.4 0.6

a6 0.3 0.3 0.3 0.3 0.4

a1 0.1 0.1 0.2 0.3

a4 0.1 0.1 0.1
a3 0.06 0.1

a5 0.04

50
Lossless Compression: Huffman Coding working
example

Source Reduction

Symbol Probability 1 2 3 4

a2 0.4 1 0.4 0.4 0.4 0.6 0

a6 0.3 00 0.3 0.3 0.3 00 0.4 1

a1 0.1 0100 0.1 0.2 010 0.3 01

a4 0.1 0100 0.1 0100 0.1 011
a3 0.06 01010 0.1 0101

a5 0.04 01011

51
Lossless Compression: Huffman Coding working
example
0.4 is represented by 1 bit
Parameter Calculation

Average Length of the L = 0.4x1 + 0.3 x 2 + 0.1 x 4 + 0.1 x 4 + 0.06 x 5 + 0.04 x 5 = 2.3 bits/symbol
code

Total bits to be 10 x 10 x 2.3 = 230bits

transmitted
It is showing, 0.06 is represented in
how many number of bits
Entropy = -p x log2 p
-0.4 x log(base 2) x 0.4 +
-0.3 x log(base 2) x 0.3 +
.
.
-0.04 x log(base 2) x 0.04 = 2.1396

How much space you (10 x 10 x 5 – 10 x 10 x 2.3 )/ 10 x 10 x 5 = 0.54 = 54%

saved =

52
Run Length Encoding
Run Length Encoding :

Repeating intensities along the row and columns often be compressed by representing runs of identical intensities
where each run length pairs specify the start of the new intensity and number of consecutive pixels that have that
intensity.
Example
111110000000011111111100111111000001111111 Total Bits in the original
11 sequence = 42

(0,2times) (0,5times)
(1,5times)
(1,6times) (1,9times)
(0,8times)

(1,9times)

53
 Run Length Encoding
Binary Representation Binary Representation

8 4 2 1
(1,5times) 10101

(0,8times) 01000

(1,9times) 11001

(0,2times) 00010

(1,6times) 10110

(0,5times) 00101

(1,9times) 11001
10101 01000 11001 00010 10110 00101 Total Bits - 35
11001
Total Bit reduced = Total number of bits in the original sequence - total number of bits in the compressed sequence = 42 -35
54
Image Compression Fundamentals
Apply Run Length Encoding to below example Binary Representation

0000000011111111 8 4 2 1

011

55
Image Compression Fundamentals
Apply Run Length Encoding to below example Binary Representation

0000000011111111 8 4 2 1
Total bits = 16

(0,8times)
(1,8times) 01000 11000 = Total bits = 10
Total compression = Original Representation - Compressed representation = 16-10 = 6

011
00001 10010 = Total bits = 10
Total compression = Original Representation - Compressed representation = 3-10 = -7

56
Lossy Compression: Discrete Cosine Transform:
DCT
Lossy Compression – In this type of compression, after performing the inverse transformation we cant get exactly
the same image as the older one (i.e. Loss of information). Overall the quality of the image get significantly reduced.

Discrete Cosine Frequency Domain

Spatial
Transform
Domain

1. Discrete Cosine Transform (DCT) is a lossy image compression techniques.

2. DCT represents an image as a sum of sinusoids of varying magnitudes and frequencies.

3. In DCT, most of the significant information or signal (image) energy, is concentrated in the few number of
coefficients (near to the origin) and rest other frequency having very small information which can be stored by using
less number of bits.
4. DCT coefficient are real valued while DFT coefficients are complex, therefore, hardware implementation of DCT
is easier than DFT.
57
Lossy Compression: DCT : Flow Chart

4
3
5
Each block is
DCT is applied on each
compressed through Inverse Discrete cosine
block from left to right top
quantization followed Transform
to bottom.
by coding)

Post-Processing
2 6
Image is broken
into 8 x 8 blocks
of window.

Decompressed
7 Image
Input Image 1

58
Lossy Compression: Discrete Cosine Transform:
DCT
DCT Equation

Equation (1)

Here p(x,y) = x th and yth element of the image represented by matrix p.

N is the size of the DCT block on which DCT is calculated.

Consider 8 by 8 block i.e N = 8, then above DCT equation becomes

59
Lossy Compression: Discrete Cosine Transform:
DCT
DCT Matrix

Here T (i,j) is the Transformation matrix which used to convert the DCT equation (1) into matrix form.
(i,j) are the pixel location.

Transformation matrix (T) for N =8 i.e for 8 x 8 block. 1/sqrt(8) = 0.3536 when i = 0

60
Lossy Compression: Discrete Cosine Transform:
DCT
Working example, Consider a 8 by 8
block of the original image. A particular
block is chosen from the very upper left- DCT is designed to work on pixel values ranging from -128 to 127,
hand corner of an image. The original image is level off by subtracting 128 from each entry.
For example 154-128 = 26, 123-128 = -5 and so on.

Discrete cosine transform (DCT) can be accomplished by matrix multiplication. DCT matrix (D) can be generated by
Matrix Multiplication between M and Transformation matrix (T). 61
Lossy Compression: Discrete Cosine Transform:
DCT

This blocks matrix now consists of 64 DCT coefficients.

Here, coefficient are ranging from 0 to 7.

The top left coefficient, correlates to the low frequencies of

the original image block and as you move away from 0,0 to
all the directions, the DCT coefficients correlate to higher
and higher frequencies of the image block where 7,7
corresponds to the highest frequency.

Quantization: Now the 8 by 8 block of DC coefficient is ready for compression by quantization.

Quantization is the process where varying levels of image compression and quality are obtainable through
selection of specific quantization matrices.

Quality levels of quantization matrix is ranging from 1 to 100 (i.e. Q1 to Q100).

Where Q1 means, quantization matrix has poorest image quality and highest compression.
Q100 means, quantization matrix has Best image quality and lowest compression.
62
Lossy Compression: Discrete Cosine Transform:
DCT
Quantization Matrix with different quality levels. These matrix are standard matrix with different quality levels.

Observations
Quantization is achieved by dividing each
element in the transformed image matrix • Coefficient situated near to
D by the corresponding element in the upper left corner corresponds
quantization matrix and then rounding off to lower
to nearest integer value. frequencies to which human
eye
Consider here Q50. is more sensitive.
first element: 162.3/16 approx. 10
Here, 162.3 is the first element of matrix • Here 0 means, high frequencies
D and 16 is the first element of Q50 that has been discarded, giving
matrix. rise to the Lossy 63 part of
Lossy Compression: Discrete Cosine Transform:
DCT
Zig Zag Coding: final step of compression is coding, where all the coefficient of C are converted by an encoder to a
stream of binary data.

[10 4 3 -7 9 2 5 1 -5 -3 -2 -5 1 2 1 0 1 -2 0 1 0 00 -1 -1 0 0 EOC]

Convert to
64
binary
Lossy Compression: Discrete Cosine Transform:
DCT
Decompression
Reconstruction of output image begins by decoding the bit stream representing the quantization matrix (C). Each
element of C
Is then multiplied by the corresponding element of the quantization matrix that was originally used. (in the quantization
process) Once (R) is generated then IDCT applied on the
matrix R, which is rounded off to the nearest integer.
Finally, 128 added to each elements of that results
to recenter it.

Here, N is the output of IDCT + 128 (Center correction)

Observation:

Compare each element of the original

image and decompressed image, there
is a loss of information.

65
Lossy Compression: Discrete Cosine Transform:
DCT

Original: Pepper image DCT of Pepper Quantized DCT of Pepper image

image

Observation:
Quantization matrix 50
is far better as compared
to 20 and 10.

66
DCT _Q50 image DCT _Q20 image DCT _Q10 image
Lossy Compression : DCT

Issues with the DCT

1 The DCT algorithm can cause block – based
artifacts when heavy compression is applied.
2 Truncation of higher spectral coefficients
results in, output image is blurred.

DCT has vast application such as –

Image Compression – JPEG, HEIF, WebP, BPG etc.

Video Compression – H.261, MJPEG, MPEG1, H.262 (MPEG2), H.265(HEVC), WebM etc.
Audio Compression – AC3(Dolby), AC4
Image watermarking.
67
Application of DCT: Digital Image Watermarking
Watermarking: It’s a digital marker that is embedded inside the image.

(b) watermark.

(c) Visible watermarked (d) Difference between a and c.

(a) Original Image. Embedded inside the image.

Compression: It is used to reduce the amount data used to represent the image.

Watermarking: Goal is to add information or data inside the images.

68
Application of DCT: Digital Image Watermarking: Why and
classification Copyright
Watermarking classification
identification

Visible Invisible
Why
watermarkin
g required ?

Fragile Robust
Authenticity Automated
determination monitoring
It is destroyed by It is designed to survive
Presence of watermarked Watermarked can be monitored performing any of the any kind of image
guarantee that an image by the systems that track when and modification in the modifications-
has not been altered. Where images are used or re-used. images - Like –
Like image Linear filtering.
Monitoring is useful for royalty collection compression and Non linear filtering.
And/or the location of illegal users. decompression. Cropping.
Rotation.
Resampling &69etc.
Application of DCT: Digital Image Watermarking: Embedding
workflow
How to add watermark
Prepare a watermark In these mid frequency
component, add watermark.
Suppose the DCT coefficient is 17.4:
It may be anything, let's •To embedd bit 0: make it even
say, someone wants to ([Link] 1 ) → 16.
hide name of college i.e. Select one block and find mid •To embedd bit 1: make it odd → 17.
BU inside the image. frequency DCT coefficient in
it
Here, BU is the
Mid-frequency coefficients
watermark. Apply Inverse DCT to each
are less perceptible or
Divide the host or the block and reconstruct the
visible to the human visual
original image into 8 by 8 original image.
system.
blocks
Apply 2D DCT 1. Stitch all the blocks together to
These 8 by 8 image blocks build the final watermarked image.
should be non
For each block, apply the 2D 2. The embedded watermark will be
overlapping.
DCT to convert the pixel values imperceptible or invisible to the
to frequency domain human eye but can be reliably
coefficients. extracted. 70
Application of DCT: Digital Image Watermarking: Extraction
workflow
Divide the watermarked Apply 2D DCT on each Reterive watermarks
image block. bits

Read the chosen mid-frequency coefficient.

[Link] Watermarked Image into 8×8 image During extraction, check whether the
blocks. coefficient is even or odd:
If even,
2. Partition exactly as done during then the bit is 0.
embedding. If odd,
then the bit is 1.

71
Color Image Compression : Flow Chart :
Compression

Color Space Chroma

Divide the image
Conversion Subsampling

RGB to Reduce Chrominance of Divide the image into

YCbCr image small blocks

Entropy Coding Zigzag Scanning Quantization DCT

Huffman Coding Arrange quantized

coefficient in the Zig Reducing
zag fashion. Precision

Decompressio 72
Lossy Compression: Discrete Cosine Transform:
DCT
Difference between DFT and DCT.

Feature Discrete Fourier Transform (DFT) Discrete Cosine Transform (DCT)

Converts a finite sequence of samples into complex

Definition Expresses finite sequence in sum of cosine functions only
sinusoidal components (cosine + sine)

Nature of Output Complex values (real + imaginary parts) Usually real numbers

Basis Functions Complex exponentials (both cosines and sines) Cosine functions only

Computational Complexity Can be computed efficiently with FFT algorithms Similar complexity to FFT; specific variants exist

Compression (image/video/audio), numerical solutions in

Typical Applications Signal analysis, filtering, convolution, spectral analysis
engineering

73
 CSET344
Image and Video Processing
03rd Oct. to 07th Nov. 2025
(Module 3 )

Overall Course Coordinator-

Dr. Shiv Shivhare Prepared By -
[Link]@[Link] Dr. Gaurav Kumar Dashondhi

Note : Any query related to course then first connect with overall course
74
coordinator.
Optical Character Recognition (OCR): Definition & Motivation
OCR: It converts different type of documents including scanned papers, PDfs and digital images into editable searchable
digital text.

Scanned Papers

Editable
PDFs OCR searchable digital
text.

Digital Images Document Scanning

Key application of OCR:

Document digitization: Converting historical documents into searchable digital archives.

Automated data entry: Automatically extracting the key information from structured documents like invoices,
receipts etc.

Accessibility: Converting printed text documents to digital format that can be used with text to speech software75for
Optical Character Recognition: Approaches

OCR Approaches

Traditional
AI Driven Approaches
Approaches

Traditional OCR relies on: Modern AI-driven OCR relies on:

Template matching for character recognition. Deep learning algorithms: Continuous improvement
through training.
Pre-defined rules and patterns.
Computer vision: Advanced image understanding.
Basic image processing techniques.
Natural Language Processing: Context-aware text
Limited accuracy with complex layouts. interpretation.

Adaptive learning: Improves accuracy over time.

76
OCR: Text Recognition : Flow Chart

Image Acquisition

Scanners, Digital cameras Text Recognition

Post-
Feature
Preprocessing Pattern Matching processing
extraction

Noise Reduction, Spell Check.

Binarization, Contrast Compares extracted Analyze geometrical
Enhancement, deskew etc. character images with features like edge and Contextual Analysis.
the stored templates. line etc.
Formatting.
Segmentation

Techniques Post -
Isolate the characters processing
that need to be
recognized. 77
Preprocessin
OCR: Preprocessing

Gray Scale Conversion Croping and ROI

Extraction Inversion (If needed)

Noise Reduction normalization Text detection, layout

and segmentation

Skew Corrections binarization Thinning

Note: Based on the requirements, preprocessing step sequence may change.

OCR: Preprocessing
1. Grayscale Conversion

•Goal: Change color images to grayscale images.

•Techniques: Simple averaging or Luminance method.

•Why: Removes unnecessary color information, reduces complexity, and enhances further binarization or thresholding.​

Color Image
Binarization or BW image

79
OCR: Preprocessing
2. Noise Reduction

•Goal: Uneven illumination, Remove specks, smudges, or background noise that can look like real text to an OCR engine.

•Techniques: Median filtering, morphological operations (like erosion/dilation), and smoothing.

Uneven illumination can be removed by contrast limited adaptive histogram equalization.

•Why: Cleaner images help the OCR systems to avoid false positives and reduce errors in character recognition.​

Image with Noise

Image without Noise

80
OCR: Preprocessing
Steps in contrast limited adaptive histogram equalization (CLAHE).

Image Tiling or The input image is divided into a grid of small, non-overlapping, and equally sized tiles.
Partitioning The contrast enhancement is performed independently for each of these tiles.

Histogram
computation and For each tile, the pixel intensity histogram is computed.
clipping or contrast Contrast Limiting: A clip limit is applied to each histogram bin. This limit, usually determined
limiting by the distribution of the histogram and a user-defined parameter, it prevents any single gray-
level bin from having too high a count.
Any pixel count in a histogram bin that exceeds the clip limit is clipped (or truncated) to that
limit.

The total number of clipped pixels (the excess) is then redistributed evenly among all the
histogram bins. This ensures that the overall brightness of the tile is maintained. This step is
what distinguishes CLAHE from standard AHE and prevents noise over-amplification.
Local histogram
equalization
Image Reconstruction via Bilinear Interpolation
A local histogram equalization has
been performed, after the histogram
81
has been clipped and redistributed.
 OCR: Preprocessing
Steps in contrast limited adaptive histogram equalization (CLAHE).

82
Optical Character Recognition: Flow diagram detail
3. Skew Correction

•Goal: Detects and corrects any tilt or rotation in the scanned image.

•Techniques: Hough Transform etc.

•Why: Even slight tilts cause major recognition problems, so aligning lines horizontally is crucial for maximal accuracy.

Skewed Image Skew corrected Image

83
OCR: Preprocessing
Skew Correction Techniques

Skew Detection Skew Angle calculation Skew Correction

Hough Transform: Detects lines Skew angle calculation is the process of • The image is rotated by the detected
in edge maps; dominant line estimating the rotation angle of the text angle in the opposite direction.
orientation reflects about skew. baseline in a document image that
causes the document's skew. • Interpolation methods (e.g., bilinear)
are used to prevent image artifacts.

4. Cropping and Region of Interest (ROI) Extraction

•Goal: Focus on relevant parts of the image, removing borders, graphics, or unrelated regions.

•Techniques: Manual/automatic cropping, contour/box detection around text.

•Why: Reduces distractions and keeps computation efficient and focused on the text content.​

84
OCR: Preprocessing

Original Image Cropped Image or ROI

[Link]

•Goal: Standardize image intensity values or scale to a uniform size and resolution.

•Techniques: Pixel normalization (e.g., min-max scaling), resizing.

•Why: Brings images into a consistent format, so OCR algorithms have predictable inputs.​
85
OCR: Preprocessing
[Link] (Thresholding)

•Goal: Convert images to only black and white pixels.

•Techniques: Fixed threshold, Otsu’s method or adaptive thresholding methods.

•Why: Strong contrast between text and background optimizes the input for most OCR algorithms, which are designed to work
on binary images.​

[Link] (If needed)

•Goal: Convert black pixel into white and white into black. It means text and background colors are swapped.

•Techniques: Thresholding followed by inversion. Color to grey then thresholding convert it into binary then inversion.

Why: OCR engines are generally optimized to recognize text that appears as dark characters on a light background.
When the input image has light-colored text on a dark background (such as white text on a black sign), inverting the
image puts it in the preferred format (dark text on light background).

86
OCR: Preprocessing
8. Text Detection, Layout, and Segmentation

•Goal: Detect lines, words, or blocks of text and segment them from one another.

•Techniques: Connected component analysis(utilize 4 connectivity or 8 connectivity to identify connected components ),

projection profiles, or deep learning-based text detectors.

•Why: Helps the OCR system understand document structure and process multi-line/column/page layouts. This is a crucial
step in complex document processing.

87
OCR: Preprocessing
9. Thinning (Skeletonization)

•Goal: Reduce character width to a single-pixel skeleton without losing shape.

•Why: Enhances the outline of handwritten or joined text, aiding recognition, especially in noisy or stylized scripts

Techniques: Morphological Based techniques like erosion etc.

88
OCR: Text Recognition Techniques
Text Recognition
Compares extracted Analyze geometrical
character images (with Feature features like edge and
Pattern Matching
stored templates. extraction line etc.

Technique Methodology Best Use Cases Limitations

Direct glyph comparison with Poor with font/style/size
Pattern Matching Typed, uniform, printed fonts.
templates. variation, handwriting.
Analyzes structural features, uses ML Handwriting, varied or Requires feature engineering
Feature Extraction
to classify. complex fonts. and more computation.

Note: glyph is the specific visual shape of a character within a particular font. 89
OCR: Flow diagram of Template matching
Image acquisition, Pre-processing, • Obtain scanned or captured image of the document.
Segmentation • Convert image to grayscale or binary.
• Detect the ROI.

Normalization • Resize each character segment to match template size.

• Perform normalization such as scaling, alignment.
Template
Database • Maintain a database of standardized character templates (A-Z, 0-9, etc.).
Preparation

• For each segmented character, compute similarity (e.g., correlation or distance metric) with each
Template Matching template.
• Use matching functions such as Euclidean distance, Correlation etc.
Best match • Identify template with highest similarity or least error to input segment.
selection • Assign label of the best matched template as recognized character.

Post- Processing • Aggregate recognized characters to form words/lines.

• Optionally, apply correction or validation with dictionary or context-aware modules.

90
OCR: Flow diagram of Feature extraction
Input pre processed character image

Edge detection Edges represent boundaries of characters and are extracted using operators like Sobel or Canny.

Corner and
Characteristic points like corners, junctions, and endpoints of strokes are detected here.
Junction
Detection

Contour and Shape Contours of the character are extracted to track boundaries and curves which can describe the
analysis overall geometry.
Statistical measures like zoning (dividing the character into zones and calculating pixel densities),
Statistical feature moments and projection profiles (horizontal/vertical pixel distributions) are computed. These
computation capture global and local shape properties complementing geometric features.

Transformation Transformations such as Fourier descriptors, Wavelets, or Discrete Cosine Transform (DCT) convert
Based Feature spatial pixel data into frequency or scale domains. Such features are powerful for capturing texture,
periodicity, and invariant aspects of characters.

Feature vector All extracted features are compiled into a fixed-length feature vector representing the
formation character uniquely. This vector feeds directly into classifiers (e.g., SVM, neural networks)
91
that perform the final character recognition.
OCR: Post Processing
The first step of post-processing is to identify which words are likely wrong.
•Method:
•Typically uses a dictionary lookup (spell-checking) or analyzes the confidence scores provided by the OCR engine for
each character/word.
•Words not found in the dictionary or those with low confidence are flagged as potential errors.
Second step: From the list of candidates, the best correction is chosen based on the context of the sentence or
document.
•Method: A language model or n-gram analysis is used to determine which candidate word is most probable given
the words surrounding it.

• For example, if the OCR output is "The house was on fre",

• The language model will select "fire" over other candidates like "free" or "frog" to complete the sentence "The house
was on fire."

Third Step: This step applies domain-specific rules to check and correct structured information that a dictionary
can't validate.
•Method: Uses regular expressions (RegEx) or other rules to ensure data fields
like dates,
phone numbers,
currency amounts, and addresses adhere to the correct format (e.g., correcting a date "10-99-2025" to "10-09-
92
2025").