Image Processing

1. Introduction

By MT
Road Map

• Introduction

• Applications

• Fundamentals

• System Components

• Image Sensing and Acquisition

• Sampling and Quantization

• Neighbors of Pixels: Connectivity, Regions, and Boundaries

• Distance Measure

By MT
Introduction

By MT
Introduction

Image

• An image is a digital representation of a visual scene. It is made up of tiny dots called pixels, each of
which has a color and brightness value. When viewed together, these pixels form a picture that we
can see and interpret. The colors of each pixel can either be:

• Grayscale: A single value representing the color. 0 (black) to 255 (white)

• RGB: 3 values representing red, green, and blue colors. Each value is between 0 and 255. A
combination of these 3 can make all other colors.

• RGBA: In addition to RGB, an alpha channel is also present for the image’s transparency.

By MT
Introduction

Image

By MT
Introduction

Image Processing

• Image processing is the technique of manipulating and analyzing digital images to improve their
quality, extract useful information, or make them suitable for specific applications.

• This involves various operations, such as enhancing contrast, removing noise, detecting edges, and
transforming images into different formats or representations.

By MT
Introduction

Image Processing

By MT
Introduction

Purpose

• The purpose of image processing is to obtain a better or more meaningful interpretation of the visual
content for tasks such as computer vision, medical imaging, and photography.

The future

• Image processing can be considered the first step in the process of analyzing and understanding the
image; leading towards Computer Vision AI, which emulates human intelligence.

• However, image processing is not the only thing that’s involved in Computer Vision.

By MT
Introduction

The future

By MT
Applications

By MT
Applications

Image processing is used across various fields, including:

• Medical imaging

• X-Ray imaging

• MRI imaging

• Ultrasound imaging

• CT Scans

• Microscopy

By MT
Applications

Image processing is used across various fields, including:

• Astronomical imaging

• Image processing is used extensively in astronomy for applications such as noise reduction,
image enhancement, image stacking, mosaicking, time-lapses, and motion analysis.

• For example in July 2015, NASA’s space probe New Horizons flew by the dwarf planet Pluto and
captured some stunning pictures of it and its moon Charon.

By MT
Applications

Image processing is used across various fields, including:

• Astronomical imaging

Pluto and Charon Pluto

By MT
Applications

Image processing is used across various fields, including:

• Astronomical imaging

• However, in reality when New Horizons flew by Pluto, it actually captured high-resolution black
and white images using its Long Range Reconnaissance Imager (LORRI) camera. It also captured
the colored images of the dwarf planet Multispectral Visible Imaging Camera (MVIC).

• The color images we often see of Pluto are typically created by combining data from LORRI and
MVIC. LORRI provides the detailed black-and-white imagery, while MVIC captures color
information using different color filters.

By MT
Applications

Image processing is used across various fields, including:

• Astronomical imaging

• Using image processing, scientists then

combine these datasets to produce the
colored images of Pluto.

• This process involves overlaying the

high-resolution black-and-white images
with the color data to create a detailed,
colorized view of the planet's surface.

Pluto in black and white

By MT
Applications

Image processing is used across various fields, including:

• Geographical imaging or remote sensing

• Land use

• Detecting changes

• Vegetation analysis

• Disaster management

• Climate monitoring

• And many more…

By MT
Applications

Image processing is used across various fields, including:

• AR and VR

By MT
Applications

Image processing is used across various fields, including:

• Robotics:

• Robots can perceive their environment and make decisions based on visual inputs.

By MT
Applications

Image processing is used across various fields, including:

• Real-time Detection Systems:

• Face recognition systems

• Security systems for monitoring and analyzing video feeds for security purposes, such as
detecting intrusions or identifying suspects.

By MT
Applications

Image processing is used across various fields, including:

• Real-time Detection Systems:

• Text recognition

By MT
Applications

Image processing is used across various fields, including:

• Real-time Detection Systems:

• Object recognition

By MT
Fundamentals

By MT
Fundamentals

By MT
Fundamentals

Image Acquisition

• Capturing an image using an imaging device such as a camera.

• Converting the captured image into a digital form if it is not already digital.

By MT
Fundamentals

Image Enhancement

• Improving the visual appearance or interpretability of an image.

• The goal is to make certain features more prominent and to make the image more suitable for a
specific application or analysis.

• Involves techniques like noise reduction, contrast enhancement, sharpening, smoothing, gamma
correction and such.

By MT
Fundamentals

Image Restoration

• Aimed at recovering an original image that has

been degraded by various factors.

• The goal of image restoration is to reconstruct or

recover an image as accurately as possible by
reversing the effects of degradation, such as
blurring, noise, or distortion.

• This is typically achieved using mathematical

models and algorithms.

By MT
Fundamentals

Color Image Processing

• Color image processing is the manipulation of images that contain color information, typically
represented in three channels corresponding to the primary colors (Red, Green, and Blue - RGB).

• It involves various techniques and operations to enhance, analyze, and manipulate color images.

By MT
Fundamentals

Color Image Processing

• Key Aspects of Color Image Processing:

• Color spaces

• RGB

• HSV

By MT
Fundamentals

Color Image Processing

• Key Aspects of Color Image Processing:

• Color Conversion

• Changing and image’s color space from one to another

• For example: RGB to HSV or HSV to RGB conversion

• Color Enhancement

• Color Filtering

• Color Segmentation

• Color Correction

• Image Smoothing

By MT
Fundamentals

Wavelets and Multi-resolution Processing

• Wavelets:

• Wavelets are mathematical functions

used to divide data into different
frequency components and then analyze
each component with a resolution matched
to its scale.

• In simpler terms, wavelets focus on smaller

part of an image at a time. This enables
analyzing images easier such as edges.

• It also zooms in/out on various parts of

images to find details.

By MT
Fundamentals

Wavelets and Multi-resolution Processing

• Multi-resolution Processing:

• Multiresolution Processing involves

analyzing and representing images
at multiple levels of resolution or scales.
This approach is beneficial for various
tasks, such as image compression,
denoising, and feature extraction.

By MT
Fundamentals

Wavelets and Multi-resolution Processing

• Multi-resolution Processing:

• A key concept of multi-resolution processing is that it produces a pyramid of images starting at

the top with low resolution to high resolution the further down
it goes.

By MT
Fundamentals

Compression

• Aims at reducing the size of images for storage or transmission purposes such as, storing the image
on the disk or show it on a web page.

• Image formats like JPG can be used to reduce image sizes and save bandwidth on the Internet which
has significant amount of digital images.

• For example, image results on Google Images are highly compressed version of the actual images
since the search result contains a lot of images. Showing all these images in their original size would
slow down the page.

By MT
Fundamentals

Morphological Processing

• It is a technique that focuses on the shape or structure of objects within an image. It uses simple
operations to manipulate the shapes and structures in binary or grayscale images.

By MT
Fundamentals

Segmentation

• It is the technique of dividing an image into

multiple segments or regions to identify and
locate objects and boundaries within the
image.

• Uses various techniques such as thresholding,

edge-based, region-based, clustering-based,
deep learning-based segmentations.

By MT
Fundamentals

Feature Extraction

• This one is usually followed immediately after segmentation, which generally produces raw pixels
as output.

• Feature extraction consists of:

• Feature detection: Finding the features such as edges, corners, textures, shapes, orientation,
and colors in an image, region, or boundary.

• Feature description: Assigning various quantitative attributes to detected features. For

example, corners in the image, their orientation and tilt.

By MT
Fundamentals

Image Pattern Classification (object detection)

• Identifies the objects described by the feature descriptions and assigns a label to them.

• For example, there is shape description “Oval shape > Nested circle > Nested circle” most likely is an
eye.

By MT
Fundamentals

Knowledge base

• A knowledge base refers to a structured collection of information, rules, and algorithms that an
image processing system uses to interpret, analyze, and understand images. This knowledge can be
used to enhance the performance of image processing tasks such as segmentation, feature
extraction, object recognition, and more

By MT
System Components

By MT
System Components

By MT
System Components

Image Sensors

• Captures the light reflected by the object that is to be imaged.

• A digitizer converts the information captured by the sensor to digital form.

• For example, a camera.

Specialized Image Processing Hardware

• This consists of the digitizer mentioned above and also hardware containing ALU such as GPU to
perform complex operations.

By MT
System Components

Computer

• Can range from a normal computer to a supercomputer dedicated for image processing tasks.

Software

• Consists of various image processing software tools such as MATLAB Image Processing Toolbox.

Mass Storage

• Consists of storage devices to store images for various purposes such as short-term storage, online
storage, archival storage

By MT
System Components

Image Displays

• Monitors used for showing images

Hard copy

• Putting images on paper or films. Various devices such as printers are used.

Networking and Cloud

By MT
Image Sensing and Acquisition

By MT
Image Sensing and Acquisition

• Images are essentially sensed illuminated objects. These objects may just reflect energy produced by
some source of electromagnetic energy or may even be a source itself.

• For example, capturing a photo of a lively sports ground during daytime. The people and the objects
in the ground are illuminated by the sun.

• It’s also possible to capture a photo of the sun itself, which is a source of the electromagnetic energy.

• Sometimes the energy is transmitted through objects instead of being reflected, such as X-rays. A
photo converter (a phosphor screen) converts this energy into visible light for various applications.

By MT
Image Sensing and Acquisition

By MT
Image Sensing and Acquisition

• The figure (a) shows a single sensing

element for capturing energy.

• Made from silicon material, it produces

voltage waveform output depending on
the amount of input energy.

• May have filters to selectively capture a

specific type of energy.

(a) Single sensor (b) Line sensor (c) Array sensor

By MT
Image Sensing and Acquisition

Image Acquisition using a Single Sensing

Element

• To capture 2D images, single sensors can

be combined with a drum structure that as
negative films mounted on it which
rotates.

• As the light passes through, it’s modified

by the film before being sensed by the
sensor.

By MT
Image Sensing and Acquisition

Image Acquisition using a Single Sensing

Element

• The amount of light causes different

voltages to be produced by the sensor
which is the converted to a digital image.

• This is an accurate and inexpensive but

slow method of obtaining high-resolution
images.

• These types of mechanical digitizers

sometimes are referred to as Transmission
Microdensitometers.

By MT
Image Sensing and Acquisition

Image Acquisition using a Strip Sensor

(a)

(b)

By MT
Image Sensing and Acquisition

Image Acquisition using a Strip Sensor

• Linear Sensor Strip:

• The strip provides imaging elements in one direction. Motion perpendicular to the strip
provides imaging in the other direction.

• Used in most flat bed scanners.

• Used routinely in airborne imaging applications, in which the imaging system is mounted on an
aircraft that flies at a constant altitude and speed over the geographical area to be imaged.

By MT
Image Sensing and Acquisition

Image Acquisition using a Strip Sensor

• Circular Sensor Strip:

• Used in medical and industrial imaging to obtain cross-sectional or slices of images of 3-D
objects.

• A rotating X-ray source provides illumination, and X-ray sensitive sensors opposite the source
collect the energy that passes through the object. This is the basis for medical and industrial
computerized axial tomography (CAT) imaging.

• The output of the sensors is processed by reconstruction algorithms whose objective is to

transform the sensed data into meaningful cross-sectional images.

• Images are not obtained directly and from the sensors by motion alone; they also require
extensive computer processing.

By MT
Image Sensing and Acquisition

Image Acquisition using a Sensor Arrays

By MT
Image Sensing and Acquisition

Image Acquisition using a Sensor Arrays

• Electromagnetic, ultrasonic devices, and cameras mostly use this arrangement to capture images.
Cameras use a sensor array type called CCD (Charged-couple Device) arrays.

• The response of each sensor is proportional to the integral of the light energy projected onto the
surface of the sensor, a property that is used in astronomical and other applications requiring low
noise images. Noise reduction is achieved by letting the sensor integrate the input light signal over
minutes or even hours.

• Energy from an illumination source is reflected from a scene. The system collects the incoming
energy and focus it onto an image plane. If the illumination is light, the front end of the imaging
system is an optical lens that projects the viewed scene onto the focal plane of the lens.

By MT
Image Sensing and Acquisition

Image Acquisition using a Sensor Arrays

• The sensor array, which is coincident with the focal plane, produces outputs proportional to the
integral of the light received at each sensor. Digital and analog circuitry sweep these outputs and
convert them to an analog signal, which is then digitized by another section of the imaging system.
The output is a digital image.

By MT
Image Sensing and Acquisition

Simple Image Formation Model

• An image is a matrix of values, denoted by the function f(x, y).

• The value of f at spatial coordinates (x, y) is a scalar quantity whose physical meaning is determined
by the source of the image, and whose values are proportional to energy radiated by a physical
source.

• Therefore, f(x, y) must be nonnegative and finite.

By MT
Image Sensing and Acquisition

Simple Image Formation Model

• Function f(x, y) is characterized by two components:

1. The amount of source illumination on the scene being viewed

2. The amount of illumination reflected by the objects in the scene

• These are called the illumination and reflectance components, and are denoted by
if(x, y) and r(x, y) respectively.

By MT
Image Sensing and Acquisition

Simple Image Formation Model

• The two functions combine to form f(x, y):

f(x, y) = in(x, y) r(x, y)

where

0 ≤ is(x, y) ≤ ∞

and

0 ≤ r(x, y) ≤ 1

By MT
Image Sensing and Acquisition

Simple Image Formation Model

• Reflectance is bounded by 0 (total absorption) and 1 (total reflectance). The nature of i(x, y) is
determined by the illumination source, and r(x, y) is determined by the characteristics of the imaged
objects.

• These expressions are applicable also to images formed via transmission of the illumination through
a medium, such as a chest X-ray. In this case, we would deal with a transmissivity instead of a
reflectivity function, but the limits would be the same.

By MT
Sampling and Quantization

By MT
Sampling and Quantization

• The output of most sensors is a continuous voltage waveform whose amplitude and spatial behavior
are related to the physical phenomenon being sensed.

• In simpler terms, most image sensors create a continuous electrical signal that changes in strength
and pattern based on what they are capturing.

• To create a digital image, we need to convert the continuous sensed data (analog image) into digital
form. This involves two processes:

• Sampling

• Quantization

By MT
Sampling and Quantization

Image Sampling

• Image Sampling is the process of converting a continuous image (usually captured by an optical
system) into a discrete form. This involves taking samples at regular intervals.

• Quantization means the intensity of the each sample. This is affected by various factors such as
illumination and the environment the image was captured in.

• In simpler terms, it is the process of converting a continuous image (like a real-world scene) into a
grid of tiny dots, called pixels. Each pixel captures a small part of the image.

• By combining all these pixels, we can recreate the entire image on a digital screen. The more pixels
we use, the clearer and more detailed the image will be.

By MT
Sampling and Quantization

Image Sampling

By MT
Sampling and Quantization

• To digitize an image, both the co-ordinate and the amplitude (intensity) of each co-ordinate are
needed.

• Digitizing the co-ordinate is known as Sampling and digitizing the amplitude is known is
Quantization.

• The first figure has an equal-spaced line from where each sample will be taken from.

• The second figure is a plot of amplitude (intensity) of the first figure. The variation in the amplitude
is due to noise.

• The dark squares in the third figure indicate the sampled area of the image. The vertical bar
represents the intensity of the gray color, from black to white.

By MT
Sampling and Quantization

Image Sampling

By MT
Neighbors of Pixels

By MT
Neighbors of Pixels

• Each pixel, denoted by P, has neighboring pixels on all 4 sides (horizontal and vertical sides), denoted
by N4(P).

By MT
Neighbors of Pixels

• The pixels on the diagonal sides are denoted by ND(P).

By MT
Neighbors of Pixels

• Both direct and diagonal neighbors make total 8 neighbors, denoted by N8(P).

By MT
Neighbors of Pixels

• Let V be a set of intensity values used to define adjacency.

• In a binary image, the value of V would be V = { 1 }. In a grayscale image, the value of V would be a
subset of the values between 0 – 255.

• Types of Adjacencies:

• 4-adjacency:

• Two pixels p and q with values from V are 4-adjacent if q is in the set N4(p).

• 8-adjacency:

• Two pixels p and q with values from V are 4-adjacent if q is in the set N8(p).

By MT
Neighbors of Pixels

• Types of Adjacencies:

• m-adjacency (or mixed-adjacency):

• Two pixels p and q with values from V are m-adjacent if:

• q is in N4(p)

OR

• q is in ND(p) and the set N4(p) ∩ N4(q) has no pixels whose values are from V.

• Mixed adjacency is a modification of 8-adjacency, and is introduced to eliminate the

ambiguities that may result from using 8-adjacency.

By MT
Neighbors of Pixels

• Types of Adjacencies:

(a) (b) (b)

(a) An arrangement of pixels.

(b) Pixels that are 8-adjacent shown by dashed lines.

(c) m-adjacency

By MT
Neighbors of Pixels

Connectivity

• Let S represent a subset of pixels in an image.

• Two pixels p and q are said to be connected in S

if there exists a path between them consisting
entirely of pixels in S.

• For a given pixel p in S, the connected component is the set of all pixels in S that are connected to p
by some path.

• If the subset S forms a single connected component, meaning any pixel in S can reach any other pixel
in S through a path that stays within S, then S is called a connected set.

By MT
Neighbors of Pixels

Region

• Let R represent a subset of pixels in an image. We call R a region of the image if R is a connected set.

• Two regions, Ri and Rj are said to be adjacent if their union forms a connected set. Regions that are
not adjacent are said to be disjoint.

• 4 and 8-adjacency is used hen referring to regions.

By MT
Neighbors of Pixels

Boundary

• The boundary of a region R is the set of pixels in R that are adjacent to pixels in the complement of
R.

• Or simply put, the boundary of a region R is made up of the pixels in R that touch pixels outside R.

By MT
Distance Measure

By MT
Distance Measure

• For pixels p, q, and s, with coordinates (x, y), (u, v), (w, z) respectively, D is a distance function or
metric if:

• Used when directly measuring distance between points using a straight line.

By MT
Distance Measure

• Another one is a city-block formula for measuring distance between pixels when only moving either
horizontally or vertically (like a city block).

By MT
Distance Measure

• Another one is a formula for moving in a chess-like fashion for measuring distance.

By MT