Preliminarie Digital Image DIP vs.
Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
01 Introduction
Computer Vision
Engr. Muhammad Ahsan Raees
ahsanraees576@[Link]
m
Department of Computer
Sciences
Barani Institute of
Management Sciences,
Rawalpindi
1
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
"One picture is worth more than ten thousand
words."
Arthur Brisbane (1911)
Figure : TESLA’s self driving
car. 4
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Prerequisites
This course is appropriate as a first course for UG/PG
students with a EE/CE/CS background, which should
have prepared them with these essential prerequisites:
Good working knowledge of MATLAB or OpenCV
with Python or C/C++ (or willingness and time to
learn quickly), data structures
Linear algebra, calculus and probability
While a background in image processing would
certainly be beneficial.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Textbook
[Link]
6
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Reference book
Multiple View Geometry in Computer Vision by Richard
Hartley
and Andrew Zisserman (latest 7
edition) Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Image Formation: Reflected Light
The colors that we
perceive are
determined by the
nature of the light
reflected from an
object.
For example, if white
light is shone onto a
green object most
wavelengths are
absorbed, while green
light is reflected from
the object
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is a digital image?
An image is a 2D projection of the 3D scene.
Mathematically, an image is a 2D function f (x, y ),
where x
and y are spatial (plane) coordinates, and f (x, y )
is the amplitude of the function at (x, y ) called
intensity or gray level.
A digital image is a f (x, y ), where x, y and f (x, y )
are all finite and discrete quantities. An image
point at (x, y ) is called a picture element, image
element, pel or pixel.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is a digital image?
A digital image is a representation of a two-dimensional
image as a finite set of digital values, called picture
elements or pixels.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is a digital image?
Common image formats include:
1 sample per point (B&W or Grayscale)
3 samples per point (Red, Green, and Blue)
4 samples per point (Red, Green, Blue, and
"Alpha", a.k.a. Opacity or Transparency)
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is a digital image?
- Pixel values typically represent gray levels, colors,
heights, opacities etc.
- Remember digitization implies that a digital image
is an
approximation of a real scene
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
What is Image Processing?
Image processing is a method to perform some
operations on an image, in order to get an enhanced
image or to extract some useful information from it.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is Digital Image Processing?
Digital image processing is the transformation of
a digital image by means of a digital computer.
Image processing starts with an image and
(usually) produces a modified version of that
image - e.g., enhancement, restoration,
compression, i.e., the output is also an image.
Machine vision / computer vision aims to
understand the content of images at a higher
level so as to take decisions based on visual input.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Image Acquisition
Figure : An example of the digital image acquisition process. (a) Energy
("illumination") source. (b) An element of a scene. (c) Imaging system. (d)
Projection of
the scene onto the image plane. (e) Digitized
image.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Sampling and Quantization
Figure : Generation of a digital image. (a) Continuous image. (b) A scan line from A to B in the
continuous image, used to illustrate the concepts of sampling and quantization. (c) Sampling and
quantization. (d) Digital scan
line
.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Result of Sampling and Quantization
A digital image is an approximation of a real
world scene.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Digital Image Representation
Number of bits required to store an image
are:
b=M×N×k
Image having 2 intensity levels
k
k = bit depth (bits per pixel) of an image.
For a grey scale image, there are 256
intensity levels (0-255), hence, k = 8.
0 represents black and 255 the white.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Coordinate Convention
Standard convention in programming is to place the
image origin at top, left corner
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
From Image Processing to Computer Vision
Low, mid and high-level
processes
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
From Image Processing to Computer Vision
Figure : Example of Low Level
Processing
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
From Image Processing to Computer Vision
Low, mid and high-level
processes
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
From Image Processing to Computer Vision
Figure : Example of Mid Level Processing -
Segmentation2 of an image into regions
2
Image segmentation is a commonly used technique in digital image processing and analysis to
image into multiple parts or regions, often based on the characteristics of the pixels 25
partition an
in the image.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
From Image Processing to Computer Vision
Low, mid and high-level
processes
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
From Image Processing to Computer Vision
Figure : Example of High Level
Processing
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
From Image Processing to Computer Vision
28
CS-477 Computer Vision by Dr. Mohsin Kamal
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is vision?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is (computer) vision?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is (computer) vision?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
What is computer vision?
The ability of computers to see.
Extract descriptions of the world from pictures or
sequences of pictures (videos) [Forsyth & Ponce,
2003].
"Vision is the process of discovering from images
what is present in the world, and where it is"
(David Marr).
Computer vision is the study of the analysis of
pictures and videos in order to achieve results
similar to those done by human.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Computer Vision
Make computers
understand images
and videos.
What kind of scene?
Where are the cars?
How far is the
building?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Vision for perception, interpretation
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
The goal of computer vision
To bridge the gap between pixels and
"meaning"
What we What a computer
see sees
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Focus of computer vision
Computer Vision focuses on the following problems:
What information should be extracted from
the visual sensors?
How is this information
extracted? How should it be
represented?
How the information be used to allow a robotics
system to perform its tasks?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Overview of classical computer vision
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Why vision?
Images are
everywhere
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Why computer vision matters?
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
A story about computer
vision
In 1966, Marvin Minsky3 at MIT asked his
undergraduate student Gerald Jay Sussman to
"spend the summer linking a camera to a computer
and getting the computer to
describe4 what it saw". We now know that the problem
is slightly more difficult than that. (Szeliski 2009,
Computer Vision)
3
Founder, MIT AI
project
4
Image
understanding Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
A story about computer
vision
In 1966, Marvin Minsky at MIT asked his undergraduate
student Gerald Jay Sussman to "spend the summer
linking a camera to a computer and getting the
computer to describe what it saw". We now know that
the problem is slightly more difficult than that.
"Needless to say, Sussman didn’t make the deadline,".
"Vision turned out to be one of the most difficult and
frustrating challenges in AI over the next four decades.
As machine vision expert Berthold Horn once pointed
out, Sussman opted never to work in vision again."
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Ridiculously brief history of computer
vision
1966: Minsky assigns computer vision
as an undergrad summer project
1960’s: interpretation of synthetic
worlds
1970’s: some progress on interpreting
selected images
1980’s: ANNs come and go; shift
toward geometry and increased
mathematical rigor
1990’s: face recognition; statistical
analysis in vogue
2000’s: broader recognition; large
annotated datasets available; video
processing starts; vision & graphis; vision
for HCI; internet vision, etc.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Optical character recognition (OCR)
-Technology to convert scanned docs to
text
-If you have a scanner, it probably came with OCR
software
License plate readers
Digit recognition, AT&T labs [Link]
http: Automatic_number_plate_
//[Link]/~yann/ recognition
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Face detection
Many new digital cameras now detect faces e.g.,
Canon, Sony, Fuji, ...
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Smile detection
The smile shutter flow
- A camera is smart enough to catch every smile.
- In Smile Shutter Mode, camera can automatically
trip the shutter at just the right instant to catch the
perfect expression.
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Object recognition (in supermarkets)
LaneHawk by Evolution Robotics
A smart camera is flush-mounted in the checkout lane, continuously
watching for items. When an item is detected and recognized, the cashier
verifies the quantity of items that were found under the basket, and
continues to close the transaction. The item can remain under the basket,
and with LaneHawk,you are assured to get paid for it.
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Vision-based biometrics
The remarkable story of
Sharbat Gula, first
photographed in 1984 aged
12 in a refugee camp in
Pakistan by National
Geographic photographer
Steve McCurry, and traced 18
years later to a remote part
"How the Afghan Girl was Identified by Her Iris Patterns" of Afghanistan where she was
Read the story at again photographed by
[Link]
McCurry. National Geographic
accepted and published this
conclusion in a sec- ond cover
issue featuring Sharbat Gula,
18 years after the first, and
the Society launched their
"Afghan Girl’s Fund" to assist
the education of Muslim girls
in cultures that dis- courage
Muhammad Ahsan Raees or prohibit female educa- tion.
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Login without a password
Face recognition systems now beginning to appear
more widely
[Link]
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Object recognition (in mobile phones)
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Special effects: motion capture
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Sports
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Smart cars
MobileEye (
Vision systems currently in many car
[Link] 54
models Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Industrial robots
Vision-guided robots position nut runners on
wheels
Muhammad Ahsan Raees
Preliminarie Digital Image DIP vs. Computer State-of-the-art in Challenges in
s Processing CV Vision CV CV
Medical imaging
Image guided surgery
[Link]
3D imaging MRI, research/
CT medical-vision-group
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Vision as a source of semantic
information
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer Vision State-of-the-art in CV Challenges in
CV
Vision as a source of semantic information
Object categorization
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer Vision State-of-the-art in CV Challenges in
CV
Vision as a source of semantic information
Scene and context categorization
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer Vision State-of-the-art in CV Challenges in
CV
Vision as a source of semantic information
Qualitative spatial information
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
1 Preliminaries
2 Digital Image
Processing
3 DIP vs. CV
4 Computer Vision
5 State-of-the-art in CV
6 Challenges in CV
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Scale
Viewpoint variation
variation
Illumination
conditions
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Occlusio
n
Deformati
on
Background clutter
Intra-class
variation
Muhammad Ahsan Raees
Preliminaries Digital Image Processing DIP vs. CV Computer State-of-the-art in Challenges in
Vision CV CV
Any Question?
Thank You
Muhammad Ahsan Raees