SRI RAMAKRISHNA ENGINEERING COLLEGE

VATTAMALAIPALAYAM, N.G.G.O. COLONY POST, COIMBATORE – 641 022.

DEPARTMENT OF ELECTRONICS AND INSTRUMENTATION ENGINEERING

Department of Electronics and Instrumentation

Engineering
20EI254 – ROBOTICS AND
MACHINE VISION SYSTEM

Course Instructor
[Link], AP/EIE
SREC, Coimbatore

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 1

 COs,POs,PEO and PSO
On successful completion of the course, the students will be able to
Course Outcomes
CO1 :Infer the basic concepts and types of robotic systems.
CO2 :Illustrate function of different sensors and kinematic representation methods in robots
CO3 :Interpret the concept of control techniques and programming methods in robot
CO4 :Summarize the basic concepts of machine vision system and image processing techniques
CO5 :Apply machine vision system and intelligent techniques for robotic applications.
CO6:Develop program for robotic arm and machine vision applications.
CO7:Communicate the results of the experiments through reports.
POs Mapping with the PO1:Engineering knowledge: Apply the knowledge of mathematics, science, engineering fundamentals, and an
COs engineering specialization to the solution of complex engineering problems.
PO2:Problem analysis: Identify, formulate, review research literature, and analyze complex engineering
problems reaching substantiated conclusions using first principles of mathematics, natural sciences, and
engineering sciences.
PO5:Modern tool usage: Create, select, and apply appropriate techniques, resources, and modern engineering
and IT tools including prediction and modeling to complex engineering activities with an understanding of the
limitations.
PO9:Individual and team work: Function effectively as an individual, and as a member or leader in diverse
teams, and in multidisciplinary settings.
PO10:Communication: Communicate effectively on complex engineering activities with the engineering
community and with society at large, such as, being able to comprehend and write effective reports and design
documentation, make effective presentations, and give and receive clear instructions.
PO12:Life-long learning: Recognize the need for, and have the preparation and ability to engage in
independent and life-long learning in the broadest context of technological change.
PEOs Mapping with the The graduates of this program after four to five years will,
Course PEO I:Engage in design of systems, tools and applications in the field of electronics and instrumentation
engineering and allied engineering industries.

PSO Mapping with the PSO II: Understand, specify and verify the principles of measuring instruments using the state of art
course equipment.
PSO III:Analyze and design process control and industrial automation using modern tools.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 2

 Syllabus
FUNDAMENTALS OF ROBOTICS
Robotics: Asimov's law, robot anatomy and classification of robots - Robot degrees of
freedom - Robot Coordinates - Work space, precision movement - Robot end effectors: types
of end effectors - Types of Gripper mechanisms - Other types of grippers: vacuum cups,
magnetic grippers and adhesive grippers - Consideration in gripper selection and design -
Robotic operating system - IIoT enabled industrial robot: Architecture, OPC robotics
companion specification.
KINEMATICS AND SENSOR FOR ROBOTICS
Kinematics of robot: introduction, matrix representation, homogeneous transformation,
forward and inverse kinematics - Inverse kinematics programming - Robot dynamics: Euler-
Lagrange equations.
Sensors: tactile sensor, velocity sensor and Light Detection and Ranging (LIDAR).
TRAJECTORY CONTROL TECHNIQUES IN ROBOTICS
Joint control: Robot dynamics equation as a control system, Trajectory tracking control, PD
controller and Neural network control design.
Robot Programming: Programming methods, Embedded Programming for Encoder, Driver
based stepper motor and servo motor speed control program - Control 2 degree of freedom
robotic arm for pick and place operation.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 3

 Syllabus
MACHINE VISION SYSTEM
Vision component: Basic components - Elements of visual perception - Types of lenses and
mount - Cameras:sensor, type, size resolution - Camera lenses interface and calculations -
Computer interfaces - Camera calibration.
Basic image processing terms - Obtaining image - Image conversion - Spatial operations -
Image segmentation:segmentation of contours, lines, circles and ellipses - Image feature
extraction: Region feature, line features, point features - Optical Character Recognition
(OCR).
APPLICATIONS
Applications: work piece recognition system with an industrial robot - development of robotic
machine vision system for quality checking of industrial components - AI based mobile robot
for stair case avoidance - OCR based quality control Printed Circuit Board(PCB).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 4

 Books Referred
TEXT BOOKS
1. Groover M.P., "Industrial Robotics - Technology, Programming and Applications", 8th Reprint,
McGraw Hill Publications, 2011.
2. S K Saha, "Introduction to Robotics", 2nd Edition, McGraw Hill Education (India) Private
Limited, 2014
3. Peter Corke, "Robotics, Vision and Control Fundamental Algorithms in MATLAB", 2nd
Edition, Elsevier Publications,2013
REFERENCES
1. Mark W Spong and M. Vidyasagar, "Robot Dynamics and Control", Reprint, Wiley India Pvt.
Limited, 2008.
2. Danny Staple "Learn Robotics Programming Build and control AI-enabled
autonomous robots using the Raspberry Pi and Python", 2nd Edition, Packt Publishing, 2021
3. Francis X Govers, "Artificial Intelligence for Robotics", 1st Edition, Packt Publishing, 2018
4. Stefan Florczyk, "Robot Vision Video-based Indoor Exploration with Autonomous and Mobile
Robots", 2nd Edition, Wiley, 2006

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 5

 MODULE IV

MACHINE VISION SYSTEM

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 6

 MACHINE VISION SYSTEM

Machine vision :
Machine vision (MV) is the technology and
methods used to provide imaging-based automatic
inspection and analysis for such applications as
automatic inspection, process control, and robot
guidance in industry.
Robot vision :
Robot vision may be defined as the process of
extracting, characterizing, and interpreting
information from images of a Two or three
dimensional world

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 7

 Video – Machine Vision System

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 8

 Function of a Machine Vision System

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 9

 Sampling, Quantization, Encoding
• Images captured by cameras are typically continuous
signals. Sampling involves capturing discrete samples
of these signals at regular intervals in both the
horizontal and vertical directions.
• quantization involves converting the sampled intensity
values (usually represented as analog voltages) into
discrete digital values. This process is necessary
because digital systems can only represent a finite
number of levels of brightness or color.
• Once the image is sampled and quantized, it needs to be
encoded into a digital format. Common encoding
schemes for images include formats like JPEG, PNG,
TIFF, etc.
04/16/26 Module IV-R&[Link], AP(O.G)/EIE 10
 Data Reduction:

Region of Interest (ROI) selection: Focusing only on

specific areas or regions within the image that are of
interest for analysis.

•Segmentation:
•Importance: Segmentation is essential for
identifying objects or regions of interest within an
image and separating them from the background or
other objects. It forms the basis for further analysis
and understanding of the visual scene.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 11

 Feature Extraction:

• Importance: Features provide quantitative

information about the visual content of the image,
which can be used for tasks such as object
recognition, classification, or tracking.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 12

 Camera and Lighting

•A simplified camera setup consists of camera, lens,

lighting, and object. The lighting illuminates the
object and the reflected light is seen by the camera.
The object is often referred to as target

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 13

 A/D Converter

• Digitization = Sampling +
Quantization + Encoding)
• Sampling
• digitization of temporal or spatial coordinates
• Quantization
• digitization of amplitude or intensity
• Encoding
• reduce/minimize the amount of data

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 14

 Frame grabber

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 15

 Frame grabber
• A frame grabber is an electronic device that
captures (i.e., "grabs") individual, digital still
frames from an analog video signal or a digital
video stream.
• It is usually employed as a component of
a computer vision system, in which video frames
are captured in digital form and then displayed,
stored or transmitted.
• Early frame grabbers typically had only
enough memory to store a single digitized video
frame, hence the singular "frame" in the device
name, whereas modern frame grabbers often can
store multiple frames.
04/16/26 Module IV-R&[Link], AP(O.G)/EIE 16
 Optical Character Recognition (OCR)

• OCR (optical character recognition) is the use of

technology to distinguish printed or handwritten
text characters inside digital images of physical
documents, such as a scanned paper document.

• The basic process of OCR involves examining the

text of a document and translating the characters
into code that can be used for data processing. OCR
is sometimes also referred to as text recognition.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 17

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 18
 ILLUMINATION TECHNIQUES

• Illumination of a scene is an important factor that

often affects the complexity of vision algorithms.
• A well designed lighting system illuminates a scene
so that the complexity of the resulting image is
minimised, while the information required for
object detection and extraction is enhanced.
• Arbitrary lighting of the environment is often not
acceptable because it can result in low contras
images, specular reflections, shadows and
extraneous details.
• There are 4 main illumination techniques for a robot
work space :
04/16/26 Module IV-R&[Link], AP(O.G)/EIE 19
 Basic components of machine vision system
• The major components of a machine vision system
include the lighting, lens, image sensor, vision
processing, and communications.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 20

 Lighting & Lens

• Lighting illuminates the part to be inspected

allowing its features to stand out so they can be
clearly seen by camera.
• The lens captures the image and presents it to the
sensor in the form of light.
• The sensor in a machine vision camera converts this
light into a digital image which is then sent to the
processor for analysis.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 21

 Vision processing & communication

• Vision processing consists of algorithms that review

the image and extract required information, run the
necessary inspection, and make a decision.

• Finally, communication is typically accomplished

by either discrete I/O signal or data sent over a
serial connection to a device that is logging
information or using it.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 22

 Machine Vision System

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 23

 Elements of visual perception

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 24

 Elements of visual perception

• Although the digital image processing field is built

on a foundation of mathematical and probabilistic
formulations, human intuition.

• Analysis play a central role in the choice of one

technique versus another, and this choice often is
made based on subjective, visual judgments.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 25

 The Human Eye

• The eye is nearly a sphere, with an average diameter

of approximately 20 mm.
Three membranes enclose the eye
• Cornea & sclera
• Choroid
• Retina

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 26

 Cornea

• The cornea is a tough, transparent tissue that covers

the anterior surface of the eye. Continuous with the
cornea, the sclera is an opaque membrane that
encloses the remainder of the optic globe.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 27

 The Choroid

• The choroid contains blood vessels for eye nutrition

and is heavily pigmented to reduce extraneous light
entrance and backscatter.
• It is divided into the ciliary body and the iris
diaphragm, which controls the amount of light that
enters the pupil (2 mm ~ 8 mm).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 28

 The Retina

• The retina lines the entire posterior portion.

• Discrete light receptors are distributed over the
surface of the retina:
• cones (6-7 million per eye) and
• rods (75-150 million per eye)

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 29

 The Lens

• The lens is made up of fibrous cells and is

suspended by fibers that attach it to the ciliary body.
• •It is slightly yellow and absorbs approx. 8% of the
visible light spectrum.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 30

 Cones

• Cones are located in the fovea and are sensitive to

color.
• Each one is connected to its own nerve end.
• Cone vision is called photopic(or bright-light
vision).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 31

 Rods

• Rods are giving a general, overall picture of the

field of view and are not involved in color vision.
• Several rods are connected to a single nerve and are
sensitive to low levels of illumination (scotopicor
dim-light vision).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 32

 Image Formation in the Eye

• The eye lens (if compared to an optical lens) is

flexible.
• It gets controlled by the fibers of the ciliary body
and to focus on distant objects it gets flatter (and
vice versa).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 33

 Image Formation in the Eye

• Distance between the center of the lens and the

retina (focal length):
• –varies from 17 mm to 14 mm (refractive power of
lens goes from minimum to maximum).
• •Objects farther than 3 m use minimum refractive
lens powers (and vice versa).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 34

 Example:

• Calculation of retinal image of an object

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 35

 Brightness Adaptation &
Discrimination
• The eye also discriminates between changes in
brightness at any specific adaptation level.

Where : : the increment of illumination

• discriminable 50% of the time and
• I : background illumination

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 36

 Cameras

• Cameras can be of two types—analog and digital.

Machine vision systems
• predominantly use digital cameras, which can be
either line scan or area
• scan cameras. The image sensor and the lens are key
components of the camera.
• The resolution of the image is determined by the
image sensor, while the
• lens determines the focal length of the camera.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 37

 CCD and CMOS Image Sensors

• The sensor in the camera that captures the image of

an object or scenery is known as the image sensor.
Image sensors are divided into two types:
• charge-coupled device (CCD)
• complementary metal-oxide semiconductor
(CMOS).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 38

 CCD sensors

• In CCD sensors, the electrical charges are

transferred line by line using
• control circuitry and read like a shift register as
shown in Figure.
• The output is converted to a digital signal using an
analog-to-digital converter (ADC).

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 39

 CCD sensors

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 40

 CMOS sensor

• In CMOS sensors, at each pixel, integrated circuits

are used to convert the electrical energy to voltage,
which is then transferred using wires.
• CMOS is illustrated in Figure.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 41

 CMOS sensor

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 42

 Types of lenses and mount

• There are many different types of lenses, including

standard, zoom, and telecentric lenses
• Each of these can be used for different applications.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 43

 Camera Type

• Machine vision systems essentially use two

different camera types: the line scan and area scan
cameras.
• Line scan cameras capture the image as a single row
of pixels.
• Area scan cameras while contain a matrix of pixels
that capture an image of a given scene or object.
• The third type of camera used in machine vision
applications is the smart camera, which has built-in
image processing capability.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 44

 Line scan camera

Fig.13 line scan camera

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 45

 Line scan camera

• Line scan cameras capture images line by line

instead of all at once.
• They typically consist of a single row of pixels and
rely on moving the object being imaged to capture
the entire scene.
• Line scan cameras are particularly useful for
inspecting objects that are moving along a
production line or conveyor belt.

Module IV-R&[Link],
04/16/26 46
AP(O.G)/EIE
 Line scan camera

Advantages:
•High resolution in the direction of motion.
•Suited for continuous processes.
•Can achieve very high speeds.
Disadvantages:
•Limited to objects moving in a linear path.
•Requires synchronization with the motion of the
object or camera.
•May require additional image processing to
reconstruct a full 2D image.
Module IV-R&[Link],
04/16/26 47
AP(O.G)/EIE
 Area scan camera

Fig.14 Area scan camera

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 48

 Area scan camera

• Area scan cameras capture the entire image at once,

similar to traditional consumer cameras.
• They consist of an array of pixels arranged in rows
and columns.
• Area scan cameras are versatile and can be used in a
wide range of applications including quality control,
part inspection, object recognition, and surveillance.

Module IV-R&[Link],
04/16/26 49
AP(O.G)/EIE
 Area scan camera

Advantages:
• Can capture still or moving objects in any direction.
• Simpler setup compared to line scan cameras.
• Suitable for a variety of applications.
Disadvantages:
• Limited by frame rate for capturing moving objects.
• Lower resolution in the direction of motion compared to
line scan cameras when imaging moving objects.

Module IV-R&[Link],
04/16/26 50
AP(O.G)/EIE
 Camera Lens

• As in the human eye, the lens is used to focus the

light reflected from a scene or object onto the image
sensors in a camera.
• The camera then converts the light energy to
electrical energy to form the digital image.
• It is important to understand the relation between
the image sensor and the lens, as it impacts the
design of machine vision solutions.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 51

 Skeletal view of a lens interior

Fig.15 Lens

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 52

 Resolution

• The resolution of the image captured by the camera

is determined by the number of pixels in the sensor
array in the camera.
• For an area scan camera, the image resolution is
marked as N × M matrix, where N denotes the
number of rows and M the number of columns in
the sensor array.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 53

 Lenses and Their Parameters

Several factors have to be considered while selecting

a lens for a machine vision application.
These include:
•The resolution of the camera sensor
•The distance between the object and the camera
•The size of the object
•The amount of light available for image capture

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 54

 Lenses with shorter and longer focal lengths.

Shorter Focal Length Lenses:

•Lenses with shorter focal lengths have a wider field
of view, allowing them to capture more of the scene
in a single frame.
Longer Focal Length Lenses:
•Longer focal length lenses have a narrower field of
view, resulting in a closer and more magnified view
of the subject.

Module IV-R&[Link],
04/16/26 55
AP(O.G)/EIE
 Lenses with shorter and longer focal lengths.

Fig.16 Field of view

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 56

 Relation between FOV and WD

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 57

 Relation between FOV and WD

• The trigonometric relationship between FOV and

WD is

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 58

 Lens with different f-numbers

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 59

 Sensor Size and Respective Diagonal Size

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 60

 Types of Lenses

• There are many different types of lenses, including

standard, zoom, and telecentric lenses.
• Each of these can be used for different applications.
• Standard lenses, also known as prime lenses, come
with a fixed focal length.
• The focal length is normally expressed in
millimeters, such as 50, 85, or 100 mm.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 61

 Types of Lenses

• Zoom lenses have focal lengths that are adjustable

over some range.
• They are useful for prototypes, where the focal
length requirement is not yet determined or for
applications where a standard lens of required focal
length
• Wide-angle lenses are used when the frame of the
object or scene that is to be captured is large.
• The field of view that they can capture is wider
than that of the standard lens and these lenses have
a shorter focal length with a greater DoF.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 62

 Lens Mounts

• Lens mounts are used to attach the lenses to

cameras.
• These mounts provide mechanical stability and
define the distance between the sensors and image.
• Lenses can be proprietary, which means that certain
lenses can be used only with cameras made by the
same manufacturer

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 63

 Lens Mounts

• Presently, mounts generally come in three major

types: C-mount, CS-mount, S-mount, and F-mount.
The sizes for each mount are based on industry
standards.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 64

 Lens Mounts

• For example, it is possible to use a C-mount lens

with a CS-mount with an extension tube.
• However, a C-mount camera cannot be used with a
CS-mount lens.
• The C-mount is the most common type of lens
mount used in machine vision applications. It can be
used with a range of lenses and accessories.
• For instance, the back focal distance of a C-mount
connector is 17.526 mm, while that of a CS-mount
is 12.526 mm.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 65

 Lens Mounts

• Hence, a C-mount lens can be used on a CS-mount

camera with an extension tube or adapter of 5 mm
fitted between the camera and lens.

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 66

 Field of View (Image Size) Is Much Larger
Than Camera Sensor Size
• For applications in this scenario, the field of view
(FOV) can typically range from about 20–30
millimeters to as large as 100 meters for outdoor
applications.
• Let us assume that we have a camera with a sensor
size of 2/3″. If an application requires the camera to
look at a FOV of 50 mm (horizontal) and a working
distance of about 200 mm, by using the equations
mentioned earlier we should be able to estimate a
lens suitable for this application as follows:

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 67

 Field of View (Image Size) Is Much Larger
Than Camera Sensor Size
• FOV = 50 mm
• Sensor size = 8.8 mm (based on 2/3″ sensor size)
• Working distance = 200 mm
• Magnification = 0.176
• Estimated focal length = 29.93 mm

04/16/26 Module IV-R&[Link], AP(O.G)/EIE 68

 Sample Calculation
• Let sensor size = 290 mm
• FOV = 28.7 mm
• Distance from sensor to measuring region (WD) =
605 mm
• Then magnification factor β = FOV/Sensor length =
28.7/290 = 1/10.1 = 1/10 (approximately).
• With magnification β and the distance from sensor
to measuring region D,
• focal length of a lens can be derived using the
following formula:
• Focal length f =D/ (1/ β + 2) = 605 mm/ (10.1 + 2) =
50 mm
04/16/26 Module IV-R&[Link], AP(O.G)/EIE 69
 Sample Calculation
• Let sensor size = 10.67 mm
• FOV = 90 mm
• Distance from sensor to measuring region (WD) =
400 mm
• Then magnification factor β = FOV/Sensor length =
28.7/290 = 1/10.1 = 1/10 (approximately).
• With magnification β and the distance from sensor
to measuring region D,
• focal length of a lens can be derived using the
following formula:
• Focal length f =D/ (1/ β + 2) = 605 mm/ (10.1 + 2) =
50 mm
04/16/26 Module IV-R&[Link], AP(O.G)/EIE 70