SRI RAMAKRISHNA ENGINEERING COLLEGE
VATTAMALAIPALAYAM, N.G.G.O. COLONY POST, COIMBATORE – 641 022.
DEPARTMENT OF ELECTRONICS AND INSTRUMENTATION ENGINEERING
Department of Electronics and Instrumentation
Engineering
20EI254 – ROBOTICS AND
MACHINE VISION SYSTEM
Course Instructor
[Link], AP/EIE
SREC, Coimbatore
04/16/26 Module V-R&[Link], AP(O.G)/EIE 1
COs,POs,PEO and PSO
On successful completion of the course, the students will be able to
Course Outcomes
CO1 :Infer the basic concepts and types of robotic systems.
CO2 :Illustrate function of different sensors and kinematic representation methods in robots
CO3 :Interpret the concept of control techniques and programming methods in robot
CO4 :Summarize the basic concepts of machine vision system and image processing techniques
CO5 :Apply machine vision system and intelligent techniques for robotic applications.
CO6:Develop program for robotic arm and machine vision applications.
CO7:Communicate the results of the experiments through reports.
POs Mapping with the PO1:Engineering knowledge: Apply the knowledge of mathematics, science, engineering fundamentals, and an
COs engineering specialization to the solution of complex engineering problems.
PO2:Problem analysis: Identify, formulate, review research literature, and analyze complex engineering
problems reaching substantiated conclusions using first principles of mathematics, natural sciences, and
engineering sciences.
PO5:Modern tool usage: Create, select, and apply appropriate techniques, resources, and modern engineering
and IT tools including prediction and modeling to complex engineering activities with an understanding of the
limitations.
PO9:Individual and team work: Function effectively as an individual, and as a member or leader in diverse
teams, and in multidisciplinary settings.
PO10:Communication: Communicate effectively on complex engineering activities with the engineering
community and with society at large, such as, being able to comprehend and write effective reports and design
documentation, make effective presentations, and give and receive clear instructions.
PO12:Life-long learning: Recognize the need for, and have the preparation and ability to engage in
independent and life-long learning in the broadest context of technological change.
PEOs Mapping with the The graduates of this program after four to five years will,
Course PEO I:Engage in design of systems, tools and applications in the field of electronics and instrumentation
engineering and allied engineering industries.
PSO Mapping with the PSO II: Understand, specify and verify the principles of measuring instruments using the state of art
course equipment.
PSO III:Analyze and design process control and industrial automation using modern tools.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 2
Syllabus
FUNDAMENTALS OF ROBOTICS
Robotics: Asimov's law, robot anatomy and classification of robots - Robot degrees of
freedom - Robot Coordinates - Work space, precision movement - Robot end effectors: types
of end effectors - Types of Gripper mechanisms - Other types of grippers: vacuum cups,
magnetic grippers and adhesive grippers - Consideration in gripper selection and design -
Robotic operating system - IIoT enabled industrial robot: Architecture, OPC robotics
companion specification.
KINEMATICS AND SENSOR FOR ROBOTICS
Kinematics of robot: introduction, matrix representation, homogeneous transformation,
forward and inverse kinematics - Inverse kinematics programming - Robot dynamics: Euler-
Lagrange equations.
Sensors: tactile sensor, velocity sensor and Light Detection and Ranging (LIDAR).
TRAJECTORY CONTROL TECHNIQUES IN ROBOTICS
Joint control: Robot dynamics equation as a control system, Trajectory tracking control, PD
controller and Neural network control design.
Robot Programming: Programming methods, Embedded Programming for Encoder, Driver
based stepper motor and servo motor speed control program - Control 2 degree of freedom
robotic arm for pick and place operation.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 3
Syllabus
MACHINE VISION SYSTEM
Vision component: Basic components - Elements of visual perception - Types of lenses and
mount - Cameras:sensor, type, size resolution - Camera lenses interface and calculations -
Computer interfaces - Camera calibration.
Basic image processing terms - Obtaining image - Image conversion - Spatial operations -
Image segmentation:segmentation of contours, lines, circles and ellipses - Image feature
extraction: Region feature, line features, point features - Optical Character Recognition
(OCR).
APPLICATIONS
Applications: work piece recognition system with an industrial robot - development of robotic
machine vision system for quality checking of industrial components - AI based mobile robot
for stair case avoidance - OCR based quality control Printed Circuit Board(PCB).
04/16/26 Module V-R&[Link], AP(O.G)/EIE 4
Books Referred
TEXT BOOKS
1. Groover M.P., "Industrial Robotics - Technology, Programming and Applications", 8th Reprint,
McGraw Hill Publications, 2011.
2. S K Saha, "Introduction to Robotics", 2nd Edition, McGraw Hill Education (India) Private
Limited, 2014
3. Peter Corke, "Robotics, Vision and Control Fundamental Algorithms in MATLAB", 2nd
Edition, Elsevier Publications,2013
REFERENCES
1. Mark W Spong and M. Vidyasagar, "Robot Dynamics and Control", Reprint, Wiley India Pvt.
Limited, 2008.
2. Danny Staple "Learn Robotics Programming Build and control AI-enabled
autonomous robots using the Raspberry Pi and Python", 2nd Edition, Packt
Publishing, 2021
3. Francis X Govers, "Artificial Intelligence for Robotics", 1st Edition, Packt Publishing, 2018
4. Stefan Florczyk, "Robot Vision Video-based Indoor Exploration with Autonomous and Mobile
Robots", 2nd Edition, Wiley, 2006
04/16/26 Module V-R&[Link], AP(O.G)/EIE 5
Machine Vision System
• A machine vision system (MVS) is a type of
technology that enables a computing device to
inspect, evaluate and identify still or moving
images.
• It is a field in computer vision and is quite similar
to surveillance cameras, but provides automatic
image capturing, evaluation and processing
capabilities.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 6
Human Vision
04/16/26 Module V-R&[Link], AP(O.G)/EIE 7
Computer Vision
04/16/26 Module V-R&[Link], AP(O.G)/EIE 8
Computer vision - Classification
04/16/26 Module V-R&[Link], AP(O.G)/EIE 9
Key Stages of digital image processing
04/16/26 Module V-R&[Link], AP(O.G)/EIE 10
work piece recognition system with an
industrial robot
04/16/26 Module V-R&[Link], AP(O.G)/EIE 11
work piece recognition system with an
industrial robot
• Simulation software
04/16/26 Module V-R&[Link], AP(O.G)/EIE 12
Video 1 - work piece recognition system
with an industrial robot
04/16/26 Module V-R&[Link], AP(O.G)/EIE 13
work piece recognition system with an
industrial robot
04/16/26 Module V-R&[Link], AP(O.G)/EIE 14
APPLICATIONS
04/16/26 Module V-R&[Link], AP(O.G)/EIE 15
development of robotic machine vision
system for quality checking of industrial
components
• Initialize the working through simulation software
• Develop the program
• Execute the program
04/16/26 Module V-R&[Link], AP(O.G)/EIE 16
APPLICATIONS
04/16/26 Module V-R&[Link], AP(O.G)/EIE 17
• What is the first mobile robot controlled by AI?
• “Shakey” was the first mobile robot with the ability
to perceive and reason about its surroundings.
• The subject of SRI's Artificial Intelligence Center
research from 1966 to 1972, Shakey could perform
tasks that required planning, route-finding, and the
rearranging of simple objects.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 18
Artificial Intelligence(AI)
• AI means a machine that exhibits some
characteristics of intelligence thinking, reasoning,
planning, learning, and adapting.
• It can also mean a software program that can
simulate thinking or reasoning.
• Examples: a robot that avoids obstacles by simple
rules (if the obstacle is to the right, go left)is not an
AI.
• A program that learns by example to recognize a cat
in a video, is an AI.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 19
Artificial Intelligence(AI)
• A mechanical arm that is operated by a joystick is
not AI, but a robot arm that adapts to different
objects in order to pick them up is AI.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 20
Characteristics of artificial intelligence
robots
There are two defining characteristics of artificial
intelligence robots
•AI robots learn and adapt to their environments,
which means that they change behaviors over time.
•Emergent behavior, where the robot exhibits
developing actions that we did not program into it
explicitly
04/16/26 Module V-R&[Link], AP(O.G)/EIE 21
Solution
• path planning and navigation for our toy-grabbing robot
helper.
• You have to admit that one of the most difficult problems in
any ground robot is navigation.
• There are two parts to the task figuring out where you are
(localization), and then figuring out where you want to go
(path planning).
• Most robots at this point would be using some sort of
Simultaneous Localization and Mapping (SLAM) algorithm
that would first map the room, and then figure out where the
robot is within it.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 22
Process
• Navigate the room avoiding obstacles (toys and
furniture) and hazards (stairs)
• Find a toy
• Pick up the toy with the robot arm
• Carry the toy to the toy box
• Put the toy in the toy box
• Go and find another toy
• If there are no more toys, then stop
04/16/26 Module V-R&[Link], AP(O.G)/EIE 23
Stairs Avoidance
Our liabilities:
•We have a small robot with a very limited sensor and compute
capability.
•We have a room full of misplaced toys and a set of deadly stairs
the robot can fall down.
Our assets:
•We have a robot with tracks that can drive around, a voice, one
camera, and a robot arm. The robot has a datalink via Wi-Fi to a
control computer.
• We have a toy box that is a distinctive color. And lots of Toys of
Usual Size (TOUS).
04/16/26 Module V-R&[Link], AP(O.G)/EIE 24
04/16/26 Module V-R&[Link], AP(O.G)/EIE 25
04/16/26 Module V-R&[Link], AP(O.G)/EIE 26
04/16/26 Module V-R&[Link], AP(O.G)/EIE 27
Neural Network
• We drive the robot around, and take a picture about four times a
second. We record what the robot is doing at each picture –
going forward, turning right, turning left, or backing up.
• We use that information to predict the robot's motion path
given the image.
• This has the advantage of avoiding obstacles and hazards
automatically.
• When we get to the stairs (remember I have stairs going down
in my game room that would damage the robot), the robot will
know to turn around, because that is what we did in training.
• We are teaching the robot to navigate the room by example.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 28
04/16/26 Module V-R&[Link], AP(O.G)/EIE 29
• Each frame is labeled by simply looking at the value
for the cmd_vel topic. If the angular velocity Z term
(angular.z) is positive, we are turning right.
• If it is negative, we are turning left, and if the term
is zero (you guessed it)
04/16/26 Module V-R&[Link], AP(O.G)/EIE 30
OCR ON ASSEMBLED PCB
COMPONENTS
• Most chips assembled onto a PCB are labeled with a
string of alphanumeric characters to track them
through production.
• Specular glare can result in low-contrast images,
which make it difficult for a machine vision system
to locate and recognize characters.
• To successfully decode characters on electronic
components and modules, an optical character
recognition (OCR) system needs to tolerate
reflective surfaces as well as deformed, skewed, and
poorly etched characters.
04/16/26 Module V-R&[Link], AP(O.G)/EIE 31
04/16/26 Module V-R&[Link], AP(O.G)/EIE 32
PCB COMPONENT PLACEMENT
GUIDANCE
04/16/26 Module V-R&[Link], AP(O.G)/EIE 33
machine vision system for quality checking
of industrial components
Module V-R&[Link],
04/16/26 34
AP(O.G)/EIE
• Machine vision is used in a variety of industrial
processes, such as material inspection, object
recognition, pattern recognition, electronic
component analysis, along with the recognition of
signatures, optical characters, and currency.
• Machine vision is the capability of a computer to
perceive the environment. One or more video
cameras are used with analog-to-digital conversion
and digital signal processing. The image data is sent
to a computer or robot controller.
Module V-R&[Link],
04/16/26 35
AP(O.G)/EIE
• A robot’s vision system consists of a number of essential
components, which include the camera that captures a
picture, to the processing mechanism that provides and
communicates the result. For any machine vision system to
work dependably and produce results that can be repeated,
it is important how these essential components interact.
• Lighting is very important to machine vision as it
illuminates the part to be viewed so that its features stand
out allowing the camera to see clearly. The lens captures
the image and communicates it to the sensor in the form of
light. The sensor in a machine vision camera translates this
light into a digital image, which is then relayed to the
processor for analysis.
Module V-R&[Link],
04/16/26 36
AP(O.G)/EIE
• Vision processing employs algorithms that review
the image and extricate the required information,
run the required inspection, and make a decision.
Finally, communication is accomplished by either a
discrete I/O signal or data transmitted over a serial
connection to a device that is logging or using the
information.
Module V-R&[Link],
04/16/26 37
AP(O.G)/EIE