MODULE 3
Robotic Vision & Kinematics
Robotic Vision: Sensing, Pre-processing, Segmentation, Description, Recognition,
Interpretation, Feature extraction -Camera sensor hardware interfacing. Representation of
Transformations - Representation of a Pure Translation - - Pure Rotation about an Axis -
Combined Transformations - Transformations Relative to the Rotating Frame.
Basic understanding of Differential-Drive Wheeled Mobile Robot, Car-Like Wheeled Mobile
Robot. Kinematic model of a differential drive and a steered mobile robot, Degree of freedom
and manoeuvrability, Degree of steerability, Degree of mobility - different wheel
configurations, holonomic and nonholonomic robots. Omnidirectional Wheeled Mobile
Robots
Robot Vision
• Essentially, robot vision is a sophisticated technology that helps a robot, usually an
automated robot, better identify things, navigate, find objects, inspect, and handle parts or
bits before an application is performed.
• Robotic vision systems allow robots to “see.” With a camera and an AI algorithm, robots
can dynamically interpret their environment and interact with it. In practice, the technology
allows robots to interpret and respond to visual information relevant to their surroundings
in near-real-time.
• For example, robots equipped with a camera can perform optical inspections, sort objects
and take measurements. So, a robot could see at the end of an assembly process if the
products are assembled properly.
Robotic vision systems
• Highly Sophisticated sensor system in robots
• These sensors are used for interaction with the environment
(External sensors).
• Uses a camera for capturing the scene.
• The images are processed using image processing techniques
for further analysis.
• This system is also known as Computer vision.
Object Detection, Face recognition Automatic Navigation
Images
• The images obtained from normal cameras are 2D images.
• They lack depth information.
• Each point in the image will be represented using 2 axes.
• Images obtained from CT scanners will be of 3D (contains depth
information).
• A digital image is a rectangular array of dots or picture elements
called PIXELS, arranged in the form of a matrix with m rows and
n columns. The expression m×n is called the resolution of the
image.
1 pixel
• The elements (values) of the matrix is the intensity of light
coming from the object.
• So an Image is the collection of data representing the light
intensities of a large number of pixels.
Stages of Robotic Vision
1. Sensing:
• The process of capturing or yielding an image.
• Images can be captured using Analog Cameras and Digital
cameras.
• But for computer vision, images should be digitized.
• These digitized images are stored in the computer memory in the
binary form (1s and 0s).
2. Preprocessing:
• Preprocessing is done to convert the image into a proper form to
be processed by a processor.
• It includes, noise remover, color conversions, enhancements etc.
3. Segmentation:
• Process of partitioning image into objects of interest.
4. Description/Feature extraction
• Required features in an image is extracted to differentiate
different objects in an image.
5. Recognition
• Identifying different objects in the images.
• Used for classification purposes, detection of objects etc.
6. Interpretation
• After doing all the processing of the objects, assign meaning to
the recognized objects.
Robot maneuverability
The overall degrees of freedom that a robot can manipulate, called the degree of maneuverability, can
be readily defined in terms of mobility and steerability:
Therefore, maneuverability includes both the degrees of freedom that the robot manipulates directly
through wheel velocity and the degrees of freedom that it indirectly manipulates by changing the
steering configuration and moving.
Representation of Transformations
A transformation is defined as making a movement in space. When a
frame or an object or a vector moves in space relative to a fixed
reference frame, we can represent this motion in a form similar to a
frame representation. This is because a transformation itself is a
change in state of a frame (representing change in its location and
orientation), and thus it can be represented as a frame. A
transformation may be one of the following forms:
• A pure translation
• A pure rotation about an axis
• A combination of translations and/or rotations
Representation of a Pure Translation
If a frame moves in space without any change in its
orientation, the transformation is pure translation.
Representation of a Pure Translation
New frame representation:
Fnew = Trans (dx ,dy ,dz )x Fold
1 0 0 d x nx ox ax p x nx ox ax px d x
0 1 0
d y ny oy ay
p y ny oy
ay py d y
Fnew
0 0 1 d z nz oz az p z nz oz az pz d z
0 0 0 1 0 0 0 1 0 0 0 1
? Example: A frame F has been moved nine units along the x axis and five units
along the z axis of the reference frame. Find the new location of the frame?
?Example: Suppose that instead of a frame, a point
P = (3,5,7)T in space was translated a distance of
d = (2,3,4)T. Find the new location of the point relative to the reference
frame.
? Example: The following frame B was moved a distance of d = (5,2,6)T. Find
the new location of the frame relative to the reference frame.
The transformation matrix representing the translation is used to find the
new location as:
Representation of a Pure Rotation about an axis
(Rotation Matrices)
• Consider a frame(n,o,a) is at origin of reference frame(x,y,z) and parallel to it.
• Let frame(n,o,a) rotate through an angle of θ about x axis of reference
frame(x,y,z).
• Let a point P is attached to the rotating frame with coordinates Px, Py, Pz
relative to reference frame and Pn, Po, Pa relative to moving frame.
• As frame rotates about x-axis, point P also rotates.
Basic Rotation Matrices
1 0 0
Rot ( x, ) 0 C S
0 S C
C 0 S
Rot ( y, ) 0 1 0
S 0 C
C S 0
Rot ( z, ) S C 0
0 0 1
? Problem: The frame F is located at 3,5,7 units, with its n-axis parallel to x, its o-
axis at 45∘ relative to the y-axis, and its a-axis at 45∘ relative to the z-axis. Describe
frame.
nx ox ax px Px 3
n oy ay p y Py 5
Fobject y
nz oz az pz Pz 7
0 0 0 1
1 0 0
Rot ( x, ) 0 C S
0 S C
45
? Problem: A point P[2, 3, 4]T is attached to a rotating frame. The frame rotates 90∘
about the x-axis
of the reference frame. Find the coordinates of the point relative to the reference
frame after the rotation.
1 0 0
Rot ( x, ) 0 C S 90
0 S C
? Problem: Find the coordinates of point P(2,3,4)T relative to the reference
frame after a rotation of
450 about the x-axis.
1 0 0
Rot ( x, ) 0 C S
45
0 S C
? Problem: Find the coordinates of point P(3,5,7)T relative to the reference
frame after a rotation of
300 about the z-axis.
C S 0
Rot ( z, ) S C 0 30
0 0 1
Composite Rotation Matrix
Basic rotation matrices can be multiplied together to represent a sequence of
finite rotations about the principal axes of the OXYZ coordinate system.
The sequence of performing rotations is important.
For example, to develop a rotation matrix representing a rotation of angle about
the OX axis followed by a rotation of angle about the OZ axis followed by a
rotation of angle about the OY axis, the resultant rotation matrix representing
these rotations is :
R = R y, R z, R x,
? Problem: Find the new location of point P(1,2,3)T relative to the reference
frame after a rotation of 300 about z-axis followed by a rotation of 600 about y-
axis.
C 0 S
C S 0
Rot ( y, ) 0 1 0
Rot ( z, ) S C 0
S 0 C
0 0 1
Homogeneous Transformation Matrix (HTM)
A 3X3 rotation matrix does not give us any provision for translation and scaling.
If we represent both orientation and position in same matrix, we will add scale
factors to 3X3 matrix to make it a 4X4 matrix. If we represent the orientation alone,
we may either drop the scale factors and use 3X3 matrices or add a fourth column
with zeros for position in order to keep the matrix square. Matrices of this form are
called homogeneous matrices.
nx ox ax px
n oy ay p y
Fobject y
nz oz az pz
0 0 0 1
Representation of Combined Transformation
Combined transformations consist of a number of successive translations and
rotations about the fixed reference frame axes or the moving current frame axes.
Any transformation can be resolved into a set of translations and rotations in a
particular order.
For example, one may rotate a frame about the x axis, translate about the x, y, z
axes, and then rotate about the y axis in order to accomplish the transformation
that is needed.
This order is very important, and if the order of two successive
transformations is changed, the result may be completely different.
Representation of Combined Transformation
Let us assume that a frame is subjected to the following three successive
transformations relative to reference frame (x, y, z):
i. Rotation of degrees about the x-axis,
i. Followed by translation of [l1, l2, l3],
ii. Followed by a rotation of β degrees about the y-axis.
Pxyz = Rot (y, β) X Trans (l1, l2, l3) X Rot (x,α) X Pnoa
? Problem: A point P(7,3,1)T is attached to (n, o, a) frame and is subjected to
transformations. Find coordinates of point relative to reference frame at
conclusion of transformations (1) rotation of 900 about the z-axis (2) followed by
a rotation of 900 about the y-axis (3) followed by a translation of [4, -3, 7].
C S 0 C 0 S
Rot ( z, ) S C 0 Rot ( y, ) 0 1 0
0 0 1 S 0 C
? Problem: A point P(7,3,1)T is attached to (n, o, a) frame and is subjected to
transformations. Find coordinates of point relative to reference frame at conclusion
of transformations (1) rotation of 900 about the z-axis (2) followed by a translation
of [4, -3, 7] (3 ) followed by a rotation of 900 about the y-axis.
C S 0 C 0 S
Rot ( z, ) S C 0 Rot ( y, ) 0 1 0
0 0 1 S 0 C