Body-Assisted Arm Movement Control for Human-Like Object
Manipulation
Isil Bozma1 and Kadir Cumali1
Abstract This paper presents computational approaches
for transferring neurobiological researches to human-inspired
coordinated control of robots in reaching movements. Inspired
by the control strategies of human reaching system, the robot
systems require incorporation of body segments (torso, arm,
and hand) for the trunk-assisted reach-to-grasp movements.
We explore for robot to coordinate all those parts of the body
while reaching for an object that is out of reach. The method is
to utilize data and output functions that appear to be instrinsic
to human reaching movements by aiming to compute integrated
controllers that lead the robot to coordinate all the segments
of the body. The results are demonstrated in simulation by
considering a robot, which has a 3-DOF robotic arm, and
two-fingered gripper. The control model is compatible with
neuroscientific findings concerning human, such as bell-shaped
velocity profile of hand, and change in grip aperture during
the reaching movement.
proposing the incorporated control of the body of robot, arm,
and gripper. The parameters of our models are estimated
based on the data recorded in the human studies.
Previous researches have studied only the velocity of the
hand [5], [6], generation of reaching motion [7], [9], the
human-like motion planning for reach-to-grasp movements
[8], and computation of human reaching trajectories using
optimization [10]. We have attempted to bridge those studies
combining and contributing to a better understanding of control of trunk-assisted reach-to-grasp movements by proposing
computational models so that robots can perform movements
in a similar way to behaviors of human during the execution
of a task.
I. INTRODUCTION
Basically, the task of reach-to-grasp can be regarded as
a series of human arm movement moving the hand from
an initial position to the object. Even though the definition
seems simple, indeed, reaching movements require the mastering of motor outputs that control the hand by assessing
environmental information collected through sensory inputs.
The central nervous system (CNS) simplifies the control of
reaching by directly computing sensorimotor transformation
of these sensory information into the muscle activations
through the combination of muscle synergies.
In recent years, some epideictic studies have been conducted regarding control of human reaching movements analyzing muscle patterns [11], [12]. However, the coordination
occurs at not only between individual muscles but also at
between joints, and between limbs following a motor control
hierarchy. The coupling between joints, muscle activities and
motions provide interjoint coordination of motor commands
in order to smoothly move the hand for a specific task.
Researches have attempted to behold synergetic control of
fast multijoint arm reaching movements [13], [14], [15],
[16]. Further, the hand must show concurrent adaptation to
the integration of muscles and joints by regulating the grip
aperture as the hand approached the object. In addition to
synchronous movement of the multijoint arm, and hand in
goal - oriented movements, in particular, when intending to
grasp an object out of reach, the motor control system needs
to not only control the moving arm, but also coordinate the
torso with the arm [17], [18], [26].
Human has a multiagency coordination system for the
trunk-assisted reach-to-grasp movements, which means that
the arm and the trunk are coordinated by motor synergies
that appear to position grip aperture for a stable closure to
grasp the object [19].
In recent years, ample neurophysiological and behavioral
results derived from the basic studies [1], [2], [3], [4]
on human reaching movements have attracted a great deal
of attention of researchers studying on humanoid robotics.
Human reach-to-grasp movements provide a pivotal and
instructive paradigm to understand how these movements
will be applied for robots that are programmed in order
to perform anthropomorphic movements for dexterous tasks.
However, due to the fact that absence of tissues and muscles
and the complexity of human articulations, to reproduce
human inspired motions in robots becomes problematic. Ideally then, the optimal way is to observe human movements
for manipulation and to understand the underlying control
process.
In human, reaching is the fundamental component of the
hand movement that requires incorporated movement of body
segments: 1) reaching by the arm and torso, and 2) adjusting
the grip aperture (distance between the index finger and
thumb) during the prehension. In reaching behavior, in some
instances which the object is located beyond the arm reaching
space, the trunk is moved to assist in the transport of the hand
to the object. In addition to trunk motion, the motion of the
arm during reaching is fully specified by the simultaneous
motion of all the joints in the arm. Finally, the grip aperture
changes with respect to the hand position and object width.
Inspired by the control of reaching movement in humans, the
motivation for the ideas presented in this paper is to assess
the experimental bases computationally in order to perform
human-like movements behaviorally for object manipulation
1 Electrical & Electronics Engineering, Intelligent Systems Laboratory,
Bogazici University, Bebek, 34342 Turkey
II. HUMAN REACHING MOVEMENTS
A. Problem Formulation
One of the well known phenomena in human reach-tograsp movements is the inherent coordination between the
motion of hand, arm, and trunk control systems. A prior
study has attempted to demonstrate that the trunk motion
is temporally coupled to both arm motion and aperture
formation [19].
Let us assume a task that the robot seeks to manipulate
an object, which is not in the reach space of the arm. It
means that the robot can not grab the object without moving
the body. There should exist a phase in which the body of
robot moves in concert with the arm. The problem becomes
how the control of comovement of arm and body is provided
considering that it alters the end-effector velocity. The second
phase is associated with the incorporation of the arm and
hand. The gripper aperture opens along the direction of the
robot motion due to the constraints of the robot. When the
hand position is controlled properly to match the center of the
object with the center point between the thumb and finger,
the problem occurs at the control of aperture size as the hand
approaches the object.
We have suggested, thus, that these problems can be
converted into the mathematical models, which descibe the
coordination of body and arm, and involvement between arm
and grip aperture. For model of multi - body incorporation,
complete configuration in robot is specified by vectors, the
position is described as given in equation 1.
qA
q = qG
(1)
qB
qi is the vector of time dependent position variables of
the related segments including body (qB ), arm (qA ), and the
gripper qG . The variable qi represents involvement of the
other parts to the movement. i shows the function that corresponds the relationships of body, arm, and gripper. fB , fA
and fG indicate the motion of body with arm, movement of
arm, and regulation of gripper.
fA (q,t) = A (qA , qA ,t)
(2)
fG (q,t) = G (qG , qG ,t)
(3)
fB (q,t) = B (qB , qB ,t)
(4)
B. Proposed Control Approaches
Identifying and modeling the behaviors in accordance with
human reaching movements provide a promotive research
direction to improve the design of similar controllers in
robots.
1) Control Model of Arm Motion: Control of the human
inspired arm systems for reaching motion should achieve
human - motion characteristics for their acceptability. In
the classic studies [2], [3], [20], [22], on human reachto-grasp movement, it has been discovered that it consists
of two interdependently optimally controlled components,
including transport component and grasp component, which
are required to finish synchronically. These components,
which computes extrinsic object properties (transport component) and objects intrinsic properties (grasp component),
constitute sequential motions that are required for humans
to execute object manipulation. It was obtained some basic
features with regard to human hand function: the velocity
profile of the hand exhibits one peak in the first half of the
movement duration [2], [3], [4].
To clarify this manner, we have proposed a feedback
scheme that computes the remaining distance to the target
as the described by the equation 5. Since the algorithm
is simulated in 3 - DOF robotic arm, qA = [q1 , q2 , q3 ] is
the vector of time dependent joint variables, which points
positions of joints. y(qA ) shows of current coordinates of
end-point of arm. g is a stationary goal coordinates. t, and
represent time, and a variable discretizing time, respectively.
1
A = t 2 ||y(qA ) g||2
(5)
1
A = t 2 (y(qA ) g)T (y(qA ) g)
(6)
The equation 6 expresses a potential function, which is based
on position of the joints and target object. In order to obtain
joint velocities, it will be taken the gradient of the function,
A , with respect to the joint variables due to fact that reaching
movements almost always comprise simultaneous motions at
several joints.
qA = DqA A
(7)
The robot manipulator will be controlled specifying target
object position by the end-effector, whose position and orientation are collected in the vector x of operational positions.
x = J(qA )qA
(8)
The Jacobian, J(qA ), for this system relates how the movement of end-effector is constructed by the joint variables. x
and qA are time derivatives of end-effetor position and joint
positions, respectively.
2) Control Model of Gripper Aperture: In human, during
the approach of arm towards a target object (transport component), the hand aperture is shaped to enclose the object (grasp
component) [2], [3]. The adjustment of gripper aperture,
which is scaled to size of the target object [24], is required
for successful prehension movement. Thus, it is inferred that
the arm and hand are directly involved in accomplishing a
task. The aperture has an identifiable maximum phase, and a
deceleration phase of the reaching movement in time [3].
In other words, there are opening and closure phases of
hand during the prehension. A number of studies have been
conducted to suggest a control method for the hand aperture
formation process in order to bring a computational approach
[14], [23], [24].
Considering the results and findings from the human, we
have modelled that the coordination of the robotic arm with
gripper is expressed with the remaining distance from the
target object and previous aperture in the hand, qG . The
variable is shown by qG = [q4 ], which is initially zero.
qG and Wo represent the instantaneous change in gripper
aperture and estimated object width, which is acquired from
the vision. y(qA ) and g are the current state of end - effector
the vector of the stationary goal coordinates, respectively. t
is given as time.
qG = 1t 2 ||y(qA ) g||2 + 2t(Wo q4 )
(9)
Positive 1 gain term implies that the gripper can open
wider than the object width, which is affected by the remaining distance to the target object. This effect is strongest
at the onset of movement. However, extreme increases in the
hand aperture are opposed by the 2 gain value, which drives
gripper aperture towards the target object width. The tradeoff between 1 and 2 generate a pattern of coordination.
Those are compensated by taking into consideration the
maximum grip aperture value derived for the human hand
during the reaching movement [21].
3) Control Model of Body Movement and Arm: The
recruitment of trunk to assist the arm in achieving a successful grasp adds significant complexity to synchronous
control since the arm movement is significantly affected
by the motion of body. For this reason, several decades
of researches have investigated how the involvement of
trunk motion during reach-to-grasp movements affect the
coordination between the transport of hand and grasping
components [17], [26], [27]. Due to coordination of trunk
with arm, the endpoint trajectory of the arm reaching toward
a target object becomes smooth [28].
III. SIMULATION RESULTS
IV. CONCLUSIONS AND DISCUSSION
APPENDIX
Appendixes should appear before the acknowledgment.
ACKNOWLEDGMENT
This project
R EFERENCES
[1] M. A. Arbib, Perceptual Structures and Distributed Motor Control,
In: Brooks VB (ed) Handbook of physiology, sect.1, vol. 2, part 2.
William and Wilkins, Baltimore, 1981, pp. 1449-1480
[2] M. Jeannerod, Intersegmental coordination during at natural visual
objects in Attention and Performance, I. X, eds J. Long and A.
Baddaley, 1981, pp. 153-168
[3] M. Jeannerod, The timing of natural prehension movements, J. Mot.
Behav., vol. 16, no.3, pp. 235-254, 1984
[4] L. S. Jakobson, and M. A. Goodale, Factors affecting higher-order
movement planning: A kinematic analysis of human prehension, Exp.
Brain Res., vol.86, pp. 199-208, 1991
[5] T. Flash, and N. Hogan, The Coordination of Arm Movements:
An Experimentally Confirmed Mathematical Model, The Journal of
Neuroscience, vol. 5, No. 7, pp. 1688-1703, 1985
[6] A. Hauck, M. Sorg, G. Farber, and T. Schenk, What Can Be Learned
from Human Reach-To-Grasp Movements for the Design of Robotic
Hand-Eye Systems?, ICRA, IEEE, 1999
[7] H. Na, E. S. Neo, and K. Yokoi, A Method Determining the Reaching
Behavior of Humanoid Robots, International Conference on Robotics
and Biomimetics, 2006
[8] B. Xie, J. Zhao and Y. Liu, Human-Like Motion Planning for
Robotic Arm System, The 15th International Conference on Advanced
Robotics, 2011
[9] T. M. Tuan, P. Soueres, M. Taix, M. N. Sreenivasa, and C. Halgand,
Humanoid human-like reaching control based on movement primitives,
19th IEEE International Symposium on Robot and Human Interactive
Communication, 2010
[10] S. Albrecht, K. Ramirez-Amaro, F. Ruiz-Ugalde, D. Weikersdorfer, M.
Leibold, M. Ulbrich, and M. Beetz, Imitating human reaching motions
using physically inspired optimization principles, 11th IEEE-RAS Int.
Conf. on Humanoid Robots, 2011, pp. 602607.
[11] A. dAvella, P. Saltiel, and E. Bizzi, Combinations of muscle synergies
in the construction of a natural motor behavior, Nature Neuroscience,
vol. 6, pp. 300-308, 2003
[12] A. dAvella, and F. Lacquaniti, Control of reaching movements by
muscle synergy combinations, Frontiers in Computational Neuroscience, vol. 7, p. 42, 2013
[13] S. R. Goodman, and G. L. Gottlieb, Analysis of kinematic invariances
of multijoint reaching movement, vol. 73, pp. 311-322, 1995
[14] P. Haggard, and A. Wing, Coordinated responses following mechanical
perturbation of the arm during prehension, Exp. Brain Res., vol. 102,
pp. 483-494, 1995
[15] J. Jacquier-Bret, N. Rezzoug, and P. Gorce, Adaptation of Joint
Flexibility During a Reach-to-Grasp Movement, The Journal of Motor
Control, vol. 13, pp. 342-361, 2009
[16] Jill C. S., [Link], and C. J. Winstein, Planning and adjustments
for the control of reach extent in a virtual environment, Journal of
NeuroEngineering and Rehabilitation, 2013
[17] R. D. Seidler, G. E. Stelmach, Trunk-Assisted Prehension: Specification of body segments with imposed temporal constaints, J. Mot.
Behav., vol. 32, no. 4, pp. 379-389, 2000
[18] R. G. Marteniuk, C. P. Bertram, Contributions of gait and trunk
movements to prehension: Perspectives from world-and body-centered
coordination, Mot. Control, vol.2, pp. 151-165, 2001
[19] J. Wang, G. E. Stelmach, Spatial and temporal control of trunk-assisted
prehensile actions, Exp Brain Res., vo. 136, pp. 231-240, 2001
[20] B. Hoff, and M. A. Arbib, Models of trajectory formation and temporal
interaction of reach and grasp, J. Mot. Behav. 25, pp. 175-192, 1993
[21] N. Kudoh, M. Hattori, N. Numata, and K. Maruyama, An analysis
of spatiotemporal variability during prehension movements: effects of
object size and distance, Exp. Brain Res. vol. 117, pp. 457-464, 1997
[22] S. Chieffi, and M. Gentilucci, Coordination between the transport and
the grasp components during prehension movements, Exp. Brain Res.,
pp. 471-477, 1993
[23] Quantitative model of transport-aperture coordination during reach-tograsp movements, Exp. Brain Res., vol. 188, pp. 263-274, 2008
[24] R. M. Bongers, F. T. J. M. Zaal, M. Jeannerod, Hand aperture patterns
in prehension, Human Movement Science, vol. 31, pp. 487-501, 2012
[25] N. Takemura, T. O. Fukui, and T. Inui, A Computational Model for
Aperture Control in Reach-to-Grasp Movement Based on Predictive
Variability, Front. Comput. Neurosci. 9, 2015
[26] R.G. Marteniuk, J. L. Leavitt, C. L. MacKenzie, and S. Athenes,
Functional relationships between grasp and transport components in a
prehension task, Hum. Mov. Sci., vol. 9, pp. 149-176, 1990
[27] J. Wang, G. E. Stelmach, Coordination among the body segments
during reach-to-grasp action involving the trunk, Exp. Brain Res., vol.
123, pp. 346-350, 1998
[28] M. K. Rand, A. W. A. V. Gemmert, A. B. M. I. Hossain, Y. P.
Shimansky, G. E. Stelmach, Control of aperture closure initiation
during trunk-assisted reach-to-grasp movements, Exp. Brain Res., vol.
219, pp. 293-304, 2012
[29] H. Poizner, A. G. Feldman, M. F. Levin, M. B. Berkinblit, W.
A. Hening, A. Patel, S. V. Adamovich, The timing of arm-trunk
coordination is deficient and vision-dependent in Parkinsons patients
during reaching movements, Exp. Brain Res., vol. 133, pp. 279-292,
2000