Vision-Based Control for Micro Aerial Vehicles
Vision-Based Control for Micro Aerial Vehicles
net/publication/336935818
CITATIONS READS
2 1,972
2 authors, including:
Henrik Hesse
University of Glasgow
54 PUBLICATIONS 671 CITATIONS
SEE PROFILE
All content following this page was uploaded by Henrik Hesse on 26 July 2021.
Abstract—This paper presents the use of an external vison-based instrument with gyroscope, accelerometer and magnetometer
positioning system for navigation and flight control of micro aerial and 1-DOF unit with a pressure sensor. The IMU therefore
vehicles. The motion capture system Optitrack is used to localize provides measurements of angular velocity, linear acceleration,
the vehicle which is a nano-quadcopter Crazyflie 2.0. An interface orientation and lastly altitude. However, the Crazyflie 2.0 has no
was created to pass positioning data to the Robot Operating 3D position measurements and the pressure sensor tends to drift
Software (ROS). ROS acts as communication layer to bridge providing inaccurate altitude measurements due to external
between Optitrack and available control nodes for Crazyflie 2.0 disturbances [1,6].
platforms.
I. INTRODUCTION
In the last decade, Micro Aerial Vehicles (MAVs) have been
used extensively for research and, recently, also find
applications in areas such as inspection surveillance [8,9,13]. To
achieve autonomous operation for such applications, we require
robust positioning and control performance of MAVs. To
Figure 1. The palm-sized nanoquadcopter, Crazyflie 2.0, attached with
localize the MAV and estimate its state, common sensor fusion reflective markers
approaches typically rely on inertial sensors and GPS to obtain
a position measurement. However, in GPS-denied environments
the vehicle has no sense of its position [18,19]. This lack of B. External Positioning System: Optitrack
position information naturally hinders the control performance In the aerial robotics research domain, quadcopters are
and is especially relevant in recent applications of MAVs for typically localized using the external positioning system VICON
inspection of enclosed areas. For such industry applications, we system [14,18,21]. Recent work [5], however, has demonstrated
typically require a reliable, inexpensive positioning system [9]. the use of the motion capturing system Optitrack as an
inexpensive solution to provide 6 DOF accuracy. Optitrack is
The choice of the MAV in this work is the open-sourced,
used to detect infrared light reflected from markers on the MAV.
open-hardware nano-quadcopter Crazyflie 2.0 as shown in
After calibration and so-called wanding [7], the system is able
Figure 1. The Robot Operating Software (ROS) is used as the
to stream data in real time to a 3D virtual reality graph.
communication layer to demonstrate the flexible and modular
code design for MAVs using ROS. To obtain external position
measurements we use the motion capture system Optitrack. C. Robot Operating System: ROS
ROS is a communication interface extensively used in the
The main objective of this work is to implement a ROS robotics community to program robots. Its strength is the
framework for the inexpensive localization and control of modular nature which allows code reuse and compatibility. It is
MAVs using vision-based position data from Optitrack. operated on an Ubuntu operating platform and features open
source libraries and tools for robotic applications. ROS nodes
A. MAV Platform: Crazyflie 2.0 are bits of codes C++ or Python to perform specific tasks. In
The research platform Crazyflie 2.0 is a nano-quadcopter Figure 2, ROS is illustrated as a communication layer between
which was developed by a team of software engineers at nodes, where nodes can publish and subscribe to topics. These
Bitcraze. It weighs 27g and can be easily flown indoors with the topics are messages sent to ROS and usually that subscribe to
dimensions of 92mm x 92mm x 29mm. these published topics can make use of this data [7,20].
The MAV comes equipped with a micro-processor which
reads measurements from a 10-degree-of-freedom (10-DOF)
inertial measurement unit (IMU). The IMU consists of a 9-DOF
∑ 𝐹⃗𝐵 = 𝑚𝑎⃗ = (𝑣⃗̇𝐵 + 𝜔
⃗⃗𝐵 × 𝑣⃗𝐵 ) = 𝐻𝐸𝐵 𝐹⃗gE + 𝐹⃗𝑎𝐵
II. MATHEMATICAL MODELLING where ℎ⃗⃗ is the total angular momentum around the center of
⃗⃗𝐵 the angular momentum with respect to the rotating
gravity, ℎ
A. Crazyflie Dynamics body frame 𝐵 and 𝑀 ⃗⃗⃗𝐵 the vector of resultant torques from the
The dynamics of the Crazyflie 2.0 quadcopter is modelled motors. With ℎ ⃗⃗𝐵 = 𝒥𝜔 ⃗⃗𝐵 , we can find the angular EoM as,
with respect to an inertial Earth frame as defined in Figure 3.
The body reference frame is also presented with the 𝑋, 𝑌 and 𝑍 ∑𝑀 ⃗⃗̇𝐵 + 𝜔
⃗⃗⃗𝐵 = 𝒥𝜔 ⃗⃗𝐵 × 𝒥𝜔
⃗⃗𝐵
axes and the Euler angles for roll, pitch and yaw denoted as 𝜑,
𝜃 and 𝜓 respectively [17]. Using these Euler angles, we can with the inertia matrix 𝒥 of the Crazyflie 2.0 assumed to be
derive a transformation between the quadcopter body-attached diagonal due to symmetry, such that
frame 𝐵 and the inertial frame 𝐸 as,
𝐼𝑥𝑥 0 0
C𝜃 C𝜓 − C𝜃 S𝜓 S𝜃
𝒥=[0 𝐼𝑦𝑦 0]
𝐻𝐵𝐸 = [ S𝜑 S𝜗 C𝜓 + C𝜑 S𝜓 C𝜑 C𝜓 + Sφ S𝜃 S𝜓 − S𝜑 C𝜃] (1)
− C𝜑 S𝜗 C𝜓 + S𝜑 S𝜓 S𝜑 C𝜓 + C𝜑 S𝜃 S𝜓 C𝜑 C𝜃 0 0 𝐼𝑧𝑧
where C is the cosine and S the sine operator. where 𝐼𝑥𝑥 = 1.395 × 10−5 𝑘𝑔𝑚2 , 𝐼𝑦𝑦 = 1.436 × 10−5 𝑘𝑔𝑚2
and 𝐼𝑧𝑧 = 2.173 × 10−5 𝑘𝑔𝑚2 [14].
The relationship between the translational and angular EoM
is given by the transformation 𝐻𝐸𝐵 defined in (1). The Euler
angles are propagated through the relationship between 𝜔⃗⃗𝐵 and
Euler rates,
𝑝 1 0 −𝑠𝑖𝑛𝜃 𝜙̇
[𝑞 ] = [ 0 𝑐𝑜𝑠𝜃 𝑠𝑖𝑛𝜙𝑐𝑜𝑠𝜃 ] [ 𝜃̇ ]
𝑟 0 −𝑠𝑖𝑛𝜙 𝑐𝑜𝑠𝜙𝑐𝑜𝑠𝜃 𝜓̇
which can be re-arranged to compute the Euler rates,
𝜙̇ 1 𝑠𝑖𝑛𝜙𝑡𝑎𝑛𝜃 𝑐𝑜𝑠𝜙 tan 𝜃 𝑝
𝜋
Figure 3. Coordinate system for Crazyflie 2.0 [ 𝜃̇ ] = [0 𝑐𝑜𝑠𝜙 −𝑠𝑖𝑛𝜙 ] [𝑞 ] for 𝜃 ≠
2
𝜓̇ 0 𝑠𝑖𝑛𝜙/𝑐𝑜𝑠𝜃 𝑐𝑜𝑠𝜙/𝑐𝑜𝑠𝜃 𝑟
The dynamic equations are derived next under the following
assumptions: where the angular velocities 𝜔⃗⃗𝐵 = [𝑝, 𝑞, 𝑟] are now the inputs
and can come from IMU measurements of the Crazyflie 2.0. To
1. The quadcopter is a rigid body that cannot deform.
avoid gimbal lock, pitching cannot reach 90 degrees (𝜃 ≠ 𝜋/2).
2. The quadcopter is symmetrical in terms of mass
Quaternions will be introduced in Section III.B. to resolve the
distribution and geometry.
gimbal lock.
3. The mass of the quadcopter is constant.
Based on Newton’s 2nd Law we can find the translational III. MOTION TRACKING WITH THE OPTITRACK SYSTEM
equations of motion (EoM) in the body frame 𝐵 as [22],
Figure 3 shows the Optitrack system with six Flex 13
cameras which can detect reflected infrared light from reflective
markers on the Crazyflie 2.0. With the six measurements the
Optitrack System can then compute the 3D coordinates of the
quadcopter with high precision and frequency. The software
Motive is used to process the 3D position data as the virtual
graph and must be calibrated through a process called wanding
[7].
Figure 5. Optitrack North-Up-East (left) versus and ROS East-North-Up B. Coordinate Axis Conversion for Orientation
(right) coordinate frame convention.
To transform the orientation of the Crazyflie 2.0 from
The six Flex 13 Cameras are connected to hub (Optihub 2) Optitrack to ROS frame conventions, we will use quaternions.
which captures the data and sends it to Motive. Motive then Quaternions are the generalization of complex numbers to 3D
computes the 3D virtual graph in real time from the and commonly used in robotics to describe rotations [17].
measurements of the reflective markers. This process is Following the Euler Theorem of Rotation, quaternions can be
illustrated in the top part of the schematic in Figure 6. introduced as the magnitude of rotation (the angle 𝛼) and a
vector as [17],
As shown in the bottom part of Figure 6, the data is then
𝛼 𝛼 𝛼 𝛼
streamed to a router which broadcasts the position data over to 𝑞𝑂𝑝𝑡𝑖 = [cos ( ) , 𝑛1 sin ( ) , 𝑛2 sin ( ) , 𝑛3 sin ( )] = [𝑤, [𝑥, 𝑦, 𝑧]]
ROS. Motive is Virtual Reality Peripheral Network (VRPN) 2 2 2 2
compatible which allows the data streaming to produce a report where the vector
identifiable as a movable tracker in ROS with position and 𝛼
orientation. However, since ROS standards follow a different 𝑛⃗⃗𝑂𝑝𝑡𝑖 = [ 𝑛1 , 𝑛2 𝑛3 ] = sin ( ) [𝑥, 𝑦, 𝑧]
2
axis coordinate system, East-North-Up (ENU) as illustrated in
Figure 5 (right), it is necessary to convert the position and defines axis of rotation. To convert the quaternion representation
orientation data. The VRPN node therefore has additional code in Optitrack, 𝑞 𝑂𝑝𝑡𝑖 , to ROS standards, we need to first isolate the
to transform between the Optitrack and ROS frame conventions angle part 𝛼 by taking inverse cosine of 𝑤 in 𝑞 𝑂𝑝𝑡𝑖 . With the
as explained next [7,12]. angle of rotation, 𝛼 , we can then compute the vector 𝑛⃗⃗𝑂𝑝𝑡𝑖 .
Finally, to convert from Optitrack to ROS standards we simply are subsequently processed by the onboard controller as
𝑅𝑂𝑆
need to apply the transformation 𝐻𝑂𝑝𝑡𝑖 to 𝑛⃗⃗𝑂𝑝𝑡𝑖 such that described next.
𝑅𝑂𝑆
𝑛⃗⃗𝑅𝑂𝑆 = 𝐻𝑂𝑝𝑡𝑖 ⋅ 𝑛⃗⃗𝑂𝑝𝑡𝑖 = [𝑁1 , 𝑁2 , 𝑁3 ]
with 𝑛⃗⃗𝑅𝑂𝑆 computed in ROS standards, we can then form the
new quaternion 𝑞 𝑅𝑂𝑆 as
𝛼 𝛼 𝛼 𝛼
𝑞 𝑅𝑂𝑆 = [cos ( ) , 𝑁1 sin ( ) , 𝑁2 sin ( ) , 𝑁3 sin ( )]
2 2 2 2
which describes the orientation of the Crazyflie 2.0 with respect
to the Earth frame 𝐸 in ROS-based (ENU) convention.
Figure 9. Generic flow of the Crazyflie 2.0 controller.
TABLE I. EXAMPLE OF QUATERNION TRANSFORMATION BETWEEN
OPTITRACK AND ROS CONVENTIONS FOR 𝛼 = 𝜋/2, 𝑛⃗⃗𝑂𝑝𝑡𝑖 = [ 0,1,2]. Figure 9 shows the generic flow of the software integration
Orientation 𝒘 𝒙 𝒚 𝒛 from a control perspective, i.e. the position and orientation data
is extracted from Optitrack which finally lead to control inputs
Optitrack -1 0 1 2 on the Crazyflie 2.0 itself. The boxes in green represent ROS
ROS -1 0 -2 1 computing, yellow represents Motive processing and orange are
operations executed directly onboard the Crazyflie 2.0.
Similar to the position transformation, the orientation
transformation effectively swaps the 𝑌 and 𝑍 coordinates and Based on the desired waypoints and current position data the
negates the 𝑍 coordinate as illustrated in Table I. offboard controller in ROS produces the velocity control inputs.
This input is then transmitted to the Crazyflie 2.0 using the
CrazyRadio and processed by the onboard controller [23].
IV. SOFTWARE INTEGRATION IN ROS FOR PATH CONTROL Finally, the onboard controller computes the motor inputs in the
OF THE CRAZYFLIE 2.0 form of Pulse-Width Modulation (PWM) signals for the four
Next, we will demonstrate the integration of the vision-based motors as [21],
navigation of the Crazyflie 2.0 in ROS for a simple waypoint 𝑃𝑊𝑀𝑚𝑜𝑡𝑜𝑟1 = 𝑇ℎ𝑟𝑢𝑠𝑡 − 𝜙/2 − 𝜃/2 − 𝜓
tracking control exercise. The software structure for the
controller in ROS is illustrated in Figure 8 where the red arrows 𝑃𝑊𝑀𝑚𝑜𝑡𝑜𝑟2 = 𝑇ℎ𝑟𝑢𝑠𝑡 + 𝜙/2 − 𝜃/2 + 𝜓
represent the topics that are subscribed by the controller and the
green-colored arrows are published by the controller. Note that 𝑃𝑊𝑀𝑚𝑜𝑡𝑜𝑟3 = 𝑇ℎ𝑟𝑢𝑠𝑡 − 𝜙/2 + 𝜃/2 + 𝜓
the software integration for the control of Crazyflie 2.0 has been 𝑃𝑊𝑀𝑚𝑜𝑡𝑜𝑟4 = 𝑇ℎ𝑟𝑢𝑠𝑡 + 𝜙/2 + 𝜃/2 − 𝜓
adapted from [12].
V. EXPERIMENTS & TEST RESULTS
The following experimental results demonstrate the software
integration of the Optitrack system with ROS, including the
transformation of position and orientation data for the first test.
The following tests show hover and waypoint tracking to
demonstrate the ROS integration for autonomous flight of the
Crazyflie 2.0.
Figure 12. 3D trajectory (blue) for waypoint tracking experiment with circles
identifying the waypoints.
Figure 11 shows the corresponding time history of the Figure 13. Time history of 3D position components for waypoint tracking
Crazyflie 2.0 during hover. The measured 𝑍 data shows that the experiment. Current waypoint in circles correspond to definition in Figure 12.
quadcopter accelerates first up to 1.2 m before leveling at 0.8m.
The oscillations in the 𝑍 direction in Figure 13 arise as the acknowledge the technical advice from Paul Beuchat at ETH
Crazyflie 2.0 is not commanded to follow a specific trajectory Zurich.
but to reach the next waypoint. To accelerate in the 𝑋 and 𝑌
directions, the onboard controller tends to also apply excess REFERENCES
thrust in the 𝑍 direction. [1] J. Foerster, “System Identification of the Crazyflie 2.0 Nano
Quadrocopter”, Bachelor Thesis, Swiss Federal Institute of Technology
(ETH) Zürich, Switzerland.
[2] M. W. Mueller and R. D’Andrea, "A model predictive controller for
quadrocopter state interception", in European Control Conference (ECC),
Zürich, Switzerland, 2013, pp. 1383-1389
[3] Landry, B. , “Planning and control for quadrotor flight through cluttered
environments”, Master’s Thesis, Massachusetts Institute of Technology,
USA, 2015.
[4] M. Binder, L. Bieri, M. Meier and N. Melchior, "Indoor Localization
Using Ultra-Wide-Band", Master’s Thesis, ETH Swiss Federal Institute
of Technology Zurich, Zurich, Switzerland, 2017.
[5] C. Hansen, D. Gibas, J. Honeine, N. Rezzoug, P. Gorce, B. Isableu, "An
inexpensive solution for motion analysis”, Journal of Sports Engineering
and Technology. Compiegne, France, 2014.
[6] L. Edwards, “Open source robotics and process control cookbook
designing and building robust dependable real time system,” Elsevier,
New York, USA, 2011.
[7] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger,
R Wheeler, A. Ng “ROS: an open-source Robot Operating System,”
Figure 14. 2D projection of the trajectory in Figure 12. ICRA Workshop on Open Source Software, Kobe, Japan, 2009.
[8] D. WONG, "Plan for drones to deliver parcels takes flight", The Straits
To demonstrate the waypoint tracking capabilities, Figure 14 Times, Feb 9, 2018.
shows the projection of the Crazyflie 2.0 trajectory on the 𝑋 − [9] G. Warwick, "Aircraft Inspection Drones Entering Service With Airline
𝑌 plane only. Figure 14 highlights that the controller can MROs", MRO network, Apr 04, 2018.
successfully command the Crazyflie 2.0 in dynamic situations [10] "Cutting-edge drones and unmanned vehicles to boost Singapore's
with only minor overshoots. The waypoint tracking can be surveillance capabilities", Channel NewsAsia, 2018.
further extended to trajectory control by discretizing the [11] Evan G. Hemingway “Perspectives on Euler Angle Singularities, Gimbal
trajectory with multiple waypoints and considering the vehicle Lock, and the Orthogonality of Applied Forces and Applied
Moments.”Article, University of California. Berkeley, CA, USA, 2017.
dynamics in the discretization.
[12] W. Hoenig, C. Milanes, L. Scaria, T. Phan, M. Bolas, N. Ayanian “Mixed
Reality for Robotics”, IEEE Intl Conf. on Intelligent Robots and Systems,
VI. CONCLUSION Hamburg, Germany. 2015.
[13] K. Kaur, The Straits Times, "Five projects to kick off unmanned aircraft
The objective of this work was to use the motion capture system trials at Singapore's first drone estate", 2018.
system Optitrack as an external localization system for
[14] C. Luis, J. Le Ny, "Design of a Trajectory Tracking Controller for a
autonomous flight control of the nano-quadcopter Crazyflie 2.0. Nanoquadcopter" Technical Report, Polytechnique Montreal. Montreal,
This paper provides a comprehensive documentation on how to Canada, 2016.
integrate the Optitrack system with ROS and it especially [15] J. Diebel, “Representing Attitude: Euler Angles, Unit Quaternions, and
addresses the transformation between coordinate conventions in Rotation Vectors”, Stanford University. Stanford, California, USA, 2006
ROS and Optitrack. This enables the use of the Optitrack [16] J. A. Preiss, W. Hoenig, G. S. Sukhatme and N. Ayanian “Crazyswarm:
localization system, which is more affordable than the standard A Large Nano-Quadcopter Swarm”, IEEE International Conference on
VICON positioning system, for educational purposes. Robotics and Automation (ICRA), Singapore, 2017.
[17] A. Hanson, “Visualizing quaternions,” Morgan Kaufmann, San
The integration of Optitrack with the communication Francisco, CA, USA, 2006, pp. 43-56.
platform ROS further makes for a versatile platform which can [18] J. Bjorn, M. Kjaegaard, J. A. Larsen “Autonomous Hover for a quad rotor
rely on the modular and open-source nature of ROS for a wide helicopter”, Aalborg University, master’s Thesis, Aalborg, Denmark,
range of robot applications. In this work, we have demonstrated 2007.
this capability by using open-source controller nodes to enable [19] M. Wierema, “Design, implementation and flight test of indoor navigation
and control system for a quadrotor UAV” TU Delft, Master of Science
waypoint tracking of the Crazyflie 2.0 nano-copter. This simple Thesis, Delft, Netherlands, 2008.
integration further highlights the benefits of ROS in combination
[20] J. Mišeikis, “INF3480 - Introduction to Robot Operating System,”
with Optitrack as an education platform in aerial robotics. Lecture Slides, University of Oslo, Norway, 2017.
[21] G.P. Subramanian, “Nonlinear control strategies for quadrotors and
ACKNOWNLEDGEMENT CubeSats”, Master Thesis, University of Illionis, USA, 2015.
[22] D. Mellinger, Q. Lindsey, M. Shomin, and V. Kumar, “Design, modeling,
The authors would like to thank Dr. Spot Srigrarom from the estimation and control for aerial grasping and manipulation,” IEEE Intl
University of Glasgow Singapore to provide access to the Conf. on Intelligent Robots and Systems, San Francisco, CA, USA, 2011.
Optitrack UAV facility. The authors would also like to