0% found this document useful (0 votes)
19 views8 pages

Autonomous Aerial Drone With Infrared Depth Tracking

G18_conference paper

Uploaded by

Harry moes
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
19 views8 pages

Autonomous Aerial Drone With Infrared Depth Tracking

G18_conference paper

Uploaded by

Harry moes
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

After the pandemic broke out and it was determined the

drone competition was not going to occur as planned, the


Autonomous Aerial Drone with project requirements were reduced to demonstrating
autonomous around two of the obstacles: a ring and a
Infrared Depth Tracking pylon. This report details our progress to accomplishing
these requirements.
Rishi Jain, Caleb Jones, Ryan Lucas, and Hamza
Siddiqui II. HARDWARE COMPONENTS
DEPT. OF ELECTRICAL AND COMPUTER The completed drone project consists of various
ENGINEERING, UNIVERSITY OF individual components used to realize the finished
CENTRAL FLORIDA, ORLANDO, product. This section provides an overview on the major
FLORIDA, 32816-2450 components selected, including a brief analysis on the
decision making process.
A. Flight Controller
ABSTRACT — Autonomous drones have broad applications
The flight controller is responsible for controlling the
including but not limited to: defense, search and rescue,
firefighting, and package delivery. Despite the different movement of the drone by adjusting the power delivered
applications that these drones may perform, they have to each motor. Flight controllers can measure the level and
similar underlying components and software architectures. speed of the drone, and use that information to correct the
This paper provides an overview on the processes and orientation of the drone during flight. The selected flight
methodologies used to implement an autonomous aerial controller for this project is the Readytosky Pixhawk. The
drone, also known as a UAV, for our senior design project. reason why this flight controller was selected was because
This project identifies known obstacles and attempts to in addition to the basic gyroscope and accelerometer
maneuver around them in specific patterns. Moreover, the sensors that all flight controllers have, this one also
project incorporates sound localization in order to identify
includes a barometer, magnetometer, and inertial
the approximate location of an audio source.
measurement unit (IMU) which provides more accurate
Index Terms — Aircraft Navigation, Autonomous Systems,
and consistent flight performance. Additionally, the
Image Recognition, Machine Learning, Sensor Systems.
Pixhawk is 32-bit, has I2C and SPI interfaces, and has the
ability to run either the PX4 or ArduPilot flight stack. We
I. INTRODUCTION utilized the expandability of this platform to test the drone
using both a PX4Flow Optical Flow Sensor and a
The purpose of this project is to create an autonomous HereFlow Optical Flow/Lidar sensor.
aerial drone in a multidisciplinary team composed of
electrical/computer engineering students as well as B. Camera
mechanical/aerospace engineering students. The original
For our camera, we were looking at a few different
objectives of this project were provided by our project
options. One option was to use a single, basic RGB
sponsor, Lockheed Martin. The main objective was to
camera. This would provide us video data from which we
create an autonomous drone to navigate through an
could feed into an object detection algorithm in which to
obstacle course consisting of a ring, a pylon, a double-
determine the location of the objects. However, we
pylon, and an acoustic waypoint.. This obstacle course
determined with this kind of set up it would be difficult to
would have been set up indoors at Lockheed’s indoor lab.
determine the distance to most objects. A solution that we
Our team’s drone’s would have competed against other
came up with was to use two RGB cameras and use the
drones designed by other teams to determine which drone
relative difference in pixels between the two images to
could navigate the most obstacles autonomously through
determine the distance to objects on the computer. The
the course. In addition to navigating the obstacles, the
downside to this approach is the large amount of
drone was also to avoid being brought down by a drone
processing power it would take to determine distances on
“mine” designed by another team to stop the drone from
a computer that is already running an computationally
completing the course. The demonstration was initially
expensive object detection algorithm.
planned to be a competition against the other drone teams,
Therefore, instead of using two RGB cameras, we
with the goal of accumulating the most points. As a part of
decided to use an Intel Realsense D435 depth camera.
this challenge, we were allocated a strict budget of $1100
This camera uses two infrared cameras to produce a depth
for the final build price of our drone and an additional
image that is calculated on a processor located in the
$550 for prototyping.
camera. The depth image along with an RGB image from
an RGB camera also located on the D435 is sent to the
drone computer via USB. The drone computer then uses
the RGB image for object recognition while the depth
image was intended to be used for determining the
distance to the object. Figure 1 shows a 3D mapped image
processed by the camera.

Figure 2: Microphone array acoustic echo cancelation


D. Companion Computer
The computer of the drone is used to control the main
functions of the drone. These functions include image
recognition on frames received from the camera, sending
commands to the flight controller, and sending video to
the ground station. When choosing a computer to integrate
into our system, we were looking for a computer that was
Figure 1: 3D mapped image of pylons and rings viewed powerful enough to run all of these systems
from the positive z-axis simultaneously in addition to being relatively light and
C. Microphone consuming relatively little power.
At first we considered using a Raspberry Pi 4 Model B.
To detect the acoustic waypoint we utilized the Mic This computer has a 4-core 1.5 GHz processor with 4GB
Array v2.0, which uses four ST MP34DT01TR-M digital of memory. It is very light at 46 grams and uses up to 5W
microphones to triangulate sound and filter out different at max power. However, this computer had the downsides
sounds into up to six channels. Sound triangulation is done of having a GPU that is not well documented, likely not
with time of arrival for a sound to each of the four very powerful, and unable to take advantage of specific
microphones, resulting in an angle which can be used to libraries designed for object detection models.
determine the location of the acoustic waypoint. Volume We instead decided to use a Nvidia Jetson Nano Dev
intensity can be used to determine the distance from the Kit. This computer has a 4-core 1.42GHz processor with
drone to the acoustic waypoint. 4GB of memory. It is slightly heavier at 116 grams and
Sound filtering is done with the onboard XMOS XVF- uses up to 10W at max power, but it uses an Nvidia
3000 chip, with the use of Acoustic Echo Cancelation CUDA core GPU that is more broadly compatible with
(AEC) which is designed to remove echoes, reverberation, software that can run object detection algorithms. We
and unwanted added. AEC utilizes an adaptive filter believed this combination of CPU and GPU would
which samples the direct audio with the echoed audio to provide enough computational power to run the object
result in one clean audio reading, which is then further detection algorithm along with any other software we
filtered with residual echo suppression and noise reduction were running. We would also be able to experiment with
(Figure 2). Nvidia specific libraries for deep learning and computer
The onboard XMOS XVF-3000 chip also has a vision. For this reason, we selected the Jetson Nano over
configuration file which allows for adjusted parameters the Raspberry Pi 4B.
for how many channels that can be filtered, as well as the
frequency range that will be recorded by the microphones. E. Microcontroller
These parameters can be adjusted because the Mic Array A microcontroller was selected to manage four laterally
v2.0 utilizes digital microphones which are compatible mounted ultrasonic range sensors, the HC-SR04. Utilizing
with a more advanced stereo sound processor codec such a separate microcontroller (as opposed to incorporating
as the XMOS. this subsystem with the companion computer) allowed us
to increase the modularity of the system. This highly
modular design allowed us to increase the reliability of the
subsystem, and allowed us to easily examine various
design iterations. The chosen architecture was based on
the Atmel ATMEGA328-PU processor which utilizes an
8-bit 20 MHz design. This integrated circuit (IC) is the battery was too low to provide adequate power of the
sufficient in driving the ultrasonic range sensors and drone’s systems.
relaying distance information to the companion computer
The “Tattu” was a 10000 mAh, 22.2 volt, 6 cell, 1400g
via UART. Finally, this architecture was chosen due to its
battery; this battery’s higher capacity and output voltage
available documentation/online resources, as well as ease
were good specifications to consider. However, the total
of use via the Arduino development platform.
weight of the battery was too high to provide adequate
F. Wi-Fi Module thrust for the drone’s takeoff of maneuverability.
The drone computer requires Wi-Fi capability in order H. Universal Battery Eliminator Circuit
to communicate with a ground control computer to send
To power the Jetson Nano using power from the drone’s
video data. The Jetson Nano does not come with a Wi-Fi
power distribution board, the ZTW 6Amp universal
module built in, so we decided to buy a Geekworm USB
battery eliminator circuit (UBEC) was used. The Jetson
Wi-Fi adapter. We decided to go with a USB Wi-Fi
Nano is rated to take a DC power supply of 5V and
module rather than a Wi-Fi card because a Wi-Fi card
4Amps, and despite providing 5V and 5.5Amps via the
would have required installation of a separate antenna in
ZTW 6Amp UBEC, it did not provide an adequate amount
order to operate properly. The USB module came with a
of power when the Jetson was running the computer
small antenna and allowed for quicker integration with the
vision algorithm, resulting in a shutdown of the CPU.
rest of the system.
The Geekworm USB Wi-Fi module was specifically III. System Overview
chosen because it uses USB 3.0 which is on the Jetson
This section will outline the physical connectivity and
Nano, allowing for data speed of up to 4,800 Mbps, which
operation of the complete system.
is ten times faster than USB 2.0 ports. A USB Wi-Fi
module with a larger antenna allows for a larger reception A. Architecture
range, and the Geekworm USB Wi-Fi module uses a 5dBi
RP-SMA which has an effective range of up to 1000 feet. Figure 3 below provides a high level overview of how
the components are connected in this project. In the figure,
G. Battery it can be seen the battery is connected to PDB, which is
connected to the ESC and motor set, the flight controller,
A Lithium Polymer (LiPo) battery will be used as the
the CPU, the camera, the microphone, and the
power source for the drone. It will be connected to the
microcontroller and sensor set.
power distribution board (PDB), which will then distribute
power to the rest of the components. The power
requirements of all the components of the drone’s system
is listed in Table 1..
Table 1: Expected Power Draw of System Components

Figure 3: System component connection overview


The battery used was a “Venom Fly” 14.8 volt, 3200
mAh, 4 cell, 330g LiPo battery. The battery was chosen As seen above, the drone has two primary systems: the
based on two primary specifications; its total output flight controller and the Jetson Nano (companion
voltage and weight. computer). The flight controller manages all the flight
controls of the drone, while the companion computer
The “Dynamite Reaction” and the Tattu batteries were manages all the image and sound recognition, processing,
taken into consideration. The “Dynamite Reaction” was a and navigation instructions of the drone. The companion
5000 mAh, 11.1 volt, 3 cell, 204g battery; this battery’s computer connects to the flight controller in place of a
higher capacity and lighter weight were good radio receiver, mimicking the signals used to fly a drone.
specifications to consider. However, the output voltage of It also provides the communication interface to control the
drone remotely. As the Jetson Nano is able to connect to a for its speed. It would allow us to perform path corrections
Wi-Fi network through the wireless adapter, a user is able at a rate at at least 30 frames per second.
to SSH into the drone and execute flight commands. This model was trained using one of our personal
Moreover, the user is able to launch the video feed and computers with a GTX 980ti GPU using Tensorflow.
stream it over the network to a remote ground control Tensorflow was chosen due to its theoretical compatibility
station (in this case, a laptop). with the Jetson hardware. The images used for training
were collected by us during our test sessions in the
B. Operation
Lockheed Martin drone lab and outside, and these images
Connecting the battery to the drone powers all of the were labeled using labelImg [1]. The configuration of our
system components immediately, however the computer training was based on that Tensorflow researchers used for
takes about 45 seconds to boot up. Once the computer has training an SSD for the COCO dataset [2].
booted, and the Jetson Nano connects to the known access Once the model was trained, the frozen model was
point, a user can remotely SSH into the computer over the transferred to the Jetson where the model was converted
same network. Once logged in via SSH, the user can into a UFF file, a form usable by TensorRT. TensorRT
launch the software for autonomous functionality. To was then used to optimize the model for use with the GPU
manually fly the drone, the remote control receiver must located on the Jetson Nano. This process was difficult due
be inserted into the flight controller, replacing the Jetson to issues in compatibility between particular versions of
Nano. Tensorflow and TensorRT. In this case, Tensorflow 14.0
was creating nodes in the graph that were unable to be
properly interpreted by our version of TensorRT. We were
IV. SOFTWARE DESIGN AND INTEGRATION
able to resolve these issues after finding a source detailing
The software for the project heavily revolves around the how these issues could be resolved [3].
Robot Operating System (ROS). Figure 4 (below) outlines B. Robot Operating System
how the various software nodes interact with one another.
The Robot Operating System (ROS) is a set of message
passing protocols typically used with autonomous
systems. ROS allows for integration of different systems
by standardizing messages including the passing of
images from the camera node to the image processing
node. Nodes can be established for various functions such
as packaging sensor data and device control instructions.
We decided to go with ROS because of the large
amount of resources that we could find a large amount of
software that would allow for the quick integration of
various components of our system.
C. Controller Node
The controller node is a ROS node designed to receive
information from the object detection node and use that
information to send commands to the flight controller via
MAVROS. The controller node contains a control loop
that switches between a set of modes. The main two
modes are Autonomous Navigation (AutoNav) and
Figure 4: Software system diagram using ROS Autonomous Maneuver (AutoMan).
A. Object Detection Model and Training While in AutoNav mode, the drone would attempt to
navigate towards a spot in front of the obstacle. The drone
Object detection is used in our system to determine the does this using object data from the object detection node.
location of obstacles compared to the drone. The decision Once the drone has navigated to a spot in front of the
of the type of model mainly focused on speed as extreme obstacle, the drone then switches into AutoMan mode.
accuracy is not necessary to navigate through hoops and The drone then would attempt to navigate around the
around pylons. Using data for models running on the obstacle. In the case of rings, the drone would attempt to
Jetson Nano [4], we decided to go with a Single Shot fly through them. In the case of pylons, the drone would
Detector (SSD) using a Mobilenet V2 to identify objects make a loop around them. Once the drone has finished
AutoMan mode, the drone would be put into AutoNav V. DRONE DESIGN
mode again to look for another obstacle.
The drone was customly designed by the
The drone would be able to be switched out of AutoNav
mechanical/aerospace engineering members of our team
and AutoMan modes by flicking a switch on the radio
with the purpose of creating a highly modular design. The
controller to throw the flight controller into a human
benefit to creating our own frame is that it can evolve to
operated mode. The drone computer will then notice the
meet the changing needs of our project. Designing our
change in mode and stop sending commands to the flight
own drone was also cost effective and allowed the team to
controller.
utilize the project budget on better components. In
D. Camera Node particular, the drone was 3D printed using PET-G with
custom mounts to securely house our components. We
The camera node we are using to interface with the Intel
were able to quickly reprint parts of the drone if we
Realsense D435 is a node called realsense-ros created by
needed to accomodate a new component or replace any
Intel [5]. This node provides an RGB image and a depth
broken parts. A model of drone is shown below in Figure
image to the other nodes.
5:
E. Object Detection Node
The object detection ROS node is created by Nvidia
employees called ros_deep_learning [5] that was edited by
us in order to accept an RGB image from the camera as
opposed to the BGR image that the program expects. The
object detection node subscribes to the RGB image output
by camera node. The node then takes our custom model in
UFF format, converts it to a form to be used by TensorRT
(if it hasn’t already from a previous run), and then uses
that model to output bounding boxes of obstacles. These
bounding boxes are labeled such that we can determine
what kind of obstacle is detected. These bounding boxes
would have been passed to the distance estimation node Figure 5: Custom design drone frame with components
for use in determining distances to each object. mounted
F. Distance Estimation Node
VI. RESULTS
The distance estimation ROS node would take an input
of a depth image from the camera node and the bounding Due to the pandemic canceling the trial runs at
boxes of objects from the object detection node to Lockheed Martin’s drone facility, the school being shut
estimate the distance to each obstacle in the field of view down and denying access to the senior design lab, and
of the camera. The closest obstacle to the drone is then generally being unable to congregate with the entire team
determined, and the bounding box for that obstacle and due to stay-at-home orders, the ability to complete this
distance to that obstacle would then have been passed to project as originally planned was severely impacted. We
the drone controller node for navigation. did our best to pull together what we could in these
circumstances.
G. Microcontroller Software
A. Object Detection
The microcontroller software was written in embedded
C, as available in the Arduino Integrated Development We were able to train an object recognition model and
Environment. The microcontroller transmits sensor successfully execute it on the drone’s onboard computer.
information over the Universal Asynchronous Receiver- We created and utilized a database of approximately eight
Transmitter (UART) interface. On the other side of the hundred images to create our model. We were able to
communication link, a ROS node on the companion correctly identify hoops and pylons. An example of this
computer can read the serial communication and publish can be seen in Figure 6. As part of the project
the data as necessary. requirements, the object recognition system should be able
to identify the type of obstacle detected, and the
confidence level of the object detected, amongst other
requirements. In figure 6, it can be seen the object
recognition model was able to detect two types of
obstacles; a ring and a single pylon. The ring was detected
with a confidence level of 83 percent, and the pylon was
detected with a confidence level of 75.7 percent.

Figure 6: Object detection with confidence level


In addition to the visual identification, we were able to Figure 7: Angles of detected sound samples (with respect
extract numerical coordinates for the identified object with to a 0° starting position)
respect to the rest of the captured frame. C. Printed Circuit Board & Microcontroller
B. Sound Triangulation We were able to design and prototype the PCB that
Using the previously mentioned microphone array, we would be able to manage the ultrasonic range sensors used
were able to identify the direction that a sound was for the measurements between the drone and potential
playing from. Primarily, we were able to extract the angles obstacles. It is based on the Atmel ATMEGA328-PU
of a sound source in reference to the zero position of the integrated circuit and an Arduino Uno was used as the
microphone. With this information, we could determine development board. While we originally wanted to
which direction the drone needed to navigate to in order to incorporate a DC/DC power converter to serve as a
complete the acoustic waypoint obstacle as part of the voltage regulator for the various components in our
competition. An example of the microphone data is shown system, the electronic speed controllers and motors draw
in Figure 7. In the figure, the angle of the sound source upwards of 30A each. We were advised to instead use a
from the drone’s current location can be seen. The angle is power distribution board designed for drone applications,
measured counterclockwise, taken from the horizontal which we then utilized to power our components. The
front end of the drone. schematic for our circuit is shown in Figure 8.
The sound that the microphones were set to listen to any
sound above 180Hz. The XMOS could limit its sound
cutoff range to 80Hz, 125Hz, and 180Hz. This was the
closest number to Lockheed’s specification of close to
500Hz for the sound emitted from the acoustic waypoint.
The 180Hz limit was enough to permit sounds from a high
frequency hand bell, and exclude lower frequency sounds
such as voices when testing for the direction of arrival for
high frequency sounds. The frequency of the sound from
the drone’s propellers to where the mic array was placed
underneath the drone, was below the 180Hz threshold.
Figure 8: PCB schematic utilizing the Atmel
ATMEGA328-PU

We tested the sensor measurements and verified the


ability to communicate data between the microcontroller
and Jetson Nano over UART to exchange data, as seen in
Figure 9.
Though we were able to establish a proof of concept for
this subsystem, we experienced unreliable performance
when using the fabricated PCB that we designed. We regulators that we used either weren’t properly rated or
believe that due to tracing errors, some signals were were not stable enough when a large amount of power is
interfering with each other and causing poor performance suddenly needed. If we had additional time, we would
in relaying information back to the drone’s companion have ordered or designed another regulator that hopefully
computer. We would be able to confirm this hypothesis would provide enough current fast enough.
with the use of the university’s lab equipment.
E. Distance Estimation
The initial plan for our distance estimation node was to
use the bounding box from the vision algorithm and line
that up with the depth image to determine the distance to
each obstacle. However, due to time constraints that
occurred partially due to the pandemic, we were unable to
implement this node. Instead, we used object detection to
determine if the drone was close enough to the obstacle to
perform a maneuver. This was able to be done because
instead of the large number of obstacles that could be in
our field of view at the same time in the originally planned
obstacle course, the drone only ever needed to navigate a
single obstacle at a time.
Figure 9: UART transmission of ultrasonic sensor data
F. Manual and Autonomous Flight
(measured in centimeters)
We were able to control the drone manually via an RC
D. Power System
controller and achieve stable flight. With all the
All of the drone’s components were powered through components attached to the drone, the drone had no
the onboard battery. The drone was originally designed to problem sustaining lift.
fly for at least 10 minutes on a full charge of a battery. Before implementing autonomous flight, we needed to
However, during testing we discovered that our battery ensure that the flight controller was able to hold the drone
was only able to provide about 6 minutes of flight with all at a constant height and position. This position hold mode
components attached. After discussions with our team, we is crucial to autonomous flight as it holds the drone in a
discovered that the power draw from our motors was stationary position in a 3D plane. We first attempted to
greater than that was what was expected. We decided to utilize a PX4Flow Optical Flow sensor to implement this
upgrade our battery to a 6000mAh, which is almost double feature. The optical flow camera would send data to the
what we had before. We also decided to upgrade to larger flight controller about how much the pixels in the frame of
motors because the additional battery weight would have the camera have shifted. However, we found that the
put significant strain on our current motors. However, we drone would drift in the air and occasionally drop out of
were unable to order these batteries due to the lack of the sky as it poorly attempted to correct itself. After doing
access to sponsor funds because of the pandemic. considerable testing, we found that the PX4Flow was
During our testing with our drone computer connected providing severely incorrect data to the flight controller,
to the battery, we noticed that we were having issues with which we believed to be causing our issues. We believed
the computer turning off automatically. When the drone initially that the issue could be due to the fact that this was
was idle, we found that each component was stable and not a new PX4Flow and was instead used in a previous
remained powered on. However, when we ran the drone project.
computer vision algorithm, the CPU load would spike and Next, we replaced the PX4Flow sensor with a HereFlow
the computer would shut off shortly thereafter. The power Optical Flow/Lidar sensor. Sensor was designed to
to the computer was being routed from the computer using provide the same basic functionality as the PX4Flow, but
a regulator rated for 5V at 6A. Since the computer is instead of a sonar sensor attached to determine height, a
designed to run at 5V and pull at max 20W with all of the LiDAR sensor was used. Through testing, we found that
peripherals attached, we believe that the reason the the HereFlow appeared to give more accurate data than the
computer kept shutting off was that the regulator that we PX4FLow, but the flight controller was still unable to
bought wasn’t able to provide the necessary amperage properly hold position.
quickly enough when the computationally intensive object We believe there are two potential reasons for why this
recognition started. The use of lab equipment such as an problem continues to occur. First, the drone was primarily
oscilloscope would have assisted us in helping to diagnose being flown over the grass. It is possible that the grass that
this as the certain cause. As it is, it seems to us that the we were flying the drone above was simply too noisy to
accurately track movement using the camera. Our project VIII. CONCLUSION
was originally intended to be flown indoors, and it is
This project was able to demonstrate a competent
possible that a more consistent floor pattern may have
design for an autonomous drone. It produced a successful
improved the performance of position hold. The second
custom object detection model to identify the provided
could be an issue with the accuracy of the height sensor.
obstacles. Although we were severely hindered in our
The height sensor is essential to determining how much a
ability to deliver a finished product due to the global
drone needs to adjust compared as pixels will appear to
pandemic, this project laid the framework to incorporate
move faster when close to ground while the pixels appear
environmental information in an autonomous device.
to move slower when farther from the ground even when
moving at the same speed. The LiDAR found on the
HereFlow likely wasn’t working as well outside in the ACKNOWLEDGEMENT
open field as it was when we were testing it indoors. We
We would like to thank Lockheed Martin in Orlando,
initially believed that the degradation in performance
Florida for sponsoring this project. In particular, we would
wouldn’t be significant enough at the heights we were
like to thank Aaron Phu from Lockheed Martin and
flying at, but it may have been enough to cause issues.
George Loubimov from the Aerospace Engineering
Although we had enough money remaining in our budget
department at the university for their time and guidance.
to order a replacement flight controller, the pandemic
We would also like to thank our ECE senior design
made it impossible to order and receive parts during the
professor, Dr. Richie.
second half of the semester.
Without the ability for a drone to properly stabilize in
the air, we felt uncomfortable running an autonomous REFERENCES
flight algorithm. The autonomous algorithm that we [1] Tzutalin. LabelImg. Git code (2015).
intended to employ relied on fairly consistent stabilization [Link]
by the drone to ensure that the drone would not drift [2] "Speed/accuracy trade-offs for modern convolutional object
significantly while carrying out movement commands detectors." Huang J, Rathod V, Sun C, Zhu M, Korattikara
based on the local coordinate system of the flight A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S,
controller. Without the assurance of stable flight, flying Murphy K, CVPR 2017.
via our autonomous algorithm posed both a risk to [Link]
ourselves and to the hardware as a whole. Additionally, object_detection/samples/configs/ssd_mobilenet_v2_coco.c
onfig
our drone power system was unable to support the power
[3] Bédorf, Jeroen. “Deploying SSD mobileNet V2 on the
requirements of the Jetson Nano Dev Board, making NVIDIA Jetson and Nano platforms.” Dec 13, 2019.
testing navigation difficult while performing flight. [Link]
the-nvidia-jetson-and-nano-platforms
VII. CONSIDERATIONS FOR THE FUTURE [4] “Jetson Nano: Deep Learning Inference Benchmarks.”
Nvidia. [Link]
A major realization that the team had when working on nano-dl-inference-benchmark
this project was how much power the Jetson Nano seemed [5] “ROS Wrapper for Intel® RealSense™ Devices.”
to consume. This not only reduced the operating time of IntelRealSense. Git code (2020). Licensed under Apache
the drone, but also gave us unreliable performance when License V2.0 ([Link]
CPU utilization spiked. The Raspberry Pi 4 Model B 2.0). [Link]
weights approximately 100 grams less by itself, does not [6] Franklin, Dustin. “ros_deep_learning.” Git code
require an external Wi-Fi dongle, and most importantly, (2020). [Link]
has a power consumption of 5W (opposed to 10W on the nv/ros_deep_learning
Jetson Nano). We believe that the Raspberry Pi 4B would
have sufficiently been able to run our object detection
model, while reducing power draw and saving weight on
the drone. Having the ability to use lab equipment to
analyze the power draw would have provided insights as
to why our power delivery was unsustainable. This may
also allow us to sustain a minimum 10 minute operating
time instead of the 6-7 minutes we estimated with the
selected battery.

You might also like