See discussions, stats, and author profiles for this publication at: [Link]

net/publication/4136714

Agent-based mission management for a UAV

Conference Paper · January 2005

DOI: 10.1109/ISSNIP.2004.1417508 · Source: IEEE Xplore

CITATIONS READS

19 611

3 authors, including:

Clinton Heinze
Australian High Commission, London, United Kingdom
37 PUBLICATIONS 611 CITATIONS

SEE PROFILE

All content following this page was uploaded by Clinton Heinze on 02 June 2014.

The user has requested enhancement of the downloaded file.

 Agent-based mission management for a UAV
#
Samin Karim 1 , Clint Heinze 2 , Shane Dunn 2
1
Information Systems, University of Melbourne
Melbourne, Australia, mkarim@[Link]

2
Defence Science and Technology Organisation
Melbourne, Australia, {[Link],[Link]}@[Link]

Abstract issues high level commands that are executed by the Flight
Control System (FCS). The system was developed in the
This article reports on the design, implementation and testing
agent-oriented paradigm. Agents are the central building block
of a mission management system for a small, electric powered
of agent-based systems. Akin to objects in the object-oriented
UAV - the Codarra Avatar. The system was designed and
software engineering paradigm, they are generally described
implemented using the agent-programming paradigm, which
as computer systems with two important and distinguishing
is a powerful, scalable and flexible framework for building
capabilities [18], [17]. Firstly, they are capable of fully au-
autonomous systems. Specifically, we have used the JACK
tonomous behaviour, which entails independent reasoning,
Intelligent Agents programming language. The system was
decision making and action in order to satisfy the agents’
integrated with an existing autopilot and auto-stabilisation
assigned goals. Secondly, they are situated in an environment
system that performs basic flight control on the UAV. The
which contains other agents with which they can interact by
mission management system performs without any human
way of social protocols such as coordination, cooperation,
intervention, and represents a step forward in UAV autonomy.
negotiation, etc.
The use of agents as the design and development platform is
The agent programming language that the system was
particularly useful in view of future work involving teams, or
implemented in was JACK 1 . JACK is a powerful language that
perhaps swarms, of UAVs.
is used to develop open systems with complex and adaptive
Two agent designs inspired by different design principles
behaviour. JACK Teams is a programming extension of JACK
will be described and compared. The simulation and actual
that is used for developing agents with requirements for
flight tests will also be discussed. In addition to the ex-
coordination and other types of team interactions. Although
perimental report, we provide a short conceptual discourse
the system presented here is a single agent, the concepts and
on autonomous agent and multi-agent systems, and how the
designs are readily extendible to a teams context for future
proposed frameworks and concepts can apply to multi-UAV
projects.
control.
2. M ULTI -AGENT S YSTEMS
1. I NTRODUCTION
The idea of using agents for mission management and planning
UAV technology has progressed steadily over recent years is particularly interesting as a basis for distributed architectures
to include highly sophisticated control and artificially intelli- and multiagent systems (MAS). In cooperative MAS, multiple
gent capabilities [11]. The Predator UCAV [13], [14], Global agents work together to achieve one or more desired common
Hawk [14] and Pegasus [6] UAVs/UCAVs all represent major goals. The overall system goal is achieved through interactions
advances in this field. Up to now, most UAV operations consist and coordination of the individual agents [17]. A distributed
of large teams of human operators that oversee and control the mobile robotic team (in mobile robotic parlance referred to as
UAV’s actions. One of the aims of this research is to explore a robot collective) has advantages over a single, complex robot
methods of reducing UAV reliance on human operators, thus in many applications [7]. For example, for search and rescue
making UAVs a more economical and scalable option. This operations, multiple robots can forage far more effectively than
paper reports on design and conceptual developments for a a single, complex robot [2].
simple UAV path planning mission. Swarm Intelligence is the study of collective intelligence,
The UAV was given the task of independently deciding originally exhibited by large groups (or swarms) of animals,
between two alternate paths to take depending on data that usually insects such as ants and bees. It is an example of
it has collected via its sensors. The mission management team coordination in MAS. In ‘robotic swarms’, the agents
system (MMS) we have designed, developed and flight tested are designed to be simplistic (in terms of cognitive decision
is different from most other similar systems in that there making and sensory processing abilities), and are unaware
is no human intervention in the decision making processes.
The system processes and reasons over sensory data and 1 [Link]

0-7803-8894-1/04/$20.00  2004 IEEE 1 ISSNIP 2004

of their organisational position within a collaborating team. not be ideal. Rather, protocols that utilise localised databases
However, the system intelligence emerges from the interac- that are only accessed by relevant agents, are more desirable.
tions of these simple agents [3]. Vincent and Rubin [16] have
recently presented an efficient framework and algorithm for 3. T HE AVATAR UAV
cooperative search using UAVs based on swarm topologies The Codarra Avatar [5] is a lightweight UAV, purpose-built
and theories. for small-scale, autonomous reconnaissance and surveillance
However, despite the appeal of relatively simple agents in missions. Codarra2 initially perceived a need for tactical
swarm intelligence, we believe team coordination and control surveillance for military platoon-sized patrols. A small, pro-
in multi-UAV applications cannot exist without some sort of grammable UAV was developed to address this need. The
supervisory presence or team awareness. Essentially, there Avatar can be disassembled and reassembled in a short time,
needs to be some sort of supervision at the cognitive level, launched by hand (ie. thrown in the air for take-off), and
filtering the reflexive, non-cognitive swarm ‘urges’. When transportable in a back-pack. Propulsion is via an electric
there is a reflexive urge to fly somewhere, it needs to be motor through a fully-folding propeller that endows the UAV
tempered by many considerations before the actual decision is with gliding and powered flight capabilities. Flight endurance
made to carry out the particular manoeuvre (eg. Do I need ATC is approximately sixty minutes. Recovery (landing) is by
(Air Traffic Control) permission?, What is my energy state?, parachute that deploys from a bulge above the wing centre-
What are the weather conditions?, etc.) An interesting research section. The payload-bay can accommodate anything from a
question then arises: How does such high-level supervision video camera that is capable of transmitting video back to a
affect the outcomes of swarm type models in UAV, and other, ground station, to other sensors up to 1.5 kg in weight and
team applications? about the size of a standard house brick. Standard sensors
The taxonomy of a robotic collective can usually determine include: GPS receiver (with a precision of 100 metres),
the best organisational and coordination framework to use barometric altimeter, and a pitot-static air speed indicator.
for a specific domain [7]. Many interaction and coordination Communication range to the Ground Control Station (GCS)
protocols have been developed, and a few will be mentioned is approximately 10 km. Arbitrary data, as well as command
here [17] as examples of effective MAS coordination frame- signals, can be communicated to the UAV via this radio link.
works:
Contract Net [15] In this protocol, a contract mechanism A. Issues in providing the Avatar with autonomous behaviour
is used to effectively delegate tasks to the most appropriate The issues in providing the Avatar with autonomy are:
agent. A manager agent broadcasts a set of tasks to other Flight time/range As the UAV is battery powered, the
contractor agents within the team. The contractor agents then relative flight time/range is small compared to, for example,
deliberate over the tasks outlined in the broadcast message, a standard petrol powered radio-controlled (RC) miniature
and ‘interested’ contractor agents submit a bid to the manager aircraft. For missions requiring longer flights, this can be
agent proposing plans or solutions to achieve the task goals. achieved by ‘refuelling’ stops (ie. the UAV returns to base
The manager then selects the bid that it perceives to yield the for a battery change, or another UAV ‘baton changing’ for
highest expected utility, resulting in a contractual obligation re-launch).
for the task to be completed by the contractor agent. This Durability The Avatar is a lightweight, small aircraft, and
process can recurse indefinitely until an agent actually carrys therefore cannot withstand turbulent, high speed wind condi-
out the task. tions. In certain conditions, the Avatar can overcome these
Blackboard Systems. [12], [17] In this protocol, communi- situations by carefully planning flight paths. Such planning
cation is achieved by a centrally located ‘database’ which all can be executed by the JACK agent mission manager onboard
agents within the system have access to. Messages to one or the UAV. However, there are situations where no amount of
more agents can be delivered by flexibly entering them on careful mission planning, by either the JACK agent or a human
the blackboard. Also, the state of the world can be stored by operator, can overcome the adverse conditions.
individual agents in the blackboard, resulting in an incremen- Flight regulations and restrictions Federal law requires
tally updated global view of sensory and reasoning data of the that all aircraft pass through stringent safety checks. In par-
entire team that can be accessed by all team members. Another ticular, autonomous aircraft must exhibit deterministic be-
benefit of blackboard systems is that knowledge sources (KS), haviour, as certification for autonomous aircraft that can be
or agents, that contribute to the blackboard contents do not demonstrated to exhibit deterministic behaviour is expected
necessarily need to know the entire problem space, and hence to be a much more straight-forward process than for non-
can make a contribution based on their knowledge and/or deterministic systems. Furthermore, for mission-critical or
expertise on a particular aspect of the problem. Of course, high risk/hazard missions, deterministic behaviour is desirable.
the only limitation with blackboard systems with respect to Deterministic behaviour means that the aircraft cannot be
UAV teams, is that all agents must have an adequately reliable overly ‘creative’ and behave in a potentially random manner
communication link to the blackboard database. This may not in known, and indeed, unknown situations. This restriction is
be possible for certain applications. Hence, with respect to this
limitation, protocols that require a centralised database may 2 [Link]

2
 Control Hierarchy On-board Controller
and
Mission Management System
Capability
Mission
Managment
agent-based JACK Agent
planning and running on
control HP iPaq Event has

Trajectory serial link

Agent
Management
ie. FMS, control of vehicle handles
Autopilot
3-axis stabilised
Attitude Managment
ie. Autopilot , auto- stabilisation
has
Plan

Avatar UAV
posts

has

Belief
Other
Events

Fig. 1: Agent - FCS Architectural design [10]

Fig. 2: An illustrated diagram of JACK agent-programming language

constructs
part of the reason why we do not believe swarm topologies
(refer to Section 2) are an adequate team model. The JACK
programming environment that is being used with the Avatar A. Plans for the future
is ideal in this regard, as the plans that it executes are known
As mentioned above, for this inaugural test flight a simple
at compile time, and run deterministically in known/unknown
HP iPaq was used as the platform for the agent. This limited
situations.
the agent’s functionally as well as the available computational
Limited computational power As the payload of the power. However, the more powerful PC104 platform is sched-
Avatar is relatively small, the size of the agent-based Mission uled to be used for upcoming test flights.
Management (MM) processing computer must be small and
therefore may be limited in computational power. In our initial 5. T HE AGENT D ESIGN
experiment, we used the HP iPAQ - a standard Pocket PC-
based PDA (Personal Digital Assistant), which was limited A. The JACK Agent Programming Language
both in function and in computational power. The iPAQ will JACK is an agent-oriented programming language. Agent-
most certainly be replaced by the more powerful PC104 or oriented languages are based around the central concept
similar cut-down PC platform in future test flights. A more of agents, as described in the introduction. Architecturally,
thorough discussion follows in Section 4. JACK consists of several constructs: Agents, Capabilities,
Limited Sensory Data As described at the start of this Events, Plans and Beliefs. Agents are at the highest level
section, the Avatar is limited to GPS, altitude and speed data, of abstraction, and represent entities with the autonomous
with the exception of other data that it may receive from the behaviour within the system. Capabilities are abstract entities
GCS. However, for longer range missions where conditions that encapsulate groups of related events and plans (described
within the vicinity of the GCS are different from that of the below).
Avatar, this is not a practical arrangement. For such complex, In JACK, multiple threads of execution are driven by
long-range missions, a more sophisticated suite of sensors separate events. The events are handled by specific plans,
must be installed in the payload, such as radar, sonar or video which in turn can post one or more other events. Autonomy
imaging. is achieved by the dynamic selection of plans during run-
time, depending on the specific situation or problem faced by
4. T HE AGENT - F LIGHT C ONTROL S YSTEM (FCS)
the agent. For example, suppose an Obstacle Detected
A RCHITECTURE
event was received by the agent, it could handle it with
The agent sits at the top of the control tree, issuing high one of several plans, such as Turn Left, Turn Right,
level waypoint commands. The FCS then simply navigates to Generate New Path or Emergency Stop (if no other
the specified waypoint. Data is sent to the agent at a regular, plan is applicable or yields the desired expected utility).
albeit slow, frequency (nominally 1 Hz). As a result, the agent The combination of these constructs results in behaviours
must be robust and able to behave proactively in order to structured around the BDI, (Belief Desire Intention) theory of
compensate for the slow data update rate. Refer to Figure 1 agency[1], [18]. The BDI framework is based on the human
for a diagram of this architecture. rationality and goal-directed thinking processes. Each agent
Commands are sent and an acknowledgement is anticipated pursues its given goals (desires), adopting the appropriate
by the agent. Environmental data (ie. wind speed, direction) plans (intentions) according to its current set of data (beliefs)
as well as internal FCS and Ground Control Station (GCS) about the state of the world. In this way, agents behave like
data are available for use by the agent. humans with all the mental attributes encompassed by the BDI

3
 Tactical Decision Making (Decide)
framework. Refer to Figure 2 for an illustration and [1] for
more information. SituationAssesment
(plan)
posts posts
MonitorWaypoint
B. The Final Design MonitorWindSpeed
(event)
Proximity
(event)

There were several design approaches and implementations of handles

the MMS. We settled on an approach adopted in many simu- posts

uses
WaypointMonitoring currentWaypoint
uses
lations of military situations, called OODA - Observe, Orient, (plan) (Waypoint )

Decide, Act. Colonel John Boyd developed the OODA model WindSpeedMonitoring posts
uses

(plan)
of decision making on the premise that decision making in
briefedWaypoints
such situations is a perpetually looping sequence of: observing situation
TimeToSelect
Waypoint
(BriefedWaypoints )

(Situation ) (event)
the environment the agent is situated in, orientating based on modifies

these (and past) observations, deciding what to do, and finally modifies
handles

executing the chosen/formulated action. Refer to Figures 3, 4

weather WaypointSelection
and 5 for JACK design diagrams of this implementation. (WindConditions ) (plan)

Situation Awareness (Observe) Fig. 4: OODA design approach for Mission Management System: Decide

SpeedAwareness Standard Operating Procedures (Act)

SASpeedUpdate handles modifies acsSpeeds
(plan) (Speed )
(event)

SituationAssesment posts MonitorWaypoint

(plan) Proximity
PositionAwareness (event)
SAPositionUpdate handles modifies position
(plan) (Position )
(event)

handles

handles
WaypointAwareness currentWaypoint
SAWaypointUpdate modifies
(plan) (Waypoint )
(event)
WaypointMonitoring TimeToSelect
posts
(plan) AWaypoint
(event)

Situation Assessment (Orient)

handles
MonitorWaypoint
Proximity
posts (event)
StartSituation handles
SituationAssesment WaypointSelection SendWaypoint
Assesment (plan) posts
(plan) ChangeCommand
(event)
posts (event)
uses
MonitorWindSpeed
(event) handles

situation
(Situation )
WaypointChange
Procedure
(plan)

Fig. 3: OODA design approach for Mission Management System: Observe

and Orient Fig. 5: OODA design approach for Mission Management System: Act

UpdateAfterDecision
C. An Alternate Design (plan)
modifies
handles

An alternate design was also developed concurrently with the FCSDataUpdate

interceptedWpts
(Waypint belief type )

previously described design. It was not flight tested, despite it (event)

handles modifies

being functionally equivalent to the OODA approach design. UpdatePilotData

(plan)

However, it underwent thorough software testing and proved modifies

to be an efficient alternative to the final design. windSpeeds

(WindSpeed belief type)

reads
The rationale for this particular design was a simplistic, StartSituation handles
SituationAssesment
(plan)
Assesment
feed-back control system. Every time a data packet arrives, (event)

the MMS either stores the relevant data, or if it is time, makes

a decision based on the stored data. Refer to Figure 6 for the posts

JACK design diagram of this implementation

TimeToMake
Decision SendWaypoint
handles
6. F LIGHT T EST (event) (plan)

DSTO’s Air Vehicle Division (AVD) in collaboration with the

waits for
Air Operations Division (AOD) and the Department of Infor-
mation Systems, The University of Melbourne, successfully Command handles
Successful
Change Made
Acknowledge
conducted a flight test of the Codarra Avatar UAV between 5 (event)
(plan)

- 7 July, 2004.
Fig. 6: Feed-back control system approach for Mission Management System
Before the actual flight tests, a standard radio-controlled
aircraft, similar in size and flight dynamics to the Avatar,
was flown to test wind conditions and aircraft visibility. The

4
 First Alternate

Decision
Point Alpha 2

Alpha 1
Backup
RC Pilot

Launch Point

Second Alternate
Command Post

Recovery Point

Fig. 7: Avatar UAV Flight Path: At Alpha 2 the AUC (Autonomous UAV
Controller) selects one of the alternate waypoints based on the local wind
conditions. Failure of the AUC will result in the aircraft flying to the decision
point and then returning home

team then proceeded to configure the Avatar MMS with

waypoint data and conduct a ‘walk around’ test. This involved,
essentially, switching on the FCS and JACK agent systems,
carrying the UAV around the flight course, traversing the
waypoints the UAV is expected to intercept during its flight,
and monitoring the FCS and JACK agent behaviour from the
Ground Control Station (GCS). This testing method can be
likened to hardware-in-the-loop testing.
After these tests were completed and all minor configuration Fig. 8: The Codarra Avatar and the test team at Greytown, Melbourne,
Australia, 5 - 7 July, 2004.
issues rectified, the flight test was conducted. The UAV was
hand launched and manually piloted to a stable altitude and
attitude by an expert Radio-Controlled (RC) pilot. The UAV 8. F UTURE W ORK
was switched over to ‘UAV mode’, and then proceeded to
Future projects planned at DSTO include more complex UAV
intercept the waypoints described in Figure 7. Figure 8 shows
path planning tasks, and of greatest interest, an applied UAV
some pictures taken during the test flight.
teams project. A layered control design (eg. [8], [4]), team
7. C ONCLUSIONS reasoning and planning under uncertainty (eg. [9], [19]), and
team coordination and task synchronisation are among the
An agent-based mission management system (MMS) for a
research considerations.
small UAV was developed using the JACK agent-oriented
Additional work involving intelligent agents and UAVs
programming language. The UAV has, as standard, a flight
within DSTO, that extends the work presented in this paper,
control system (FCS) that performs simple tasks such as
concerns the development of UAV systems that approach
waypoint navigation and three axis auto-stabilisation. The test
the safety and reliability of manned aircraft. In working
mission was to make a single, fully autonomous, intelligent
towards this goal, agents are being developed that replicate
path planning decision mid-flight using JACK as the MMS
a pilot’s functionality in the management of his/her aircraft.
platform, coupled to the existing FCS.
These agents will form beliefs about the vehicle state and
Agent-oriented languages, and particularly the JACK lan-
environment based on sensors and data from other sources.
guage, offer higher levels of autonomy, intelligence, and above
Using such data, the agents will reason appropriately in the
all, the possibility for complex multiagent systems based teams
nature of a pilot about actions in the light of the overall mission
of UAVs capable of complex interactions such as coordination
goals.
and negotiation. UAV teams can achieve tasks more efficiently
and flexibly than a single UAV, or can tackle problems that are ACKNOWLEDGMENTS
not possible with a lone UAV. Contract Net and blackboard
systems are examples of effective coordination mechanisms We would like to thank Agent-Oriented Software for their
that can be used in dynamic multiagent system domains such support, in particular David Sheperdson and Richard Jones
as UAV teams. who provided assistance during implementation. From DSTO,
we would like to thank Erdal Akgol of AOD and Stephen
van der Velden of AVD for their assistance in the Agent-FCS
interface.

5
 R EFERENCES
[1] Agent-Oriented Software Pty. Ltd. (AOS), P.O. Box 639, Carlton
South, Victoria, 3053. JACK Intelligent AgentsT M : JACK Manual,
4.1 edition, April 2003. The Main JACK Manual describing the
JACK agent programming language, available at [Link]
[Link]/shared/resources/[Link].
[2] Tucker Balch. The impact of diversity on performance in multi-robot
foraging. In Proceedings of the third annual conference on Autonomous
Agents, pages 92–99. ACM Press, 1999.
[3] Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm Intelligence:
From Natural to Artificial Intelligence. Santa Fe Institute Studies in the
Sciences of Complexity. Oxford University Press, 1999.
[4] R. Peter Bonasso and David Kortenkamp. Using a layered control
architecture to alleviate planning with incomplete information. In
Planning with Incomplete Information for Robot Problems: Papers from
the 1996 AAAI Spring Symposium, pages 1–4. AAAI Press, Menlo Park,
California, 1996.
[5] Codarra. Codarra Avatar fact page. Web:
[Link] July 2004. Includes
a fact sheet: [Link]
[6] John Croft. Pegasus: UCAVs look seaward. Aerospace America, 9:36–
42, September 2003.
[7] G. Dudek, M. Jenkin, E. Milios, and D. Wilkes. A taxonomy for multi-
agent robotics, 1996.
[8] E. Gat. Artificial Intelligence and Mobile Robotics, chapter 8: Three-
Layer Architectures, pages 195–210. AAAI Press/MIT Press, 1997.
[9] Sarit Kraus, Onn Shehory, and Gilad Taase. Coalition formation with
uncertain heterogeneous information. In Proceedings of the second
international joint conference on Autonomous agents and multiagent
systems, pages 1–8. ACM Press, 2003.
[10] Andrew Lucas. First Flight True UAV Autonomy At Last. Agent-
Oriented Software Press Release, July 2004.
[11] David M. North, editor. Aviation Week, volume 159. McGraw-Hill,
September 2003.
[12] Grantham K. H. Pang. Development of a blackboard system for robot
programming. In Proceedings of the third international conference on
Industrial and engineering applications of artificial intelligence and
expert systems, pages 123–130. ACM Press, 1990.
[13] Predator. RQ-1 Predator Medium Altitude Endurance (MAE) UAV.
Web: [Link] November 6
2002. Includes fact sheets: [Link]
[14] David L. Rockwell. Sensing the future of UAVs. Aerospace America,
9:26–31, September 2003.
[15] Reid G. Smith. The contract net protocol: High level communication
and control in a distributed problem solver. In IEEE Transactions on
Computers, volume C-29, pages 1104–1113, December 1980.
[16] Patrick Vincent and Izhak Rubin. A framework and analysis for
cooperative search using UAV swarms. In Proceedings of the 2004 ACM
symposium on Applied computing, pages 79–86. ACM Press, 2004.
[17] Gerhard Weiss, editor. Multiagent Systems: A Modern Approach to
Distributed Artificial Intelligence. MIT Press, 2000.
[18] Michael J. Wooldridge. Introduction to Multiagent Systems. John Wiley
& Sons, Inc., 2001.
[19] Yang Xiang and Victor Lesser. On the Role of Multiply Sectioned
Bayesian Networks for Cooperative Multiagent Systems. IEEE Systems,
Man, and Cybernetics (Part A), 33(4):489–501, 2003.

View publication stats