George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and

Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

Modeling and Control of Unmanned Aerial Vehicles

– Current Status and Future Directions
George Vachtsevanos, Ben Ludington, Johan Reimann, Georgia Institute of Technology
Panos Antsaklis, Notre Dame University
Kimon Valavanis, University of South Florida

Abstract
Recent military and civil actions worldwide of distributed (in the spatio-temporal sense)
have highlighted the potential utility for sensors that must be coordinated to complete a
Unmanned Aerial Vehicles (UAVs). Both fixed complex mission. Current R&D activities are
wing and rotary aircraft have contributed discussed that concern issues of modeling,
significantly to the success of several military planning and control. Here, optimum terrain
and surveillance/rescue operations. Future coverage, target tracking and adversarial
combat operations will continue to place reasoning strategies require new technologies to
unmanned aircraft in challenging conditions deal with issues of system complexity,
such as the urban warfare environment. uncertainty management and computational
However, the poor reliability, reduced autonomy efficiency [Vachtsevanos, et al, 2004]. We will
and operator workload requirements of current pose the major technical challenges arising in
unmanned vehicles present a roadblock to their the “system of systems” approach and state the
success. It is anticipated that future operations need for new modeling, networking,
will require multiple UAVs performing in a communications and computing technologies
cooperative mode, sharing resources and that must be developed and validated if such
complementing other air or ground assets. complex unmanned systems as UAVs are to
Surveillance and reconnaissance tasks that rely perform effectively and efficiently, in
on UAVs require sophisticated modeling, conjunction with manned systems, in a variety
planning and control technologies. This paper of application domains. We will conclude by
reviews the current status of UAV technologies proposing possible solutions to these challenges.
with emphasis on recent developments aimed at
UAV improved autonomy and reliability and I. INTRODUCTION
discusses future directions and technological The future urban warfare , as well search and
challenges that must be addressed in the rescue, border patrol, Homeland security and
immediate future. We view the assembly of other applications, will utilize an unprecedented
multiple and heterogeneous vehicles as a level of automation in which human-operated,
“system of systems” where individual UAVs are autonomous, and semi-autonomous air and
functioning as sensors or agents. Thus, ground platforms will be linked through a
networking, computing and communications coordinated control system. Networked UAVs
issues must be considered as the UAVs are bring a new dimension to future combat systems
tasked to perform surveillance and that must include adaptable operational
reconnaissance missions in an urban procedures, planning and deconfliction of assets
environment. The same scenario arises in coupled with the technology to realize such
similar civil applications such as forest fire concepts. The technical challenges the control
detection, rescue operations, pipeline monitoring, designer is facing for autonomous collaborative
etc. A software (middleware) platform enables operations stem from real-time sensing,
real time reconfiguration, plug-and-play and computing and communications requirements,
other quality of service functions. Multiple environmental and operational uncertainty,
UAVs, flying in a swarm, constitute a network hostile threats and the emerging need for
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

improved UAV and UAV team autonomy and Figure 1: Autonomous Control Level Trend
reliability. Figure 1 shows the autonomous
control level trend according to the DoD UAV
Roadmap [Office of Secretary of Defense, 2002].
The same roadmap details the need for new
technologies that will address single vehicle and
multi-vehicle autonomy issue. The challenges
increase significantly as we move up the
hierarchy of the chart shown in Figures 2 (a) and
(b) from single vehicle to multi-vehicle
coordinated control. Moderate success has been
reported thus far in meeting the lower echelon
challenges. Achieving the ultimate goal of full
autonomy for a swarm of vehicles executing a
complex surveillance and reconnaissance
mission still remains a major challenge. To
meet these challenges, innovative coordinated
planning and control technologies such as
distributed artificial intelligence (DAI),
computational intelligence and soft computing,
as well as game theory and dynamic
optimization, have been investigated intensively Figure 2(a): The Autonomous Control Level Chart
in recent years. However, in this area, more
work has been focused on solving particular
problems, such as formation control and
autonomous search, while less attention has been
paid to the system architecture, especially from
an implementation and integration point of view.
Other significant concerns relate to inter-UAV
communications, links to command and control,
contingency management, etc.

Figure 2(b): The Autonomous Control Level Chart

We will review briefly in this paper a few of the

challenges referred to above and suggest
possible approaches to these problems. The
intent is to motivate through application
examples the modeling, control and
communication concerns and highlight those

2
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

new directions that are needed to assist in

arriving at satisfactory solutions. We will Urban Warfare

emphasize the synergy of tools and GTMax Manned Vehicle

methodologies stemming from various domains GTMav

Fixed Wing UAV GTMax

as well as the resurfacing of classical OAV Sniper

mathematical notions that may be called upon

now to solve difficult spatio-temporal dynamic
Ground
situations. Recent advances in computing and Ground
Sensor
Sensor

communications promise to accommodate the Moving

on-line real time implementation of such Target

mathematical algorithms that were considered Soldiers Ground

Sensor
intractable some years back.

II. System Architecture Commander

While networked and autonomous UAVs can be Operator

centrally controlled, this requires that each UAV

communicates all the data from its sensors to a
central location and receives all the control
signals back. Network failures and Figure 3: A Team of 5 UAVs Executing RS Missions in
communication delays are one of the main an Urban Warfare Environment
concerns in the design of cooperative control A “system of systems” approach suggests a
systems. On the other hand, distributed hierarchical architecture for the coordinated
intelligent agent systems provide an control of multiple UAVs. The hierarchical
environment in which agents autonomously architecture, shown in Figure 4, features an
coordinate, cooperate, negotiate, make decisions upper level with global situation awareness and
and take actions to meet the objectives of a team mission planning, a middle level with local
particular application or mission. The knowledge, formation control and obstacle
autonomous nature of agents allows for efficient avoidance, and a low level that interfaces with
communication and processing among onboard baseline controllers, sensors,
distributed resources. communication and weapon systems. Each level
consists of several interacting agents with
For the purpose of coordinated control of dedicated functions. The formation control
multiple UAVs, each individual UAV in the problem is viewed as a Pursuit Game of n
team is considered as an agent or sensor with pursuers and n evaders. Stability of the
particular capabilities engaged in executing a formation of vehicles is guaranteed if the
portion of the mission. The primary task of a vehicles can reach their destinations within a
typical team of UAVs is to execute faithfully specified time, assuming that the destination
and reliably a critical mission while satisfying points are avoiding the vehicles in an optimal
local survivability conditions. In order to define fashion. Vehicle model is simplified to point
the application domain, we adopt an assumed mass with acceleration limit. Collision
mission scenario of a group of UAVs executing avoidance is achieved by designing the value
reconnaissance and surveillance (RS) missions function so that it ensures that the vehicles move
in an urban warfare environment, as depicted in away from one another when they come too
Figure 3. close to each one. Simulation results are
provided to verify the performance of the
proposed algorithms.

3
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

Command Manned
& Control Vehicle

Level 3
Global
Knowledge Team Mission Planning Global Situation Knowledge Fusion
/Re-planning Agent Awareness Agent Agent

Global Performance QoS Assessment

Measurement Agent Agent

Level 2
Local Formation Control Moving Obstacle Local Situation
Knowledge Agent Avoidance Agent Awareness Agent

Local Mission FDI/Reconfigurable

Planning Agent Control Agent

Level 1
Behavioral
Vehicle Weapon System
Knowledge Communication Sensing Agent
Control Agent Agent Agent

……

Figure 4: A Generic Hierarchical Multi-agent System Architecture

The highest level of the control hierarchy Planning the UAVs’ flight route is also an
features functions of global situation awareness integral part of mission planning. A modified A*
and teamwork. The mission planning agent is search algorithm, which attempts to minimize a
able to generate and refine mission plans for the suitable cost function consisting of the weighted
team, generate or select flight routes, and create sum of distance, hazard and maneuverability
operational orders. It is also responsible for measures [Bertsekas, 1992], [Vachtsevanos et. al,
keeping track of the team’s plan, goals, and team 1997], can be utilized to facilitate the design of
members' status. The overall mission is usually the route planner. In the case of a leader-
planned by the command and control center follower scenario, an optimal route is generated
based on the capabilities of each individual for the leader, while the followers fly in close
UAV agent, and is further decomposed into formation in the proximity of the leader. The
tasks/subtasks which are finally allocated to the global situation awareness agent, interacting
UAV assets (individually or in coordination with with the knowledge fusion agent, evaluates the
other vehicles). This can usually be cast as a world conditions based on data gathered from
constrained optimization problem and tackled each UAV (and ground sensors if available) and
with various approaches, such as integer reasons about the enemy's likely actions.
programming, graph theory, etc. Market based Adversarial reasoning and deception reasoning
methods [Dunbar and Murray, 2002] [Voos, are two important tasks executed here. The
1999] and especially auction theory [Clearwater, global performance measurement agent
1996], [Walsh and Wellman, 1998], measures the performance of the team and
[Engelbrecht, et. Al 1983] can be applied as a suggests team re-configuration or mission re-
solution to autonomous mission re-planning. planning, whenever necessary. Quality of
service (QoS) is assessed to make the best effort

4
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

to accomplish the mission and meet the problem is essentially reduced to the standard
predefined quality criteria. Real world but much more complex pursuit game.
implementation of this level is not limited to the
agents depicted in the figure. For example, in Differential Game Theory was initially used to
heterogeneous agent societies, knowledge of determine optimal military strategies in
coordination protocols and languages may also continuous time conflicts governed by some
reside [Sousa and Pereira, 2003]. given dynamics and constraints [Isaacs, 1965].
One such application is the so-called Pursuit
III. Formation Control Game in which a pursuer has to collide with an
The problem of finding a control algorithm, evading target. Naturally, in order to solve such
which will ensure that multiple autonomous a problem it is advantageous to know the
vehicles can maintain a formation while dynamics and the positional information of both
traversing a desired path and avoid inter-vehicle the evader and the pursuer, that is, the Pursuit
collisions, will be referred to as the formation Game will be viewed as a Perfect Information
control problem. The formation control problem Game.
has recently received considerable attention due
in part to its wide range of applications in Stability of the formation of vehicles is
aerospace and robotics. A classic example guaranteed if the vehicles can reach their
involving the implementation of the virtual destination within some specified time,
potential problem is presented in [Howard et. al, assuming that the destination points are avoiding
2000]. The authors performed simulations on a the vehicles in an optimal fashion. It seems
two-dimensional system, which proved to be counterintuitive that the destination points
well behaved. However, as they mention in their should be avoiding the vehicles optimally,
conclusion, the drawback of the virtual potential however if the vehicles can reach the points
function approach is the possibility of being under such conditions then they will always be
“trapped” in local minima. Hence, if local able to reach their destination.
minima exist, one cannot guarantee that the
system is stable. In [Baras, et. al, 2003], the As a consequence of our stability criterion, it is
individual trajectories of autonomous vehicles necessary not only to determine the control
moving in formation were generated by solving strategies of the vehicles but also the optimal
the optimal control problem at each time step. avoidance strategies of the desired points. Let us
This is computationally demanding and hence label the final control vector of the vehicles by
not possible to perform in real-time with current φ and the control final vector of the desired
hardware. points by ψ . Then, the main equation which has
to be satisfied is:
This paper views the formation control problem
from a two player differential game perspective, ⎡ ⎤
which provides a framework to determine
min max ⎢
φ ψ ⎢
∑V j ⋅ f j ( x ,φ ,ψ ) + G ( x ,φ ,ψ )⎥ = 0
⎥
⎣ j ⎦ (1)
acceptable initial vehicle deployment conditions
but also, provides insight into acceptable
formation maneuvers that can be performed which has to be true for both φ and ψ .
while maintaining the formation. The f j ( x, φ ,ψ ) term is the jth dynamic equation
governing the system, and the V j is the
The formation control problem can be regarded
as a Pursuit Game, except that, it is in general, corresponding Value of the game. G ( x, φ ,ψ ) is a
much more complex in terms of the combined predetermined function which, when integrated,
dynamical equations, since the system consists provides the payoff of the game. Notice, that the
of n pursuers and n evaders instead of only one only quantity that is not specified in the equation
of each. However, if the group of vehicles is is the V j term.
viewed as the pursuer and the group of desired
points in the formation as the evader, the

5
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

From the main equation it is possible to xi = v xi

determine the retrograde path equations (RPEs), v xi = Fi cos(φ2i −1 ) sin(φ2i ) − ki ⋅ v xi
which will have to be solved to determine the yi = v yi
actual paths traversed by the vehicles in the
formation. However, initial conditions of the v yi = Fi sin(φ2i −1 ) sin(φ2i ) − ki ⋅ v yi
retrograde path equations will have to be zi = v zi
considered in order to integrate the RPEs. These v zi = Fi cos(φ2i ) − ki ⋅ v zi
initial condition requirements provide us with
Where i = 1,2 .
the ability to introduce tolerance boundaries,
within which we say that the formation has The two desired “points” are described by one
settled. Such boundaries naturally add set of dynamic equations. This simply implies
complexity to the problem, however they also that there is a constant distance separating the
provide a framework for positional measurement two desired points, and that the formation can
errors. only perform translations and not rotations in the
three dimensional space. Hence the dynamic
The above formulation suggests a way for equations become:
approaching the solution to differential game. xd = v xd
However, how does one ensure that inter-vehicle v xd = Fd cos(ψ 1 ) sin(ψ 2 ) − k d ⋅ v xd
collisions are avoided? To ensure this, it is yd = v yd
necessary to consider the payoff function v yd = Fd sin(ψ 1 ) sin(ψ 2 ) − k d ⋅ v yd
determined by the integral of G ( x, φ ,ψ ) . As an
z d = v zd
example, if we simply seek that the vehicles
v zd = Fd cos(ψ 2 ) − k d ⋅ v zd
must reach their goal within a certain time τ,
then G ( x, φ ,ψ ) = 1 . This can be verified by
τ In the above dynamical systems, the k i and k d
evaluating ∫ G ( x, φ ,ψ )dt = τ . Hence, we have factors are simply linear drag terms to ensure
0 that the velocities are bounded, and the Fd and
restricted our solutions to the initial vehicle Fi terms are the magnitudes of the applied
deployment, which will ensure that the vehicles
forces. Figure 5 shows the coordinate system
will reach the desired points in τ time. However,
and the associated angles.
if G ( x, φ ,ψ ) is changed to penalize proximity of
vehicles to one-another, only initial conditions
that ensure collision free trajectories will be
valid.

However, G ( x, φ ,ψ ) does not provide the means

to perform the actual collision avoidance, but
merely limits the solution space. So, in order to
incorporate collision avoidance into the
controller, one can either change the value
function or add terms to the system of dynamic
equations.
Figure 5: Definition of Angles

IV. Two-Vehicle Example Substituting the dynamical equations into the

In order to illustrate some of the advantages and main equation (1), we obtain the following
disadvantages with the differential game expressions:
approach to formation control, consider the
following system of simple point “Helicopters”,
that is, points that can move in three dimensions
governed by the following dynamic equations:

6
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

min[ F1 ⋅ (Vvx1 ⋅ cos(φ1 ) ⋅ sin(φ2 ) + Vvxd Vvyd

φ cos(ψ 1 ) = + , sin(ψ 1 ) = +
Vvy1 ⋅ sin(φ1 ) ⋅ sin(φ2 ) + Vvz1 ⋅ cos(φ2 ) + ) ρ d1 ρ d1
Vvzd ρ
F2 ⋅ (Vvx 2 ⋅ cos(φ3 ) ⋅ sin(φ4 ) + cos(ψ 2 ) = + , sin(ψ 2 ) = + d 1
ρd 2 ρd 2
Vvy 2 ⋅ sin(φ3 ) ⋅ sin(φ4 ) + Vvz 2 ⋅ cos(φ4 ) ] )
And From this, we see that the retrograde equations
max[ Fd ⋅ (Vvxd ⋅ cos(ψ 1 ) ⋅ sin(ψ 2 ) + have the following form:
ψ
Vvyd ⋅ sin(ψ 1 ) ⋅ sin(ψ 2 ) + Vvzd ⋅ cos(ψ 2 ) ] ) (2) o Vvx1
v x1 = − F1 ⋅ + k1 ⋅ v x1
ρ2
To obtain the control law that results from the o
max-min solution of equation (2), the following x1 = −vx1
lemma is used: o
Lemma 1: V x1 = 0
o
Let a , b ∈ ℜ : V vx1 = Vx1 − k1 ⋅ Vvx1

For this example, the final value will be zero,

2 2 and occurs when the difference between the
Then ρ = a + b
desired position and the actual position is zero.
Naturally, to obtain a more general solution, a
is obtained where solution manifold should be used; however, in
max(a ⋅ cos(θ ) + b ⋅ sin (θ )) order to display the utility of this approach, the
θ
previously mentioned final conditions will
suffice. The closed form expression of the value
a b function is then of the form:
cos(θ ) = , and sin (θ ) =
ρ ρ
1 − e − k1t
Vvx1 = ( x1 − xd ) ⋅
and the max is ρ k1

By combining Lemma 1 with Equation 2, the It should be noted that the above analysis could
following control strategy for vehicle 1 is found: be performed on a reduced set of differential
equations, where each equation would express
Vvy1 the differences in distance and velocity, and
Vvx1
cos(φ1 ) = − , sin(φ1 ) = − hence reduce the number of differential
ρ1 ρ1 equations by a factor of 2. However, for the sake
Vvz1 ρ1 of clarity, the analysis is performed on the actual
cos(φ2 ) = − , sin(φ 2 ) = −
ρ2 ρ2 position and velocity differential equations.
Where
ρ1 = Vvx21 + Vvy21 Furthermore, it should also be noted that this
solution closely resembles the isotropic rocket
and
pursuit game described in [Isaacs, 1965]. This is
ρ 2 = Vvx21 + Vvy21 + Vvz21 due to the fact that the dynamic equations are
decoupled, and hence working within a three-
Similar results are obtained for vehicle 2. For the dimensional framework will not change the
optimal avoidance strategy of the desired problem considerably.
points, we obtain the following:

7
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

vehicles can move quickly enough to actually

V. Simulation Results reach the desired trajectories, while in Figure 7
From the closed form expression of the control the velocities of the vehicles are not sufficient to
presented in the previous section, it is obvious reach the desired trajectories. In the latter case,
that the optimal strategies are in fact bang-bang the vehicles simply move in a smaller circle,
controllers. Since the forces in the system are which ensures that the error remains constant.
not dependent on the proximity of the vehicles
to the desired points, there will always exist The term target tracking is often used to refer to
some positional error. It is however possible to the task of finding/estimating the motion
resolve this problem simply by switching parameters (mainly the location and direction) of
controllers at some error threshold, or a moving target in a time sequence of
introducing terms that minimize the force terms measurements. This task is achievable as long as
F1 and F2 as the vehicles approach the desired the target is within the sensor's field of view
points. (FOV). If it happens that the target keeps
moving away to the point it runs o® the FOV,
the target tracking task will fail to track the
moving target until the target re-enters the
sensors FOV. To address such problem, the
sensor is mounted on a moving platform such as
a UAV. We call the new setup (the sensor plus
the UAV) an agent. Thus, we can start a second
task, other than the target tracking task, to
(reactively or proactively) move the sensor to
guarantee that the target stays in view. That
second task is what we call the agent placement
task. The work presented in this paper is of the
active sensing-based target tracking variety, in
which both tasks discussed above are integrated.
Figure 6: Two-Vehicle Simulation with Sufficient
Vehicle Velocities
VI. Target Tracking
There exists a number of efforts to formally
describe the dynamic agent placement problem
for target tracking. The choice is made to use a
formulation of the variety of Weighted
Cooperative Multi-robot Observation of
Multiple Moving Targets (W-CMOMMT)
(Werger and Mataric, 2000), (Werger and
Mataric, 2001) since it captures the multiple-
observer-multiple-target scenario with target
prioritization. W-CMOMMT can be shown to be
an NP-hard problem [Hegazy and Vachtsevanos,
2004].

Figure 7: Two-Vehicle Simulation with Insufficient The agent (sensor) placement problem is
Vehicle Velocities formulated by defining a global utility function
to be optimized given a graph representing the
The above plot shows the tracking capabilities of region of interest, a team of agents and a set of
the derived controller. The two vehicles are targets. A coarse motion model is developed
attempting to follow two parameterized circular first where target transitions follow a stochastic
trajectories with a radius of three. In Figure 6 the model described by an M th order Markov chain.

8
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

Agents use the model to predict the target In most cases, the above equations cannot be
locations at future time instants as probability determined analytically. The Kalman filter is a
distributions. The algorithm attempts to well-known exception. However, when a
maximize the coverage by searching for a set of Kalman filter is used, the system must be linear
observation points at each time step. A real time with
dynamic programming tool is called upon to Gaussian distributions. The particle filter is one
solve the maximization problem. Details of the way to estimate the above equations.
approach can be found in [Hegzy and A particle filter iteratively approximates the
Vachtsevanos, 2004]. posterior pdf as a set

Particle filters have recently been successful in

S k = ⎧⎨ x k(i ) , w k(i ) i = 1, … , n⎫⎬
tracking mobile targets in video (P. Perez and ⎩ ⎭
Blake, 2004) (P. Perez and Vermaak, 2002). The
video tracking problem consists of determining where x k(i ) represents a point in the state space,
the position of a target within a particular video
frame based on information from all past frames. and w (i ) is the importance weight associated
k
Information such as size, color, and motion with this point. The w k(i ) are non-negative, and
characteristics of the target is known a priori. In
the particle filter framework, this information is sum to unity. At each iteration, the particles are
used to initialize the filter in the first few frames updated using the system dynamics and
of video. Thereafter, using a model similar to (P. sampling from
Perez and Blake, 2004) the state of each particle
is updated as the video progresses from one p⎛⎜ x k(i ) x k(i−)1 ⎞⎟
frame to the next. At each step, color and motion ⎝ ⎠
data is collected for each particle to determine
which particles have a high probability of Measurements are then taken at each particle
correctly tracking the target. On the next and the weights are updated using

( )
iteration, particles are drawn according to this
probability. Thus, successful particles “survive” w k(i ) ∝ w k(i−)1 p z k x k(i )
and are used in subsequent frames, while the
other particles “die”. If the particles are resampled at each iteration,
then the previous weights may be neglected and
Particle Filtering in a Bayesian Framework Eq. 12 becomes
The objective of Bayesian state estimation is to
estimate the posterior pdf of a state, x k , based (
w k(i ) ∝ p z k x k(i ) )
on all previous measurements, z1:k . This pdf,
p(x k z1:k ) can be determined in two steps, After the weights are determined to at least a
prediction and update. In the prediction step, the scale factor, they are normalized such that their
state update model is used to determine the prior sum is equal to unity. It has been shown
pdf p(x k x k −1 ) . If a first-order Markov model is (Arulampalam and Maskell, 2002) that the
posterior pdf estimated using particle filtering
assumed, then the prior is given as
converges to the actual pdf as the number of
p(x k z1: k −1 ) = ∫ p(x k x k −1 ) p (x k −1 z1:k −1 )dx k −1
particles increases.

A particle filter was used to track a soldier as he

After the measurement, z k is made, the prior is maneuvered in an urban environment 2. Frames
updated using Bayes' rule: were grabbed from a movie at a rate of 30 Hz.
The movie camera was held by a human
p(x k z k ) p (x k z1:k −1 ) operator.
p(x k z1:k ) =
p (z k z1:k −1 )

9
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

Therefore, there are a number of vibrations in planning and control of networked

the video, and the zoom is adjusted during the systems.
video.
• Control– Hierarchical/Intelligent control
A few frames of the output are shown in Figure of multiple networked systems (agents,
8. The box represents a weighted average of the sensors); new reasoning paradigms for
ten best particles. The set of “lights” in the upper tracking, pursuit-evasion,
left corner of each frame are used to indicate the surveillance/reconnaissance,
output of the neural network. If the lowest coordinated control, planning and
“light” is “illuminated,” the neural network has scheduling, obstacle avoidance, etc.
output the lowest confidence level. If the second
lowest is “illuminated,” the neural network has • Networking and Communications –
output the second lowest confidence level. If the Inter-and intra-systems reliable and
middle two are “illuminated,” the neural secure communication protocols; need
network has output the second highest for command and control and
confidence level. If the top three are supervisory functions; bandwidth and
“illuminated,” the neural network has output the other Quality of Service requirements.
highest confidence level.
• Computing– On-platform computational
requirements; hardware and software
architectures; open systems architectures.

• Sensors and Sensing Strategies –

Hardware/Software requirements;
performance and effectiveness metrics;
networked sensors.

• Performance Metrics/Verification and

Validation – Defining metrics for design
and performance assessment; formal
methods for verification and validation.

The Enabling Technologies:

• New modeling techniques are required
to capture the coupling between
individual system/sensor dynamics,
communications, etc. with system of
systems behaviors. Hybrid system
approaches will play a key role. Means
Figure 8: Typical output frames. Each frame is to represent and manage uncertainty.
approximately 1.7 seconds apart from each other
Software models for improved QoS.
Spatio-temporal models of distributed
agents (sensors) are required to integrate
VII. New Directions/Technological
system and motion dependencies,
Challenges
contingency planning, etc.
Technological Challenges:
• Control – Intelligent and
*From single system to “system of systems”
hierarchical/distributed control concepts
• Modeling– Spatio-temporal modeling
must be developed and expanded to
paradigms are needed for real-time
address “system of systems”
configurations. Game – theoretic notions

10
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

and optimization algorithms running in called upon to play a major role in the
almost real time to assist in cooperative introduction of breakthrough technologies in this
control and adversarial reasoning. exciting area.
Control of networks of dynamic agents.
IX. REFERENCES
• Networking and Communications – 1. Vachtsevanos, G., Tang, L. and Reimann, J.
Communication protocols and standards. “An Intelligent Approach to Coordinated
Control of Multiple Unmanned Aerial
• Computing – Embedded processing Vehicles,” Proceedings of the American
requirements; new and reliable, fault- Helicopter Society 60th Annual Forum,
tolerant computing platforms; software Baltimore, MD, June 7-10, 2004.
reliability issue. 2. Office of the Secretary of Defense
(Acquisition, Technology, & Logistics), Air
• Sensors and Sensing Strategies - Warfare. "OSD UAV Roadmap 2002-2027."
Innovative concept and technologies in December 2002.
wireless communications; improved and 3. Sousa, J. B., and Pereira, F., "A framework
reliable/cost-effective sensor suites; for networked motion control," Proceedings
“smart” sensors and sensing strategies; of the 42nd IEEE Conference on Decision
data processing, data mining, sensor and Control, pp. 1526-1531, Hawaii, USA,
fusion, etc. December 2003.
4. Howard, M., Hoff, B., Lee, C., "Hierarchical
• Performance Metrics/V&V- Need new Command and Control for Multi-agent
system of systems performance and Teamwork", Proceedings of 5th Intl. Conf.
effectiveness metrics to assist in the on Practical Application of Intelligent
design, verification/validation and Agents and Mult-Agent Technology
assessment of networked systems. (PAAM2000), pp. 1-13, Manchester, UK.
Apr. 10, 2000.
5. Baras, J. S., Tan, X., and Hovareshti, P.,
VIII. CONCLUDING REMARKS "Decentralized Control of Autonomous
Federated systems consisting of multiple Vehicles," Proceedings of the 42nd IEEE
Unmanned Aerial Vehicles performing complex Conference on Decision and Control, pp.
missions present new challenges to the control 1532-1537, Hawaii, USA, December 2003.
community. UAVs must possess attributes of 6. Dunbar, W. B., Murray, R. M., "Model
autonomy in order to function effectively in a Predictive Control of Coordinated Multi-
“system of systems” configuration. Vehicle Formation," Proceedings of the 41st
Coordinated/collaborative control of UAV IEEE Conference on Decision and Control,
swarms demands new and novel technologies pp. 4631-4636, Las Vegas, USA, December
that integrate modeling, control and 2002.
communications/computing concerns into a 7. Voos, H., “Market-based Algorithms for
single architecture. Typical application domains Optimal Decentralized Control of Complex
include reconnaissance and surveillance Dynamic Systems,” Proc. of the 38th IEEE
missions in an urban environment, target Conference on Decision and Control,
tracking and evasive maneuvers, search and Vol.40, pp.3295-3296, Phoenix, AZ. 1999.
rescue operations, Homeland security, etc. 8. Clearwater, S. H. E., "Market-Based
Major technological challenges remain to be Control: A Paradigm for Distributed
addressed for such UAV swarms, or similar Resource Allocation," Singapore: World
federated system of systems configurations to Scientic, 1996.
perform efficiently and reliably. Excessive 9. Walsh, W., and Wellman, M., “A Market
operator load, autonomy issues and reliability Protocol for Decentralized Task
concerns have limited thus far their widespread Allocation,” Proc. of the 3rd International
utility. The systems and controls community is Conference on Multiagent Systems, 1998.

11
 George Vachtsevanos, Ben Ludington, Johan Reimann, Panos Antsaklis, Kimon Valavanis, “Modeling and
Control of Unmanned Aerial Vehicles– Current Status and Future Directions,” Workshop on Modeling and
Control of Complex Systems (MCCS), Ayia Napa, Cyprus, June 30-July 1, 2005. Also Chapter 9, in
Modeling and Control of Complex Systems, CRC Press 2007.

10. Engelbrecht, W. R., Shubik, M. , and Stark, 20. Loy, G., Fletcher, L. and Apostoloff, N.
R.M., "Auctions, Bidding, and Contracting: (2002). An adaptive fusion architecture for
Uses and Theory," New York, NY: New target tracking. Proceedings of the Fifth
York University Press, 1983. International Conference on Automatic
11. Bertsekas, D., “Auction Algorithms for Face and Gesture Recognition. pp. 248-253.
Network Flow Problems: A Tutorial 21. Hegazy, T. and Vachtsevanos, G. (2004).
Introduction,” Computational Optimization Dynamic agent deployment for tracking
and Applications, Vol.1, pp.7-66, 1992. moving targets. Proceedings of the Twelfth
12. Vachtsevanos, G., Kim, W., Al-Hasan, S., Mediterranean Conference on Control and
Rufus, F., Simon, M., Schrage, D., and Automation. Kusadasi, Aydin, Turkey.
Prasad, J. V. R., "Mission Planning and 22. Hochbaum, D. (1995). Approximate
Flight Control: Meeting the Challenge with Algorithms for NP-Hard Problems. PWS
Intelligent Techniques," Journal of Publishing Company.
Advanced Computational Intelligence, Vol. 23. Perez, P., Hue, C. and Vermaak, J. (2002).
1, (1), pp. 62-70, Oct., 1997. Color-based probabilistic tracking.
13. Al-Hasan, S. and Vachtsevanos, G., Proceedings of the European Conference
"Intelligent Route Planning for Fast on Computer Vision. pp. 134-149.
Autonomous Vehicles Operating in a Large 24. Perez, P., Vermaak, J. and Blake, A. (2004).
Natural Terrain," Journal of Robotics and Data fusion for visual tracking with
Autonomous Systems, Vol. 40, pp. 1-24, particles. Proceedings of the IEEE. Vol. 92.
2002. pp. 495-513.
14. Al-Hasan, S. and Vachtsevanos, G., "A 25. Han, Q.,Ye, Y., Zhang, H. and Zhang, J.
Neural Fuzzy Controller for Moving (2002). On approximation of max-vertex-
Obstacle Avoidance," Third International cover. European Journal of Operational
NAISO Symposium on Engineering Of Research (2), 207-220.
Intelligent Systems, Malaga, Spain, 26. Werger, B. and Mataric, M. (2000).
September 24 - 27, 2002. Broadcast of local eligibility: Behavior-
15. Isaacs, R. "Differential Games: a based control for strongly cooperative robot
Mathematical Theory with Applications to teams. Proceedings of the Fourth
Warfare and Pursuit, Control and International Conference on Autonomous
Optimization." New York: John Wiley and Agents. pp. 21-22.
Sons, Inc., 1965. 27. Werger, B. and Mataric, M. J. (2001). From
16. Arulampalam, A. and Maskell, S. (2002). A insect to internet: Situated control for
tutorial on particle fiters for online networked robot teams. Annals of
nonlinear/non-gaussian bayesian tracking. Mathematics and Artificial Intelligence
IEEE Transactions on Signal Processing 31(1-4), 173-197.
50, 174-188.
17. Berchtold, A. and Raftery, A. (2002). The
mixture transition distribution model for
high-order Markov chains and non-
gaussian time series. Statistical Science
17(3), 328-356.
18. Kwok, C., Fox, D. and Meila, M. (2003).
Adaptive real-time particle fitlers for robot
locatlization. In: Proc. of the IEEE
International Conference on Robotices &
Automation. pp. 2836-2841.
19. Fox, D. (2003). Adapting the sample size in
particle filters through kld-sampling.
International Journal of Robotics Research
pp. 985-1002.

12