2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

October 25-29, 2020, Las Vegas, NV, USA (Virtual)

Adaptive Dynamic Window Approach for Local Navigation

Matej Dobrevski1 and Danijel Skočaj1

Abstract— Local navigation is an essential ability of any

mobile robot working in a real-world environment. One of
the most commonly used methods for local navigation is the
Dynamic Window Approach (DWA), which heavily depends on
the settings of the parameters in its cost function. Since the
optimal choice of the parameters depends on the environment
that may significantly vary and change at any time, the param-
eters should be chosen dynamically in a data-driven way. To
cope with this problem, we propose a novel deep convolutional
neural network, which dynamically predicts these parameters
considering the sensor readings. The network is trained using
a state-of-the art reinforcement learning algorithm. In this
way, we combine the power of data-driven learning and the
dynamic model of the robot, enabling adaptation to the current
environment as well as guaranteeing collision-free movement
and smooth trajectories of the mobile robot. The experimental
results show that the proposed method outperforms the DWA
method as well as its recent extension.
Fig. 1. On the left: some canonical navigation scenarios in which DWA
I. INTRODUCTION can have problems. Parameters that are optimal for one situation can lead
to a local minimum or a longer path in another situation. On the right:
The ability to navigate unstructured environments is a our proposal, to adapt the parameters at each step based on a pre-trained
crucial capability for mobile robots operating in the real network.
world. Applications like service and inspection robots, de-
livery drones, unmanned surface vehicles, search and rescue
robots and many more, are all limited by the capabilities of the network. However, in this navigation paradigm it is not
the navigation software in use. Rightfully, robot navigation easy to make targeted improvements to the robot behaviour.
has been an extensively researched topic [1]. Additionally, they need mechanisms for ensuring collision
The usual approach for designing navigation systems is to free navigation. These shortcomings can be overcome by
split the problem into global navigation and local navigation. considering classical navigation approaches that specifically
Global navigation, or commonly path-planning [2], supposes address these issues.
that a map of the environment is available and produces a One of the most commonly used classical methods for
plan for reaching the goal position, while taking in consid- local-navigation today is the DWA [7]. In their extensive
eration the geometric properties of the robot. Unfortunately, experiments for indoor navigation, Willow-Garage decided
if the environment is not strictly managed, the map used for to include DWA as the local navigation method [8] and this
navigation quickly becomes outdated. Potentially adding the is the default local navigation method used in the navigation
difficulty of moving obstacles, a need for local navigation stack in ROS [9]. DWA generates control velocities to
quickly arises. Local navigation, in contrast to global navi- achieve a goal position by using the dynamic model of the
gation does not rely on a map of the environment, but only on robot to optimize the distance to obstacles, its velocity, and
the immediate sensor readings of the robot. It alleviates the the heading towards the goal. The reasons for its continued
problems of unforeseen changes in the map by considering usage are numerous: it generates smooth trajectories, the
the dynamic properties of the robot and making short-term trade-of between precision and computational power can be
decisions on avoiding obstacles while following the global controlled via the sampling, it works in velocity space so
plan as close as possible. there is no need for an additional controller, and it guarantees
Fueled by the availability of computing power, data, and collision free trajectories. As such, it has been extended to
new techniques for their training, deep neural-networks have numerous applications such as global navigation [10] and
shown impressive results on a number of domains in the last navigation in presence of known moving obstacles [11].
decade. Consequently, many approaches that use a neural- On the other hand, as pointed out in the original paper [7]
network to navigate a robot have appeared [3], [4], [5], [6], and other research [12], the performance of DWA depends
mostly using reinforcement learning (RL) algorithms to train on the weights we choose for the obstacle clearance, speed
1 Visual Cognitive Systems Laboratory, Faculty of Computer and Infor-
and heading in the cost function. As illustrated in Figure 1
mation Science, University of Ljubljana, Večna pot 113, Ljubljana, Slovenia it is dependent on these weights whether a robot will pass
[Link]@[Link] through a narrow gap, whether it will circumvent a given

978-1-7281-6211-9/20/$31.00 ©2020 IEEE 6930

obstacle or come completely to a stop. Additionally, there is Predictive Control. Recently, Missura and Bannewitz [11]
no general rule for setting these weights, but researchers and presented a variant of DWA which also addresses navigation
practitioners set them through trial-and-error. scenarios with moving obstacles, which are modeled as
This problem has been addressed in [13], [14] and [15]. In moving polygons.
[13] and [14] the authors propose a fuzzy system for adapting To address the problem of needing different weights for
these weights. However the rules for adapting these weights different scenarios Hong et al. [13] proposed the usage of
are still set manually, and only 8 configurations of obstacles a fuzzy system to adapt the weights of the algorithm based
are considered. In [15] the authors propose the usage of an on the distribution of obstacles around the robot. However,
Artificial Neuro Fuzzy Inference System (ANFIS) to learn the obstacle distribution is one of eight types, and the fuzzy
to adapt these weights from experience. However, they do rules are set by hand. Betoño et al. [15] expand this idea
not consider the configuration of obstacles in the learning and introduce an ANFIS for predicting the weights, and
system. introduce optimization of the cost function for two steps
In this work we present a new method for local naviga- ahead. In the inputs to their neuro-fuzzy predictor they only
tion, which combines the adaptability of learned methods consider the mean of the obstacle distances, the heading
while guaranteeing collision free trajectories and generating and the current speed, and not the detailed distribution of
smooth trajectories. Our method is based on the DWA, but obstacles. In contrast to these approaches we consider the
the cost function that DWA optimizes is changed at each distribution of obstacles in 128 measurement points around
step by a deep neural network. We train the neural network the robot and we use a deep convolutional network to predict
in a reinforcement learning setting with the usage of the the optimal parameters for the cost function. Additionally,
state of the art Proximal Policy Optimization (PPO) [16] our network is trained on experience using the state-of-the
method. Our method takes as an observation a laser-scan art PPO method [16].
of the environment and the position of the next goal and In the last few years numerous approaches that use a
output the new weights for the cost function. We evaluate our neural network to model the complete navigation policy
method extensively on the Gibson dataset [17] consisting of have appeared [3], [4], [5], [6], [23], using reinforcement
3D scans of real indoor spaces and show that we outperform learning to train the network. Xie et al. [3] have developed a
both DWA and its recent extension [15]. monocular image obstacle avoidance method based on the
The paper is organised as follows. In the following section dueling and double-Q mechanisms. This method however
we first present the related work. This is followed by the does not navigate to a goal location. Zhu et al. [4] developed
presentation of the proposed method Adaptive Dynamic a method for navigating to a target given as an image in
Window Approach (ADWA). Evaluation of the proposed a previously known environment. Leiva et al. [24] learn
method and comparison to related work is presented in a collision avoidance policy which as input combines the
Sec. IV. The paper closes with the concluding remarks in data from the RGB, depth image and lasers on the robot.
Sec. V. Lei at al. [23] train a navigation policy for the Turtlebot
using a Deep RL algorithm. One downside of purely us-
II. RELATED WORK ing a neural network is that targeted improvements in the
Local navigation, or navigation based on the robot sensor navigation behavior are impossible. Other downsides are the
readings, has been studied since the development of the lack of guarantees for collision free trajectories, and the
first mobile robots. From these various approaches the most generated smoothness of the robot movement. In our work
used ones have been methods operating in velocity space, we combine the strength of both, learning based and classical
as they consider the kinematic and dynamic constraints of approaches, and offer improvement over classical approaches
the robot, and in contrast to geometrical approaches, can be while also guaranteeing collision free navigation and smooth
easily translated to motor commands. Velocity space methods trajectories.
include several curvature based methods [18], [19], [7], [20], Our method is based on deep reinforcement learning.
from which the most popular has been the DWA, and is Since the incredible results achieved by Mnih et al. [25] and
included as the default local navigation method in ROS. their Deep Q-Networks (DQN) there has been a resurgence
The DWA addresses the problem of navigation by search- of research in reinforcement learning, such as methods that
ing in the collision free part of the reachable velocity space, improve the DQN like Dueling DQN [26] and Double
for speeds which maximize a weighted cost function in order DQN [27]. Techniques for improving convergence by break-
to maximize the clearance from obstacles, heading towards ing the correlation between successive samples, like Expe-
the goal, and movement speed of the robot. Brock and rience Replay [28] and Prioritized Experience Replay [29]
Khatib [10] expanded the algorithm so that it is applicable have also been used. Another direction has been research
to global navigation scenarios. Their approach requires a into methods applied to continuous outputs like Deep De-
costly calculation of a global navigation function, and is terministic Policy Gradients [30] and other methods directly
not suitable for scenarios where the environment is expected optimizing the policy like Trust Region Policy Optimization
to change. Ogren et al. [21] inspired by the work done by (TRPO) [31] which addresses some of the problems with
Rimon and Koditschek [22] developed a version of DWA the stability in reinforcement learning methods, by making
that is provably convergent by viewing the method as Model sure that the policy does not change drastically between

6931
successive updates. It does this by limiting the Kullback- B. Deep reinforcement learning approach
Leibler divergence between updates, with the size of the We model the problem of setting these weights as a
updates to the network parameters. PPO [16] is a kind of Markov Decision Process (MDP) and solve it in a reinforce-
successor method to TRPO, however the costly Killback- ment learning setting. The state at time t is defined as a
Leibler divergence calculation has been replaced with a vector st = [l1 , l2 , ..., l128 , θg , dg , νr , ωr ] where l1 to l128 are
much computationally simple limitation on the maximal laser range readings around the robot, measuring distance
permitted gradient update. We use PPO because of its elegant [0, 4]m in ±120◦ about the forward direction of the robot; θg
and efficient implementation and demonstrated convergence is the angle to the goal; dg is the distance to the goal; νr is the
properties. translational speed of the robot, and ωr is the rotational speed
of the robot. After each observation of the state, the robot
III. A DAPTIVE DWA produces an action at = [∆α, ∆β, ∆γ]. During the training
we consider the discount factor to be γdiscount = 0.99. The
A. DWA method reward function R(st ) does not contain any intermediate
rewards and is defined as:
In the DWA we assume that the control intervals are 
sufficiently small so that in each control interval [ti , ti+1 ] 100 , dg < 0.3m

we can assume the velocity of the robot is constant. In our R = −30 , robot stuck (3)
experiments this interval is ∆ti = ti+1 − ti = 0.2s. If we 

0 , otherwise
assume a synchro-drive robot and if the translational velocity
of the robot in the [ti , ti+1 ] interval is νi , its rotational The transition function of the MDP is implicitly defined by
velocity is ωi and its orientation at time ti is θ(ti ), then the dynamics of the robot and the environment in which it
we can derive that during the given interval the robot will is learning.
be moving along a circular arc with a center in: The laser-scan data in the physical world is highly cor-
related. The network should learn to extract from this low-
νi νi level data information like locations of sharp edges, small ob-
(− sinθ(ti ), cosθ(ti )) (1) stacles, and traversable areas, and combine this information
ωi ωi
with the movement parameters of the robot and the location
and a radius of r = ωνii assuming ωi 6= 0 [7]. If ωi = 0 towards it is headed and these ideas are reflected in the design
then the robot will be moving along a straight line in the of the network.
direction θ(ti ). Knowing this, at each control interval the As input, the network takes a 128 laser-range measure-
two-dimensional search space (ν, ω) for the command is ments in ±120◦ around the robot, passes this information
reduced to the intersection of the admissible velocities and through four 1D convolutional layers and concatenates it with
the dynamic window. Admissible velocities are all velocities the relative distance and orientation to the goal, as well as
that will not cause a collision, in other words, velocities that, the current translational and rotational velocity of the robot,
if applied, the robot will be able to stop before reaching as shown in Figure 2. The combined vector is processed
an obstacle on the current curvature, given its deceleration by four more fully connected layers, before the output layer.
limitations. The dynamic window are all velocities that can The outputs of the network have the tanh activation function
be reached within the next control cycle, given the limited so they are normalized in the range [−1, 1]. The laser-scan,
acceleration of the robot. as well as the four additional parameters to the network,
According to the DWA from this search space we choose are normalized before they are fed in the network. In PPO,
the velocities that maximize the function: during training we use an additional network for estimating
the value of the states, which is in turn used for calculating
G(ν, ω) = σ(α·heading(ν, ω)+β ·dist(ν, ω)+γ ·vel(ν, ω)) the advantage function. As is usual, the value estimation
(2) network which is used for calculating the advantage function
where heading(·) is the angle between the forward direction during learning, shares the same structure but has a single
of the robot and the goal, a measure of how much we are output with a linear activation function.
moving towards the goal; dist(·) is a measure of clearance, The network produces ∆α, ∆β and ∆γ as outputs
or the distance to the closest obstacle on the curvature; which are scaled to the ranges [−0.5, 0.5], [−0.3, 0.3] and
vel(·) is simply the translatonal velocity of the robot, so [−0.3, 0.3] respectively, then summed with the default values
as to maximize its speed; σ(·) is a smoothing operator that of α = 0.8, β = 0.1 and γ = 0.1, and the underlying DWA
normalizes heading(ν, ω), dist(ν, ω) and vel(ν, ω) to the operates with these adapted parameters.
range [0, 1]. The behaviour of the robot will change with
different values of α, β and γ. Our experiments showing C. Training
this dependence can be seen in the next section. The optimal The network is trained in simulation in a reinforcement
setting of these weights depends on the configuration of learning setting using an artificially constructed obstacle
obstacles around the robot, and in this work we design a course depicted in Figure 3. The obstacle course is signif-
neural network that does exactly that. icantly different to most maps of real world environments

6932
 happen if: the robot reaches the target location (distance to
the goal dg < 0.3m), the robot exceeds 300 control cycles,
or the robot gets stuck in a minimum from which it can not
get out (has not moved for 20 control cycles).
These experiences are gathered into a buffer and when
the buffer reaches 5000 steps an update to the policy is
performed according to the PPO rules. This represents one
epoch of training. We train the policy for 600 epochs, or a
total of 3M steps. The training lasts for a couple of hours,
when the simulation is using 4 cores of the Intel i7-6700 CPU
and the policy is trained on a Nvidia GTX 1070 GPU. The
trained network is then directly applied to the new domain,
without any further training.

IV. EVALUATION
Fig. 2. The neural network transforms the range-scan of the robot through A. Experimental setup
four convolutional layers. The output from the last convolutional layer is
fed into a fully connected layer, after which it is concatenated with the Evaluating the effects of a local navigation on a real-
information about the goal and the velocity of the robot. This complete istic navigation scenario requires some consideration. For
information is then fed through four fully connected layers, before finally example, the ROS implementation of DWA1 contains several
outputting the new parameters.
techniques on top of the bare-bones algorithm for signifi-
cantly improving its performance: the local goal is updated
at each control-cycle; additional cost is added for adhering
to the detailed path of the global planner; a flag for limiting
oscillatory behaviour; additional scoring point away from
the center of the robot; recovery behaviours etc. To produce
comparable results, we do not implement these additional
techniques, but use the method as it was originally presented
and is implemented in previous research [7], [15], [11].
To isolate the effects of the local-planner on the success
of navigation we thus simplify the interaction between the
local and the global planner as much as possible. The global
Fig. 3. The polygon used for training the network is a 20m by 40m hall
planner is run only to create navigation scenarios and does
filled with randomly generated convex obstacles. not interact with the evaluated navigation methods in any
other way. This also results in some goals being placed
in places where pure local-navigation can not reach the
(some of which are shown in Figure 4). We created this goal effectively. In production, a navigation system would
obstacle space entirely of randomly generated obstacles in include a more complex interaction between the global and
order to prevent over-fitting of the trained network to existing the local planner, such as taking care where the targets for
features of certain environments, and force it to learn the the local planner were placed given the velocity, position and
underlying physics of the problem. Randomization during orientation of the robot, would include recovery behaviours
training has been found to be an effective technique for and other interactions. However, in this evaluation we focus
alleviating issues with over-fitting in reinforcement learning only on the local planer, since the improvement in the
problems [32], [33], [34]. performance of this component naturally translates in the
The simulation is implemented in ROS, with nodes sim- improvement of the overall performance as well.
ulating the robot and range-scanner. For our robot, the For evaluation we used a subset of the Gibson dataset [17],
Turtlebot, we performed parameter identification and adapted which contains 3D data of scanned indoor spaces, to gen-
the transition functions for the translational and rotational erate realistic 2D floor-plans (six of which are depicted in
velocities in the simulator, to match them as close as possible Figure 4) of indoor office and home environments2 . If the
to the real robot. space contained multiple floors we generated a separate 2D
Each episode of interaction begins by randomly initial- map for each floor. In each map we randomly sampled 10
izing the starting and target location of the robot within a starting and goal locations, while taking care that there is at
predefined distance δ = 4m. In each control cycle (5Hz) least one obstacle between the start and the goal. In this way
the neural network outputs ∆α,∆β and ∆γ values which we generated a total of 170 non-trivial navigation episodes.
are added to the default values suggested by the authors of 1 [Link]
the original DWA. The system then navigates the robot to melodic-devel/dwa_local_planner
the target location until an episode is terminated, which can 2 [Link]

6933
 TABLE II
E VALUATION OF APPROACHES ON THE G IBSON DATASET, SEQUENTIAL
NAVIGATION

method # completed ep. # reached goals

Best DWA[7] 66 511
ANFIS DWA[15] 39 337
Ours 89 624

Fig. 4. Visualizations of 6 of the total 17 environments used for

evaluation of the navigation methods. The environments are real floor plans
of 3D scanned indoor spaces and contain realistic configurations of various
obstacles. One randomly chosen navigation episode is visualized for each
environment. These maps were loaded in ROS and a simulation of the
Turtlebot robot was navigated on each map.

The T heta∗ [35] global planner was executed on each

pair of starting and goal location for every map and gen-
erated the shortest trajectories. On this trajectory we chose
intermediate goals for navigation such that consecutive goals
are 1m apart (Euclidean distance), except the final goal Fig. 5. In the navigation episode in A all three methods reach the goal.
which is within (0, 1]m away from the previous goal. The In B the ANFIS DWA method is stuck near a sharp corner, while in C the
DWA gets stuck in front of a protruding obstacle (a wall).
robot was initialized in the starting location and navigated
to each intermediate goal until it reached the final goal,
after which an episode was ruled successful. If it failed to
and γ. In the instances where this was not the case, the
reach an intermediate goal the complete episode was ruled
algorithm failed almost completely. These results show that
unsuccessful. Each goal was considered reached when the
the performance clearly depends on these parameters.
center of the robot was within 0.3m of its location.
B. Sensitivity of DWA C. Experimental results
Using the same obstacle course as depicted in Figure 3,
TABLE I
we generated 10000 episodes of experience and trained
T HE BEST PARAMETERS FOR DWA
an ANFIS controller for DWA as specified in [15]. This
navigation method was then run on the same navigation
α β γ # completed ep. # reached goals
0.8 0.1 0.1 66 511
episodes as the DWA. As can be seen in Table II, it did
1.0 0.1 0.3 60 488 not manage to outperform DWA with a good combination of
1.0 0.1 0.5 52 403 parameters. We evaluated our trained network on the same
1.0 0.1 0.1 50 435 pre-generated episodes.
0.8 0.5 0.1 50 392
The results of all three methods are presented in Table II.
As can be seen, our approach can navigate about 35% more
To examine the sensitivity of DWA to the weights in the episodes than the vanilla DWA with the best combination of
cost function we performed a grid search of the α, β and γ parameters that we managed to find. The ANFIS method
weights, evaluating the performance of the algorithm with the on the other hand scored worse than the DWA with the
different weights on the complete dataset for each generated best parameters, completing about 40% less episodes. A
combination. The five best results achieved in this search are visualization of the generated trajectories on three navigation
shown in Table I. From the results we can confirm that the episodes is shown in Figure 5.
DWA is sensitive to the weights in the cost function. More We also note that for all navigation approaches most
precisely, the results show that in order to get acceptable episodes contain an intermediate goal which the approaches
behaviours α should be about 4 − 10 times larger than β did not manage to reach. Upon closer inspection we noticed

6934
 Fig. 6. Completely finished episodes for each environment in the dataset.

that these are mostly situations where the next goal is both V. CONCLUSIONS
around a corner and close to an obstacle (wall).
We examined the results per different environments in the The main contribution of this paper is the proposed new
dataset, the results of which are presented in Figure 6. Our local navigation method which uses the dynamic model of
method outperforms the DWA approach even if we compare the robot and a deep neural network to reach a given goal.
our method to the best performing parameters tuned for each In this way we combine the power of data-driven learning,
environment separately, which confirms our hypothesis that while also guaranteeing collision free movement and smooth
we are able to effectively predict suitable parameters for the trajectories.
cost-function on the basis of the distribution of obstacles Our method is based on the established DWA algorithm,
around the robot. and addresses the sensitivity of performance to the setting
To examine the behaviour of the navigation methods of the weights in the cost function. We do this by designing
on longer navigation tasks, we generated new navigation a deep convolutional neural network which predicts this
episodes. In this test, each episode contains a starting location parameters on the basis of the sensor readings of the robot,
and just a single goal such that the distance between the and we train this network using the state-of-the art PPO
start and the goal is between 0 and 4 meters. The start and reinforcement learning algorithm. The network is trained in
goal locations are sampled containing at least one obstacle a reinforcement learning setting in order to optimize the long
in between, so that there are no trivial navigation tasks. We term goal of reaching the target location. Our usage of the
generated 100 tasks per environment or 1700 in total. On DWA as the backbone, guarantees that the generated robot
this dataset we evaluated the best DWA, ANFIS DWA and trajectories are smooth and collision free.
our method. The results are presented in Table III. We also create a benchmark dataset, generated from the 3D
scanned Gibson dataset, on which we evaluate our navigation
method versus the vanilla DWA and previous work on adapt-
TABLE III
ing these parameters. Furthermore, by evaluating against
E VALUATION OF APPROACHES ON THE G IBSON DATASET, SINGLE
the best setting of parameters for each environment in the
DISTANT GOAL .
dataset, we conclusively show that our method outperforms
simple adaptations of the cost function parameters.
method # completed ep.
Best DWA[7] 294 Continuing this work, we will address the problems of
ANFIS DWA[15] 340 DWA that current approach did not manage to solve, namely
Ours 520 the problems of navigating around corners to goals that are
close to the wall. The solution of this problem will enable
our algorithm to be used in longer horizon navigation tasks,
From the presented results we can see that the performance making it applicable to a lot of navigation scenarios without
of DWA approach drops as the distance of the goal location is the usage of a global planner.
increased, being able to complete only 17% of the tasks. On
these longer navigation tasks the ANFIS DWA outperforms
the vanilla DWA, finishing about 16% more navigation tasks. ACKNOWLEDGEMENTS
Our method outperforms both methods again, reaching about
76% more episodes than DWA. The experiments also show This work was in part supported by the ARRS research
that our method needs about 3.3% additional computation project DIVID (J2-9433) and research programme Computer
time compared to the vanilla DWA. vision (P2-0214).

6935
 R EFERENCES [18] R. Simmons, “The curvature-velocity method for local obstacle avoid-
ance,” in In Proc. of the IEEE International Conference on Robotics
[1] J.-A. Meyer and D. Filliat, “Map-based navigation in mobile and Automation, 1996, pp. 3375–3382.
robots:: Ii. a review of map-learning and path-planning strategies,” [19] Nak Yong Ko and R. G. Simmons, “The lane-curvature method for lo-
Cognitive Systems Research, vol. 4, no. 4, pp. 283 – 317, 2003. cal obstacle avoidance,” in Proceedings. 1998 IEEE/RSJ International
[Online]. Available: [Link] Conference on Intelligent Robots and Systems. Innovations in Theory,
S138904170300007X Practice and Applications (Cat. No.98CH36190), vol. 3, Oct 1998,
[2] A. S. H. H. V. Injarapu and S. K. Gawre, “A survey of autonomous pp. 1615–1621 vol.3.
mobile robot path planning approaches,” in 2017 International Con- [20] B. P. Gerkey and K. Konolige, “Planning and control in unstructured
ference on Recent Innovations in Signal processing and Embedded terrain,” in In Workshop on Path Planning on Costmaps, Proceedings
Systems (RISE), Oct 2017, pp. 624–628. of the IEEE International Conference on Robotics and Automation
(ICRA, 2008.
[3] L. Xie, S. Wang, A. Markham, and N. Trigoni, “Towards monocular
[21] P. Ogren and N. E. Leonard, “A convergent dynamic window approach
vision based obstacle avoidance through deep reinforcement learning,”
to obstacle avoidance,” IEEE Transactions on Robotics, vol. 21, no. 2,
in RSS 2017 workshop on New Frontiers for Deep Learning in
pp. 188–195, April 2005.
Robotics, 2017.
[22] E. Rimon and D. E. Koditschek, “Exact robot navigation using
[4] Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and
artificial potential functions,” IEEE Transactions on Robotics and
A. Farhadi, “Target-driven visual navigation in indoor scenes using
Automation, vol. 8, no. 5, pp. 501–518, Oct 1992.
deep reinforcement learning,” in 2017 IEEE International Conference
[23] L. Tai, G. Paolo, and M. Liu, “Virtual-to-real deep reinforcement
on Robotics and Automation (ICRA), May 2017, pp. 3357–3364.
learning: Continuous control of mobile robots for mapless navigation,”
[5] P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, CoRR, vol. abs/1703.00420, 2017. [Online]. Available: [Link]
A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, org/abs/1703.00420
D. Kumaran, and R. Hadsell, “Learning to navigate in complex [24] F. Leiva, K. Lobos-Tsunekawa, and J. Ruiz-del Solar, “Collision avoid-
environments,” CoRR, vol. abs/1611.03673, 2016. [Online]. Available: ance for indoor service robots through multimodal deep reinforcement
[Link] learning,” in RoboCup 2019: Robot World Cup XXIII, S. Chalup,
[6] H. L. Chiang, A. Faust, M. Fiser, and A. Francis, “Learning navigation T. Niemueller, J. Suthakorn, and M.-A. Williams, Eds. Cham:
behaviors end-to-end with autorl,” IEEE Robotics and Automation Springer International Publishing, 2019, pp. 140–153.
Letters, vol. 4, no. 2, pp. 2007–2014, April 2019. [25] V. Mnih, K. Kavukcuoglu, D. Silver, A. a. Rusu, J. Veness,
[7] D. Fox, W. Burgard, and S. Thrun, “The dynamic window approach M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland,
to collision avoidance,” IEEE Robotics Automation Magazine, vol. 4, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou,
no. 1, pp. 23–33, March 1997. H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis,
[8] E. Marder-Eppstein, E. Berger, T. Foote, B. Gerkey, and K. Konolige, “Human-level control through deep reinforcement learning,” Nature,
“The office marathon: Robust navigation in an indoor office envi- vol. 518, no. 7540, pp. 529–533, 2015. [Online]. Available:
ronment,” in 2010 IEEE International Conference on Robotics and [Link]
Automation, May 2010, pp. 300–307. [26] Z. Wang, T. Schaul, M. Hessel, H. V. Hasselt, M. Lanctot, N. D.
[9] Stanford Artificial Intelligence Laboratory et al., “Robotic operating Freitas, and G. Deepmind, “Dueling Network Architectures for Deep
system.” [Online]. Available: [Link] Reinforcement Learning,” Journal of Machine Learning Research,
[10] O. Brock and O. Khatib, “High-speed navigation using the global vol. 48, no. 9, 2016.
dynamic window approach,” in Proceedings 1999 IEEE International [27] H. van Hasselt, A. Guez, and D. Silver, “Deep Reinforcement
Conference on Robotics and Automation (Cat. No.99CH36288C), Learning with Double Q-learning,” AAAI Publications, Thirtieth
vol. 1, May 1999, pp. 341–346 vol.1. AAAI Conference on Artificial Intelligence, 2016. [Online]. Available:
[11] M. Missura and M. Bennewitz, “Predictive collision avoidance for [Link]
the dynamic window approach,” in 2019 International Conference on [28] L. ji Lin, “Self-improving reactive agents based on reinforcement
Robotics and Automation (ICRA), May 2019, pp. 8620–8626. learning, planning and teaching,” in Machine Learning, 1992, pp. 293–
[12] A. Maroti, D. Szaloki, D. Kiss, and G. Tevesz, “Investigation of 321.
dynamic window based navigation algorithms on a real robot,” in 2013 [29] T. Schaul, J. Quan, I. Antonoglou, D. Silver, and G. Deepmind,
IEEE 11th International Symposium on Applied Machine Intelligence “Prioritized experience replay,” International Conference on Learning
and Informatics (SAMI), Jan 2013, pp. 95–100. Representations, 2016.
[13] Z. Hong, S. Chun-Long, Z. Zi-Jun, A. Wei, Z. De-Qiang, and W. Jing- [30] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa,
Jing, “A modified dynamic window approach to obstacle avoidance D. Silver, and D. Wierstra, “Continuous control with deep reinforce-
combined with fuzzy logic,” in 2015 14th International Symposium ment learning,” 2015.
on Distributed Computing and Applications for Business Engineering [31] J. Schulman, S. Levine, M. Jordan, and P. Abbeel, “Trust Region
and Science (DCABES), Aug 2015, pp. 523–526. Policy Optimization,” Icml-2015, p. 16, 2015. [Online]. Available:
[14] O. A. Abubakr, M. A. K. Jaradat, and M. A. Hafez, “A reduced [Link]
cascaded fuzzy logic controller for dynamic window weights optimiza- [32] OpenAI, “Learning dexterous in-hand manipulation,” CoRR, vol.
tion,” in 2018 11th International Symposium on Mechatronics and its abs/1808.00177, 2018. [Online]. Available: [Link]
Applications (ISMA), March 2018, pp. 1–4. 00177
[15] D. Teso-Fz-Betoño, E. Zulueta, U. Fernandez-Gamiz, A. Saenz- [33] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel,
Aguirre, and R. Martinez, “Predictive dynamic window approach “Domain randomization for transferring deep neural networks from
development with artificial neural fuzzy inference improvement,” simulation to the real world,” in 2017 IEEE/RSJ International Confer-
Electronics, vol. 8, no. 9, 2019. [Online]. Available: https: ence on Intelligent Robots and Systems (IROS), Sep. 2017, pp. 23–30.
//[Link]/2079-9292/8/9/935 [34] F. Sadeghi and S. Levine, “(CAD)2 RL: Real Single-Image Flight
[16] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and without a Single Real Image,” arXiv:1611.04201, 2016. [Online].
O. Klimov, “Proximal policy optimization algorithms,” ArXiv, Available: [Link]
vol. abs/1707.06347, 2017. [35] K. Daniel, A. Nash, S. Koenig, and A. Felner, “Theta*: Any-
[17] F. Xia, A. R. Zamir, Z.-Y. He, A. Sax, J. Malik, and S. Savarese, angle path planning on grids,” Journal of Artificial Intelligence
“Gibson env: real-world perception for embodied agents,” in Computer Research, vol. 39, p. 533–579, Oct 2010. [Online]. Available:
Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on. [Link]
IEEE, 2018.

6936