2021 International Conference on Automation, Robotics and Applications

Deep Learning Localization with 2D Range Scanner

Giuseppe Spampinato, Arcangelo Bruna, Ivana Guarneri, Davide Giacalone
System, Research and Application
STMicroelectronics
Catania, Italy
{[Link], [Link], [Link], [Link]}@[Link]
2021 7th International Conference on Automation, Robotics and Applications (ICARA) | 978-1-6654-0469-3/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICARA51699.2021.9376424

Abstract—In recent years, the use of 2D laser range scanners localization [7]. These approaches usually make use of
is increasing in industrial products, thanks to decreasing cost of Bayesian filtering to transform the robot localization into
this kind of devices and increasing accuracy. Nevertheless, the probability distribution estimation problem based on grid maps
localization estimation of the moving objects (vehicles, robots, [8, 9]. Another classical method called Iterative Closest Point
drones and so on) between consecutive laser range scans is still a (ICP) estimates the transformation between two point- clouds
challenging problem. In this paper, we explore different neural by minimizing the matching error between them [10]. Another
network approaches, using only a 2D laser scanner to address more complex method called Lidar Odometry and Mapping
this problem. The proposed neural network shows promising (LOAM) runs two different algorithms in parallel to achieve
results in terms of average accuracy (about 1cm in translation
real time processing [11]. Since we have distance information,
and 1°in rotation of Mean Absolute Error (MAE)) and in terms
of overall used parameters (less than one hundred thousand),
these techniques usually achieve precise localization, but they
being an interesting method that could complement or integrate suffer in the case of presence of moving obstacles in the scene.
traditional localization approaches. The proposed neural In fact, in this case, we have lack of information due to
network processes about 8000 pairs of compacted scans per occlusions and transformation estimation can fail.
second on Nvidia Titan X (Pascal) GPU. Inspired by the recent success of deep learning algorithms
in extracting and classifying features, several recent methods
Keywords—localization, laser scans, dataset generation, neural
used these kind of approach to estimate camera position in
network
vision-based localization: PoseNet used Convolutional Neural
Networks (CNNs) for pose regression task [12], Deepvo uses
I. INTRODUCTION Recurrent Neural Networks (RNNs) to reach the same goal
Localization is a very important step in the field of robot [13], while UndeepVO is an unsupervised deep learning
navigation, and it is considered one of the most difficult method to estimate the pose of a monocular camera [14]. It is
problems in robotics. The main goal is to estimate the punctual important to underline that deep learning-based methods still
position of the moving robot using current sensor data and do not reach performances of classic vision-based localization,
previously stored data (usually arranged creating an evolving in terms of pose accuracy estimation.
map). It is to be considered a preliminary step for further tasks Following the trials of the deep learning algorithms in
like: Simultaneous Localization and Mapping (SLAM), point vision-based localization, some deep learning methods for
to point navigation, target tracking, path planning and so on. laser-based localization have been proposed: Nicolai et al.
Different approaches have been proposed in the past to processed 3D laser scanner data in CNNs to estimate odometry
perform localization. Two main categories can be easily [15], Pfeiffer et al. give steering commands training a target-
identified: vision-based localization and laser-based oriented navigation model [16], Li et al. propose a CNN
localization. network to perform scan matching and loop closure [17] and
recently Valente et al. propose a RNN to learn temporal
In vision-based localization, since images have useful features in order to improve the odometry results [18].
visual information, they are used to be matched among them to
find best features alignment, achieving best instant localization. The same consideration made for deep learning vision-
Examples of these kind of techniques are: Effective Prioritized based localization are still valid for deep learning laser-based
Matching [1], ORB-SLAM [2], Semi-direct monocular Visual localization: results are still not so performant compared to
Odometry (SVO) [3], Camera Pose Voting [4], Probabilistic efficiency of existing classic techniques. In fact, many authors
Feature Map-Based Localization [5] and so on. Other suggest to integrate the deep learning methods with classical
approaches make use of multi-resolution image pyramid methods: Velas et al. suggest to use CNNs with 3D laser
method to achieve robust scan-to-scan matching [6]. Even if scanners for Inertial Measurement Unit (IMU) assisted
these techniques are enough robust, they suffer for the lack of odometry [19] and Xu at al. propose to use CNN based image
distance information. retrieval as input seed to speed-up Monte Carlo localization
algorithm convergence and increase robustness and precision
Laser-based localization techniques use laser scans, which [20].
contain much less information than images. The weak
variations in range measurements for features (e.g. corners) As aforementioned, the machine learning techniques for
and the subsequent lack of distinctive features does not allow a moving robots localization, in particular for 2D Lidar data, is
simple equivalent feature-based approach like in vision-based still a challenging process and it has not yet been exhaustively

978-0-7381-4290-6/21/$31.00 ©2021 IEEE 206

Authorized licensed use limited to: UNIVERSITA MODENA. Downloaded on August 01,2025 at 06:43:30 UTC from IEEE Xplore. Restrictions apply.
explored. In fact, up to this moment only few papers have been optimal angle resolution we tried to minimize the collisions per
investigated about this field. In our work we choose to continue frame, that is the number of data ranges which are in the same
the investigation, developing a simple neural network method bin, and to maximize the array density, that is the number of
to localize a moving robot, using as input sequences only data non-zero value bins for each scans.
acquired from 2D laser scanners. Since we are interested in
low-cost real-time solutions, we reduced the dimension of Since RPLidar A2 covers a full 360°, laser scanner data is
parameters of the neural network, to fit with low-cost resources. separated into β° bins, for a total of N=360°/β° bins. Table I
shows how different choices of angle resolution β impact on
This paper is structured as follows: in Section II the total number of bins N, on average collisions per frame and
proposed system is explained, while the experimental results average array density. Of course, increasing β the array density
are shown in Section III, followed by conclusion in final will become bigger, but also collisions per frame will increase.
Section IV.
Another important task is to understand how to solve
collisions, particularly present for higher angle resolution.
II. PROPOSED SYSTEM Valente et al. chose to take the average of all data distances
To implement our approach, we firstly generate a custom falling in the same bin [18]. Considering that all laser range
data set with different indoor environments (office, apartment, scanners are more precise for lower distances and that the
and laboratory), equipping a wheeled robot with a RPLidar A2 average can introduce false objects distances, in the case of
single laser range scanner and recording related depth scans. collisions, we chose to take the minimum distance.
This sensor has a maximum range of 12 meters, covering a full
360°at up to 15 Hz. TABLE I. DATASET BINARIZATION
After dataset generation, to generate the ground truth β N
Collisions Array
position (x, y, α) for each sample, we used the MATLAB per frame Density
Navigation Tool. It is very slow but efficient method, based on 0.10 3600 0.05 9%
Google Cartographer [21]. This algorithm is based on building
0.25 1440 0.13 22%
multiple submaps and aligning new scans to nearby submaps to
generate constrains on a graph. Although we do not obtain the 0.50 720 0.48 46%
real position of the sensor, we can still use data obtained for 1.0 360 2.54 89%
estimating localization with the neural network.
At last, we tried different neural network configurations to
find the best compromise between complexity and quality B. Neural Network
results. We perform all the experiments with this data set in a To obtain the robot displacement between a pair of 2D
Python environment using the TensorFlow framework and Lidar scans, the proposed network tries to estimate their
Keras wrapper. relative pose transformation:

A. Dataset Generation
 ΔxΔyΔα
It is important to note that acquisition of a large amount of
data is still one of the main challenges for the application of where Δx and Δy represent horizontal and vertical translation,
Deep Learning methods. We generate a custom dataset while Δα represents the rotation angle between two
consisting in about 51,000 samples, which is enough for our consecutive 2D Lidar scans (st-1,st). Since we are using 2D
experiments. Every sample is composed by two Lidar scans sensor only, we only estimate the 2D displacement of the robot.
(each one expressed by distances and angles), which is the
potential input for the neural network. The final goal is to learn the unknown function g(), which
maps (st-1,st) to the pose transformation T at time t:
Unfortunately, we cannot use each scan as it is, but we need
to encode it into a panoramic depth image, before being paired
with the subsequent scan. Data encoding is based on previous  tg(st-1,st)
work [17], where the laser scanner point set is encoded into a
1D vector. This is done binning the distances into angles bins, The function g() is learned during the training phase. Loss
depending on the angle resolution chosen. Considering all the functions used are Mean Square Error (MSE) and Mean
bins of a 360°rotation range, we store the depth values into a Absolute Error MAE. The global position of the robot at time t
vector, where each possible bin angle depth is represented. is obtained accumulating the estimated local poses from the
Once two 1D vectors from sequential scans are processed, we beginning up to time t.
concatenate them to use as input for the network.
Basic idea is to use similar standard 2D convolution-based
Differently from other kind of laser range scanner, which image networks to be adapted to our problem. In fact, the
usually give distances for constant angles scan by scan, in obtained encoded panoramic depth image format allows to use
RPLidar A2 the number of measurement and related angles can a series of standard 2D convolutional neural network layers to
vary scan by scan. In this way, with RPLidar A2 we cannot use extract spatial features detected by the sensor in the tested
a fixed angle resolution like in other approaches, e.g. Li et al. environment. With additional dense (fully connected) layers
use 0.25°[17] and Valente et al. use 0.10°[18]. To understand

207
Authorized licensed use limited to: UNIVERSITA MODENA. Downloaded on August 01,2025 at 06:43:30 UTC from IEEE Xplore. Restrictions apply.
the network could learn the patterns inside the extracted spatial TABLE II. NEURAL NETWORK PARAMETERS
features to provide better robot position estimation.
N NN Model NN Parameters
In Fig. 1 is shown an architecture of the proposed network. 3600 CNN+LSTM 21,078,563
The number of elements N depends on the angle resolution, as
previously indicated in Table I. We started from the network 3600 CNN+Dense 5,343,779
suggested by Valente et al. [18], but we experimented different 1440 CNN+LSTM 15,835,683
network structures to find the best compromise between
complexity and quality results. Two consecutive pre-processed 1440 CNN+Dense 2,722,339
laser scans, encoded as a 1D vector of size N, are concatenated 720 CNN+LSTM 14,262,819
to obtain the input tensor of the neural network. This input
tensor is fed to six 1D convolutional layers, each layer 720 CNN+Dense 1,935,907
followed by a Rectified Linear Unit (ReLU) activation, to learn 360 CNN+LSTM 13,214,243
the spatial features between the two laser scans. Differently
from Valente et al. [18] and according to Li et al. [17], between 360 CNN+Dense 1,411,619
convolutional layers, a max pool layer is introduced to reduce
to reduce computation complexity by extracting the most
important spatial features. We obtain better results with max III. EXPERIMENTAL RESULTS
pooling, since starting pre-processed laser scans are sparse, as As indicated in Table II, different neural network
exposed in previous Section IIA. At last, like indicated by Li et configurations have been tested with different indoor
al. [17], the obtained spatial features are received by two dense environments. We noticed in general that classical CNNs
(fully connected) layers to obtain the local position estimation. suffers a bit in the case of sparse dataset, so even if firstly we
As aforementioned, different neural network configuration conduct our experiments with all different configuration, we
have been tested, and, in particular, we vary: choose to use angle resolution β = 1.0°and then N = 360 for
final experiments. This choice also allows us to reduce the total
 The angle resolution β and then the number of bins N, neural network parameters and then the overall complexity.
which determine the input data dimension;
First experiments, conduct with input distances and output
 The last two layers of the neural network, using Dense position in millimeters, gives us a bad result: 2.48 MSE train
layers [17] and Long Short Term Memory (LSTM) loss, 216 MSE validation loss and 233 MSE test loss. Fig. 2
layers [18]. (left) shows the train and validation loss curves. We can notice
that the train curve correctly decreases, while the validation
Table II shows how different choices of total number of
curve is flat, so the neural network is not able to generalize the
bins N and different choice of model impact on the total
proposed regression problem.
number of neural network parameters. Of course, increasing N
and using two LSTM with the same size of Dense layers In this approach, the problem is the differences in the scales
(1024), the number of the neural network parameters and then across input variables, which may increase the difficulty of the
its execution time will increase. problem being modeled. In practice, it is often advantageous to
apply pre-processing transformations to the input data and to
C. Training the reference outputs before it is presented to a network [22].
As aforementioned, the input for the proposed neural Since one of the most common forms of pre-processing
network is a pair of consecutive encoded laser scans, that is a 2 consists of a simple linear rescaling of the input variables [22],
X N matrix, with N depending on the angle resolution, as we decided to change the reference distance measure from
previously indicated in Table I. The reference output of the millimeter to centimeter. In this case we obtained better results:
neural network is a vector T = [Δx,Δy,Δα], obtained by usage 0.13 MSE train loss, 3.84 MSE validation loss and 3.683 MSE
of MATLAB Navigation Tool. test loss. Fig. 2 (right) shows the train and validation loss
We used for all experiments the TensorFlow framework curves. It is to note that since MSE is a quadratic measure, the
with Keras wrappers in a Python environment. Training was error is reduced compared to previous experiment and that now
executed on a dedicated Workstation, octacore Intel Xeon ES- the validation curve is not flat, but starts to decrease, even if
2630 machine with 62GB RAM and an Nvidia GTX TITAN X not so well like the train curve. Also, in this case the neural
(Pascal) GPU. The TITAN X has 12GB of total RAM with network is not able to make a good generalization of the
3584 CUDA cores, allowing for fast parallelization in deep proposed regression problem.
learning training. At this point we obtained a good hint to work with, so
About training details, we used a low learning rate of different kind of normalization have been tested in TensorFlow
0.0001 for function cost minimization and we trained the framework: different type of scaling, MinMaxScaler() and
neural network for 500 epochs, with a small batch size of 32. StandardScaler(). The difficult is to apply the same
As training optimizer, we used Adam. We also tried other normalization in both input and reference output data. We
training optimizers, but no significant difference in obtained best results scaling input data and translation output
performance was noticed. data by 1,000 (that is using meters as measure) and output
rotation data by 100, to maintain similar scale also in output

208
Authorized licensed use limited to: UNIVERSITA MODENA. Downloaded on August 01,2025 at 06:43:30 UTC from IEEE Xplore. Restrictions apply.
variables. Moreover, we decide to use MAE, instead of MSE, Since we are interested in light solutions, to implement it in
because we obtain similar curves behavior with more intuitive a microcontroller with low memory (RAM and FLASH), we
results. tried to reduce parameters in the net, reducing the number of
With the correctly scaled dataset and output references we elements of the last two Dense layers. We found that also
finally obtained good results and no substantial differences for eliminating these two layers, obtaining a low number of total
the various configuration indicated in Table II (also using parameters (96,547), we obtain similar results. Fig. 3 (right)
LSTM). Using simplest configuration with 1,411,619 neural shows that train and validation curves are very similar also in
network parameters, we obtain 0.011 MAE train loss, 0.011 this last experiment and starting loss values are higher than
MSE validation loss and 0.010 MSE test loss. Fig. 3 (left) previous experiment because the net is simpler.
shows that train and validation curves are similar, indicating
that the neural network can generalize the regression problem.

Fig. 1. Proposed Neural Network

Fig. 2. Improvement obtained with different input dataset scaling: millimeter (left), centimeter (right).

Fig. 3. Final results obtained scaling input dataset in meters and with different parameters: 1,411,619 (left), 96,547 (right).

209
Authorized licensed use limited to: UNIVERSITA MODENA. Downloaded on August 01,2025 at 06:43:30 UTC from IEEE Xplore. Restrictions apply.
 [8] S.I. Roumeliotis, G. A. Bekey, W. Burgard, and S. Thrun, "Bayesian
IV. CONCLUSION estimation and Kalman filtering: A unified framework for mobile robot
localization", Proceedings of the IEEE International Conference on
In this paper we presented a simple approach based on Robotics and Automation (ICRA), 2000.
CNNs to estimate the global localization of a moving robot [9] S. Park, and K. S. Roh, "Coarse-to-Fine Localization for a Mobile Robot
using only the 2D laser scanner data. For the test data set, this Based on Place Learning With a 2-D Range Scan", IEEE Transactions
network architecture achieved about 1 cm in translation and on Robotics, 2016.
about 1° in rotation of MAE test error with only 96K [10] P. Besl, and H.D. McKay, "Method for registration of 3-D shapes",
parameters. Sensor Fusion IV: Control Paradigms and Data Structures, International
Society for Optics and Photonics, 1992.
Although the preliminary results presented were almost [11] J. Zhang, and S. Singh, "LOAM: Lidar Odometry and Mapping in Real-
comparable with state-of-the-art global localization techniques, time." Robotics: Science and Systems, Vol. 2, 2014.
at moment the deep learning methods could not entirely replace [12] A. Kendall, M. Grimes, and R. Cipolla. "Posenet: A convolutional
classic algorithms, since they are more flexible, and they can network for real-time 6-dof camera relocalization", Proceedings of the
IEEE International Conference on Computer Vision, 2015.
provide better results. However, the proposed system is a good
proof of concept and it is promising for further investigation. [13] S. Wang, R. Clark, H. Wen, and N. Trigoni, "Deepvo: Towards end-to-
end visual odometry with deep recurrent convolutional neural networks",
Moreover, since the proposed network runs in real-time, IEEE International Conference on Robotics and Automation (ICRA),
2017.
taking about 125 μs to provide an estimate on Nvidia Titan X
(Pascal) GPU, it could be an interesting complement or [14] R. Li, S. Wang, Z. Long, and D. Gu, "Undeepvo: Monocular visual
odometry through unsupervised deep learning", IEEE International
integration for classic localization methods. Conference on Robotics and Automation (ICRA), 2018.
[15] S. Wang, R. Clark, H. Wen, and N. Trigoni, "Deepvo: Towards end-to-
REFERENCES end visual odometry with deep recurrent convolutional neural networks",
IEEE International Conference on Robotics and Automation (ICRA),
[1] T. Sattler, B. Leibe, and L. Kobbelt, "Efficient & Effective Prioritized 2017.
Matching for Large-Scale Image-Based Localization", IEEE Transaction
on Pattern Analysis and Machine Intelligence, 2017. [16] R. Li, S. Wang, Z. Long, and D. Gu, "Undeepvo: Monocular visual
odometry through unsupervised deep learning", IEEE International
[2] R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, "ORB-SLAM: a Conference on Robotics and Automation (ICRA), 2018.
versatile and accurate monocular SLAM system", IEEE Transactions on
Robotics, 2015. [17] J. Li, H. Zhan, B. M. Chen, I. Reid, and G. H. Lee, "Deep learning for
2D scan matching and loop closure", International Conference on
[3] C. Forster, M. Pizzoli, and D. Scaramuzza. "SVO: Fast semi-direct Intelligent Robots and Systems (IROS), 2017.
monocular visual odometry", IEEE International Conference on
Robotics and Automation (ICRA), 2014. [18] M. Valente, C. Joly, and A. de La Fortelle, "An LSTM Network for
Real-Time Odometry Estimation", arXiv:1902.08536, 2019.
[4] B. Zeisl, T. Sattler, and M. Pollefeys, "Camera Pose Voting for Large-
Scale Image-Based Localization", Proceedings of the IEEE International [19] M. Velas, M. Spanel, M. Hradis, A. Herout, "CNN for IMU assisted
Conference on Computer Vision (ICCV), 2015. odometry estimation using velodyne LiDAR", IEEE International
Conference on Autonomous Robot Systems and Competitions
[5] H. Kim, D. Lee, T. Oh, and H. Myung, "A Probabilistic Feature Map- (ICARSC), 2018.
Based Localization System Using a Monocular Camera", Sensors, doi:
10.3390/s150921636, 2015. [20] S. Xu, W. Chou, and H. Dong, "A Robust Indoor Localization System
Integrating Visual Localization Aided by CNN-Based Image Retrieval
[6] E. Olson, “M3rsm: Many-to-many multi-resolution scan matching”, with Monte Carlo Localization", Sensors 19(2), 249, 2019.
IEEE International Conference on Robotics and Automation (ICRA),
2015. [21] W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop closure in
2d lidar slam”, IEEE International Conference on Robotics and
[7] G. D. Tipaldi, and K. O. Arras, “Flirt-interest regions for 2d range data”, Automation (ICRA), 2016.
IEEE International Conference on Robotics and Automation (ICRA),
2010. [22] C. M. Bishop, "Neural Networks for Pattern Recognition", Clarendon
Press, 1995.

210
Authorized licensed use limited to: UNIVERSITA MODENA. Downloaded on August 01,2025 at 06:43:30 UTC from IEEE Xplore. Restrictions apply.