2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

October 25-29, 2020, Las Vegas, NV, USA (Virtual)

GndNet: Fast Ground Plane Estimation and Point Cloud Segmentation

for Autonomous Vehicles
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) | 978-1-7281-6212-6/20/$31.00 ©2020 IEEE | DOI: 10.1109/IROS45743.2020.9340979

Anshul Paigwar, Özgür Erkent, David Sierra-Gonzalez, Christian Laugier

Abstract— Ground plane estimation and ground point seg-

mentation is a crucial precursor for many applications in
robotics and intelligent vehicles like navigable space detection
and occupancy grid generation, 3D object detection, point cloud
matching for localization and registration for mapping. In this
paper, we present GndNet, a novel end-to-end approach that es-
timates the ground plane elevation information in a grid-based
representation and segments the ground points simultaneously
in real-time. GndNet uses PointNet and Pillar Feature Encoding
network to extract features and regresses ground height for
each cell of the grid. We augment the SemanticKITTI dataset to
train our network. We demonstrate qualitative and quantitative
evaluation of our results for ground elevation estimation and
semantic segmentation of point cloud. GndNet establishes a new
state-of-the-art, achieves a run-time of 55Hz for ground plane
estimation and ground point segmentation.

I. I NTRODUCTION
Fully autonomous driving is an important but challenging
goal, for which a reliable perception of the local environment
is crucial [1]. 3D-LiDARs are popular and widely used
sensors in robotics and intelligent vehicles. LiDARs generate Fig. 1. Overview of GndNet architecture: GndNet takes point cloud
from 3D LiDAR as input, estimates ground plane elevation, and segments
high-resolution 3D point clouds of the environment while point cloud into the ground and non-ground categories at 55Hz.
remaining unaffected by varying illumination. Point clouds
are used for several applications including object recognition,
navigation, and path planning. Navigation for intelligent
The segmentation of the points belonging to the ground
vehicles requires accurate detection of the navigable space
is also an important task. If these points can be segmented,
and classification of obstacles. For path planning and to
other tasks such as object detection and classification or lo-
estimate the risks on the path, occupancy grids are often used
calization and mapping would be improved both in accuracy
[2]. Occupancy grids are 2D spatial maps of the environment
and speed.
around the vehicle which can be constructed by processing
Therefore, we formulate our problem as finding a method
the LiDAR point cloud data. Typical steps that precede the
that would estimate the ground plane and segment the ground
occupancy grid generation are the estimation of the ground
points simultaneously in real-time for an autonomous vehicle
plane and the segmentation of the ground points [3].
setting where the computational efficiency of the method is
The goal of the ground plane estimation is to find out the
a critical aspect.
height of the ground for each of the grid cells. Due to the
sparse nature of the point clouds, this is a challenging task. In this work, we present GndNet, a deep learning-based
Furthermore, the occupation of some of the cells by tempo- approach for ground plane estimation and point cloud seg-
rary obstacles such as vehicles or pedestrians, lack of points mentation. GndNet uses PointNet to extract point-wise fea-
in some of the cells due to occlusions, and unevenness in the tures [6] and 2D convolutions on the grid to extract spatial
surface of the ground are some of the other challenges for features. By leveraging these features, the proposed network
estimating the ground height of the grid cells. The benefits learns the appearances, analyzes the scene and estimates the
of a correct estimation of the ground plane go beyond the ground elevation in real-time (Figure 1).
computation of occupancy grid maps. For instance, knowing One issue with learning-based approaches is the require-
the height of each LiDAR point in relation to the ground ment of a large amount of annotated data for training. Cur-
can benefit the plethora of LiDAR-based 3D object detection rently, no public dataset is available with an annotated ground
approaches [4], [5]. elevation map. To overcome this difficulty, we propose two
approaches: morphological operations and a CRF-based [3]
1 Univ. Grenoble Alpes, Inria, 38000, Grenoble, France; e-mail: approach to obtain the ground-truth elevation map from the
[Link]@[Link] SemanticKITTI dataset [7], [8].

978-1-7281-6212-6/20/$31.00 ©2020 European Union 2150

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 The key contributions of this study can be summarized as learning the representation of point clouds. However, in
follows: contrast to images, point clouds lack the detailed texture
• We present a novel deep neural network architecture information. Pointclouds obtained from 3D LiDARs are
called GndNet for real-time ground elevation estimation unordered, sparse and have a variable point density.
and point cloud segmentation. The network directly Recent work proposes novel types of network architec-
operates on sparse 3D points and is end-to-end trainable. tures, dealing with challenges associated with point cloud
• We show the network’s ability to analyze the scene, processing. Among these, PointNet has shown encouraging
learn the appearances, and distinguish the points be- results for single object classification and semantic segmen-
longing to the ground and non-ground. tation [6]. PointNet can learn point features from a set of
• We demonstrate how to generate ground truth datasets points but its applicability to entire large scale point cloud
for ground elevation from the SemanticKITTI dataset. has not been studied yet. 3D object detection approaches
We perform real experiments and quantitative com- like VoxelNet discretize the point cloud into a voxel grid
parisons to prove the integrity of our learning-based [4]. PointPillars takes a similar route but uses pillars instead
approach. of voxels [5]. PointNet is then applied to the points inside
each voxel/pillar to learn voxel/pillar wise features. This
II. R ELATED W ORK voxel/pillar feature encoding network is then used with
In this section, we first discuss methods related to ground convolution filters for region proposals and object detection.
plane estimation and segmentation. Next, we discuss meth- Inspired by VoxelNet and Pointpillars we choose to use
ods that solely focus on ground points segmentation. Finally, PointNet together with a pillar feature encoding network as
we cover deep learning methods for feature extraction from the backbone for our model.
point clouds.
One of the initial methods that appeared for ground III. G ND N ET A RCHITECTURE
estimation was based on elevation maps [9]. This method
was used in the DARPA Urban Challenge and is based on GndNet accepts raw point clouds as input and produces
projecting 3D points as a 2.5D grid and then using a Min- a grid-based ground elevation estimation and a point cloud
Max elevation map. However, this approach suffers from segmentation into two categories (ground and non-ground).
large errors in the case of bridges or treetops. Another It consists of three main stages as shown in Figure 2: (1)
group of studies relies on a 2D-line-extraction-based fast Discretization of the point cloud into a 2D grid; (2) Pillar
algorithm [10]. However, these algorithm-based methods feature encoding network that converts a point cloud to a
have problems with scalability to a large set of cases, for sparse pseudo image; and (3) 2D convolutional encoder-
example, curved terrains. Several other methods use the decoder network to process the pseudo-image and produce
gradient information of the terrain to model the ground a high-level representation and a regression of the ground
plane using a Markov Random Field (MRF) or a Conditional elevation per cell.
Random Field (CRF) [11], [12], [13], [3]. However, due to
A. Point cloud discretization
limited learning capacity the results in cases of occlusions
and sparse points are not satisfactory and the segmentation To extract features from the point cloud, we first discretize
of ground points require thus additional processing time. it into an evenly spaced grid in the x − y plane. Unlike Vox-
Narksri et al, propose a two-stage method to estimate elNet [4], which discretizes the point cloud into a 3D voxel
the ground points and then fit a plane by using RANSAC, map, the PointPillars authors argued that there is no need for
which makes the approach slow [14]. Liu et al integrate binning the z dimension, as it does not affect the accuracy
Gaussian Process Regression (GPR) and Robust Locally of the detection and is significantly more computationally
Weighted Regression (RLWR) to form a hybrid regression efficient [5]. This is equivalent to creating a set of pillars P .
model for the ground plane, which is also not real-time due We denote by l a point in a point cloud with coordinates x,
to the computational complexity of the GPs [15]. In another y, z, and reflectance r. Similar to Pointpillars, the points in
study, Velas et al use a discriminative method to segment each pillar are then augmented with (xc , yc , zc , xp , yp ) where
the ground points by using a neural network that performs the c subscript denotes distance to the arithmetic mean of all
in real-time [16]. However, the network is capable only of points in the pillar and the p subscript denotes the offset
ground point segmentation. A ground plane is not obtained from the pillar (x, y) center. Four-dimensional lidar point l
for occluded regions which would be necessary for tasks is now augmented to a D = 9 dimension. A large set of
related to autonomous vehicles. pillars will be mostly empty due to the sparsity of the point
One of the common points in most of the discussed meth- cloud, and the non-empty pillars will, in general, have only
ods is the use of handcrafted features. They generally fail a few points in them. We fixed the number of points N per
to scale, are complex to implement, and are computationally pillar to create a dense tensor of size (D, P, N ), where P
expensive. The sparsity, occlusions and the roughness of the is the number of non-empty pillars. If a pillar holds more
terrain are not considered. than N points then we randomly sample the points to fit the
Recently, Deep Learning methods has emerged as the tensor. Conversely, if a pillar has too few data points then a
prominent alternative for manual feature engineering for zero padding is applied to populate the tensor.

2151
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 Fig. 2. GndNet architecture: We use a pillar feature network with a fully convolutional encoder-decoder network to regress local ground elevation
values. The raw point cloud is discretized in a 2D grid forming a set of pillars. PointNet uses points in the pillars to learn pillar wise features that can
be scattered back to a 2D pseudo image. A convolutional encoder-decoder network uses the pseudo image to learn spatial features and regresses ground
elevation values per cell in the grid. The point cloud is then segmented in the ground and non-ground category using elevation values with a threshold.

B. Pillar Feature Encoding and Pseudo Image IV. T RAINING AND E XPERIMENTS
Next, we extract features per non-empty pillar using Supervised learning using deep neural networks requires,
a simplified version of PointNet. Our simplified PointNet in general, a large amount of annotated data to train. Absence
consists of a linear layer that is applied to each point followed of annotated ground elevation and point cloud data is the
by Batchnorm and ReLU to generate a (C, P, N) sized tensor. reason learning-based approaches have not been explored
Then, a max-pooling operation over the channels creates an for this task. Also because of the lack of annotated data,
output tensor of size (C, P). These encoded pillar features none of the existing ground detection approaches carry-
are then placed to their original locations in the grid to create out exhaustive quantitative evaluation and cross-comparison.
a pseudo-image of size (C, H, W) where H and W indicate Generating annotated ground elevation data is a crucial and
the height and width of the grid. important challenge we try to solve in this work.

C. Encoder-Decoder Network A. Dataset generation

To generate an annotated ground elevation dataset we use
To extract spatial features we use a convolutional encoder-
the SemanticKITTI [7] dataset, which is based on the KITTI
decoder network in a similar fashion as SegNet [17]. The
Vision benchmark and uses sequences from the odometry
encoder part of the network consists of 4 convolutional
task [8]. SemanticKITTI provides dense annotations for
layers. Max-pooling with a 2×2 window and stride 2 (non-
each scan of sequences 00-10. The dataset contains 28
overlapping window) is performed after every two convo-
classes including classes distinguishing non-moving and
lution layers. The resulting output of the encoder is sub-
moving objects (which covers traffic participants), and also
sampled by a factor of 4. We selected the number of layers
functional classes for ground, such as parking areas and
such that the ground elevation of a cell is only dependent
sidewalks.
upon its close neighboring cells. Each encoder layer has a
corresponding decoder layer and hence the decoder network
Definition of ground: We define ground elevation as the
has 4 convolution layers and max-unpooling after every two
height in the LiDAR sensor frame of reference at which
convolution layers. The final decoder output is of the same
object boundaries start or objects can be placed. For example,
shape as the input pseudo image (grid), and is fed to 3x3
the ground can be any terrain where a vehicle can traverse
convolution filter which regresses the ground elevation value
(does not imply that it is legal to traverse). In the case of
for each pixel in the pseudo image (cell in the grid).
non-moving objects like walls and trees, the ground elevation
is at the bottom where the actual object boundary starts as
D. Point cloud segmentation
shown in Figure 3.
We segment the point cloud into two categories as ground To generate ground elevation dataset from the
or obstacle points. In each cell, the points above threshold T SemanticKITTI dataset, we first remove all the points
of predicted ground elevation are segmented as obstacles and that do not belong to the ground, and keep only those
points below are segmented as ground points. These ground belonging to the categories road, sidewalk, parking, other
points then can be removed to generate an occupancy grid ground, and terrain as shown in Figure 4. We divide the
or to facilitate object detection. Ground elevation itself can entire environment into a 2D grid of shape (100, 100)
be used to find navigable and non-navigable space for path with cell resolution of 1m x 1m. We then calculate the
planning. ground-truth ground elevation per cell. One approach for

2152
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 Fig. 3. Definition of the ground plane as defined and utilized for this work.

Fig. 5. Morphological Operations: (a) Occupancy map generated using

only ground points. (b) Dilated occupancy map to fill the void cells. (c)
Occupancy map subtracted from dilated occupancy map resulting into a
mask for void cells. (d) Elevation map. (e) Inpainted elevation map. (f) Final
elevation map after applying average filter. The colors represent elevation
of the cells and the side bar shows the scale in meters.

Fig. 4. Left: 3D point cloud from SemanticKITTI dataset; Right: point

cloud with only ground points.

this task would be to align the point clouds and estimate

global elevation field for the whole driving sequence. We
observe that there are still many cells without the ground
points, mainly due to the occlusions. We explore two
different methods to tackle this issue:

Morphological method: We average the z values of all Fig. 6. Qualitative comparison of ground truth elevation map generated
using morphological operations vs CRF-method. The colors represent ele-
the ground points in the corresponding cells and create an vation of the cells and the side bar shows the scale in meters.
elevation map as shown in Figure 5 (d). Many cells belonging
to the ground do not have any points in them resulting in the
elevation of the cell as zero. Through this work, we aim to dependence and the reconstruction resulted in more struc-
achieve a smooth, uniform ground plane elevation even in turally defined and consistent output than a morphology-
regions with sparse points and in occluded regions. To fill based approach. A comparison between outputs from both
the holes in the elevation map we use an image processing the method is shown in Figure 6.
technique of in-painting. The procedure is as follows: Finally, we choose to use the CRF-based method to
1) The first step is to create a mask for the cells that need to generate the ground elevation dataset for the training of
be inpainted. We create an occupancy map using the ground our network. For each scan, we save the complete point
points as in Figure 5 (a). We use dilation to fill the holes in cloud (including ground and non-ground points) and the
occupancy map, Figure 5 (b). We subtract occupancy map ground elevation map generated using only ground points.
from dilated occupancy map resulting in the desired mask, To help our network generalize better on challenging terrains
Figure 5 (c). with steeper slopes than those present in the SemanticKITTI
2) Inpainting is the process of reconstructing lost or dataset, we augment the point cloud by randomly applying
deteriorated parts of images using the information from 3D rotations in the range of (−10, 10) degrees with respect
neighboring pixels. We use a biharmonic function based to the x and y axes. The SemanticKITTI dataset has 11
inpainting and surface completion method [18] implemented annotated sequences; we use 7 sequences for the training
in scikit-image [19] to fill the holes in the elevation map. set and 4 sequences for the validation set. The final dataset
Inpainting resulted in the far superior interpolation of ground consists of 6584 frames for training and 3040 frames for
elevation than just averaging over neighboring cells—see validation.
Figure 5 (e). Finally, to further smoothen small crests and
troughs we apply an average filter over the entire inpainted B. Loss Function
elevation map, Figure 5 (f). We perform end-to-end training of the full model including
pillar feature encoding layer and encoder-decoder module
CRF-based Method: Another more mathematically consis- using a combination of regression loss Lreg and spatial-
tent approach for surface reconstruction is using CRFs. We smoothing loss Lsmooth :
used a similar CRF model as proposed in [3] to model
the ground plane. The CRF based method uses temporal ˆ = αLreg (I, I)
L(I, I) ˆ + βLsmooth (I)
ˆ (1)

2153
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 where I denotes the ground truth elevation map of the grid
with shape (H, W ), and Iˆ denotes the predicted elevation
map. The hyper-parameters α and β are used for balancing
the two losses. For the regression of elevation values, we
use the Huber loss. For spatial smoothness, we minimize
the L1 norm of the second-order gradients for the predicted
elevation maps (similar to [20]). We penalize the norm of
second-order gradients across adjacent cells to encourage not
constant but rather smoothly changing elevation values:

ˆ = ∇2x Iˆ + ∇x ∇y Iˆ + ∇y ∇x Iˆ + ∇2y Iˆ
Lsmooth (I) (2)
Fig. 7. Qualitative comparison of ground elevation estimation in case of
where ∇x and ∇y are, respectively, the gradients in the x occluded areas (yellow). (Left) Output from CRF-method [3]; (Right) Output
and y direction of the elevation map. from the proposed method GndNet.

V. N ETWORK AND T RAINING D ETAILS

VI. R ESULTS
A. Network Real-time accurate ground plane estimation and semantic
segmentation of point clouds is a challenging task. We
We do not pre-train our networks, all weights were initial-
compare the proposed method with several top-performing
ized randomly using a uniform distribution. Our simplified
approaches, including a CRF-based method [3], Hybrid-
PointNet in the pillar feature encoding network has one
regression [15] and a CNN-method [16]. At the time of
fully connected layer with C = (9, 64) input-output features.
writing this work, no ground elevation dataset was available
In the encoder-decoder architecture, all the convolutional
publicly. Most previous approaches only reported qualitative
layers have kernel size 3, and stride and padding 1. Each
results, while others evaluated their approach on small cus-
convolutional layer is followed by batch normalization and
tom datasets, providing no common basis for quantitative
element-wise rectified-linear non-linearity (ReLU) except for
comparison.
the last regression layer. Max-pooling and max-unpooling are
In this work, we first deal with the challenge of estab-
performed with a 2×2 window and stride 2. The specifics
lishing a common basis for a quantitative evaluation of the
of the encoder-decoder network design with input-output
ground estimation method. We use the publicly available
channels for the convolutional layers are: Conv1(64, 128),
SemanticKITTI dataset to generate a ground elevation dataset
Conv2(128, 128), Max-pool, Conv3(128, 256), Conv4(256,
as described in subsection IV-A. We perform a qualitative
256), Max-pool, Max-unpool, Conv5(256, 256), Conv6(256,
and quantitative evaluation on this dataset and use the CRF-
128), Max-unpool, Conv7(128, 64), Conv8(64, 64), and
based method [3] as our baseline. For ease of comparison, we
regression layer Convreg (64, 1).
follow the same color schematic and representation of ground
elevation as in [3]. We perform two different quantitative
B. Training Settings evaluations to show the effectiveness of our approach:
GndNet takes a raw point cloud with approximately A. Quantitative Evaluation
100,000 points as input. We discretize the environment into
Ground plane estimation: To evaluate the ground plane
a 2D grid of size (100, 100) meters. The x, y, z range is
estimation we use the Root Mean Squared Error (RMSE)
[(−50, 50), (−50, 50), (−4, 4)] meters, and we remove all the
metric. We calculate the RMSE for all the cells that contain
points outside that range. The maximum number of points
ground points. We use the elevation map in Figure 5 (d)
per pillar (N) is kept at 100. We keep the cell resolution of
as ground truth for elevation values. We only qualitatively
1m x 1m as it provides us better trade-off between accuracy
evaluate the cells that do not have ground points as only
and run-time. Compared to the PointPillars, which keeps the
interpolated elevation values are available for these regions.
cell resolution of 0.2 m is very small our task as it results
Ground elevation estimation results for GndNet are shown
in very few ground points per cell, and most of the cells
in Table I.
being empty. The hyper-parameters α and β were set to 0.9
and 0.1 respectively. We use the stochastic gradient descent TABLE I
(SGD) optimizer to train the model with a momentum of E VALUATION OF G ROUND P LANE E STIMATION
0.9, weight decay of 0.0005 and batch size 2. The learning
Method Dataset Frames RMSE (m)
rate is kept 0.01 for the first 40 epochs and then lowered it Hybrid-reg [15] custom 10 0.182
to 0.001 for further epochs. We observed that the network CRF-method[3] SemKITTI 3040 0.201
converges in roughly 120 epochs. Training on our custom GndNet SemKITTI 3040 0.195
ground elevation dataset takes 6 to 8 hours to converge using
an Nvidia GTX 1080 GPU. We use the PyTorch [21] machine Segmentation of ground points: To evaluate the segmen-
learning framework for the development of our model. tation of the point clouds we use the mean Jaccard or

2154
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 t

Fig. 8. Ground elevation estimation by GndNet: Two diverse scenes with uphill and downhill slopes. GndNet outputs smooth and uniform ground
elevation even in the regions with very sparse points up to the distance of 50m. GndNet equally perform well in occluded regions as the network not only
uses local features for regressing elevation values but also the spatial features from the points in nearby areas.

Fig. 9. Point cloud segmentation using GndNet: Diverse scenes with crossroads, urban environment, slopes, vehicles, and pedestrians. GndNet learns
to distinguish points that are relevant for ground elevation estimation. We segment point cloud in two categories, the points below the estimated elevation
values as ground points, those above as non-ground points. The first column shows failure cases with inconsistencies with ground segmentation, some
points on the car in (a) are wrongly segmented as the ground while in (d) some road points are segmented as non-ground.

TABLE II
intersection-over-union (mIoU) over two classes: ground and
E VALUATION OF P OINT C LOUD S EGMENTATION
non-ground.
C
Method Dataset Frames Prec Recall mIoU
1 X T Pc Hybrid-reg [15] custom 10 0.98 - -
mIoU = (3) CNN-method [16] custom 252 0.929 0.993 -
C c=1 T Pc + F Pc + F Nc CRF-method [3] SemKITTI 3040 0.801 0.993 0.782
GndNet SemKITTI 3040 0.841 0.993 0.836
where, T Pc , F Pc and F Nc correspond to the number of
true positive, false positive, and false-negative predictions
for class c, and C is the number of classes. We also provide time for GndNet is given in Table III. Note that the pre-
precision and recall value and compare those with other processing and point segmentation task is performed on CPU
approaches in Table II. GndNet shows comparable results using Numpy and Numba which could be further accelerated
with other approaches. Hybrid-reg [15] and CNN-method using GPU.
[16] report very high precision but their approaches are
evaluated on a limited set of data. TABLE III
Inference: The GndNet is a lightweight network and A NALYSIS OF COMPUTATION TIME REQUIRED BY G ND N ET
achieves runtime performance of 55 Hz compared to CRF-
Task Time Device
method [3] of 9.8 Hz for processing ≈ 100, 000 points in the
Pre - processing 7.59 ms CPU
grid size of 100x100m. The Hybrid-reg [15] approach is not Model forward pass 6.57 ms GPU
real-time, while the CNN-method [16] reports a runtime of Point segmentation 3.88 ms CPU
140Hz but only performs the segmentation task (the ground Total 17.98 ms (55.61 Hz)
elevation is not estimated). Detailed analysis of the inference

2155
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.
 B. Qualitative Evaluation [3] L. Rummelhard, A. Paigwar, A. Nègre, and C. Laugier, “Ground esti-
mation and point cloud segmentation using spatiotemporal conditional
The illustrations in Figure 8 and Figure 9 show the random field,” in 2017 IEEE Intelligent Vehicles Symposium (IV).
ground elevation estimation and point cloud segmentation by IEEE, 2017, pp. 1105–1110.
GndNet in diverse and challenging scenarios. The main ad- [4] Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud
based 3d object detection,” in Proceedings of the IEEE Conference on
vantage of GndNet is in occluded regions with no data points Computer Vision and Pattern Recognition, 2018, pp. 4490–4499.
and regions with sparse data. Figure 7 depicts a scenario [5] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom,
where a vehicle in front occludes a section of the ground “Pointpillars: Fast encoders for object detection from point clouds,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern
and part of a wall. The competing CRF-based method uses Recognition, 2019, pp. 12 697–12 705.
the lowest points in the cells to estimate ground elevation; for [6] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning
the cells that do not have any points, the ground elevation on point sets for 3d classification and segmentation,” Proc. Computer
Vision and Pattern Recognition (CVPR), IEEE, vol. 1, no. 2, p. 4,
values are interpolated using values from the neighboring 2017.
cells. As in Figure 7, the CRF-method incorrectly estimates [7] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stach-
ground elevation as a steep slope starting from the bottom niss, and J. Gall, “SemanticKITTI: A Dataset for Semantic Scene
Understanding of LiDAR Sequences,” in Proc. of the IEEE/CVF
of the vehicle to the visible part of the wall and continues International Conf. on Computer Vision (ICCV), 2019.
further. GndNet rather learns the appearances, analyzes the [8] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for Autonomous
scene to accurately estimate ground elevation. Driving? The KITTI Vision Benchmark Suite,” in Proc. of the IEEE
Conf. on Computer Vision and Pattern Recognition (CVPR), 2012, pp.
VII. C ONCLUSIONS 3354–3361.
[9] S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron,
In this paper, we tackled the challenging problem of J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, et al., “Stanley:
The robot that won the darpa grand challenge,” Journal of field
ground plane estimation and point cloud segmentation for Robotics, vol. 23, no. 9, pp. 661–692, 2006.
intelligent vehicles. We presented a deep learning-based [10] M. Himmelsbach, F. v. Hundelshausen, and H. J. Wuensche, “Fast
approach called GndNet, which uses PointNet and Pillar segmentation of 3d point clouds for ground vehicles,” in Intelligent
Vehicles Symposium (IV), 2010 IEEE, June 2010, pp. 560–565.
Feature encoding networks to learn point features and output [11] C. Guo, W. Sato, L. Han, S. Mita, and D. McAllester, “Graph-based
ground elevation in a grid-based representation. We provide 2d road representation of 3d point clouds for intelligent vehicles,” in
a qualitative and quantitative evaluation of our approach on Intelligent Vehicles Symposium (IV), 2011 IEEE, June 2011, pp. 715–
721.
a large dataset and compare our results with other competing [12] J. Byun, K.-i. Na, B.-s. Seo, and M. Roh, Drivable Road Detection
approaches. GndNet achieves comparable results in terms of with 3D Point Clouds Based on the MRF for Intelligent Vehicle.
the accuracy and establishes a new state-of-the-art in terms Cham: Springer International Publishing, 2015, pp. 49–60. [Online].
Available: [Link] 4
of the run-time of 55Hz for ground plane estimation and [13] M. Zhang, D. D. Morris, and R. Fu, “Ground segmentation based on
ground points segmentation. loopy belief propagation for sparse 3d point clouds,” in 3D Vision
Currently, for training, we use a dataset derived from (3DV), 2015 International Conference on, Oct 2015, pp. 615–622.
[14] P. Narksri, E. Takeuchi, Y. Ninomiya, Y. Morales, N. Akai, and
the SemanticKITTI dataset using a CRF-based method. This N. Kawaguchi, “A slope-robust cascaded ground segmentation in 3d
derived dataset can be inconsistent in certain scenarios and point cloud for autonomous vehicles,” in 2018 21st International
using a carefully annotated dataset by hand can improve Conference on Intelligent Transportation Systems (ITSC). IEEE,
2018, pp. 497–504.
the accuracy of the proposed network. Also, our proposed [15] K. Liu, W. Wang, R. Tharmarasa, J. Wang, and Y. Zuo, “Ground sur-
network uses only instantaneous data and spatial features. In face filtering of 3d point clouds based on hybrid regression technique,”
the future, we plan to use temporal information and features IEEE Access, vol. 7, pp. 23 270–23 284, 2019.
[16] M. Velas, M. Spanel, M. Hradis, and A. Herout, “Cnn for very fast
collected from previous frames; this could potentially im- ground segmentation in velodyne lidar data,” in 2018 IEEE Interna-
prove the ground elevation results in the cases of occluded tional Conference on Autonomous Robot Systems and Competitions
areas and regions with sparse points. Another line of future (ICARSC). IEEE, 2018, pp. 97–103.
[17] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep
work could be directly segmenting point cloud as a predicted convolutional encoder-decoder architecture for image segmentation,”
output by the network rather than as a post-processing step. IEEE transactions on pattern analysis and machine intelligence,
vol. 39, no. 12, pp. 2481–2495, 2017.
ACKNOWLEDGMENT [18] S. Damelin and N. Hoang, “On surface completion and image in-
painting by biharmonic functions: Numerical aspects,” International
This work was conducted at Inria, team Chroma. The Journal of Mathematics and Mathematical Sciences, vol. 2018, 2018.
authors would like to thank all the team members for their [19] S. van der Walt, J. L. Schönberger, J. Nunez-Iglesias, F. Boulogne,
constant support on this research work. This work has been J. D. Warner, N. Yager, E. Gouillart, T. Yu, and the scikit-image
contributors, “scikit-image: image processing in Python,” PeerJ, vol. 2,
conducted within the scope of ES3CAP (Embedded Smart p. e453, 6 2014. [Online]. Available: [Link]
Safe Secure Computing Autonomous Platform) project. [20] T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, “Unsupervised
learning of depth and ego-motion from video,” in Proceedings of the
R EFERENCES IEEE Conference on Computer Vision and Pattern Recognition, 2017,
pp. 1851–1858.
[1] J. Leonard, J. How, S. Teller, M. Berger, S. Campbell, G. Fiore, [21] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan,
L. Fletcher, E. Frazzoli, A. Huang, S. Karaman, et al., “A perception- T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf,
driven autonomous urban vehicle,” Journal of Field Robotics, vol. 25, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner,
no. 10, pp. 727–774, 2008. L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-
[2] L. Rummelhard, A. Nègre, and C. Laugier, “Conditional monte carlo performance deep learning library,” in Advances in Neural Information
dense occupancy tracker,” in 2015 IEEE 18th International Conference Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer,
on Intelligent Transportation Systems, Sept 2015, pp. 2485–2490. F. dAlché-Buc, E. Fox, and R. Garnett, Eds. Curran Associates, Inc.,
2019, pp. 8024–8035.

2156
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on June 05,2023 at 10:05:48 UTC from IEEE Xplore. Restrictions apply.