Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Contents lists available at ScienceDirect

Engineering Applications of Artificial Intelligence

journal homepage: [Link]/locate/engappai

Research paper

Real-time point-cloud-based vehicle fleet monitoring system on the edge,

edge computing, and deep learning technique
Tun Jian Tan a , Zhaoyu Su b , Jun Kang Chow a , Tin Long Leung a , Pin Siang Tan a ,
Mei Ling Leung a , Wai Yin Gavin Wu a , Hai Yang a , Dasa Gu a , Yu-Hsing Wang a ,∗
a
Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
b Intelligent Automotive Solution, Huawei Technologies. Co., Ltd., Shenzhen, Guangdong, China

ARTICLE INFO ABSTRACT

Keywords: Intelligent Transportation Systems (ITS) increasingly rely on real-time 3-Dimensional (3D) data for vehicle
Intelligent transportation systems dynamics, making edge computing crucial for timely and scalable 3D object detection, vehicle tracking, and
3-dimensional object detection counting. This paper introduces a real-time vehicle fleet monitoring system operating at 10 Hertz with a latency
Vehicle monitoring
of 62.71 ms using advanced hardware and software. The primary objective of ‘‘fleet monitoring’’ is accurate
Real-time edge computing
vehicle counting and classification across diverse vehicle types. The hardware consists of a multi-beam flash
Light Detection and Ranging (LiDAR), and an edge computing device. The software architecture is comprised
of four modules: (1) a Graphics Processing Unit (GPU)-accelerated sensor interface module, which effectively
processes high-density LiDAR point clouds by filtering out irrelevant background data and extracting regions
of interest (RoIs); (2) Dynamic Voxelization Detector (DV-Det), a 3D object detection model which identifies
and categorizes various vehicle types. It demonstrated exceptional performance, achieving 75 Hertz on the
Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset and surpassing bird’s eye
view (BEV)-based methods in all evaluation metrics; (3) A-Baseline-for-3D-Multi-Object-Tracking (AB3DMOT)
algorithm, a robust multi-vehicle tracking module; (4) a specialized multi-vehicle counting algorithm, tailored
for accurate vehicle enumeration in 3D environments. Field experiments validate the system’s capability to
perform fine-grain vehicle classification with high precision and recall, achieving at least 80 Mean Average
Precision (mAP) at a 3D Intersection-over-Union (IoU) threshold of 0.5 for most vehicle classes, along with a
recall of 100% and a precision of at least 80% for the majority of the 10 fine-grain vehicle classes.

1. Introduction 2022; Sankari and Meena, 2010; Yang and Qu, 2018; Zhang et al.,
2021; Rezaei et al., 2022), radar (Sánchez-Oro et al., 2013; Tan et al.,
Vehicle detection is not only a critical component of Intelligent 2024), and Light Detection and Ranging (LiDAR) (Zhang et al., 2020;
Transportation Systems (ITS) but also serves a variety of applica- Zhao et al., 2019; Tan et al., 2024), offer broader area coverage
tions such as emission inventory estimation (Ghaffarpasand et al., and simpler installation, often utilizing existing infrastructure such as
2020), geographic information system (GIS) integration (Tay et al., poles or traffic lights. Video cameras have become prevalent in non-
2023), and Vehicle-to-Everything (V2X) communication (Adnan Yusuf intrusive vehicle sensing, particularly with the aid of advancements
et al., 2024). Vehicle detection technologies are broadly classified in computer vision (Zaidi et al., 2022). In addition, the advent of
into intrusive and non-intrusive methods. Intrusive methods, including deep learning techniques has significantly enhanced the capabilities of
pneumatic tube counters (Che Puan et al., 2019; Tan et al., 2024), computer vision, with models such as Single Shot Detector (SSD) (Liu
piezoelectric sensors (Zhang et al., 2015; Tan et al., 2024), and in- et al., 2016), You Only Look Once (YOLO) (Redmon et al., 2016),
duction loops (Padrón et al., 2023; Tan et al., 2024), are installed and their iterations (Redmon and Farhadi, 2017, 2018) demonstrating
within the road infrastructure to measure vehicle speed and volume superior performance in real-time traffic surveillance systems. These
at specific locations. Although effective, these methods often involve advances have outperformed traditional object detection and classi-
higher installation and maintenance costs due to their subsurface place- fication methods in various benchmarks (Everingham et al., 2010;
ment and limited spatial coverage. In contrast, non-intrusive methods, Geiger et al., 2012; Sun et al., 2020; Caesar et al., 2020). However,
such as video cameras (Brahme and Kulkarni, 2011; Trivedi et al.,

∗ Corresponding author.
E-mail address: ceyhwang@[Link] (Y.-H. Wang).

[Link]
Received 21 June 2024; Received in revised form 11 March 2025; Accepted 13 June 2025
Available online 2 July 2025
0952-1976/© 2025 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 1. Real-time point-cloud-based vehicle monitoring system.

the rise of autonomous driving technologies has underscored the im- the high data rates of LiDARs, the sheer volume of raw data generated
portance of accurate 3-Dimensional (3D) information for safety-critical by a network of sensors in a city is immense. In particular, transmitting
applications. this data to the cloud for processing becomes impractical. A more
While camera-based systems are more economical for basic traffic feasible approach is to process the data at the edge, transmitting only
monitoring, they face fundamental technical limitations in obtaining essential high-level information, such as vehicle class, position, size,
accurate measurements. Speed calculations using cameras require com- and velocity, to the cloud for further usage by downstream applications.
plex calibration procedures with known reference objects, and measure- Real-time edge computation not only reduces latency for time-critical
ment accuracy is heavily dependent on viewing angles and perspective applications like V2X but also conserves bandwidth for data trans-
transformations. Vehicle dimension measurements from cameras can mission (Yang et al., 2024). However, the point cloud data produced
vary significantly with viewing angle, particularly problematic at low by LiDARs are often sparse and unstructured, necessitating complex
angles of attack, where bounding-box estimations become unreliable. processing that can be computationally intensive (Bello et al., 2020).
Additionally, handling occlusions in complex traffic scenarios requires Deep learning models, which are highly parallelizable on Graphics
multiple cameras with sophisticated calibration procedures, increasing Processing Units (GPUs), have shown potential for real-time operation
both system complexity and maintenance requirements. These limi- on powerful computing units such as the RTX 2080Ti and A100
tations become particularly critical in applications requiring precise GPUs. However, deploying these models on resource-constrained edge
vehicle classification and speed measurements, such as emission inven-
computing devices presents significant challenges.
tory estimation and V2X communication, where measurement errors
Fine-grain vehicle classification is another crucial aspect of vehicle
can significantly impact system effectiveness.
fleet monitoring systems, distinguishing between vehicles of similar
3D traffic data, including vehicle speed, dimensions, and positions,
size based on additional characteristics like geometric appearance and
can be integrated into a geospatial indexing system, allowing for a
occupancy. While point cloud data has been widely used for vehicle
unified view of traffic conditions across multiple intersections and
detection in autonomous driving (Geiger et al., 2012; Sun et al., 2020;
roads. The 3D traffic data and 3D traffic scene captured by the LiDAR
Caesar et al., 2020), the focus in that domain has been more on spatial
is able to be georeferenced to form a fully connected grids of traffic
occupancy and object dynamics, with less emphasis on vehicle classifi-
network that generates insight in low latency. This real-time 3D traffic
data that has information such as vehicle type, vehicle speed and cation. Ref. Tan et al. (2024) covers a comprehensive list of vehicle
vehicle location could benefit applications such as better estimation of classification approaches that utilized various sensing technologies.
traffic volume (Tay et al., 2023) on a larger scale. It highlights the ITS applications (Liu et al., 2020; Li et al., 2021;
LiDAR sensors therefore have emerged as the most effective non- Rezaei et al., 2022) that performs detailed vehicle classification on
intrusive tools for capturing the environment in three dimensions LiDAR data. For instance, Li et al. (2021) classified trucks based on the
(Zhang et al., 2020; Zhao et al., 2019), despite their higher initial number of wheel axles, but their method was limited to single-vehicle
costs. Unlike cameras, which rely on complex operations like inverse detection and classification. Hence, need to have an efficient model
perspective mapping and distance estimation between satellite images for fine-grain vehicle classification is evident. The existing point cloud
and roadside surveillance camera images (Rezaei et al., 2022), LiDARs detection methods using deep learning algorithms can be classified into
provide native 3D measurements of their surroundings, enabling direct two categories: grid-based methods and point-wise methods. Point-wise
calculation of vehicle positions, dimensions, and speeds without elabo- methods excel at learning precise point-wise feature representations;
rate calibration processes. Multiple LiDAR units can be easily combined however, their reliance on multi-layer perceptrons (MLPs) often re-
to create comprehensive point clouds that effectively handle occlusion, sults in heavy network architectures (Qi et al., 2017a,b; Shi et al.,
while their consistent performance across varying environmental con- 2019a). In contrast, grid-based methods are generally more efficient.
ditions reduces long-term maintenance requirements. However, given However, they encounter the issue of feature dilution, where feature

2
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 2. Real-time vehicle monitoring system on edge using deep learning technique and point cloud data.

maps at higher network layers often have low resolution, hindering the distinct vehicle classes based on Environmental Protection De-
network’s ability to learn precise local feature representations (Lang partment (EPD) Hong Kong specifications. This granular classi-
et al., 2019; Yan et al., 2018). Our work focuses on performing vehi- fication enables more accurate estimation of emission inventory
cle detection and classification simultaneously through deep learning by precisely identifying vehicle types and their corresponding
approaches. emission factors, improved traffic flow analysis through detailed
To address the challenges in vehicle fleet monitoring, this paper vehicle type distribution data, and improved traffic management
introduces a system that integrates hardware and software components, strategies tailored to specific vehicle types. This addresses a
as depicted in Fig. 1. The hardware setup consists of a multi-beam critical gap in existing LiDAR-based vehicle detection systems
flash LiDAR and an edge computing device. The software leverages that often focus solely on spatial occupancy and object dynamics.
a cutting-edge deep learning model to process 3D point cloud data 3. Development of an efficient edge processing pipeline featuring
a GPU-accelerated sensor interface module that significantly
in real-time on edge devices. The data processing pipeline, illustrated
reduces data processing requirements, the novel DV-Det model
in Fig. 2, includes four key modules: a sensor interface and the deep
achieving 75 frames-per-second (FPS) on the Karlsruhe Insti-
learning model, Dynamic Voxelization Detector (DV-Det), both GPU-
tute of Technology and Toyota Technological Institute (KITTI)
accelerated, as well as two Central Processing Unit (CPU)-efficient
dataset (Geiger et al., 2012) while maintaining high accuracy,
modules for multi-vehicle tracking and counting. The sensor interface
and efficient multi-vehicle tracking and counting algorithms.
module utilizes statistical background filtering and point-in-polygonal This integrated approach enables real-time processing on
prism cropping to streamline the input for DV-Det. This novel deep resource-constrained edge devices, making widespread deploy-
learning model, DV-Det, combines the precision of point-wise methods ment of advanced vehicle monitoring systems practically feasi-
with the efficiency of grid-based approaches, avoiding their respective ble.
drawbacks. The A-Baseline-for-3D-Multi-Object-Tracking (AB3DMOT) 4. Introduction of DV-Det, which combines grid-based (Chen et al.,
tracking algorithm (Weng et al., 2020) monitors vehicles over time, 2017; Zhou and Tuzel, 2018; Yan et al., 2018) and point-
while a unique 3D multi-vehicle counting algorithm capitalizes on the wise methods (Qi et al., 2017a) with dynamic voxelization and
spatial separability of vehicles in 3D space to count and refine vehicle location-aware region-of-interest (LA-RoI) pooling. The model
class predictions, considering 10 fine-grain vehicle classes in this study. achieves three times faster processing than state-of-the-art 3D
It is important to clarify the scope and boundaries of our vehicle perceptron methods while maintaining superior accuracy com-
fleet monitoring approach. In this context, ‘‘vehicle fleet monitoring’’ pared to bird’s-eye view (BEV)-based methods. These improve-
refers to the detection, classification, counting, and basic tracking ments enable practical applications like real-time V2X commu-
of individual vehicles passing through monitored areas, rather than nication and dynamic traffic management that were previously
tracking specific groups of vehicles moving as coordinated units. The infeasible with slower or less accurate systems.
tracking component serves primarily to support accurate counting and 5. Comprehensive validation through on-site deployment demon-
classification by preventing double-counting and enabling temporal strates 80 Mean Average Precision (mAP) at a 3D Intersection-
class label smoothing, rather than analyzing inter-vehicle relationships over-Union (IoU) threshold of 0.5 for most vehicle classes, 100%
or formation patterns. This targeted approach aligns with practical ITS recall, and at least 80% precision for most vehicle classes, with
applications that require accurate vehicle counts and classifications, reliable operation under various weather conditions and traf-
fic patterns. These results confirm the system’s readiness for
such as emission inventory estimation and traffic flow analysis.
practical deployment in real-world ITS applications.
The primary contributions of this paper are as follows:
The remainder of the paper is structured as follows: Section 2
1. Development of a comprehensive real-time 3D vehicle monitor- reviews recent literature on vehicle monitoring systems that utilize
ing system that operates at the edge, achieving 10 Hz processing video, aerial cameras, and LiDARs, as well as fine-grain vehicle classifi-
with just 62.71 ms latency. This enables immediate traffic in- cation from point cloud data and real-time traffic monitoring systems.
sights and supports critical downstream applications, including Section 3 offers a detailed description of our real-time point cloud-
the estimation of emission inventory, the integration of GIS based vehicle fleet monitoring system. Section 4 presents the evaluation
and the communication with V2X, representing a significant results of DV-Det. Sections 5 and 6 detail the on-site experiment and
advancement in the practical deployment of ITS. discuss the results, respectively. Section 7 concludes the paper, summa-
2. To the best of our knowledge, it is the first attempt to introduce rizing the main findings and suggesting potential directions for future
fine-grain vehicle detection in 3D point clouds, distinguishing 10 research.

3
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

2. Literature review 2.2. 2D-based aerial camera video images

Vehicle detection is an essential component of ITS, serving as a To address the issue of occlusion, some research efforts have instead
cornerstone for subsequent ITS tasks such as vehicle classification, proposed the use of aerial images for vehicle detection, where a camera
tracking, counting, and speed estimation. In essence, a robust vehicle is mounted on an unmanned aerial vehicle (UAV) to capture video
detection algorithm is a prerequisite for these downstream applications. images from a BEV. This setup offers a wide field of view and minimal
Non-intrusive sensors are increasingly favored for vehicle detection occlusion, assuming that the UAV is positioned at an optimal altitude
due to their ability to provide extensive measurement coverage when above the road. For example, Cao et al. (2011) developed a support
configured with appropriate sensor placement and orientation. The vector machine to detect and classify moving vehicles on roads using
related work in the literature on these topics is briefly reviewed in the aerial images collected from low-altitude airborne platforms. Aerial
following sections. images, as opposed to roadside surveillance cameras, tend to yield
superior results for vehicle detection and subsequent vehicle trajectory
2.1. 2-Dimensional (2D)-based roadside camera video images analysis. However, the cost of operating an UAV for extended periods
makes it impractical for continuous traffic monitoring.
Video cameras, utilizing non-intrusive sensing technologies, have
2.3. 3D-based CV on roadside camera video images
been extensively deployed, leading to the development of a range of
classical image processing techniques for vehicle detection from im-
Despite the use of surveillance cameras located on the roadside
ages. Background subtraction, also known as the motion-based method,
or mounted on UAVs, the outputs of the aforementioned algorithms
is one of the common techniques. This approach assumes that ev-
are fundamentally 2D and lack the accurate 3D spatial information
erything in the images is static except for the vehicles in a traffic
of vehicles necessary for ITS applications. There are limited studies
scene (Brahme and Kulkarni, 2011; Sankari and Meena, 2010; Yang
related to CV-based 3D vehicle detection using surveillance cameras.
and Qu, 2018; Zhang et al., 2021; Trivedi et al., 2022). A reference
For example, Rezaei et al. (2022) introduced Traffic-Net, which com-
background model is constructed to detect moving vehicles that sig-
bines images from roadside surveillance cameras with satellite images
nificantly deviate from this model, based on a predefined threshold.
for 3D vehicle detection. This process enables the decoding of 3D
Following background subtraction, vehicle detection is performed on
world scene information that has been encoded as a 2D perspective
the foreground, which consists of multiple blobs representing the target
image, thereby retrieving the 3D attributes. However, this approach
objects. The effectiveness of subsequent vehicle detection is highly
required extensive calibration to determine the intrinsic and extrinsic
dependent on the quality of background subtraction, specifically the
camera parameters necessary to perform inverse perspective mapping
amount of noise in the extracted foregrounds. Although background
and distance mapping.
subtraction can be executed rapidly, it is often impractical in real-world
traffic scenarios due to the wide variations in lighting and the presence
2.4. 3D-based CV on LiDAR point clouds
of irrelevant moving objects, such as the swaying of tree branches.
Other research efforts bypass background subtraction and directly LiDAR presents an appealing alternative for 3D vehicle detection
perform vehicle detection based on visual features, such as color, due to its capability to directly capture objects in 3D with high spatial
texture, edge, and shape. For instance, Chhadikar et al. (2019) de- accuracy and temporal resolution. Thus, this approach is also beneficial
veloped a Haar Cascade Classifier for vehicle detection. In addition, for subsequent vehicle classification, tracking, counting, and speed
the significant advancements in artificial intelligence (AI), particularly estimation.
in deep learning, have markedly enhanced the performance of com- Many existing deep learning algorithms for 3D vehicle detection
puter vision (CV)-based vehicle detection. Relevant works (Feng et al., from LiDAR point clouds have evolved from 2D object detection from
2018; Chhadikar et al., 2019; Yao, 2019; Abdelwahab, 2019; Song images. These methods often involve projecting 3D point clouds onto
et al., 2019; Liu et al., 2020; Li et al., 2021) typically employ gen- 2D images in BEV (Ali et al., 2018; Beltrán et al., 2018; Simony et al.,
eral 2D object detection models, such as Region-based Convolutional 2018; Yang et al., 2018b,a; Lang et al., 2019; Ye et al., 2020) and
Network method (RCNN) (Girshick et al., 2014), Faster R-CNN (Ren front view (Li et al., 2016; Minemura et al., 2018), and then applying
et al., 2015), SSD (Liu et al., 2016), YOLO (Redmon et al., 2016), 2D convolutional neural networks (CNNs) for vehicle detection. Other
YOLOv2 (Redmon and Farhadi, 2017), and YOLOv3 (Redmon and approaches have attempted to encode LiDAR point clouds into regular
Farhadi, 2018). Compared to classical image processing techniques, voxels, which are then used as inputs for 3D vehicle detection (Li,
deep learning-based object detectors demonstrate greater robustness 2017; Zhou and Tuzel, 2018; Yan et al., 2018; Ye et al., 2023; Fan
and generalization across a wide range of road scenes, accounting for et al., 2022; Deng et al., 2021). However, these 3D vehicle detection
variations in illumination, shadows, and partial occlusions in videos. algorithms were designed primarily based on the KITTI dataset for
It is important to note that vehicle type classification is sometimes autonomous driving platforms and did not consider the upstream task
integrated into the detection task. For example, Song et al. (2019) of processing raw LiDAR point clouds or the downstream task of
utilized YOLOv3 to detect vehicles on highways and simultaneously generating traffic insights, e.g., identifying different types of vehicle
classify them into categories such as cars, buses, and trucks. Liu et al. and associated speeds.
(2020), and Tourani and Shahbahrami (2015) classified individual ve- Examples of LiDAR-based ITS applications include the following:
hicles based on the width and length of the predicted bounding boxes. Zhang et al. (2020) removed the furthest points, presumed to be static
However, vehicle detection using images from roadside surveillance background, and then applied the Euclidean cluster extraction algo-
cameras has limitations. Partial or complete occlusion of vehicles, rithm to isolate individual clusters of point clouds representing moving
especially those located at a distance, is sometimes unavoidable. Since objects. The cluster features were encoded as a 1D vector and used
the field of view of surveillance cameras is typically limited, multiple as inputs for a support vector machine and random forest to classify
cameras may be required to cover a test site, necessitating complex objects as vehicles or non-vehicles. The detected vehicles were then
calibration to align all raw images to the same reference plane. More tracked for speed estimation. Zhao et al. (2019) utilized density-based
critically, depth information (distance from the camera) is absent from spatial clustering of applications with noise (DBSCAN) on background-
camera data, unless a specific calibration is performed to establish filtered point clouds to identify clusters representing target objects. The
the relationship between pixel distance and real-world distance from total number of points, the scanning distance, and the direction of the
certain perspectives. clusters were used as input for a single-layer neural network for the

4
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 3. The three main types of convolution kernels used in point-cloud based 3D object detection models. The yellow kernels represents voxel and point convolution operations.
The red feature tiles and points represents the centroid of the convolutional kernels. The blue feature tiles and points are the neighboring feature tiles and points that partakes in
the convolution.

binary classification of pedestrians and vehicles. The identified vehicles 2020), where both approaches are alternated at the framework level to
were tracked for speed and moving direction estimation. Li et al. (2021) optimize performance. Despite achieving impressive accuracy, heavy
applied background subtraction and DBSCAN to identify target object MLPs are still used in Shi et al. (2020), and inference speed remains
clusters. The point cloud reconstruction was then performed using a constraint. Currently, mainstream point cloud detection methods
points belonging to the same truck in consecutive frames, followed by operate at around 20 FPS for the KITTI dataset (with a few reaching
the use of PointNet (Qi et al., 2017a) for the truck classification. up to 45 FPS1 Lang et al. (2019)) on a high-computing power server
These Refs. Zhao et al. (2019), Zhang et al. (2020) and Li et al. GPU, covering a 90-degree horizontal front-view field of view (FOV).
(2021) remove unwanted point cloud regions from raw data prior to This speed might be insufficient for real-world scenarios with a full
vehicle detection, as the number of points in a single LiDAR frame is 360-degree FOV and a LiDAR sampling rate of 10 Hz, considering that
vast, and the scene representations are typically sparse. Moreover, in a edge devices have limited computational resources. Moreover, vehicle
single frame, the vehicle representation from the LiDAR point clouds detection is only a component of a larger application system in the real-
is often incomplete. In such circumstances, clustering methods and world. The detection component should run as fast as it can to share its
machine learning methods struggle to provide satisfactory predictions computational resources and latency constraint with other components
regarding the position, dimensions, and orientation of the bounding box of the application system.
of the detected vehicles in the 3D space. Deep learning methods, on the Hence, to address these issues, we propose a novel method that com-
other hand, generate better predictions as their training objective is to bines the strengths of both grid-based and point-wise methods at the
directly predict the parameters of the bounding box. perceptron level. Illustrated in Fig. 3(c), we first extract key points from
a point cloud and gather neighboring points around each key point,
2.5. 3D point cloud object detection using deep learning algorithms similar to point-wise methods. However, instead of using MLPs to learn
from point coordinates, we employ local-scale 3D voxels to preserve
This section delves into a review of published deep learning algo- the 3D geometry and then apply 3D convolutions to learn features.
rithms for point cloud object detection. This review aims to underscore This approach eliminates the need for costly MLPs and addresses the
the necessity of developing a new model. Generally, existing deep quantization issue (to be further discussed in Section 3.3).
learning algorithms for this task can be categorized into two main
types: grid-based methods and point-wise methods. 2.6. Real-time vehicle monitoring systems
Grid-based methods typically employ a strategy akin to 2D image
processing by converting point clouds into regular grid formats, such The ability to generate traffic insights in real time can significantly
as 3D voxels (Chen et al., 2017; Zhou and Tuzel, 2018; Shi et al., enhance the efficiency of ITS. Hu et al. (2004) highlighted the im-
2019b; Chen et al., 2019; He et al., 2020; Deng et al., 2021) or 2D- portance of high inference speed. Song et al. (2019) and Li et al.
BEV projections (Yan et al., 2018; Yang et al., 2018b,a; Lang et al., (2021) employ single-stage object detection models (e.g., YOLO and its
2019). Following this conversion, 3D or 2D CNNs are applied to ex- variants) as the vehicle detection algorithm in their detection system.
tract features and predict bounding boxes. Nonetheless, this process of However, the image-based system does not generate 3D information.
transforming point clouds into regular grids invariably leads to a loss of However, to deploy LiDAR solutions in ITS, existing cloud-based
information. Due to quantization, feature maps in the higher-level CNN approaches struggle to provide the extremely high network bandwidths
layers often fail to maintain precise regional feature representations, required to rapidly transmit vast amounts of raw point cloud data
as illustrated in Fig. 3(a). This precision is essential for an accurate to a central server, leading to significant latency in generating traf-
regression of the bounding boxes. fic insights. Given the constraints of limited on-site computational
Conversely, point-wise methods, which have evolved from the pio- resources, deep learning-based vehicle detection algorithms must be
neering work of Deep Learning on Point Sets for 3D Classification and lightweight and efficient enough to process traffic data at test sites.
Segmentation (PointNet) (Qi et al., 2017a,b), directly process raw 3D Consequently, only compact traffic data should be transmitted for
LiDAR points and learn point-wise feature representations from point downstream applications. In essence, multiple factors must be con-
coordinates using MLPs (Qi et al., 2018; Shi et al., 2019a; Yang et al., sidered in the development of a traffic monitoring system, so that
2019; Chen et al., 2019; Yang et al., 2020). As depicted in Fig. 3(b), individual ITS edge devices can be effectively scaled up to monitor and
point-wise methods avoid the quantization issues associated with con- manage traffic across an entire city’s road network in real time.
ventional convolutions. However, this benefit comes at a cost: learning In short, to fulfill these requirements, an optimal vehicle fleet
representations from unstructured point coordinates is challenging, and monitoring system needs to provide real-time 3D data that accurately
these methods typically involve expensive operations e.g. Farthest Point
Sampling (Gonzalez, 1985), and KD-Tree (Bentley, 1975), resulting in
slower inference speeds. 1
Without NVIDIA’s TensorRT optimization. The TensorRT inference library
Some attempts have been made to combine grid-based and point- provides a general-purpose AI compiler and an inference runtime that deliver
wise methods, such as in PointVoxel-RCNN (PV-RCNN) (Shi et al., low latency and high throughput for production applications.

5
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 4. The hardware features of LiDAR system. (a) The custom mounting supports roll, pitch, and yaw adjustment for the ease of calibrating the orientation of LiDAR. (b) The
sunshade designed and installed for additional weather protection, preventing overheating from solar radiation.

reflect vehicle dynamics, including exact location, size, type, and speed. On-site deployment environments often have limited resources. An
Therefore, the system should prioritize LiDAR technology over tra- edge computing unit is essential for reducing setup costs, thereby
ditional cameras, supported by an edge computing device for data making the system more economical to scale. The NVIDIA AGX Orin
processing. Additionally, the system should incorporate a streamlined is chosen as the specialized computing unit to deploy deep learning
and efficient data processing pipeline with a rapid and proficient 3D at the edge in this work. It operates at less than 40 Watts (W) and
deep object detector, all engineered to enhance data throughput and is equipped with 1792 NVIDIA CUDA cores, 56 Tensor Cores, an 8-
reduce response time. Core ARM v8.2 64-Bit CPU, 2 megabyte (MB) L2 cache, 4 MB L3
cache, and 32 gigabyte (GB) of 256-bit Low Power Double Data Rate 5
3. Methodology (LPDDR5) memory. Despite the moderate computing capabilities of the
edge device, it can support the operation of a lightweight deep learning
As shown in Fig. 1, our proposed real-time vehicle monitoring model at high speeds without the need for extensive optimization,
system is composed of both hardware and software components. The such as using TensorRT. Moreover, the low thermal dissipation power
hardware includes a LiDAR sensor and an edge computing device. The (TDP) of NVIDIA AGX Orin enables the edge computing unit to adopt
software, namely, the data processing pipeline, consists of four major a fanless design, as illustrated in Fig. 1. The whole edge computing
modules: (1) sensor interface, (2) 3D deep object detector, (3) multi- device is sealed, dust proof and waterproof, resilient for prolonged
vehicle tracker, and (4) multi-vehicle counter. The interconnections usage, offering lower maintenance cost.
between these modules are strategically designed as a pipeline, op-
timizing throughput, and minimizing system latency. The following
3.2. Sensor interface
section will first detail the hardware components and then provide a
comprehensive explanation of each software module.
The sensor interface module is responsible for receiving 3D point
cloud data from the LiDAR sensor and processing it through a series
3.1. Hardware part of the monitoring system
of steps: filtering information, repackaging data, and transmitting it
Given that vehicle classification in this work addresses fine-grain to downstream modules. A point cloud is inherently a collection of
classification challenges, capturing a comprehensive range of vehicle unordered points that represent a sparse version of the scene. For
features is crucial to aid the deep learning model in achieving accurate example, the deep learning model discussed in Section 3.3 processes
classification. As shown in Fig. 1, the OS-2-128 Ouster LiDAR has been the point cloud in both point and voxel representations, and the model
selected for its high resolution and substantial point generation rate. operating speed is inversely proportional to the number of input points.
This multi-beam flash LiDAR utilizes a unique combination of vertical Therefore, to reduce the computational load on downstream modules,
cavity surface emitting lasers (VCSELs) and single photon avalanche the number of points is decreased through filtering operations before
diodes (SPADs), producing a point cloud that is resistant to motion the data is propagated down the system pipeline. Background points are
blur. The OS-2-128 Ouster LiDAR features 128 laser channels, a 360- initially removed through statistical background filtering. The filtered
degree horizontal FOV, a 22.5-degree vertical FOV, a rotation rate of point cloud is then cropped to the size of the effective detection region,
10 Hz, and a detection range of up to 240 meters (m). With a vertical which is determined by the physical installation configuration of the
resolution of 0.18◦ and a horizontal resolution of 0.176◦ , the LiDAR is LiDAR sensor on-site.
capable of generating a dense point cloud that captures high-resolution Fig. 5 illustrates the statistical background filtering process. The
details of vehicles within the scene. To enhance adaptability for various LiDAR sensor, consisting of an array of miniature laser sensors, gener-
monitoring tasks, we have specially designed a Computerised Numeri- ates a repetitive scanning pattern in each captured frame. This pattern
cal Control (CNC) aluminium mount in-house, which is shown in Fig. allows for efficient and effective computation of static background
4(a). This mount allows for adjustments in multiple directions within filtering. The point cloud frame produced by the LiDAR sensor is a
a three-dimensional space. In addition, we have developed a weath- collection of 𝑁 points with raw attributes, beam azimuth angle 𝜙, beam
erproof cover, shown in Fig. 4(b), to ensure durability and protection altitude angle 𝜃, range measurement 𝑟, intensity 𝐼, ambience 𝜓, reflectivity
against environmental conditions. 𝛾. The 𝑥, 𝑦, 𝑧 coordinates of the points are calculated based on the beam

6
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 5. Overview of the statistical background filtering process adopted in the sensor interface.

angles and the range measurement. The point cloud is thus defined the curve travels counterclockwise around the point – is calculated
as 𝑃 = {𝑝𝑖 = [ 𝑥, 𝑦, 𝑧, 𝜙, 𝜃, 𝑟, 𝐼, 𝜓, 𝛾 ]𝑇 }, where 𝑖 ∈ {1, … , 𝑁}. In using Algorithm 1.
the statistical background filtering algorithm, 𝑀 point cloud frames, Points within the polygon have a winding number greater than 0,
𝑃1 , … , 𝑃𝑀 , are collected and identified as containing only background while those outside have a winding number less than 0. Consequently,
information. Due to the repetitive scanning pattern of the LiDAR sensor, points whose 𝑥 and 𝑦 coordinates fall outside the polygon of the
the relationship for every point, 𝑝𝑛 , in point cloud frame 𝑃𝑡 across all RoIs are removed, as are points whose 𝑧 coordinates are outside the
time frames can be expressed as: height range of the RoIs’ polygonal prisms. In addition, considering that
point-in-polygonal-prism cropping is a point-wise operation, it has been
𝜙𝑝𝑖 ,𝑃𝑡 = 𝜙𝑝𝑖 ,𝑃𝑡
1 2 optimized for speed by implementing a parallelized version that utilizes
(1)
𝜃𝑝𝑖 ,𝑃𝑡 = 𝜃𝑝𝑖 ,𝑃𝑡 the GPU on the edge device.
1 2

for 𝑖 ∈ [1, 𝑁], 𝑡1 ∈ [1, 𝑀] and 𝑡2 ∈ [1, 𝑀]. 3.2.1. Comparative study of ground filtering approaches
The 5- to 95-percentile range, 𝑅𝑛 of range attributes of each point In addition to our statistical background filtering and point-in-
𝑝𝑛 , is calculated from all the 𝑀 point cloud frames. The 5- to 95- polygonal-prism cropping, various ground filtering methods exist in the
percentile range is chosen to remove outlier measurements. In deploy- literature. Table 1 presents a comparative overview of some widely
ment, any 𝑝𝑛 lying in 𝑅𝑛 is considered as a background point, therefore, used techniques—both ‘‘simple’’ approaches such as plane detection
will be filtered out. On the other hand, if 𝑅𝑛 of the point 𝑝𝑛 is larger (e.g., RANSAC) and registration-based filtering, and more sophisti-
than the minimum precision of the LiDAR sensor (which is 800 mm for cated pipelines. The table contrasts the approaches in terms of (i)
the Ouster LiDAR used in this work), the point is not filtered. whether they remove only ground points or also off-ground static
In many applications, not all areas captured by the LiDAR sensor clutter (e.g., buildings, signage, vegetation), (ii) time complexity, (iii)
are region-of-interests (RoIs). We introduce point-in-polygonal-prism parallelizability on GPUs, and (iv) typical reported performance. Our
cropping to extract only points within the RoIs, further reducing the method is highlighted in the last row.
number of points. Polygonal prisms define the volumes of interest, and When analyzing time complexity, we denote the number of points
points are filtered in two steps: point-in-polygon filtering and elevation in the point cloud as 𝑁. RANSAC plane fitting (Schnabel et al.,
filtering. The point-in-polygon algorithm from Refs. Weiler (1994) and 2007; Zeineldin and El-Fishawy, 2016) achieves 𝑂(𝑁) complexity.
Hormann and Agathos (2001) is selected for its efficiency and ability to Registration-based filtering (such as Iterative Closest Point (ICP) al-
handle convex and concave polygons. To determine whether a point is gorithm (Besl and McKay, 1992)) typically requires 𝑂(𝐾𝑁𝑙𝑜𝑔𝑁) op-
within a given polygon, the winding number – the total number of times erations, where 𝐾 represents the number of iterations needed for

7
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Table 1
Comparison of different filtering approaches for point cloud processing. Times are approximate; many methods have reported CPU-based
runtimes, whereas our pipeline exploits GPU parallelization.
Criteria RANSAC plane fitting Registration-Based Voxel grid or range Our proposed:
filtering image segmentation Statistical Filtering +
Polygonal Cropping
Ground Yes Yes Yes Yes
Filtering?
Off-ground Static No No (unless extra map No Yes (buildings, roadside
Filtering? differencing) clutter)
Time Complexity 𝑂(𝑁) 𝑂(𝐾𝑁𝑙𝑜𝑔𝑁) 𝑂(𝑁) or 𝑂(𝑠𝐵𝑁) 𝑂(𝑁)
GPU-friendly? Parallelizable Often partial (ICP steps Possible (depending on Highly
are CPU-heavy) BFS) GPU-parallelizable
(point-wise)
Typical Perfor- Good for near-planar Effective if ground is Often used in real-time ∼18.67 ms per frame
mance/Notes roads; can remove parts truly static; limited for object detection (Jetson AGX Orin).
of sloped vehicles large coverage (ground vs. Robust on uneven
non-ground) slopes and clutter

Algorithm 1 Winding number of a point with respect to a clipping on road regions, removing off-ground objects such as vegetations,
polygon. buildings, bus stops, round-about, railing etc. This combined approach
Input: reliably removes both ground and persistent off-ground background. As
𝑝 - point to be evaluated. evidenced by the final row in Table 1, our approach achieves near 𝑂(𝑁)
𝐐 - an array of vertices of closed clipping polygon arranged in counter- complexity on a GPU-equipped Jetson AGX Orin, processing each frame
clockwise order. in 18.67 ms.
Output:
𝑤𝑛 - winding number of the evaluated point 𝑝.
3.3. 3D deep object detector: DV-Det
1: 𝑤𝑛 ← 0
2: 𝑀 ← number of vertices in 𝐐 {/* In a closed polygon, the first vertices in
After the module of sensor interface, a 3D deep object detector,
the array is equal to the last vertices in the array. */}
DV-Det follows (referring to Fig. 2). The DV-Det proposed in this
3: for 𝑖 = 0 to 𝑀 − 1 do
4: 𝑞1 ← 𝐐[𝑖]
study is an efficient and rapid two-stage 3D deep object detector that
5: 𝑞2 ← 𝐐[𝑖 + 1] combines the advantages of both point-wise methods and grid-based
6: if 𝑞1 .𝑦 ≤ 𝑝.𝑦 then methods. DV-Det processes raw input point clouds, while preserving
7: if 𝑞2 .𝑦 > 𝑝.𝑦 then the raw positional information that is often lost in fully grid-based
8: if ((𝑞2 .𝑥 − 𝑞1 .𝑥) × (𝑝.𝑦 − 𝑞1 .𝑦) − (𝑝.𝑥 − 𝑞1 .𝑥) × (𝑞2 .𝑦 − 𝑞1 .𝑦)) > 0 then methods (Chen et al., 2017; Zhou and Tuzel, 2018; Yan et al., 2018;
9: 𝑤𝑛 ← 𝑤𝑛 + 1 Shi et al., 2019b). As illustrated in Fig. 3(a), the transformation of point
10: end if cloud data into uniform grids typically results in information loss, and
11: end if the issue of quantization exacerbates this problem. Consequently, the
12: else feature maps in the upper layers of CNNs frequently fail to capture
13: if 𝑞2 .𝑦 ≤ 𝑝.𝑦 then
the detailed local feature representations essential for precise bound-
14: if ((𝑞2 .𝑥 − 𝑞1 .𝑥) × (𝑝.𝑦 − 𝑞1 .𝑦) − (𝑝.𝑥 − 𝑞1 .𝑥) × (𝑞2 .𝑦 − 𝑞1 .𝑦)) < 0 then
ing box regression. Furthermore, unlike other existing state-of-the-art
15: 𝑤𝑛 ← 𝑤𝑛 − 1
16: end if
methods (Fan et al., 2022; Ye et al., 2023), DV-Det scales linearly
17: end if with the number of input points, allowing the model to be tailored to
18: end if the computational capabilities of the platform to meet the demands of
19: end for time-critical systems.
As a two-stage detection framework, RoI pooling plays an essential
part in the second refinement stage. However, performing efficient
yet accurate RoI pooling still remains an open problem for 3D point
convergence. Range image segmentation approaches generally main- cloud detection. The concept of RoI pooling was officially introduced
tain 𝑂(𝑁) complexity (Bogoslavskyi and Stachniss, 2016), though some by Shi et al. (2019b), and was later adopted in successive works (Qi
implementations like the work of Anand et al. (2021) require 𝑂(𝑠𝐵𝑁) et al., 2017b; Shi et al., 2020). But how to efficiently aggregate the
operations, where 𝐵 is the number of bins when partitioning along point-wise features into RoI pooling grids still remains a problem. Shi
a line segment, and 𝑠 is the number of line segmentations computed et al. (2020) proposed approach where the features in each pooling
after radially partitioning the 3D point cloud around the 𝑧-axis. Our grid are learned from additional convolutions, e.g., for RoI pooling
proposed method combines statistical filtering at 𝑂(𝑁) with polygonal with grid resolution of 6 × 6 × 6, a total number of 216 convolution
cropping at 𝑂(𝑉 𝑁), where 𝑉 represents the number of vertices in the calculations are involved, that is, for one single RoI only; therefore,
polygon. Since 𝑉 is a fixed constant during deployment, our overall it is unaffordable for an efficient detection framework. To address
complexity remains effectively 𝑂(𝑁). this challenge, we introduce the LA-RoI pooling method, which is
As shown above, simple plane-fitting or registration-based methods elaborated in Section 3.3.3.
can be highly effective in controlled conditions or predominantly planar A comprehensive performance evaluation of DV-Det on the bench-
roads. However, they generally focus on removing ground returns only, mark KITTI and Waymo Open Dataset (Sun et al., 2020) will be present
and they often cannot remove additional off-ground clutter such as later in Section 4. The model architecture evaluated is depicted in Fig.
roadside structures or tall vegetation. Furthermore, plane-fitting tech- 6. The following subsections provide detailed descriptions of the point-
niques like RANSAC may inadvertently remove portions of vehicles on wise convolution layer based on the dynamic voxelization operation
sloped surfaces or ramps, since they assume a global plane. By contrast, plus the LA-RoI pooling, together with the loss functions adopted.
our statistical filtering explicitly models each LiDAR beam’s range The point-wise convolution layer consists of the sampling layer, the
profile and the point-in-polygon filtering effectively extract only points grouping layer, and feature extraction layer.

8
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 6. Visualization of the proposed two-stage DV-Det framework. In the first stage, raw 3D points are processed by point-wise convolutional layers with dynamic voxelization
to extract features and obtain 3D bounding-box proposals. In the second stage, the proposals are refined based on the LA-RoI pooling method to get the final prediction results.

Fig. 7. The point-wise convolution operation consists for three processes. (a) grid down sampling; (b) grouping; (c) feature extraction.

3.3.1. Sampling layer — grid downsampling 3.3.2. Grouping layer and feature extraction layer — dynamic voxelization
As illustrated in Fig. 7(a), DV-Det employs a grid downsampling convolution
As illustrated in Fig. 7, the grouping and feature extraction layers
approach, which is significantly faster than the widely used farthest
together perform dynamic voxelization convolution. First, the grouping
point sampling for point downsampling (Qi et al., 2017a). In grid
layer collects the neighboring points for each key point. Using the
downsampling, within a specified volume of interest with dimensions same 3D buffer described in the previous section, constructed from a
𝐷𝑥 , 𝐷𝑦 , 𝐷𝑧 , a 3D buffer of size ⌈𝐷𝑥 ∕𝛿𝑥 ⌉, ⌈𝐷𝑦 ∕𝛿𝑦 ⌉, ⌈𝐷𝑧 ∕𝛿𝑧 ⌉ is instanti- point cloud 𝑃 , the neighbors of a key point can be directly accessed
ated to −1, indicating the absence of points at the corresponding buffer using the voxel coordinates, resulting in a time complexity of 𝑂(1).
positions. As illustrated in Fig. 7(a), given a set of input point clouds To enhance the efficiency of the neighbor search operation, the search
with coordinates 𝑥, 𝑦, 𝑧, and voxel grids resolutions, 𝛿𝑥 , 𝛿𝑦 , 𝛿𝑧 , the point space for a point’s neighbors can be further refined. With the voxel
clouds are voxelized into 3D grid coordinates or voxel coordinates, coordinates (𝑔𝑥 , 𝑔𝑦 , 𝑔𝑧 ) of a keypoint, the kernel radius 𝑟𝑘𝑒𝑟𝑛𝑒𝑙 and the
(𝑔𝑥 , 𝑔𝑦 , 𝑔𝑧 ) = (⌊𝑥∕𝛿𝑥 ⌋, ⌊𝑦∕𝛿𝑦 ⌋, ⌊𝑧∕𝛿𝑧 ⌋). The order of points, 𝐎 = [1, … , 𝑁] offset group defined by a Cartesian product of the offsets along the
𝑥, 𝑦, 𝑧-axes, 𝛯 = 𝛥𝑥 × 𝛥𝑦 × 𝛥𝑧 where 𝛥 = [⌊−𝑟𝑘𝑒𝑟𝑛𝑒𝑙 ∕𝛿⌋, ⌈𝑟𝑘𝑒𝑟𝑛𝑒𝑙 ∕𝛿⌉], and
in the point cloud 𝑃 with 𝑁 points is assigned to the 3D buffer based
the neighbors of the keypoints are then guaranteed to be within the set
on the voxel coordinates of the points. This assignment has a time
of voxel coordinates { (𝑔𝑥 , 𝑔𝑦 , 𝑔𝑧 ) + 𝛯 }.
complexity of 𝑂(𝑁) and the access to the buffer has a time complexity In the subsequent feature extraction layer, neighboring points are
of 𝑂(1), which is overall faster than farthest point sampling which has divided into 𝑘 × 𝑘 × 𝑘 blocks based on the size of the kernel weights
time complexity of 𝑂(𝑁 2 ), at the cost of a higher memory complexity (for example, Fig. 7(c) shows a scenario where 𝑘 = 3). The features
of 𝑂(𝑁 3 ). of all points within each block are averaged to produce features that
Furthermore, grid downsampling can be parallelized using GPU correspond in size to the kernel weights. Then a kernel convolution ∗ is
executed between the kernel weights (𝑊 ) and the aggregated features
threading, unlike farthest point sampling, which is a sequential algo-
(𝑉 𝐹 ), as illustrated in Fig. 7.
rithm. This parallelism significantly enhances the speed of the down-
sampling operation. After dynamic voxelization, a single voxel grid may 3.3.3. LA-RoI pooling
contain multiple points, from which a single point is randomly selected As shown in Fig. 8, during the pooling operation, additional weights
as the key point (red points in Fig. 7(a)) for the grouping and feature are assigned to the input point-wise features according to their Eu-
extraction layers. clidean distances to the pooling grid center. Specifically, given a 3D

9
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 8. LA-RoI pooling illustrated in 2D with two proposals and pooling resolution of 2 × 2. With LA-RoI pooling, our network can capture the tiny variance between to the two
proposals that are very close to each other.

RoI of dimension 𝑊 × 𝐿 × 𝐻 and the target pooling grid resolution 𝑘,

we first gather all the interior point-wise features {𝑓𝑖 } within a certain 3.4. Efficient 3D multi-vehicles tracking algorithm: AB3DMOT
pooling grid 𝑔 among the 𝑘 × 𝑘 × 𝑘 grids. Next, representative features
𝑓𝑔 is computed as follows: Previous works by Wojke et al. (2017) and Wojke and Bewley
(2018) addressed incorrect tracking identification (ID) assignments in
1∑
𝑛
𝑑 𝑊 𝐿 𝐻
𝑓𝑔 = 𝑤 𝑓 ; 𝑤 = 𝑒𝑥𝑝(1 − 𝑖 ); 𝑟 = 𝑚𝑎𝑥{ , , } (2) 2D image-based tracking using deep image features from neural net-
𝑛 𝑖=1 𝑖 𝑖 𝑖 𝑟 𝑘 𝑘 𝑘
works. Although effective, these methods involved a trade-off between
where 𝑑𝑖 is the Euclidean distance between the feature point 𝑓𝑖 and speed and accuracy due to the additional computational overhead.
the grid center 𝑔, 𝑟 is the maximum dimension of pooling grid among However, tracking in 3D space offers inherent advantages: objects
𝑊 , 𝐿 and 𝐻 orientations. 𝑓𝑔 is normalized by the total number of at different distances remain spatially distinct, even when occlusion
interior points 𝑛 in each pooling grid, so that the scale of 𝑓𝑔 will not be occurs in the point cloud representation. This spatial separation en-
heavily affected by 𝑛. In practice, the largest value of 𝑛 is set as 5, which sures more reliable tracking, as multiple vehicles in 3D space cannot
means for each grid 𝑔, only at most 5 interior point-wise features 𝑓𝑖 are physically occupy the same volume.
taken into consideration, and the redundant points are simply omitted. For our system, we employed the AB3DMOT algorithm (Weng et al.,
By doing so, tiny location perturbations between different RoIs can be 2020), which is specifically designed for efficient 3D multi-vehicle
captured, while avoid the expensive convolution computations. tracking. Unlike previous approaches that use computationally expen-
sive deep learning models to compute matching scores, AB3DMOT
3.3.4. Loss functions utilizes 3D IoU as a simple, yet effective metric to match predicted vehi-
The model is trained end-to-end and optimized using a multi-task cles with tracked vehicles. The algorithm has demonstrated exceptional
loss, 𝐿𝑡𝑜𝑡𝑎𝑙 . The first component of this loss is the foreground point performance, operating at 207.4 FPS in the KITTI benchmark dataset
classification loss, 𝐿𝑜𝑏𝑗 , which reflects the confidence that a key point while achieving zero identity switches.
is a foreground point. 𝐿𝑜𝑏𝑗 is calculated using the focal loss (Cao et al., The selection of AB3DMOT was driven by several practical con-
2011) as follows: siderations. While newer tracking algorithms like ‘‘Robust 3D Multi-
𝐿𝑜𝑏𝑗 = −𝛼(1 − 𝑝)𝛾 log 𝑝 (3) Object Tracking by Observational Noise and State Estimation Drift
Mitigation on LiDAR PointCloud’’ (RobMOT) (Nagy et al., 2024) offer
where 𝑝 is the class probability of the keypoint. 𝛼 and 𝛾 are set to advanced features for handling observational noise and state estimation
0.25 and 2, respectively, the same as the parameters in Lin et al. drift, AB3DMOT’s lightweight architecture (running at 207.4 FPS on
(2017). Similar to Ref. Su (2021), the 3D IoU loss 𝐿3𝐷𝐼𝑜𝑈 is the CPU alone) is particularly well-suited for our edge computing envi-
main optimization objective, as it accounts for all the parameters of ronment. Our use case involves roadside monitoring where vehicles
the bounding boxes, [ 𝑥, 𝑦, 𝑧, 𝑤, 𝑙, ℎ, 𝜃𝑧 ]. However, the 3D IoU loss is are spatially well separated in 3D space, making complex tracking
insensitive to the 𝜃𝑧 of the bounding box. The angle regression loss 𝐿𝜃𝑧
algorithms unnecessary. Furthermore, AB3DMOT’s straightforward ar-
between the ground truth orientation 𝜃𝑔 and predicted orientation 𝜃𝑝 is
chitecture using Kalman filtering and the Hungarian algorithm enables
introduced as an auxiliary learning objective:
{ easier integration and maintenance in production environments.
0.5𝑏2 , if ‖𝑏‖ < 1 The robustness of our tracking module is ensured by careful param-
smoothL1(𝑏) = (4) eter tuning of the Kalman filter. The measurement uncertainty matrix
‖𝑏‖ − 0.5, otherwise
𝑅 and the process uncertainty matrix 𝑄 are configured as
𝐿𝜃𝑧 = smoothL1(𝑠𝑖𝑛(𝜃𝑔 − 𝜃𝑝 )) (5)
𝑅 = 𝐼7×7 [𝜖𝑥 , 𝜖𝑦 , 𝜖𝑧 , 𝜖𝑤 , 𝜖𝑙 , 𝜖ℎ , 𝜖𝜃𝑧 ]𝑇
The classification loss 𝐿𝑐𝑙𝑠 used is the softmax cross entropy loss: (8)
𝑄 = 𝐼10×10 [𝜆𝑥 , 𝜆𝑦 , 𝜆𝑧 , 𝜆𝑤 , 𝜆𝑙 , 𝜆ℎ , 𝜆𝜃𝑧 , 𝜆𝑣𝑥 , 𝜆𝑣𝑦 , 𝜆𝑣𝑧 ]𝑇
𝑁+
1 ∑ 𝑒𝑥𝑝(𝑠+𝑖 ) where 𝜖 represents the measurement uncertainties based on the bound-
𝐿𝑐𝑙𝑠 = − 𝑙𝑜𝑔( ∑𝐶 ) (6)
𝑁+ 𝑖 ing box detection output by DV-Det. Given that typical vehicle length
𝑗=1 𝑒𝑥𝑝(𝑠𝑗 )
is around 4 m, we calibrated the position and dimension measure-
where 𝑠+𝑖 is the predicted probability score for the positive class
ments uncertainties (𝜖𝑥 , 𝜖𝑦 , 𝜖𝑧 , 𝜖𝑤 , 𝜖𝑙 , 𝜖ℎ ) to 1.0 m to accommodate detec-
𝑖 evaluated at the positive keypoint, 𝑁+ is the number of positive
tion variations during partial occlusions. This value accounts for cases
keypoints and 𝐶 is the total number of classes.
where partial vehicle visibility can cause location estimates to deviate
The multi-task loss is the sum of all the loss functions:
by up to 1 meter from the true position. For orientation measurement
𝐿𝑡𝑜𝑡𝑎𝑙 = 𝐿𝑜𝑏𝑗 + 𝐿3𝐷𝐼𝑜𝑈 + 𝐿𝜃𝑧 + 𝐿𝑐𝑙𝑠 (7) uncertainty (𝜖𝜃𝑧 ), we set it to 1.0 rad based on observed DV-Det

10
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

behavior during partial occlusions — when only one corner of a vehicle

is visible, orientation estimates can vary by approximately 1 rad from
the actual vehicle pose. The process uncertainties 𝜆 are derived from
typical vehicle motion patterns: 1.0 for position and dimension states to
allow natural vehicle movement, and 0.01 for velocity states to ensure
smooth tracking under varying speeds.
The system’s robustness is particularly evident in three scenarios:

1. During dense traffic conditions, where the spatial separation in

3D space prevents track confusion even when vehicles are close
together.
2. Through partial occlusions, where the Kalman filter’s state pre-
diction maintains tracking continuity.
3. During vehicle turns, where the orientation uncertainty param-
eters allow for smooth tracking through varying vehicle poses.

The effectiveness and robustness of the selected algorithm,

AB3DMOT, is validated by our experimental results, where we achieve
100% recall and at least 80% precision for most vehicle classes. These
characteristics make AB3DMOT an ideal choice for our real-time vehi-
cle monitoring system, meeting our operational requirement of 10 Hz
with a latency of 62.71 ms while maintaining high tracking accuracy.
It is important to note the specific role of tracking within our sys-
tem’s boundaries. Unlike traditional multi-object tracking (MOT) appli-
cations that emphasize persistent identity maintenance and inter-object
relationship analysis, our tracking component serves two primary func-
tions: (1) preventing double-counting by associating detections across
frames, and (2) supporting temporal class label smoothing to im-
prove classification accuracy. The system does not require analyzing
changes in the order, position, or quantity within vehicle groups,
nor maintaining vehicle identities beyond what is necessary for ac-
curate counting. This focused tracking approach is sufficient for our
vehicle fleet monitoring objectives and contributes to the system’s
computational efficiency.

3.5. Multi-vehicles counter

Vehicle movement on roads typically follows well-defined patterns, Fig. 9. Two processes of multi-vehicles counter algorithm. (a) alignment score com-
with vehicles in designated lanes traveling in specific directions. We putation, and (b) temporal class label smoothing strategy.

propose a novel vehicle counting algorithm that leverages this under-

standing to improve counting accuracy and prevent common issues
such as double counting or counting non-vehicular objects. The al- the vehicle’s appearance at the moment of detection, the same vehicle
gorithm takes as input the parameters [x, y, z, w, l, h, 𝜃𝑧 , classID, may be classified into different categories at different timestamps, as
confidence, trackID ] of the vehicles tracked by AB3DMOT, as detailed shown in Fig. 9(b). To address this, we introduce a temporal class label
in the previous section. smoothing strategy to yield a more stable and accurate vehicle classifi-
As illustrated in Fig. 9(a), at each timestamp, the trajectory of the cation. When a vehicle is closer to the LiDAR sensor, its critical features
vehicle 𝑡𝑟𝑎𝑐𝑘𝐼𝐷 that has not yet been counted, denoted as 𝑆𝑡𝑟𝑎𝑐𝑘𝐼𝐷 , is are more likely to be captured, making the classification prediction
assessed within the RoI 𝜉 (white polygon in Fig. 9(a)) is evaluated. For a at closer range more trustworthy. Consequently, a distance-weighted
given hyperparameter 𝜂, the line segment 𝐿𝑡𝑟𝑎𝑗 (the purple arrow in Fig. scheme is apt for this context to determine the vehicle’s class label. As
9(a)) represents the connection between the point 𝑞𝑡−𝜂 on the trajectory demonstrated in Fig. 9(b), for each timestamp 𝑡, the weight 𝑤𝑡 of the
within 𝜉, at most 𝜂 time steps prior to the current timestamp 𝑡, and the class at that position, and at each position 𝑞𝑡 in the trajectory 𝑆𝑡𝑟𝑎𝑐𝑘𝐼𝐷 ,
current position 𝑞𝑡 on the trajectory, where 𝑞 ∈ 𝑆𝑡𝑟𝑎𝑐𝑘𝐼𝐷 and 𝑞 ∈ R2 . To is computed as:
ensure accurate counting and minimize errors, two conditions must be 1.0
𝑤𝑡 = , 𝑞 ∈ R2 (10)
met before a vehicle is marked as counted: ‖𝑞𝑡 ‖2
The weights for all positions are cumulatively tallied by class, and the
1. The vehicle must cross the virtual counting line 𝐿𝑐𝑜𝑢𝑛𝑡 (the red
final class label assigned to the vehicle corresponds to the class with
line in Fig. 9(a)).
the highest aggregate weight.
2. 𝐿𝑡𝑟𝑎𝑗 (the purple line in Fig. 9(a)) must intersect 𝐿𝑐𝑜𝑢𝑛𝑡 , and the
By implementing these strategies, our algorithm aims to signifi-
⃗ 𝑡𝑟𝑎𝑗 = 𝑞⃖⃖⃖⃖⃖⃖⃖⃖⃖
direction of 𝐿𝑡𝑟𝑎𝑗 , represented by the vector 𝐿 𝑡 𝑞𝑡−𝜂⃗, should cantly reduce counting errors caused by double counting, misclassi-
align with the traffic flow vector 𝑇⃗𝑓 𝑙𝑜𝑤 (the green arrow in Fig. fication, or the erroneous inclusion of non-vehicular objects, thereby
9) within the region. improving the overall accuracy of vehicle counting in complex traffic
⃗ 𝑡𝑟𝑎𝑗 , and 𝑇⃗𝑓 𝑙𝑜𝑤 is calculated as: scenarios.
The alignment score s between 𝐿

⃗ 𝑡𝑟𝑎𝑗 ⋅ 𝑇⃗𝑓 𝑙𝑜𝑤 > 0)

𝑠 = 1(𝐿 (9) 4. Evaluation of DV-Det

Only vehicle trajectories with positive alignment scores are counted. The evaluation of DV-Det is conducted prior to the on-site ex-
Since the class detected by the deep learning model can vary based on periment to assess the performance of our proposed deep learning

11
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

methodologies. Unless stated otherwise, all experiments related to the 4.4. Evaluation on Waymo open dataset
DV-Det in this study were performed on a server equipped with RTX
2080 Ti GPUs, which has a computing power of 13.5 TFLOPs and In addition to the KITTI dataset, we tested our model on the Waymo
supports efficient processing of model training and inference. Open Dataset for a comprehensive evaluation. We trained our model
on 798 training sequences with approximately 158k point cloud scenes
4.1. Network architecture
and tested our performance on 202 validation sequences with approxi-
mately 40 000 point cloud scenes. Unlike the KITTI dataset, the Waymo
As shown in Fig. 6, our backbone network is based on the hi-
Open Dataset features a full 360-degree FOV with denser point clouds,
erarchical point-wise convolutional layers with dynamic voxelization.
requiring more efficient models to achieve real-time processing speeds.
The backbone is composed of four convolution blocks, and each block
We evaluated our method using the official tools released by Waymo.
includes two point-wise convolutional layers. The downsampling reso-
The mAP performance of DV-Det is reported in Table 3 for vehicle
lution 𝑟 is set as 0.1 m, 0.2 m, 0.4 m, 0.8 m for each block, and the
kernel resolution 𝑘 is 3 for all the blocks with the number of feature detection with an IoU threshold of 0.7, considering only examples with
channels 16, 32, 64 and 128. The output features from each block are difficulty LEVEL_1 (objects with more than 5 LiDAR points). Our DV-
then concatenated and fed into the second stage as input to the LA-RoI Det is outperformed by state-of-the-art 3D perceptron-based methods
pooling layer. The LA-RoI pooling transforms the point-wise features like Voxel-RCNN (Deng et al., 2021) in terms of detection accuracy,
into 5 × 5 × 5 dense 3D voxels, and pass them to the dense 3 × 3 × 3 3D but our method ran at a speed of 25 FPS, which is faster than 2D BEV
convolutions. We apply ‘‘valid’’ padding strategy for the 3D convolution methods like PointPillars and with significantly higher accuracy.
and the voxel dimensions are reduced to 1 × 1 × 1 after two successive
dense convolutional layers. Finally, we feed these features into MLPs 5. On-site experiment
to return the final predictions.
An on-site experiment was conducted to evaluate our real-time vehi-
4.2. Training and inference setup
cle fleet monitoring system on vehicle detection, vehicle classification,
vehicle counting, and system speed.
DV-Det is trained end-to-end from scratch on the KITTI 3D object
detection dataset and the Waymo Open Dataset. The model was trained
on 16 GPUs for 80 epochs with a learning rate of 0.01. The batch size 5.1. Testing site and hardware setup
was set to 64 for the KITTI dataset and 32 for the Waymo Open Dataset.
In the second stage, we resampled the negative and positive proposals As depicted in Fig. 10, the experiment was carried out at the South
at a 2 ∶ 1 ratio with an IoU threshold of 0.5. During the training phase, entrance of Hong Kong University of Science and Technology (HKUST).
input data were augmented with random rotations along the 𝑧-axis in The location features a bus stop on the left and a roundabout on
the range [−𝜋∕4, 𝜋∕4] and random scaling along all three axes in the the right (Fig. 10(a)). Unlike previous studies on ITS that often select
range [0.95, 1.05]. Following practices in Refs. Shi et al. (2020) and Yan straight or less complex road conditions, our chosen test site presents
et al. (2018), we also randomly ‘‘pasted’’ ground truth objects from uneven topography and road geometry, with the entrance situated on
different scenes within the same dataset to enrich the training data. a slope. This complexity adds the challenge of tracking and counting
vehicles in the roundabout.
4.3. Evaluation on KITTI dataset
The LiDAR sensor was mounted on a roof top at a pitch angle of
57.29◦ (calculated based on the mounting height of 4.5 m to optimize
We first evaluated our performance on the 3D object detection
KITTI dataset, which consists of a training dataset (7481 samples) coverage of the detection areas), providing the LiDAR sensor with an
and a testing dataset (7518 samples). We further divided the training effective horizontal field of view (FOV) of 235◦ and a coverage area of
partition into a train set with 3712 samples and a val set with 3769 50.2 m by 70.2 m.
samples. In the training phase, we used 80% of the samples from the Three RoIs were designated as vehicle counting areas, as shown in
𝑡𝑟𝑎𝑖𝑛 + 𝑣𝑎𝑙 combination as training data and reserved the remaining Fig. 10(c). The central area is defined as vehicle counting Region 1,
20% for validation. We evaluated performance using the KITTI official capturing vehicles entering the campus from the south. The left area,
online evaluation server’s testing dataset and reported our accuracy for vehicle counting Region 2, covers the sole exit from the bus stop. The
the car and cyclist classes in Table 2. The speed of the deep learning right area, Region 3, monitors vehicles leaving the campus heading
model is evaluated locally. We only compared with pure point cloud- southward. The monitoring period is from 1 December 2020 to 31
based methods and categorized existing methods into two groups: December 2020, and 21 July 2021.
3D Perceptron methods that parse point clouds in 3D space using
perceptrons like 3D convolution or PointNet, and BEV methods that 5.2. Vehicle taxonomy
convert 3D point clouds onto a 2D BEV plane and process data with
2D convolutions. Note that all existing 3D perceptron-based methods
have an inference speed of less than 25 FPS on a dedicated GPU, Given the geographical context of this, the class of vehicles was
and considering the 90-degree front-view FOV in the KITTI dataset, defined according to the specifications of the EPD of Hong Kong. This
they are not yet ready for real-world applications. BEV-based methods study is the first to attempt fine-grain vehicle detection in 3D point
run considerably faster than 3D perceptron-based methods; however, clouds across 10 vehicle classes, derived from the 18 classes defined by
converting 3D point clouds into 2D BEV projections results in the per- the EPD. The 10 classes include private cars, taxis, small goods vehicles,
manent loss of 3D spatial information, often leading to unsatisfactory public minibuses, private minibuses, coach buses, double-decker buses,
performance on challenging examples. small trucks, large trucks, and motorbikes.
In contrast, our method achieved an impressive 75 FPS on the Fig. 11 presents examples of point clouds for each vehicle class. The
KITTI dataset, which is three times faster than the state-of-the-art 3D 10 classes of vehicles include some cases of intra-class distinctions:
perceptron-based method. Meanwhile, compared with existing BEV-
based methods, our method outperformed them in all evaluation met- 1. Private cars and small goods vehicles. As shown in Fig. 12,
rics and achieved a 45% faster inference speed than the widely adopted a van-shaped vehicle without sealed side windows at the rear is
‘‘Fast Encoders for Object Detection from Point Clouds’’ (PointPillars) considered a private car, while a similar vehicle with sealed side
method (Lang et al., 2019). windows is classified as a small goods vehicle.

12
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Table 2
Performance comparison on the KITTI testing dataset based on mAP.
Method Speed Car — 3D detection Car — BEV detection Cyclist — 3D detection Cyclist — BEV detection
(FPS) Easy Mod. Hard Easy Mod. Hard Easy Mod. Hard Easy Mod. Hard
Point-RCNN (Shi et al., 2019a) ∼10 86.96 75.64 70.70 92.13 87.39 82.72 74.96 58.82 52.53 82.56 67.24 60.28
STD (Yang et al., 2019) <20 87.95 79.71 75.09 94.74 89.19 86.42 78.69 61.59 55.30 81.36 67.23 59.35
3D Perceptron

Fast Point-RCNN (Chen et al., 2019) ∼20 85.29 77.40 70.24 90.87 87.84 80.52 – – – – – –
PV-RCNN (Shi et al., 2020) ∼15 90.25 81.43 76.82 94.98 90.65 86.14 78.60 63.71 57.65 82.49 68.89 62.41
SECOND (Yan et al., 2018) ∼30 83.34 72.55 65.82 89.39 83.77 78.59 71.33 52.08 45.83 76.50 56.05 49.45
Voxel-RCNN (Deng et al., 2021) ∼25 90.90 81.62 77.06 – – – – – – – – –
SA-SSD (He et al., 2020) ∼25 88.75 79.79 74.16 95.03 91.03 85.96 – – – – – –
3DSSD (Yang et al., 2020) ∼25 88.36 79.57 74.55 – – – 82.48 64.10 56.90 – – –
PIXOR (Yang et al., 2018b) ∼10 – – – 81.70 77.05 72.95 – – – – – –
BEV

HDNET (Yang et al., 2018a) ∼25 – – – 89.14 86.57 78.32 – – – – – –

PointPillars (Lang et al., 2019) >45 82.58 74.31 68.99 90.07 86.56 82.81 77.10 58.65 51.92 79.90 62.73 55.58
DV-Det (Ours) ∼75 85.32 76.74 70.02 91.53 87.65 83.27 77.94 58.77 52.49 80.92 63.08 57.32

Table 3
mAP performance comparison on the Waymo Open Dataset with 202 validation sequences for the vehicle class at different distances and IoU matching thresholds.
Method Speed 3D mAP (IoU = 0.7) BEV mAP (IoU = 0.7)
(FPS) Overall 0-30 m 30 m–50 m 50 m-Inf Overall 0–30 m 30 m–50 m 50 m-Inf
Point-RCNN (Shi et al., 2019a) <5 70.30 91.92 69.21 42.17 82.96 97.53 82.99 64.97
Voxel-RCNN (Deng et al., 2021) <10 75.59 92.49 74.09 53.15 88.19 97.62 87.34 77.70
PointPillars (Lang et al., 2019) ∼15 56.62 81.01 51.75 27.94 75.57 92.10 74.06 55.47
DV-Det (Ours) ∼25 63.42 85.72 55.89 30.73 78.38 95.12 76.93 59.36

Table 4
The number of ground truth vehicle count in all three regions (see white polygons in Fig. 10) in the vehicle counting evaluation dataset. ‘‘–’’ means that there are no ground truth
instance appearing in the region.
Vehicle counting region Vehicle count
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Region 1 152 28 16 – 6 18 – 27 9 16
Region 2 3 3 – 37 – – 14 – – –
Region 3 104 24 23 – 5 13 – 28 7 12

2. Public minibuses and private minibuses. These vehicles often The same frame sampling and labeling procedure was applied, yielding
share the same base model, such as the Toyota Coaster. The 2124 point cloud frames for evaluation. The class distribution in the
primary differences include the presence of a top plate indicating evaluation dataset is shown in Fig. 15. We have ensured that the period
the bus terminus and the color of the vehicle, as depicted in Fig. of data sampled for training dataset and evaluation dataset does not
13. overlap in time.
3. Private minibuses, coach buses, and double-decker buses. The evaluation dataset for vehicle counting was prepared differ-
Buses vary in size, freight loading capacity, and engine power ently, with the addition of trackID to assist in evaluation. Point cloud
and capacity, as illustrated in Fig. 11. Being able to classify footage was collected between 8 A.M. and 12 P.M. of 2 December 2020,
buses helps applications such as road maintenance and emission and subsequently split into 5 min interval point cloud video clips. Point
inventory estimation. cloud video clips are randomly selected, resulting in a total of 29 point
4. Small trucks and large trucks. Trucks also vary in size, freight cloud video clips. Table 4 lists the number of vehicles in each counting
loading capacity, and engine power, as shown in Fig. 11. Appli- class. The three vehicle counting regions can be referred to the white
cations such as road maintenance and the estimation of emission polygons indicated in Fig. 10(c).
inventory require information that can distinguish between the
two types. 5.4. Configurations for DV-Det and multi-vehicle tracker

DV-Det Configuration. Tables 2 and 3 provide an overview of

5.3. Dataset preparation the performance comparison among various models, highlighting the
superior speed of our proposed DV-Det model. The evaluations were
For this study, 1.2 Terabytes (TB) of point cloud data were collected. conducted on a server. However, the on-site edge computing environ-
A total of 34 point cloud video clips, collected during the period of 2 ment employs an NVIDIA AGX Orin, which has significantly lower GPU
December 2020 and July to August 2021, each spanning 5 min and compute capability at 3.37 TFLOPs. Moreover, the LiDAR data acquired
recorded at 10 Hz, were used to create the training dataset. Frames during the on-site experiment were more extensive than those in the
were sampled at 1 Hz for labeling, with the exception of frames benchmark datasets. Despite DV-Det’s commendable speed, the con-
containing scarce vehicle classes like private minibuses and motorbikes, straints posed by limited edge computing resources necessitate adap-
which were labeled at 5 Hz. Frames without objects were excluded, tation. To address this, we have devised an economical engineering
resulting in 4619 labeled point cloud frames for training. The class solution by refining DV-Det into a single-stage detection model, exclud-
distribution in the training dataset is shown in Fig. 14. ing the second-stage refinement module, thereby enhancing inference
The evaluation dataset for 3D object detection was compiled from speed without compromising accuracy to meet practical application
13 point cloud video clips, totaling 45 min of point cloud footage. requirements (as will be demonstrated in the subsequent section).
The point cloud footage was collected on 2 and 31 December 2020. The model architecture is illustrated in Fig. 16. As edge computing

13
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 10. Setup of the LiDAR sensor for the on-site experiment. (a) image showing a part of the view from the sensor’s perspective; (b) the installation configuration of the LiDAR
sensor; and (c) this image depicts the point cloud captured by the LiDAR sensor, where the white polygon represents the vehicle counting regions, the red line represents the
counting line, and the green arrow indicates the direction of traffic flow.

capabilities continue to advance, there is potential to reintegrate the 𝑈 = 𝐼10∗10 [ 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, (13)
second-stage refinement module, thereby boosting the system’s per- 𝑇
10.0, 10000.0, 10000.0, 10000.0 ] (14)
formance in vehicle monitoring, provided the computational resources
𝑇
permit. 𝑄 = 𝐼10∗10 [ 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.01, 0.01, 0.01 ] (15)
The DV-Det deep learning model underwent training for 120 epochs, These initializations provide a baseline for the Kalman filter to begin
starting with an initial learning rate of 0.001. An Adam optimizer was tracking the state of each vehicle effectively, allowing for real-time
utilized, featuring a learning rate decay factor of 0.5 and a weight decay adjustments as more data is gathered during the monitoring process.
of 0.0005, to minimize the total loss 𝐿𝑡𝑜𝑡𝑎𝑙 (refer to Eq. (7)). The focal
loss hyperparameters, 𝛼 and 𝛾, were configured to values of 0.25 and 6. On-site experimental results and discussion
2, respectively.
To enhance the diversity of the training dataset, various data aug- 6.1. Vehicle detection results and discussion
mentation strategies for 3D object detection were employed. These
included ground truth instance pasting, random flipping along the 𝑌 - The primary evaluation metric utilized in this study was mAP,
axis, global scaling with a random scaling factor drawn from the range which quantifies the area under the precision–recall curve (PR-curve).
[0.95, 1.05], and global rotation around the 𝑍-axis with a random This metric is consistent with the benchmark used in the KITTI dataset.
angle selected from the interval [−1.4, 1.4]. The matching criteria between predicted and ground truth bounding
Multi-Vehicle Tracker Configuration. For each new tracker, the boxes are based on both 2D and 3D IoUs. A predicted bounding box
Kalman filter parameters 𝑅, 𝑈 and 𝑄 were initialized with the following is matched with a ground truth bounding box if the IoU score meets
values: or exceeds a predefined threshold 𝜇, resulting in a True Positive (TP).
[ ] Predicted bounding boxes without a corresponding ground-truth match
𝐻 = 𝐼7∗7 07∗3 (11)
are considered False Positives (FP), and if multiple predictions match
𝑅 = 𝐼7∗7 (12) a single ground truth, only one is counted as TP while the others are

14
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 11. Non-exhaustive examples of point clouds of each vehicle class.

marked as FP. Ground truth bounding boxes without predictions are

deemed False Negatives (FN). Precision and recall are calculated as
follows:
𝑇𝑃
𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃 (16)
𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃 + 𝐹𝑁
Adhering to the updated KITTI benchmark rules (Simonelli et al.,
2022), the precision of the model is assessed at 41 uniformly sampled
recall values within the range [0, 1]. The mAP is estimated by:
1 ∑
𝑚𝐴𝑃 = 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛(𝑅𝑒𝑐𝑎𝑙𝑙) (17)
41 𝑅𝑒𝑐𝑎𝑙𝑙∈{𝑘∕40,𝑘∈[0,40]∩Z}
In this study, mAP is computed using the 3D IoU matching function
with an IoU threshold of 0.5 (Table 5), and the 2D IoU matching
function with the same IoU threshold (Table 6). To gauge the overall
performance of the detection model, the results were evaluated across
Fig. 12. Distinction between a private car (private van) and small goods vehicle (van the entire effective region of 50.2 m by 70.2 m. In addition, it is crucial
for carrying goods). to assess the detection performance within the three RoIs designated for
vehicle counting.
As indicated in Table 5, the 3D object detection model achieves
an mAP score of at least 80 for both 2D and 3D IoU at the 0.5

15
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Table 5
Detection performance of model on vehicle counting region 1, 2, and 3 based on the mAP with an IoU threshold of 0.5 (𝑚𝐴𝑃0.5 ) with 41 recall positions and 3D IoU as matching
metric. ‘‘–’’ means that there are no ground truth instance appearing in the region.
Vehicle counting region 3D IoU 𝑚𝐴𝑃0.5
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Region 1 93.15 95.12 78.26 – 100.00 99.11 – 87.92 89.51 90.27
Region 2 – – – 96.64 – – 95.12 – – –
Region 3 58.60 83.28 99.55 – 53.66 – – – 65.85 83.07

Table 6
Detection performance of model on vehicle counting region 1, 2, and 3 based on the 𝑚𝐴𝑃0.5 with 41 recall positions and 2D IoU as matching metric. ‘‘–’’ means that there are no
ground truth instance appearing in the region.
Vehicle counting region 2D IoU 𝑚𝐴𝑃0.5
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Region 1 93.15 95.12 78.26 – 100.0 99.11 – 87.92 93.82 95.62
Region 2 – – – 96.64.2 – – 95.12 – – –
Region 3 63.59 85.37 99.55 – 60.98 – – – 65.85 82.93

Fig. 15. Class distribution in evaluation dataset.

subtle differences in classification specifications can cause confusion

Fig. 13. Distinction between a public mini bus (bus that serves as mode of public
between vans and small goods vehicles.
transportation) and private mini bus.
The lower mAP for private minibuses is attributed to confusion with
public minibuses, as evidenced by the confusion matrix in Table 8. In
Hong Kong, minibuses are predominantly based on the Toyota Coaster
model, depicted in Fig. 13. The public minibus is a modified version of
this model, distinguished by a top plate indicating the bus terminus.
The scarcity of training examples for the private minibus (refer to
Fig. 14) could hinder the model’s ability to differentiate between the
two. In addition, the ‘large truck’ category includes both goods and
construction vehicles. The training dataset’s lack of comprehensive
coverage of construction vehicle types could lead to confusion between
small and large trucks, as observed in Table 9.

6.2. Vehicle counting results and discussion

Table 10 summarizes the performance of the vehicle counting based

Fig. 14. Class distribution in training dataset.
on the precision and recall of each individual vehicle class. The perfor-
mance was assessed using an additional 29 video clips of point clouds,
which included the 4 clips previously used to evaluate the detection
matching threshold for vehicle classes with fewer model variations,
model. As noted, the three RoIs for vehicle counting are outlined in
such as taxis, small goods vehicles, public minibuses, coach buses, small
Section 5.1 and depicted in Fig. 10(c). Within each region, a red virtual
trucks, and motorbikes. However, certain classes, including small goods
line, referred to as the count line, is established. Vehicles are counted
vehicles, private minibuses, small trucks, and large trucks, exhibit lower
detection performance. To investigate this, a subsidiary experiment only if they intersect this count line and travel in the general direction
was conducted in which another model was trained with the same of traffic flow, as indicated by a green arrow in Fig. 10(c). The precision
3D object detection training dataset, but with a slight modification and recall metrics from Eq. (16) were adapted to evaluate vehicle
to the classes. A new ‘van’ class was introduced, extracted from the counting performance. In this context, FP represent the count of excess
private car category, to represent van-shaped vehicles similar to small predicted vehicles, while FN denote the count of missed ground truth
goods vehicles. Consequently, the model was trained to recognize an vehicles. TP are calculated as the total number of ground truth vehicle
additional ‘van’ class. The confusion matrix in Table 7 reveals that counts minus FN.

16
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Fig. 16. The single-stage DV-Det framework deployed in real-time vehicle monitoring system.

Table 7
Confusion matrix among 4 classes: private car, taxi, van, and small goods vehicle generated from the 3D object detection
evaluation dataset, with a matching metric 2D IoU and matching threshold of 0.5. The bold values indicates that small goods
vehicle has tendency to be classified as van (a subset class within the private car class defined in the Section 6.1).
Predicted class
Private car Taxi Van Small goods vehicle
Private car 323 3 32 5
Taxi 6 124 0 1
Actual class
Van 92 230 210 26
Small goods vehicle 2 3 15 130

Table 8
Confusion matrix of ground truth class public mini bus and private mini bus against 10 predicted classes generated from the 3D object detection evaluation dataset, with a matching
metric 2D IoU and matching threshold of 0.5. The bolded values indicates that there is tendency of confusion between public mini bus and private mini bus.
Predicted class
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Public mini bus 0 0 0 116 0 0 0 0 0 0
Actual class
Private Mini Bus 2 0 0 5 33 0 0 0 0 0

Table 9
Confusion matrix of ground truth class small truck and large truck against 10 predicted classes generated from the 3D object detection evaluation dataset, with a matching metric
2D IoU and matching threshold of 0.5. The bolded values indicates that there is tendency of confusion between small truck and large truck.
Predicted class
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Small truck 1 0 1 0 0 3 0 93 7 0
Actual class
Large truck 0 0 0 1 0 0 1 8 121 0

Table 10
Performance of vehicle counting system on vehicle counting region 1, vehicle counting 2 and vehicle counting region 3 based on the precision and recall of each individual vehicle
class. ‘‘–’’ means that there are no ground truth instance appearing in the region.
Vehicle counting region 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛∕𝑅𝑒𝑐𝑎𝑙𝑙
Private Taxi Small goods Public mini Private Coach Double- Small Large Motorbike
car vehicle bus mini bus bus decker bus truck truck
Region 1 90.0/ 100.0/ 100.0 84.38/ – 100.00/ 94.74/ – 100.0/ 81.82/ 100.0/ 100.0
100.0 100.0 100.0 100.0 96.15 100.0
Region 2 100.0/ 75.0/ 100.0 – 100.00/ – – 100.00/ – – –
100.0 97.30 100.0
Region 3 84.55/ 88.89/ 100.0 95.83/ – 100.0/ 100.0/ – 100.0/ 63.64/ 100.0/ 100.0
100.0 100.0 100.0 100.0 82.14 100.0

Overall, as summarized in Table 10, the vehicle monitoring system in Table 10 also indicates that the proposed temporal class label
demonstrated excellent performance in counting vehicles across various smoothing strategy yields higher precision and recall for small goods
classes, with most achieving a recall of 100% and a precision of at vehicles, despite their lower detection performance (refer to Table
least 80 𝑚𝐴𝑃 . Detailed discussion is arranged according to different 5). However, the precision for small goods vehicles is comparatively
monitoring regions as follows: lower than for other classes. This discrepancy arises from the definition
Region 1. This area includes all vehicle classes except for public provided by the EPD of Hong Kong, where a small goods vehicle
minibuses and double-decker buses, as evidenced in Table 10. Results is defined as a van-shaped vehicle with sealed side windows at the

17
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Table 11 Our DV-Det method achieved a remarkable 75 FPS on the KITTI

The performance breakdown of the vehicle monitoring system on edge device, NVIDIA
dataset, tripling the speed of state-of-the-art 3D perceptron-based meth-
AGX Orin.
ods. It also outperformed existing BEV-based methods in all evaluation
Device Module FPS
metrics and achieved a 45% faster inference speed than the widely
Sensor Interface 53.56
NVIDIA AGX adopted PointPillars method. On the Waymo Open Dataset, DV-Det
3D Deep Object Detector 39.39
Orin demonstrated faster processing at 25 FPS compared to 2D BEV meth-
Multi-Vehicle Tracker and 162.15
Multi-Vehicle Counter ods like PointPillars, while maintaining significantly higher accuracy,
despite being outperformed by methods like Voxel-RCNN in detection
accuracy.
The on-site experimental results demonstrated that our monitoring
rear, as shown in Fig. 12. Conversely, a van-shaped vehicle without
system could achieve fine-grain vehicle classification with at least 80
sealed side windows is considered a private car. The subtle distinction
mAP at a 3D IoU threshold of 0.5 for most classes, a recall of 100%,
in appearance complicates the vehicle counting process due to the
and a precision of at least 80% for the majority of the 10 fine-grain
variability in class labels assigned by the detection model for the same
vehicle classes. Moreover, the system operated in real-time at 10 Hz
vehicle.
with a latency of 62.71 ms. The proposed system enables real-time
Region 2. This bus stop area is designated primarily for public
extraction of 3D spatial measurements and fine-grain vehicle classes for
minibuses and double-decker buses, with private cars and taxis entering
applications requiring rapid and distinct vehicle specification analysis,
only occasionally. The standardized physical design and color scheme
such as emission inventory estimation and spatial traffic flow analysis.
of public minibuses and double-decker buses facilitate the model’s abil-
With vehicle classes defined according to the EPD’s specifications, this
ity to distinguish them from other vehicle classes, given the sufficient
work can be extended to estimate pollutant emissions using models like
number of training samples.
Emission Factor Hong Kong (EMFAC-HK).
Region 3. The lower precision in vehicle counts for private cars,
The current system’s limitation lies in the computation power on
taxis, and large trucks, relative to other classes, can be linked to the
edge device. This limitation can be addressed by leveraging a more
diminished detection performance of the 3D object detection model in
powerful edge device to deploy a two-stage DV-Det model for improved
this region (see Table 6).
vehicle detection and classification. Enhancements such as temporal
6.3. System performance extraction modules (e.g., attention layers and long-short-term memory
(LSTM)) and expanded datasets for underrepresented vehicle classes
The system’s performance was evaluated on-site using the NVIDIA could mitigate variable class identification across frames, enabled by
AGX Orin edge device, which was connected to the single LiDAR sensor. the sensor interface module’s efficient point reduction. Additionally,
The comprehensive performance breakdown is presented in Table 11. future research could explore the impact of varying LiDAR installation
The vehicle monitoring system operated at 10 Hz, which is constrained heights, angular configurations, and weather conditions on system
by the LiDAR’s operational frequency, with a latency of 62.71 ms performance. Expanding the study to a multi-LiDAR framework may
from processing the point cloud frame to generating vehicle count further optimize the system’s ability to handle large-scale scenes with
statistics. The sensor interface module achieved a rate of 53.56 FPS, high-density point clouds.
the deep learning 3D object detection operated at 39.39 FPS, and the
combined tracking and counting modules functioned at up to 162.15 CRediT authorship contribution statement
FPS. The extra 37 ms of latency constraint could be allocated for
other downstream processes. For example in the V2X application, the Tun Jian Tan: Writing – review & editing, Writing – original draft,
extracted scene information can be transmitted over network to the Visualization, Validation, Software, Project administration, Methodol-
autonomous driving vehicle to perform planning and control. ogy, Investigation, Formal analysis, Data curation, Conceptualization.
Zhaoyu Su: Writing – review & editing, Writing – original draft, Soft-
7. Conclusion ware, Methodology, Investigation, Formal analysis, Conceptualization.
Jun Kang Chow: Writing – review & editing, Writing – original draft,
In this paper, we proposed a real-time vehicle fleet monitoring sys- Data curation. Tin Long Leung: Writing – review & editing, Writing –
tem that leverages both hardware and software components, utilizing original draft, Software. Pin Siang Tan: Methodology, Conceptualiza-
deep learning techniques and point cloud data. The hardware setup tion. Mei Ling Leung: Resources, Data curation. Wai Yin Gavin Wu:
included a multi-beam flash LiDAR, chosen for its high point rate and Resources, Data curation. Hai Yang: Supervision, Funding acquisition.
operating frequency, coupled with edge computing capabilities. The Dasa Gu: Supervision, Funding acquisition. Yu-Hsing Wang: Writing
software comprised a data processing pipeline with four key modules: – review & editing, Writing – original draft, Supervision, Funding
(1) sensor interface, (2) 3D deep object detector, (3) multi-vehicle acquisition.
tracker, and (4) multi-vehicle counter. We developed a novel sensor
interface module to efficiently extract points of interest from dense Declaration of Generative AI and AI-assisted technologies in the
point clouds, capitalizing on the capabilities of multi-beam flash LiDAR writing process
for effective background removal. Additionally, a GPU-based point-
in-polygonal-prism algorithm was designed to delineate RoIs. The 3D Statement: During the preparation of this work, the author(s) used
object detection model, DV-Det, was created to detect and classify on- GPT4 in order to improve readability and language. After using this
road vehicles within the processed point cloud into various classes tool/service, the author(s) reviewed and edited the content as needed
at high speed. The AB3DMOT algorithm, a 3D multi-vehicle tracking and take(s) full responsibility for the content of the publication.
system, assigns unique IDs to detected vehicles and estimates their
velocities. Using the trajectory information from tracked vehicles, our Funding
specialized counting algorithm for 3D scenes determines vehicle counts.
Our definition of vehicle fleet monitoring focuses specifically on This work was supported by the ITF Midstream Research Pro-
the detection, classification, and counting of individual vehicles, with gramme for Universities (project no. MRP/003/21X); the Hong Kong
tracking serving as a supporting component rather than as an end Research Grants Council (project no. 16205021); and HKUST-Kaisa
goal requiring complex tracking metrics or analysis of inter-vehicle Joint Research Institute. The authors are also grateful to reviewers for
relationships. their valuable comments.

18
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Declaration of competing interest Geiger, A., Lenz, P., Urtasun, R., 2012. Are we ready for autonomous driving? The
KITTI vision benchmark suite. In: Conference on Computer Vision and Pattern
Recognition. CVPR.
The authors declare that they have no known competing finan-
Ghaffarpasand, O., Talaie, M.R., Ahmadikia, H., Khozani, A.T., Shalamzari, M.D.,
cial interests or personal relationships that could have appeared to 2020. A high-resolution spatial and temporal on-road vehicle emission inventory
influence the work reported in this paper. in an Iranian metropolitan area, isfahan, based on detailed hourly traffic data.
Atmospheric Pollut. Res. 11 (9), 1598–1609. [Link]
06.006.
Data availability
Girshick, R., Donahue, J., Darrell, T., Malik, J., 2014. Rich feature hierarchies for
accurate object detection and semantic segmentation. In: Proceedings of the IEEE
Data will be made available on request. Conference on Computer Vision and Pattern Recognition. pp. 580–587.
Gonzalez, T.F., 1985. Clustering to minimize the maximum intercluster distance. The-
oret. Comput. Sci. 38, 293–306. [Link]
References 5.
He, C., Zeng, H., Huang, J., Hua, X.-S., Zhang, L., 2020. Structure aware single-stage
Abdelwahab, M.A., 2019. Accurate vehicle counting approach based on deep neural 3D object detection from point cloud. In: Proceedings of the IEEE Conference on
networks. In: 2019 International Conference on Innovative Trends in Computer Computer Vision and Pattern Recognition.
Engineering. ITCE, IEEE, pp. 1–5. Hormann, K., Agathos, A., 2001. The point in polygon problem for arbitrary polygons.
Adnan Yusuf, S., Khan, A., Souissi, R., 2024. Vehicle-to-everything (V2X) in the Comput. Geom. 20 (3), 131–144.
autonomous vehicles domain – A technical review of communication, sensor, and Hu, W., Tan, T., Wang, L., Maybank, S., 2004. A survey on visual surveillance of object
AI technologies for road user safety. Transp. Res. Interdiscip. Perspect. 23, 100980. motion and behaviors. IEEE Trans. Syst. Man, Cybern. Part C ( Appl. Reviews) 34
[Link] (3), 334–352.
Ali, W., Abdelkarim, S., Zidan, M., Zahran, M., El Sallab, A., 2018. Yolo3d: End-to-end Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O., 2019. Pointpillars:
real-time 3d oriented object bounding box detection from lidar point cloud. In: Fast Encoders for Object Detection from Point Clouds. [Link]
Proceedings of the European Conference on Computer Vision (ECCV) Workshops. CVPR.2019.01298.
Anand, B., Senapati, M., Barsaiyan, V., Rajalakshmi, P., 2021. LiDAR-INS/GNSS-Based Li, B., 2017. 3D fully convolutional network for vehicle detection in point cloud. In:
real-time ground removal, segmentation, and georeferencing framework for smart 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS,
transportation. IEEE Trans. Instrum. Meas. 70, 1–11. [Link] IEEE, pp. 1513–1518.
TIM.2021.3117661. Li, Y., Allu, K.R., Sun, Z., Tok, A.Y., Feng, G., Ritchie, S.G., 2021. Truck body type
Bello, S.A., Yu, S., Wang, C., Adam, J.M., Li, J., 2020. Review: Deep learning on 3D classification using a deep representation learning ensemble on 3D point sets.
point clouds. Remote. Sens. 12 (11), [Link] URL Transp. Res. Part C: Emerg. Technol. 133, 103461.
[Link]
Li, B., Zhang, T., Xia, T., 2016. Vehicle detection from 3d lidar using fully convolutional
Beltrán, J., Guindel, C., Moreno, F.M., Cruzado, D., Garcia, F., De La Escalera, A.,
network. arXiv preprint arXiv:1608.07916.
2018. Birdnet: a 3d object detection framework from lidar information. In: 2018
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P., 2017. Focal loss for dense object
21st International Conference on Intelligent Transportation Systems. ITSC, IEEE,
detection. In: Proceedings of the IEEE International Conference on Computer Vision.
pp. 3517–3523.
ICCV.
Bentley, J.L., 1975. Multidimensional binary search trees used for associative searching.
Commun. ACM 18 (9), 509–517. [Link] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C., 2016.
Ssd: Single shot multibox detector. In: European Conference on Computer Vision.
Besl, P., McKay, N.D., 1992. A method for registration of 3-D shapes. IEEE Trans.
Springer, pp. 21–37.
Pattern Anal. Mach. Intell. 14 (2), 239–256. [Link]
Bogoslavskyi, I., Stachniss, C., 2016. Fast range image-based segmentation of sparse Liu, C., Huynh, D.Q., Sun, Y., Reynolds, M., Atkinson, S., 2020. A vision-based pipeline
3D laser scans for online operation. In: 2016 IEEE/RSJ International Conference for vehicle counting, speed estimation, and classification. IEEE Trans. Intell. Transp.
on Intelligent Robots and Systems. IROS, pp. 163–169. [Link] Syst. 22 (12), 7547–7560.
IROS.2016.7759050. Minemura, K., Liau, H., Monrroy, A., Kato, S., 2018. LMNet: Real-time multiclass
Brahme, Y.B., Kulkarni, P., 2011. An implementation of moving object detection, object detection on CPU using 3D LiDAR. In: 2018 3rd Asia-Pacific Conference
tracking and counting objects for traffic surveillance system. In: 2011 International on Intelligent Robot Systems. ACIRS, IEEE, pp. 28–34.
Conference on Computational Intelligence and Communication Networks. IEEE, pp. Nagy, M., Werghi, N., Hassan, B., Dias, J., Khonji, M., 2024. RobMOT: Robust 3D
143–148. multi-object tracking by observational noise and state estimation drift mitigation
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., on LiDAR PointCloud. arXiv preprint arXiv:2405.11536.
Baldan, G., Beijbom, O., 2020. nuScenes: A multimodal dataset for autonomous Padrón, J.D., Hernández-Orallo, E., Calafate, C.T., Soler, D., Cano, J.-C., Manzoni, P.,
driving. In: CVPR. 2023. Realistic traffic model for urban environments based on induction loop data.
Cao, X., Wu, C., Yan, P., Li, X., 2011. Linear SVM classification using boosting HOG Simul. Model. Pr. Theory 125, 102742. [Link]
features for vehicle detection in low-altitude airborne videos. In: 2011 18th IEEE 102742.
International Conference on Image Processing. IEEE, pp. 2421–2424. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J., 2018. Frustum pointnets for 3d object
Che Puan, O., Muhamad Nor, N.S., Mashros, N., Hainin, M.R., 2019. Applicability of an detection from rgb-d data. pp. 918–927.
automatic pneumatic–tube–based traffic counting device for collecting data under Qi, C.R., Su, H., Mo, K., Guibas, L.J., 2017a. PointNet: Deep learning on point sets
mixed traffic. IOP Conf. Ser.: Earth Environ. Sci. 365, 012032. [Link] for 3D classification and segmentation. In: Proceedings of the IEEE Conference on
10.1088/1755-1315/365/1/012032. Computer Vision and Pattern Recognition. CVPR, pp. 77–85. [Link]
Chen, Y., Liu, S., Shen, X., Jia, J., 2019. Fast point R-CNN. In: 2019 IEEE/CVF 1109/CVPR.2017.16.
1175International Conference on Computer Vision. ICCV, pp. 9774–9783, URL Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017b. PointNet++: Deep hierarchical feature
[Link] learning on point sets in a metric space.
Chen, X., Ma, H., Wan, J., Li, B., Xia, T., 2017. Multi-View 3D Object Detection Network
Redmon, J., Divvala, S., Girshick, R., Farhadi, A., 2016. You only look once: Unified,
for Autonomous Driving. [Link]
real-time object detection. In: Proceedings of the IEEE Conference on Computer
Chhadikar, N., Bhamare, P., Patil, K., Kumari, S., 2019. Image processing based tracking Vision and Pattern Recognition. pp. 779–788.
and counting vehicles. In: 2019 3rd International Conference on Electronics,
Redmon, J., Farhadi, A., 2017. YOLO9000: better, faster, stronger. In: Proceedings of
Communication and Aerospace Technology. ICECA, IEEE, pp. 335–339.
the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7263–7271.
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H., 2021. Voxel r-cnn: Towards
Redmon, J., Farhadi, A., 2018. Yolov3: An incremental improvement. arXiv preprint
high performance voxel-based 3d object detection. In: Proceedings of the AAAI
arXiv:1804.02767.
Conference on Artificial Intelligence. Vol. 35, pp. 1201–1209.
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A., 2010. The pascal Ren, S., He, K., Girshick, R., Sun, J., 2015. Faster r-cnn: Towards real-time object
visual object classes (voc) challenge. Int. J. Comput. Vis. 88 (2), 303–338. detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28.
Fan, L., Pang, Z., Zhang, T., Wang, Y.-X., Zhao, H., Wang, F., Wang, N., Zhang, Z., 2022. Rezaei, M., Azarmi, M., Mir, F.M.P., 2022. Traffic-net: 3D traffic monitoring using a
Embracing single stride 3D object detector with sparse transformer. In: Proceedings single camera. arXiv arXiv:2109.09165.
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR, Sánchez-Oro, J., Fernández-López, D., Cabido, R., Montemayor, A., Pantrigo, J., 2013.
pp. 8458–8468. Radar-based road-traffic monitoring in urban environments. Digit. Signal Process.
Feng, M., Hu, S., Lee, G., Ang, M., 2018. Towards precise vehicle-free point cloud 23 (1), 364–374. [Link]
mapping: An on-vehicle system with deep vehicle detection and tracking. In: 2018 Sankari, M., Meena, C., 2010. Adaptive background estimation and object detection
IEEE International Conference on Systems, Man, and Cybernetics. SMC, IEEE, pp. applying in automated visual surveillance. IJCSIS) Int. J. Comput. Sci. Inf. Secur.
1288–1293. 8 (4).

19
T.J. Tan et al. Engineering Applications of Arti cial Intelligence 159 (2025) 111534

Schnabel, R., Wahl, R., Klein, R., 2007. Efficient RANSAC for point-cloud shape Wojke, N., Bewley, A., Paulus, D., 2017. Simple online and realtime tracking
detection. Comput. Graph. Forum 26 (2), 214–226. [Link] with a deep association metric. In: 2017 IEEE International Conference on Im-
1467-8659.2007.01016.x, arXiv:[Link] age Processing. ICIP, IEEE, pp. 3645–3649. [Link]
j.1467-8659.2007.01016.x. URL [Link] 8296962.
j.1467-8659.2007.01016.x. Yan, Y., Mao, Y., Li, B., 2018. Second: Sparsely embedded convolutional detection.
Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., Li, H., 2020. Pv-rcnn: Point-voxel Sensors 18 (10), 3337.
feature set abstraction for 3d object detection. pp. 10529–10538. Yang, B., Liang, M., Urtasun, R., 2018a. Hdnet: Exploiting hd maps for 3d object
Shi, S., Wang, X., Li, H., 2019a. PointRCNN: 3D Object Proposal Generation and detection. In: Conference on Robot Learning. PMLR, pp. 146–155.
Detection from Point Cloud. [Link] Yang, B., Luo, W., Urtasun, R., 2018b. Pixor: Real-time 3d object detection from point
Shi, S., Wang, Z., Shi, J., Wang, X., Li, H., 2019b. Part-A2 net: 3D part-aware and clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern
aggregation neural network for object detection. Proc. the IEEE Comput. Soc. Conf. Recognition. pp. 7652–7660.
Comput. Vis. Pattern Recognit.. Yang, H., Qu, S., 2018. Real-time vehicle detection and counting in complex traffic
Simonelli, A., Bul00F2, S.R., Porzi, L., Antequera, M.L., Kontschieder, P., 2022. Dis- scenes using background subtraction model with low-rank decomposition. IET
entangling monocular 3D object detection: From single to multi-class recognition. Intell. Transp. Syst. 12 (1), 75–85.
IEEE Trans. Pattern Anal. Mach. Intell. 44 (3), 1219–1231. [Link] Yang, Z., Sun, Y., Liu, S., Jia, J., 2020. 3DSSD: Point-based 3D single stage object
1109/TPAMI.2020.3025077. detector. Proc. the IEEE Conf. Comput. Vis. Pattern Recognit..
Simony, M., Milzy, S., Amendey, K., Gross, H.-M., 2018. Complex-yolo: An euler-region- Yang, Z., Sun, Y., Liu, S., Shen, X., Jia, J., 2019. Std: Sparse-to-dense 3d object detector
proposal for real-time 3d object detection on point clouds. In: Proceedings of the for point cloud. pp. 1951–1960,
European Conference on Computer Vision (ECCV) Workshops. Yang, K., Sun, P., Yang, D., Lin, J., Boukerche, A., Song, L., 2024. A novel hierarchical
Song, H., Liang, H., Li, H., Dai, Z., Yun, X., 2019. Vision-based vehicle detection and distributed vehicular edge computing framework for supporting intelligent driving.
counting system using deep learning in highway scenes. Eur. Transp. Res. Rev. 11 Ad Hoc Netw. 153, 103343. [Link]
(1), 1–16. Yao, L., 2019. An effective vehicle counting approach based on CNN. In: 2019 IEEE 2nd
Su, Z., 2021. Advanced 2D and 3D Computer Vision for a Smarter City : From Image International Conference on Electronics and Communication Engineering. ICECE,
to Point Cloud. [Hong Kong University of Science and Technology], Hong Kong]. IEEE, pp. 15–19.
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Ye, Y., Chen, H., Zhang, C., Hao, X., Zhang, Z., 2020. Sarpnet: Shape attention regional
Zhou, Y., Chai, Y., Caine, B., Vasudevan, V., Han, W., Ngiam, J., Zhao, H., proposal network for lidar-based 3d object detection. Neurocomputing 379, 53–63.
Timofeev, A., Ettinger, S., Krivokon, M., Gao, A., Joshi, A., Zhang, Y., Shlens, J., Ye, M., Meyer, G.P., Chai, Y., Liu, Q., 2023. Efficient transformer-based 3D object de-
Chen, Z., Anguelov, D., 2020. Scalability in perception for autonomous driving: tection with dynamic token halting. In: Proceedings of the IEEE/CVF International
Waymo open dataset. In: Proceedings of the IEEE/CVF Conference on Computer Conference on Computer Vision. ICCV, pp. 8438–8450.
Vision and Pattern Recognition. CVPR. Zaidi, S.S.A., Ansari, M.S., Aslam, A., Kanwal, N., Asghar, M., Lee, B., 2022. A survey
Tan, S.H., Chuah, J.H., Chow, C.-O., Kanesan, J., Leong, H.Y., 2024. Artificial intelligent of modern deep learning based object detection models. Digit. Signal Process. 126,
systems for vehicle classification: A survey. Eng. Appl. Artif. Intell. 129, 107497. 103514. [Link]
[Link] Zeineldin, R., El-Fishawy, N., 2016. Fast and Accurate Ground Plane Detection for the
Tay, L., Lim, J.M.-Y., Liang, S.-N., Keong, C.K., Tay, Y.H., 2023. Urban traffic volume Visually Impaired from 3D Organized Point Clouds. pp. 373–379. [Link]
estimation using intelligent transportation system crowdsourced data. Eng. Appl. org/10.1109/SAI.2016.7556009.
Artif. Intell. 126, 107064. [Link] Zhang, J., Guo, X., Zhang, C., Liu, P., 2021. A vehicle detection and shadow elimination
Tourani, A., Shahbahrami, A., 2015. Vehicle counting method based on digital method based on greyscale information, edge information, and prior knowledge.
image processing algorithms. In: 2015 2nd International Conference on Pattern Comput. Electr. Eng. 94, 107366.
Recognition and Image Analysis. IPRIA, IEEE, pp. 1–6. Zhang, J., Lu, Y., Liu, C., Sun, G., Li, Z., 2015. A new smart traffic monitoring
Trivedi, J.D., Mandalapu, S.D., Dave, D.H., 2022. Vision-based real-time vehicle method using embedded cement-based piezoelectric sensors. Smart Mater. Struct.
detection and vehicle speed measurement using morphology and binary logical 24, [Link]
operation. J. Ind. Inf. Integr. 27, 100280. Zhang, J., Xiao, W., Coifman, B., Mills, J.P., 2020. Vehicle tracking and speed
Weiler, K., 1994. An incremental angle point in polygon test. In: Graphics Gems. estimation from roadside lidar. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.
Weng, X., Wang, J., Held, D., Kitani, K., 2020. 3D multi-object tracking: A baseline and 13, 5597–5608.
new evaluation metrics. In: 2020 IEEE/RSJ International Conference on Intelligent Zhao, J., Xu, H., Liu, H., Wu, J., Zheng, Y., Wu, D., 2019. Detection and tracking of
Robots and Systems. IROS, IEEE, pp. 10359–10366. pedestrians and vehicles using roadside LiDAR sensors. Transp. Res. Part C: Emerg.
Wojke, N., Bewley, A., 2018. Deep cosine metric learning for person re-identification. Technol. 100, 68–87.
In: 2018 IEEE Winter Conference on Applications of Computer Vision. WACV, IEEE, Zhou, Y., Tuzel, O., 2018. Voxelnet: End-to-end learning for point cloud based 3d object
pp. 748–756. [Link] detection. pp. 4490–4499,

20