Received 11 May 2023, accepted 26 May 2023, date of publication 1 June 2023, date of current version 8 June 2023.

Digital Object Identifier 10.1109/ACCESS.2023.3282110

Transformer-Based Attention Network for

In-Vehicle Intrusion Detection
TRIEU PHONG NGUYEN 1 , HEUNGWOO NAM 2, (Member, IEEE),
AND DAEHEE KIM 1 , (Member, IEEE)
1 Department of Mobility Convergence Security, Soonchunhyang University, Asan 31538, South Korea
2 Department of Computer Engineering, Daegu University, Gyeongsan 38453, South Korea
Corresponding authors: Daehee Kim (daeheekim@[Link]) and Heungwoo Nam (hwnam@[Link])
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (No.
2021R1A4A2001810, 2022H1D8A3038040), by Institute for Information & communications Technology Planning & Evaluation (IITP)
grant funded by the Korea government (MSIT) (No. 2022-0-01197, Convergence security core talent training business (SoonChunHyang
University)), by ‘‘Regional Innovation Strategy (RIS)’’ through the National Research Foundation of Korea (NRF) funded by the Ministry
of Education (MOE) (2021RIS-004), by Basic Science Research Program through the National Research Foundation of Korea (NRF)
funded by the Ministry of Education (No. 2022R1I1A3066416), and this work was supported by the Soonchunhyang Research Fund.

ABSTRACT Despite the significant advantages of communication systems between electronic control units,
the controller area network (CAN) protocol is vulnerable to attacks owing to its weak security structure. The
persistent development of intrusion detection systems (IDS) is geared toward preventing vehicles from being
targeted by malicious attacks. Recurrent neural networks (RNNs) have emerged as a prominent approach
in this domain, contributing significantly to the evolution of IDS. Nonetheless, RNN-based methods have
certain limitations in step-by-step processing. Their feature extraction at any given point in time only relies
on the hidden state of previously observed information, possibly resulting in missing features in the context
vector. In this paper, we propose a novel multi-class IDS using a transformer-based attention network
(TAN) for an in-vehicle CAN bus. Our model builds on the self-attention mechanism, removing RNNs and
classifying attacks into multiple categories. Furthermore, the proposed models can detect replay attacks by
aggregating sequential CAN IDs. The experimental results indicate that the TAN is more efficient than the
baselines for different input data types and datasets. The highlight is that, although sequential CAN IDs are
used, our model can identify intrusion messages without requiring message labeling. Finally, by inheriting
the advantages of transformers, TAN employs transfer learning to improve the performance of models trained
on small data from other car models.

INDEX TERMS CAN, intrusion detection system, multi-class classification, replay attacks, self-attention,
transformer.

I. INTRODUCTION work (IVN) is the controller area network (CAN). The CAN
Due to recent developments in the field of intelligent vehi- exhibits many advantages over other networks, including the
cles, modern cars can provide drivers with an excellent local interconnected network (LIN) and FlexRay, owing to
experience. However, rapidly evolving utilities and functions its high speed and efficiency. In addition to serving as an
require expanding car control systems [1]. Electronic control IVN, the CAN has been effectively employed in numerous
units (ECUs), which serve as building blocks for complex industrial applications, including aircraft, railways, and X-ray
in-vehicle systems, regulate the message broadcast network machines [3]. However, owing to its efficiency and simple
to ensure smooth operation between components such as design, the CAN bus is vulnerable to cyberattacks because
the throttle, brake, engine, and steering wheel [2]. The units intruders can easily influence physical subsystems. For exam-
transmit control signals through networks by using a network ple, an attacker can inject harmful messages directly through
protocol. The most extensively implemented in-vehicle net- physical onboard diagnostic (OBD-II) ports or access the
network remotely using telematics services [4].
The associate editor coordinating the review of this manuscript and Many manual approaches, such as certification, scan-
approving it for publication was Vicente Alarcon-Aquino . ners, access control, and firewalls, have been proposed to

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

VOLUME 11, 2023 For more information, see [Link] 55389
 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

FIGURE 2. Structure of the CAN data frame.

ment without requiring a message level labeled dataset.

FIGURE 1. Voltage levels of controller area network bus. We exploit the attention scores to identify malicious
CAN IDs in the segment, enabling the model to remove
resolve the vulnerability of CAN buses. A point of con- only the attack messages from the CAN bus.
cern is that manual methods can only passively prevent 4) We considered the possibility of applying transfer
malicious intrusions by using limited rules. The security learning to our model. Our data problem-solving efforts
problem in IVN requires a system that can analyze and yielded positive results when experience from the
learn the features in CAN data and use them to recog- source model improved the performance of the target
nize abnormal messages outside the scope of configurations. model.
Machine-learning algorithms have joined security races with
The remainder of this paper is organized as follows.
great potential. Research on recurrent neural network (RNN)
In Section II, we present background material pertaining to
algorithms yields state-of-the-art (SOTA) performance in
the CAN bus and various attack scenarios. Section III pro-
intrusion detection systems [5]. However, RNNs face bottle- vides an overview of previous studies on intrusion detection
necks in the extraction of unselective information. Accord- systems in CAN bus. In Section IV, the core of the proposed
ingly, this paper proposes a novel approach to improving model is presented. In Section V, we compare the perfor-
intrusion detection. mance of our approach with that of other machine learning
Transformers perform excellently in many fields, includ- methods and examine the results of transfer learning. Finally,
ing natural language, image, and audio processing. With we conclude the paper and describe the future directions of
the ability to extract features through correlation calculation our research in Section 7.
data components, we consider the transformer efficiency of
multi-class classification in IVN. The key contributions of our II. BACKGROUND
approach are summarized as follows: The following section summarizes the inner workings of
1) We propose an intrusion detection system (IDS) based the CAN protocol and describes the most common attack
on a transformer attention network for a CAN bus that scenarios in in-vehicle IDS.
uses a single message. We processed the data by remov-
ing unnecessary elements in each message, merging the A. CONTROLLER AREA NETWORK BUS
datasets, relabeling them for multi-class classification, In the 1980s, Robert Bosch developed a CAN that allowed
and converting them to a data type suitable for machine ECUs to communicate with each other [6]. As the com-
learning. Experimental results demonstrate the superior munication standard for ECU in in-vehicle networks, CAN
ability of our model to classify intrusion messages on design considers three objectives: reliability, effectiveness,
the Car Hacking dataset compared to that of SOTA and economics. The modern CAN bus transfers information
RNN-based IDS [5]. regarding the steering wheel, fuel gauge, brakes, and engine
2) A single-message IDS can easily confuse malicious to maintain data synchronization in vehicles [7]. Despite
and normal messages if an attacker injects messages these advantages, CAN lack security features. The initial
extracted from the CAN bus. We proposed a model design of the CAN bus did not consider authentication of
that uses sequential CAN IDs to solve this problem. messages because the ECU did not connect to the external
We used sliding windows to aggregate CAN IDs into network.
segments and label sequential CAN ID inputs at the The underlying concept of the CAN bus is a multi-master
segment level. Our experiment was conducted with sev- competition [8] wherein each ECU node can send and receive
eral window sizes to evaluate the tradeoff between each messages from hundreds of sensors without permission.
model and determine the optimal parameters. When The unique CAN ID determines the priority of each CAN
evaluating the effectiveness of our proposed model message.
using the replay attack, which involves reusing mes- As shown in Fig. 1, the two lines in the CAN bus have
sages extracted from the CAN bus, we observed that certain voltage levels to transmit either the dominant bit
it achieved the lowest error rates compared with the (0 bit) or recessive bit (1 bit), where the dominant bit is
selected baseline models. prioritized over the recessive bit. Consequently, a lower ID
3) To the best of our knowledge, other models that use indicates a higher priority [9]. In multi-master and priority-
sequential CAN IDs can detect only abnormal seg- based attacks, an attacker can inject messages with the high-
ments. Our results indicate that the proposed model est priority into the CAN bus and then flood the in-vehicle
can identify intrusive messages in any malicious seg- network through denial-of-service (DoS) attacks.

55390 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

ACK (acknowledgment field). The confirmation compo-

nent indicates the success status (two bits).
EOF (end of the frame field). The 7-bit delimiter verifies
bit stuffing and frame finalization.
These two specifications exhibit few structural differences.
The only distinction is that the arbitration ID of the standard
frame accommodates 11- or 29-bit options, whereas that of
FIGURE 3. DoS attack. the extended frame allows only 29 bits.
Several studies have indicated that a period exists between
messages with the same identifier in the CAN bus [4], [10].
Because each ECU sends data every 10 ms, we assume loss
of periodicity upon the injection of a malicious message.
Thus, sequential IDs can be used in the CAN bus to recognize
malicious threats.

B. ATTACK MODELS
FIGURE 4. Fuzzy attack.
The lack of network segmentation, authenticity, and
data encryption represents a fundamental vulnerability of
in-vehicle CAN [11]. Lee et al. [12] and Song et al. [13]
demonstrated that ECU nodes do not consider the sources
of CAN messages. Consequently, cyber-attackers can easily
carry out common network attacks on the CAN bus. The
attack scenarios considered in this study are introduced in
this section and depicted in Figs. 3–6.
FIGURE 5. Spoofing attack.
1) FLOOD ATTACK
Common (denial-of-service) DoS attacks. The flood attack
method involves submitting a large number of valid requests
that exceed the CAN bus processing capabilities. The attacker
intends to exhaust the system resources and paralyze the
functions.

2) FUZZY ATTACK
FIGURE 6. Replay attack.
The attacker repeatedly injected the CAN bus by using ran-
dom messages. To disrupt the vehicle’s functions, the attacker
examines in-vehicle information CAN packets and selects the
There are two design criteria for the CAN message format:
target CAN IDs to cause unexpected activity.
a standard frame (CAN 2.0A) and an extended frame (CAN
2.0B). Fig. 2 illustrates the general architecture of a CAN
frame. The following functional descriptions were applied to 3) SPOOFING ATTACK
the constituent fields: This is also known as a malfunction (Mal) attack. During
SOF (start of the frame field). The dominant bit repre- a spoofing attack, the intruder uses the CAN IDs obtained
sents the synchronization and the start of the frame. to target the specific subsystem functions in the vehicle.
CAN ID (arbitration field). Identification and frame pri- Abnormal messages are generated by gradually changing the
ority for arbitration. payload.
RTR (remote transmission request). The identity bit
indicates whether the frame is remote or a data frame. 4) REPLAY ATTACK
IDE (identifier extension). The bit identifies the CAN An attacker captures a legitimate message and injects it
frame type. A dominant bit defines the 11-bit standard, randomly into the CAN bus. These abnormal messages
whereas a recessive bit defines the 29-bit extended frame. force vehicles to repeat actions with potentially dangerous
DLC (data length code). The size of the DLC ranged results. These attacks are difficult to detect without examining
from 0 to 8 bits, indicating the byte length of the data field. sequential patterns in the CAN.
Data field. The 64-bit payload is responsible for commu-
nication between ECUs. III. RELATED WORK
CRC (cyclic redundancy check field). 16 control the data An IDS is a network security technology built to detect signs
checksum. of suspicious presence and generate alerts when cybersecurity

VOLUME 11, 2023 55391

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 1. Comparision of deep learning studies for CAN IDS.

threats are discovered [9]. Because CAN lacks built-in secu- as evaluated using their dataset. They also demonstrated that
rity features, the in-vehicle network system utilizes an IDS to an LSTM-based IDS is more efficient for in-vehicle intrusion
monitor malicious activities between the CAN and external detection than a survival analysis method [18]. Khan et al.
interfaces. Many approaches have been developed to identify developed a multi-stage deep learning framework which com-
intrusions on the CAN buses. bines normal state-based and a bidirectional LSTM archi-
According to its design, Wu et al. categorized in-vehicle tecture [19]. Their outcomes indicated that the framework
network IDS a fingerprints-based (bus level), parameter- outperforms a variety of current state-of-the-art systems and
monitoring-based (message level), information-theoretic- can detect zero-day outbreaks from Internet of Vehicles
based (data-flow level), and machine-learning-based networks. In another periodicity-based study, Song et al.
(functional level) [14]. Machine learning and deep learning developed a CNN-based IDS to extract spatial features from
represent the primary approaches in the research field of 29 sequential CAN ID data frames [4]. Furthermore, their
in-vehicle network IDS. With outstanding advantages in model was optimized to minimize unnecessary components
learning data behaviors, machine- and deep-learning-based in the inception-resnet architecture. In another approach,
IDS are capable of defending against unidentified attacks Hoang et al. presented a semi-supervised learning method
within the CAN bus through input data analysis. using convolutional adversarial autoencoders (CAAE) [20].
In terms of data mining, Sun et al. categorized intrusion Their methodology leveraged the autoencoder mechanism
detection algorithms into three groups: semantic-based, data for unsupervised training, followed by supervised fine-tuning
domain-based, and periodicity-based [10]. with partial data to enhance the precision of the model. Their
Semantic-based methods attempt to analyze the semantics lightweight model achieved a high performance within a 40%
of CAN messages. Wasicek et al. developed a context-aware dataset compared to the baseline model.
IDS to classify cyber threats in in-vehicle network sys- Other studies have attempted to combine the spatial and
tems [15]. Their approach exploits the information obtained temporal features [5], [21]. Lo et al. used a convolutional neu-
by an existing sensor to improve the precision of an IDS ral network (CNN) and LSTM to learn representative features
by using an ANN. The report showed that semantic-based from CAN traffic [5]. The author performed binary classifica-
methods, such as chip tuning, can unveil anomalous manip- tion to demonstrate that the hybrid deep learning-based IDS
ulations. In another approach, a data-domain-based IDS successfully recognized suspicious behaviors in the CAN
addresses the vulnerability of CAN by exploiting the regu- bus. The results on the Car Hacking Dataset [4] showed that
larity of CAN IDs. Based on RNN algorithms, these meth- their method outperformed other related works.
ods predict the next bit or package. For instance, Qin et al. CNN-LSTM achieved SOTA performance in the binary
suggested long short-term memory (LSTM) to forecast CAN classification of the CAN bus. However, this approach does
messages in two data formats [16] and compared several loss not consider the natural sequential CAN IDs. These short-
functions for time-series prediction to achieve optimal results. comings can be exploited for replay attack scenarios in which
Similarly, periodicity-based IDS use sequential patterns of the attacker utilizes injected malicious messages extracted
CAN messages to identify malicious sequences in data traffic. from the CAN bus.
Hossain et al. proposed an LSTM-based IDS that learns the Research related to transfer learning has also been applied
temporal features from multivariate time-series CAN traf- to periodicity-based models such as LSTM. Kang et al.
fic [17]. The overall accuracy of this approach was 99.995% deployed LSTM-based transfer learning for in-vehicle CAN

55392 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

FIGURE 7. Overview of proposed architecture.

and evaluated it using a survival dataset [22]. Their report most studies have ignored that recurrent neural network
indicated that the performance of a model trained on a small (RNN) algorithms incur vanishing and exploding gradient
dataset can be improved significantly. The transfer learning problems [25].
approach related to vehicle intrusion detection systems has With advancements in computer processing capabilities,
yielded promising results. Tariq et al. used transfer learning various recent research studies have proposed utilizing
combined with CNN-LSTM to demonstrate the ability to self-attention mechanisms to tackle the limitations of RNNs.
transfer knowledge from large to small datasets [23]. Through Nam et al. deployed bidirectional GPT for an in-vehicle
one-shot learning, CNN-LSTM transfer learning achieved a IDS [26]. The author performed binary classifications to
performance improvement of 26.06% over the baselines. demonstrate that their model could detect the anomalous
In addition to the spectral and temporal features, sev- segment in the case of an extremely sparsely attacked CAN
eral recent studies have considered the correlation between ID. However, the authors did not provide any comparisons
each element in CAN traffic. Wei et al. proposed an intru- regarding the time inference. Moreover, the limitation of their
sion detection model based on an attention mechanism and work is discussed, as the model can only detect malicious
autoencoder [24]. They applied an encoder and decoder to message segments without identifying injected CAN IDs.
represent the potential features of the CAN messages. The Wu et al. introduced a robust transformer-based approach
attention layer then learns the importance of the extracted for IDS on network traffic [27]. According to their report,
hidden features and feeds them into the classification layer their model outperforms support vector machine and deep
to calculate the prediction result. According to their exper- learning algorithms, including RNN, on both individual
imental results, a hybrid of an autoencoder and an atten- attack types and multi-class datasets in terms of efficiency.
tion mechanism is highly promising for the OTIDS dataset. Their model had a longer training time than other mod-
Other methodologies based on the attention mechanism have els, and comparisons of inference time were not presented.
investigated the selection of points of significance in a target In another study, Sharma et al. approach transformer model
from hidden features. Sun et al. proposed a CNN-LSTM and by using CNN for embeding layer [28]. The results show
attention model using bit flip rate preprocessing [10] and that using CNN can improve model performance in pre-
demonstrated the model’s robustness and reliability for intru- dicting driver intent. Our study focus on developing an
sion detection systems. They also discussed the shortcomings intrusion detection system based on the transformer architec-
and future research directions to improve their performance. ture for in-vehicle CAN bus. Table 1 presents several compar-
The attention mechanism was combined with other algo- isons between our proposed method and previous CAN IDS
rithms to preserve useful features in data traffic. However, studies.

VOLUME 11, 2023 55393

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

Algorithm 1 Training for sequential CAN IDs

Input:
D = [{x0 , y0 } , {x1 , y1 } , . . . , {xw , yw }] : the dataset with
xi is sequential CAN IDs, yi is the label for each xi , w is
window size.
n_epochs : the number of epochs for training.
window_size : the number of CAN IDs in a sequence.
num_heads : the number of attention heads.
ff_dim : the hidden layer size in feed forward network.
embed_size : the embedding dimension for each token.
Output: θC : The weight for classifier
[α0 , . . . , αw ] : The attention score for a sequential CAN IDs
1: for epoch from 1 to n_epochs do
2: Embedding input by fembed ([x0 , . . . , xw ])
3: Adding positional encoding:
input =f embed ([x0 , . . . , xw ]) + fembed ([0, . . . , w])
4: Updating multi-head attention:
Multihead(Q, K, V) = Concat(head1 , . . . , headn )WO
where
Q
headi = Attention(QWi , KWK
i , VWi )
V

WO ∈ Rhdv ×dm
5: Updating θC by optimizing Lcross_entropy
6: Updating [α0 , . . . , αw ] by Multihead(Q, K, V)
FIGURE 8. Architecture of transformer-based attention network.
7: end for

IV. METHODOLOGY
The following section proposes a novel multi-class IDS using
a transformer-based attention network for an in-vehicle CAN
bus. Our proposed IDS comprises the two models shown in
Fig. 7: an IDS using only a single message and an IDS using
sequential CAN IDs. Note that the first model detects DoS,
fuzzy, and spoofing attacks better than SOTA RNN-based
IDS [5]. However, it cannot detect replay attacks because
it uses only a single message. Accordingly, we designed a FIGURE 9. Single message preprocessing.
second model to detect replay attacks by using sequential
CAN IDs.
identity mappings to the input layer. These mappings are
A. OVERALL ARCHITECTUTE then added to the output layer. Context vectors are generated
For a comprehensive evaluation, the proposed system (Fig. 7) at the end of the encoding phase. To mitigate overfitting
consists of two components corresponding to different data during training, we employed a technique that dropout lay-
types: a single message and sequential CAN IDs. ers with 0.15 rate are interleaved with normalization layers.
In the single-message component, we fed the preprocessed Subsequently, selective information is fed to a simple neural
data into a classification model based on the transformer network such as feed-forwarding layers. The output of the
architecture. The two primary components of a transformer multi-class classifier was a linear layer that performed prob-
are the encoder and decoder [29]. The encoder is the most abilistic predictions for each class. The probabilities are fed
essential component for classification. As shown in Fig. 8, into a softmax function, and the class with the highest return
to remove recursion and convolution, the transformer-based value results from classification.
attention network must embed positional encoding to mark For sequential CAN IDs, we performed data processing
the arrangement of the tokens in the input. The combina- using only the identified fields. Packets containing sequential
tion of input and positional encoding enables the model to CAN IDs are fed into the classification model. We use the
obtain sequence features. After adding positional informa- same model as that used in the single-message approach.
tion, the self-attention algorithm calculates the correlation Unlike single-message inputs, the components in sequen-
between tokens in a single message. To maintain propaga- tial CAN IDs represent identifiers. The use of sequential CAN
tion of signals, we used residual connections that perform IDs overcomes the repeated intrusions extracted from the

55394 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

Algorithm 2 Label Generation for Sequential CAN IDs

Input:
The batch of sequential CAN ID and each label
[{x0 , y0 } , {x1 , y1 } , . . . , {xw , yw }] × b with
{xi , yi } are CAN ID and message label
w is window size
b is batch size
Output:
The sequential CAN IDs and segmentation label Xj , Yj
 

1: for j P from 0 to (b − w) do
2: if i=0 w yi = 0 then Yj = 0
3: else
FIGURE 10. Sequential CAN ID preprocessing.
4: for i = 0 to w do
5: if yi ̸ = 0 then Yj = yi
6: end for
we use sliding windows to generate sequential CAN IDs,
enabling the detection of anomalies at the segment level.
In a single message, we use the arbitration field, data length
code, and data payload (Fig. 9). All identifications and values
CAN bus. Consider the example of an accepted CAN message in the data field were converted from hexadecimal to decimal
with a brake signal. The attacker can extract these messages formats. Each data field value is a byte (8 bits) in size and
from the CAN and inject them into an in-vehicle network. is assigned a value from 0 to 255. To maintain a uniform
The single-message IDS then accepts intrusive brake signals data length, we assigned null values of 256. However, the
as safety messages, posing a danger to the driver. The training actual length of the message is still the specified by the DLC.
process for the sequential CAN IDs data type is summarized We then define 0 and 1 as normal and anomalous messages,
in Algorithm 1. respectively [5].
The use of a single-message input can identify malicious For sequential CAN IDs, we considered only sequential
messages in the CAN bus, whereas sequential CAN IDs can patterns on the CAN bus. As shown in Fig. 10, CAN IDs
only detect anomalous segments. To identify malicious mes- were similarly converted into numeric forms. In each mes-
sages in sequential CAN IDs, we extracted attention scores. sage, the T and R tags represent an anomaly and a normal
Following classification, these scores highlight malicious message, respectively. We assume that only one attack type
messages in the segment. occurs in each segment and consider continuous CAN IDs as
The transformer is well-known for its transfer learning time-series data for each time step. A sliding window divides
capabilities in language processing, image classification, and all identifications in the dataset among the data segments and
audio classification tasks [30], [31], [32]. In this study, uses them as the input data. Suppose that, for a dataset with
we considered the feasibility of using a pre-trained model to n records, the input sequence at the ith time window is
improve the performance of limited data. The transfer learn-
ing technique can provide knowledge trained on a model with Xi = [xi1 , xi2 , . . . , xiw ], i ≤ (n − w) (1)
extensive data from one car to a model with less data from
another car. We applied the same data processing method to where w is the window size and xit represents the identifier of
the target dataset. The target and source data were trained the CAN message at time t. The label generation process is
using the same model. For multi-class classification, the out- presented in Algorithm 2.
put layer can be adjusted to fit the target data.
C. IDENTIFICATION FOR EACH MALICIOUS MESSAGE
B. PREPROCESSING DATA As previously mentioned, the self-attention mechanism rep-
The quality of data has a significant impact on the quality of resents the core of the transformer model. That is, self-
training. Because machine learning methods cannot directly attention is the most significant development step that
use the raw data extracted from the vehicle, preprocessing differentiates transformers from traditional recurrent neu-
must be applied to convert the data into a suitable type. ral networks, such as RNNs and LSTM. Because the
We removed redundant values, filled in null values, and self-attention mechanism assigns higher weights to important
transformed the data into a standardized form to ensure data information, we assume divergence between the attention
usability. We consider two critical data types: single messages scores of normal and abnormal messages in each CAN ID
and sequential CAN IDs. Although single-message intrusion segment. We extracted the attention scores to demonstrate
detection approaches yield high performance [5], they present the ability to identify malicious IDs. We used the multi-head
a vulnerability if the attacker injects replay messages that attention layer’s output to calculate the attention score and
were previously verified to be secure. To address this issue, exploit it for message intrusion identification.

VOLUME 11, 2023 55395

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

The underlying concept of the self-attention mechanism is TABLE 2. Car hacking dataset statistics.
to map a set of query and key-value vector pairs to an out-
put vector. Similar to the attention mechanism, self-attention
receives (Q, K , V ) as the input, where Q, K , and V are the
query, key, and value matrices, respectively. (Q, K , V ) is sep-
arated into sub-space representations, also known as heads,
in which each attention weight is focused on a different head
in parallel. Because of parallel computation, we concatenated
the heads’ individual results to obtain the final attention TABLE 3. INV intrusion detection dataset statistics.
weights. Therefore, the multi-head self-attention function can
be expressed as follows:
Q
headi = Attention(QWi , KWiK , VWiV ) (2)
Multihead(Q, K , V ) = Concat(head1 , . . . , headn )WO (3)
where WO ∈ Rhdv ×dm are the output parameter matrices, dv
is a dimension of value matrices, dm is a dimension of the
model. TABLE 4. Survival analysis dataset statistics.
Instead of allowing the output to attend to important fea-
tures of the input, the self-attention mechanism allows input
tokens to interact with each other. The query and key are iden-
tical in the self-attention mechanism [29]. Assume that the
CAN input is X = [x1 , x2 , . . . , xn ], with dm
 ntokens, xi ∈d R×d .
The initialized trainable parameters WQ , WK ∈ R m k,

WV ∈ R d m ×dk were randomly generated and updated regu-

larly for attention calculation. K and V can be expressed as
follows: the attention scores of the injected CAN IDs increased, as the
(
K = XWK other attention scores decreased. We deployed malicious seg-
(4) ments with 16 CAN IDs to evaluate the performance of
V = XWV
the proposed model in identifying intrusive messages. The
Each attention score is a dot product of the attention of a attention scores of each segment were flattened by the sum
query and a set of key values [29]. Each correlation score function before fed into the sigmoid function to classify
between a query qi and values in Kis presented as weight through the threshold. Normal IDs with low attention scores
j
wi through the softmax function. The attention weights were returned 0, whereas injected IDs with high attention scores
then combined with the values to represent the context vector, returned 1.
indicating where the model pays more attention. The fol-
lowing equations can be used to calculate the dot-product D. TRANSFER LEARNING
attention: Transfer learning solves critical problems in in-vehicle intru-
qi K T = [qi k1T qi k2T . . . qi knT ] (5) sion detection, training, and data collection. As the data
varied with respect to the vehicle model, data capture was
SoftMax(qi K ) = [wi1 wi2 . . . win ]
T
(6) performed whenever a new car model was released. Extract-
Xn
SoftMax(qi K T )V = wij vj (7) ing sufficient data from new car models is time-consuming
j=1
and expensive. The advantage of transfer learning is its ability
With all queries, the formula for the scaled dot product atten- to reuse the information trained on a large dataset. Thus,
tion matrix is a target model trained on a small dataset can be improved

QK T
 using knowledge from the source model.
Attention(Q, K , V ) = SoftMax √ (8) As described in [33], D = {X , P(X )} and T = {Y , f }
dk
√ represent the domain and task of each dataset, respectively.
where d k is the scaling factor to counteracts extremely We prepared the source (XS ) and target (XT ) datasets with
small gradients in softmax function as the growth of the dot equivalent feature spaces XS = XT , and different marginal
product in magnitude. distributions P (XS ) ̸= P (XT ) [34]. Thus, DS ̸= DT . We used
Our experiment used only the data labeled at the seg- the Survival dataset for XT and the Car Hacking dataset for
ment level, where malicious segments included both normal XS . Our objective was to improve the performance of learned
and attack messages. Although we did not require a labeled decision functions, which are expressed as the probability
message level dataset for the training phase, our model distribution of P(YT |XT ) in DT , by utilizing the knowledge
could identify malicious messages. Through self-attention, implied in DS [35].

55396 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 5. Hyperparameter settings.

After obtaining an optimal model from the source data,

we unfroze the model and modified the output layer. The
model was trained on the source data with an output of five, FIGURE 11. Embedding dimension comparison.
different from the target data with four classes. Subsequently,
the new model was trained on the target dataset using the
appropriate output. The target model utilizes source model library (version 2.4.1). Training was performed on an Intel(R)
parameters to train the limited data. Hyperparameters, such Xeon(R) Silver 4108 CPU @ 1.80GHz, 128 GB RAM, and
as the learning rate, were adjusted to avoid overfitting. Fol- four NVIDIA TITAN RTX GPU. We implemented a data
lowing the training phase, the target model was fine-tuned to generator based on the Keras functions to handle massive
achieve an optimal performance. datasets. The model checkpoints were set regularly.

V. EXPERIMENTS C. EVALUATION METRICS

A. DATASETS Accuracy, precision, recall, F-measure, and error rate were
To test our model, we implemented it on three com- adopted as the performance metrics to evaluate the effective-
monly used datasets from the Hacking and Countermeasures ness of the transformer-based attention network for intrusion
Research Lab in Korea. All the attack datasets were concate- detection systems. We were particularly interested in the error
nated to perform multi-class classification. rate, which presented false-negative and false-positive results.
A high false-negative rate corresponds to a significant
1) CAR HACKING DATASET [4] number of missed intrusions, which pose a danger to drivers
As shown in Table 2, the Car Hacking dataset encompasses in long-term attack scenarios. In contrast, a high false-
four attack datasets from the Hyundai Sonata car model: a positive rate may overwhelm users with false alarms [37].
DoS attack, fuzzy attack, and two spoofing attacks (corre- In addition to the multi-class classification model, we used a
sponding to the drive gear and revolutions per minute (RPM) multi-label model to evaluate the ability to identify intrusion
gauge). Each dataset included both the normal and attack messages in the CAN bus segments. We used the results of
records. We used the Car Hacking dataset to evaluate the the confusion matrices for the calculations, as follows:
performance of the two input data types from the CAN bus. TP
Precision(Prec) = (9)
TP + FP
2) IVN INTRUSION DETECTION DATASET [36] TP
Table 3 lists the statistics for the In-Vehicle Intrusion Detec- Recall(Rec) = (10)
TP +
 FN
tion dataset. The Attack dataset was extracted from two car Prec × Rec

models, Hyundai Sonata and Kia Soul. Although this dataset F − measure(F1) = 2 × (11)
Prec + Rec
is smaller than the Car Hacking dataset, it is more balanced.
FP + FN
We considered the ability to detect replay attack scenarios on Errorrate(ER) = (12)
an In-Vehicle Network Intrusion Detection dataset. TP + TN + FP + FN

3) SURVIVAL ANALYSIS DATASET [18] D. HYPERPARAMETERS

The statistics in Table 4 show the composition of the Survival Hyperparameters also play an essential role in improving the
Analysis dataset, which comprises data from the three car model performance. Throughout the experiment, we chose
models. We used a pre-trained model on the Car Hacking suitable values with a tradeoff via hyperparameter tuning.
dataset. Then we extended our transfer learning experience For instance, the model achieves higher accuracy when the
to the Kia Soul and Chevrolet Spark car models, with the embedding dimension is increased. However, the message
exception of Hyundai Sonata. embedding size affects the inference time. Based on empirical
evaluations conducted on the Car Hacking Dataset with a
B. EXPERIMENTAL SETUP fixed window size (Fig. 11), an embedding dimension of 32 is
The proposed transformer-based attention network IDS for found to be optimal for the model, offering both high accu-
the IVN is implemented in Python using the TensorFlow racy and small inference time. The optimal hyperparameter

VOLUME 11, 2023 55397

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 6. Comparison with other models on single-message data. TABLE 8. Comparison on sequential data with a window size of 32.

TABLE 7. Comparison on sequential data with a window size of 16.

TABLE 9. Comparison on sequential data with a window size of 64.

settings are listed in detail in Table 5. The sliding window

size also affects model performance for a sequential CAN ID The SOTA CNN-LSTM model exhibited impressive perfor-
input. Therefore, we analyzed the performance for different mance, with an error rate of 0% and the highest F1 scores for
window sizes. DoS and spoofing attacks, accurately classifying abnormal
messages. However, our model yielded better results than
E. RESULTS ON CAR HACKING DATASET CNN-LSTM for fuzzy attacks, as the latter only achieved an
1) SINGLE CAN MESSAGE F1 score of 0.99992. Furthermore, our model achieved the
Table 6 presents performance comparison results between highest F1 scores for all attack types.
our model and existing in-vehicle intrusion detection systems Although the use of a single-message input achieved good
using single messages, representing RNN variants. The input results, the injection of extracted messages was not detected
data were independent CAN signals with a length of 10. by the model trained on single-message data. This may
Among the baselines, CNN-LSTM is a SOTA deep learning prove dangerous for the driver if unexpected actions such
model in IDS for CAN [21], which performs multi-class as turning on the light, acceleration, or sudden braking are
classification on DoS, fuzzy, and spoofing attacks. Although continuously sent to the CAN bus without being detected.
all tested models yielded good results in detecting DoS and We used sequential data in our experiment to evaluate the
spoofing attacks, the complexity of fuzzy attack scenarios model performance. The transformer-based supervised learn-
makes them challenging for traditional models to detect. ing model can learn sequential patterns in the CAN bus and

55398 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 10. Comparison on sequential data with a window size of 128. model with a more comprehensive view of the abnormality
in the CAN bus. Our experiment started with data from Car
Hacking dataset.

2) SEQUENTIAL CAN ID
As mentioned above, the model performance is significantly
affected by hyperparameters. Therefore, we evaluate the per-
formance for window sizes ranging from 16 to 256. Each
sequential input was labeled as normal or abnormal, with the
output being a multi-class classification value. The super-
vised learning process continuously updates the weights in
the model to recognize the sequence of CAN IDs in the time
domain. The results presented in Tables 7 and 11 indicate that
the transformer model easily outperforms the baselines, par-
ticularly in fuzzy attack scenarios. Furthermore, an increase
in the window size from 16 to 128 helped all models extract
more features from sequential CAN IDs, thereby improv-
ing the classification results. Our model classified abnor-
mal intrusions almost perfectly in the DoS dataset for every
TABLE 11. Comparison on sequential data with a window size of 256. window size. The reason is that the attacker is forced to
use a high-priority message – CAN ID 0x000 – for this
attack, making it easy for the system to detect intrusions.
The results for the spoofing attack dataset showed ERs of
less than 1 when the window size exceeded 32. In particular,
in the fuzzy dataset, the F1 score of CNN-LSTM increased
from 0.71820 to 0.99496, whereas our model improved from
0.99985 to 0.99993, with a gradual increase in the input
data length. Thus, the proposed model yields better extracted
information from longer sequential CAN IDs, thereby achiev-
ing better results for larger window sizes. Our results indicate
that most F1 scores decreased as the CAN ID chain length
increased from 128 to 256, as the baselines experienced bot-
tlenecks when the window size increased significantly. Our
model also achieved the lowest ER values for most attack
types compared to the other models. As shown in Table 10,
the error rates of the baseline models were greater than 5%,
and the error rates of the Transformer model for the DoS,
Fuzzy, Gear and RPM Spoofing datasets were 0.0%, 0.001%,
0.112%, and 0.009%, respectively. We also presented ROC
curves with AUC values of the proposed model and baseline
models in Fig. 12 in order to compare the overall performance
between models with a window size of 128. The results indi-
cate that our model improves safety by detecting the majority
of intrusions and minimizing false alarms.

F. RESULTS ON THE INTRUSION DETECTION DATASET

Replay attacks are a type of attack that inject extracted CAN
messages, which confuse a model trained on single-message
input. To examine the performance of our model with replay
attacks, we used an In-Vehicle Intrusion Detection dataset.
Our data analysis revealed CAN messages with the same
FIGURE 12. ROC curves with a window size of 128. identifiers as safe messages used for attack purposes. In other
words, the messages exhibited the same CAN IDs with dif-
ferent labels. We found that the number of messages labeled
use them for classification purposes. When an attacker injects as normal was more extensive than that of injected malicious
the extracted intrusions, sequential CAN IDs provide the messages with the same identifier. Therefore, supervised

VOLUME 11, 2023 55399

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 12. Comparison the proposed model and CNN-LSTM on replay attack dataset.

FIGURE 13. Confusion matrices of CNN-LSTM, replay attack dataset, KIA FIGURE 16. Confusion matrices of transformer, replay attack dataset,
soul, using sequential CAN ID (a) and single message (b). Hyundai Sonata, using sequential CAN ID (a) and single message (b).

decreased to 0.288% when sequential data were used instead

of single messages. Furthermore, applying the sequential
data type reduced the error rate of the transformer-based
attention network from 15.87% to 0.058%. This script was
similar to other experiments on the KIA Soul car model. For
CNN-LSTM, ER decreased from 14.18% to 2.006%. By con-
trast, the ER of the proposed model decreased from 12.75% to
0.314%. With the sequential CAN ID input type, CNN-LSTM
obtained satisfactory results, with ERs of 0.288% for Hyundai
FIGURE 14. Confusion matrices of transformer, replay attack dataset, KIA Sonata and 2.006% for Kia Soul. In contrast, our model
soul, using sequential CAN ID (a) and single message (b). achieved ERs of 0.058% and 0.314% for Hyundai Sonata and
Kia Soul, respectively.
To optimize each model’s performance, we selected win-
dow sizes of 16 and 128 for the transformer-based attention
network and CNN-LSTM, respectively. The two different
sliding windows ensured that the CNN-LSTM model yielded
optimal performance, and the inference time of the pro-
posed model was suitable. Data were allocated to the training
and testing sets at a 9:1 ratio. Subsequently, we found that
the proposed multi-class model with sequential CAN IDs
detects replay attack intrusions more effectively than the
FIGURE 15. Confusion matrices of CNN-LSTM, replay attack dataset, CNN-LSTM model.
Hyundai Sonata, using sequential CAN ID (a) and single message (b).

G. INTRUSION MESSAGE IDENTIFICATION

learning methods tend to be biased and confuse malicious and Our results indicate that the use of sequential CAN IDs
safety messages. provides a more comprehensive vehicle protection. Intrusion
We compared CNN-LSTM and the proposed model on detection systems can stop a vehicle and notify a driver when
two car models, with the results listed in Table 12 and abnormal segments are detected. These malicious segments
the confusion matrices presented in Figs. 13–16. First, the include normal and attack messages, so they must not be
results indicate that sequential input data performs better than entirely discarded. Training with data labeled at the segment
single-message data in replay attack scenarios. In the case level can provide warnings when intrusions are detected,
of Hyundai Sonata, the CNN-LSTM error rate of 17.52% this approach faces challenges when identifying anomalous

55400 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 13. F1 scores of intrusion identification with 16 CAN IDs on Car TABLE 14. Comparison of transfer learning on KIA soul.
Hacking dataset.

and transfer learning (TL) models. Owing to data limitations,

the training and testing datasets were divided in a 2:1 ratio to
ensure that sufficient data were available for training. First,
we partially frozen the source model. Owing to the change
in the number of classes, it was necessary to replace the
messages. In the following experiment, we trained on the classifier in the last layer and then perform one iteration of
labeled dataset at the segment level and then used attention training with the target dataset. Next, we unfroze the model
scores to locate malicious messages in segments. The F1 and conducted fine tuning. The results show that the target
scores obtained from the Car Hacking dataset are listed in model was successfully obtained from the source model,
Table 13. thereby improving its performance. For example, in a fuzzy
As previously mentioned, the threshold is a factor that attack, the ER of the transformer decreases from 1.09% to
affects the results. We considered all thresholds from 0.3 to 0.85%. These results demonstrate the effectiveness of the
0.6 with step 0.05, where Avg represents the average value transfer learning method in reducing the error rate in both the
of the results, and the optimal threshold yielded the best CNN-LSTM and transformer networks.
value. The DoS and fuzzy attack datasets exhibited effective Conversely, if we compare the transformer and CNN-LSTM
performance at a threshold of 0.5, whereas the RPM and Gear models, the proposed model sometimes exhibits a poorer
attack datasets demonstrated expected results at 0.3 and 0.4, performance. A limitation of our transformer model is that
respectively. Our results demonstrate that the attention score the algorithm is effective only for large datasets. However,
can perfectly identify attack messages in the DoS dataset. The the transformer TL can obtain higher F1 scores and lower
identification of malicious messages in a fuzzy attack dataset ERs than the CNN-LSTM TL. For instance, experiments
is not challenging for transformer-based attention networks. on malfunction attacks revealed an F1 score of 0.9936 for
The F1 scores on the fuzzy attack dataset mostly reached CNN-LSTM, which was higher than the transformer SA
0.99, with slight performance drops on the RPM and gear of 0.9917.
attack datasets. However, the transformer TL achieved better overall
However, the results at the optimal threshold show that results than the CNN-LSTM TL, with F1 scores of 0.9947 and
the model successfully identifies the messages injected into 0.9923, respectively. Furthermore, the TL transformer model
a CAN segment, with F1 scores on the RPM and gear exhibited an excellent intrusion detection performance for
attack datasets reaching values of 0.92 and 0.95, respectively. DoS attacks. From the results, we conclude that the proposed
Notably, labels for individual messages were not used during model can perform well when applying transfer learning with
the training phase. Our experiment proves that the proposed a suitable pretrained model.
model can pay attention to malicious components appearing
in segments of the CAN bus. By highlighting these intrusive I. MODEL COMPLEXITY ANALYSIS
CAN IDs, an in-vehicle attack protection system can elimi- This section focuses on analyzing the time costs associated
nate them from the CAN bus and return the vehicle to working with the proposed model. Specifically, we conducted a com-
mode without interrupting operations. prehensive analysis of the model’s performance on a CPU
to ensure its compatibility with vehicle systems with limited
H. TRANSFER LEARNING hardware capabilities. Using FLOPs analysis, we estimated
The experimental results are presented in Tables 14 and 15, the computational complexity of our proposed model and
respectively. We compared the results of the standalone (SA) compared its training and inference times with those of

VOLUME 11, 2023 55401

 T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

TABLE 15. Comparison of transfer learning on Chevrolet Spark. IDs, extracts information more selectively, and outperforms
other excellent methods in our evaluations. Various experi-
ments were conducted to demonstrate the advantages of the
proposed model. With a single-message input, our model
achieved perfect results on the car-hacking dataset, with an F1
score of 1.00, and error rates of approximately 0% for all four
attack scenarios. In particular, our model yielded a superior
performance when evaluated using sequential data. Although
the transformer requires a longer inference time with the
same input length, the result of our model with 16 sequential
CAN IDs still outperforms that of others. We found that
the proposed model can provide better results on the replay
attack dataset than CNN-LSTM, which yields SOTA intru-
sion detection performance for CAN buses. We employed
the self-attention mechanism to identify intrusive messages in
CAN segments without requiring message level labeled data
TABLE 16. Comparison of model complexity.
in the training phase. The results of our experiment support
the safe maintenance of vehicle operations following attacks.
Furthermore, we presented an application of transfer learning
to our model. The results show that the transformer can share
knowledge from the source model with the target model to
improve intrusion detection. A limitation of our study lies
in the fact that we experimented with an extracted dataset.
Classification becomes more difficult when the model is
baseline models. Previous experiments in [4] have shown applied to vehicles operating in a real-time environment.
that reducing the batch size can optimize the inference time, The direction of our future development will be unknown
thereby minimizing the overall detection latency of IDS. attack detection in real-life attack scenarios. Finally, using the
Therefore, we evaluated the performance of the model using sliding window results in the appearance of many malicious
a batch size of 1. That is, we evaluate the processing time messages in single data segments. This complication can be
required by the model to handle given input data (s/batch). resolved using multi-label multi-class classification methods
The results of the time costs analysis are listed in Table 16. in the future.
In accordance with the J1939 standards [37], the Society of
Automotive Engineers (SAE) recommends a signaling rate REFERENCES
of 250kb/s for the CAN bus, despite its maximum capacity [1] G. Leen and D. Heffernan, ‘‘Expanding automotive electronic systems,’’
of 1 Mb/s [37]. At this recommended rate, the CAN bus can Computer, vol. 35, no. 1, pp. 88–93, 2002.
[2] N. A. Stanton, M. Young, and B. McCaulder, ‘‘Drive-by-wire: The case of
transmit up to 1908 CAN frames per second. Regarding the driver workload and reclaiming control with adaptive cruise control,’’ Saf.
CPU, the proposed model was capable of processing more Sci., vol. 27, nos. 2–3, pp. 149–159, Nov. 1997.
than 10,000 messages and making approximately 85 predic- [3] B. Galloway and G. P. Hancke, ‘‘Introduction to industrial control net-
works,’’ IEEE Commun. Surveys Tuts., vol. 15, no. 2, pp. 860–880,
tions per second, given an inference time of 11.6 ms/batch. 2nd Quart., 2013.
Compared to the ability to process 4,300 messages per second [4] H. M. Song, J. Woo, and H. K. Kim, ‘‘In-vehicle network intrusion detec-
by performing 150 inferences on DCNN [4], despite having a tion using deep convolutional neural network,’’ Veh. Commun., vol. 21,
Jan. 2020, Art. no. 100198.
longer inference time with a larger window size, our proposed
[5] W. Lo, H. Alqahtani, K. Thakur, A. Almadhor, S. Chander, and G. Kumar,
model is capable of processing more messages per second. ‘‘A hybrid deep learning based intrusion detection system using spatial–
A comparison shows that the proposed model is suitable for temporal representation of in-vehicle network traffic,’’ Veh. Commun.,
vol. 35, Jun. 2022, Art. no. 100471.
real-time detection.
[6] Bosch, Robert Bosch GmbH, Gerlingen, Germany, 1991.
[7] M. Farsi, M. Barbosa, and K. Ratcliff, ‘‘An overview of controller area
VI. CONCLUSION AND FUTURE WORK network,’’ Comput. Control Eng. J., vol. 10, no. 3, pp. 113–120, Jun. 1999.
[8] R. De Andrade, K. N. Hodel, J. F. Justo, A. M. Laganá, M. M. Santos, and
In this paper, we proposed a transformer-based attention Z. Gu, ‘‘Analytical and experimental performance evaluations of CAN-FD
network as an intrusion detection system for CAN buses bus,’’ IEEE Access, vol. 6, pp. 21287–21295, 2018.
in an in-vehicle environment. This study aimed to present [9] H. J. Jo and W. Choi, ‘‘A survey of attacks on controller area networks and
a novel approach for better intrusion detection that can corresponding countermeasures,’’ IEEE Trans. Intell. Transp. Syst., vol. 23,
no. 7, pp. 6123–6141, Jul. 2022.
take over the role of RNNs. The core of the proposed [10] H. Sun, M. Chen, J. Weng, Z. Liu, and G. Geng, ‘‘Anomaly detection for
approach is a self-attention mechanism. Taking advantage in-vehicle network using CNN-LSTM with attention mechanism,’’ IEEE
of the correlations between the components in CAN data, Trans. Veh. Technol., vol. 70, no. 10, pp. 10880–10893, Oct. 2021.
[11] R. Buttigieg, M. Farrugia, and C. Meli, ‘‘Security issues in controller area
the transformer-based model offers powerful feature extrac- networks in automobiles,’’ in Proc. 18th Int. Conf. Sci. Techn. Autom.
tion capabilities, performs well on longer sequential CAN Control Comput. Eng. (STA), Dec. 2017, pp. 93–98.

55402 VOLUME 11, 2023

T. P. Nguyen et al.: Transformer-Based Attention Network for In-Vehicle Intrusion Detection

[12] H. Lee, S. H. Jeong, and H. K. Kim, ‘‘OTIDS: A novel intrusion detection [35] T.-N. Hoang and D. Kim, ‘‘Supervised contrastive ResNet and trans-
system for in-vehicle network by using remote frame,’’ in Proc. 15th Annu. fer learning for the in-vehicle intrusion detection system,’’ 2022,
Conf. Privacy, Secur. Trust (PST), Aug. 2017, p. 5709. arXiv:2207.10814.
[13] H. M. Song, H. R. Kim, and H. K. Kim, ‘‘Intrusion detection system based [36] The In-Vehicle Network Intrusion Detection Dataset of Information Secu-
on the analysis of time intervals of CAN messages for in-vehicle network,’’ rity R&D Data Challenge, Hacking Countermeasure Res. Lab, Korea
in Proc. Int. Conf. Inf. Netw. (ICOIN), Jan. 2016, pp. 63–68. Univ., Seoul, South Korea, 2019.
[14] W. Wu, R. Li, G. Xie, J. An, Y. Bai, J. Zhou, and K. Li, ‘‘A survey of [37] J1939 Recommended Practice for a Serial Control and Communications
intrusion detection for in-vehicle networks,’’ IEEE Trans. Intell. Transp. Vehicle Network, SAE Int., Warrendale, PA, USA, 2013.
Syst., vol. 21, no. 3, pp. 919–933, Mar. 2020.
[15] A. Wasicek, M. D. Pese, A. Weimerskirch, Y. Burakova, and K. Singh,
‘‘Context-aware intrusion detection in automotive control systems,’’ in
Proc. 5th ESCAR USA Conf., 2017, pp. 21–22.
[16] H. Qin, M. Yan, and H. Ji, ‘‘Application of controller area network (CAN)
bus anomaly detection based on time series prediction,’’ Veh. Commun., TRIEU PHONG NGUYEN received the B.S.
vol. 27, Jan. 2021, Art. no. 100291. degree in control engineering and automation
[17] M. D. Hossain, H. Inoue, H. Ochiai, D. Fall, and Y. Kadobayashi, ‘‘LSTM- system from the Ho Chi Minh City University
based intrusion detection system for in-vehicle can bus communications,’’ of Transport (UT-HCMC), Vietnam, in 2021.
IEEE Access, vol. 8, pp. 185489–185502, 2020. He is currently pursuing the M.S. degree with
[18] M. L. Han, B. I. Kwak, and H. K. Kim, ‘‘Anomaly intrusion detection the Department of Mobility Convergence Security
method for vehicular networks based on survival analysis,’’ Veh. Commun., in Engineering, Soonchunhyang University, Asan,
vol. 14, pp. 52–63, Oct. 2018.
South Korea. From January 2020 to January 2021,
[19] I. A. Khan, N. Moustafa, D. Pi, W. Haider, B. Li, and A. Jolfaei, he was an Assistant Researcher with the Artificial
‘‘An enhanced multi-stage deep learning framework for detecting mali-
Intelligence Laboratory, Graduate Institute,
cious activities from autonomous vehicles,’’ IEEE Trans. Intell. Transp.
UT-HCMC. From February 2021 to January 2022, he was an Artificial
Syst., vol. 23, no. 12, pp. 25469–25478, Dec. 2022.
Intelligence Engineer with Heligate Japan Software Company, Vietnam. His
[20] T.-N. Hoang and D. Kim, ‘‘Detecting in-vehicle intrusion via semi-
research interest includes deep learning, the Internet of Things, and security
supervised learning-based convolutional adversarial autoencoders,’’ Veh.
Commun., vol. 38, Dec. 2022, Art. no. 100520. networks.
[21] A. K. Desta, S. Ohira, I. Arai, and K. Fujikawa, ‘‘Rec-CNN: In-vehicle
networks intrusion detection using convolutional neural networks trained
on recurrence plots,’’ Veh. Commun., vol. 35, Jun. 2022, Art. no. 100470.
[22] L. Kang and H. Shen, ‘‘A transfer learning based abnormal CAN bus
message detection system,’’ in Proc. IEEE 18th Int. Conf. Mobile Ad Hoc
Smart Syst. (MASS), Oct. 2021, pp. 545–553. HEUNGWOO NAM (Member, IEEE) received
[23] S. Tariq, S. Lee, and S. S. Woo, ‘‘CANTransfer: Transfer learning based the B.S. degree in computer science and engi-
intrusion detection on a controller area network using convolutional LSTM neering from Kyung Hee University, South Korea,
network,’’ in Proc. 35th Annu. ACM Symp. Appl. Comput., Mar. 2020, in 2002, and the M.S. degree in computer science
pp. 1048–1055. and engineering and the Ph.D. degree in electron-
[24] P. Wei, B. Wang, X. Dai, L. Li, and F. He, ‘‘A novel intrusion detection ics and computer engineering from Korea Univer-
model for the CAN bus packet of in-vehicle network based on atten- sity, South Korea, in 2005 and 2010, respectively.
tion mechanism and autoencoder,’’ Digit. Commun. Netw., vol. 9, no. 1, From September 2010 to May 2015, he was a
pp. 14–21, Feb. 2023. Research Professor with Korea University. From
[25] S. Hochreiter and J. Schmidhuber, ‘‘Long short-term memory,’’ Neural May 2015 to December 2017, he was a Visiting
Comput., vol. 9, no. 8, pp. 1735–1780, 1997. Researcher with the Scripps Institution of Oceanography (SIO), University
[26] M. Nam, S. Park, and D. S. Kim, ‘‘Intrusion detection method using bi- of California, San Diego (UCSD). From December 2017 to August 2020,
directional GPT for in-vehicle controller area networks,’’ IEEE Access, he was a Research Professor with the Smart Submerged Floating Tunnel
vol. 9, pp. 124931–124944, 2021. System Research Center, KAIST. Since September 2020, he has been an
[27] Z. Wu, H. Zhang, P. Wang, and Z. Sun, ‘‘RTIDS: A robust transformer- Assistant Professor with the Department of Computer Engineering, Daegu
based approach for intrusion detection system,’’ IEEE Access, vol. 10, University. His research interests include communication networks and pro-
pp. 64375–64387, 2022. tocols, underwater acoustic communication systems, autonomous underwa-
[28] O. Sharma, N. C. Sahoo, and N. B. Puhan, ‘‘Kernelized convolutional ter vehicles, wireless and underwater sensor networks, and structural health
transformer network based driver behavior estimation for conflict res- monitoring.
olution at unsignalized roundabout,’’ ISA Trans., vol. 133, pp. 13–28,
Feb. 2023.
[29] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
and I. Polosukhin, ‘‘Attention is all you need,’’ in Proc. Adv. Neural Inf.
Process. Syst., vol. 30, 2017, pp. 1–11.
[30] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ‘‘BERT: Pre-training
DAEHEE KIM (Member, IEEE) received the
of deep bidirectional transformers for language understanding,’’ 2018,
arXiv:1810.04805. B.S. degree in electrical and electronic engineer-
[31] B. Han, Z. Chen, and Y. Qian, ‘‘Local information modeling with self- ing from Yonsei University, Seoul, South Korea,
attention for speaker verification,’’ in Proc. IEEE Int. Conf. Acoust., Speech in 2003, and the M.S. and Ph.D. degrees in electri-
Signal Process. (ICASSP), May 2022, pp. 6727–6731. cal and electronic engineering from Korea Univer-
[32] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, sity, Seoul, in 2006 and 2016, respectively.
T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszko- From 2006 to 2016, he was a Senior Engineer
reit, and N. Houlsby, ‘‘An image is worth 16×16 words: Transformers for with Samsung Electronics, Suwon, South Korea,
image recognition at scale,’’ 2020, arXiv:2010.11929. where he conducted research on WiMAX and LTE
[33] S. J. Pan and Q. Yang, ‘‘A survey on transfer learning,’’ IEEE Trans. Knowl. systems. He is currently an Assistant Professor
Data Eng., vol. 22, no. 10, pp. 1345–1359, Oct. 2010. with the Department of Internet of Things, Soonchunhyang University, Asan,
[34] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, South Korea. His research interest includes the Internet of Things, energy
‘‘A comprehensive survey on transfer learning,’’ Proc. IEEE, vol. 109, management, blockchain, 5G/6G, and security for wireless networks.
no. 1, pp. 43–76, Jan. 2021.

VOLUME 11, 2023 55403