Available online at [Link].

com

ScienceDirect
ICT Express 9 (2023) 1077–1084
[Link]/locate/icte

Energy-harvesting Q-learning secure routing algorithm with

authenticated-encryption for WSN
Cuiran Li ∗, Jixuan Wu, Zepeng Zhang, Anqi Lv
School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou, China
Received 26 September 2022; received in revised form 4 March 2023; accepted 18 May 2023
Available online 26 May 2023

Abstract
Wireless sensor networks are susceptible to a variety of network attacks. Due to the limited energy of nodes and selfish nodes in the network,
the packet delivery rate is lower. To address these issues, we innovatively propose an energy-harvesting Q-learning secure routing algorithm
with authenticated-encryption. The algorithm uses physical unclonable functions and optimized Q-learning to ensure that the transmission
path is reliable. Meanwhile, we combine the LSTM-based prediction model to predict the energy value that the nodes replenish. In addition,
simulations are performed to compare the performances of the proposed algorithm with other algorithms under different attacks. The proposed
algorithm has greater improvements in the packet delivery rate, filtering selfish nodes, and reducing node energy consumption.
© 2023 The Author(s). Published by Elsevier B.V. on behalf of The Korean Institute of Communications and Information Sciences. This is an open
access article under the CC BY license ([Link]
Keywords: Energy harvesting; Power prediction; Q-learning; Secure routing; Wireless sensor network

1. Introduction topology make multi-hop routing vulnerable to various types

of attacks. A challenging problem for WSN is how to consider
The technical characteristics of 5G network with high speed power consumption based on improving the security of WSN
and low latency enable the rapid development of emerging in order to transmit data efficiently.
application technologies based on the Internet of Things (IoT). A node rank-based evaluation algorithm has been proposed
Information and communication technology plays an increas- in the literature [5] to identify compromised nodes based on
ingly important role with the development of the IoT [1]. the difference in rank between parent and child nodes, but it
Wireless sensor network (WSN) provides a new way to collect, is only applicable to small WSN. An algorithm for detecting
process, and communicate data between different types of and responding to intrusions has been proposed in the lit-
devices. It is a network consisting of a large number of sensor erature [6], which sets up observation nodes in the network
nodes, all of which are randomly distributed throughout the and identifies compromised nodes on the basis of the number
monitoring area and feed the detected relevant information of packets lost by the nodes. This algorithm, however, has a
to the user. WSN can be used extensively in the military, single evaluation criterion and is unable to identify multiple
environmental monitoring, disaster relief, medical health, and attacks. Literature [7] proposes a selfish node identification
other areas. In the future, it is foreseeable that wireless sensor algorithm based on node energy that improves the packet
networks will greatly affect every aspect of people’s lives [2,3]. delivery rate of the network by computing the trust value
Sensor nodes are typically deployed in harsh environments of nodes through the packet generation rate and the survival
that are difficult for humans to access and thus cannot be period. The literature [8] proposes a distributed trust model for
replenished by replacing batteries [4]. At the same time, wireless sensor networks, which can effectively improve the
the openness of wireless channels and the dynamic network efficiency of compromised node identification by building trust
values embedded in nodes through both direct and indirect
∗ Correspondence to: Lanzhou Jiaotong University, No. 88 Anning West
trust. However, the weights between its direct trust value and
Road, Anning District, 730070 Lanzhou City, Gansu Province, China. indirect trust value are fixed values, which cannot track the
E-mail addresses: licr@[Link] (C. Li), JSywan_W@[Link]
(J. Wu), 1115092414@[Link] (Z. Zhang), 2810282717@[Link] (A. Lv).
time-varying characteristics of the network in real time. A
Peer review under responsibility of The Korean Institute of Communica- trust-aware adaptive routing algorithm based on trust has been
tions and Information Sciences (KICS). proposed in the literature [9], but its node trust value does
[Link]
2405-9595/© 2023 The Author(s). Published by Elsevier B.V. on behalf of The Korean Institute of Communications and Information Sciences. This is an
open access article under the CC BY license ([Link]
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

Table 1
The behavioral characteristics of typical attacks.
Network attack Drop data Modify data Disrupting
packets packets routing
Sink hole attack ✓ ✓ ✓
Black hole attack ✓ × ✓
Gray-hole attack ✓ × ×
Selective forwarding attack ✓ × ×
Tampering attack × ✓ ×
Witch attack × × ✓

not take energy into account. In the literature [10], a secure

routing algorithm based on node trust and energy consumption
was proposed to improve network security performance and
efficiently balance the energy consumption of nodes, but with
a high overhead. Fig. 1. The model of EH-WSN.
To address the above problem, we propose an energy-
harvesting Q-learning secure routing algorithm with
(3) The broadcast range of the sink node can cover all the
authenticated-encryption (EH-QSREA) for WSN. The contri-
sensor nodes, and each node can adapt the performance to the
butions of this paper include the following:
transmitted range.
(1) To solve the problem of selecting relay nodes, we model
(4) Each sensor node begins with the same initial energy
them as a Markov decision process (MDP) and solve the
and has the same battery capacity, computing power, and
optimal value function for energy harvesting wireless sensor
communication capability.
networks (EH-WSN). This paper proposes an improved Q-
learning based algorithm to realize the shortest path of the
2.2. Types of network attacks
network and dynamically evaluate the comprehensive trust
value of nodes. The proposed algorithm effectively improves
Network attacks can be divided into routing protocol at-
the packet delivery rate (PDR). tack and trust model attack according to the different targets.
(2) In this paper, we analyze the characteristics of typ- During routing protocol attacks, selfish nodes disrupt trans-
ical network attacks on WSN. An authenticated-encryption missions by packet spoofing or routing forging [13]. Table 1
mechanism based on physical unclonable functions (PUF) is shows the behavioral characteristics of typical attacks.
designed for unauthorized users and selfish nodes in the public Among them, the behavioral characteristics of sinkhole at-
network environment, which improves the network’s efficiency tack and blackhole attack include modifying packets, dropping
in identifying selfish nodes. packets, and disrupting routing. Thus, we chose these two
(3) The proposed algorithm considers the expected replen- attacks to enhance the applicability of the proposed algorithm.
ishable energy of nodes and the remaining energy of nodes to
select relay nodes. Therefore, network energy consumption is 2.3. Solar power prediction model
balanced.
As shown in Fig. 2, we measured the solar irradiance over
2. Models and secure routing algorithm a 24-hour period. It can be seen that the solar irradiance
has a strong non-smoothness. If the prediction model is con-
2.1. EH-WSN model structed directly, the sudden changes in solar irradiance (as
shown in the box in Fig. 2) can seriously affect the prediction
The EH-WSN model is shown in Fig. 1. The model consists accuracy.
of sensor nodes equipped with energy harvesting equipment For this reason, we remove the influence of sudden changes
(solar panels), and grid-powered sink nodes. They form a on prediction accuracy through empirical ensemble mode de-
network by self-organization. Monitoring data is transmitted composition (EEMD) and build the prediction model by com-
hop by hop along the sensor nodes to sink node. During bining networks from long-short-term memory (LSTM) [14].
the transmission process, monitoring data can be processed The basic idea of the EEMD algorithm is as follows: The
by several nodes. For the model, we make the following added white noise would uniformly populate the entire time–
assumptions [11,12]: frequency space with the constituent components of different
(1) The deployment area is a regular graph. All sensor scales. Upon addition of a signal to this uniformly distributed
nodes are deployed randomly and remain stationary after white-noise background, each component in different scales
deployment. of the signal is automatically projected onto clean reference
(2) Each sensor node and sink node can locate and know scales set by white noise in the background noise. Since
their relative positions to other nodes. each of the noise-added decompositions consists of the signal
1078
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

irradiance sequence xi (t).

x i (t) = x(t) + ωi (t) (1)
(2) EMD is based on the local characteristic time scales
of a signal and could decompose the complicated signal into
a set of complete and almost orthogonal components named
intrinsic mode function (IMF). The xi (t) is processed by the
EMD method to obtain the IMF components of each order.
m
∑
x i (t) = I M Fi j (t) + Ri (t) (2)
j=1

where m denotes the number of IMF components obtained by

each decomposition, IMFi j (t) and Ri (t) denote the jth IMF
component and residual component of the decomposition after
the ith addition of white noise, respectively.
Fig. 2. 24 h of solar irradiance. (3) Since the mean value of the white noise Gaussian
spectrum is zero, the mean for the above components is solved.

N
1 ∑
I M F j (t) = I M Fi j (t) (3)
N i=1
N
1 ∑
R(t) = Ri (t) (4)
N i−1
where IMF j (t) denotes the jth IMF component after the
integrated average, and R(t) denotes the residual component.
(4) After normalizing the IMF component, input it to LSTM
to obtain the IMF component predicted value {H1 , H2 , . . . ,
H N } and the residual component predicted value H R . For {H1 ,
H2 , . . . , H N } are inverse normalized and superimposed to
obtain the final solar irradiance prediction H.

2.4. PUF-based authenticated-encryption mechanism

Physical unclonable function (PUF) uses the random phys-

ical characteristics of the device to establish special mapping
Fig. 3. PUF-based authenticated-encryption mechanism. relationships [16]. The PUF-based authentication-encryption
mechanism prevents unauthorized users from accessing net-
work data and prevents malicious nodes from entering the
plus the added white noise, it is certainly possible that each
network. As Fig. 3 shows, it consists of the following 6 steps:
individual trial produces very noisy results. But the noise
(1) The sink node sends the challenge Ci to the sensor node
in each trial is different in separate trials. Thus, it can be
i, and the sensor node i receives and generates the response
decreased or even completely canceled out in the ensemble
Ri ;
with enough trails. We treat the ensemble mean as the true
response because ultimately the only part that is persistent is Ri = f PU F (Ci ) (5)
the signal as more and more trials are added to the ensemble.
Since the sensor nodes collect solar energy to recharge (2) The sensor node i returns the Ri to the sink node. The
energy, solar irradiance is determined by weather and other sink node receives and forms a challenge response pair C-R as
factors, its value is random in time and space, and the ef- the IDi of sensor node i;
ficiency of converting solar energy into electricity is low— I Di = Ci ⊕ Ri (6)
generally not more than 30%. And the sink nodes are powered
by the grid and have a stable energy source. Both the training (3) The sink node stores each sensor node’s ID and creates
and prediction processes in this work are performed on the the database.
sink node, and sensor nodes periodically obtain prediction (4) When an unknown node θk sends a connection request
results from the sink node [15]. to the network through identity tampering, impersonation, etc.,
(1) Gaussian white noise ωi (t) is added to the original the sink node sends a challenge Ci to the unknown node θk .
solar irradiance sequence x(t) to obtain the reconstructed solar The unknown node θk receives and generates a response Rk ;
1079
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

(5) The sink node receives Rk and obtains Ck from 2.6. The Q-learning secure routing algorithm based on
Eq. (7) authenticated-encryption
Ck = I Di ⊕ Rk (7) In Q-learning, the ε-greedy strategy performs the random
(6) If Ck = Ci , it determines the unknown node as exploration with equal probability ε, it is not easy to make
a legitimate node; otherwise, its connection request will be optimal decisions about actions [17]. The improved Q-learning
rejected. strategy uses a decreasing ε-greedy strategy, exploring all ac-
tions with a decreasing probability εT , so that the proportion of
exploration actions decreases with state space, making finding
2.5. The model for selecting relay nodes the optimal solution easier. The probability εT is:

Since the selection of relay nodes is only related to the εT = [(T − Ti )/T ]ε0 (15)
current state, such as behavior and energy, it is independent where Ti denotes the current training round, T denotes the
of previously experienced states, i.e., it possesses a Markov total training round, ε0 denotes the initial probability. Set the
property. Therefore, we model it as MDP, as shown in Eq. (8). minimum probability εmin = 0.05 to prevent the agent from
falling into a non-exploration state.
The proposed algorithm is based on the comprehensive
G t = gt + γ (gt+1 + γ (gt+2 + · · · )) = gt + γ G t+1 (8)
evaluation of node behavior and location. In the trust detection
where Gt denotes the total reward of the model after time t, process, watchdog mechanism is used to detect malicious
gt denotes the reward for node action, γ denotes the discount behaviors in routing [13]. Assuming that the coordinates of
factor, and γ ∈[0, 1]. The agent in the environment state y, the sink node are (sx , s y ) and the coordinates of the sensor
according to the strategy π :Y→X selects the action x from node i are (nix , niy ), the distance between them is:
the action space X, denoted as x = π (y). Then the total value √
ρsni = (n ix − sx )2 + (n iy − s y )2 (16)
function obtained after executing the strategy π in state y is
The distance between sensor node i and the next hop node
V π (y) = G(y) + γ PYπ (Y ) (y∗)V π (y∗)
∑
(9)
j can be expressed as:
y∗∈Y √
j j
where Vπ (y) denotes the state value in the current state, G(y) ρni n j = (n ix − n x )2 + (n iy − n y )2 (17)
denotes the immediate reward feedback that the agent can get
Then the behavior trust value of the sensor node i is shown
after performing action x in the environment state y, and the in Eq. (18):
combined formula denotes the sum of future rewards that the
agent can get after performing action x in the environment state n i → Q t = (1 − α) · n i → Q t−1 + α · R (18)
y. Denote the state value function in each state y as V π (y) , then where α denotes the learning rate, Qt denotes sensor node
the optimal state value function is defined as i’s the latest behavioral trust value, and Qt−1 denotes sensor
node i’s the behavioral trust value in the last training round. R
∑
V ∗ (y) = G(y) + max γ Px y (y∗)V ∗ (y∗) (10)
x∈X
y∗∈Y
denotes the reward factor, as shown in Eq. (19):
Rmax , p = 0 and ρsni = ρni n j
{
From Eq. (10), the optimal strategy function can be ob- R= (19)
tained as Eq. (11), and the inequality Eq. (12) can be obtained Rmin , p = 1
from where p denotes sensor node i’s behavioral discrimination
∑ value. When p = 1, the behavior of sensor node i is judged
π ∗ (y) = arg max Px y (y∗)V ∗ (y∗) (11)
x∈X to be negative; when p = 0, the behavior of sensor node i
y∗∈Y
is judged to be positive; Rmax denotes the maximum reward,
π∗
V ∗ (y) = V (y) ≥ V π (y) (12) and Rmin denotes the minimum reward. When the behavior
of the sensor node i is judged to be positive, update the
The state–action value function Qπ (y, x) can be used instead comprehensive trust value with the node i’s location.
of the V π (y) . Qπ (y, x) denotes the reward obtained by the agent
in the current environment after receiving and executing the n i → Q t = (1 − α) · n i → Q t−1 + α · (R ′ + γ · Q max ) (20)
strategy π , as shown in Eq. (13). where γ denotes the discount factor, and Qmax denotes the
π π
Q (y, x) = E [gt + γ gt+1 + γ gt+2 + · · · |y, x]
2 maximum comprehensive trust value of the surrounding relay
(13) nodes. R′ denotes the reward under the positive behavior of
= E y ∗ [gt + γ Q π (y∗, x∗)|y, x]
sensor node i, as shown in Eq. (21):
when the Q value obtains the maximum value, the correspond- {
Rmid , p = 0 and ρsni > ρsn j
′
ing optimal value function is also the maximum value and is R = (21)
0, p = 0 and ρsni < ρsn j
unique. As shown in Eq. (14).
where Rmid denotes the medium reward. The Algorithm 1
Q ∗ (y, x) = max Q π (y, x) (14) depicts the comprehensive trust value update algorithm.
π
1080
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

The node is able to transmit when Er e reaches the energy

threshold λen . Otherwise, it cannot participate in transmission.
In summary, the Q-learning secure routing algorithm based on
authenticated-encryption is shown in Algorithm 2.

3. Experimental results
In this section, we have experimented with and analyzed the
proposed algorithm. In addition, we compare the performance
of the algorithm with several other algorithms under different
types of attacks. Where we obtained results from the compari-
son algorithm by repetition multiple simulations are performed
to average the experimental results. The ratio of selfish nodes
is increased from 5% to 50%. Simulation parameters are given
in Table 2.
The most secure routing algorithm selects the relay node Table 2
with the highest comprehensive trust, which will increase Simulation parameters.
the energy consumption of the node. Therefore, the energy Parameters Value
consumption of the network needs to be balanced. The energy Monitoring region 100 m × 100 m
consumption of transferring l bit of data to a distance of d is Number of nodes deployed 100
Communication distance 20 m
E T R = 2E elec l + ld 2 εamp (22) Node initial energy 10 J
Location of sink node (50, 50)
where Eelec is the energy consumption per bit, εamp is the Eelec /(J bit−1 ) 50 × 10−9
energy consumed to achieve a certain signal to noise ratio. εamp /(J bit−1 m−4 ) 0.0013 × 10−12
Suppose the initial energy of node is E0 , and the expected εmin 0.05
λen 3 J
supplementary energy of sensor node in a training round is λtr 1000
E f , then its residual energy is Er e .
Er e = E 0 − E T R + E f (23) 3.1. Packet delivery rate

PDR is to be defined as the ratio of the sum of packets

received by the target node to the sum of packets sent by the
source node [5]. Fig. 4 shows the comparison of PDR with
different selfish node ratios.
Under the sinkhole attack, the PDR of the four algo-
rithms decreases as the ratio of selfish nodes increases. At
the 50% ratio, the PDR of EH-QSRAE is 51.1% higher
than that of EDTM. It is because EH-QSRAE introduces

Fig. 4. Comparison of PDR under different selfish node ratios.

1081
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

Fig. 6. Comparison of TPR under different selfish node ratios.

Fig. 5. Comparison of DR under different selfish node ratios.

and ATRP assesses nodes’ trust value without considering

dynamic reward to compute the comprehensive trust value of
abnormal node behavior during packet processing. Thus, the
nodes and decreases the comprehensive trust value of sinkhole
DR of these three algorithms is inferior to that of EH-QSRAE.
nodes rapidly. In the meantime, the nodes in EH-QSRAE
are equipped with solar panels, so that the nodes have more
3.3. True positive rate
residual energy. Therefore, when the ratio reaches 50%, the
PDR of EH-QSRAE is 71.7%, and the PDR of SOS-RPL,
InDReS, and EDTM is only 63.1%, 47.1%, and 20.6%. Under TPR is the ratio of the number of correctly identified selfish
the blackhole attack, the PDR of EH-QSRAE decreases slowly nodes to the number of nodes identified as selfish [5]. Fig. 6
with an increasing selfish node ratio, and the decreasing trend shows the comparison of TPR with different selfish node
of the other three algorithms is evident. When the ratio reaches ratios.
50%, the PDR of EH-QSRAE is 54.9%, 61%, and 59.5% Under the sinkhole attack, the TPR of the four algorithms
greater than that of SQEER, ELPC, and ATRP, respectively. decreases as the selfish node ratio increases. At the 50%
Indeed, SQEER, ELPC, and ATRP all adopt a static trust ratio, EH-QSRAE has the highest TPR with 89%. InDReS
evaluation mechanism. Therefore, their PDR is significantly has the lowest TPR with 72%. Next, SOS-RPL and EDTM
lower than EH-QSRAE’s. have TPR of 82% and 72.8%, respectively. EH-QSRAE uses
step detection to detect the behavior and location of nodes.
3.2. Detection rate of selfish nodes Despite the sinkhole nodes’ disguising or broadcasting false
routes to cheat packets, EH-QSRAE can still discriminate
DR is the ratio of the number of identified selfish nodes by the locations recorded by surrounding nodes. And SOS-
to the total number of selfish nodes [5]. Fig. 5 shows the RPL detects nodes by the rank difference between parent and
comparison of DR with different selfish node ratios. child nodes, which cannot be identified quickly and effectively.
Under the sinkhole attack, the DR of the four algorithms InDReS only calculates nodes’ trust value by the number of
decreases with an increasing selfish node ratio. At the 50% packet losses. Even though EDTM has both direct and indirect
ratio, the DR of EH-QSRAE is 31.3% higher than that of trust, the weight of these two trust values cannot be adaptive.
InDReS. This is because the ID database is established by the With the increase of the ratio, there is a large difference
PUF-based authenticated-encryption mechanism. In addition, between the trust value of the nodes computed by EDTM and
EH-QSRAE checks the node behavior and also calculates the the actual values. Therefore, the TPR of EH-QSRAE is higher
node location. It therefore has a high DR. When the ratio than the other three algorithms. Under the blackhole attack,
reaches 50%, the DR of EH-QSRAE is 92%, while the DR of the TPR of the four algorithms decreases with an increasing
EDTM, SOS-RPL, and InDReS is 83%, 77.9%, and 60.7%. selfish ratio. With a ratio of 50%, the TPR of EH-QSRAE
Under the blackhole attack, the DR of the four algorithms de- is 90%, whereas the TPD of SQEER, ELPC, and ATRP are
creases with the increase of the selfish ratio. Among them, the 83.5%, 87.4%, and 84.3%, respectively.
decreasing trend of EH-QSRAE is slower, and the decreasing
trend of the other three algorithms is faster. At a ratio of 50%, 3.4. Average residual energy of nodes
the DR of EH-QSRAE is 38.5%, 40.9%, and 49.3% higher
than that of ELPC, SQEER, and ATRP, respectively. This is Fig. 7 shows the comparison of the average residual energy
because ELPC detects only abnormal energy consumption of of nodes with different selfish node ratios. Under the sinkhole
nodes; SQEER does not dynamically detect abnormal nodes; attack, the average residual energy of the four algorithms
1082
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

this algorithm with reinforcement learning tools to improve its

resistance to trust model attack.

CRediT authorship contribution statement

Cuiran Li: Conceptualization, Methodology, Writing – re-
view & editing, Supervision, Resources, Funding acquisition.
Jixuan Wu: Software, Validation, Formal analysis, Investiga-
tion, Data curation, Writing – original draft. Zepeng Zhang:
Writing – review & editing. Anqi Lv: Writing – review &
editing.

Declaration of competing interest

The authors declare that they have no known competing
financial interests or personal relationships that could have
appeared to influence the work reported in this paper.
Fig. 7. Comparison of average residual energy under different selfish node
ratios. Acknowledgments
This work was partially supported by the National Natural
shows a decreasing trend with an increasing selfish node ratio. Science Foundation of China (62161016), and the Science and
At 50%, the average residual energy of EH-QSRAE is 84%, Technology Plan of Gansu Province, China (20JR10RA273).
which is much higher than the 59% of EDTM and higher than
SOS-RPL and InDReS at 71% and 69.3%, respectively. This References
is due to the fact that the nodes in EH-QSRAE are able to [1] J. Wang, H. Han, H. Li, et al., Multiple strategies differential privacy
replenish the energy consumed by solar energy. In addition, on sparse tensor factorization for network traffic analysis in 5G, IEEE
EH-QSRAE adjusts routing based on node energy so that Trans. Ind. Inform. 18 (3) (2021) 1939–1948.
[2] J. Wang, Y. Cao, B. Li, et al., Particle swarm optimization based
low-energy nodes that do not reach λen can replenish energy.
clustering algorithm with mobile sink for WSNs, Future Gener.
As a result, nodes have a higher average residual energy. Comput. Syst. 76 (2017) 452–457.
EDTM’s multi-dimensional evaluation mechanism consumes [3] J. Wang, C. Ju, Y. Gao, et al., A PSO based energy efficient coverage
more energy and cannot replenish energy, so the average control algorithm for wireless sensor networks, Comput. Mater. Contin.
residual energy of nodes decreases most rapidly. Under the 56 (3) (2018) 433–446.
[4] J. Wang, Y. Gao, X. Yin, et al., An enhanced PEGASIS algorithm with
blackhole attack, the average residual energy of the four al- mobile sink support for wireless sensor networks, Wirel. Commun.
gorithms shows a decreasing trend as the selfish node ratio Mob. Comput. 2018 (2018) 1–9.
increases. At 50%, the average residual energy of EH-QSRAE [5] M. Zaminkar, R. Fotohi, SoS-RPL: securing internet of things against
is 85%, which is much higher than the 51.8% and 50.9% of sinkhole attack using RPL protocol-based node rating and ranking
mechanism, Wirel. Pers. Commun. 114 (2) (2020) 1287–1312.
ELPC and ATRP, and is slightly higher than SQEER at 80.9%, [6] M. Surendar, A. Umamakeswari, InDReS: An intrusion detection
respectively. Since the SQEER node evaluation metric has less and response system for internet of things with 6LoWPAN, in:
overhead, the average residual energy of nodes has a slow 2016 International Conference on Wireless Communications, Signal
decreasing trend. In ELPC, a large amount of node information Processing and Networking (WiSPNET), IEEE, 2016, pp. 1903–1908.
[7] N. Dharini, N. Duraipandian, J. Katiravan, ELPC-trust framework
has to be stored when evaluating nodes’ trust values, which
for wireless sensor networks, Wirel. Pers. Commun. 113 (4) (2020)
leads to high overhead. Since ATRP ignores node energy, it 1709–1742.
cannot balance the energy consumption of the network, which [8] J. Jiang, G. Han, F. Wang, et al., An efficient distributed trust model
results in a low average residual energy of nodes. for wireless sensor networks, IEEE Trans. Parallel Distrib. Syst. 26
(5) (2015) 1228–1237.
[9] N.A. Khalid, Q. Bai, A. Al-Anbuky, Adaptive trust-based routing
4. Conclusion protocol for large scale WSNs, IEEE Access 7 (2019) 143539–143549.
[10] T. Kalidoss, L. Rajasekaran, K. Kanagasabai, et al., Qos aware trust
This paper proposes an energy-harvesting Q-learning se- based routing algorithm for wireless sensor networks, Wirel. Pers.
cure routing algorithm with authenticated-encryption, which Commun. 110 (4) (2020) 1637–1658.
is implemented to verify the nodes, detect selfish nodes, and [11] A. Puviarasu, M. Balaji, R. Thirukkumaran, et al., Dynamic uneven
clustering protocol for efficient energy management in EH-WSNs,
balance the network energy consumption. For the node’s lim- Mater. Today Proc. 57 (2022) 2092–2095.
ited energy problem, solar panels are used to charge the [12] J. Wang, Y. Gao, C. Zhou, et al., Optimal coverage multi-path
nodes. Besides, the EEMD-LSTM model is used to predict scheduling scheme with multiple mobile sinks for WSNs, Comput.
the supplemental energy of nodes. Experimental results show Mater. Contin. 62 (2) (2020) 695–711.
[13] Qin Danyang, Jia Shuang, Yang Songxiang, Ma Jingya, Zhang Yan,
that the proposed algorithm can improve the PDR, balance the Ding Qun, Research on wireless sensor cyber security routing mech-
network energy consumption, and improve the DR and TPR. anism based on trust perception, Chin J Commun 38 (10) (2017)
As a next step, we will optimize the trust evaluation model of 60–70.
1083
C. Li, J. Wu, Z. Zhang et al. ICT Express 9 (2023) 1077–1084

[14] P. Zou, B. Hou, J. Lei, et al., Bearing fault diagnosis method based on [16] W. Wang, C. Qiu, Z. Yin, et al., Blockchain and PUF-based lightweight
EEMD and LSTM, Int. J. Comput. Commun. Control 15 (1) (2020). authentication protocol for wireless medical sensor networks, IEEE
[15] Y. Li, X. He, C. Yin, Energy aware opportunistic routing for energy Internet Things J. 9 (11) (2022) 8883–8891.
harvesting wireless sensor networks, in: 2020 IEEE 31st Annual [17] R. Lu, S.H. Hong, X. Zhang, A dynamic pricing demand response
International Symposium on Personal, Indoor and Mobile Radio algorithm for smart grid: reinforcement learning approach, Appl.
Communications, IEEE, 2020, pp. 1–6. Energy 220 (2018) 220–230.

1084