Available online at [Link].

com

ScienceDirect
IFAC PapersOnLine 58-30 (2024) 332–337

Learning-based Characterization of Noise

Statistics for SoC Estimation via Kalman
Filtering
Ahmet Can Erdem ∗,∗∗ Volkan Mert ∗ Barış Tekin ∗∗
Tuncay Altun ∗∗∗ Derya Ahmet Kocabaş ∗∗
∗
Turkish Aerospace, Air Vehicle Technologies Research Center,
Istanbul, Türkiye.
∗∗
Faculty of Electrical and Electronics, Department of Electrical
Engineering, Istanbul Technical University, Istanbul, Türkiye.
∗∗∗
Faculty of Engineering and Architecture, Department of Electrical
Electronics Engineering, Yozgat Bozok University, Yozgat, Türkiye.

Abstract: In the field of Battery Management Systems (BMS), the State of Charge (SoC) is
a crucial metric that represents the available energy capacity and directly affects the opera-
tional strategy. Accurately determining SoC is inherently complex due to the electrochemical
characteristics that exhibit non-linear responses under various operational conditions. BMS
operates model-based on an Extended Kalman Filter (EKF). In contrast, the employed Deep
Deterministic Policy Gradient (DDPG) algorithm represents a model-free reinforcement learning
methodology. This study aims to investigate the learning-based characterization of the process
and measurement noise statistics for state of charge estimation via Kalman filtering. The method
iteratively updates a value-function based on a reward mechanism, facilitating the selection
of actions that minimize estimation error without a model of the environment. The model
proposed in this study enhances EKF, a method known for its robustness in tracking SoC,
by incorporating a RL paradigm. This paradigm is tailored to optimize parameter estimation
despite sparse datasets. The adaptive mechanism is governed by a reward function that is based
on minimizing SoC estimation error. This represents a judicious calibration between model-
based and data-driven estimation techniques. The integration of DDPG improves our model’s
adaptability to SoC dynamics, promising enhanced estimation accuracy and improved reliability
and efficiency in BMS across diverse applications.
Copyright © 2024 The Authors. This is an open access article under the CC BY-NC-ND license
([Link]
Keywords: Battery; Kalman Filter; State of Charge; Deep Deterministic Policy Gradient;
DDPG;

1. INTRODUCTION capacity. Since the SoC cannot be measured directly from

the (+) and (-) terminals of the battery, its estimation is
A battery is basically a storage device that converts made from measured time-series data [3].
chemical energy into electrical energy. The battery is a
complex system with many state variables. Estimating the Various methodologies for estimating the states of bat-
SoC and SoH is fundamental to the ability of Battery teries are classified into four primary categories: direct
Management System (BMS) operation and control of the measurement, empirical, model-based, and data-driven ap-
battery. proaches. Examples of direct measurement methods in-
clude the Open Circuit Voltage Method, Terminal Volt-
Battery states such as charge status and health status age Method, Impedance Method, and Impedance Spec-
cannot be measured directly. In order to control the troscopy Method. The empirical methods comprise the
battery system, prediction is made using inputs from Coulomb Counting Method and Modified Coulomb Count-
voltage, current, and temperature measurements [1]. In ing Method. In the context of model-based methods,
order to make these predictions, voltage, current, and techniques such as the Kalman Filter, Equivalent Circuit
temperature values are recorded using sensors and data Model, and Incremental Capacity Analysis are employed.
acquisition system [2]. Finally, data-driven methods include Fuzzy Logic, Arti-
ficial Neural Networks (ANN), Support Vector Machines
SoC is a critical status indicator for the battery, essential
(SVM), and Genetic Algorithms.
for its operation, control, and maintenance. A precise SoC
estimation is very important to ensure the safe operation Equivalent Circuit Model (ECM) is the most commonly
of the battery and prevent situations such as overload and used method for off-the-shelf BMSs. ECM is created
over discharge. In simple terms, SoC can be defined as the to electrically simulate the dynamics of the battery by
ratio of current battery capacity to the maximum battery analysing the physical and chemical reactions that take
2405-8963 Copyright © 2024 The Authors. This is an open access article under the CC BY-NC-ND license.
Peer review under responsibility of International Federation of Automatic Control.
10.1016/[Link].2025.01.203
 Ahmet Can Erdem et al. / IFAC PapersOnLine 58-30 (2024) 332–337 333

place in the battery. ECM uses a combination of circuit 2. BACKGROUND

elements such as resistance, capacitance and voltage source
[4]. When modelling the battery as an electrical circuit, 2.1 SoC estimation based on EKF with equivalent circuit
the most commonly used RC network consists of parallel model
connected RC networks connected in series with the inter-
nal resistance R0 . The number of RC networks indicates The equivalent circuit model is a popular choice for battery
the degree of the model. To illustrate, if the model has 2 modeling due to its minimal computational demands [9].
parallel RC networks connected in series with each other, The equivalent circuit model for Li-ion batteries consists of
the degree of the model is considered to be second order a voltage source, resistors, and capacitors, with the battery
[5]. dynamics represented by RC branches. Although adding
In addition to the use of Equivalent Circuit Model (ECM), more RC branches enhances the model’s accuracy, it also
another prevalent technique utilized in off-the-shelf BMS increases its complexity. In this paper, 1RC equivalent
is state estimation through the implementation of the circuit model is selected to decrease computational burden
Kalman Filter. The model-based Kalman Filter is the of SoC estimation. The 1RC equivalent circuit model is
optimal estimator used to estimate the internal states of shown in Fig.1.
the system [6]. The Kalman Filter has been proposed
to accurately estimate the state of systems by remov-
ing unwanted noise from a dataset, especially for multi-
input systems. In [7], the study demonstrates that noise
characterization may be achieved through the estimation
of process and measurement noise covariances, Q and R,
Fig. 1. 1RC equivalent circuit model
derived from the covariance function of the innovations
obtained from any stable filter. For a multi-input system
such as an BMS, the Kalman Filter can better represent Here, Vt represents the terminal voltage, I indicates the
the internal dynamics of the battery. However, the efficacy current, R0 refers to the series resistance, R1 and C1
of the Kalman filter is contingent upon the precise char- denote the resistance and capacitance, respectively, of the
acterization of process and measurement noise. Two key RC branch, and OCV stands for the Open Circuit Voltage.
factors must be considered: the accuracy of the prediction The state-space model is derived from the 1RC equivalent
model (in this case, the ECM model) and the precision in circuit model. The voltage across the RC branch is denoted
modeling noise characteristics. [3]. as U1 . The state variables are defined as U1 and SoC. The
RL is emerging as a potent method for tackling diffi- structure of the circuit dictates that the state-space model
culties associated with tuning parameters in the Kalman is characterized as follows:
 
Filter. As a sophisticated machine learning strategy, RL   0   1
˙ 0 −
allows an agent to learn ideal strategies by engaging SoC SoC  
ẋ = = 1 +  C1n  I (1)
with its environment, effectively managing the trade-off U̇1 0 − U1
R1 C 1 −
between exploration and exploitation. Given its capacity C1
to autonomously adjust to evolving conditions, RL holds Vt = OCV (SoC) − U1 − R0 I (2)
promise for refining Kalman Filter Parameters, thereby
improving the precision and reliability of SoC estimation where Cn is Columb Efficiency.
[8].
The EKF must be utilized with the state space model
In the described RL estimation method, the training equations because the measurement equation is nonlinear.
phase utilizes a learning model derived from interactions The EKF operates in two stages: the model-based pre-
with the battery model and actual current and voltage diction phase (5) and (6), and the measurement-based
measurements under Constant Current Constant Voltage update phase (7), (8) and (9). During the prediction
(CCCV) conditions. The algorithm’s reward is determined phase, the state and covariance are forecasted as follows:
by the integral of the error between the observed battery xk+1 = Ak xk + Bk uk + wk (3)
voltage and that predicted by a SoC observer.
zk = Hk xk + vk (4)
The subsequent sections delineate the organization of this x̂k|k−1 = Ak−1 x̂k−1|k−1 + Bk−1 uk−1 (5)
paper. In Section II, we present an in-depth discussion
of the battery model and elaborate on RL methodology Pk|k−1 = Ak−1 Pk−1|k−1 ATk−1 + Qk−1 (6)
−1
employed in this study. Section III introduces innovative Kk = Pk|k−1 HkT (Hk Pk|k−1 HkT+ Rk ) (7)
approaches and methodologies designed to enhance the ac-
x̂k|k = x̂k|k−1 + Kk (zk − Hk x̂k|k−1 ) (8)
curacy and efficiency of SoC estimation. Section IV focuses
on the simulation results, providing a comprehensive anal- Pk|k = (I − Kk Hk )Pk|k−1 (9)
ysis of the data and findings. Finally, Section V presents
a synthesis of the key conclusions and implications of In the Kalman Filter, xk represents the state vector, Ak
our research, highlighting the contributions and potential the state transition matrix, Bk the control input matrix,
directions for future work. and wk the process noise, which reflects model uncer-
tainties. The measurement vector zk includes observations
influenced by measurement noise vk , with Hk mapping the
state space to the observed space. The filter operates in two
phases: predicting the next state and its uncertainty, and
334 Ahmet Can Erdem et al. / IFAC PapersOnLine 58-30 (2024) 332–337

updating these predictions by integrating new measure- expected return. The feedback from the critic is used to
ments using the Kalman Gain as given in (7) to minimize optimize the actor’s policy [8]. The overview diagram of
error covariance. the DDPG algorithm is shown in Fig.2.
The matrices Q and R are essential for the performance of
the filter. In order to avoid confusion, it is necessary to de-
fine the terms used in this study. The Process Covariance
Matrix will be referred to as Qp and the Measurement
Covariance Matrix as Rm . The matrix Qp , which is the
covariance matrix for process noise, is typically set up as
follows:  2 
σSoC 0
Qp = (10)
0 σV2 1
2
Here, σSoC and σV2 1 represent the variances of the SoC and
V1 due to internal fluctuations. The measurement noise Fig. 2. Overview diagram of the DDPG algorithm
covariance matrix Rm is configured as:
 
Rm = σV2 t (11)
3. PROPOSED METHOD
where σV2 t reflects the variance of the error in terminal
voltage measurements. This study presents an advanced method for SoC esti-
Calibrating these matrices, along with thorough testing mation using an EKF with adaptively tuned covariance
and simulation, confirms the effectiveness of the EKF matrices, enhancing estimation accuracy under variable
method in practical settings [10]. operational conditions. The novel aspect of this method-
ology is the dynamic adjustment of Qp and Rm during
2.2 Reinforcement Learning - Deep Deterministic Policy battery charging and discharging cycles.
Gradient The adjustments to Qp are based on the rate of SoC change
and environmental factors like temperature, particularly
Reinforcement Learning is a machine learning algorithm during conditions such as rapid charging where internal
that draws inspiration from the way humans or animals chemical dynamics are less predictable. This necessitates
progressively learn to perform tasks through repeated tri- an increased Qp to accommodate the greater uncertainty
als [13]. RL has gained popularity in artificial intelligence. in state predictions, ensuring the EKF’s accuracy by ac-
It functions on a reward and punishment system, program- counting for fluctuating internal states of the battery.
ming agents based on trial-and-error interactions within a Simultaneously, Rm is modified in response to changes
dynamic environment, guided by exploration mechanisms. in measurement accuracy affected by sensor noise and
Essentially, agents perceive their surroundings and act to temperature-induced variations in internal resistance. El-
maximize rewards, ultimately aiming to identify optimal evated temperatures, which can degrade the precision of
actions for the greatest returns [11]. voltage and current measurements, necessitate higher Rm
values to reflect the decreased reliability of sensor outputs.
Learning-based optimization techniques have been devel-
This adaptive strategy ensures that the EKF maintains
oped to address the limitations of conventional optimiza-
robust performance by appropriately balancing the confi-
tion methods. RL stands out as a common decision-making
dence in predicted states against actual measurements.
approach that has found extensive application across var-
ious fields, though it is infrequently utilized in physical The adaptive tuning of Qp and Rm employs the DDPG
control tasks due to its sample inefficiency and the curse of algorithm, a reinforcement learning technique suitable
dimensionality. Deep Learning (DL), with its deep neural for continuous action spaces, facilitating real-time adjust-
networks, excels in feature extraction. Combining DL with ments. This dynamic approach contrasts with conven-
RL, Deep Reinforcement Learning (DRL) emerges with tional methods that use static covariance values, which
superior representation and decision-making capabilities often result in suboptimal performance during rapid state
[12]. changes or shifts between operational modes. By dynam-
ically adapting these matrices, the proposed method al-
The Deep Q-Network (DQN) algorithm marked the initial
lows the EKF to respond to real-time changes in battery
significant advancement in the field of DRL, outperforming
conditions, significantly enhancing the accuracy and re-
humans in video game playing. Consequently, numerous
liability of SoC estimations. These estimations are cru-
DRL-focused papers have been published, aiming either
cial for optimizing battery management, improving safety,
to enhance the performance of DQN or to extend DRL
and extending battery lifespan. The DDPG algorithm’s
methodologies to different areas. Since DQN is limited
implementation involves a critic network with 9 layers,
to making decisions within a discrete action space, other
starting with an input layer of 2 features, followed by a
approaches such as Deep Deterministic Policy Gradient
fully connected layer with 50 units, a ReLU activation,
have been developed to address this limitation [13].
another fully connected layer with 25 units, a second input
The DDPG algorithm features two main elements: an layer of 1 feature, another fully connected layer with 25
actor and a critic. The actor is a network that determines units, an addition layer for element-wise addition, another
an action based on the current state of the system, ReLU activation, and a final fully connected layer with 1
whereas the critic is a network that assesses the state- unit. The actor network comprises 8 layers, beginning with
action combination and produces a Q-value, indicating the an input layer of 2 features, followed by fully connected
 Ahmet Can Erdem et al. / IFAC PapersOnLine 58-30 (2024) 332–337 335

layers with 50, 25, and 1 unit, interspersed with ReLU The selected SoC range for isdone was [0.25 0.95] with
activations, a hyperbolic tangent activation, and a scaling a margin of 0.05. However, the actual charge/discharge
layer. range is between 0.3 and 0.9. Subsequent analysis of the
simulations revealed that the values never approached
4. SIMULATION AND RESULTS these limits. The following subsections will address the
training of Rm and Qp , respectively.
This section presents the results of a simulation study
investigating the efficacy of a reinforcement learning ap- 4.1 Enhancing SoC Accuracy through Rm Training
proach with a DDPG agent for estimating the bat-
tery SoC. In this study, to ensure the reliability of the The Rm value is set to be 0.7 for the estimation of SoC
charging and discharging processes, extensive simulations with EKF. In the training process with Rm , the value
were conducted using MATLAB® /SIMULINK® . The of Rm was constrained to a range between 0 and 1. For
cell under consideration in this paper is the A123 ALM 100 episodes, an illustration of the Reinforcement Learning
12V7. The simulations utilized a comprehensive charg- Episode Manager is provided in Fig.3.
ing/discharging dataset spanning 21600 seconds, providing
robust insights and validating the efficacy of the pro-
posed methodologies.[15]. A total of 100, 500, 1000, and
5000 episodes of 21600 seconds of charging states were
trained according to the simulation computer available
for this study. The simulation computer is equipped with
a 13th generation INTEL® Core i9-13900KF processor,
comprising 32 cores operating at a base clock speed of
3.0 GHz. The system is equipped with 64 GB of random-
access memory (RAM) and an NVIDIA® GeForce RTX
4090 graphics processing unit (GPU) with 24 GB of video
RAM (VRAM) and a total GPU memory of approximately
55.5 GB. The system is operational in accordance with Fig. 3. RL Training for Rm (100 Episodes)
BIOS version 8.20. It was observed that the training
period remained consistent up to a certain number of
episodes. Therefore, as a comparison, the results of a fast In the remainder of this paper, the SoC obtained with EKF
training trial were compared with the longest simulations and adaptive Rm and Qp will be referred to as ”Estimated
available. These simulation studies were conducted us- SoC-DDPG” (blue line in graphs). The SoC obtained with
ing MATLAB® R2023b with the Reinforcement Learning EKF without adaptive Rm and Qp will be referred to as
Toolbox. ”Estimated SoC-EKF” (red line in graphs).

The Reinforcement Learning Toolbox comprises three in- Upon completion of 100 training episodes, the SoC ob-
puts for the RL Agent block: observation, reward and tained with EKF with adaptive Rm , the SoC obtained with
isdone. The selected inputs for observation were error EKF without adaptive Rm and Qp , and the Real SoC are
and estimated SoC. Here, error is defined as the difference presented in Fig.4.
between the real and estimated SoC values. The reward
function is 
defined as follows:
 t


1 − 0.1 e2 dt − 0.5 RMS2 , |e| ≤ 0.02
Reward = 0  t


−0.1 − 0.1 e2 dt − 0.5 RMS2 , |e| > 0.02
0
(12)
This reward function encourages the prediction of accurate
values by offering a positive reward if the absolute error,
denoted by the symbol |e|, is within a small margin, defined Fig. 4. Comparison of the SoC Estimations (100 Episodes)
as 0.02 at most. Upon examination of other studies on the
subject, it was determined that the error value is set to The same training procedure is conducted for 5000
0.02 [14]. Conversely, a negative reward is assigned when episodes for Rm covariance matrix. For 5000 episodes, an
t
the absolute error exceeds the threshold. The term 0 e2 illustration of the Reinforcement Learning Episode Man-
represents the cumulative squared error over time, thereby ager is provided in Fig.5.
encouraging consistent performance across the episode. A
In the same manner, upon completion of 5000 training
scaling factor of 0.1 provides a balance between immediate
episodes, the SoC obtained with EKF with adaptive Rm ,
accuracy and long-term performance, offering clear and
the SoC obtained with EKF without adaptive Rm and Qp ,
effective feedback to the agent. Furthermore, the Root
and the Real SoC are presented in Fig.6.
Mean Square (RMS) error is also considered, and the term
RM S 2 is included in the reward function. This indicates The results of the training process yielded RMS error
that higher RMS error values contribute negatively to values for 100 and 5000 episodes. Table 1 presents a
the reward. The same reward function was employed for comparison of EKF without adaptive Rm and Qp and EKF
training the Rm and Qp covariance matrices. with adaptive Rm according to RMS values.
336 Ahmet Can Erdem et al. / IFAC PapersOnLine 58-30 (2024) 332–337

Fig. 7. RL Training for Qp (100 Episodes)

Fig. 5. RL Training for Rm (5000 Episodes)

Fig. 8. Comparison of the SoC Estimations (100 Episodes)

Fig. 6. Comparison of the SoC Estimations (5000
Episodes)
Table 1. Comparison of RMS Errors for EKF
without adaptive Rm and Qp [RMS EKF]
and EKF with adaptive Rm Methods [RMS
DDPG]
EPISODE NO RMS EKF RMS DDPG IMPROVEMENT
100 0.01087 0.009782 10.01%
5000 0.01087 0.009129 16.02%

Table 1 illustrates the substantial enhancement in SoC

estimation precision achieved through EKF with adap- Fig. 9. RL Training for Qp (5000 Episodes)
tive Rm in comparison to EKF without adaptive Rm
and Qp . Following 100 episodes of training, RMS error In the same manner, upon completion of 5000 training
for EKF with adaptive Rm decreased by 10.01%, while episodes, the SoC obtained with EKF with adaptive Qp ,
after 5000 episodes, the reduction reached 16.02%. The the SoC obtained with EKF without adaptive Rm and Qp ,
corresponding plots of the estimated SoC reveal that EKF and the Real SoC are presented in Fig.10.
with adaptive Rm method closely follows the real SoC
values, outperforming EKF without adaptive Rm and Qp .
The graphs of the reinforcement learning training progress
further illustrate that the DDPG agent’s reward values in-
crease substantially over the training episodes, indicating
enhanced learning and improved performance.

4.2 Enhancing SoC Accuracy through Qp Training

The Qp value is set to be 10−4 for the estimation of SoC

with EKF. In the training process with Qp , the value of Fig. 10. Comparison of the SoC Estimations (5000
Qp was constrained to a range between 0 and 0.01. For Episodes)
100 episodes, an illustration of the Reinforcement Learning
Episode Manager is provided in Fig.7. The results of the training process yielded RMS error
values for 100 and 5000 episodes. Table 2 presents a
Upon completion of 100 training episodes, the SoC ob- comparison of EKF without adaptive Rm and Qp and EKF
tained with EKF with adaptive Qp , the SoC obtained with with adaptive Qp according to RMS values.
EKF without adaptive Rm and Qp , and the Real SoC are
Table 2 illustrates the substantial enhancement in SoC
presented in Fig.8.
estimation precision achieved through EKF with adaptive
The same training procedure is conducted for 5000 Qp in comparison to EKF without adaptive Rm and Qp .
episodes for Qp . For 5000 episodes, an illustration of the Following 100 episodes of training, RMS error for EKF
Reinforcement Learning Episode Manager is provided in with adaptive Qp decreased by 16.67%, while after 5000
Fig.9. episodes, the reduction reached 39.50%. The correspond-
 Ahmet Can Erdem et al. / IFAC PapersOnLine 58-30 (2024) 332–337 337

Table 2. Comparison of RMS Errors for EKF [2] Samanta, A., Chowdhuri, S., Williamson, S.
without adaptive Rm and Qp [RMS EKF] S. Machine learning-based data-driven fault
and EKF with adaptive Rm Methods [RMS detection/diagnosis of lithium-ion battery: A critical
DDPG] review. Electronics, 10(11), 1309. (2021).
[3] Arunagirinathan, S., Subramanian, C. Analysis of a
EPISODE NO RMS EKF RMS DDPG IMPROVEMENT Li-ion battery state of charge by artificial neural net-
100 0.01087 0.009058 16.67%
work. Bulletin of Electrical Engineering and Informat-
5000 0.01087 0.004291 39.50%
ics, 12(2), 792-799.(2023).
ing plots of the estimated SoC reveal that EKF with adap- [4] Hu, X., Xu, L., Lin, X., Pecht, M. Battery lifetime
tive Qp closely follows the real SoC values, outperforming prognostics. Joule, 4(2), 310-346.(2020).
EKF without adaptive Rm and Qp . The graphs of the rein- [5] Li, L., Li, Z., Zhao, J., Guo, W. Lithium-ion bat-
forcement learning training progress further illustrate that tery management system for electric vehicles. Interna-
the DDPG agent’s reward values increase substantially tional Journal of Performability Engineering, 14(12),
over the training episodes, indicating enhanced learning 3184.(2018).
and improved performance. [6] Hussein, A. A. Capacity fade estimation in electric
vehicle li-ion batteries using artificial neural networks.
The RMS error comparisons for Rm and Qp demonstrate IEEE Transactions on Industry Applications, 51(3),
that the EKF with adaptive Rm or Qp method consis- 2321-2330. (2014).
tently achieves a lower RMS error than the EKF without [7] Bulut, Y., Vines-Cavanaugh, D., & Bernal, D. (2011).
adaptive Rm and Qp method, indicating a more accurate Process and measurement noise estimation for Kalman
SoC estimation for both noise covariance matrices. This filtering. In Structural Dynamics, Volume 3: Proceed-
suggests that adapting the Rm and Qp matrices via RL ings of the 28th IMAC, A Conference on Structural
positvely impact SoC estimation accuracy. In conclusion, Dynamics, 2010 (pp. 375-386). Springer New York.
DDPG based RL is more effective in handling noise char- [8] Yin, Y., Zhu, X., Zhao, X. A deep reinforcement learn-
acteristics represented by Rm and Qp , leading to more ing approach for state of charge and state of health esti-
accurate SOC estimations than EKF without adaptive mation in lithium-ion batteries. AIP Advances, 13(10).
Rm and Qp . This highlights the potential advantages of (2023).
advanced reinforcement learning techniques in scenarios [9] Kim, M., Kim, K., Kim, J., Yu, J., & Han, S. (2018).
with complex noise characteristics. State of charge estimation for lithium ion battery
based on reinforcement learning. IFAC-PapersOnLine,
5. CONCLUSION 51(28), 404-408.
[10] Jiani, D., Youyi, W., & Changyun, W. (2013, June).
In conclusion, our study on hybrid SoC estimation em- Li-ion battery SoC estimation using particle filter based
ploying a combination of DDPG and the EKF represents on an equivalent circuit model. In 2013 10th IEEE
a potential advancement in Battery Management Systems International Conference on Control and Automation
technologies. By leveraging the adaptive capabilities of (ICCA) (pp. 580-585). IEEE.
DDPG to optimize the covariance matrices Qp and Rm , [11] Ganesh, A. H., & Xu, B. (2022). A review of rein-
we have managed to improve the noise characterization forcement learning based energy management systems
or model and thus the performance of the EFK, result- for electrified powertrains: Progress, challenge, and po-
ing in acceptable estimation accuracy even under varying tential solution. Renewable and Sustainable Energy
noise conditions. This approach not only demonstrates Reviews, 154, 111833.
the potential for integrating machine learning algorithms [12] Chen, W., Wu, N., & Huang, Y. (2021, July). Real-
with traditional estimation techniques but also promises Time Optimal Dispatch of Microgrid Based on Deep
ensuring scalability and robustness, catering to different Deterministic Policy Gradient Algorithm. In 2021 In-
battery types and operational conditions. The results of ternational Conference on Big Data and Intelligent
our investigation indicate that the DDPG-optimized EKF Decision Making (BDIDM) (pp. 24-28). IEEE.
framework has the potential to be a viable solution for [13] Domı́nguez-Barbero, D., Garcı́a-González, J., &
real-world applications and represents a promising avenue Sanz-Bobi, M. Á. (2023). Twin-delayed deep determin-
for next generation BMS. istic policy gradient algorithm for the energy manage-
ment of microgrids. Engineering Applications of Arti-
ACKNOWLEDGEMENTS ficial Intelligence, 125, 106693.
[14] Hossain, Monowar and Haque, ME and Arif, Moham-
We thank Assoc. Prof. Dr. Emre Koyuncu for providing mad Taufiqul, Kalman filtering techniques for the on-
access to the Cyber Airspace Laboratory at ITU ARC, line model parameters and state of charge estimation of
which was crucial for our experiments. We also appreciate the Li-ion batteries: A comparative analysis}, Journal
the support of Dr. Murad Abu-Khalaf throughout our of Energy Storage, volume=51, pages=104174, 2022
study. Elsevier
[15] MathWorks® Help Center, Battery State
REFERENCES of Charge Estimation, [Link]
[Link]/help/simscape-battery/ug/
[1] Wu, B., Widanage, W. D., Yang, S., Liu, X. Battery
[Link],
digital twins: Perspectives on the fusion of models, data
Accessed: 2024-05-25.
and artificial intelligence for smart battery manage-
ment systems. Energy and AI, 1, 100016.(2020).