Computers in Biology and Medicine 173 (2024) 108345

Contents lists available at ScienceDirect

Computers in Biology and Medicine

journal homepage: [Link]/locate/compbiomed

Cardiovascular disease detection using a novel stack-based ensemble

classifier with aggregation layer, DOWA operator, and feature
transformation
Mehdi Hosseini Chagahi a , Saeed Mohammadi Dashtaki a , Behzad Moshiri a,b ,1 , M.d. Jalil Piran c ,1
a
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
b Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada
c Department of Computer Science and Engineering, Sejong University, Seoul 05006, South Korea

ARTICLE INFO ABSTRACT

Keywords: Due to their widespread prevalence and impact on quality of life, cardiovascular diseases (CVD) pose a
Cardiovascular diseases considerable global health burden. Early detection and intervention can reduce the incidence, severity, and
Stack-based ensemble classifier progression of CVD and prevent premature death. The application of machine learning (ML) techniques to
Aggregation layer
early CVD detection is therefore a valuable approach. In this paper, A stack-based ensemble classifier with an
Machine learning
aggregation layer and the dependent ordered weighted averaging (DOWA) operator is proposed for detecting
Classifier selection
Dependent ordered weighted averaging
cardiovascular diseases. We propose transforming features using the Johnson transformation technique and
(DOWA) operator normalizing feature distributions. Three diverse first-level classifiers are selected based on their accuracy, and
Feature transformation predictions are combined using the aggregation layer and DOWA. A linear support vector machine (SVM)
meta-classifier makes the final classification. Adding the aggregation layer to the stacking classifier improves
classification accuracy significantly, according to the study. The accuracy is enhanced by 5%, resulting in an
impressive overall accuracy of 94.05%. Moreover, the proposed system significantly increases the area under
the receiver operating characteristic (ROC) curve compared to recent studies, reaching 97.14%. It further
reinforces the classifier’s reliability and effectiveness in classifying cardiovascular disease by distinguishing
between positive and negative instances. With improved accuracy and a high area under the curve (AUC), the
proposed classifier exhibits robustness and superior performance in the detection of cardiovascular diseases.

1. Introduction such conditions [7]. The goal of CVD detection is to identify heart
and blood vessel anomalies early on, allow for timely management
Cardiovascular disease (CVD) detection is a critical focus within and treatment, and ultimately, reduce mortality and improve quality of
healthcare due to its status as the leading cause of death globally. A life [8]. Effective detection strategies can also minimize the economic
substantial burden is placed on healthcare systems by these diseases, in- burden of CVDs through the prevention of costly complications, thus
cluding coronary artery disease, heart failure, and stroke [1]. According
reducing the overall healthcare expenditure associated with chronic
to the World Health Organization report, in 2019, about 17.9 million
management of late-stage heart disease [9]. In research studies, various
people died due to CVDs, which includes 32% of all global deaths.
Also, out of 17 million premature deaths (under 70 years old) due to approaches and methodologies were explored to develop advanced
non-communicable diseases, 38% were caused by CVDs [2]. The most detection models for CVDs.
important behavioral risk factors for cardiovascular disease include Machine learning (ML) methods were extensively used to detect
unhealthy diet, inactivity, smoking, and alcohol consumption [2,3]. CVDs, using algorithms to extract patterns from data [10–12]. The ef-
The effects of these risk factors may appear in people in the form of fectiveness of traditional ML algorithms such as logistic regression (LR),
increased blood pressure, blood glucose, blood lipids, and overweight decision trees (DTs), support vector machines (SVMs), random forest
and obesity [2–6]. (RF), K-nearest neighbors (KNN), naïve bayes (NB), and multilayer
Therefore, Prompt and accurate detection of CVDs can lead to early perceptron (MLP) for predicting CVD risk was investigated [13–15]. In
intervention, significantly improving the prognosis for individuals with

E-mail addresses: [Link]@[Link] (M. Hosseini Chagahi), saeedmohammadi.d@[Link] (S. Mohammadi Dashtaki), moshiri@[Link] (B. Moshiri),
piran@[Link] (M.d. Jalil Piran).
1
Senior Member, IEEE.

[Link]
Received 3 November 2023; Received in revised form 14 March 2024; Accepted 17 March 2024
Available online 27 March 2024
0010-4825/© 2024 Elsevier Ltd. All rights reserved.
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

addition, ensemble learning methods, such as RFs and gradient boost- 2. Related work
ing, were used to enhance predictive accuracy by combining multiple
models [16–19]. Detecting CVD using these approaches demonstrated The purpose of this section is to discuss several research works
promising results, underscoring their potential for clinical use. and the different methodologies that other researchers have used to
Due to its ability to automatically extract intricate patterns from approach the problem. The number of patients with cardiovascular
complex data, deep learning (DL), a subset of ML, has gained attention disease has increased exponentially [31]. To predict the occurrence of
in CVD detection. In this study, convolutional neural networks (CNNs) cardiovascular disease, scientists use a variety of methodologies and
were used to analyze electrocardiograms (ECGs), medical images, and algorithms. Cardiovascular disease prognosis has been the subject of
electronic health records to predict CVD outcomes [20–22]. Models many research studies over the years. Studies related to this topic will
based on DL can capture subtle features and improve CVD detection be discussed in the following paragraphs.
accuracy. An ensemble classifier was used to predict cardiovascular disease
A recent study incorporated multi-omics data, including genomics, early in [32]. ExtraTrees classifier, RF, extreme gradient boosting (XG-
proteomics, metabolomics, and transcriptomics, to enhance CVD pre- Boost), and others were incorporated into the proposed framework.
diction models [23]. These approaches combined information from According to the authors, the ensemble classifier outperformed individ-
different molecular levels to identify novel biomarkers and molecular ual classifiers on the IEEE Data Port dataset, achieving a 2% increase
pathways associated with CVD. It was demonstrated that the integra- in accuracy. In [33], the authors presented a design for building an
tion of multi-omics data can improve the accuracy and precision of Intelligent Hybrid Framework using different algorithms and 10-fold
CVD models, providing a more comprehensive understanding of the cross-validation methods on both complete and chosen characteristics.
underlying biological mechanisms [24]. Four different algorithms were used to select features, including LASSO
The use of explainable AI (XAI) techniques in CVD detection is in and Relief algorithms. The authors calculated Chi-Square and 𝑃 -value
line with the growing emphasis on transparency and interpretability in after implementing feature selection techniques.
healthcare applications. To provide interpretable explanations for pre- With the technique of majority voting ensemble in [34], various
dictions and support clinical decision-making, models such as decision algorithms were utilized, resulting in an accuracy rate of 90%. They
trees, rule-based systems, and Bayesian networks were employed [25, conducted a majority vote of all algorithms in order to improve their
26]. By highlighting key features and risk factors for CVD outcomes, overall accuracy. As part of the investigation, the authors computed
these approaches enhanced trust and understanding among healthcare correlations with the target and examined the relationship between
professionals. other attributes and positive data points. Using a Decision Tree-like
CVD detection gained traction as a result of personalized risk assess- structure, the authors in [35] performed data splits at multiple lev-
ment, a concept that aims to tailor preventative strategies based on an els using the chi-squared automated interaction detector technique
individual’s health profile and characteristics [27]. ML algorithms were (CHAID). Using the CHAID decision tree, cardiologists can gain a
combined with patient-specific data, including demographics, clinical deeper understanding of a patient’s health status, enhancing their
history, and lifestyle factors, to develop personalized risk prediction ability to distinguish diseases. The dataset was analyzed using CHAID
models. Health providers could use such models to identify high-risk to improve understanding. To enhance the overall accuracy, a majority
individuals and take proactive measures to prevent CVDs [28,29]. voting process was implemented. MLP, LR, and NB were used for the
While CVD detection has advanced, challenges such as handling voting.
large datasets, ensuring model interpretability, achieving accurate pre- Alghahtani et al. in [36], the stacking classifier structure Was uti-
dictions, and generalizing across diverse populations persist [30]. lized twice. Initially, a stack model was implemented using DT, KNN,
This study introduces the dependent ordered weighted averaging and XGBoost classifiers as first-level classifiers, with a RF serving as the
(DOWA) operator and incorporates it as an aggregation layer in a stack- meta-classifier. The second stack model comprises deep neural network
ing classifier, presenting a new classifier. In this new stacking classifier, (DNN), keras deep neural network (KDNN), and the first stacking
first-level classifiers are selected based on two criteria: accuracy and model as first-level classifiers, utilizing majority voting as the meta-
diversity. The classifier chosen as the meta-classifier is the one that classifier. Their experimental results indicated that the ML ensemble
demonstrates the best performance in detecting CVD. Additionally, raw model forecasts the disease most accurately, with 88.70% accuracy.
data undergoes a better representation through feature transformation In [37], the authors presented an ensemble framework using ten
techniques and is used to feed the introduced classifier. The main different classifiers, including SVM, KNN, LR, and so on. To avoid
contributions of this paper can be summarized as follows. overfitting, the ensemble framework used the LR as a meta learner to
select the best base learners. A fusion of Heart Datasets from various
• By introducing a DOWA operator, we propose a new version of
UCI ML repositories and a publicly available Heart Attack Dataset were
the stacking classifier.
used to validate the proposed algorithm. Experimental results showed
• Feature transformation techniques enhance feature representation
that the stacking classifier performed significantly better than ten single
and quality in our model. We perform comprehensive hyperpa-
classifier models in terms of accuracy and applicability.
rameter tuning for maximum accuracy.
The purpose of this research is to present a new version of the
• Using the two criteria of accuracy and diversity, the best clas-
stacking classifier using a novel layer. Furthermore, the effectiveness
sifiers are selected to be used as first-level classifiers and meta-
of the proposed classifier is determined by the careful selection of first
classifier.
and second-level classifiers.
• Our proposed model achieve an impressive 94.05% accuracy and
97.14% area under the curve (AUC) for cardiovascular diseases.
Consequently, a more robust and reliable model was innovated. 3. Methodology

The rest of this article is structured as follows. Section 2 introduces For the purpose of predicting cardiovascular diseases, this section
related work on predicting cardiovascular diseases using the stacking provides details about the dataset. In Section 3.3, our focus shifts to
classifier. Section 3 provides an overview of the comprehensive expo- introducing a novel classifier specifically designed for cardiovascular
sition of the employed dataset and the methodologies. The simulation disease prediction. A meticulous examination of the selection of single
results and accompanying discussion are presented in Section 4. Sec- classifiers within the stacking classifier is carried out. Through the
tion 5 discusses future research directions. Finally, Section 6 draws the aggregation layer, we also introduce the DOWA operator and explain
conclusions. how it is integrated into the stacking classifier.

2
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

Fig. 1. The statistical characteristics of the continuous features of the dataset, (a) Age, (b) Height, (c) Weight, (d) Systolic blood pressure, (e) Diastolic blood pressure.

3.1. Dataset probability plot of the transformed data, as depicted in Fig. 3. This
process enhances the data’s distributional properties and contributes to
For the ML task in cardiovascular diseases, we utilize a Kaggle improved model performance.
dataset [38] 2 . Each patient is represented by 11 features in this dataset, 𝐴 sinh(𝑋 − 57.3035)
which consists of information from 70,000 patients. In the provided 𝑇 𝐹 = −1.41815 + 2.10448 × . (1)
21.2098
dataset, three types of input features are present including; (i) ‘‘Objec-
tive’’ features which include factual information such as age (in days), 3.3. Proposed CVD system: Architecture, design, and analysis
height (in cm), weight (in kg), and gender (1, 2). (ii) ‘‘Examination’’
features which encompass medical examination results like systolic and
The proposed cardiovascular disease prediction system is described
diastolic blood pressure, cholesterol (1: normal, 2: above normal, 3:
in this section in terms of its architecture, design, and analysis. As a
well above normal), and glucose levels (1: normal, 2: above normal, 3:
first step, we examine the stacking classifier, followed by the dependent
well above normal). And (iii) ‘‘Subjective’’ features involving patient-
ordered weighted averaging operator. The DOWA operator is used to
provided information like smoking (0, 1), alcohol intake (0, 1), and
integrate the aggregation layer into the stacking classifier.
physical activity (0, 1). The dataset’s target variable is the presence or
absence of cardiovascular disease, categorized as binary. All the values
in the dataset are collected during the patients’ medical examinations. 3.3.1. Stacking classifier
Fig. 1 provides statistical information about the continuous features The stacking classifier, a popular ensemble learning technique, com-
in the dataset, including weight, diastolic blood pressure, systolic blood bines the predictions of multi base classifiers to improve prediction
pressure, and others. On the other hand, Fig. 2 displays statistical accuracy and robustness [39]. In our proposed system, we carefully
information about the discrete features such as cholesterol, glucose, select a set of base classifiers with diverse characteristics to capture dif-
physical activity, and others. ferent aspects of the complex relationships present in CVD prediction.
As illustrated in Fig. 4, it operates in two stages including:
3.2. Feature transformation
• Training Stage:
It involves applying mathematical or statistical operations to the – Multi base classifiers, denoted as 𝐶1, 𝐶2, and 𝐶3, are trained
features of a dataset to improve the representation and performance
on the transformed dataset.
of ML models. In our study, we employ the Johnson transformation to
– Each base classifier learns from the input features and cor-
normalize the data and mitigate potential biases introduced by features
responding labels to generate diverse predictions.
with larger values. In the johnson transformation, we apply a function
– The training process allows the base classifiers to capture
to the data to achieve a more symmetric and Gaussian distribution.
different patterns and relationships in the dataset.
Specifically, we apply it to all continuous features.
Fig. 3 provides a visualization of the feature transformation process • Prediction Stage:
specifically for the ‘‘weight’’ feature. By utilizing (1), we achieve a
transformation from the probability plot of the original data to the – After training, the base classifiers are used to make predic-
tions on new, unseen data.
– Each base classifier independently provides a prediction
2
[Link] (𝑃 1, 𝑃 2, 𝑃 3) for the test data based on its learned knowl-
dataset edge.

3
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

Fig. 2. The statistical characteristics of the discrete features of the dataset, along with the target variable, (a) Gender, (b) Cholesterol, (c) Smoking, (d) Alcohol intake, (e) Glucose,
(f) Physical activity, (g) Presence or absence of CVD.

Fig. 3. Feature transformation using the Johnson transformation for the ‘Weight’ feature from the dataset, (a) Probability plot for original data, (b) Select a transformation, (c)
Probability plot for transformed data.

we consider both accuracy and diversity metrics when choosing the

first-level classifiers, namely 𝐶1, 𝐶2, and 𝐶3. The classifier with the
highest correct classification rate on the test data is chosen as the
meta-classifier.

[Link]. First-level classifier selection. Our initial priority is to select

classifiers that are highly accurate individually. According to Table 1,
we can assess each classifier’s performance based on its accuracy on
the test data. We aim to incorporate classifiers that provide distinct
predictions for different samples, however, recognizing the importance
of diversity in ensemble learning. The diversity of perspectives ensures
a broader range of perspectives and enhances the system’s predictive
capability.
The concept of correlation among classifiers is used to evaluate di-
Fig. 4. The stacking classifier employing three first-level classifiers 𝐶1, 𝐶2, 𝐶3 for initial versity. In order to increase the likelihood of obtaining complementary
predictions, and a meta-classifier for the final predictions.
predictions, we select classifiers that do not exhibit high correlation.
With this selection process, the ensemble is able to capture a wider
range of patterns and improve prediction accuracy. During the pre-
– These predictions serve as input to the meta-classifier, diction generation of the first-level classifiers, we employ the K-Fold
which combines and aggregates them to generate the final validation method to mitigate potential data leakage issues. Overfitting
prediction. is prevented by this technique, which ensures robustness.
By leveraging the collective wisdom of the base classifiers, the stacking Based on their exceptional accuracy in predicting cardiovascular
classifier improves prediction accuracy and robustness. With multiple diseases, we carefully select three classifiers, namely decision tree, ran-
classifiers, each with its own strengths and weaknesses, a stacking dom forest, and gradient boosting. These classifiers are chosen for their
classifier can capture a greater range of information and make more ability to capture complex relationships and exhibit strong predictive
accurate predictions. power. Additionally, we assess their diversity by examining the corre-
lation matrix, as depicted in Fig. 5. Classifiers based on decision trees,
3.3.2. Classifier selection random forests, and gradient boosting are correlated visually in the
To achieve optimal prediction performance, first-level classifiers correlation matrix. In the matrix, each cell represents the correlation
and meta-classifiers need to be selected carefully. In our approach, coefficient between two classifiers. A matrix’s color intensity indicates

4
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

Table 1
The accuracy of the individual classifiers on the test
data.
Classifier Accuracy
Logistic Regression 70.20 ± 0.28%
Support Vector Machine 59.33 ± 0.33%
k-Nearest Neighbors 68.46 ± 0.21%
Gradient Boosting 73.75 ± 0.19%
Stochastic Gradient Descent 69.52 ± 0.25%
Decision Tree 72.65 ± 0.24%
Random Forest 73.12 ± 0.18%
Multi-Layer Perceptron 66.42 ± 0.23%
Adaptive Boosting 71.87 ± 0.20%
Hard Voting 70.81 ± 0.31%

Table 2
Performance metrics for selecting the meta-classifier.
Classifier Accuracy AUC
Logistic Regression 92.87 ± 0.19% 95.33 ± 0.13%
Linear SVM 94.05 ± 0.19% 97.14 ± 0.12%
k-Nearest Neighbors 92.66 ± 0.25% 94.63 ± 0.15%
Gradient Boosting 93.94 ± 0.20% 95.22 ± 0.10%
Decision Tree 90.48 ± 0.29% 90.69 ± 0.27% Fig. 5. The correlation between the predictions of the first-level classifiers in the
Random Forest 90.67 ± 0.26% 95.32 ± 0.13% stacking classifier, which serves as a measure of diversity among the classifiers.
Multi-Layer Perceptron 94.03 ± 0.19% 97.14 ± 0.12%
Adaptive Boosting 92.61 ± 0.15% 96.75 ± 0.12%

the level of correlation, with darker colors indicating lower correla-

tion and lighter colors indicating higher correlation. According to this
analysis, the selected classifiers displayed relatively diverse prediction
patterns, making them suitable candidates for ensemble learning.
Currently, researchers are working on determining the optimal num-
ber of classifiers for fusion. A stepwise selection process is used in
our approach. By starting with the two classifiers that achieve the
highest individual accuracy, we progressively add more classifiers. The
process continues as long as adding additional classifiers improves the
overall accuracy of the system. As a result of this iterative approach, we
determine that three classifiers are optimal for our ensemble: decision
trees, random forests, and gradient boosting. Besides exhibiting high
individual accuracy, these classifiers also demonstrate a wide range of
predictions, enhancing the performance of the ensemble.

[Link]. Meta-classifier selection. Various criteria exist for the selection

of the meta-classifier. In this study, we employ two primary criteria,
namely accuracy and AUC (an indicator of model robustness), as de-
picted in Table 2 Among various classifiers, the linear SVM is identified
as the optimal choice based on these criteria. Not only does the linear Fig. 6. SVM classifier operates by finding a hyperplane that minimizes classification
SVM exhibit superior performance, but it also boasts better inter- errors and maximizes the margin between the two classes.

pretability compared to its competitive models. A linear SVM identifies

the optimal hyperplane that effectively separates the different classes
within the input feature space by identifying the optimal hyperplane. where 𝑦𝑖 represents the 𝑖th class and 𝑁 represents the number of
It is defined by a linear decision boundary that maximizes the margin, samples. By employing the linear SVM as the meta-classifier in our
which is the distance between the hyperplane and the nearest data proposed CVD prediction system, we aim to leverage its ability to
points from each class. It is associated with better generalization and effectively classifying and separating different classes based on the
performance of unseen data when there is a larger margin. learned decision boundary.
The training process of a linear SVM involves finding the parameters
that define the hyperplane. This is accomplished by solving a constraint 3.3.3. Aggregation layer
optimization problem, as shown in (2), (3), (4). The objective is to By introducing an aggregation layer, which employs dependent
minimize classification errors while simultaneously maximizing the ordered weighted averaging, we extend the capabilities of the stacking
margin. The optimization process determines the optimal set of weights classifier. By combining the predictions from the first-level stacking
(coefficients) that can best separate the classes while satisfying the classifiers, we can further refine and enhance the overall predictive
margin constraint. Fig. 6 illustrates the operational principles of the ability. In this operator, the soft outputs, which represent the confi-
linear support vector machine classifier. dence of each sample belonging to a specific class, are combined into a
2 single prediction. The functioning of the DOWA operator is described
maximize , (2) in Definition 1 [40].
𝑤,𝑏 ‖𝑤‖
s.t.: (𝑤𝑇 𝑥𝑖 + 𝑏) ≥ 1, for 𝑦𝑖 = 1, 𝑖 = 1, 2, … , 𝑁. (3)
Definition 1. The OWA operator is a mathematical operator that
𝑇
(𝑤 𝑥𝑖 + 𝑏) ≤ −1, for 𝑦𝑖 = −1, 𝑖 = 1, 2, … , 𝑁. (4) combines inputs based on their weights and order. Decision-making and

5
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

aggregation processes commonly use it. The OWA operator aggregates

inputs by arranging them in a specific order and assigning weights
accordingly. Let 𝑃1 , 𝑃2 , … , 𝑃𝑛 be the soft outputs or predictions from
the first-level classifiers for a given sample. These predictions represent
the probabilities of the sample belonging to each class. and let 𝜇
be the average value of these predictions for each class, i.e., 𝜇 =
1
∑𝑛

𝑛
𝑃𝑗 , (𝛽(1), 𝛽(2), … , 𝛽(𝑛)) is a permutation of (1, 2, … , 𝑛) such that
𝑗=1
𝑃𝛽(𝑗−1) ≥ 𝑃𝛽(𝑗) for all 𝑗 = 2, … , 𝑛, then we call
|𝑃𝛽(𝑗) − 𝜇|
𝑠(𝑃𝛽(𝑗) , 𝜇) = 1 − ∑𝑛 , 𝑗 = 1, 2, … , 𝑛, (5)
𝑗=1 |𝑃𝑗 − 𝜇|
Fig. 7. The stacking classifier after incorporating the aggregation layer into its
the similarity degree of the 𝑗th largest argument 𝑃𝛽(𝑗) and the average structure.
value 𝜇.
The DOWA operator computes the weighted average of the pre-
dictions based on a weight vector where each weight represents the
importance or significance of the corresponding prediction. Let the
𝑤 = (𝑤1 , 𝑤2 , … , 𝑤𝑛 )𝑇 be the weight vector of the OWA operator, then
we define the following:
𝑠(𝑃𝛽(𝑗) , 𝜇)
𝑤𝑗 = ∑ 𝑛 , 𝑗 = 1, 2, … , 𝑛, (6)
𝑗=1 𝑠(𝑃𝛽(𝑗) , 𝜇)
where 𝑠(𝑃𝛽(𝑗) , 𝜇) is defined by (5). Clearly, we have 𝑤𝑗 ∈ [0, 1] and
∑𝑛
𝑗=1 𝑤𝑗 = 1. Since

∑
𝑛 ∑
𝑛
𝑠(𝑃𝛽(𝑗) , 𝜇) = 𝑠(𝑃𝑗 , 𝜇), (7)
𝑗=1 𝑗=1

then (6) can be rewritten as

𝑠(𝑃𝛽(𝑗) , 𝜇)
𝑤𝑗 = ∑𝑛 , 𝑗 = 1, 2, … , 𝑛. (8)
𝑗=1 𝑠(𝑃𝑗 , 𝜇)
In this case, we have
∑𝑛
𝑗=1 𝑠(𝑃𝛽(𝑗) , 𝜇)𝑃𝛽(𝑗)
𝑂𝑊 𝐴(𝑃1 , 𝑃2 , … , 𝑃𝑛 ) = ∑𝑛 . (9)
𝑗=1 𝑠(𝑃𝑗 , 𝜇) Fig. 8. Decision boundary determined by the linear support vector machine classifier
for the two classes.
According to (4), we can write
∑𝑛
𝑗=1 𝑠(𝑃𝑗 , 𝜇)𝑏𝑗
𝑂𝑊 𝐴(𝑃1 , 𝑃2 , … , 𝑃𝑛 ) = ∑𝑛 . (10)
𝑗=1 𝑠(𝑃𝑗 , 𝜇)
4. Experiment results and discussion

Depending on the characteristics of the dataset and the first-level In this section, we expound upon the outcomes and introduce
classifier, the weights assigned to each prediction can be adjusted. various performance metrics used to appraise the algorithm’s efficacy,
Weights can be optimized to maximize prediction accuracy or align including accuracy, sensitivity, precision, F1-Score, specificity, and
with domain-specific requirements. A meta-classifier makes the final AUC.
prediction based on the aggregated predictions from the aggregation
layer. To predict the target variable, the meta-classifier uses its own 4.1. Performance metrics
learning algorithm to combine the aggregated predictions.
By utilizing the DOWA operator in aggregation, our proposed CVD 4.1.1. Precision
prediction system offers several advantages. Having the ability to assign Precision measures the proportion of true positive predictions
higher weights to predictions that are closer to the average prediction
among all positive predictions made by a model. It focuses on the
of all classifiers is a key advantage. Outlier classifiers that produce sig-
accuracy of positive predictions, e.g.: Precision = 𝑇 𝑃𝑇+𝐹
𝑃
𝑃
.
nificantly different predictions from the majority are mitigated by this
feature. The DOWA operator ensures a more balanced and robust final
4.1.2. Specificity
prediction by assigning higher weights to classifiers that exhibit closer
Specificity evaluates the proportion of true negative predictions
alignment with the collective knowledge of all classifiers. A consensus-
among all negative predictions made by a model. It emphasizes the
based approach reduces the potential bias introduced by individual 𝑇𝑁
classifiers by considering the collective wisdom and agreement among accuracy of negative predictions, e.g., Specificity = 𝑇 𝑁+𝐹 𝑃
.
the base classifiers.
To calculate the weights for each classifier, we employ (8), which 4.1.3. Accuracy
considers the predictions made by the first-level classifiers in the stack- Accuracy gauges the overall correctness of a model’s predictions by
ing classifier. With the weights determined, we aggregate the predic- considering both true positives and true negatives relative to the total
𝑃 +𝑇 𝑁
tions of all the first-level classifiers for each class using (10). This number of predictions, e.g., Accuracy = 𝑇 𝑃 +𝑇𝑇𝑁+𝐹 𝑃 +𝐹 𝑁
.
aggregation process, facilitated by the DOWA operator, combines the
individual predictions into a unified prediction that encapsulates the 4.1.4. Recall
collective knowledge and expertise of the ensemble. After adding the ‘‘Recall’’, also known as ‘‘sensitivity’’ or ‘‘true positive rate’’, mea-
aggregation layer to the stacking classifier, the resulting classifier is sures the proportion of actual positive instances that were correctly
shown in Fig. 7. The new classifier demonstrates improved and more identified by a model. It focuses on the model’s ability to capture all
reliable performance. positive instances, e.g., Recall = 𝑇 𝑃𝑇+𝐹
𝑃
𝑁
.

6
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

Fig. 10. ROC curve illustrating the classification performance of the proposed model.
The area under the curve (AUC) is 97.14%, indicating excellent discriminative power
and predictive accuracy.

Fig. 9. The confusion matrix for the proposed stack-based ensemble classifier, which
achieved an accuracy of 94.05% on the test data. 5. Future research directions

Table 3 There are several challenges associated with the newly proposed
Performance metrics of the proposed method. model that future research can address.
Metric Value
Accuracy 94.05 ± 0.19%
• As mentioned, in selecting the optimal number of first-level classi-
Precision 94.54 ± 0.15% fiers, we employed the stepwise selection method. The first chal-
Recall 93.50 ± 0.17% lenge could involve exploring alternative approaches for select-
Specificity 94.60 ± 0.12% ing the optimal number of first-level classifiers, such as random
F1-Score 94.02 ± 0.14%
search, grid search, and others.
AUC 97.14 ± 0.12%
• Another challenge could involve exploring the use of alternative
attention-based approaches or leveraging other extensions of the
ordered weighted averaging (OWA) operator to implement the
4.1.5. F1-Score aggregation layer.
F1-score is a balanced metric that considers both precision and • The final challenge could involve exploring a deep stacking
recall. It provides a single value that balances the trade-off between model. This entails incorporating multiple layers of prediction
precision and recall, making it useful when dealing with imbalanced and aggregation within the stacking classifier structure, poten-
datasets, e.g., F1-Score = 2×Precision×Recall , where 𝑇 𝑃 is true positive, tially leading to a more sophisticated model.
Precision+Recall
𝐹 𝑃 is false positive, 𝑇 𝑁 is true negative and 𝐹 𝑁 is false negative. • Given that a historical dataset was used in this research, it is
recommended that a cohort study be designed to test the stated
After incorporating the aggregation layer into the stacking classi-
methodology on a longitudinal dataset and to evaluate its perfor-
fier using the DOWA operator and selecting the first-level classifiers
mance.
and meta-classifier, we evaluate the performance of the meta-classifier
on the test data using the K-Fold validation method with 𝐾 = 15. 6. Conclusion
The linear support vector machine classifier is selected as the meta-
classifier to distinguish between the two classes, as shown in Fig. 8. The rapid and timely detection of cardiovascular diseases is of
The meta-classifier achieves an accuracy of 94.05% on the test data. special importance. Therefore, in this study, by adding an aggregation
The confusion matrix of the meta-classifier on the test data is layer to the structure of the stacking classifier, a new classification
presented in Fig. 9, indicating that 5.4% of samples from class ‘0’ and method was devised. In the proposed classifier, the aggregation layer
6.5% of samples from class ‘1’ are misclassified. The receiver operating was implemented using the DOWA operator, and the first-level clas-
characteristic curve, which illustrates the relationship between the sifiers and meta-classifier were carefully selected based on the two
true positive rate (TPR) and the false positive rate (FPR) at various criteria of accuracy and diversity. Additionally, raw data from poten-
classification thresholds, is depicted in Fig. 10. The ROC curves of both tial CVD patients, after appropriate preprocessing and using feature
classes exhibit a considerable overlap, indicating a similar performance transformation techniques, were utilized to feed the proposed classifier.
between the two classes. The area under the ROC curve (AUC) serves as The proposed classifier demonstrated a significant increase in accuracy
and AUC in the detection of CVD. Furthermore, as it is based on the
a metric for evaluating the classifier’s performance, with a higher AUC
aggregation of opinions from multiple base classifiers, the proposed
value, closer to 1, indicating better classifier performance. The pro-
classifier is more reliable.
posed system exhibits an AUC of 97.14%. Other performance metrics
of the proposed system are depicted in Table 3. CRediT authorship contribution statement
Based on the results, the linear SVM meta-classifier achieves high
accuracy on the test data and exhibits balanced performance in classify- Mehdi Hosseini Chagahi: Writing – review & editing, Writing –
ing both classes. ROC curve overlap and low misclassification rates indi- original draft, Visualization, Validation, Software, Resources, Method-
cate the effectiveness of the meta-classifier in predicting cardiovascular ology, Investigation, Funding acquisition, Formal analysis, Data cu-
diseases. ration, Conceptualization. Saeed Mohammadi Dashtaki: Writing –

7
M. Hosseini Chagahi et al. Computers in Biology and Medicine 173 (2024) 108345

original draft, Software, Resources, Methodology, Project administra- [18] S. Hossain, A. Chakrabarty, T.R. Gadekallu, M. Alazab, M.J. Piran, Vision
tion, Funding acquisition, Data curation, Conceptualization. Behzad transformers, ensemble model, and transfer learning leveraging explainable ai
for brain tumor detection and classification, IEEE J. Biomed. Health Inf. (2023).
Moshiri: Writing – review & editing, Writing – original draft, Su-
[19] A. Abdellatif, H. Abdellatef, J. Kanesan, C.-O. Chow, J.H. Chuah, H.M. Gheni,
pervision, Project administration, Methodology, Investigation, Formal Improving the heart disease detection and patients’ survival using supervised
analysis, Data curation, Conceptualization. M.d. Jalil Piran: Writing infinite feature selection and improved weighted random forest, IEEE Access 10
– review & editing, Writing – original draft, Visualization, Validation, (2022) 67363–67372.
Supervision, Resources, Project administration, Investigation, Funding [20] F. Khozeimeh, D. Sharifrazi, N.H. Izadi, J.H. Joloudari, A. Shoeibi, R. Al-
izadehsani, M. Tartibi, S. Hussain, Z.A. Sani, M. Khodatars, et al., RF-CNN-F:
acquisition. Random forest with convolutional neural network features for coronary artery
disease diagnosis based on cardiac magnetic resonance, Sci. Rep. 12 (1) (2022)
Declaration of competing interest 11178.
[21] R. Rajmohan, M. Pavithra, T.A. Kumar, P. Manjubala, Exploration of deep RNN
architectures: LSTM and gru in medical diagnostics of cardiovascular and neuro
The authors declare that they have no known competing finan-
diseases, in: Handbook of Deep Learning in Biomedical Engineering and Health
cial interests or personal relationships that could have appeared to Informatics, Apple Academic Press, 2021, pp. 167–202.
influence the work reported in this paper. [22] D. Xu, J.Q. Sheng, P.J.-H. Hu, T.-S. Huang, C.-C. Hsu, A deep learning–based
unsupervised method to impute missing values in patient records for improved
References management of cardiovascular patients, IEEE J. Biomed. Health Inf. 25 (6) (2020)
2260–2272.
[23] S. Doran, M. Arif, S. Lam, A. Bayraktar, H. Turkez, M. Uhlen, J. Boren,
[1] P. Greenland, D.M. Lloyd-Jones, Role of coronary artery calcium testing for risk
A. Mardinoglu, Multi-omics approaches for revealing the complexity of
assessment in primary prevention of atherosclerotic cardiovascular disease: A
cardiovascular disease, Brief. Bioinform. 22 (5) (2021) bbab061.
review, JAMA Cardiol. (2022).
[24] R.-S. Wang, B.A. Maron, J. Loscalzo, Multiomics network medicine approaches
[2] World Health Organization. Cardiovascular diseases (CVDs) [Available from:
to precision medicine and therapeutics in cardiovascular diseases, Arterioscler.
[Link]
Thromb. Vasc. Biol. 43 (4) (2023) 493–503.
(cvds) -11 June 2021.] Access date: 29 September 2023.
[25] P. Guleria, P. Naga Srinivasu, S. Ahmed, N. Almusallam, F.K. Alarfaj, XAI
[3] S. Islam, F.A. Nobel, S. Sabrina, A. Islam, M.J. Islam, Assessment and comparison
framework for cardiovascular disease prediction using classification techniques,
of cardiovascular disease risk factors and biochemical parameters among men
Electronics 11 (24) (2022) 4086.
and women: A cross-sectional study, J. Adv. Biotechnol. Exp. Ther. 6 (1) (2023)
[26] A. Brankovic, D. Cook, J. Rahman, W. Huang, S. Khanna, Evaluation of popular
25–34.
XAI applied to clinical prediction models: Can they be trusted? 2023, arXiv
[4] D. Lovic, V. Stojanov, B. Jakovljevic, M. Krotin, V. Jurisic, D. Djordjevic, K.
preprint arXiv:2306.11985.
Paunovic, M. Zdravkovic, D. Simonovic, D. Bastac, et al., Prevalence of arterial
[27] H. Tada, N. Fujino, A. Nomura, C. Nakanishi, K. Hayashi, M. Takamura, M.-a.
hypertension in Serbia: PAHIS study, J. Hypertens. 31 (11) (2013) 2151–2157.
Kawashiri, Personalized medicine for cardiovascular diseases, J. Hum. Genet. 66
[5] M. Marinković, N. Ilić, D. Djokić, V. Andrejevic, G. Damjanović, G. Samardžić,
(1) (2021) 67–74.
S. Tufegdžić, M. Vučic-Janković, Prevalence of hypertension in adults in the
[28] S. Patil, S. Bhosale, A novel feature selection-based algorithm for medical
Šumadija district, Serbia–a cross-sectional study, Vojnosanit. Pregl. 71 (3) (2014).
correlation of high dimensional data, in: 2023 IEEE IAS Global Conference on
[6] B. Che, C. Zhong, R. Zhang, L. Pu, T. Zhao, Y. Zhang, L. Han, Triglyceride-glucose
Emerging Technologies, GlobConET, IEEE, 2023, pp. 1–7.
index and triglyceride to high-density lipoprotein cholesterol ratio as potential
[29] J. Kanani, S. Patel, B. Parmar, M. Vaghasia, K. Patel, Big data predictive analytics
cardiovascular disease risk factors: An analysis of UK biobank data, Cardiovasc.
model for cardiovascular risk detection using machine learning techniques,
Diabetol. 22 (1) (2023) 1–11.
in: 2022 IEEE 2nd International Symposium on Sustainable Energy, Signal
[7] A. Kachhawa, J. Hitt, An intelligent system for early prediction of cardiovascular
Processing and Cyber Security, ISSSC, IEEE, 2022, pp. 1–6.
disease using machine learning, J. Stud. Res. 11 (3) (2022).
[30] J. Lewey, Challenges and opportunities to improving research in maternal
[8] B. Rahman, B.S. Sabarguna, H.S. Warnars, W. Budiharto, Early Detection of
cardiovascular health, Nat. Cardiovasc. Res. (2023) 1–2.
Heart Disease Based on Medical Check-Up Datasets Using Multilayer Perceptron
[31] G.A. Roth, G.A. Mensah, C.O. Johnson, G. Addolorato, E. Ammirati, L.M.
Classifier, 2023.
Baddour, N.C. Barengo, A.Z. Beaton, E.J. Benjamin, C.P. Benziger, et al., Global
[9] M.J. Oude Wolcherink, C.M. Behr, X.G. Pouwels, C.J. Doggen, H. Koffijberg,
burden of cardiovascular diseases and risk factors, 1990–2019: Update from the
Health economic research assessing the value of early detection of cardiovascular
GBD 2019 study, J. Am. Coll. Cardiol. 76 (25) (2020) 2982–3021.
disease: A systematic review, PharmacoEconomics (2023) 1–21.
[32] A. Tiwari, A. Chugh, A. Sharma, Ensemble framework for cardiovascular disease
[10] Y. Jiang, X. Zhang, R. Ma, X. Wang, J. Liu, M. Keerman, Y. Yan, J. Ma, Y. Song, J.
prediction, Comput. Biol. Med. 146 (2022) 105624.
Zhang, et al., Cardiovascular disease prediction by machine learning algorithms
[33] Y. Muhammad, M. Tahir, M. Hayat, K.T. Chong, Early and accurate detection
based on cytokines in kazakhs of China, Clin. Epidemiol. (2021) 417–428.
and diagnosis of heart disease using intelligent computational model, Sci. Rep.
[11] S. Ghorashi, K. Rehman, A. Riaz, H.K. Alkahtani, A.H. Samak, I. Cherrez-Ojeda,
10 (1) (2020) 19747.
A. Parveen, Leveraging regression analysis to predict overlapping symptoms of
[34] R. Atallah, A. Al-Mousa, Heart disease detection using machine learning majority
cardiovascular diseases, IEEE Access (2023).
voting ensemble method, in: 2019 2nd International Conference on New Trends
[12] G. Muhammad, S. Naveed, L. Nadeem, T. Mahmood, A.R. Khan, Y. Amin, S.A.O.
in Computing Sciences, Ictcs, IEEE, 2019, pp. 1–6.
Bahaj, Enhancing prognosis accuracy for ischemic cardiovascular disease using
[35] K. Raza, Improving the prediction accuracy of heart disease with ensemble
k nearest neighbor algorithm: A robust approach, IEEE Access (2023).
learning and majority voting rule, in: U-Healthcare Monitoring Systems, Elsevier,
[13] V.S.K. Reddy, P. Meghana, N.S. Reddy, B.A. Rao, Prediction on cardiovascular
2019, pp. 179–196.
disease using decision tree and Naïve Bayes classifiers, in: J. Phys. Conf. Ser.,
[36] A. Alqahtani, S. Alsubai, M. Sha, L. Vilcekova, T. Javed, Cardiovascular disease
2161, (1) IOP Publishing, 2022, 012015.
detection using ensemble learning, Comput. Intell. Neurosci. 2022 (2022).
[14] M.M. Saim, H. Ammor, Comparative study of machine learning algorithms (SVM,
[37] J. Liu, X. Dong, H. Zhao, Y. Tian, Predictive classifier for cardiovascular disease
logistic regression and KNN) to predict cardiovascular diseases, in: E3S Web of
based on stacking model fusion, Processes 10 (4) (2022) 749.
Conferences, Vol. 351, EDP Sciences, 2022, p. 01037.
[38] Cardiovascular disease dataset, retrieved from Kaggle repository on 18.01.2021,
[15] R. Kapila, T. Ragunathan, S. Saleti, T.J. Lakshmi, M.W. Ahmad, Heart disease
[Link]
prediction using novel quine McCluskey binary classifier (QMBC), IEEE Access
[39] Z. Rustamov, J. Rustamov, M.S. Sultana, J. Ywei, V. Balakrishnan, N. Zaki,
(2023).
Cardiovascular disease prediction using ensemble learning techniques: A stacking
[16] P. Rubini, C. Subasini, A.V. Katharine, V. Kumaresan, S.G. Kumar, T. Nithya, A
approach, in: 2023 19th IEEE International Colloquium on Signal Processing &
cardiovascular disease prediction using machine learning algorithms, Ann. Rom.
Its Applications, CSPA, IEEE, 2023, pp. 93–98.
Soc. Cell Biol. (2021) 904–912.
[40] W. Zeng, D. Li, Y. Gu, Monotonic argument-dependent OWA operators, Int. J.
[17] P. Theerthagiri, J. Vidya, Cardiovascular disease prediction using recursive
Intell. Syst. 33 (8) (2018) 1639–1659.
feature elimination and gradient boosting classification techniques, Expert Syst.
39 (9) (2022) e13064.