Journal of Biomedical Informatics 111 (2020) 103591

Contents lists available at ScienceDirect

Journal of Biomedical Informatics

journal homepage: [Link]/locate/yjbin

Detection and classification of breast cancer using logistic regression

feature selection and GMDH classifier
Ziba Khandezamin a, Marjan Naderan b, *, Mohammad Javad Rashti c
a
[Link]. of Artificial Intelligence, Department of Computer Engineering, Faculty of Engineering, Shahid Chamran University of Ahvaz, Ahwvz, Iran
b
Associate Professor, Department of Computer Engineering, Faculty of Engineering, Shahid Chamran University of Ahvaz, Ahvaz, Iran
c
Assistant Professor, Department of Computer Engineering, Faculty of Engineering, Shahid Chamran University of Ahvaz, Ahvaz, Iran

A R T I C L E I N F O A B S T R A C T

Keywords: Breast cancer is the most common cancer among women such that the existence of a precise and reliable system
Breast cancer for the diagnosis of benign or malignant tumors is critical. Nowadays, using the results of Fine Needle Aspiration
Feature selection (FNA) cytology and machine learning techniques, detection and early diagnosis of this cancer can be done with
Logistic regression
greater accuracy. In this paper, we propose a method consisting of two steps: in the first step, to eliminate the less
Machine learning
Group method data handling
important features, logistic regression has been used. In the second step, the Group Method Data Handling
(GMDH) neural network is used for the diagnosis of benign and malignant samples. To evaluate the performance
of the proposed method, three datasets WBCD, WDBC and WPBC are investigated with metrics: precision, the
Area Under the ROC (AUC), true positive rate, false positive rate, accuracy and F-criteria. Simulation results
show that the proposed method reaches a precision of 99.4% for WBCD, 99.6% for WDBC and a precision of
96.9% for WPBC dataset.

1. Introduction may lead to different interpretations since different radiologists have

different inferences for a single mammogram [3]. The accuracy of
Breast cancer is the most common type of cancer among women mammography is between 68 and 79 percent. When mammography
which is highly heterogeneous, and caused by the interplay of hereditary detects a tumor, biopsy is done to measure its malignancy. The accuracy
and environmental risk factors [1]. It is caused by irregular growth of of surgical biopsy is near to 100 percent, but it is costly, aggressive, time-
cells in the breast tissue, which leads to the creation of tumors that could consuming and painful.
be benign (non-cancerous) or malignant (cancerous) [2]. Nearly, 25% of Among the diagnostic methods described above, FNA is extensively
deaths due to breast cancer are in the women aged from 40 to 49 years. used for breast cancer detection, since it is outpatient, low-cost, easy and
This type of cancer is rarely seen before the age of 25–30, but it has been fast [4]. The accuracy of FNA with visual interpretation can vary from
reported even in adolescent women. Based on the statistics published by 35 to 95 percent depending of the experience of the doctor [3]. In this
the World Health Organization, one in 8 to 10 women suffers from breast method, the liquid extracted from the breast tissue is put under micro­
cancer. In Iran, one in 10 to 15 women is affected with this type of scope to study its cytological properties. After examining cytology re­
cancer. Researchers argue that the high mortality rate of breast cancer is sults, the malignance of the tumor can be identified [2]. When the doctor
due to its late diagnosis [1]. is doubtful and cannot determine whether the tumor is benign or ma­
Three types of breast cancer diagnosing has been developed so far, lignant, data mining and machine learning techniques can play an
as: physical examination, mammography and biopsy. Biopsy can be important role in diagnosis. In fact, using a computer based system to
done by using four methods: Fine Needle Aspiration (FNA), core needle diagnose breast cancer in early stages can lead to longevity after diag­
biopsy, surgical biopsy or lymph node biopsy. On the other hand, nosis of cancer. Hence, researches on finding more accurate methods for
mammography is one of the common methods used to diagnose breast diagnosis of breast cancer have being conducted during the previous
cancer, but the results should be interpreted by expert radiologists. This decade.

* Correspondence to: Marjan Naderan, Associate Professor of Department of Computer Engineering, Faculty of Engineering, Shahid Chamran University of Ahvaz,
Ahvaz, Iran.
E-mail addresses: [Link]@[Link], z-khandezamin@[Link] (Z. Khandezamin), [Link]@[Link] (M. Naderan), [Link]@scu.
[Link] (M.J. Rashti).

[Link]
Received 11 February 2020; Received in revised form 1 October 2020; Accepted 4 October 2020
Available online 8 October 2020
1532-0464/© 2020 Elsevier Inc. This article is made available under the Elsevier license ([Link]
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Computer-based breast cancer methods usually classify patients to 2.1. Feature selection methods in prediction of breast cancer
the benign class (which do not have cancer) and the malignant class
(which have cancer). There have been numerous intelligent methods to In [4], a combination of genetic algorithm and particle swarm
classify breast cancer data [2–27], which some of them include feature optimization is incorporated. To this end, chromosomes are defined as
selection methods, and some perform classification without feature se­ masks for features. In other words, each chromosome is a set of features.
lection. For example, Naïve Bayes (NB) and Weighted Naïve Bayes The size of chromosome (the number of genes) is equal to the number of
classifiers have been incorporated in a number of studies to identify the features obtained from a person. Each chromosome is represented in a
type of tumor, such that in some of them a feature selection method is binary form, in 0 or 1. A 1 means the related feature is selected, and
used before classification, while in others feature selection has not been 0 means that it is not selected. The genetic algorithm chooses a subset of
performed. Common classifiers such as Support Vector Machine (SVM), features as a chromosome, and each chromosome is sent to the particle
Random Forest (RF), and Decision Tree (DT) have been also incorpo­ swarm optimization based classifier to calculate its fitness value. The
rated to detect breast cancer. In one research, methods such as Principal genetic algorithm uses this fitness value for evolution of chromosomes.
Component Analysis (PCA) and Relief has been used for feature selec­ In the end, the genetic algorithm finds an optimal subset of features. The
tion. Gaussian Bayes is also used alongside Linear Discriminant Analysis number of features are reduced from 9 to 4 in the WBCD dataset, from 30
(LDA) for classification. In another research, using the same feature to 14 in the WDBC dataset, and from 33 to 16 in the WPBC dataset. After
selection method, the Mamdani fuzzy inference system has been used to feature selection, three methods, namely a neural network, a particle
classify data. Neural networks are also used in previous works to predict swam optimization and a genetic algorithm are used to classify data. The
breast cancer. In one of these works, before classification, features are accuracy result for WPBC is 79.2 which is relatively low, and this is one
selected by means of a Genetic Algorithm (GA). Evolutionary algorithms of the shortcomings of this paper.
such as GA and Particle Swarm Optimization (PSO) are incorporated in In [5] and [6], LDA is incorporated for feature selection. The
the literature which have been used as classifiers and feature selection methods of these studies are similar. Their feature selection method is as
methods. However, researchers are trying to find better solutions to help follows: features are grouped into subsets with four elements, and for
physicians. each subset, the accuracy of LDA is calculated. Then, the subset with the
In this paper, to eliminate ineffective features, the logistic regression highest accuracy is selected as the best feature set, and used for classi­
algorithm is used which to the best of our knowledge, has not been used fication. In both studies, the WBCD dataset is used. However, it is
in any of the previous studies. Using this algorithm, which is linear, and obvious that a brute-force approach for feature selection and creation of
cost-effective, feature selection is performed effectively. In fact, the all of the possible subsets is time-consuming. After finding the best
features are weighted and using these weights, irrelevant features are subset of features, a Gaussian naïve bayes classifier is incorporated for
eliminated. In the next phase, a novel classification algorithm is applied, classification, and its accuracy is 96.6 percent. In [5], there are no dis­
which is based on deep neural networks, namely Group Method Data cussions and analyses on the results. However, in [6], from nine features,
Handling or GMDH. Finally, these methods are applied on three datasets just four of them are selected for classification, and after feature selec­
for evaluation. tion, the Mamdani fuzzy inference system is used for classification which
Concisely, the contributions of this paper are as follows: has resulted in an accuracy of 93.6 percent.
Fuzzy logic has been also incorporated in [7] and [8] for cancer
• Using three different datasets to evaluate the proposed methods prediction; in [7], the feature values are fuzzificated.
• Reaching a high level of accuracy for all three datasets In [9], both feature selection and feature extraction are considered.
• Using a new feature selection method based on logistic regression to For feature selection, the Pearson Correlation Coefficient (PCC) method
reduce computational cost is applied on the WDBC dataset which consists of 30 features. After
• Using a novel deep neural network, which can be tuned to increase feature selection with PCC, the binning technique is used for pre­
accuracy or decrease computational cost. processing of data. The results with 5 and 19 selected features are
• Using a comprehensive set of evaluation metrics, including false compared to three classifiers. The highest accuracy is yielded when 19
positive rate features are fed into support vector machines, which is 98.51 percent.
• A deep investigation of previous works and a qualitative comparison Also, using five features and a Bayesian classifier results in an accuracy
among them of 97.39. The advantage of this method is that it has reduced the number
of features significantly, and thus has reduced the computational load.
The rest of this paper is as follows: in section 2, previous works are Furthermore, in this research, PCA is considered for extracting features.
assessed; in section 3, the proposed method is explained in detail; in PCA maps the data into a space with lower dimensions. After applying
section 4, results are reported and compared with previous works; PCA, the features of the WDBC dataset are reduced to five features.
finally, in section 5, conclusions and suggestions for future works are However, as said before, in this method no feature selection is incor­
presented. porated, but the dimensionality of the data is reduced.
In [10], for feature selection, PCA, Information Gain (IG), and Relief
2. Previous works are applied. After feature selection, four classifiers, Naïve Bayes,
Bayesian network, Random Forest and J48 Decision Tree are used to
In this section, previous works in the field of feature selection and classify data of the WBCD, WDBC and WPBC datasets. However, the
classification on breast cancer data are reviewed. Some of these works number of selected features for these data is not reported. This method
focused on feature selection, some focused on the classification method has performed better on the WBCD dataset than the other two datasets.
and a number of them have contributions in both phases. In the Likewise, in [11] the PCA method is used for feature selection. Three
following, first the feature selection methods, and then the classification classifiers are used, of which the J48 decision tree has yielded the best
methods are explained, and finally, a qualitative comparison of them is results on the WBCD dataset. In [12], genetic algorithm is used for
shown in a table. feature selection and for fitness function, the accuracy of the model
using rotation forest is calculated. Likewise, in [13], genetic algorithm is

2
 Z. Khandezamin et al.
Table 1
Qualitative comparison of previous works on breast cancer detection and classification.
Paper Publication Classification Method Feature Selection Dataset Weaknesses/Strengths
Year

[3] 2015 Weighted Bayesian – WBCD Using grid search to find optimum weights, and high computational cost for this search, the
need to initialize this weight vector
Overcoming the NB limitations, including assignment of labels to training data.
[4] 2016 GA, PSO, ANN GA + PSO WBCD, WDBC, Using a combination of GA and PSO to find effective features and to reduce the number of
WPBC features.
[5] 2016 Gaussian Bayes LDA WBCD Not applying the proposed method on other datasets, such as WDBC.
Using feature selection, feature reduction, and reducing the time needed for prediction.
[6] 2015 The Mamdani fuzzy inference system LDA WBCD Low true negative rate, low accuracy.
Using a small portion of the whole dataset.
Low time complexity and feature reduction.
[7] 2012 PDM-FNA-Fuzzy Fuzzy logic WDBC Low detection rate for benign samples compared to the high fraction of malignant samples.
Better detection of malignant samples compared to non-automatic methods.
[8] 2017 CART with logical fuzzy Fuzzy logic WBCD Using the EM method for clustering and easing the clustering process.
Using PCA method to dominate the multi-linear property of the problem.
[9] 2016 SVM and Bayesian PCA and PCC WDBC Low detection rate compared to the proposed method. Using the PCA method for
dimensionality reduction and feature selection.
[10] 2016 NB, Bayesian network, RF, J48 DT Information Gain (IG), PCA, WBCD, WDBC, Not being able to find the best algorithm for feature selection.
and Relief WPBC Dependency of performance of algorithms to type and features of data.
Using feature selection and combining the Bayesian classifier and the J48 DT to enhance
accuracy.
[12] 2015 Rotation forest, SVM, RF GA WBCD Using of rotation forest with the SVM base. Enhancing the accuracy with this classifier. Using
GA to select more effective features.
[13] 2015 ANN (Resilient back propagation) GA WBCD Using the ANN to optimize the parameters of the genetic algorithm.
[14] 2014 SVM Genetic-SVM WBCD Reducing unnecessary features and optimization of SVM parameters using GA.
3

[15] 2016 SVM PSO WDBC Using the PSO method for feature selection.
[16] 2015 k-NN, neural network, neural network based on RBF, SVM ICA WDBC Reducing the dimensionality of features with the ICA method to one dimension.
Reducing the computational complexity.
[17] 2018 SVM, KNN, LR PCC Feature selection using the PCC method.
[18] 2019 Normalized Neural Networks – WBCD Normalizing the samples before training using the neural network which reduces the training
time.
[19] 2016 SVM, Naïve bayes, C4.5, k-NN – WBCD Low true positive rate for k-NN.
Slow learning rate for k-NN and low performance of k-NN.
Testing different classifiers and comparing results.
[20] 2016 SVM, Random Forest, Bayesian Network – WBCD Testing and comparing the results between three methods: SVM, random forest and Bayesian
network.
[21] 2018 Two semi-supervised fuzzy methods in the first phase and – WBCD Using two fuzzy clustering methods, FCM and GK, to obtain feature weights and using them to
SVM in the second phase classify with SVM.

Journal of Biomedical Informatics 111 (2020) 103591

[22] 2017 Neural network – WBCD Increase in errors when the training iterations are increased.
Testing with multiple activation functions and different number of neurons and layers to find
the optimal solution which is time-consuming
[23] 2015 GONN – WBCD Optimizing the parameters of the neural network with GA and using it for classification.
[24] 2016 AIS – WBCD Using a semi-supervised method which reduces the execution time.
WDBC
[25] 2013 J48, MLP, NB, SMO, IBK PCA WBCD, WDBC, Using feature selection and dimensionality reduction.
WPBC Combining several classifiers to increase the classification accuracy.
[26] 2019 K-NN, SVM, LR, C4.5, RF PCA + SMOTE WBCD, Using combination of several classifiers to increase the classification accuracy.
WDBC, Using two PCA and SMOTE methods to reduce the dimensionality of features.
WPBC
[27] 2018 GRNN – WDBC, Improving the GRNN method with changing the Euclidean distance to absolute distance.
WPBC
This 2019 GMDH LR WBCD, WDBC, Using feature selection for ignoring ineffective features.
paper WPBC Applying on three datasets
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Fig. 1. Schematic diagram of the proposed method.

incorporated to select features. are used, one of which is FCM. In the second phase, SVM is incorporated
In [14], for feature selection, a combination of genetic algorithm and which is finally applied to WBCD dataset.
Support Vector Machine (SVM) is incorporated. The fitness function of In [12], the random forest method is applied on WBCD and WDBC
the genetic algorithm is the accuracy of the model using SVM. In this datasets; without feature selection, the accuracy on these datasets are
paper, the WBCD dataset is used and again, the number of selected 96.3% and 96.1%, respectively. For results after feature selection, only
features is not reported. The accuracy after feature selection is improved the result for the WDBC classifier is reported, which is 95.4% percent.
from 96.5 percent to 97 percent. Hence, after feature selection, the accuracy has been affected negatively.
In [15], similar to [4], PSO is used for feature selection. In [16], the Another classification method used in [12], is the rotation forest algo­
authors have used the independent component analysis (ICA) method rithm. As the basis, a support vector machine with a polynomial kernel is
for feature selection, and have applied it on the WBCD dataset. In [17], incorporated. The distinct feature set is converted to distinct rotations.
another method is used for feature selection which selects the best set of Consequently, individual classifiers are created with high performance.
features on the WBCD dataset. Another method used in [12] for classifying benign and malignant in­
stances is the C4.5 algorithm. This classifier has been applied on the
WBCD and WDBC datasets; without feature selection, the accuracy
2.2. Classification methods in prediction of breast cancer values are 96% and 93.2%, respectively. After feature selection, the
accuracy on WDBC is increased to 94%.
Bayesian classifier has been incorporated in various forms to detect Neural networks are another type of classifiers used to predict breast
breast cancer. For example, in [3], a weighted Bayesian classifier is cancer. In [4], a neural network is incorporated to classify WBCD, WDBC
proposed for classification, in which all of the features of the dataset are and WPBC datasets. In [12], a multilayer perceptron classifies the data
fed into a weighted Bayesian classifier. The weights of the classifier are in the WBCD and WDBC datasets. In [22], a feed-forward neural network
tuned in the training phase beforehand. The suggested weighting is used for classification and the number of neurons in the hidden layer
method is applied on the input features, and the weighted Bayesian are 20 to 24 neurons. Classification is done for one, two and three hidden
classifier classifies the data. To obtain optimum weights, grid search layers and mean of squared error is calculated for three activation
mechanism is used and the accuracy of this method has reached to 98.54 functions: Tansig, LogSig and Pureline. In [23], after feature selection
percent on WBCD. In [5], to classify breast cancer instances, a Gaussian with genetic algorithm, a neural network is used for classification, in
Naïve Bayes classifier is used. Authors in [10] and in [18] also used which for fine-tuning the weights, three new back propagation methods
Naïve Bayes for classification of breast cancer data. are incorporated. The reported results on the WBCD are 99.24% and
In [6], the researchers have used the Mamdani fuzzy inference sys­ 98.29%.
tem. After feature selection with a linear discriminant model, four out of In [16], after feature selection using ICA, multiple classifiers are used
nine features are selected, and the proposed fuzzy model maps these four for classification, including: k-NN, neural network, RBF-based neural
input variables to one output variable. The authors have applied their network and SVM. In [24], artificial immune systems are used as a novel
model on the WBCD dataset and have reached the accuracy of 93.6 solution for classification, in which a semi-supervised method is incor­
percent. porated. The proposed method in this paper is applied on the WBCD
The SVM classifier has been also used in a number of breast cancer dataset.
prediction methods. For example, in [12], this classifier has been In [25], a combination of different classifiers is used to find the best
applied on WBCD and WDBC datasets; without feature selection, the combination. These classifiers include: the J48 decision tree, Multi-
accuracy is 96.7% and 96.8%, respectively. In addition, this classifier Layer Perceptron (MLP), Naïve Bayes, Sequential Minimal Optimiza­
has been applied on the WDBC classifier with feature selection, and its tion (SMO) and Instance-Based k-NN (IBK). Different combinations of
accuracy is 98.9%. In [14], the support vector machine classifier has classifiers are evaluated on WBCD, WDBC and WPBC datasets. The au­
been applied on the WBCD classifier and has reached the accuracy of thors showed that the best results are yielded when MLP and J48 are
96.7%. In [15], after a feature selection phase based on PSO, the SVM combined, with PCA feature selection.
classifier has been applied on WBCD for classification. In [19], SVM is In summary, most of feature selection methods in previous works are
used on the WBCD dataset and an accuracy of 96.13% is obtained. In computation-intensive. Furthermore, previous methods have used
[20], this classifier classifies the WBCD dataset instances and its accu­ common machine learning classifiers and in most of the studies, classi­
racy is 97%. In [21], semi-supervised and supervised methods are fiers are similar to each other. As a conclusion for this section, a
combined. To this end, in the first phase, two fuzzy clustering methods

4
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Table 2 Table 5
Features of the WBCD dataset. Distribution of Samples in Two Classes in the WDBC dataset.
Feature Number Feature Name Feature Range Dataset Name Benign Malignant Total Number of Instances

1 Clump Thickness 1–10 Training 283 172 455

2 Uniformity of Cell Size 1–10 Test 73 41 114
3 Uniformity of Cell Shape 1–10
4 Marginal Adhesion 1–10
5 Single Epithelial Cell Size 1–10
6 Bare Nuclei 1–10
Table 6
7 Bland Chromatin 1–10
8 Normal Nucleoli 1–10
Distribution of Samples in Two Classes in the WPBC dataset.
9 Mitoses 1–10 Dataset Name Benign Malignant Total Number of Instances

Training 116 39 155

Test 32 7 39
Table 3
Distribution of Samples in Two Classes in the WBCD dataset.
benign or malignant. 241 instances are malignant and 458 instances are
Dataset Name Benign Malignant Total Number of Instances
benign. Table 2 shows the features of this dataset and Table 3 shows the
Training 367 192 559 instances distribution in training and test data.
Test 91 49 140
The WDBC dataset has 569 patients with 31 features. The first
feature is again, the patient ID and the remaining 30 features are
calculated based on the digital image of the FNA test of the breast mass.
Table 4 These features represent the characteristics of the cell nuclei in the
Features of the WDBC dataset. images.
Feature Feature Description The 10 features that are shown in Table 4 for the WDBC dataset are
Number calculated for each nuclei and for these features, mean, standard devi­
1 Radius (Mean of distances from centers to points on the ation and maximum value (the highest of these values) are calculated.
perimeter) Hence, 30 features are extracted with real values for each instance. Each
2 Texture (standard deviation of gray-scale values) instance is labeled as benign or malignant. Among the 569 instances,
3 Perimeter
4 Area
356 of them are benign and other 213 are malignant. Table 5 shows the
5 Smoothness (local variation in radius lengths) distribution of instances in training and test data.
6 Compactness (perimeter^2 / area − 1.0) The WPBC dataset, similar to the other datasets has benign and
7 Concavity (severity of concave portions of the contour) malignant instances. It contains 194 instances, of which 148 are benign
8 Concave points (number of concave portions of the contour)
and 46 are malignant. The features of this dataset are as follows:
9 Symmetry
10 Fractal dimension (“coastline approximation” − 1)
• The first feature is the patient ID.
• The second feature is the output; R is for “recur” and N is for “Non-
summary of previous methods is presented in Table 1 based on classi­ Recur”.
fication method, feature selection method, dataset and strengths/ • The third feature is time, which is the time of recurrence for R and
weaknesses of these methods. time of being healthy for N.
• Features 3 to 33, like the WDBC dataset, represent 10 real values for
3. The proposed method cell nuclei: area, radius, perimeter (size and shape of nucleus),
symmetry, concavity and concave points, fractal dimension (coast­
In this section, the details of the proposed method are explained. To line approximation), compactness, smoothness (local variation in
this end, first, the datasets that will be used are introduced. Since each radius lengths), and texture (standard deviation of gray scale values).
dataset has different number of features, the input to the feature selec­ • The 34th feature is the tumor size, which is the diameter of the tumor
tion method will be different. A feature selection method based on lo­ in centimeters. The tumor size is represented in four groups:
gistic regression is proposed. Next, the GMDH neural network is o T-1: the size is from 0 to 2 cm.
explained. Fig. 1 demonstrates a schematic graph of the proposed o T-2: the size is from 2 to 5 cm.
method. o T-3: the size is higher than 5 cm.
o T-4: The tumor is broken through.
• The 35th feature is the lymph node status. It shows the number of
3.1. The datasets positive axillary lymph nodes that are observed during surgery. The
negative lymph means that the lymph nodes are not cancerous and
In this paper, three datasets are used and each has a different number the positive value means that they are cancerous.
of features, to better evaluate the feature selection method. The WBCD,
WDBC and WPBC datasets are chosen; these datasets were used in most Similar to the WDBC features, for 10 real values of the WPBC, three
of the previous works as well. They have been retrieved from the UCI values are calculated: mean, standard deviation and worst mean (or
machine learning repository [28]. highest mean). These values form 30 features, from the 3rd to the 33rd
The WBCD dataset contains the data of 699 patients with 10 features. feature. Mean is calculated as follows:
The first feature is the patient ID and the other features are quantitative
results of the FNA test for each patient. Each instance is labeled as

5
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

∑n
i=1 xi
function and m is the number of instances. To calculate unknown pa­
mean = (1)
n rameters for each class, the label of that class is set to 1, and the label of
other classes is set to 0. Hence, the value of y is either 0 or 1. The value of
Standard deviation is calculated as follows:
θ should be calculated such that the cost function in (7) is minimized.
Se = δ
s
(2) The cost function is convex and differentiable; hence, a gradient method
n can be used to find the optimal result. Hence,
in which, δ is the standard error parameter and s is the standard ( ( ))
θJ = θj + α yi − hθ xi xθ (i) (8)
deviation. Table 6 shows the distribution of instances in training and test
phases. Some examples of gradient methods are gradient descent and con­
At the end of this subsection, it is also worth mentioning that despite jugate gradient. In this paper, the conjugate gradient method is used
we used these three well-known data sets, the proposed method does not which does need to calculate the learning rate, i.e. α by trial and error
have any specific limitation for test on real data sets. In fact, if a real data and the optimum θ is found in the optimal time. Furthermore, it con­
set is going to be used, the following operations must be taken into ac­ verges faster than the gradient descent method. The implementation of
count to have a clean data set: this algorithm can be found in Matlab as the fminunc function and it is
presented in Appendix 1.
• Features must be extracted from real samples After classification of the data by the logistic regression algorithm,
• Valid samples must be separated and those which are damaged or not nine values for θ for the nine features of the WBCD dataset, 30 values for
valid must be removed. θ for the WDBC dataset, and 33 values for θ for the WPBC dataset are
• The data samples must be tagged (as benign or malignant) by an calculated. The results of feature selection and the final selected features
expert doctor or physician. are reported in Section 4.

3.2. Feature selection based on logistic regression 3.3. The Group method of data Handling neural network

Before feature selection, data should be pre-processed. This pre- The Group Method Data Handling (GMDH) is inspired from a
processing is the normalization, in which the data are mapped from deduction approach for mathematical modelling of complex systems
one range to another range. To this end, we map the data to the range [0, [30]. GMDH is not a classic statistical modeling method, but a regular
1], according to: process to overcome statistics and neural networks shortcomings. It is
Real value− Minimum value of dataset also one of the best methods for solving problems such as pattern
Thenormalized value =
Maximum value of dataset− Minimum value of dataset recognition in complex systems. The GMDH algorithm is also known as a
(3) heuristic learning algorithm, in the sense that it creates models for
complex systems as high-order regression which has advantages
The method used in this paper is logistic regression, which is a su­ compared to classical models. The variables that affect the process are
pervised learning method. To select or delete a feature, feature considered as the inputs of the system. The linear, non-liner or clustered
weighting is used. In logistic regression [29], the Sigmoid function is models can be obtained by minimizing an external measure and the
used for classification, which ensures that the output is in the range algorithm receives the desired information directly by means of the
[0–1]: input data. The effective input variables, number of layers, the neurons
( ) 1 in the hidden layers and the optimal structure of the model are deter­
hθ (x) = g θT x = (4)
1 + e− θT x mined automatically. This is done by the effect of minimizing the
external measure while developing the structure.
The input of this function is as follows, based on the dataset that the
The GMDH algorithm can be beneficial for classification due to the
data comes from:
following reasons:
∑
M
θ T x = θ0 + θ i xi (5) (1)- For real world problems, with small amount and noisy data, it can
produce an accurate and optimal model.
i=1

in which, θ0 is the bias parameter, for the first dataset, 1 ≤ i ≤ 9, for (2)- The number of layers and neurons in the hidden layers are
the second dataset, 1 ≤ i ≤ 30, and for the third dataset, 1 ≤ i ≤ 33. The determined automatically.
θi values are the unknown values that should be calculated, and the xi (3)- The algorithm guarantees finding accurate and unbiased models.
values are feature values. The objective function is defined as in (6): (4)- It finds relations between data and effective input variables
{ automatically.
− log(hθ (x) ) if y = 1
Cost(hθ (x), y ) = (6)
− log(1 − hθ (x)) Otherwise
Overall, the GMDH algorithm can be assessed from two different
in which, y is the class label and y ∈{0,1}. In fact, the cost function perspectives:
defined in (6) can be written in a simplified way:
• mathematical foundations
1 ∑m
( ( ) )
• system modeling analysis
J(θ) = Cost hθ xi , yi
m i=1
[ ]
1 ∑ m
( i) For detailed discussions, the interested reader is referred to [30]. It is
= − i i i
y loghθ x + (1 − y )log(1− hθ (x ) (7) also worth mentioning that despite neural networks need somehow
m i=1
large numbers of samples, the GMDH network is designed in such
such that, θ values (unknown parameters) are input values to the manner to overcome this limitation of neural networks. Example studies

6
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

such as [31] shows promising results which acknowledges this property dataset are considered. Applying the algorithm on other datasets is
of GMDH, as the authors in [31] used less than 1000 samples for training similar, and the only difference is in the number of inputs. Suppose that
their network. the WBCD dataset matrix is presented as below:
In the following, the application of GMDH for classification breast To train the network, the training data are used as shown in the
()
cancer data is explained.
matrix. From the nine variables of the dataset, 92 ordered pairs are
For a better understanding of the model, samples from the WBCD

Fig. 2. Structure of the GMDH neural network with the WBCD dataset variables.

7
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

extracted as follows: Table 7

Values of features coefficients for each dataset by the logistic regression method.
{(x1 , x2 )(x1 , x3 )(x1 , x4 )(x1 , x5 )(x1 , x6 )(x1 , x7 )(x1 , x8 )(x1 , x9 )⋯⋯..(x8 , x9 )}
The The The WPBC
in which, each ordered pair contains two variables or features and WBCD WDBC dataset
each ordered pair is considered as a matrix. For example, for the ordered dataset dataset
pair (x1 , x2 ), a matrix is composed as follows: Feature Feature Feature Feature Feature Feature
Number Coefficient Number Coefficient Number Coefficient

1 4.8115 1 − 1.3024 1 − 3.2780

2 0.1020 2 2.0438 2 − 11.3772
3 2.9135 3 − 1.0786 3 − 2.6679
4 2.1383 4 1.3705 4 11.1558
5 0.52482 5 − 1.9971 5 0.1281
6 3.8534 6 − 1.5165 6 − 2.8620
7 3.7090 7 6.2651 7 − 0.5785
8 1.4242 8 6.8728 8 0.5539
9 4.8224 9 − 1.5886 9 − 5.0867
– – 10 − 3.9961 10 1.8613
– – 11 8.6271 11 3.0986
– – 12 − 1.7333 12 12.2787
– – 13 4.4734 13 2.7660
– – 14 10.3790 14 7.4105
– – 15 3.1223 15 − 34.1072
– – 16 − 3.4533 16 − 7.8273
For each ordered pair, a polynomial is constructed as in (9): – – 17 − 3.6955 17 6.9381
– – 18 − 1.0389 18 0.4881
( )
yi = G xi , xj = a0 + a1 x1 + a2 x2 + a3 x21 + a4 x22 + a5 x1 x2 (9) – – 19 − 1.4708 19 − 6.1501
– – 20 − 4.2051 20 0.4854
such that, based on (10) and (11): – – 21 2.4333 21 8.5749
⎧ – – 22 4.3291 22 16.5298
⎪ – – 23 1.8190 23 − 1.9413
⎪
⎪ a0 + a1 x11 + a2 x12 + a3 x11 x12 + a4 x211 + a5 x212 = y1
⎪
⎪ – – 24 5.6579 24 − 17.6330
⎪
⎪ 25 2.7231 25 17.2522
⎪
⎪ a0 + a1 x21 + a2 x22 + a3 x21 x22 + a4 x221 + a5 x222 = y2 – –
⎨ – – 26 0.7375 26 6.3823
. – – 27 3.2892 27 − 8.6171
⎪
⎪
⎪
⎪
⎪ . – – 28 3.1402 28 − 0.3201
⎪
⎪ – – 29 5.0230 29 2.0842
⎪
⎪ + a4 x2559 + a5 x2559
⎩ a0 + a1 x559 1 + a2 x559 2 + a3 x559 1 x559 2 1 2 = y559
– – 30 2.5903 30 − 2.2644
– – – – 31 − 4.2999
(10) – – – – 32 − 0.5825
– – – – 33 − 1.6332
( )− 1 ( )−
(11)
1
Y = aT X→Y × X T = aT XX T →Y × X T XX T = aT XX T XX T

1
→aT = YAT (X T X)−
the coefficients (a0, a1, a2, a3, a4, a5) are obtained. By placing these
often passes all of the layers. Moreover, for the stopping criteria in
coefficients in (13), the output, y, is calculated. These coefficients are
implementation, a maximum number of layers is set. In the final layer, if
calculated for all of the ordered pairs, and each ordered pair approxi­
a number of neurons (or ordered pairs) with lower error is present, this
mates the output for all instances. Next, the output error of each ordered
error is selected for the final output. On the other hand, to increase the
pair is calculated based on minimum squared errors:
network speed, the maximum number of neurons for each layer is set to
∑
n (m) 20 neurons. This is due to the fact that the number of neurons (ordered
d2j = (Yi − yij )2 j = 1, 2, ⋯. (12) ( )
m
2 pairs) is and in each layer this number could be high which results
2
i=nT+1

Now, from the list of all of ordered pairs, those with more errors are in slowing the network. Fig. 2 shows a GMDH neural network with three
removed. The measure for selecting ordered pairs with less error is as: layers, in which the network inputs are variables or features of the
dataset and the output of each neuron, y, is the error of that ordered pair.
e = αemin + (1 − α)emax (13)
Accordingly, the ordered pairs with lower errors are fed to the next 4. Simulation, results, and evaluation
layer. α, the selection pressure, is considered 0.6, which is achieved by
trial and error and explained in Section 4. In fact, when this value is close For simulation of the proposed method, Matlab 2013 is used. In this
to 1, the ordered pairs with lower errors are selected. If it is set to 1, only section, first, the measures of evaluation are introduced, and then the
the ordered pair with the lowest error is selected. This is repeated in all selected features and the way of their selection by the logistic regression
of the next layers until convergence or other stopping criteria are algorithm are presented. Accordingly, the results of feature selection
reached. evaluation, and finally, the results of simulating the classification with
In the implemented network in this study, a maximum number of the GMDH algorithm compared with other methods are presented.
layers is considered. The network either passes all the predefined layers
or less layers are needed. In the conducted experiments, the network

8
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Fig. 3. AUC chart for sorted features based on their coefficients on WBCD dataset.

Fig. 4. AUC chart for sorted features based on their coefficients on WDBC dataset.

Fig. 5. AUC chart for sorted features based on their coefficients on WPBC dataset.

9
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Table 8
The numbers of selected features and their numbers inside the datasets.
Dataset Number of features Selected features
selected

WBCD 5 1–3-6–7-9
WDBC 15 2–7-8–11-13–14-15–21-22–24-25–27-
28–29-30
WPBC 16 4–5-8–10-11–12-13–14-17–18-20–21-
22–25-26–29

Fig. 8. Comparison of results of the GMDH classifier with and without feature
selection on WPBC dataset.

Table 9
Results of the proposed method without feature selection.
Dataset No. of true No. of true No. of false No. of false Accuracy
positives negatives positives negatives %

WBCD 233 457 8 1 98.7

WDBC 211 351 2 5 98.8
Fig. 6. Comparison of results of the GMDH classifier with and without feature WPBC 45 143 3 5 95.9
selection on WBCD dataset.

Table 10
Results of the proposed method with feature selection.
Dataset No. of true No. of true No. of false No. of false Accuracy
positives negatives positives negatives %

WBCD 240 455 1 3 99.4

WDBC 212 355 1 1 99.6
WPBC 40 147 6 1 96.4

exact point for the classifier on which the number of true positives
are maximum and the number of true negatives are minimum.
• Recall (TPR): shows how much the system is capable of classifying
the real cancerous instances as in (15).
• Precision: shows the ratio of correct classifications of the classifier to
all of the instances in the dataset as in (16).
• Error Rate: which is opposite to accuracy. The best performance is
Fig. 7. Comparison of results of the GMDH classifier with and without feature achieved when it is zero, and the worst performance is when it is 1, as
selection on WDBC dataset. in (17).
• F-Measure: It is a proper parameter for evaluating the quality of
4.1. Evaluation metrics classification, and describes the harmonic mean of precision and
recall, as in (18).
The instances of the datasets are split into training and test data with
80%-20% ratio. To evaluate the performance of the proposed method,
these measures are calculated: accuracy, Area Under the Curve of ROC Table 11
(AUC), True Positive Rate (TPR), False Positive Rate (FPR), precision, Results of experiments with different maximum number of layers on the WBCD
and f-measure. The measures are explained as follows: dataset.
Maximum No. of No. of true No. of No. of false Accuracy
• True positive: the number of positive instances that are classified no. of layers true negatives false negatives %
correctly. positives positives
• True negative: the number of negative instances that are classified 10 235 447 6 11 97.6
correctly. 20 237 450 4 8 98.3
• False positive: the number of negative instances that are classified as 30 239 450 2 8 98.6
positive. 40 235 458 6 0 99.1
50 240 455 1 3 99.4
• False negative: the number of positive instances that are classified as
60 236 457 5 1 99.1
negative. 70 237 452 4 6 98.6
• Accuracy: the percentage of correct classifications by the classifier on 80 238 453 3 5 98.9
the test set, as in (14). 100 239 450 2 8 98.6
• Area Under Curve of ROC (AUC): is used to distinguish data in the
classes (such as benign or malignant). The objective is to find the

10
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Table 12 coefficients. The vertical axis shows the value of AUC for each number of
Results of experiments with different maximum number of layers on the WDBC features (obtained through the regression classification method). Ac­
dataset. cording to these diagrams, the lowest number of features with accept­
Maximum No. of No. of true No. of No. of false Accuracy able AUC values are selected. Finally, considering Figs. 3-5, features
no. of layers true negatives false negatives % selected for each dataset are reported in Table 8. In fact, these are the
positives positives features that are used for classification in the next stage.
10 205 350 8 6 97.5 To clarify this issue consider the following example: for the WBCD
20 205 353 8 3 98.1 dataset, first feature number 9 is considered as the only feature (as it has
30 208 354 5 2 98.8
the highest coefficient value in Table 7) and classification is performed
40 210 353 3 3 98.9
50 210 356 3 0 99.5 using the regression method. Next, the AUC value is obtained and the
60 212 355 1 1 99.6 first point is added to the chart of Fig. 3. The second feature with the
70 209 354 4 2 98.9 highest coefficient for WBCD is feature 1 (Table 7), and the same process
80 206 352 7 4 98.1 is repeated. This is repeated for all the features, and as can be seen from
100 206 355 7 1 98.6
Fig. 3, from (after) the 5th added feature, the AUC does not experience
higher changes any more. Therefore, the first 5 added features are the
most important ones, as reported in Table 8. As another example, in
Table 13 Fig. 4, the AUC values after adding either 15 features or 30 features are
Results of experiments with different maximum number of layers on the WPBC identical. Hence, the minimum number of features is selected, which is
dataset.
15 features.
Maximum No. of No. of true No. of No. of false Accuracy However, for the WPBC dataset, the trend is different. As seen from
no. of layers true negatives false negatives %
Fig. 5, the AUC value is constant till 15 features, and after selecting more
positives positives
features, this value increases suddenly. In fact, with increasing the
10 25 141 21 7 85.6 number of features for classification, the AUC also increases and does
20 31 145 15 3 90.7
30 40 142 6 6 93.8
not remain constant. In addition, the AUC value is less for WPBC
40 40 146 6 2 95.9 compared to WBCD and WDBC (82 compared to 98 from Figs. 3-5). Our
50 41 146 5 2 96.4 solution for this problem was to select different subsets of features in a
60 41 147 5 1 96.9 try and error manner and finally it was concluded that with 16 features,
70 37 142 9 6 92.3
a desirable result is yielded. To prove this, the feature selection method
80 35 145 11 3 92.8
100 41 141 5 7 93.8 is evaluated in the next section.

TP + TN 4.3. Performance evaluation of the feature selection method

Accuracy = × 100 (14)
TP + FP + FN + TN
To evaluate the performance of the feature selection method based
Recall =
TP
× 100 (15) on logistic regression, the results of applying and not applying the
TP + FN feature selection method should be compared. To this end, three mea­
sures, namely accuracy, precision and recall are chosen, and the results
TP
Precision = × 100 (16) were calculated for three datasets, which are shown in Figs. 6, 7, and 8.
TP + FP
As seen from Figs. 6 and 7, the accuracy and precision measures for
FN + FP GMDH method are improved on the WBCD and WDBC datasets after
ER = = 1 − Accuracy (17) feature selection. The recall measure for the WDBC dataset is also
TP + FP + FN + TN
improved by feature selection. On the other hand, for the WPBC dataset,
2*precision*recall accuracy and recall measures were improved, but not the precision
F − measure = (18)
precision + recall measure. This may be because of the small number of samples of this
dataset and the unusual behavior of the feature selection reported in
Fig. 5.
4.2. Selected features and the method of feature selection by logistic
regression
Table 15
Results of the GMDH classifier on the datasets.
As said in the section 3.2, the unknown parameters of each dataset
are calculated using conjugate gradient. These unknown parameters are Evaluation Metric GMDH
the coefficients that the regression algorithm has assigned to each WBCD WDBC WPBC
feature of the dataset. These values are shown in Table 7. Accuracy 0.9940 0.9960 0.9690
Next, the diagram of number of features based on the AUC values is AUC 0.9947 0.9930 0.9533
drawn according to this method: features with higher coefficient values False Positive Rate 0.0021 0.0028 0.032
are added to the AUC chart sooner. These diagrams are presented in Recall 0.9876 0.9953 0.9756
Precision 0.9958 0.9953 0.8695
Figs. 3, 4 and 5 for each of the three datasets and on the whole training
F-Measure 0.9916 0.9953 0.9194
samples. In these figures, the horizontal axis represents the number of
features added in ascending order according to the values of their

Table 14
Results of the experiments with different values for alpha.
No. of layers Dataset 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

50 WBCD 98.7 99.1 99.1 98.7 99 99.4 96.6 96.6 96.1 95.7
60 WDBC 98.2 98.2 99.1 98.8 98.6 99.6 97.7 98.2 97.4 95.3
60 WPBC 93.8 95.9 94.8 93.3 95.4 96.9 94.3 89.2 79.4 67.3

11
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Fig. 9. Confusion matrix for the WDBC dataset with 60 layers.

Fig. 10. Confusion matrix for the WBCD dataset with 50 layers.

12
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Fig. 11. Confusion matrix for the WPBC dataset with 60 layers.

Fig. 12. The ROC curve for the three datasets.

13
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Table 16 Table 19
Comparison of the proposed method with baselines [12] on the WBCD and Comparison of the proposed method with some previous works based on
WDBC datasets. accuracy.
Classifier WBCD WDBC Reference Year Detection technique Dataset Accuracy
%
C4.5 Decision Tree 96.05 96.05
Random Forest 96.34 96.34 [3] 2015 Weighted naïve Bayesian WBCD 98.54
Bayes Network 97.22 97.22 [4] 2016 ANN-GA WPBC 79.2
Multilayer Perceptron Neural Network 96.05 96.05 [5] 2016 Gaussian Naïve Bayes WBCD 96.6
Support Vector Machine 96.78 96.78 [6] 2015 Mamdani Fuzzy WBCD 93.6
Radial Basis Function Networks 96.75 96.75 [8] 2017 CART WDBC 93.2
Rotation Forest 96.78 96.78 [10] 2016 NB + J48 WBCD 97
The Proposed Method without Feature Selection 98.7 98.8 [11] 2018 PCA-J48 WBCD 97.36
The Proposed Method with Feature Selection 99.40 99.6 [13] 2014 GAANN_RP WBCD 98.29
[17] 2018 PLCC-KNN WBCD 99.28
[19] 2016 SVM WBCD 97.13
[21] 2018 Fuzzy-SVM WBCD 99.04
Table 17 [22] 2017 Neural Network (FFBPN) WBCD 98
Comparison of the proposed method with baselines methods in [12,26] on the [23] 2015 Neural Network + GA WBCD 99.66
WPBC dataset. [24] 2016 AIS WDBC 98.0
[24] 2016 AIS WBCD 98.3
Classifier Precision% [25] 2012 Fusion of SMO, J48, MLP and WPBC 77.31
KNN
Naïve Bayes 67.17
[27] 2018 Euclidean distance GRNN WPBC 96.14
Bayesian Network 74.75
Proposed 2019 LR-GMDH WBCD 97.9
Random Forest 79.29
method
J48 Decision Tree 73.74
Proposed 2019 LR-GMDH WDBC 99.1
K-NN 68.68
method
SVM 76.26
Proposed 2019 LR-GMDH WPBC 84.6
C4.5 Decision Tree 75.25
method
The Proposed Method without Feature Selection 95.9
The Proposed Method with Feature Selection 95.6

The results of the proposed method with and without feature selec­
tion are reported in Tables 9 and 10 in detail. As the results suggest, the
Table 18 detection rate has increased by feature selection, confirming Figs. 6, 7
Comparison of the proposed method with some previous works. and 8. It should be noted that the number of layers in the network for
Reference Dataset Precision% Recall% F-measure% experiments without feature selection and with feature selection are
equal. The number of layers will be discussed in the next section.
[24] WDBC 95.9 95.9 97.3
[24] WBCD 98.5 94.3 96.4
[19] WBCD 96.5 96.5 97
[9] WBCD 96.53 99.11 – 4.4. Performance evaluation of the classification method
[6] WBCD 96 94 –
Proposed method WBCD 98.83 97.70 98.26 To select the GMDH classification network with best parameters, the
Proposed method WDBC 100 98.64 99.31
Proposed method WPBC 84.37 96.42 89.99
following experiments are performed by testing different maximum
number of layers. In Tables 11-13, the results of experiments on WBCD,
WDBC and WPBC datasets are reported. The average number of exper­
iments on each dataset and each neuron is 10. As reported in Tables 11-
13, the highest accuracy value in the WBCD dataset is achieved with 50
layers and in the WDBC and WPBC dataset are achieved with 60 layers.

Fig. 13. Comparison of the proposed method with previous works.

14
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

In addition, despite the accuracy for the WPBC dataset is much better as well. The precision has increased for both datasets; since it is
than previous works, it is few percents less than that of WBCD and dependent to both true positives and false positives. This shows that in
WDBC. This may be a result of the less number of samples of this dataset the proposed method, the false positive predictions have decreased for
in addition to its different behavior of feature selection. both datasets. The F-measure in the proposed method is better than the
In Table 14, obtained values for the number of layers are tested with other works, since this measure depends on both precision and recall,
different α values (in formula (13)) and the results are reported. Ac­ and the improvement in precision and recall has led to the increase in F-
cording to this table, the best value for alpha is 0.6. In Table 15, the measure.
desired evaluation measures are reported with the obtained values for Finally, Table 19 and Fig. 13 compare the proposed method with
the number of layers and α = 0.6. previous works on the accuracy measure. In fact, the precision measure
In Figs. 9, 10 and 11, the confusion matrices of the three datasets are reported in Table 19 is on the test data, resulted from the confusion
shown according to Table 15. The confusion matrix shows the perfor­ matrices in Figs. 9-11. This is due to the fact to have a more fair com­
mance of the classification algorithm based on the input data and the parison with other studies, despite there is no specific mention in pre­
classes. vious work whether their results are based only on the test data or the
In this matrix, the green box in the top left represents true negative whole dataset. As can be seen, the proposed method has a better accu­
and the green box in the bottom right represents true positive. The red racy than the other works. In most of the previous works, there are no
boxes on the bottom left and top right represent false positive and false feature selection phases, and the most important features have not been
negative, respectively. identified. In the proposed method, by selecting the most important
Fig. 12 shows the ROC curve for each of the datasets. An ROC curve features, and using a powerful neural network, the accuracy has
makes it possible to visually compare a set of classifiers. Different points improved, and false positive rate has decreased.
in the ROC curve are of interest. The bottom left point (0, 0) shows a
strategy in which a classifier does not produce positive values. The 5. Conclusion and future works
opposite strategy, which unconditionally produces positive outputs is
shown on the top right point (1, 1). The (0, 1) point shows the perfect In this paper, for selecting an optimal set of features to predict breast
classifier. In general, a point in the ROC curve is better than another cancer, the logistic regression method has been incorporated. After
point if it is located on the northwestern side. According to the ROC selecting the most effective features, a novel classification method based
curve and the recall measure in Table 15, it can be seen that the pro­ on the GMDH neural network has been utilized to classify the instances
posed method has yielded better results on the WDBC dataset than the to benign and malignant classes. A complete set of evaluation measures
other ones. (TPR, TNR, FPR, FNR, accuracy, AUC, Recall, precision, error rate and F-
measure) have been tested in simulations together with confusion
4.5. Comparison of the proposed method with baseline methods according matrices and ROC. In addition, comparison with seven baseline methods
to accuracy and four other studies has been conducted. The results show that the
GMDH outperforms many of the previous works according to the ac­
In [12], an intelligent breast cancer prediction system is imple­ curacy measure for the three datasets, WBCD, WDBC and WPBC. Also,
mented on two datasets (WBCD and WDBC), with 7 classifiers: C4.5 the results show that feature selection can improve accuracy.
decision tree, random forest, Bayesian network, multilayer perceptron As future works, it is suggested to use other deep learning methods,
neural network, support vector machine, radial basis function networks, which are accurate and can select features alongside classification. This
and rotation forest. In Table 16, the results of these classifiers are field has yet to be explored. Furthermore, creating real datasets from
compared to those of the proposed method in accuracy on the WBCD and hospitals is also of concern for this problem and exploring more features
WDBC datasets. The results show that the proposed method significantly by the aid of physicians.
outperforms the baselines. The proposed method using feature reduction
has higher generalization ability. The removal of less relevant features, CRediT authorship contribution statement
and selecting a limited number of features for each dataset leads to
better accuracy. Ziba Khandezamin: Methodology, Software, Formal analysis, Data
Table 17 shows the comparison of the proposed method with the curation, Visualization. Marjan Naderan: Conceptualization, Investi­
baseline classifiers [12] and [26] on the WPBC dataset. The results show gation, Resources, Writing - review & editing, Supervision, Project
that the proposed method has a desirable accuracy compared to the administration. Mohammad Javad Rashti: Writing - review & editing.
baselines. It should be noted that most of previous works have used the
WBCD and WDBC datasets, and the WPBC dataset has rarely been used. Declaration of Competing Interest

4.6. Comparison of the proposed method with other works based on The authors declare that they have no known competing financial
precision, recall, and F-measure interests or personal relationships that could have appeared to influence
the work reported in this paper.
Table 18 shows the comparison of the proposed method with some
other works based on precision, recall, and F-measure. This table shows Acknowledgements
that the proposed method significantly outperforms other methods on
the WDBC dataset in these measures. The improvement on recall is The authors would like to thank Shahid Chamran University of
because of increase in the true positives, and decrease in false negatives. Ahvaz for financial support of this project under grant 97/3/02/26247.
This shows that in the proposed method, the true positive rate has The authors would also like to thank Shahid Chamran University of
increased, and accordingly, the false negative has decreased. For the Ahvaz High Performance Computing Center (SCU-HPCC) for providing
WBCD dataset, in addition to recall, the two other methods are increased computing resources for this project.

15
Z. Khandezamin et al. Journal of Biomedical Informatics 111 (2020) 103591

Appendix 1

Pseudo code of the conjugate gradient algorithm used in this paper

References [15] A. Ahmadi, P. Afshar, Intelligent breast cancer recognition using particle swarm
optimization and support vector machine, J. Exp. Theor. Artif. Intell. 28 (6) (2016)
1021–1034.
[1] R. Sheikhpour, R. Sheikhpour, Breast cancer diagnosis using non-parametric kernel
[16] A. Mert, N. Kılıç, E. Bilgili, A. Akan, Breast Cancer Detection with Reduced Feature
density estimation, Razi Journal of Medical Sciences (RJMS) 23 (144) (2016)
Set, Comput. Math. Methods Med. (2015).
30–40.
[17] M. Kumari, V. Singh, Breast Cancer Prediction system, Procedia Computer Science
[2] R. Sheikhpour, M. Agha Sarram, R. Sheikhpour, Particle swarm optimization for
132 (2018) 371–376.
bandwidth determination and feature selection of kernel density estimation based
[18] R. Tina, S. Patil, S. Sherekar, Performance Analysis of Naive Bayes and J48
classifiers in diagnosis of breast cancer, Appl. Soft Comput. 40 (2016) 113–131.
Classification Algorithm for Data Classification, International Journal Of Computer
[3] M.A. Karabatak, A new classifier for breast cancer detection based on Naïve
Science And Applications 6 (2) (2013).
Bayesian, Measurement 32–36 (2015).
[19] H. Asri, H. Mousannif, H. Al Moatassime, T. Noel, Using machine learning
[4] Sh. Aalaei, H. Shahraki, A.R. Rowhanimanesh, S. Eslami, Feature selection using
algorithms for breast cancer risk prediction and diagnosis, Procedia Computer
genetic algorithm for breast cancer diagnosis: experiment on three different
Science (2016).
datasets, Iranian Journal of Basic Medical Sciences 19 (2016) 476–482.
[20] D. Bazazeh, R.M. Shubair, Comparative study of machine learning algorithms for
[5] B.M. Gayathri, C.P. Sumathi, An Automated Technique using Gaussian Naïve Bayes
breast cancer detection and diagnosis, in: 5th International Conference on
Classifier to Classify Breast Cancer, International Journal of Computer Application
Electronic Devices, Systems and Applications (ICEDSA), 2016, pp. 1–4.
148 (6) (2016).
[21] M. S. Abdullah, [Link]-Anzi, S. Al-Sharhan, “Hybrid multistage fuzzy clustering
[6] B. M. Gayathri, C. P. Sumathi, Mamdani Fuzzy Inference system for Breast cancer
system for medical data classification”, International Conference on Computing
risk detection, IEEE International Conference on Computational Intelligence and
Sciences and Engineering (ICCSE), Kuwait City, Kuwait, March 2018.
Computing Research (ICCIC), Madurai, India, Dec. 2015.
[22] A. Abdel-Ilah, L. Sahinbegoviü,“Using machine learning tool in classification of
[7] G. R. Sizilio, C. R Leite, A. M Guerreiro and A. D. D. Neto, “Fuzzy method for pre-
breast cancer”, Springer Nature Singapore Pte Ltd, 3-8, 2017.
diagnosis of breast cancer from the Fine Needle Aspirate analysis”, Biomedical
[23] A. Bhardwaj, A. Tiwari, Breast Cancer diagnosis using genetically optimized neural
Engineering Online. 11(83), 2012.
network model, Expert Syst. Appl. (2015).
[8] M. Nilshahi, O. Ibrahim, H. Ahmadi, M. Shahmoradi, A knowledge-based system
[24] L. Peng, W. Chen, W. Zhou, F. Li, J. Yang, J. Zhang, An immune-inspired semi-
for breast cancer classification using fuzzy logic method, Telematics Inform. 34
supervised algorithm for breast cancer diagnosis, Comput. Methods Programs
(2017) 133–144.
Biomed. 134 (2016) 259–265.
[9] A Hazra, S. Kumar Mandal, A. Gupta, “Study and analysis of breast cancer cell
[25] G.I. Salama, M.B. Abdelhalim, M. Abd-elghany Zeid, Breast Cancer Diagnosis on
detection using naïve Bayes, SVM and ensemble algorithms”, International Journal
Three Different Datasets Using Multi-Classifiers, International Journal of Computer
of Computer Applications, 145(2), 39-45, July 2016.
and Information Technology 1 (1) (2012) 36–43.
[10] N. Modi, K. Ghanchi, A Comparative analysis of feature selection methods and
[26] G. Naga RamaDevi, K. Usha Rani, D. Lavanya, “Ensemble-Based Hybrid Approach
associated machine learning algorithms on Wisconsin breast cancer dataset
for Breast Cancer Data”, International Conference on Communications and Cyber
(WBCD), International Conference on ICT for Sustainable Development 1 (2016)
Physical Engineering, 713-720, 2018.
215–224.
[27] S. Babaei Ghalejoughi, N. Lotfivand, “Enhanced Generalized Regression Neural Net
[11] Ph. Douangnoulack, V. Boonjing, “Building Minimal classification rules for breast
for Breast Cancer detection”, International Journal of Computer Science and
cancer diagnosis”, 10th International Conference on Knowledge and Smart
Information Security, 16(1), 2018.
Technology (KST), Chiang Mai, Thailand, Jan.-Feb. 2018.
[28] UCI Machine Learning Repository, Breast Cancer Wisconsin (Original) Dataset
[12] E. Alickovic, A. Subasi, Breast cancer diagnosis using GA feature selection and
[Link]
Rotation Forest, Neural Comput. Appl. 28 (2015) 753–763.
[29] E. Besharati, M. Naderan, E. Namjoo, LR-HIDS: Logistic Regression Host-based
[13] F. Ahmad, N.A. Mat Isa, Z. Hussain, M.K. Osman, S.N. Sulaiman, A GA-based
Intrusion Detection System for Cloud Environments, J. Ambient Intell. Hum.
feature selection and parameter optimization of an ANN in diagnosing breast
Comput. 10 (9) (2019) 3669–3692.
cancer, Pattern Analysis and Applications, Nov. 18 (4) (2015) 861–870.
[30] S Farlow, “the GMDH algorithm of Ivakhnenko”, The American Statistician, Vol.35,
[14] G. Ravi Kumar, G. A. Ramachandra, K. Nagamani, “An Efficient Feature Selection
No.4.
System to Integrating SVM with Genetic Algorithm for Large Medical Datasets”,
[31] X. Jia, Y. Di, J. Feng, Q. Yang, H. Dai, J. Lee, Adaptive virtual metrology for
International Journal of Advanced Research in Computer Science and Software
semiconductor chemical mechanical planarization process using GMDH-type
Engineering, 272-277, 2014.
polynomial neural networks, J. Process Control 62 (2018) 44–54.

16