International Journal of Computational Intelligence Systems (2024) 17:241

[Link]

RESEARCH ARTICLE

Knee Osteoporosis Diagnosis Based on Deep Learning

Amany M. Sarhan1 · Mohamed Gobara2 · Shady Yasser2 · Zainab Elsayed2 · Ghada Sherif2 ·
Nada Moataz2 · Yasmen Yasir2 · Esraa Moustafa2 · Sara Ibrahim2 · Hesham A. Ali2,3

Received: 9 January 2024 / Accepted: 22 July 2024

© The Author(s) 2024

Abstract
Osteoporosis, a silent yet debilitating disease, presents a significant challenge due to its asymptomatic nature until fractures
occur. Rapid bone loss outpaces regeneration, leading to pain, disability, and loss of independence. Early detection is piv-
otal for effective management and fracture risk reduction, yet current diagnostic methods are time-consuming. Despite its
importance, research addressing early diagnosis remains limited. Deep learning, particularly convolutional neural networks
(CNNs), has emerged as a potent tool in image analysis. This paper presents a novel approach utilizing transfer learning
with CNNs for osteoporosis detection from X-ray images. The proposed approach not only achieves a high accuracy of
osteoporosis diagnosis but also offers a revealed feature map that can guide medical professionals for osteoporosis diagnosis.
The innovation lies in a dual strategy: (i) a model integrating transfer learning from CNN architectures such as AlexNet,
VGG-16, ResNet-50, VGG-19, InceptionNet, XceptionNet, and a custom CNN, and (ii) a dataset collection augmentation
mechanism to enhance learning accuracy. The study includes binary and multiclass classification of knee joint X-ray images
into normal, osteopenia, and osteoporosis groups, utilizing a dataset of 1947 knee X-rays for training and testing. Performance
comparisons against state-of-the-art models reveal the proposed VGG-19 model achieves the highest accuracy at 92.0% for
multiclass and 97.5% for binary. These findings underscore the potential of deep learning with transfer learning in aiding
early osteoporosis detection, thereby mitigating fracture risks.

Keywords Knee osteoporosis · Deep learning · Convolutional neural network · CNN · Medical imaging · Image
preprocessing

1 Introduction osteoporosis. Osteoporosis is detected when both bone min-

eral density and bone mass decrease resulting in defect of
Knee pain is a frequent complaint that affects people of all the structural of bone tissue.
ages. It is also an irreversible condition that causes issues Numerous studies have delved into the classification
and has an impact on our lives and future. The first type of of knee osteoporosis, each presenting unique findings and
knee problems is knee osteoarthritis, which is the most com- methodological nuances. For instance, Wani and Arora’s
mon joint disorder, while the second case is knee osteopo- research published in 2020 reported a commendable accu-
rosis, which can progress without symptoms until a broken racy of 91%. However, upon closer examination of the con-
bone occurs. The later begins as osteopenia, which is loss fusion matrix, the actual accuracy was found to be lower. In
of bone mass or bone mineral density, then escalates to be addition, the models exhibited high validation loss, suggest-
ing potential limitations in generalization. In contrast, Kumar
et al.’s study in 2023 showcased a higher accuracy compared
* Amany M. Sarhan to Wani & Arora’s findings. Nevertheless, the research noted
amany_sarhan@[Link]
significant drops in model performance during training, indi-
1
Department of Computer and Control Engineering, Tanta cating potential instability or overfitting issues. Similarly,
University, Tanta, Egypt Abubakar et al.’s research in 2022 demonstrated promising
2
Department of Artificial Intelligence, Faculty of Artificial accuracy in binary classification. However, the study lacked
Intelligence, Delta University, Gamassa, Egypt reporting on validation loss and encountered difficulties in
3
Department of Computer and Systems Engineering, correctly classifying certain instances, highlighting the need
Mansoura University, Mansoura, Egypt

Vol.:(0123456789)
241 Page 2 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

for improved model robustness. These studies collectively Machine learning has an effective rule in understanding
underscore several crucial gaps in the current research land- diagnosis and treatment of osteoporosis, as its algorithms
scape. Key areas of concern include limited dataset size, can analyze large datasets to identify patterns to how indi-
insufficient picture quality, and the necessity for a more viduals respond to different treatments. It can also monitor
robust and comprehensive convolutional neural network changes about health overtime and through natural language
(CNN)-based architecture for osteoporosis detection from processing (NLP) algorithms can extract valuable informa-
X-ray images. Moreover, our own research endeavors have tion from electronic health records that facilitates cohesive
yielded promising results, particularly in achieving higher analysis of patient histories. Convolutional neural network
accuracy in both multiclass and binary classification tasks. (CNN) models have gained prominence [4, 5] due to their
Addressing the identified gaps presents an opportunity for revolutionary success in diagnosing numerous diseases from
advancing the field and enhancing the efficacy of knee osteo- pictures and many other useful applications, such as brain
porosis classification methodologies. Methodologies such tumor detection and segmentation [6], COVID detection [7],
as constrained multi-dimensional (CM) mathematical algo- cancer detection [8], human activity identification [9], and
rithms and meta-heuristic algorithms (MAs) [43] showcase age and face detection [10].
the potential for optimizing complex models in healthcare Osteoporosis can be found in many parts of the human
diagnostics. body including teeth, hip, spine, hand, and knee. Therefore,
Osteoporosis is a major public health problem in many automated systems can detect suitable changes in bone den-
countries around the world. It is a silent danger that increases sity and structure that may be indicative of osteoporosis.
the incidence of it among the elderly. The statistics collected Several work have been proposed for osteoporosis in teeth
by the World Health Organization (WHO) stated that Osteo- [11], hip [12], and spine [13], while few have been directed
porosis is considered a health problem of great importance to knee osteoporosis despite the fact that the knee is the
to the elderly, as the rate of bone fractures in women due to most strained joint since it bears the body’s weight and is
this disease ranges between 30 and 40%100 people. That responsible for movement. With an aging population, the
is, an increase of 132 percent, and by 2050, experts expect prevalence of osteoporotic fractures around the knee rises
this number to reach one billion. It affects men and women especially in with women. X-ray images have been the
from all racial groups, and can rob people of their inde- major media in osteoporosis detection. It is the most com-
pendence and quality of life. However, osteoporosis could be mon imaging technique among the medical community to
prevented and treated; therefore, its effect can be eliminated find bone pathologies as it also takes images of whole body.
if it is early detected [1–3]. That makes it very important to X-ray images greatly help in fracture diagnosis, dislocation
check our bone health and to detect osteoporosis. of joints, and detecting changes in the bone density and
The normal bone modeling process involves the creation architecture. Although it is the main tool for the doctors
of new bone and the removal of the old bone. Osteoporosis to diagnose osteoporosis, it does not always easily detect
causes imbalance in this process so bone loss occurs more osteoporosis unless the doctor is professional. There are
rapidly than bone formation. Its symptoms are not obvious some researches that dedicated their work to osteoporosis
that leads to fracture in many people, especially the elderly, detection [11, 14–21]; however, their accuracies are insuf-
causing pain, disability and loss of free movement. The ficient to rely on.
treatment of osteoporosis can take long time, where that In this research, we introduce two deep learning mod-
time can be reduced if we detect its early stage called osteo- els aimed at detecting knee osteoporosis in X-ray images,
penia. X-rays are used in the bone density test to determine considering the detection or absence of osteoporosis. We
how many grams of calcium and other bone minerals are also introduce several deep learning models targeting the
packed into a section of bone. Doctors often require this test differentiation between knee osteoporosis and osteopenia
if a fracture occurs, a noticed loss of patient’s height (about in X-ray images as multiclass classification problem. Our
3.5 cm) or suffer of certain hormone drop. models not only excel in classification but also assimilate
Figure 1a shows a sample of X-ray image for three cases: intricate abstract representations from raw inputs, courtesy
normal, osteopenia and osteoporosis knee. We can clearly of their layered architecture. Our study is geared towards
see that the knee is still having good bone mass even it has addressing pivotal challenges in knee osteoporosis detection,
osteopenia (Fig. 1b) than that of the one having osteoporosis with the following distinctive contributions:
(Fig. 1c). Figure 1d also shows areas of decreased bone den-
sity of patellas, femoral and tibial condyles. Finally, Fig. 1e 1. Model architecture: We investigate various deep learn-
gives a detailed measures of knee densities in an X-ray ing models of convolutional neural networks utilizing
image. Through these images and due to its importance, it transfer learning approach to ensure detection accuracy
is a must to detect this disease in early stages to enable better and efficiency, including VGG-19, VGG-16, AlexNet,
treatment results. ResNet-50, ExceptionNet, and InceptionNet, as well as
International Journal of Computational Intelligence Systems (2024) 17:241 Page 3 of 33 241

Fig. 1  X-ray images for a normal, b osteopenia, c osteoporosis knee, d noticed change in bone density, and detailed measures of knee densities

a CNN specifically designed for accurate knee osteopo- 2 Background and Related Work
rosis detection.
2. The detection aim is binary classes for osteoporosis As the latest trend in complex and powerful deep learning
detection and multiclass for osteoporosis and osteopenia models, deep neural networks are designed to model geo-
detection. As such, our work serves both directions. metric transformations and are designed such that they are
3. Dataset Use: The model is trained and evaluated on an maximally activated throughout the dataset, facilitating the
assembled dataset [22], which is a combination of four task of helping us understand the structure of the model,
independent datasets [23–26]. Furthermore, after com- i.e., mapping deep features to truth labels. Convolutional
bining the datasets, data augmentation is employed to neural networks (CNNs), a network inspired by the visual
solve the unbalanced dataset problem. functions of animal brains, were first proposed by Fuku-
shima in the late 1970s, but the current network framework
This research paper is structured as follows: an intro- was developed by LeCun in 1998. The development of
duction followed by background and related work, and CNNs is based on the backpropagation algorithm, which is
then a full discussion of the suggested models. The experi- It is necessary to adjust model parameters repeatedly and
ments, findings, and analysis are presented in order, fol- automatically. Various CNN designs have been developed
lowed by the conclusion and list of references.
241 Page 4 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

to control the structure, learning method, number of train- 2.2 Transfer Learning in Deep Learning
ing epochs of the corresponding layers, and hyperparam-
eters, and CNN models can maintain deep networks for Deep learning (DL), or more precisely the class of machine
complex learning tasks [30–34]. learning called artificial neural networks, is dominant as a
Transfer learning (TL) and deep learning (DL) are two tool for solving high-dimensional prediction and control
important theories in machine learning, and it is essen- problems in computer vision, language, and controlling
tial to understand both above all in the field of artificial agents interacting with relatively simple environments. In
intelligence (AI). The motivation behind learning is the the context of computer vision, a deep learning model is
assumption that intelligence gained from learning in one given an image as input, is expected to produce a vector of
domain can be usefully reused to solve learning problems class probabilities, and ranks the identified key elements
in other related or completely unrelated learning tasks. present in the image according to their probability of pres-
In the context of distance learning, such reuse learning ence. These class probability vectors are generated by soft-
is typically achieved by transferring early or middle level max transformation after computing a linear combination
layers of deep models which involves demonstrating trans- of inputs and weights at each layer of the network [37,
fer strategies and how much knowledge and complexity is 38, 42].
transferred to the structure of the target model [37, 38]. In the deep learning field, deep structures are mostly
learned on a dataset that consists of pairs of data and
labels. Deep learning systems typically need large super-
2.1 Significance of Knee Osteoporosis Diagnosis vised training, and they require large labeled data to per-
form well. But in some cases, large labeled datasets are not
The importance of achieving more accurate measurements available, and labeling is costly. To this end, the learned
of bone density in the knee is enormous in reducing the features in one deep network can be reused to improve the
serious consequences mentioned above. Dual-energy performance of another neural network that has a different
X-ray absorptiometry (DXA) examination or whole leg task than the original task. This practice is called transfer
models for accurate estimation of knee mineral density are learning. As a result of using a pre-trained network for
useless because BMD over the entire knee area is not of the new task, it is also possible to use a good model with
direct clinical significance, although most BMD measure- less training.
ments are reported to contain error. In clinical practice, Matching the right task and model in the right way is a
the diagnosis of osteoarthritis of the knee is mainly based complex and resource-intensive process. One way to make
on X-ray radiographs according to the qualitative evalua- this work easier is to use known representations from other
tion of bone tissue. Furthermore, there is a real unsaleable learning processes and use them as a starting point. One
demand for a rapid, non-invasive and accurate diagnostic of the technologies for this purpose, transfer learning, is a
tool for knee osteoarthritis. Our study aims to develop technique that takes advantage of the features obtained in
a rapid diagnostic tool to aid widespread screening for the source task to improve learning in the target task. The
early detection of knee osteoarthritis and support decision- criteria for selecting the source and target tasks in transfer
making in clinical practice. learning are not clear. For that reason, there are many ways
The underlying path of knee osteoarthritis is bone min- to apply transfer learning in deep learning.
eral density disease. Loss of bone mineral density in the We can describe osteoporosis as medical condition
region of the subarticular trabecular bone located in the characterized by weakened bones and increased fracture
intercondylar notch beneath the thickened cartilage has a risk due to a loss of bone density. It commonly affects
negative correlation with the loss of mechanical proper- older adults, particularly postmenopausal women, osteo-
ties. Therefore, it is considered a major indicator of knee porosis, a silent disease causing weak and fragile bones,
osteoarthritis. However, accurately estimating knee min- often leads to fractures with no evident symptoms until
eral density in this discrete region is a difficult task, as they occur, causing pain and disability. The traditional
the subarticular region of the knee is considered to have diagnostic methods are time-consuming. Early detection
a peculiar response in postmenopausal women compared is crucial for effective management. Motivations to address
to bone mineral density in other parts of the human skel- osteoporosis stem from the desire to improve health out-
eton. Significant and complex challenges have generally comes, prevent debilitating fractures in an aging popula-
been reported in the literature such as knee degeneration, tion, reduce the economic burden on healthcare systems,
delayed treatment, increased duration of post-fracture hos- and advance scientific knowledge and innovations in medi-
pitalization, and major depressive disorder secondary to cal science. The goal is to enhance overall well-being,
femoral neck fractures, which are particularly severe when emphasizing the importance of prevention, early detection,
the patient is an elderly woman. and effective management of osteoporosis.
International Journal of Computational Intelligence Systems (2024) 17:241 Page 5 of 33 241

Fig. 2  The most common osteoporosis diagnose methodologies

2.3 Osteoporosis Diagnose Methodologies iv. Quantitative ultrasound (QUS) which is a non-invasive

test that measures bone density using sound waves. It
Generally, the diagnosing of osteoporosis can be achieved is quick procedure with no radiation exposure, suitable
using the next most common four different methodologies, for certain population.
as depicted in Fig. 2. These four methodologies are:
Although several studies had observed that the sensitiv-
i. X-ray and DXA where important signs of osteoporo- ity of X-ray and DXA in the diagnosing of osteoporosis is
sis in an X-ray may appear such (cortical thinning— higher than other methods, the American College of Radi-
increased radiolucency). They gives very precise ology (ACR) has issued guidance that X-ray and DXA is
measurements at clinically relevant skeletal sites; accurate tools for diagnosing. Insights gained from study-
however, the major disadvantages of DXA are that ing bone health can potentially influence a range of medical
the machine is large (not portable) and expensive, and disciplines, from orthopedics to geriatrics, opening doors
that it uses ionizing radiation. to novel treatments and preventive strategies for a variety
ii. Numerical laboratory test—postmenopausal women of age-related conditions. Moreover, a concerted effort
with low BMD (T-score below − 2.5) and/or fragility to address osteoporosis aligns with broader public health
fracture have the some basic tests such as: biochem- initiatives.
istry profile, 25-hydroxyvitamin D (25[OH]D) and In exploring the landscape of AI applications in medical
complete blood count (CBC). imaging, previous studies have predominantly focused on
iii. Computed tomography scan which is an imaging tech- image recognition, feature extraction, and diagnostic assis-
nique that provides detailed cross-sectional images of tance. In this section, we will briefly introduce the most
bones to check bone density, and structure, and iden- recent works in these two fields. In [17], Wani & Arora
tifying fractures. handled the osteoporosis diagnosis in knee X-rays using
241 Page 6 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

transfer learning based on convolution neural network (ReLU), and batch normalization, elucidating their roles in
using multiclass dataset from Mendeley dataset [26]. Four shaping the behavior and performance of neural networks.
models were used: AlexNet, VGG-16, ResNet and VGG-19
with accuracies: 91%, 86.30, 86.30% and 84.20%, respec- 2.4.1 Softmax Function
tively. In Ref. [18], Kumar et al. utilized a fuzzy rank-based
ensemble model for accurate diagnosis of knee osteoporosis The softmax function is a cornerstone in multiclass clas-
built using three models: Inception v3, Xception and ResNet sification tasks, particularly in the output layer of neural
18. A multiclass dataset from Mendeley data [26] for Knee networks. It transforms the raw output scores into proba-
X-ray images was used. The models accuracies were: Incep- bilities, ensuring that the sum of these probabilities across
tion v3 (89.8%), Xception (90.9%), and ResNet 18 (91.4%). all classes equals one. Mathematically, given a vector of
In [21], by Abubakar, et al. used transfer learning models for raw scores z, the softmax function computes the probabil-
osteoporosis classification on knee radiograph of RGB and ity distribution σ(z) as follows:
grayscale images using binary Knee X-ray images dataset
from Kaggle [23]. Two models were employed: GoogleNet
�� ezi
𝜎 ⃗z i = ∑k
with accuracy 90.0% and VGG-16 with accuracy 87%. j=1
ezi
Yang 2022 [27] aimed to identify and classify knee dis-
eases (osteoporosis and osteoarthritis) through X-ray images where zi represents the raw score for class i, and K denotes
using deep learning. A binary classes dataset from kaggle the total number of classes. The softmax function ensures
[23] for Knee X-ray images was employed. They used three that the network outputs a valid probability distribution,
models: custom CNN with accuracy 77%, late-fusion with enabling intuitive interpretation and facilitating tasks such
accuracy 71%, and VGG-16 with accuracy 82%. In Ref. as classification.
[16], Dodamani and Danti worked with a binary dataset
from Zydus Hospital [16] for spine, hand, leg, and knee 2.4.2 Sigmoid Function
X-ray images utilizing transfer learning for osteoporosis
classification. Five models were trained: VGG-16, VGG- The sigmoid function is a widely used activation function,
19, DenseNet-121, ResNet-50 and Inception V3, where the especially in binary classification tasks. It squashes the
reported accuracies were 78%, 86%, 93%, 89% and 90%, input values to the range [0, 1], effectively interpreting
respectively. A summary of the previous X-ray-based knee them as probabilities. The sigmoid function is defined as
osteoporosis diagnosis using deep learning models are listed
below along with their accuracy in Table 1. 𝜎(z) =
1
As a conclusion of these studies that investigated using 1 + e−z
deep learning models for osteoporosis diagnosis using knee where z represents the input to the function. Despite its van-
X-ray images, we have identified that the performance (pre- ishing gradient issue for extreme input values, the sigmoid
sented as model accuracy) are poor. More advanced mod- function remains valuable in scenarios where binary deci-
els are required to enhance the detection accuracy of knee sions are required, such as determining the probability of a
osteoporosis. given input belonging to a certain class.

2.4 Mathematical Background and Formulations 2.4.3 Rectified Linear Unit (ReLU)

for Deep Learning Models
ReLU is a popular activation function known for its sim-
Deep learning models are associated with several building plicity and effectiveness in mitigating the vanishing gra-
blocks that enable them to perform the feature extraction and dient problem. It replaces negative input values with zero
classification process they are built to perform. Each deep while leaving positive values unchanged. Mathematically,
learning model is built using some or all of these blocks, the ReLU function is defined as
where each model has different architecture and, therefore, f (z) = max(0, z)
different performance. In the area of neural networks and
deep learning, understanding the mathematical underpin- ReLU’s simplicity and computational efficiency make
nings of various activation functions and normalization it a preferred choice in many deep learning architectures,
techniques is crucial [35, 36, 39]. This section delves into promoting faster convergence during training and alleviat-
the fundamentals of softmax, sigmoid, rectified linear unit ing issues associated with vanishing gradients.
Table 1  Previous work for knee osteoporosis
References Bone type Image type Classes Dataset Data size Classifier Reported CV train/valida- Disadvantages Problem faced
model accu- tion/test
racy

Wani & Arora, Knee X-ray Multiclass Dataset from 239 AlexNet 91.0% 95:5 1. High valida- The CNN dataset
2022 [17] Mendeley Data normal: 36 VGG-16 86.30% Training:testing tion loss is limited by a
[26] Osteopenia: 154 ResNet 86.30% 2. The accuracy small number of
Osteoporosis 49 VGG-19 84.20% mentioned in images
the research The T-score
paper was not yielded unstable
the same after results as a bone
calculating the parameter
accuracy from The collected
the confusion clinical and other
matrix factors were not
utilized in the
classification
International Journal of Computational Intelligence Systems

process
Kumar et al., Knee X-ray Multiclass Dataset from 239 Inception v3 89.8% Not reported There were many Not reported
2023 [18] Mendeley Data Normal: 36 Xception 90.9% drops in the
[26] Osteopenia: 154 ResNet 18 91.4% model while
Osteoporosis 49 training
Abubakar et al., Knee X-ray Binary Dataset from 646 GoogleNet 90% 207 Training 1. The researcher 1. Picture quality is
(2024) 17:241

2022 [21] Kaggle [23] normal: 372 VGG-16 87% 65 Testing does not men- insufficient
Osteoporosis: tion the valida- 2. Existence of
372 tion loss overlapping cells
2. The high- 3. The suggested
est accuracy ensemble model
is 90% for a has difficulty cor-
binary model rectly classifying
a few images
Yang, 2022 [27] Knee X-ray Binary Dataset from 646 Custom CNN 77% 71% 70 Training 1. The researcher 1. The OS training
Kaggle [23] Normal: 372 Late-Fusion 82% 10 Testing does not men- data shows a
Osteoporosis: VGG-16 tion the Valida- slight imbalance,
372 tion Loss to be addressed
2. The algorithm later using the
is not efficient class weight
enough due to method for bal-
low accuracy ance
2. All images in
the dataset have
been resized to
224 × 224
Page 7 of 33
241
 241 Page 8 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

2.4.4 Batch Normalization

ually distinguish-
1. Difficulty in vis-

two conditions is
ing between the

effectively train
large dataset to
necessity for a

deep learning
placed on the
2. Emphasis is
Problem faced

highlighted
Batch normalization is a normalization technique that

networks
addresses the internal covariate shift problem during
training by normalizing the activations of each layer. The
process involves the following steps:
1. High valida-
Disadvantages

1. Calculate mini-batch mean: compute the mean μB of the

mini-batch
tion loss

m
1∑
𝜇B = x
m i=1 i
CV train/valida-

2. Calculate Mini-batch variance: compute the variance σ2B

Not reported

of the mini-batch
model accu- tion/test

m
1∑
σ2B = (x − 𝜇B )2
m i=1 i
Reported

3. Normalize: the activations within the mini-batch using

78%
86%
93%
89%
90%
racy

the mean and variance

x − 𝜇B
DenseNet-121

Inception V3

xi = √i
̂
ResNet-50
Classifier

σ2B + 𝜖
VGG-16
VGG-19

4. Scale and shift: the normalized activations using learn-

able parameters γ and β
Not reported

xi + 𝛽
yi = γ̂
Data size

Here, ϵ is a small constant added to the variance to pre-

Dataset, Dahod,

vent division by zero, and γ and β are learnable parameters

Zydus Hospital

per feature dimension. Batch normalization not only acceler-

ates convergence but also acts as a regularizer, reducing the
Gujarat
Dataset

reliance on techniques like dropout.

2.5 Convolutional Neural Network (CNN) Models

Image type Classes

Binary

Convolutional Neural Networks (CNNs) are structured with

three primary layers: convolutional, pooling, and fully con-
nected. The convolutional layer employs filters to extract
features from input images, while pooling reduces spatial
X-ray

dimensionality, promoting shift-invariance. Stacking multi-

ple convolutional and pooling layers facilitates the extraction
Pooja and Sakshi, Spine, hand, leg,

of higher-level feature maps. Fully connected layers typically

follow this stack in a CNN, occurring before the output layer
Bone type

to perform reasoning tasks [28].

knee

In the realm of deep learning, transfer learning involves

Table 1  (continued)

two main approaches: feature extraction and fine-tuning.

Feature extraction utilizes a standard dataset like ImageNet
to remove the top classification layer. The remaining model
2023 [16]
References

serves as a feature extractor, capturing relevant features for

a new dataset. A new classification model is then trained
International Journal of Computational Intelligence Systems (2024) 17:241 Page 9 of 33 241

on the extracted features. Fine-tuning, on the other hand, fully connected layers are adjusted using techniques such as
utilizes pre-trained model weights initially and updates them backpropagation and gradient descent to minimize the differ-
as the training process progresses. During fine-tuning, the ence between the predicted probabilities and the true labels.
weights learned from the original dataset serve as a starting During inference or testing, an input image is passed through
point and are adjusted for more precise features related to the feature extraction layers of our models, and the resulting
the new dataset, aiming to adapt general features to specific features are fed into the fully connected layers.
ones [29]. The output of the SoftMax layer represents the probabili-
To mitigate overfitting due to limited training data, this ties of the image belonging to each class. The class with the
study employed pre-trained weights from ImageNet and highest probability is considered the predicted class for the
implemented a transfer learning strategy. For multi and input image. Generally speaking, classification’s objective
binary classification, the final layer of advanced models can be binary or multiclass. Binary classification is a method
such as VGG-16, VGG-19, AlexNet, XceptionNet, Incep- where the diagnostic outcome is divided into two catego-
tionNet, and ResNet-50 underwent fine-tuning [30, 32, 40, ries or classes. In the case of knee osteoporosis diagnosis,
41]. This involved replacing the fully connected layers with a binary classification approach may involve categorizing
a SoftMax activation function and adding a dropout of 0.5 patients into two groups: those with knee osteoporosis and
for regularization. Finally, a layer with dense connections those without knee osteoporosis. This approach simplifies
and a SoftMax activation function was added, produc- the classification into a yes/no or positive/negative outcome.
ing either two probability outputs in binary classification: Whereas multiclass classification, on the other hand,
‘Normal’ and ‘Osteoporosis’, or three probability outputs involves classifying the diagnostic outcomes into more than
‘Normal’, ‘Osteopenia’ and ‘Osteoporosis’ for the multiclass two categories. In the context of knee osteoporosis diagno-
classification. sis, a multiclass classification approach may involve catego-
rizing patients into multiple groups based on the severity
2.5.1 Feature Extraction of osteoporosis or different subtypes of knee-related bone
disorders. For example, patients may be classified as having
Feature extraction in our models involves using pre-trained normal bone density, mild osteoporosis, moderate osteopo-
convolutional layers to extract important features from rosis, or severe osteoporosis affecting the knee joint.
images. These layers apply filters to the input image, cre-
ating feature maps that capture visual patterns. Activation 2.6 Problem Formulation
functions and max pooling further enhance and reduce the
features. The extracted features can then be used for tasks Osteoporosis, identified by weakened and brittle bones,
like image classification. Using pre-trained layers, our is frequently referred to as a “silent disease” as it remains
models leverages learned knowledge from a large dataset to symptom-free until fractures manifest. The usual bone
extract meaningful features effectively. regeneration cycle is disturbed in osteoporosis, caus-
ing faster bone loss than formation. Symptoms are sub-
2.5.2 Classification tle, resulting in fractures that induce pain, disability, and
a decline in independence. Existing diagnostic procedures
The classification process in our models refers to how the are lengthy, underscoring the significance of early identifica-
extracted features are used to assign a label or category to tion to effectively manage and diminish the risk of fractures.
an input image. After the feature extraction step, the output Deep learning has gained widespread acceptance in image
from the last convolutional layer is typically flattened into a analysis, representing a notable progress in recent decades.
vector. This vector is then passed through one or more fully Existing research papers on osteoporosis have previously
connected layers, also known as the classifier. These layers implemented algorithms, but many of them have reported
are responsible for making the final predictions based on the relatively low accuracy rates [16–18, 21, 27]. In response
extracted features. The fully connected layers in our models to this limitation, our study focuses on introducing an algo-
usually consist of one or more dense layers, followed by a rithm that achieves notably higher accuracy in the context
SoftMax activation function. The dense layers have learn- of knee osteoporosis diagnosis. By incorporating advanced
able weights that are adjusted during the training process to deep learning techniques, specifically employing transfer
map the extracted features to specific classes. learning with VGG-19 and exploring various architectures
The SoftMax activation function then normalizes the out- such as VGG-16 and ResNet, our model aims to surpass the
put of the dense layers into a probability distribution over accuracy levels documented in these earlier studies. This
the possible classes. During training, the our models model novel approach highlights our commitment to addressing the
is presented with a labeled dataset, and the weights of the
241 Page 10 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 3  Proposed architecture for knee osteoporosis diagnosis

limitations of prior algorithms and pushing the boundaries projection in DL to affirm the importance of the indicators
of accuracy in osteoporosis detection. associated with the ground truth, and then performing trans-
fer learning on the widely used convolutional neural net-
2.7 Plan of Solution works. Subsequently, we demonstrate the clinical value of
the selected feature map in the CNN learning process from
Our ongoing research is focused on the creation of an another perspective to show the guiding properties of the
advanced algorithm tailored for osteoporosis diagnosis, rely- CNN-derived feature representation for knee osteoporosis.
ing on a sophisticated blend of neural network architectures. Diverse artificial intelligence (AI) empowered by machine
These include renowned models such as AlexNet, VGG-16, learning and deep learning (DL) has significantly aided a
ResNet, VGG-19, CNN, InceptionNet, and XceptionNet. more efficient and accurate decision support system in the
This comprehensive integration of diverse architectures biomedical field in the past decade. Convolutional neural
underscores our commitment to elevating the accuracy and networks (CNN), a type of DL method, has been extensively
efficacy of osteoporosis detection. By harnessing the capa- applied in ensemble learning techniques such as boosting
bilities of these cutting-edge models, we aim to not only and bagging. Transfer learning (TL) is also a strategy to
enhance diagnostic precision but also contribute signifi- improve the performance of CNN based on the combination
cantly to the evolution of methodologies in medical image of CNN’s feature extraction power and knowledge transfer
analysis. Through this extensive exploration, our goal is to across different datasets or domains. Threshold imitation-
propel advancements in the understanding and application volumetric fractal dimension is closely associated with bone
of osteoporosis diagnosis techniques, ultimately benefiting microarchitecture geometry and is also reported as a reliable
the broader landscape of healthcare research. In addition, index for knee osteoporosis diagnosis. However, the utility
we prepare a new dataset, which is a collection of available of deep learning empowered by TL and the guiding sig-
datasets [23–26] used in binary and multiclass classification nificance of the CNN-derived feature map on osteoporosis
for osteoporosis in previous work [16–18, 21, 27]. diagnosis has not been well explored in the literature.
Figure 3 depicts the different components and phases of
the proposed architecture, first, the data are collected via
3 Proposed Detection Models bone scanner (or available datasets), and the next phase is
to label each image. For the collected images, we split them
The purpose of this study is to construct a deep learning into training data and testing data. For the training data, we
model based on CNN-empowered TL. In this proposed strat- apply data augmentation and data preprocessing then we
egy, a specialized model is first used to mimic the significant split them to training and validation. Training the model is
International Journal of Computational Intelligence Systems (2024) 17:241 Page 11 of 33 241

achieved using the training data, afterwards we calculate the expansive and diverse medical image datasets encompassing
model performance. In the testing phase, we test the model various musculoskeletal conditions.
after applying the same preprocessing steps on the images. The essence of transfer learning lies in its transformative
The goal of the testing phase is to predict the knee condi- impact on model adaptation and generalization. By fine-
tion through the knee image and classify it into one of three tuning pre-trained models on knee X-ray datasets specific to
classes: normal, osteopenia or osteoporosis. osteoporosis, we imbue these models with a nuanced under-
The proposed methodology initiates with meticulous data standing of bone density variations and related pathologies
curation, involving the collection of datasets and precise observed in knee imaging. This nuanced understanding is
labeling of knee images, as detailed in Sect. 4. This forms crucial for accurate and sensitive detection of osteoporotic
the groundwork for subsequent model training. The data are manifestations, facilitating early diagnosis and intervention.
then stratified into training and testing subsets, followed by a Moreover, transfer learning serves us in accelerating the
comprehensive treatment of the training data. This treatment learning process and convergence during model training.
includes sophisticated techniques such as augmentation and The transfer of knowledge from source tasks, such as bone
preprocessing, as outlined in Sect. 4.1, aimed at enhancing density analysis or musculoskeletal imaging, empowers the
model generalization. model to discern subtle yet clinically significant features in
To address the challenge of training deep learning models knee X-rays, thereby enhancing diagnostic precision and
from scratch with a vast amount of data, we opt for transfer efficacy. Beyond its immediate impact on diagnostic perfor-
learning—a technique widely used in deep learning. Trans- mance, transfer learning fosters a dynamic learning para-
fer learning constructs new models by reusing previously digm characterized by continual adaptation and refinement.
learned ones, leveraging prior task expertise to improve gen- In essence, transfer learning stands as a cornerstone in
eralization. Knee X-ray images, sourced from various chan- our approach to knee osteoporosis diagnosis using deep
nels as discussed in Sect. 4.1, undergo data augmentation to learning, embodying a fusion of domain-specific expertise
expand the image dataset and enhance model training accu- and broad-based knowledge derived from diverse medical
racy. The dataset is subsequently divided into training and imaging domains. This fusion not only elevates diagnostic
testing components. The training dataset is employed to train accuracy but also instills resilience and adaptability in our
CNN classification models, while the testing dataset assesses models, positioning them at the forefront of precision medi-
the performance of these trained models. The training phase cine in musculoskeletal health.
involves a bifurcation into training and validation data, cul-
minating in the deployment of a sophisticated model for 3.2 VGG‑19 for Multiclass Model
predicting knee conditions. This intricate classification task
distinguishes among three classes: normal knee, osteopenia VGG-19 is a convolutional neural network (CNN) architec-
knee, and osteoporosis knee, providing nuanced diagnostic ture that was developed by researchers at the Visual Geom-
insights. Following the exploration of multiclass classifica- etry Group (VGG) at the University of Oxford [40, 41]. It
tion models, the subsequent section introduces multi-classes is an extension of the VGG-16 architecture and was intro-
classification models. The framework’s robustness is then duced in the 2014 paper titled “Very Deep Convolutional
validated through rigorous performance evaluation on the Networks for Large-Scale Image Recognition”. VGG-19 is
test dataset, ensuring the model’s effectiveness in accurately known for its simplicity and uniformity in design. It consists
classifying knee X-ray images across diverse cases. Detailed of 19 layers, including 16 convolutional layers and 3 fully
results are presented in Sect. 5, establishing the model’s reli- connected layers. All the convolutional layers in VGG-19
ability in clinical applications. have a 3 × 3 filter size and a stride of 1, and they are followed
by rectified linear unit (ReLU) activation functions. The max
3.1 Utilizing Transfer Learning in Our Models pooling operation with a 2 × 2 window and a stride of 2 is
applied after every two convolutional layers, reducing the
Transfer learning is a pivotal strategy in the realm of knee spatial dimensions of the feature maps. In our system, we
osteoporosis diagnosis, where deep learning models are used VGG-19 for multiclass classification, where the clas-
endowed with the ability to transcend their original train- sification result is: normal, osteopenia or osteoporosis as
ing domains and assimilate knowledge from broader medi- shown in Fig. 4.
cal imaging contexts. This methodology capitalizes on the
wealth of information encoded within pre-trained neural 3.3 VGG‑16 for Multiclass Model
network architectures, particularly convolutional neural
networks (CNNs), which have been extensively trained on VGG-16 represents a convolutional neural network (CNN)
architecture recognized as one of the most advanced vision
241 Page 12 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 4  Architecture of VGG-19 for multiclass

Fig. 5  Architecture of VGG-16 for multiclass model

model designs to date. Widely acknowledged, VGG-16 3.4 InceptionNet for Multiclass Model
serves as a foundational architecture for diverse computer
vision applications, encompassing tasks such as image clas- InceptionNet, introduced by Google in 2014, stands as a
sification and object detection, and image segmentation. Its convolutional neural network architecture distinguished
pre-trained weights are often used as initialization for other for its utilization of inception modules—collections of lay-
models or as a feature extractor for transfer learning. The ers adept at capturing both local and global features from
details of the VGG-16 model for multiclass problem are input data [34]. Engineered with the aim of enhanced effi-
given in Fig. 5. ciency and quicker training compared to alternative deep
International Journal of Computational Intelligence Systems (2024) 17:241 Page 13 of 33 241

Fig. 6  Architecture of InceptionNet for multiclass model

convolutional neural networks, InceptionNet has found dataset comprising over a million images sourced from the
application in tasks such as image classification and object ImageNet database [35]. The term “ResNet” is derived
detection, and face recognition and has been the basis for from “Residual Network,” delineating a specific category
popular neural network architectures such as Inception-v4 of convolutional neural networks (CNN) detailed in the
and Inception-ResNet. The details of the InceptionNet model 2015 paper “Deep Residual Learning for Image Recogni-
for multiclass problem are given in Fig. 6. tion” by He Kaiming, Zhang Xiangyu, Ren Shaoqing, and
Sun Jian. Commonly employed in computer vision applica-
3.5 XceptionNet for Multiclass Model tions, CNNs are integral to ResNet-50, featuring a structure
of 50 layers, including 48 convolutional layers, one Max-
XceptionNet, also known as Extreme Inception, is a deep Pool layer, and one average pool layer. Residual neural net-
convolutional neural network architecture that was intro- works, a subtype of artificial neural networks (ANN), are
duced in 2016 by François Chollet, the creator of the Keras constructed by stacking residual blocks. The details of the
deep learning library [35]. XceptionNet is designed to per- ResNet-50 model for multiclass problem in given in Fig. 8.
form image classification tasks and is known for its excep-
tional performance and efficiency. The name “Xception” is 3.7 AlexNet for Multiclass Model
a combination of “Extreme” and “Inception” because the
architecture is an extension of the Inception architecture, The model comprises convolutional layers with ReLU acti-
which was introduced by Google researchers. The Inception vation and max pooling, capturing hierarchical features.
architecture was originally designed to improve computa- An adaptive average pooling layer ensures consistent input
tional efficiency using multiple parallel convolutional layers size, and a sequence of fully connected layers with dropout
of different sizes. The details of the XceptionNet model for and ReLU forms the classifier [33]. The forward method
multiclass problem are given in Fig. 7. orchestrates the flow of input data through these layers. This
implementation is versatile for image classification tasks,
3.6 ResNet for Multiclass Model with the flexibility to adjust parameters such as the number
of classes for specific use cases. The details of the AlexNet
ResNet-50 represents a convolutional neural network char- model for multiclass problem are given in Fig. 9.
acterized by its depth of 50 layers. It allows for the loading
of a pre-trained version, having been trained on an extensive
241 Page 14 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 7  Architecture of XceptionNet for multiclass model

Fig. 8  Architecture of ResNet for multiclass model

International Journal of Computational Intelligence Systems (2024) 17:241 Page 15 of 33 241

Fig. 9  Architecture of AlexNet for multiclass model

Fig. 10  Architecture of custom CNN for multiclass model

3.8 Custom CNN for Multiclass Model from 128 to 512, with kernel sizes from 1 × 1 to 8 × 8. Batch
normalization is applied after each convolutional layer. The
A custom CNN refers to a CNN architecture that is tailored output layer has 3 units with softmax activation. The model
to solve a specific task or problem. It involves designing the has a total of approximately 15 million parameters. The
network architecture by choosing the appropriate number model structure is depicted in Fig. 10.
and type of layers, as well as their configurations, based on
the characteristics of the input data and the objectives of the
task. The proposed model is a deep convolutional neural 3.9 VGG‑19 for Binary Model
network (CNN) for image classification. It consists of 13
convolutional layers, 4 max pooling layers, and 3 fully con- VGG-19 is also employed in our work for binary class clas-
nected layers. The convolutional layers have filters ranging sification, where the result is normal and osteoporosis. The
details of the VGG-19 model for binary class problem are
given in Fig. 11.
241 Page 16 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 11  Architecture of VGG-19 for binary classes’ model

Fig. 12  Architecture of custom CNN for binary classes

3.10 Custom CNN for Binary Classes to acquire the multiclass dataset. Our collection currently
includes 793 pictures for osteoporosis, 374 for osteopenia,
A custom CNN is also built for binary classes’ classification and 780 for healthy (normal) patients. The datasets details
problem. The details of the custom CNN model for binary are freely accessible via the URLs [23–26].
classes’ problem in given in Fig. 12. In our work, we use two levels of data augmentation.
The first level’s goal is to increase the number of images in
the osteopenia class category, which is now lower than the
4 Dataset Description other two classes (normal and osteoporosis, as shown in
Fig. 13. The second level of data augmentation is applied
Instead of utilizing a single dataset to train our models, we to the entire dataset in order to enhance the size of the
construct our gathered dataset [22], which is made up of dataset to accommodate the deep learning models’ learn-
four datasets [23–26]. Figure 13 depicts the method we used ing process.
International Journal of Computational Intelligence Systems (2024) 17:241 Page 17 of 33 241

Normal 372

Osteoporosis
372
Dataset 1 - Binary Dataset from kaggle

Normal 186

Osteoporosis Normal 780

186
Osteopenia 374
Dataset 2 - Binary Dataset from kaggle [24]

Normal 186 Osteoporosis 793

Osteoporosis
186
Dataset 5 - Our collected dataset [22]
Dataset 3 - Binary Dataset from kaggle [25]

Normal 36

Osteopenia 154 Data

Osteoporosis
Dataset 4 - Multiclass Dataset from Mendeley data [26]

Fig. 13  Building our dataset from 4 different datasets

Algorithm 1  Proposed dataset collection algorithm

241 Page 18 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

(a)

X- ray Image Classification

Normal

Osteopenia

Osteoporosis

(b)

Fig. 14  a Dataset images augmentation using flipping, scaling, and rotation functions, b examples of dataset images

4.1 Data Augmentation diverse training dataset. The resulting images showcase

the variations introduced by each augmentation technique,
In this section, we delve into the practical implementa- highlighting their potential to enhance a neural network’s
tion of image data augmentation using the Python Imaging ability to handle diverse scenarios.
Library (Pillow). Figure 14 depicts an illustration of the It is worth noting that the parameters, such as the scal-
result of the used three fundamental augmentation tech- ing factor and rotation angle, can be adjusted based on the
niques: flipping, scaling, and rotation. The implementation specific requirements of the dataset and the targeted appli-
emphasizes the simplicity and versatility of data augmen- cation. Fine-tuning these parameters allows for a tailored
tation. These transformations, when applied individually augmentation strategy that aligns with the characteristics
or in combination, contribute to the creation of a more of the data.
International Journal of Computational Intelligence Systems (2024) 17:241 Page 19 of 33 241

5 Experiments and Results training and validating our models, ADAMAX optimizer
and suitable fitness functions were used, where each model
This section presents an evaluation of the results achieved ran around 14–100 epochs according to the model type (as
by the proposed system in osteoporosis detection using knee will be shown in results figures) and a batch size of 64. Only
X-ray images. The evaluation includes metrics such as accu- in the custom CNN models we used different number of
racy, AUC, precision, F1-score, recall, support, and error epochs (up to 20), dropout and batch size values to adjust
rate. The various classification models in this paper were the performance of the model results. It was compiled with
implemented using Python 3 and the Keras framework. We categorical cross-entropy loss and SGD optimizer with a
utilized the Google Colab pro version with proper RAM learning rate of 0.001.
size, and a graphical processing unit (GPU) processor assis- In the multiclass models, which classify the images into
tance. Python Imaging Library (Pillow) was used to raise one of three classes: normal, osteopenia, and osteoporosis,
the number of images in each class after merging the dataset the dataset was split into 80% for training and 20% for test-
from the different sources. All images were processed by ing, utilizing a learning rate of 0.001. In the binary models,
ImageDataGenerator class in Keras to perform preprocess- in which the classes are: normal and osteoporosis, the data-
ing operations, such as resizing, and normalization. The set comprises a total of 1573 images, with 793 for osteopo-
generated images were fed into our proposed binary and rosis and 780 for healthy cases. This dataset was also split
multiclass classification deep learning models. into 80% for training and 20% for testing, utilizing a learn-
We have conducted two types of experiments accord- ing rate of 0.001. The architecture of the models is sum-
ing to the output of the models, as we have binary clas- marized in Table 2. The prediction steps is given in details
sification models and multiclass classification models. For in Algorithm 2.

Algorithm 2  Proposed prediction algorithm

241 Page 20 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Table 2  The architecture

details of the trained models
for osteoporosis detection using
knee X-ray images
International Journal of Computational Intelligence Systems (2024) 17:241 Page 21 of 33 241

Table 3  Performance metrics formulas

Measure Formula Definition

Precision (PR) TP Precision is used to measure the positive patterns that are correctly predicted from the total
TP+FP
predicted patterns in a positive class
Recall/sensitivity (RE) TP Recall is used to measure the fraction of positive patterns that are correctly classified
TP+FN
Accuracy (AC) TP + TN The accuracy metric measures the ratio of correct predictions over the total number of
TP + TN + FP + FN
instances evaluated
Error (ER) 1-Accuracy Misclassification error measures the ratio of incorrect predictions over the total number of
instances evaluated
Macro-average “for Precision” The average of the precision and recall of the system on various n classes. It is used for
∑n
P i ∕n
∑i=1 knowing how the system acts overall across the sets of data
“for Recall”
n
R
i=1 i ∕n
Micro-average TP1+ TP2 Summation of the individual true positives, false positives, and false negatives of the system
TP1+TP2+FP1+FP2
for various sets and then applies them to get the statistics. It can be a useful measure when
“for precision” the size of dataset varies
TP1+ TP2
TP1+ TP2+ FN1+ FN2
“for Recall”
F-measure 2*P+R Metric for merge both recall and precision into a single score that take both properties
P∗R

Fig. 15  VGG-19 results for multiclass classification

5.1 Evaluation Metrics Table 3. Finally, the run time of knee osteoporosis detec-
tion algorithms should be measured using the second unit.
The subsequent deep learning classification metrics were Through these performance metrics, we utilized many of
employed to gain a deeper insight into the models’ perfor- those mentioned in Table 3, such as Accuracy, Recall, Pre-
mance across the two different classification approaches: cision, and F-measure, to confirm the effectiveness of the
binary and multiclass. During the next experiments, the models we developed.
evaluation parameters such as accuracy, error, recall, and
precision will be calculated. Then, F1-score, micro-average 5.2 Models Classification Results
and macro-average related to precision and recall will be
measured. The confusion matrix is used to calculate the Figures 15, 16, 17, 18, 19, 20, and 21 illustrate the perfor-
values of these parameters. Various formulas are used as mance results of our trained models for multiclass models
a summarization of the confusion matrix as depicted in on the collected dataset. The figures present the performance
 241 Page 22 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 16  Custom CNN results for multiclass classification

Fig. 17  VGG-16 results for multiclass classification

of various deep learning models on classifying knee X-ray Figures 22 and 23 also illustrate the performance results
images. The models were trained and tested on the collected of VGG-19 and custom CNN trained models for binary
dataset [22]. The performance of each model is measured models on the collected dataset. These binary models are
by its classification accuracy. In the multiclass setting, the arranged from the highest to the lowest accuracy, showcas-
models used are VGG-19, custom CNN, VGG-16, Incep- ing their performance in descending order.
tionNet, XceptionNet 3, ResNet, and AlexNet. These models Table 4 augments the comparison of our models accura-
are arranged from the highest to the lowest accuracy, show- cies, precision, F1-score and recall for binary and multi-
ing their performance in descending order. class classification, while Table 5 gives a comparison of our
International Journal of Computational Intelligence Systems (2024) 17:241 Page 23 of 33 241

Fig. 18  InceptionNet results for multiclass classification

Fig. 19  XceptionNet results for multiclass classification

 241 Page 24 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Fig. 20  ResNet-50 results for multiclass classification

Fig. 21  AlexNet results for

multiclass classification

models accuracies for binary and multiclass against state-of- CNN performs slightly lower at 95.6%. Based on these fig-
the-art models [16–18, 21, 27]. ures, it can be concluded that VGG-19 is the best-performing
From the figures and the tables, we can conclude that the model for classifying knee X-ray images in both multiclass
highest classification accuracy is achieved by VGG-19 at and binary classification settings, while custom CNN also
92.0%, followed by custom CNN at 90.4%, and VGG-16 at demonstrates competitive performance.
89.9%. The other models have relatively lower performance,
with InceptionNet at 89.7%, XceptionNet 3 at 87.9%, ResNet
at 81.4%, and AlexNet at 78.4%. 5.3 Evaluation of Training Results
In the binary classification setting, only two models are
compared: VGG-19 and custom CNN. VGG-19 achieves an As well as the accuracy comparison, Fig. 24 displays the
impressive classification accuracy of 97.5%, while custom confusion matrix for the six classification models, depicting
International Journal of Computational Intelligence Systems (2024) 17:241 Page 25 of 33 241

Fig. 22  VGG-19 results for binary class classification

Fig. 23  Custom CNN results for binary class classification

the system’s predictions for our multiclass models on the a clear visual representation of their performance in binary
dataset. These models are arranged from the highest to the classification.
lowest accuracy, providing a visual representation of their Tables 6 and 7 show the training and testing results of
performance. multiclass custom CNN with different dropout and batch
Figure 25 displays the confusion matrix for the two clas- size values, while Tables 8 and 9 show the training and test-
sification models, depicting the system’s predictions for ing results of binary class custom CNN with different drop-
our binary models on the dataset. These binary models are out and batch size values. We can see the consistency of the
arranged from the highest to the lowest accuracy, providing model across the various values of dropout and batch size
241 Page 26 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Table 4  Comparison of accuracy, precision, F1-score and recall of our models for binary and multiclass classification
Model Dataset Accuracy % Precision % F1-score % Recall %

VGG-19 for multiclass Our collected dataset [22] 92.0 92 92 92

Custom CNN for multiclass 90.4 90 90 90
VGG-16 for multiclass 89.9 90 90 90
InceptionNet for multiclass 89.7 90 90 90
XceptionNet 3 for multiclass 87.9 88 88 88
ResNet for multiclass 81.4 82 81 81
AlexNet for multiclass 78.4 83 78 78
VGG-19 for binary 97.5 98 97 97
Custom CNN for binary 95.6 96 96 96

Table 5  Comparison of accuracy of our models for binary and multiclass against state-of-the-art models
Authors and years Bone type Image type Classes Dataset Classifier Reported
model
accuracy

Wani & Arora, 2022 [17] Knee X-ray Multiclass Dataset from Mendeley data AlexNet 91.0%
[26] VGG-16 86.30%
ResNet 86.30%
VGG-19 84.20%
Kumar et al., 2023 [18] Knee X-ray Multiclass Dataset Inception v3 89.8%
from Mendeley data [26] Xception 90.9%
ResNet 18 91.4%
Abubakar et al., 2022 [21] Knee X-ray Binary Dataset from kaggle GoogleNet 90%
[23] VGG-16 87%
Yang, 2022 [27] Knee X-ray Binary Dataset from kaggle [23] Custom CNN 77% 71%
Late-Fusion 82%
VGG-16
Pooja and Sakshi, 2023 [16] Spine, X-ray Binary Zydus Hospital Dataset, VGG-16 78%
Hand, Dahod, Gujarat VGG-19 DenseNet-121 86%
Leg, Knee ResNet-50 93%
Inception V3 89%
90%
Our Work Knee X-ray Multiclass Our collected dataset [22] VGG-19 92.0%
Custom CNN 90.4%
VGG-16 89.9%
InceptionNet 89.7%
XceptionNet 3 87.9%
ResNet 81.4%
AlexNet 78.4%
Our Work Knee X-ray Binary classes Our collected dataset [22] VGG-19 97.5%
Custom CNN 95.6%

values, which indicates the robustness of the model. The best • Testing results of multiclass
model training and testing parameters are:
No. of epochs = 20, dropout = 0.5, batch size = 16 which
• Training results of multiclass yields loss= 0.3346 and accuracy = 91.03%

• Training results of binary class

No. of epochs = 20, without dropout, batch size = 4
which yields loss= 0.0025 and accuracy = 99.94% No. of epochs = 15, without dropout, batch size = 4
which yields loss= 9.9635e-04 and accuracy = 100%
International Journal of Computational Intelligence Systems (2024) 17:241 Page 27 of 33 241

Healthy 222 6 10 Healthy 219 6 13

Osteopenia 1 96 14 Osteopenia 3 95 13
Osteoporosis 5 11 220 Osteoporosis 14 9 213
Healthy Osteopenia Osteoporosis Healthy Osteopenia Osteoporosis

(a) VGG-19 (b) Custom CNN

Healthy 214 9 15 Healthy 209 8 21
Osteopenia 1 92 18 Osteopenia 1 96 14
Osteoporosis 7 9 220 Osteoporosis 8 8 220
Healthy Osteopenia Osteoporosis Healthy Osteopenia Osteoporosis

(c) VGG-16 (d) Inception

Healthy 219 6 13 Healthy 194 5 39

Osteopenia 3 95 13 Osteopenia 9 74 28
Osteoporosis 14 9 213 Osteoporosis 23 5 208
Healthy Osteopenia Osteoporosis Healthy Osteopenia Osteoporosis

(e) Xception (f) ResNet

Fig. 24  Confusion matrices for the 6 multiclass classification models

Healthy 231 0 Healthy 217 14

Osteoporosis 12 229 Osteoporosis 7 234
Healthy Osteoporosis Healthy Osteoporosis

(a) VGG-19 (b) Custom CNN

Fig. 25  Confusion matrices for VGG-19 and custom CNN binary classification models

• Testing results of binary class diagnosis employing advanced deep learning techniques.
Our approach, featuring transfer learning with VGG-19,
No. of epochs = 15, without dropout, batch size = 4 achieves a compelling 92.0% accuracy for multiclass clas-
which yields loss = 0.2054 and accuracy = 94.29% sification, outperforming the previous work [17] accuracy of
The custom CNN models (both multiclass and binary 84.20%. Across various architectures, our VGG-16 achieves
classes) have smaller architecture compared to VGG-19 89.9% accuracy, surpassing the 86.30% reported in Ref. [17],
model, thus have smaller number of trainable parameters. It while ResNet demonstrates competitive accuracy at 81.4%
took only 20 epochs to reach reliable accuracies compared compared to 86.30% in Ref. [17]. Although our AlexNet
to 100 epochs for VGG-19 model training. accuracy of 78.4% is lower than the results in Ref. [17] 91%,
it highlights nuanced model performance differences. We
5.4 Results’ Analysis also notice that the results in Ref. [17] are taken as the best
results in each epoch rather than the average of all epochs
This study rigorously compares the performance of as we have implemented. That was the reason to report our
several classification models for knee osteoporosis results in Table 7.
241 Page 28 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Table 6  Training results of multiclass custom CNN with different dropout and batch size values
No. of epochs Dropout Batch size Loss Accuracy Precision Recall AUC​ F1 Specificity Sensitivity

10 Without 2 0.0701 0.9756 0.9762 0.9743 0.9981 0.9752 0.9881 0.9743

4 0.0276 0.9917 0.9917 0.9917 0.9999 0.9917 0.9958 0.9917
8 0.0639 0.9737 0.9749 0.9730 0.9991 0.9740 0.9875 0.9730
16 0.1178 0.9499 0.9541 0.9486 0.9967 0.9514 0.9772 0.9486
0.2 2 0.1714 0.9345 0.9388 0.9255 0.9918 0.9321 0.9698 0.9255
4 0.1507 0.9435 0.9474 0.9371 0.9936 0.9422 0.9740 0.9371
8 0.1592 0.9428 0.9481 0.9377 0.9927 0.9428 0.9743 0.9377
16 0.2034 0.9261 0.9297 0.9178 0.9886 0.9237 0.9653 0.9178
0.5 2 0.3502 0.8626 0.8745 0.8459 0.9665 0.8599 0.9393 0.8459
4 0.3463 0.8690 0.8842 0.8536 0.9676 0.8686 0.9441 0.8536
8 0.3384 0.8606 0.8724 0.8433 0.9687 0.8576 0.9383 0.8433
16 0.3945 0.8452 0.8639 0.8317 0.9574 0.8475 0.9345 0.8317
15 Without 2 0.0088 0.9974 0.9974 0.9974 1.0000 0.9974 0.9987 0.9974
4 0.0064 0.9994 0.9994 0.9994 0.9995 0.9994 0.9997 0.9994
8 0.0116 0.9968 0.9968 0.9968 1.0000 0.9968 0.9984 0.9968
16 0.0303 0.9929 0.9929 0.9923 0.9999 0.9926 0.9965 0.9923
0.2 2 0.1228 0.9525 0.9561 0.9505 0.9955 0.9533 0.9782 0.9505
4 0.0584 0.9769 0.9800 0.9762 0.9991 0.9781 0.9900 0.9762
8 0.0987 0.9634 0.9645 0.9602 0.9969 0.9623 0.9823 0.9602
16 0.1383 0.9416 0.9469 0.9383 0.9948 0.9426 0.9737 0.9383
0.5 2 0.2416 0.9049 0.9179 0.8979 0.9838 0.9078 0.9599 0.8979
4 0.2279 0.9088 0.9854 0.8992 0.9854 0.9132 0.9650 0.8992
8 0.2363 0.9114 0.9215 0.9043 0.9846 0.9128 0.9615 0.9043
16 0.2528 0.9088 0.9197 0.8979 0.9823 0.9087 0.9608 0.8979
20 Without 2 0.0040 0.9994 0.9994 0.9994 1.0000 0.9994 0.9997 0.9994
4 0.0025 0.9994 0.9994 0.9994 1.0000 0.9994 0.9997 0.9994
8 0.0091 0.9981 0.9981 0.9981 1.0000 0.9981 0.9990 0.9981
16 0.0095 0.9994 0.9994 0.9994 1.0000 0.9994 0.9997 0.9994
0.2 2 0.0815 0.9705 0.9717 0.9698 0.9976 0.9707 0.9859 0.9698
4 0.0344 0.9917 0.9917 0.9917 0.9992 0.9917 0.9958 0.9917
8 0.0544 0.9820 0.9826 0.9801 0.9993 0.9814 0.9913 0.9801
16 0.1210 0.9525 0.9554 0.9493 0.9959 0.9523 0.9778 0.9493
0.5 2 0.1568 0.9416 0.9460 0.9345 0.9933 0.9402 0.9733 0.9345
4 0.1607 0.9383 0.9453 0.9332 0.9931 0.9392 0.9730 0.9332
8 0.1635 0.9403 0.9460 0.9345 0.9924 0.9402 0.9733 0.9345
16 0.2082 0.9197 0.9256 0.9107 0.9882 0.9181 0.9634 0.9107

Notably, our research extends beyond multiclass scenarios the adaptability and effectiveness of our proposed mod-
to include binary classifications, showcasing the efficacy of els in addressing both multiclass and binary classification
custom CNNs with accuracies of 90.4% and 95.6% in multi- challenges. As deep learning in medical image analysis
class and binary contexts, respectively. InceptionNet attains advances, our research contributes valuable insights and
a robust 89.7% accuracy in multiclass scenarios. Diverging benchmarks to propel advancements in knee osteoporosis
from Ref. [17], our explicit consideration of binary clas- diagnosis. Upon meticulous examination of the confusion
sification scenarios, particularly with VGG-19 for binary matrix calculations detailed in their research paper, it has
classes (97.5%) and custom CNN for binary classes (95.6%), come to our attention that discrepancies exist between the
underscores the versatility and discriminative capabilities of reported accuracy figures and the actual values derived from
our proposed models. the confusion matrix for several key models.
This comprehensive analysis positions our methodolo- In the case of VGG-19, the reported accuracy stands at
gies as superior in multiclass scenarios and emphasizes 84.30%, but our scrutiny of the confusion matrix reveals a
International Journal of Computational Intelligence Systems (2024) 17:241 Page 29 of 33 241

Table 7  Testing results of multiclass custom CNN with different dropout and batch size values
No. of epochs Dropout Batch size Loss Accuracy Precision Recall AUC​ F1 Specificity Sensitivity

10 Without 2 0.5349 0.8538 0.8594 0.8462 0.9439 0.8526 0.9307 0.8461

4 0.4373 0.8949 0.8943 0.8897 0.9570 0.8920 0.9474 0.8897
8 0.3899 0.9103 0.9149 0.9103 0.9566 0.9126 0.9577 0.9103
16 0.3380 0.9026 0.9117 0.9000 0.9676 0.9058 0.9564 0.9000
0.2 2 0.3896 0.9026 0.9046 0.9000 0.9657 0.9012 0.9525 0.9000
4 0.4301 0.8641 0.8630 0.8564 0.9602 0.8596 0.9320 0.8564
8 0.3701 0.8795 0.8906 0.8769 0.9678 0.8837 0.9462 0.8769
16 0.3113 0.9051 0.9162 0.8974 0.9710 0.9067 0.9590 0.8974
0.5 2 0.4226 0.8205 0.8584 0.7769 0.9507 0.8156 0.9358 0.7769
4 0.3835 0.8795 0.8910 0.8590 0.9600 0.8747 0.9474 0.8590
8 0.4137 0.8744 0.8793 0.8590 0.9535 0.8690 0.9410 0.8590
16 0.4292 0.8410 0.8464 0.8333 0.9512 0.8398 0.9244 0.8333
15 Without 2 0.5889 0.8487 0.8505 0.8462 0.9368 0.8482 0.9256 0.8461
4 0.5243 0.8923 0.8923 0.8923 0.9497 0.8923 0.9462 0.8923
8 0.4418 0.9051 0.9096 0.9026 0.9533 0.9060 0.9551 0.9026
16 0.3487 0.8949 0.8943 0.8897 0.9705 0.8920 0.9474 0.8897
0.2 2 0.4567 0.8897 0.8897 0.8897 0.9567 0.8897 0.9448 0.8897
4 0.4272 0.9051 0.9051 0.9051 0.9657 0.9051 0.9525 0.9051
8 0.3800 0.9103 0.9119 0.9026 0.9609 0.9072 0.9564 0.9026
16 0.3557 0.9077 0.9121 0.9051 0.9672 0.9086 0.9564 0.9051
0.5 2 0.4850 0.8256 0.8346 0.8154 0.9439 0.8248 0.9192 0.8153
4 0.3757 0.8667 0.8667 0.8689 0.9657 0.8678 0.9346 0.8667
8 0.3379 0.8974 0.9058 0.8872 0.9687 0.8964 0.9538 0.8872
16 0.3544 0.8923 0.9058 0.8872 0.9658 0.8964 0.9538 0.8872
20 Without 2 0.6079 0.8538 0.8579 0.8513 0.9395 0.8545 0.9294 0.8512
4 0.5602 0.8923 0.8923 0.8923 0.9496 0.8923 0.9462 0.8923
8 0.4672 0.8974 0.8974 0.8974 0.9524 0.8974 0.9487 0.8974
16 0.3905 0.8821 0.8843 0.8821 0.9632 0.8832 0.9423 0.8821
0.2 2 0.4818 0.8897 0.8941 0.8872 0.9542 0.8905 0.9474 0.8871
4 0.5583 0.8949 0.8995 0.8949 0.9531 0.8972 0.9500 0.8949
8 0.4442 0.8923 0.8920 0.8897 0.9612 0.8909 0.9462 0.8897
16 0.3575 0.8949 0.8990 0.8897 0.9686 0.8943 0.9500 0.8897
0.5 2 0.4057 0.8462 0.8494 0.8385 0.9635 0.8438 0.9256 0.8384
4 0.3565 0.8897 0.8958 0.8821 0.9711 0.8889 0.9487 0.8821
8 0.3486 0.8846 0.8950 0.8744 0.9699 0.8846 0.9487 0.8744
16 0.3346 0.9103 0.9145 0.9051 0.9693 0.9098 0.9577 0.9051

more accurate figure of 81.81%. Similarly, for VGG-16, the and multiclass classifications as it achieved 92.0% accu-
reported accuracy of 86.30% is contradicted by our calcula- racy, 92.0% recall, 92.0% F1-score, and 92.0% precision for
tions, which yield a more accurate value of 81.81%. While multiclass classification, and 97.5% accuracy, 97.0% recall,
ResNet aligns closely with the reported accuracy of 86.36%, 97.0% F1-score, and 98.0% precision for binary classifica-
a significant discrepancy is apparent in the case of AlexNet. tions. However, the proposed custom CNN has comparative
Their reported accuracy of 91% contrasts starkly with the results with lower number of parameters and lower number
rigorously derived accuracy of 77.27% from our in-depth of epochs for training. We opt that the presented deep mod-
examination of the confusion matrix. els and their results may serve as a first step toward develop-
Consequently, and from the results of the conducted ing a knee problems diagnosis system from X-ray images,
experiments after validating the accuracy of the proposed according to the high accuracy reached by the models.
models, we recommend the VGG-19 model for both binary
241 Page 30 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

Table 8  Training results of binary class custom CNN with different dropout and batch size values
No. of epochs Dropout Batch size Loss Accuracy Precision Recall AUC​ F1 specificity Sensitivity

10 Without 2 0.0183 0.9921 0.9921 0.9921 0.9998 0.9921 0.9921 0.9921

4 0.0039 0.9992 0.9992 0.9992 1.0000 0.9992 0.9992 0.9992
8 0.0235 0.9913 0.9913 0.9913 0.9998 0.9913 0.9913 0.9913
16 0.0536 0.9762 0.9762 0.9762 0.9987 0.9762 0.9762 0.9762
0.2 2 0.1063 0.9610 0.9610 0.9610 0.9928 0.9610 0.9610 0.9610
4 0.0370 0.9881 0.9881 0.9881 0.9993 0.9881 0.9881 0.9881
8 0.0565 0.9833 0.9833 0.9833 0.9970 0.9833 0.9833 0.9833
16 0.0875 0.9730 0.9730 0.9730 0.9944 0.9730 0.9730 0.9730
0.5 2 0.3502 0.8626 0.8745 0.8459 0.9665 0.9054 0.9054 0.9054
4 0.3463 0.8690 0.8842 0.8536 0.9676 0.9388 0.9388 0.9388
8 0.3384 0.8606 0.8724 0.8433 0.9687 0.9372 0.9372 0.9372
16 0.3945 0.8452 0.8639 0.8317 0.9574 0.9245 0.9245 0.9245
15 Without 2 0.0011 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
4 9.9635e-04 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
8 0.0056 0.9992 0.9992 0.9992 1.0000 0.9992 0.9992 0.9992
16 0.0104 0.9976 0.9976 0.9976 1.0000 0.9976 0.9976 0.9976
0.2 2 0.0487 0.9833 0.9833 0.9833 0.9984 0.9833 0.9833 0.9833
4 0.0184 0.9968 0.9997 0.9968 0.9997 0.9968 0.9968 0.9968
8 0.0284 0.9913 0.9913 0.9913 0.9995 0.9913 0.9913 0.9913
16 0.0669 0.9730 0.9642 0.9642 0.9918 0.9730 0.9730 0.9730
0.5 2 0.2416 0.9049 0.9179 0.8979 0.9838 0.9523 0.9523 0.9523
4 0.2279 0.9088 0.9278 0.8992 0.9854 0.9754 0.9754 0.9754
8 0.2363 0.9114 0.9215 0.9043 0.9846 0.9587 0.9587 0.9587
16 0.2528 0.9088 0.9197 0.8979 0.9823 0.9531 0.9531 0.9531
20 Without 2 3.9341e-04 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
4 6.4387e-04 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
8 0.0035 0.9984 0.9984 0.9984 1.0000 0.9984 0.9984 0.9984
16 0.0061 0.9984 0.9984 0.9984 1.0000 0.9984 0.9984 0.9984
0.2 2 0.0259 0.9897 0.9897 0.9897 0.9997 0.9897 0.9897 0.9897
4 0.0046 0.9992 0.9992 0.9992 1.0000 0.9992 0.9992 0.9992
8 0.0092 0.9968 0.9968 0.9968 1.0000 0.9968 0.9968 0.9968
16 0.0623 0.9769 0.9769 0.9769 0.9979 0.9769 0.9769 0.9769
0.5 2 0.1568 0.9416 0.9460 0.9345 0.9933 0.9722 0.9722 0.9722
4 0.1607 0.9383 0.9453 0.9332 0.9931 0.9825 0.9825 0.9825
8 0.1635 0.9403 0.9460 0.9345 0.9924 0.9754 0.9754 0.9754
16 0.2082 0.9197 0.9256 0.9107 0.9882 0.9650 0.9650 0.9650

6 Conclusion a learning and classification model was presented, which

is based on integrating transfer learning from CNN archi-
The diagnosis of knee osteoarthritis using X-ray image tectures based on deep learning. A new mechanism for
currently faces difficulties in automated and accurate clas- collecting the data set was also presented based on the
sification. In this paper, we propose a new deep learning image enlargement method provided by different sources.
method to classify different osteoarthritis diseases of the The detection has been applied to both binary and multiple
knee. The main objective contribution of this work is to classes to classify X-ray images of the knee.
presented an exceptionally automated procedure for osteo- In the introduced study, we have evaluated and com-
arthritis diagnosis utilizing deep learning. In this research, pared the performance of popular CNN architectures,
International Journal of Computational Intelligence Systems (2024) 17:241 Page 31 of 33 241

Table 9  Testing results of binary class custom CNN with different dropout and batch size values
No. of epochs Dropout Batch size Loss Accuracy Precision Recall AUC​ F1 Specificity Sensitivity

10 Without 2 0.2374 0.9397 0.9397 0.9397 0.9625 0.9397 0.9397 0.9397

4 0.1876 0.9397 0.9397 0.9397 0.9840 0.9397 0.9397 0.9397
8 0.1261 0.9429 0.9429 0.9429 0.9897 0.9429 0.9429 0.9429
16 0.1594 0.9429 0.9429 0.9429 0.9870 0.9429 0.9429 0.9429
0.2 2 0.2211 0.9270 0.9270 0.9270 0.9721 0.9270 0.9270 0.9270
4 0.2554 0.9333 0.9333 0.9333 0.9725 0.9333 0.9333 0.9333
8 0.2443 0.9333 0.9333 0.9333 0.9715 0.9333 0.9333 0.9333
16 0.1446 0.9270 0.9270 0.9270 0.9886 0.9270 0.9270 0.9270
0.5 2 0.3074 0.8540 0.8540 0.8540 0.9410 0.8540 0.8540 0.8540
4 0.1892 0.9238 0.9238 0.9238 0.9785 0.9238 0.9238 0.9238
8 0.2360 0.9175 0.9175 0.9175 0.9695 0.9175 0.9175 0.9175
16 0.2196 0.8857 0.8857 0.8857 0.9720 0.8857 0.8857 0.8857
15 Without 2 0.2643 0.9333 0.9333 0.9333 0.9607 0.9333 0.9333 0.9333
4 0.2054 0.9429 0.9429 0.9429 0.9796 0.9429 0.9429 0.9429
8 0.1523 0.9365 0.9365 0.9365 0.9915 0.9365 0.9365 0.9365
16 0.1443 0.9333 0.9333 0.9333 0.9916 0.9333 0.9333 0.9333
0.2 2 0.2383 0.9175 0.9175 0.9175 0.9695 0.9175 0.9175 0.9175
4 0.2676 0.9175 0.9175 0.9175 0.9797 0.9175 0.9175 0.9175
8 0.2183 0.9429 0.9429 0.9429 0.9742 0.9429 0.9429 0.9429
16 0.1300 0.9397 0.9397 0.9397 0.9881 0.9397 0.9397 0.9397
0.5 2 0.4407 0.8381 0.8381 0.8381 0.9286 0.8381 0.8381 0.8381
4 0.1551 0.9365 0.9857 0.9365 0.9365 0.9365 0.9365 0.9365
8 0.1719 0.9111 0.9111 0.9111 0.9841 0.9111 0.9111 0.9111
16 0.1416 0.9238 0.9238 0.9238 0.9899 0.9238 0.9238 0.9238
20 Without 2 0.2728 0.9333 0.9333 0.9333 0.9629 0.9333 0.9333 0.9333
4 0.2184 0.9429 0.9429 0.9429 0.9723 0.9429 0.9429 0.9429
8 0.1521 0.9365 0.9365 0.9365 0.9919 0.9365 0.9365 0.9365
16 0.1664 0.9365 0.9365 0.9365 0.9909 0.9365 0.9365 0.9365
0.2 2 0.2248 0.9397 0.9397 0.9397 0.9740 0.9397 0.9397 0.9397
4 0.2737 0.9175 0.9175 0.9175 0.9767 0.9175 0.9175 0.9175
8 0.2534 0.9429 0.9429 0.9429 0.9669 0.9429 0.9429 0.9429
16 0.1636 0.9333 0.9333 0.9333 0.9858 0.9333 0.9333 0.9333
0.5 2 0.2750 0.8794 0.8794 0.8794 0.9700 0.8794 0.8794 0.8794
4 0.2066 0.9270 0.9270 0.9270 0.9826 0.9270 0.9270 0.9270
8 0.1821 0.9206 0.9206 0.9206 0.9851 0.9206 0.9206 0.9206
16 0.1794 0.9238 0.9238 0.9238 0.9867 0.9238 0.9238 0.9238

namely VGG-19 for binary, custom CNN for binary, VGG- with 97.5% accuracy and was achieved by VGG-19 for
19 for multi-classes, custom CNN for multiclass, VGG- multiclass with 92.0% accuracy. The outcomes derived
16 for multiclass, InceptionNet, XceptionNet, ResNet from all convolutional neural networks (CNNs) exhibited
and AlexNet in diagnosing osteoporosis from knee X-ray commendable diagnostic performance. These findings
images. The X-ray images used were taken from the cus- suggest that employing transfer learning with CNNs for
tom dataset that was classified into normal, osteopenia, and osteoporosis diagnosis from knee X-rays can represent a
osteoporosis group with the help of a medically. Our dem- cost-effective and readily accessible diagnostic tool. In
onstration highlighted the successful application of trans- subsequent efforts, additional data collection, particularly
fer learning from the object detection domain to achieve from normal and osteoporotic subjects, could enhance the
effective knee joint segmentation. The results show that dataset’s comprehensiveness. Furthermore, exploring the
the best performance was achieved by VGG-19 for binary correlation between knee osteoporosis and osteoporosis
 241 Page 32 of 33 International Journal of Computational Intelligence Systems (2024) 17:241

at other anatomical sites could contribute to the devel- 5. Yadav, S.S., Jadhav, S.M.: Deep convolutional neural network
opment of a universal osteoporosis diagnostic system. based medical image classification for disease diagnosis. J. Big
Data 6(1), 1–18 (2019)
Lastly, there is potential to construct a system capable of 6. Mahmud, I., Mamun, M., Abdelgawad, A.: A deep analysis of
detecting osteoporosis by integrating clinical factors with brain tumor detection from mr images using deep learning net-
image analysis. Importantly, our study underscores that, works. Algorithms 16(4), 176 (2023). [Link]
with adept model structuring and parameterization, a deep a1604​0176
7. Arafa, D.A., Moustafa, H.E.D., Ali, H.A., Ali-Eldin, A.M.T.,
convolutional network can mitigate computational com- Saraya, S.: A deep learning framework for early diagnosis of
plexities without compromising robustness, particularly Alzheimer’s disease on MRI images. In: Multimedia Tools and
in managing extensive datasets. Applications, pp. 1–33. Springer, Cham (2022)
8. Balaha, H.M., Balaha, M.H., Ali, H.A.: Hybrid COVID-19 seg-
mentation and recognition framework (HMB-HCF) using deep
Author Contributions AS and HAA: conceptualization of this study, learning and genetic algorithms. Artif. Intell. Med. 119, 102–156
adding the basic ideas writing and editing the manuscript. DS and YY: (2021)
formal analysis, methodology, software, validation, writing original 9. Shalaby, E., ElShennawy, N., Sarhan, A.: Utilizing deep learn-
draft preparation SY, ZE, GS, NM, EM, AND SI: investigation, meth- ing models in CSI-based human activity recognition. In: Neural
odology, validation. computing and applications, pp. 1–18. Springer, Cham (2022)
10. Sathyavathi, S., Baskaran, K.R.: An intelligent human age pre-
Funding Open access funding provided by The Science, Technology & diction from face image framework based on deep learning algo-
Innovation Funding Authority (STDF) in cooperation with The Egyp- rithms. Inform. Technol. Control 52(1), 245–257 (2023)
tian Knowledge Bank (EKB). 11. Lee, K., Jung, S.K., Ryu, J.J., Shin, S.W., Choi, J.: Evaluation
of transfer learning with deep convolutional neural networks for
Data Availability The dataset used in this paper is publicly available screening osteoporosis in dental panoramic radiographs. J. Clin.
online. Med. 9(2), 392 (2020)
12. Feng, S., Lin, S., Chiang, Y., Lu, M., Chao, Y.: Deep learning-
based hip X-ray image analysis for predicting osteoporosis. Appl.
Declarations Sci. 14(1), 133 (2023)
13. Zhang, B., Yu, K., Ning, Z., Wang, K., Dong, Y., Liu, X., Liu,
Conflict of Interest The authors declare that they have no known com- S., Wang, J., Zhu, C., Yu, Q., Duan, Y.: Deep learning of lumbar
peting financial interests or personal relationships that could have ap- spine X-ray for osteopenia and Osteoporosis screening: a multi-
peared to influence the work reported in this paper. center retrospective cohort study. Bone 140, 115561 (2020)
14. Chen, Z., Zheng, H., Duan, J., Wang, X.: GLCM-based FBLS: a
Open Access This article is licensed under a Creative Commons novel broad learning system for knee osteopenia and osteoporosis
Attribution-NonCommercial-NoDerivatives 4.0 International License, screening in athletes. Appl. Sci., MDPI 13(20), 11150 (2023)
which permits any non-commercial use, sharing, distribution and repro- 15. Sebro, R., De la Garza-Ramos, C.: Machine learning for oppor-
duction in any medium or format, as long as you give appropriate credit tunistic screening for Osteoporosis from CT scans of the wrist and
to the original author(s) and the source, provide a link to the Creative forearm. Diagnostics, MDPI 12(3), 691 (2022)
Commons licence, and indicate if you modified the licensed material. 16. Dodamani, P.S., Danti, A.: Transfer learning-based osteoporosis
You do not have permission under this licence to share adapted material classification using simple radiographs. Int. J. Onl. Biomed. Eng.
derived from this article or parts of it. The images or other third party 19(8), 66 (2023)
material in this article are included in the article’s Creative Commons 17. Wani, M.I., Arora, S.: Osteoporosis diagnosis in knee X-rays by
licence, unless indicated otherwise in a credit line to the material. If transfer learning based on convolution neural network. Multim.
material is not included in the article’s Creative Commons licence and Tools Appl. 82(9), 14193–14217 (2023)
your intended use is not permitted by statutory regulation or exceeds 18. Kumar, S., Goswami, P., Batra, S.: Fuzzy rank-based ensemble
the permitted use, you will need to obtain permission directly from model for accurate diagnosis of osteoporosis in knee radiographs.
the copyright holder. To view a copy of this licence, visit [Link] IJACSA Int. J. Adv. Comput. Sci. Appl. (2023). [Link] oi.o​ rg/1​ 0.​
[Link]/licenses/by-nc-nd/4.0/. 14569/​IJACSA.​2023.​01404​30
19. Ashames, M.M., Ceylan, M., Jennane, R.: Deep transfer learning
and majority voting approaches for Osteoporosis classification.
Int. J. Intell. Syst. Appl. Eng. (2021). [Link]
ijisae.​20214​73646
References 20. Dzierżak, R., Omiotek, Z.: Application of deep convolutional
neural networks in the diagnosis of Osteoporosis. Sens., MDPI
22(21), 8189 (2022)
1. Cooper, C., Campion, G., Melton, L.: Hip fractures in the 21. Abubakar, U.B., Boukar, M.M., Adeshina, S., Dane, S.: Transfer
elderly: a world-wide projection. Osteoporos Int. 2(6), 285–289 learning model training time comparison for Osteoporosis classifi-
(1992) cation on knee radiograph of RGB and grayscale images. WSEAS
2. [Link] Trans. Electron. 13, 45–51 (2022)
stics 22. [Link]
3. [Link] 0a19d​88074​cad74​e8f0e​0cfa2​b236a​6904
4. Li, Q., Cai, W., Wang, X., Zhou, Y., Feng, D.D., Chen, M.: 23. [Link]
“Medical image classification with convolutional neural net- 24. [Link]
work," Proceedings of the IEEE 13th International Conference is-​knee-​datas​et-​prepr​ocess​ed128​x256
on Control Automation Robotics & Vision (ICARCV), pp. 25. [Link] ww.k​ aggle.c​ om/d​ atase​ ts/s​ tevep​ ython/O
​ steop​ orosi​ s-k​ nee-​
844–848, (2014) xray-​datas​et
International Journal of Computational Intelligence Systems (2024) 17:241 Page 33 of 33 241

26. [Link] Conference on Neural Information Processing, pp. 535–545.

27. Yang, T.S.: "Recognition and classification of knee osteoporosis Springer, Cham (2019)
and osteoarthritis severity using deep learning techniques," (Doc- 37. Yu, X., Wang, S.-H.: Abnormality diagnosis in mammograms
toral dissertation, Dublin, National College of Ireland), (2022) by transfer learning based on ResNet18. Fund. Inform. 168(2),
28. Vishnu, T., Saranya, K., Arunkumar, R., Devi, M.G.,:"Efficient 219–230 (2019)
and early detection of Osteoporosis using trabecular region," Pro- 38. Ebrahimi-Ghahnavieh, A., Luo, S., Chiong, R.: "Transfer learning
ceedings of the IEEE Online International Conference on Green for Alzheimer's disease detection on MRI images," Proceedings
Engineering and Technologies (IC-GET), pp. 1–5, (2015) of the IEEE International Conference on Industry 4.0, Artificial
29. Bengio, Y.: "Deep learning of representations for unsupervised Intelligence, and Communications Technology (IAICT), pp. 133–
and transfer learning," Proceedings of ICML workshop on unsu- 138, (2019)
pervised and transfer learning, pp. 17–36, (2012) 39. Howard, J., Gugger, S.: A layered API for deep learning. Inform.,
30. Hosny, K.M., Kassem, M.A., Foaud, M.M.: Classification of skin MDPI 11(2), 108 (2020)
lesions using transfer learning and augmentation with Alexnet. 40. Khan, Z., Khan, F.G., Khan, A., Rehman, Z., Shah, S., Qummar,
PLoS ONE 14(5), e0217293 (2019) S., Pack, S.: Diabetic retinopathy detection using VGG-NIN a
31. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classifica- deep learning architecture. IEEE Access 9, 61408–61416 (2021)
tion with deep convolutional neural networks. Commun. ACM 25, 41. Militante, S.V.: "Malaria disease recognition through adaptive
1097–1105 (2012) deep learning models of convolutional neural network," Proceed-
32. Lu, S., Wang, S.-H., Zhang, Y.-D.: Detection of abnormal brain ings of the IEEE 6th International Conference on Engineering
in MRI via improved AlexNet and ELM optimized by chaotic Technologies and Applied Sciences (ICETAS), pp. 1–6, (2019)
bat algorithm. Appl. MDPI (2020). [Link] 42. Simonyan, K., Zisserman, A.: "Very deep convolutional networks
s00521-​020-​05082-4 for large-scale image recognition," arXiv preprint arXiv:​1409.​
33. Salih, S.Q., Hawre, Kh., et al.: "Modified Alexnet convolution 1556, (2014)
neural network for Covid-19 detection using chest X-ray images. 43. Tutsoy, O.: Graph theory based large-scale machine learning with
KJAR (2020). [Link] multi-dimensional constrained optimization approaches for exact
34. Guo, M., Du, Y.: "Classification of thyroid ultrasound standard epidemiological modeling of pandemic diseases. IEEE Trans. Pat-
plane images using ResNet-18 Networks," Proceedings of the tern Anal. Mach. Intell. 45(8), 9836–9845 (2023). [Link]
IEEE ­13th International Conference on Anti-counterfeiting, Secu- 10.​1109/​TPAMI.​2023.​32564​21
rity, and Identification (ASID), pp. 324–328, (2019)
35. He, K, Zhang, X, Ren, S, Sun, J.: "Deep residual learning for Publisher's Note Springer Nature remains neutral with regard to
image recognition," Proceedings of the IEEE conference on com- jurisdictional claims in published maps and institutional affiliations.
puter vision and pattern recognition, pp. 770–778, (2016)
36. Liu, D., Liu, Y., Dong, L.: G-ResNet: Improved ResNet for
brain tumor classification. In: Proceedings of the International