Diagnosis of Lumbar Spondylolisthesis via

Convolutional Neural Networks

1st Fatih Varçın 2nd Hasan Erbay 3rd Eyüp Çetin
Computer Engineering Department Computer Engineering Department Neurosurgery Clinic
Kırıkkale University Kırıkkale University Dörtyol State Hospital
Kırıkkale, Turkey Kırıkkale, Turkey Hatay, Turkey
fatihvarcin@[Link] hasan_erbay@[Link] eyupcet@[Link]

4th İhsan Çetin 5th Turgut Kültür

Department of Medical Biochemistry Department of Physical Medicine and Rehabilitation
Hitit University Kırıkkale University
Çorum, Turkey Kırıkkale, Turkey
ihsancetinilim@[Link] kurgut@[Link]

Abstract—Spondylolisthesis, due to bilateral defects of the Furthermore, the methods related to the classification of
posterior vertebral arch, refers to a slippage of a vertebral vertebral column disorders and vertebral recognition usually
body over another, usually with the superior vertebral body use machine learning and image processing techniques [6],
slipping anteriorly relative to an adjacent inferior vertebral body.
The most common site for spondylolisthesis is L4 or L5. As [7]. Studies on the classification of vertebral column disorders
with most spinal diseases, radiological findings are the main based on machine learning algorithms such as Artificial Neural
ingredients in diagnosing spondylolisthesis. Therefore, computer- Network (ANN), Support Vector Machine (SVM) and Naive
assisted systems can be used in diagnosing spondylolisthesis Bayes have used data from the UCI machine learning database.
when adequate experience doctors in the outpatient clinics are Each sample in the dataset has six attributes and belongs to
not present. In this paper, we looked for a solution to the
problem of diagnosis of spondylolisthesis by using two well- one of the three classes of spondylolisthesis, disc hernia and
known artificial neural networks AlexNet and GoogleLeNet. normal. Thanks to these studies, the classification accuracy of
The data set consists of 272 X-ray images in total. 136 of this task has reached 93.87% [8]. Other studies using image
these images belong to patients suffering from spondylolisthesis, processing include a number of tasks such as identification of
and 136 images without spondylolisthesis. Experimental results vertebral locations, vertebra naming and diagnosing vertebral
demonstrate that GoogleLeNet is 93.87% accuracy and performs
slightly better than AlexNet with an accuracy of 91.67%. diseases [9], [10].
Index Terms—Convolutional Neural Networks, Transfer On the other hand, Convolutional Neural Networks (CNNs)
Learning, Spondylolisthesis are pixel-based multilayer ANN methods, i.e. a type of deep
neural networks. They, at least, include one input layer, more
I. I NTRODUCTION than two hidden layers, and one output layer. They are used
primarily to classify images and have demonstrated better
Lower spin diseases such as disc hernia, spondylolysis, and performance in image classification compared to feature-based
spondylolisthesis are common in adults. Spondylolysis is an methods and show promising performance in medical imaging.
anatomical defect or fracture in the vertebral arch. It may be There are many successful studies done with CNNs, such as
caused by mechanical stress. It occurs at the L5 vertebrae diagnosis of breast cancer [11], [12], diagnosis of brain tumors
between 85 and 95% and also it’s incidence is found at the L4 [13], diagnosis of lung lesions [14].
vertebrae by 5–15% [1]. Progression of spondylolysis is called To develop a successful CNN method, a large amount of
spondylolisthesis which is defined as the anterior slippage labeled training data is a must requirement, in addition, the
of a lumbar vertebra relative to the adjacent vertebra [2]. quality of the images is among the important requirements
Spondylolisthesis has been divided into different categories [15], [4], [16]. However, in applications like medical imaging,
based on aetiology by [3] in 1976 and the categories updated the data is limited, so, data augmentation and transfer learning
in 1990. The categories are dysplastic, isthmic, degenerative, are the compulsory techniques to overcome the need for large
traumatic, pathologic and iatrogenic. dataset.
Computerized methods in medical imaging have been used ImageNet is a growing image dataset and as of April 30,
to help physicians during the diagnosis process [4]. Most of 2010, it contains slightly more than 14 million images in
these methods are feature-based. For example, Yurttakal et. al. 27 high-level categories [17]. There are state of art CNN
[5] developed a method to discriminate malignant and benign networks developed using ImageNet dataset such as Alexnet
breast lesions. [18], SqueezeNet [19], Resnet [20], VGG [21], GoogLeNet

Authorized licensed use limited to: National Institute of Technology- Delhi. Downloaded on May 01,2024 at 09:00:20 UTC from IEEE Xplore. Restrictions apply.
 Cases 272
[22] with different characteristics and performances. Normal 136
In this study, we adopted two different pre-trained deep Lumbar spondylolisthesis 136
neural networks, AlexNet and GoogLeNet, for the diagnosis Image Resolution 227 × 227 × 3
Image Format X-Ray images in JPG
of spondylolisthesis. In addition, data augmentation technique
has been used during the training process. Later, the perfor- TABLE II: Characteristics of the dataset
mances of these networks were compared. The performance
metrics state that GoogLeNet performs slightly better than Cases Training Test
Normal 112 24
AlexNet. Lumbar spondylolisthesis 112 24
II. M ATERIALS AND M ETHODS Total 224 48

A. Transfer Learning TABLE III: Training set and test set statistics
One of the most important problems in deep learning is the
insufficient size and variety of the dataset [23]. This problem
b) Data Augmentation: Image data augmentation is a
adversely affects the training of the CNNs. The goal of transfer
way of preventing CNN from learning irrelevant patterns,
learning is to take advantage of the number of training data in
overfitting and memorizing the exact details of the training
one domain of interest to facilitate learning in another domain
images, while boosting overall performance. For geometric
of interest with little training data. Through transfer learning,
and color augmentations, a lot of practices are performed such
the need for large datasets has greatly reduced. An ImageNet-
as cropping, translating and reflecting the image, increasing
trained network is a very reasonable choice for pre-training
or decreasing brightness and adding Gaussian noise [26].
because of the number of training data and various visual
We, therefore, applied online data augmentation by randomly
challenges that may be important in feature extraction can be
translating the images up to three pixels horizontally and
overcome without fine-tuning [24]. Also, data augmentation
vertically, and rotating the images through an angle of up to
techniques can produce artificial data to obtain sufficient data
20 degrees.
sets using image processing methods. In this augmentation
c) Transfer Learning CNN Models: State-of-the-art pre-
process, new training data has the label of the original data
trained networks such as AlexNet and GoogLeNet were the
from which they were generated. Nonetheless, the number
starting point to implement the transfer learning CNN model.
of data should be at least a few hundred for a considerable
Initially, the original classifier was removed and binary classi-
accuracy rate, which may vary depending on the complexity
fier added. Since the dataset is relatively scarce, during training
of the task [25].
lower layers’ weights were kept frozen, however, higher layers
B. Image Acquisition were fine-tuned.
The raw dataset consists of X-Ray images showing vertebral Table IV presents some features of AlexNet and
columns of 286 patients with various dimensions. The 150 GoogLeNet. These architectures were chosen because of their
images in the raw dataset belong to patients suffering from good performance in the diagnosis of medical disease recog-
spondylolisthesis and 136 images without spondylolisthesis nition [27], [28].
(normal). The radiographs were cropped with the same aspect
Network Year Deep Layers Architecture
ratio of 2/3 to obtain images involving L5, L4, L3, L2 Alexnet 2012 8 25 Traditional sequential network
vertebras and a part of S1 vertebra. Vertebrae radiographs were GoogleNet 2015 22 144 Inception
classified as normal and spondylolisthesis radiographs by two
TABLE IV: Some features of AlexNet and GoogLeNet
experienced experts, brain neurosurgery, and physical therapy
and rehabilitation. Detail of the raw dataset is presented in
Table I. III. R ESULTS
Cases 286 Transfer learning CNN models were implemented using
Normal 136
Lumbar spondylolisthesis 150 the Neural Network Toolbox available in Matlab 2018b. The
Image Resolution Various networks were trained on a dataset containing 272 radiographs
Image Format X-Ray images in JPG of vertebrae, of which 136 were normal and 136 with spondy-
TABLE I: Characteristics of the raw dataset lolisthesis, and validated them on 48 radiographs, of which 24
were normal and 24 with spondylolisthesis. Figure 1 presents
a) Data Pre-processing and Splitting: Some radiographs some images from the dataset.
in the dataset were not technically good, some radiographs Figures 2a and 3a show plots of accuracies over iterations
had zippers, keys, medical supplies, etc. These images were of the transfer learning models via AlexNet and GoogLeNet,
excluded from the dataset. The characteristics of the final respectively, on the other hand, Figures 2b and 3b illustrate the
dataset are shown in Table II. Each image was then resized to loss. We observe that the two models behave similarly. In other
227 × 227 × 3 dimensions. words, both the training accuracy (blue line) and validation
The dataset was then randomly split into a training set and accuracy (dashed black line) getting close to each other during
a test set. Table III shows this splitting. the training process for each model. It also can be seen from

Authorized licensed use limited to: National Institute of Technology- Delhi. Downloaded on May 01,2024 at 09:00:20 UTC from IEEE Xplore. Restrictions apply.
 healthy (without spondylolisthesis) and true negative the num-
ber of cases correctly identified as the patient (with spondy-
lolisthesis).
Confusion Matrix (AlexNet) Confusion Matrix (GoogLeNet)
21 1 95.5% 22 1 95.7%
normal normal
43.8% 2.1% 4.5% 45.8% 2.1% 4.3%

Output Class

Output Class
3 23 88.5% 2 23 92.0%
spondylolishesis spondylolishesis
6.3% 47.9% 11.5% 4.2% 47.9% 8.0%

87.5% 95.8% 91.7% 91.7% 95.8% 93.8%

12.5% 4.2% 8.3% 8.3% 4.2% 6.3%
Fig. 1: Some images from the dataset( on the top, patients

al

s
al

si
rm
si
rm

he
he
without spondylolisthesis and on the bottom, patients with

no
no

is
lis

ol
lo

yl
dy

d
on
spondylolisthesis)

on

sp
sp
Target Class
Target Class

(a) Using AlexNet (b) Using GoogLeNet

the loss subplots that the training loss and the validation loss Fig. 4: Confusion Matrix for the Transfer Learning using
are small, and decrease during the learning progress for the AlexNet and GoogLeNet
models.
Training Accuracy (AlexNet)
The efficiency of the transfer models was evaluated using
100
performance metrics (see Table V) such as classification accu-
Accuracy(%)

80
racy, specificity, precision, sensitivity, F1 score, false positive
Accuracy
60 Training
rate, false discovery rate, false negative rate, and negative pre-
Training (smoothed)

40
Validation dictive value. These performance metrics are evaluated using
0 50 100 150 200
Iteration
250 300 350 400 450 the confusion matrices given in Figure 4. Using only pixel
information, two different transfer learning based multi-layer
(a) Training accuracy using AlexNet
Training Loss (AlexNet) CNN models with online data augmentation were designed.
1.5
Loss
Training
1 Training (smoothed)
Value
Measure
Loss

Validation AlexNet GoogLeNet

0.5 Sensitivity 0.9545 0.9565
Specificity 0.8846 0.9200
0
0 50 100 150 200 250 300 350 400 450 Precision 0.8750 0.9167
Iteration Negative Predictive Value 0.9583 0.9583
(b) Training loss using AlexNet False Positive Rate 0.1154 0.0800
False Discovery Rate 0.1250 0.0833
False Negative Rate 0.0455 0.0435
Fig. 2: Transfer Learning using Pretrained AlexNet Accuracy 0.9167 0.9375
F1 Score 0.9130 0.9362

TABLE V: Performance statistics

Training Accuracy (GoogLeNet)
100
Accuracy(%)

80

60 Accuracy
IV. C ONCLUSION
Training
40 Training (smoothed)
Validation
Throughout this study, two different deep neural networks,
20
0 50 100 150 200 250 300 350 400 450
AlexNet and GoogLeNet were used for the diagnosis of
Iteration
spondylolisthesis and their performances were compared. To
(a) Training accuracy using GoogLeNet the best of our knowledge, there is no publicly available
Training Loss (GoogLeNet)
1
Loss
dataset for spondylolisthesis. Therefore, we have used our
Training
Training (smoothed)
private dataset containing X-ray images. Because the data
Loss

Validation
0.5
set does not contain many images, transfer learning and
data augmentation techniques have been used for the training
0
0 50 100 150 200 250 300 350 400 450 process. Performance evaluation was carried out by means of
Iteration
metrics such as sensitivity, specificity, precision and accuracy
(b) Training loss using GoogLeNet which are frequently used in the literature. The results show
that GoogLeNet is a bit more successful in all performance
Fig. 3: Transfer Learning using Pretrained GoogLeNet metrics than AlexNet.
Our study is important because it is an encouraging start
Figure 4 shows the confusion matrices for the Transfer to suggest a radiological diagnosis to help diagnose lumbar
Learning using AlexNet and GoogLeNet. It is assumed that pathologies. Further studies are needed on this subject, in-
true positive is the number of cases correctly identified as cluding other lumbar pathologies.

Authorized licensed use limited to: National Institute of Technology- Delhi. Downloaded on May 01,2024 at 09:00:20 UTC from IEEE Xplore. Restrictions apply.
 R EFERENCES [23] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans-
actions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–
[1] S. S. Hu, C. B. Tribus, M. Diab, and A. J. Ghanayem, “Spondylolisthesis 1359, 2010.
and spondylolysis,” JBJS, vol. 90, no. 3, pp. 655–671, 2008. [24] A. W. Harley, A. Ufkes, and K. G. Derpanis, “Evaluation of deep
[2] L. L. Wiltse, J. E. Widell, and D. Jackson, “Fatigue fracture: the basic convolutional nets for document image classification and retrieval,”
lesion is inthmic spondylolisthesis.” The Journal of bone and joint in 2015 13th International Conference on Document Analysis and
surgery. American volume, vol. 57, no. 1, pp. 17–22, 1975. Recognition (ICDAR). IEEE, 2015, pp. 991–995.
[3] L. L. Wiltse, P. Newman, and I. Macnab, “Classification of spondyloisis [25] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi,
and spondylolisthesis,” Clinical Orthopaedics and Related Research®, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sánchez,
vol. 117, pp. 23–29, 1976. “A survey on deep learning in medical image analysis,” Medical image
[4] H. Greenspan, B. Van Ginneken, and R. M. Summers, “Guest editorial analysis, vol. 42, pp. 60–88, 2017.
deep learning in medical imaging: Overview and future promise of [26] L. Perez and J. Wang, “The effectiveness of data augmentation in image
an exciting new technique,” IEEE Transactions on Medical Imaging, classification using deep learning,” arXiv preprint arXiv:1712.04621,
vol. 35, no. 5, pp. 1153–1159, 2016. 2017.
[5] A. H. Yurttakal, H. Erbay, T. İkizceli, S. Karaçavuş, and G. Çinarer, [27] N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B.
“A comparative study on segmentation and classification in breast mri Gotway, and J. Liang, “Convolutional neural networks for medical image
imaging,” IIOAB JOURNAL, vol. 9, no. 5, pp. 23–33, 2018. analysis: Full training or fine tuning?” IEEE transactions on medical
[6] J. J. Corso, A. Raja’S, and V. Chaudhary, “Lumbar disc localization and imaging, vol. 35, no. 5, pp. 1299–1312, 2016.
labeling with a probabilistic model on both pixel and object features,” in [28] J. Chi, E. Walia, P. Babyn, J. Wang, G. Groot, and M. Eramian,
International Conference on Medical Image Computing and Computer- “Thyroid nodule classification in ultrasound images by fine-tuning deep
Assisted Intervention. Springer, 2008, pp. 202–210. convolutional neural network,” Journal of digital imaging, vol. 30, no. 4,
[7] A. Raja’S, J. J. Corso, V. Chaudhary, and G. Dhillon, “Toward a clinical pp. 477–486, 2017.
lumbar cad: herniation diagnosis,” International journal of computer
assisted radiology and surgery, vol. 6, no. 1, pp. 119–126, 2011.
[8] S. Ansari, F. Sajjad, N. Naveed, I. Shafi et al., “Diagnosis of vertebral
column disorders using machine learning classifiers,” in 2013 Interna-
tional Conference on Information Science and Applications (ICISA).
IEEE, 2013, pp. 1–6.
[9] Y. Cai, M. Landis, D. T. Laidley, A. Kornecki, A. Lum, and S. Li,
“Multi-modal vertebrae recognition using transformed deep convolution
network,” Computerized medical imaging and graphics, vol. 51, pp. 11–
19, 2016.
[10] T. Klinder, J. Ostermann, M. Ehm, A. Franz, R. Kneser, and C. Lorenz,
“Automated model-based vertebra detection, identification, and segmen-
tation in ct images,” Medical image analysis, vol. 13, no. 3, pp. 471–482,
2009.
[11] H. Li, M. L. Giger, B. Q. Huynh, and N. O. Antropova, “Deep
learning in breast cancer risk assessment: evaluation of convolutional
neural networks on a clinical dataset of full-field digital mammograms,”
Journal of Medical Imaging, vol. 4, no. 4, p. 041304, 2017.
[12] A. H. Yurttakal, H. Erbay, T. İkizceli, and S. Karaçavuş, “Detection of
breast cancer via deep convolution neural networks using mri images,”
Multimedia Tools and Applications, pp. 1–19, 2019.
[13] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumor seg-
mentation using convolutional neural networks in mri images,” IEEE
transactions on medical imaging, vol. 35, no. 5, pp. 1240–1251, 2016.
[14] A. A. A. Setio, F. Ciompi, G. Litjens, P. Gerke, C. Jacobs, S. J. Van Riel,
M. M. W. Wille, M. Naqibullah, C. I. Sánchez, and B. van Ginneken,
“Pulmonary nodule detection in ct images: false positive reduction
using multi-view convolutional networks,” IEEE transactions on medical
imaging, vol. 35, no. 5, pp. 1160–1169, 2016.
[15] K. J. Dreyer and J. R. Geis, “When machines think: radiology’s next
frontier,” Radiology, vol. 285, no. 3, pp. 713–718, 2017.
[16] D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image
analysis,” Annual review of biomedical engineering, vol. 19, pp. 221–
248, 2017.
[17] ImageNet. Summary and statistics (updated on april 30, 2010). [Online].
Available: [Link]
[18] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in neural infor-
mation processing systems, 2012, pp. 1097–1105.
[19] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally,
and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer
parameters and< 0.5 mb model size,” arXiv preprint arXiv:1602.07360,
2016.
[20] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proceedings of the IEEE conference on computer vision
and pattern recognition, 2016, pp. 770–778.
[21] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[22] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”
in Proceedings of the IEEE conference on computer vision and pattern
recognition, 2015, pp. 1–9.

Authorized licensed use limited to: National Institute of Technology- Delhi. Downloaded on May 01,2024 at 09:00:20 UTC from IEEE Xplore. Restrictions apply.