See discussions, stats, and author profiles for this publication at: [Link]

net/publication/319453582

Lung Cancer Detection and Classification with 3D Convolutional Neural Network

(3D-CNN)

Article in International Journal of Advanced Computer Science and Applications · January 2017
DOI: 10.14569/IJACSA.2017.080853

CITATIONS READS
266 19,441

3 authors, including:

Wafaa Alakwaa Mohammad Nassef

Cairo University Cairo University/University of Jeddah
4 PUBLICATIONS 275 CITATIONS 29 PUBLICATIONS 682 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Mohammad Nassef on 01 November 2017.

The user has requested enhancement of the downloaded file.

 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

Lung Cancer Detection and Classification with 3D

Convolutional Neural Network (3D-CNN)

Wafaa Alakwaa Mohammad Nassef Amr Badr

Faculty of Computers & Info. Faculty of Computers & Info. Faculty of Computers & Info.
Cairo University, Egypt Cairo University, Egypt Cairo University, Egypt

Abstract—This paper demonstrates a computer-aided diag- early detection and improved survival. The goal is to construct
nosis (CAD) system for lung cancer classification of CT scans a computer-aided diagnosis (CAD) system that takes as input
with unmarked nodules, a dataset from the Kaggle Data Science patient chest CT scans and outputs whether or not the patient
Bowl, 2017. Thresholding was used as an initial segmentation has lung cancer [2].
approach to segment out lung tissue from the rest of the CT
scan. Thresholding produced the next best lung segmentation. Though this task seems straightforward, it is actually a
The initial approach was to directly feed the segmented CT needle in the haystack problem. In order to determine whether
scans into 3D CNNs for classification, but this proved to be or not a patient has early-stage cancer, the CAD system would
inadequate. Instead, a modified U-Net trained on LUNA16 data have to detect the presence of a tiny nodule (< 10 mm in
(CT scans with labeled nodules) was used to first detect nodule
candidates in the Kaggle CT scans. The U-Net nodule detection
diameter for early stage cancers) from a large 3D lung CT
produced many false positives, so regions of CTs with segmented scan (typically around 200 mm × 400 mm × 400 mm). An
lungs where the most likely nodule candidates were located as example of an early stage lung cancer nodule shown in within
determined by the U-Net output were fed into 3D Convolutional a 2D slice of a CT scan is given in Fig. 1. Furthermore, a
Neural Networks (CNNs) to ultimately classify the CT scan as CT scan is filled with noise from surrounding tissues, bone,
positive or negative for lung cancer. The 3D CNNs produced air, so for the CAD systems search to be efficient, this noise
a test set Accuracy of 86.6%. The performance of our CAD would first have to be preprocessed. Hence our classification
system outperforms the current CAD systems in literature which pipeline is image preprocessing, nodule candidates detection,
have several training and testing phases that each requires malignancy classification.
a lot of labeled data, while our CAD system has only three
major phases (segmentation, nodule candidate detection, and In this paper, we apply an extensive preprocessing tech-
malignancy classification), allowing more efficient training and niques to get the accurate nodules in order to enhance the
detection and more generalizability to other cancers. accuracy of detection of lung cancer. Moreover, we perform an
Keywords—Lung cancer; computed tomography; deep learning; end-to-end training of CNN from scratch in order to realize the
convolutional neural networks; segmentation full potential of the neural network i.e. to learn discriminative
features. Extensive experimental evaluations are performed on
I. I NTRODUCTION a dataset comprising lung nodules from more than 1390 low
dose CT scans.
Lung cancer is one of the most common cancers, ac-
counting for over 225,000 cases, 150,000 deaths, and $12 an axial slice) o
billion in health care costs yearly in the U.S. [1]. It is also in DICOM form
one of the deadliest cancers; overall, only 17% of people in the Kaggle data
the U.S. diagnosed with lung cancer survive five years after in our malignan
the diagnosis, and the survival rate is lower in developing Because the
countries. The stage of a cancer refers to how extensively it quate to accura
the patient lung
has metastasized. Stages 1 and 2 refer to cancers localized to
the LUng Nodu
the lungs and latter stages refer to cancers that have spread to train a U-Ne
to other organs. Current diagnostic methods include biopsies dataset contains
and imaging, such as CT scans. Early detection of lung cancer vide into a train
(detection during the earlier stages) significantly improves the Figure Figure
1: 2D1: CT scan
2D CT scanslice containing
slice containing a small
a small (5mm) (5mm) size 178. For ea
early early
chances for survival, but it is also more difficult to detect early
stage lung cancer nodule.
stage lung cancer nodule [3]. and a nodule lab
stages of lung cancer as there are fewer symptoms [1]. ameter). For ea
variable numbe
Our task is a binary classification problem to detect the 2. Background image is an axia
presence of lung cancer in patient CT scans of lungs with and The paper’s arrangement is as follows: Related work is LUNA16 da
without early stage lung cancer. We aim to use methods from summarized briefly
Typical CAD in Section
systems II. Dataset
for lung for literature
cancer from this paper tection,
is one of t
computer vision and deep learning, particularly 2D and 3D haveintheSection
described followingIII.
pipeline: image preprocessing
The methods → de-
for segmentation are
problem is to a
convolutional neural networks, to build an accurate classifier. tection
presented of cancerous
in section IV. nodule candidates
The nodule → nodule candidate
segmentation or ‘no cancer’)
is introduced
false positive reduction → malignancy prediction for each We will use ac
An accurate lung cancer classifier could speed up and reduce in Section V based on U-Net architecture. Section VI presents
nodule candidate → malignancy prediction for overall CT the ROC to eval
costs of lung cancer screening, allowing for more widespread 3D Convolutional Neural Network for nodule classification and
scan [4]. These pipelines have many phases, each of which Kaggle test set.
is computationally expensive and requires well-labeled data
[Link] 409 | P a g e
during training. For example, the false positive reduction 4. Methods
phase requires a dataset of labeled true and false nodule
candidates, and the nodule malignancy prediction phase re- We preproce
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

patient classification. Our discussion and results are described In [12], R. Golan proposed a framework that train the
in details in Section VII. Section VIII concludes the paper. weights of the CNN by a back propagation to detect lung
nodules in the CT image sub-volumes. This system achieved
sensitivity of 78.9% with 20 false positives, while 71.2% with
II. R ELATED W ORK 10 FPs per scan, on lung nodules that have been annotated by
all four radiologists
Recently, deep artificial neural networks have been ap-
plied in many applications in pattern recognition and machine Convolutional neural networks have achieved better than
learning, especially, Convolutional neural networks (CNNs) Deep Belief Networks in current studies on benchmark com-
which is one class of models [3]. Another approach of CNNs puter vision datasets. The CNNs have attracted considerable
was applied on ImageNet Classification in 2012 is called an interest in machine learning since they have strong representa-
ensemble CNNs which outperformed the best results which tion ability in learning useful features from input data in recent
were popular in the computer vision community [4]. There years.
has also been popular latest research in the area of medical
imaging using deep learning with promising results. III. DATA

Suk et al. [5] suggested a new latent and shared feature Our primary dataset is the patient lung CT scan dataset
representation of neuro-imaging data of brain using Deep from Kaggles Data Science Bowl (DSB) 2017 [13]. The dataset
Boltzmann Machine (DBM) for AD/MCI diagnosis. Wu et al. contains labeled data for 1397 patients, which we divide into
[6] developed deep feature learning for deformable registration training set of size 978, and test set of size 419. For each
of brain MR images to improve image registration by using patient, the data consists of CT scan data and a label (0 for
deep features. Xu et al. [7] presented the effectiveness of no cancer, 1 for cancer). Note that the Kaggle dataset does
using deep neural networks (DNNs) for feature extraction in not have labeled nodules. For each patient, the CT scan data
medical image analysis as a supervised approach. Kumar et consists of a variable number of images (typically around 100-
al. [8] proposed a CAD system which uses deep features 400, each image is an axial slice) of 512 × 512 pixels. The
extracted from an autoencoder to classify lung nodules as slices are provided in DICOM format. Around 70% of the
either malignant or benign on LIDC database. In [9], Yaniv provided labels in the Kaggle dataset are 0, so we used a
et al. presented a system for medical application of chest weighted loss function in our malignancy classifier to address
pathology detection in x-rays which uses convolutional neural this imbalance.
networks that are learned from a non-medical archive. that Because the Kaggle dataset alone proved to be inadequate
work showed a combination of deep learning (Decaf) and to accurately classify the validation set, we also used the
PiCodes features achieves the best performance. The proposed patient lung CT scan dataset with labeled nodules from the
combination presented the feasibility of detecting pathology Lung Nodule Analysis 2016 (LUNA16) Challenge [14] to
in chest x-ray using deep learning approaches based on non- train a U-Net for lung nodule detection. The LUNA16 dataset
medical learning. The used database was composed of 93 contains labeled data for 888 patients, which we divided into
images. They obtained an area under curve (AUC) of 0.93 a training set of size 710 and a validation set of size 178. For
for Right Pleural Effusion detection, 0.89 for Enlarged heart each patient, the data consists of CT scan data and a nodule
detection and 0.79 for classification between healthy and label (list of nodule center coordinates and diameter). For each
abnormal chest x-ray. patient, the CT scan data consists of a variable number of
images (typically around 100-400, each image is an axial slice)
In [10], Suna W. et al., implemented three different deep
of 512 × 512 pixels.
learning algorithms, Convolutional Neural Network (CNN),
Deep Belief Networks (DBNs), Stacked Denoising Autoen- LUNA16 data was used to train a U-Net for nodule
coder (SDAE), and compared them with the traditional image detection, one of the phases in our classification pipeline. The
feature based CAD system. The CNN architecture contains problem is to accurately predict a patient’s label (‘cancer’ or
eight layers of convolutional and pooling layers, interchange- ‘no cancer’) based on the patient’s Kaggle lung CT scan. We
ably. For the traditional compared to algorithm, there were will use accuracy, sensitivity, specificity, and AUC of the ROC
about 35 extracted texture and morphological features. These to evaluate our CAD system’s performance on the Kaggle test
features were fed to the kernel based support vector machine set.
(SVM) for training and classification. The resulted accuracy for
the CNN approach reached 0.7976 which was little higher than IV. M ETHODS
the traditional SVM, with 0.7940. They used the Lung Image
Database Consortium and Image Database Resource Initiative Typical CAD systems for lung cancer have the following
(LIDC/IDRI) public databases, with about 1018 lung cases. pipeline: image preprocessing, detection of cancerous nodule
candidates, nodule candidate false positive reduction, malig-
In [11], J. Tan et al. designed a framework that detected nancy prediction for each nodule candidate, and malignancy
lung nodules, then reduced the false positive for the detected prediction for overall CT scan [15]. These pipelines have
nodules based on Deep neural network and Convolutional many phases, each of which is computationally expensive and
Neural Network. The CNN has four convolutional layers and requires well-labeled data during training. For example, the
four pooling layers. The filter was of depth 32 and size 3,5. false positive reduction phase requires a dataset of labeled
The used dataset was acquired from the LIDC-IDRI for about true and false nodule candidates, and the nodule malignancy
85 patients. The resulted sensitivity was of 0.82. The False prediction phase requires a dataset with nodules labeled with
positive reduction gotten by DNN was 0.329. malignancy.

[Link] 410 | P a g e
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

Figure 2: 3D convolutional neural networks architecture.

After segmentation, we normalize the 3D image by apply-

After
ing the segmentation,
linearforscaling we normalizeall
to squeezed the 3D image
of theby apply-
True/False labels nodule candidates pixels
and malignancy original
ing the linear scaling to squeezed all pixels of the original
labels for unsegmented
nodules are image sparseto for
values lungbetween
cancer,0 and
and 1. may Thenbe we
unsegmented image to to values between each
0 and3D 1. Then by we
nonexistentuseforspline
someinterpolation
other cancers, downsample
so CAD systems thatimage
rely
use
a scalespline
of 0.5interpolation to downsample each 3D image by
on such data would not in each of the
generalize to three
other dimensions. Finally,
cancers. In order to we
a scale
zero-center of 0.5
the in each
data be of the three
subtracting dimensions.
the
achieve greater computational efficiency and generalizability to mean of Finally,
all the we
im-
zero-center
ages
other cancers, thethe
fromproposed
the dataCAD
training be
[Link]
system has theshorter
mean ofpipeline
all the im-
ages from the training set.
and only requires the following data during training: a dataset
of CT scans4.1.1withThresholding
true nodules labeled, and a dataset of CT
scans with 4.1.1
an overall malignancy label. State-of-the-art CAD
Thresholding
systems that predict
Typical malignancyoffrom
radiodensities CT scans
various parts of achieve
a CT AUCscan are
of up to 0.83Typical
shown radiodensities
[16].inHowever, isof variousabove,
as mentioned
Table 1. Air typically parts these
around of a CT
−1000 scanlung
systems
HU, are
take as inputshown in
tissuevarious Table 1.
labeled
is typically Air
around is typically
data around
thatwater,
−500, is not −1000 HU,
usedandinother
blood, lung
this tis-
tissue
[Link] is around
typically
Thearemain around
goal0 of theand
HU, bonewater,
−500,
proposed blood,
issystem is and
typically to other700
reach
around tis-
close to thissues are around
performance. 0 HU, and bone is typically
HU, so we mask out pixels that are close to −1000 or above around 700
HU, so we mask out pixels that are close to −1000 or above
−320 toCADleave lung tissue
The proposed
−320 to leavesystem startsasas
lung tissue
the only
with segment. The
the preprocessing
segment. the The3D
distri-
atonly distri- (a)(a)Histograms
Histograms of pixel
of pixel values
values in for
in HU HUsample
for sample patients
patient’s CT scan
CT scans bution
using of pixel Hounsfield
segmentation, units
normalization, various axial slices
downsampling,
bution of pixel Hounsfield units at various axial slices for a
for a (a)
atCT Histograms of pixelslices.
scanslices.
various at various values in HU for sample patient’s CT scan
sample
and [Link] are
The initial shown in Figure
approach 2.
was Pixels thresholded
input atat at various slices.
sample patient are shown in Figure 2. to simply
Pixels thresholded
400
the preprocessedHU are
3DareCTshown in Figure
scansininto 3, and
3D3,CNNs, the mask is shown in
400 HU shown Figure and thebut the isresults
mask shown in
were [Link]
So 4 additional
an However, topreprocessing
account for thewas possibility
performed
Figure 4 However, to account for the possibility that some
that to
some
input onlycancerous
regions
cancerous ofgrowth
interests
growth
couldintooccur
could the 3D
occur
within the bronchioles
CNNs.
within To identify(air
the bronchioles (air
regions ofpathways)
interest, ainside
U-Net thewaslung, whichforarenodule
trained showncandidate
pathways) inside the lung, which are shown in Figure 5,
in Figure 5,
detection. we choose
Then input to include
regionsthis air to nodule
around create the finalized mask
candidates
we choose to include this air to create the finalized mask as
de- as
tected by shown in Figure
the U-Net was6. fed into 3D CNNs to ultimately
shown in Figure 6.
classify the CT scans as positive or negative for lung cancer.
The overall architecture Substance
isSubstance
shown in Fig. 2, all details
Radiodensity (HU)of layers
Radiodensity (HU)
will be described in the next Air sections.
Air
-1000
-1000
Lung tissue -500
Lung tissue -500
A. Proprocessing andwater and blood
Segmentation
water and blood
0
0
bone 700
For each patient, pixel bone 700
values was first converted in each
image to Hounsfield units (HU), a measurement of
Table 1: typical radiodensities in HU of various substances radioden-
sity, and 2D Table 1:are typical radiodensities in HU3D of various substances
ininaslices
a
CT
CT
scan
scan
stacked
[8]
[8]
into a single image. Because
tumors form on lung tissue, segmentation is used to mask out
the bone, outside air, and other substances that would make
data noisy, and leave only lung tissue information for the
classifier. 4.1.2
A4.1.2 Watershed
number of segmentation approaches were tried,
Watershed
including thresholding, clustering (Kmeans and Meanshift), (b) Corresponding 2D axial
axial slices
The (b)(b) Corresponding2D
Corresponding 2Daxial slices
slices.
and [Link]
segmentation
K-means andobtained
obtained from
Meanshift from thresholding
thresholding
allow
has aa lot
very littlehassuper-lot of
of
noise- many voxels that were part of lung tissue, especially Figure
vision and didnoise-notmany voxels
produce thatqualitative
good were part ofresults.
lung tissue, especially
Watershed Figure 2: (a) Histogram of HU values at (b)
2: (a) Histogram of HU values at (b) corresponding
corresponding
voxels at the edge of the lung, tended to fall outside the Figure
produced the voxels
best atqualitative
the edge of the lung,
results, tendedtoo
but took to long
fall outside
to run the axial slices for sample patient 3D imagecorresponding
3:
axial 3a Histogram
slices for of
sample HU values
patient 3D at 3b
image at
at various axial
axial
various axial
range of lung tissue radiodensity due
to use by the deadline. Ultimately, thresholding was used.
range of lung tissue radiodensity due to
to CT
CT scan
scan noise.
noise. This
This slices for
slice sample patient 3D image at various axial.
slice
means
meansthat
thatour
ourclassifier
classifierwill
willnot
notbe
beable
abletoto correctly
correctly clas-
clas-
After segmentation,
sify the 3Dcancerous
image is normalized by apply-
sifyimages
imagesininwhichwhich cancerousnodules
nodules are
are located
located atat the
the
ing the linear
edge scaling to squeezed all pixels of the voxels
original we
wewere
were unable
unable toto preprocess
preprocess all
all CT
CT scans
scans using Watershed,
edge of the lung. To filter noise and include voxels from
of the lung. To filter noise and include from using Watershed,
unsegmentedthe image we
edges, to values
use between 0 watershed
Marker-driven and 1. Spline inter-
segmentation, 1)
so
soThresholding:
we
we used
used Typical radiodensities of various parts of
thresholding.
thresholding.
the edges, we use Marker-driven watershed segmentation,
polation downsamples
asasdescribed each 3D image etbyal.a [9].scale of original
0.5 in each CT a CT scan are shown in Table I. Air is typically around -1000
describedininAl-Tarawneh
Al-Tarawneh et al. [9]. An An original 2D 2D CT
of the three dimensions.
slice ofofaasample Finally,
patient zero-centering
isisgiven [Link]
The resulting HU,4.2.
lungU-Net
4.2. tissuefor
U-Net is Nodule
for typicallyDetection
Nodule around -500, water, blood, and
Detection
slice
on data by2Dsubtracting sample the patient
mean ofgiven
all the Figure
images The
from resulting
the other tissues are around 0 HU, and bone is typicallysegmented
around 700
slice of the lung segmentation mask created by thresh-
2D slice of the lung segmentation mask created by thresh- HU, so We We initially
initially tried directly
triedclose
directly inputting
inputting the
the entire
entire segmented
training set.
olding is shown in Figure 8, and the resulting 2D slice of pixels that are to -1000 or above -320 are
lungs into malignancy classifiers, but the results were poor. masked
olding is shown in Figure 8, and the resulting 2D slice of lungs into malignancy classifiers, but the results were poor.
the
thelung
lungsegmentation
segmentationmask
maskcreated
createdby
byWatershed
Watershed isisshown
shown ItIt was
[Link] was likely
likely the
the case
case that
that the
the entire
entire image
image was too |large
was411
too large
P a gaa e
ininFigure
Figure10.
[Link],
Qualitatively,this
thisproduces
producesaamuch
muchbetter
betterseg-
seg- search
search space.
space. Thus
Thus wewe need
need aa way
way ofof inputting smaller re-
inputting smaller re-
mentation
mentationthan
thanthresholding.
[Link]
Missingvoxels
voxels(black
(blackdots
dots in
in gions
gions ofof interest
interest instead
instead of
of the
the entire
entire segmented 3D image.
segmented 3D image.
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

in Fig. 5b, and the resultingFigure 8: 2Dlungslice of the lung

segmentation mask segmentation
by thresholding of sam-
mask created by Watershed ple patientis shown in Fig. 5d. Qualitatively,
this produces a much better segmentation than thresholding.
Missing voxels (black dots in Fig. 5b) are largely re-included.
However, this is much less efficient than basic thresholding,
so 6:due
Figure to time
Sample patient limitations, it was
final mask in which not possible to preprocess
bronchioles
Figure 4: Sample patient initial mask with no air
are all CT scans using Watershed, so thresholding is used instead.
included
0 is far more common in the mask than a
trained U-Net is then applied to the segme
0 is
candidates, we train a modified version of the U-Net as de- scan slices to generate nodule candidates.
trai
scribed in Ronneberger et al. on LUNA16 data [10]. U-Net sca
is a 2D CNN architecture that is popular for biomedical im-
(a)age segmentation. We designed a stripped-down
(b) version of
the U-Net to limit memory expense. A visualization of our
U-Net architecture is included in Figure 11 and is described
in detail in Table 2. During training, our modified U-Net
takes as input 256 × 256 2D CT slices, and labels are pro-
vided (256×256 mask where nodule pixels are 1, rest are 0).
The model is trained to output images of shape 256 × 256 Figure 9: final watershed segmentation mask of sample pa-
were each pixels of the output has a value between 0 and 1 tient
indicating the probability the pixel belongs to a nodule. This Figure 8: lung segmentation mask by thresholding of sam-
(a) 8: lung segmentation(b)
Figurepatient
is done by taking the slice corresponding to label one of the ple patient Figure 7: original 2D slice of sample mask by thresholding of sam-
ple patient

4 Figure 11: Modified U-Net archi

Layer Params Activation

Input L
(c) (d)
Conv1a 3x3x3 ReLuIn
Conv1b 3x3x3 ReLuC
Max Pool 2x2, stride 2 C
Figure 4: (4a) Sample patient 3D image with pixels values Conv2a 3x3x3 ReLu
M
greater than 400 HU reveals the bone segment, (4b) Sample Conv2b 3x3x3 ReLu
C
C
patient bronchioles within lung, (4c) Sample patient initial Max Pool 2x2, stride 2
M
Conv3a 3x3x3 ReLu
mask with no air, and (4d) Sample patient final mask in which Conv3b 3x3x3 ReLu
C
C
bronchioles are included. (c) Figure
Figure 9: final watershed segmentation mask of sample pa- Max
(d) Pool 2x2, stride 2 M
10: final
Figure watershed
9: final watershedlung segmentation
segmentation
Conv4a mask of of sample
sample
3x3x3 pa- pa-
ReLuC
tient tient
tient Conv4b 3x3x3 ReLuC
Figure 5: (5a) Original 2D slice of sampleUp Conv4b
patient,
Concat
(5b) Lung 2x2
Conv4b,Conv3b
U
out to leave lung tissue as the only segment. The distribution segmentation mask by thresholding of sample Conv5a patient, (5c) 3x3x3
C
ReLuC
of pixel Hounsfield units at various axial slices for a sample softmax of the
Final watershed segmentation mask of sample final U-Net layer. Corresponding
patient, and (5d)
Conv5b U-Net
3x3x3 in-
ReLuC
patient are shown in Fig. 3. Pixels thresholded at 400 HU are Final watershed lungputs, labels, and predictions
segmentation of sample UponConv5b
apatient.
patient from the2x2 LUNA16 U
shown in Fig. 3a, and the mask is shown in Fig. 3b. However, validation set is shown in Concat Figures 12,Conv5b,Conv2b
13, and 14 respec- C
tively. most nodules are much Conv6a
smaller A weighted3x3x3softmax ReLuC
to account for the possibility that some cancerous growth could Conv6b 3x3x3 ReLuC
cross-entropy loss calculated for each pixel, as 2x2 a label of U
occur within the bronchioles (air pathways) inside the lung, V. U-N ET FOR N ODULE D ETECTION
Up Conv6b
Concat Conv6b,Conv1b C
which are shown in Fig. 4c, this air is included to create the Conv6a 3x3x3 ReLuC
finalized mask as shown in Fig. 4d. Feeding the entire segmented lungs into malignancy clas-
Conv6b 3x3x3 ReLu5
C
sifiers made results very poor. It was likely the case that 3x3x3
Conv7 the C

entire image was too largeFigure search space.

10: final Thus
watershed feeding
lung2: smaller
segmentation of sample pa- Tab
Table I: Typical Radiodensities in HU of Various SubstancesFigure regions
10: final watershed lung segmentation of sample
of interest insteadtient
pa-
of the entire segmented
Table U-Net architecture
3Dconv
image
(Dropout wit
is training,afte‘
tient after each ‘a’ layer during
in a CT Scan more convenient. This was achieved by selecting small
sizing of image boxesinterpolation,
via bilinear siziA
lea
Substance Radiodensity (HU) containing top cancerous nodule candidates. To rate
learning find= 0.0001)
these top
softmax of the final U-Net layer. softmaxU-Net
Corresponding of the in-
final U-Net layer. Corresponding U-Net in-
Air -1000 nodule candidates, a modified version
puts, labels, of the U-Net
and predictions wasfrom
on a patient trained
the LUNA16
puts, labels, and predictions on a patient from the LUNA16 4.3
Lung tissue -500 as described in Ronneberger et al.
validation onshown
set is
validation set is shown in Figures 12, 13, and 14 respec-
LUNA16
4.3.
in Figuresdata
Malignancy [18].
12, 13, U-respec-
Classifiers
and 14
tively. Net is a 2D CNNsmaller
architecture
A weighted that is popular Onceforwe biomedical
tively. most nodules are much smaller A weighted softmax
Water and Blood 0 most nodules are much softmax trained the U-Net on the
cross-entropy loss calculated for each pixel, as a label of we
imageloss
cross-entropy segmentation. A pixel,
calculated for each stripped-down
as a label of versionwe ranof it the
on 2D U-Net is Kaggle data
slices of
Bone 700
designed to limit memory expense. A visualization of the U- 5
Net architecture is included in Fig. 6 and
5 is described in detail
2) Watershed: The segmentation obtained from threshold-
in Table II. During training, the modified U-Net takes as input
ing has a lot of noise. Many voxels that were part of lung
256 × 256 2D CT slices, and labels are provided (256 × 256
tissue, especially voxels at the edge of the lung, tended to fall
mask where nodule pixels are 1, rest are 0).
outside the range of lung tissue radiodensity due to CT scan
noise. This means that our classifier will not be able to cor- The model is trained to output images of shape 256 ×
rectly classify images in which cancerous nodules are located 256 were each pixels of the output has a value between 0
at the edge of the lung. To filter noise and include voxels from and 1 indicating the probability the pixel belongs to a nodule.
the edges, we use Marker-driven watershed segmentation, as This is done by taking the slice corresponding to label one
described in Al-Tarawneh et al. [17]. An original 2D CT slice of the softmax of the final U-Net layer. Corresponding U-Net
of a sample patient is given in Fig. 5a. The resulting 2D slice of inputs, labels, and predictions on a patient from the LUNA16
the lung segmentation mask created by thresholding is shown validation set is shown in Fig. 7a, 7b, and 7c, respectively.
[Link] 412 | P a g e
 (IJACSA) International Journal of Advanced Computer Science and Applications,
0 is far more common in the mask than a label of 1. The
Vol. 8, No. 8, 2017
trained U-Net is then applied to the segmented Kaggle CT
scan slices to generate nodule candidates.

Table II: U-Net Architecture (Dropout with 0.2 Probability

after each ‘a’ Conv. Layer during Training, ‘Up’ Indicates
Resizing of Image via Bilinear Interpolation, Adam Optimizer,
Learning Rate = 0.0001)
Layer Params Activation Output
Input 256 × 256 × 1
Conv1a 3 × 3× 32 ReLu 256 × 256 × 32
resholding of sam- Conv1b 3 × 3× 32 ReLu 256 × 256 × 32
Max Pool 2× 2, stride 2 128 × 128 × 32
Conv2a 3 × 3× 80 ReLu 128 × 128 × 80
Figure 12: U-Net sample input from LUNA16 validation Figure 14: U-Net predicted output from LUNA16 valida-
set. Note that the above image has the largest nodule from tion set Conv2b 3×3× 80 ReLu 128 × 128 × 80
Figure Figure6: 11:Modified
Modified U-Net U-Net architecture
architecture.
the LUNA16 validation set, which we chose for clarity- Max Pool 2× 2, stride 2 64 × 64 × 80
most nodules are significantly smaller than the largest one
in this image. Conv3a 3× 3× 160
Because our U-Net generates more suspicious regions than
ReLu 64 × 64 × 160
Conv3b
actual nodules, we located the top 3×8 nodule
3× 160 candidates ReLu 64 × 64 × 160
(32 × 32 × 32 volumes) by sliding a window over the data
Layer Params Activation Output Max Pool 2× 2, stride 2
and saving the locations of the 8 most activated (largest
32 × 32 × 160
Input 256 x 256 x 1 Conv4a
L2 norm) sectors. To prevent the 3× 3× from
top sectors 320simply ReLu 32 × 32 × 320
being clustered in the brightest region of the image, the 8
Conv1a 3x3x3 ReLu 256 x 256 x 32 Conv4b 3× 3× 320
sectors we ultimately chose were not permitted to overlap
ReLu 32 × 32 × 320
Conv1b 3x3x3 ReLu 256 x 256 x 32 with each Upother. Conv4b
We then combined 2×2 these sectors into a 64 × 64 × 320
single 64 ×Concat
64 × 64 image, which will serve as
Conv4b,Conv3b the input to 64 × 64 × 480
Max Pool 2x2, stride 2 128 x 128 x 32 our classifiers, which assign a label to the image (cancer or
Conv2a 3x3x3 ReLu 128 x 128 x 80 not cancer).Conv5a 3× 3× 160 ReLu 64 × 64 × 160
We use Conv5b
a linear classifier as a3× 3×a 160
baseline, vanilla 3D ReLu 64 × 64 × 160
Conv2b 3x3x3 ReLu 128 x 128 x 80 CNN, and a Googlenet-based 3D CNN. Each of our clas-
Up Conv5b 2× 2 128 × 128 × 160
Max Pool 2x2, stride 2 64 x 64 x 80 sifiers uses weighted softmax cross entropy loss (weight for
Concat
a label is the Conv5b,Conv2b
inverse of the frequency of the label in the 128 × 128 × 240
Conv3a 3x3x3 ReLu 64 x 64 x 160 training set) and Adam Optimizer, and the CNNs use ReLU
Conv6a 3× 3× 80
activation and droupout after each convolutional layer dur-
ReLu 128 × 128 × 80
Conv3b 3x3x3 ReLu 64 x 64 x 160 ing training.Conv6b
The vanilla 3D CNN is3× based3× 80CNN de-
on a 3D ReLu 128 × 128 × 80
Figure 12: U-Net Figure
mask of that
set. Note sample
sample12:
the above pa-
input
image
set. Note
from sample
U-Net
thathas
thethe
LUNA16
Max
largest
above
(a)validation
input
Pool
nodule
image
from
hasfrom
LUNA16Figure
the largesttion
14: U-Net predicted
validation
2x2,
set
nodule from
Figure
stridetion (b)
2 set
output
14: U-Netfrompredicted
LUNA16
Figure valida-
output
13: U-Netfrom
32 x 32 x 160 LUNA16
sample (c)
labelsvalida-
mask
dation set showing ground truth nodule location
from LUNA16 vali-
signed for this task [11]. We shrunk the network to prevent
Up Conv6b 2×2 256 × 256 × 80
parameter overload for the relatively small Kaggle dataset.
the LUNA16 validation set, which
the LUNA16 validationConv4a
we choseset,for we chose for clarity-3x3x3
clarity-
which ReLu 32 x 32 x 320 Concat
A visualization of our vanilla 3D Conv6b,Conv1b
CNN architecture is in- 256 × 256 × 112
most nodules are significantly
most nodulessmaller than the largest
are significantly onethan the largest one
smaller
in this image. in this image. Conv4b 3x3x3
Because our U-Net generates moreU-Net ReLu
suspicious2D regions
32
than x 32 x 320 cluded in Figure 15 and described in detail in Table 3
Conv6a 3× 3× 32 ReLu 256 × 256 × 32
Figure 7: (7a) U-Net
Up Conv4b sample
actual nodules, we2x2locatedinput
Because our
the top 8 from
generates
nodule
slices
LUNA16
more
64
candidates
processing
back
and
actual nodules, we located the top 8 nodule candidates
to generate
suspicious
x 64 x
reading of validation
regions than
320
nodule candidates (Pre-
LUNA16 data code based
We also designed a 3D Googlenet-based model is based
Conv6b
on the 2D model designed in Szegedy3×et al.
3× for 32
image clas- ReLu 256 × 256 × 32
(32 × 32 × 32 volumes) by32sliding
× 32 avolumes)
window on over the[Link]
data
set. Note Concatthat theConv4b,Conv3b
andabove image
(32 ×
saving the locations of the the
and saving 8has
most theof the(largest
activated
locations largest
by sliding
64
8 most activated nodule
a window over the data
x 64 x 480 (largest
candidate-generation-and-luna16-preprocessing). from Conv7
sification [12]. A visualization of 3×
our 3D3× 3 is in-
Googlenet 256 × 256 × 2
L2 norm) sectors. To prevent the top sectors from simply cluded in Figure 16 and described in detail in Table 4. Re-
the LUNA16 Conv5a validation being clusteredset,
3x3x3 which
L2 norm) sectors.
in the brightest region of we
ReLu
being clustered in thetions
ToIdeally
chose
preventthe
the image,
the
of allthe
brightest
64
the 8nodules,
region
forximage,
topxsectors
output 64
of U-Net
of theand
clarity-most
160
from would
simplygive us the exact loca-
we would
the 8 be able to say images
fer to Szegedy et al. for more information on the inception

nodulesConv5b 3x3x3 chose were notReLu

were64 asxpermitted
64 x 160
sectors we ultimately permitted to overlap module [12].
are significantly with each other. smaller
Up Conv5b single 64 × 64 × 642x2
sectors we ultimately
We then combined than with
the
chosenodules
these sectors intolargest
not
a
detected by
one in this
U-Net are positive for lung
to overlap
image, which will serve as the128 input x to 128 x 160
with each other. Wecancer, and images
then combined without
these sectorsanyinto
nodules
a detected by U-Net
image, (7b) ConcatU-NetConv5b,Conv2b ourpredicted
single
classifiers, which assignoutput
64 a label to thefrom
× 64 × 64
our classifiers, which 14,
are negative
image, which
a128
image (cancer
U-Net
assign
LUNA16
for
will lung
serve
xto 128
or
produces
label
cancer.
ax
the image
as the
240
strong
validation
signalorfor the actual nodule, into a single 64 × 64 × 64 image, which will serve as the input
However,
input
(cancer
to as shown in Figure 5. Results
not cancer).
set, (7c)Conv6a U-Net sample We use a labels
3x3x3
linear mask
not cancer).
classifier ReLu
as a baseline,
We use3Da CNN.
from
linear Each
but also 128
a vanilla
an additional
classifier
LUNA16
produces
3D
asclas-
x a lotxof80false
128
classifier athat
a baseline,
validation
positives, so we need
determines
vanilla 3D the malignancy.
to classifiers, which assign a label to the image (cancer or not
The results are shown in Table 5, and ROC curves for the
Vanilla CNN and 3D Googlenet are shown in Figure 17.
set showing Conv6bground truth nodule
3x3x3
CNN, and a Googlenet-based
CNN, and a location.
ReLu
Googlenet-based
sifiers uses weighted softmax cross entropy loss (weight for
of our128
3D CNN.x 128
Each xof80our clas- cancer).
Up Conv6b a label is the inverse 2x2
sifiers
of theuses weighted
frequency ofsoftmax 256
the labelcross
in the x 256
entropy lossx(weight
80 for 6
training set) and Adam a label is theand
Optimizer, inverse of theuse
the CNNs frequency
ReLU of the label in the
Concat Conv6b,Conv1b
activation and droupout training
after set)
eachand Adam Optimizer,
convolutional 256
layer x 256
and
dur- the CNNsx 112 use ReLU A 3D CNN is used as linear classifier. It uses weighted
Conv6a ing training. The 3x3x3 activation
vanilla 3D CNN andis droupout
ReLu
based on aafter
3D CNNeach
256 convolutional
de- x 256 x 32 layer dur-
Figure 13: U-Net sample labels mask from
dation set showingFigure
ground13:
truthU-Net
nodulesample
LUNA16 vali-
labels
Conv6b
location mask from
signed for this task [11].
LUNA16 vali-
parameter overload 3x3x3
ing training.
signed
for
We shrunk Thethe
for this task
the relatively
vanilla
ReLu
network
[11].
small
3D to
CNN is based on a 3D CNN de-
prevent
256 the
We shrunk
Kaggle dataset. x 256network x to
32prevent
softmax cross entropy loss (weight for a label is the inverse
dation set showing ground truth nodule location
Most nodules Conv7 are Acludedmuch smaller.
visualization of our
3x3x3
in Figure 15 and
parameter
vanilla 3D
A visualization
A
overload
weighted
for the relatively
CNN architecture
of ourinvanilla
described in detail
softmax
is in- small Kaggle dataset.
256 x 256
Table 33D CNN architecture is in-
x 2 cross- of the frequency of the label in the training set) and Adam
processing and reading entropy loss calculated for each pixel, as a label of 0 is far
2D slices back to generate nodule candidates (Pre-
of LUNA16
2D slices back to data code nodule
generate
We also designed cluded
based candidates (Pre-
in Figure 15 andmodel
a 3D Googlenet-based described in detail in Table 3
is based Optimizer, and the CNNs use ReLU activation and droupout
on the 2D model designed Weinalso
Szegedy et al.a for
3D image clas-
ation of[Link]
sample pa- and reading Table of 2: U-Net architecture (Dropout designed
with 0.2 probability
Googlenet-based model is based
after each convolutional layer during training. The network is
on more common indatasification
the mask onthan of a label ofisRe-
in-1.
et [Link] trained U-
on processing LUNA16 code based
[12]. A visualization
the 2D model our designed
3D Googlenet in Szegedy for image clas-
candidate-generation-and-luna16-preprocessing).
[Link]
of U-Net would giveafter us the each ‘a’ conv cluded layer
in Figure during
16 and training,
described
sification [12].in Adetail in‘Up’ indicates
Table [Link]
visualization our 3D Googlenetre- is in-
Net is then applied
Ideally the output candidate-generation-and-luna16-preprocessing).
Ideally the output ofsizing
exact loca-
fer to the
to Szegedy et segmented
al. cluded
for morein information
Figure 16 and Kaggle
on described
the inception CT
in detail in scan slices
Table 4. Re- to shrunk to prevent parameter overload for the relatively small
tions of all the nodules, and we would be able to
of
say
U-Net would image
images
give us the via
exactbilinear
module loca-
[12]. interpolation,
to Szegedy et al. forAdam Optimizer,
with nodules as detected
cancer, and imageswith generate
without
by the
tions of all
any
U-Net
nodules nodule
are positive
nodules, and we for
learning
detected
nodules as detected by
lung
would
rate
U-Net
by U-Net
candidates.
be able to say images
are = 0.0001)
positive for lung
fer
module [12].
more information on the inception
Kaggle dataset. The 3D CNN architecture is described in detail
are negative for lung
ponding U-Net
14, U-Net produces
cancer.
cancer, andHowever,
in- signal
area negative
strong
as shown
images without
for cancer.
for lung
anyinnodules
the actual
Figure detected
5. Results
by U-Net
nodule, as shown in Figure
However,
in Table III.
but also produces14, a lot of false positives, so we need The
5. Results
results are shown in Table 5, and ROC curves for the
U-Net produces a strong signal for the actual nodule,
from the LUNA16
an additional classifier
but also thatproduces
determines
4.3.
a lotVI.
theofmalignancy.
MalignancyM Vanilla
ALIGNANCY
false positives, so we CNN
Classifiers The3D
needand 3D Googlenet results CNN
areare
shownshown in C
in Figure 17.5, and ROC curves for the
LASSIFIERS
Table
Convolutional neural network consists of some number of
13, and 14 respec- an additional classifier that determines the malignancy. Vanilla CNN and 3D Googlenet are shown in Figure 17.

weighted softmax OnceOnce

6
we trained convolutional layers, followed by one or more fully connected
the U-Net wasthe trained U-Net on on the the LUNA16 LUNA16 data, data, it is ran
pixel, as a label of
6
we ran it on 2D slices of Kaggle data and stacked the layers and finally an output layer. An example of this archi-
on 2D slices of Kaggle data and stacked the 2D slices back
tecture is illustrated in Fig. 8.
to generate nodule candidates 1 . Ideally the output of U-Net
5 give the exact locations of all the nodules, and it would
would
be able to declare images with nodules as detected by U-Net
are positive for lung cancer, and images without any nodules
detected by U-Net are negative for lung cancer. However, as
shown in Fig. 7c, U-Net produces a strong signal for the actual
nodule, but also produces a lot of false positives, so we need
an additional classifier that determines the malignancy. Figure 4. An example architecture of a 3D Convolutional Neural Network used here. On the left is the input 3D volume, followed by two
convolutional layers, a fully connected layers and an output layer. In the convolutional layers, each filter (or channel) is represented by a
Because U-Net generates more suspicious regions than Figurevolume.
8: An example architecture of a 3D convolutional neural
actual nodules, the top 8 nodule candidates are located (32 × network usedin thehere.
its orientation CT scan. On the left
In the remainder of this is
sec- the input
treated 3D
as a single volume,
vector. The output of a followed
fully connected

by two convolutional layers, a fully

tion, we describe the technical details of the neural network
networkconnected
architecture. The outputlayers
is a 1D vector I
of neuron i inand
layer m isan
whose dimension is a parameter of the
(m)

32×32 volumes) by sliding a window over the data and saving architecture we used and how it was trained.

the locations of the 8 most activated (largest L2 norm) sectors. output2.2.1layer. In the
Convolutional Neuralconvolutional
Networks layers,
given by
each filter X
(or channel)


To prevent the top sectors from simply being clustered in the is represented by
A convolutional neural a
network volume.
consists of some number
of convolutional layers, followed by one or more fully con-
I =f b(m)
i + I
ReLU
(m,i)
W  (4)
j
(m−1)
j
(m,i)
j

brightest region of the image, the 8 sectors were not permitted nected layers and finally an output layer. An example of
this architecture is illustrated in Figure 4. Formally, we de-
where W (m,i) and b(m,i) are the parameters of neuron i in
layer m and the sum over j is a sum over all dimensions
to overlap with each other. Then these sectors are combined note the input to layer m of the network by I (m) . The
input to a 3D convolutional layer m of a neural network is
of the input. The activation function fReLU (·) here is cho-
sen to be a Rectified Linear Unit (ReLU) with fReLU (a) =
(m−1) (m−1) (m−1) (m−1)
a n1 × n2 × n3 3D object with nc chan- max(0, a). This activation function has been widely used in
1 Preprocessing Formally, we denote the input to layer m of the network
nels, so I (m−1) ∈ Rn1
(m−1)
×n2
(m−1)
×n3
(m−1)
×n(m−1)
c and its a number of domains [24, 16] and is believed to be particu-
and reading of LUNA16 data code based on (m,`)
elements are denoted by Ii,j,k where i, j, and k index larly helpful in classification tasks as the sparsity it induces

[Link] candidate-generation-and-luna16- by I . The input to a 3D convolutional layer m of a neural

(m)
the 3D volume and ` selects the channel. The output of
a convolutional layer m is defined by its dimensions, i.e.,
in the outputs helps create separation between classes dur-
ing learning [17, 3].
(m)(m−1) (m−1) (m−1) (m−1)
preprocessing network is a n
(m)
×n(m)
×n 3D object with n
n1 × n2 × n3 as well as the number of filters or chan-
nels it produces nc1 . The output of layer
(m) 2 m is a convolu-
The last fully connected layer is used as input to the out-
3 put layer. The structure and form of the output clayer de-
tion of its input with a filter and is computed as pends on the particular task. Here we consider two different
types of output functions. In classification problems with K
[Link] 413 | P a g e
(m,`)
X (m−1,`0 ) (m,`)
Ii,j,k = ftanh (b(m,`) + Ii0 ,j 0 ,k0 Wi−i0 ,j−j 0 ,k−k0 ,`0 ) classes, a common output function is the softmax function
i0 ,j 0 ,k0 ,`0 (o)
(3) exp(Ii )
fi = P (5)
where W (m,`)
and b (m,`)
are the parameters which define (o)
j exp(Ij )
the `th filter in layer m The locations where the filters are K
(m,`) X
evaluated (i.e., the values of i, j, k for which Ii,j,k is com- (o)
Ii = (o,i)
b +
(o,i) (N )
Wk I k (6)
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

(m−1) (m−1) (m−1)

so I (m−1) ∈ (Rn1 ×n2 ×n3
and its elements are
(m,`)  −1
denoted by Ii,j,k where i, j, and k index the 3D volume and X
` selects the channel. The output of a convolutional layer m is f = a + (b − a) 1 + exp(b(o) +
(o) (N ) 
Wj I j (5)
(m) (m) (m)
defined by its dimensions, i.e., n1 × n2 × n3 as well as j
(m)
the number of filters or channels it produces nc . The output
of layer m is a convolution of its input with a filter and is which provides a continuous output f which is restricted to
computed as lie in the range (a, b) with parameters b(o) and W (o) . We call
(m,`)
X (m−1,`) ˜ (m,`)
this the scaled logistic output function. We note that when
Ii,j,k = ftanh (b(m,`) + Iĩ,j̃,k̃ Wi−ĩ,j−j̃,k−k̃,`˜) (1) considering a ranking-type multi-class classification problem
ĩ,j̃,k̃,`˜ like predicting the malignancy level this output function might
be expected to perform better.
where, W (m,`) and b(m,`) are the parameters which define
the `th filter in layer m The locations where the filters Table III: 3D CNN Architecture (Dropout with 0.2, Adam
(m,`)
are evaluated (i.e., the values of i, j, k for which Ii,j,k is Optimizer, Learning Rate = 0.0001)
computed) and the size of the filters (i.e., the values of W (m,`) ) Layer Params Activation Output
which are non-zero) are parameters of the network architecture. Input 28 × 28 × 28
Finally, we use a hyperbolic tangent activation function with Conv1 5 × 5× 5 ReLu 28 × 28 × 28× 7
Max Pool 1× 1× 1, stride 2× 2× 4 14 × 14 × 7 × 7
ftanh (a) = tanh(a). Conv2 5 × 5× 3 ReLu 14 × 14 × 7 × 17
Max Pool 2× 2 × 2, stride 1× 1 × 0 6 × 6 × 3 × 17
Convolutional layers preserve the spatial structure of the Dense 256
inputs, and as more layers are used, build up more and Dense 2

more complex representations of the input. The output of the

convolutional layers is then used as input to a fully connected
network layer. To do this, the spatial and channel structure is A. Training
ignored and the output of the convolutional layer is treated Given a collection of data and a network architecture, the
as a single vector. The output of a fully connected is a 1D main goal is to fit the parameters of the network to that data.
vector I (m) whose dimension is a parameter of the network To do this we will define an objective function and use gradient
architecture. The output of neuron i in layer m is given by based optimization to search for the network parameters which
 
X minimize the objective function. Let D = ni , yi D i=1 be the
(m) (m−1) (m,i) 
Ii = fReLU b(m,i) + Ij Wj (2) set of D (potentially augmented) training examples where n
j is an input (a portion of a CT scan) and y is the output
(the malignancy level or a binary class indicating benign or
where, W (m,i) and b(m,i) are the parameters of neuron i in malignant) and Θ denote the collection of all weights W and
layer m and the sum over j is a sum over all dimensions of the biases b for all layers of the network. The objective function
input. The activation function fReLU (.) here is chosen to be has the form
X D
a Rectified Linear Unit (ReLU) with fReLU (a) = max(0, a).
E(Θ) = L(yi , f (ni , Θ)) + λEprior (Θ) (6)
This activation function has been widely used in a number of
i=1
domains [19], [20] and is believed to be particularly helpful
in classification tasks as the sparsity it induces in the outputs where, f (ni , Θ)) is the output of the network evaluated on
helps create separation between classes during learning. input n with parameters Θ, L(yi , f (ni , Θ)) is a loss function
which penalizes differences between the desired output of the
The last fully connected layer is used as input to the output network y and the prediction of the network ŷ. The function
layer. The structure and form of the output layer depends on Eprior (Θ) = kW k2 is a weight decay prior which helps
the particular task. Here we consider two different types of prevent over-fitting by penalizing the norm of the weights and
output functions. In classification problems with K classes, a λ controls the strength of the prior.
common output function is the softmax function:
(o) We consider two different objective functions in this paper
exp(Ii ) depending on the choice of output function. For the softmax
fi = P (3)
(o) output functionP we use the standard cross-entropy loss function
j exp(Ij ) K
L(yi , ŷ) = − k=1 yk log(ŷk ) where y is assumed to be a
K
X binary indicator vector and ŷ is assumed to be a vector of
(o)
Ii =b (o,i)
+ Wk
(o,i) (N )
Ik (4) probabilities for each of the K classes. A limitation of a cross-
k=1
entropy loss is that all class errors are considered equal, hence
mislabeling a malignancy level 1 as a level 2 is considered just
as bad as mislabeling it a 5. This is clearly problematic, hence
for the scaled logistic function we use the squared error loss
where, N is the index of the last fully connected layer,
function to capture this. Formally, L(yi , ŷ) = (y − ŷ)2 where
b(o,i) and W (o,i) are the parameters of the ith output unit and
we assume y and ŷ to be real valued.
fi ∈ [0, 1] is the output for class i which can be interpreted as
the probability of that class given the inputs. We also consider Given the objective function E(Θ), the parameters Θ are
a variation on the logistic output function: learned using stochastic gradient descent (SGD) [21]. SGD
[Link] 414 | P a g e
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

operates by randomly selecting a subset of training examples The experiments are implemented on computer with CPU
and updating the values of the parameters using the gradient i7, 2.6 GHz, 16 RAM, Matlab 2013b, R-Studio, and Python.
of the objective function evaluated on the selected examples. Initially speaking, the nodules in DSB dataset are detected and
To accelerate progress and reduce noise due to the random segmented using thresholding and U-Net Convolutional Neural
sampling of training examples we use a variant of SGD with Network. The diameters of the nodules range from 3 to 30 mm.
momentum [22]. Specifically, at iteration t, the parameters are Each slice has 512 × 512 pixels and 4096 gray level values in
updated as Hounsfield Unit (HU), which is a measure of radiodensity.
Θt+1 = Θt + 4Θt+1 (7) In the screening setting, one of the most difficult decisions
is whether CT or another investigation is needed before the
4Θt+1 = ρ4Θt − ∇Et (Θt ) (8) next annual low-dose CT study. Current clinical guidelines are
complex and vary according to the size and appearance of the
where, ρ = 0.9 is the momentum parameter, 4Θt+1 is nodule. The majority of nodules were solid in appearance. For
the momentum vector, t is the learning rate and ∇Et (Θt ) pulmonary nodule detection using CT imaging, CNNs have
is the gradient of the objective function evaluated using only recently been used as a feature extractor within a larger CAD
the training examples selected at iteration t. At iteration 0, system.
all biases are set to 0 and the values of the filters and For simplicity in training and testing we selected the
weightsqare initialized byq uniformly sampling from the inter- ratings of a single radiologist. All experiments were done using
val [− fan in +fan out , fan in +f
6 6
] as suggested by [23] 50% training set, 20% validation set and 30% testing set. To
an out
where fan in and fan out respectively denote the number of evaluate the results we considered a variety of testing metrics.
nodes in the previous hidden layer and in the current layer. The accuracy metric is the used metric in our evaluations. In
Given this initialization and setting t = 0.01, SGD is running our first set of experiments we considered a range of CNN
for 2000 epochs, during which t is decreased by 10% every architectures for the binary classification task. Early experi-
25 epochs to ensure convergence. mentation suggested that the number of filters and neurons per
layer were less significant than the number of layers. Thus, to
simplify analysis the first convolutional layer used seven filters
VII. S IMULATION R ESULTS with size 5×5×5, the second convolutional layer used 17 filters
with 5×5×3 and all fully connected layers used 256 neurons.
The experiments are conducted using DSB dataset. In These were found to generally perform well and we considered
this dataset, a thousand low-dose CT images from high-risk the impact of one or two convolutional layers followed by one
patients in DICOM format is given. The DSB database consists or two fully connected layers. The networks were trained as
of 1397 CT scans and 248580 slices. Each scan contains a described above and the results of these experiments can be
series with multiple axial slices of the chest cavity. Each scan found in Table I. Our results suggest that two convolutional
has a variable number of 2D slices (Fig. 9), which can vary layers followed by a single hidden layer is one of the optimal
based on the machine taking the scan and patient. The DICOM network architecture for this dataset. The average error for
files have a header that contains the necessary information training is described in Fig. 10.
about the patient id, as well as scan parameters such as the
slice thickness. It is publicly available in the Kaggle [13]. 3D CNN Training Error
0.7
Dicom is the de-facto file standard in medical imaging. This
pixel size/coarseness of the scan differs from scan to scan
(e.g. the distance between slices may differ), which can hurt 0.6

performance of our model.

Number of Slices for each Patient 0.5
600
Average Training Error

500 0.4

400
0.3
Number of Slices

300
0.2

200
0.1

100

0
0 10 20 30 40 50 60 70 80 90 100
0 Iteration Number
0 200 400 600 800 1000 1200 1400
Patient Number
Number of Slices Figure 10: Average training error in 3D CNN.
Figure 9: Number of slices per patient in data science bowl
dataset. Another important parameter in the training of neural
networks is the number of observations that are sampled
[Link] 415 | P a g e
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

at each iteration, the size of the so-called minibatch. The R EFERENCES

use of minibatches is often driven in part by computational
[1] W.-J. Choi and T.-S. Choi, “Automated pulmonary nodule detection
considerations but can impact the ability of SGD to find a system in computed tomography images: A hierarchical block classifi-
good solution. Indeed, we found that choosing the proper cation approach,” Entropy, vol. 15, no. 2, pp. 507–523, 2013.
minibatch size was critical for learning to be effective. We [2] A. Chon, N. Balachandar, and P. Lu, “Deep convolutional neural
tried minibatches of size 1, 10, 50 and 100. While the nature networks for lung cancer detection,” tech. rep., Stanford University,
of SGD suggests that larger batch sizes should produce better 2017.
gradient estimates and therefor work better, our results here [3] Y. LeCun, K. Kavukcuoglu, and C. Farabet, “Convolutional networks
show that the opposite is true. Smaller batch sizes, even as and applications in vision.,” in Proceedings of the IEEE International
Symposium on Circuits and Systems (ISCAS), pp. 253–256, IEEE, 2010.
small as 1, produce the best results. We suspect that the added
noise of smaller batch sizes allows SGD to better escape poor [4] K. Alex, I. Sutskever, and G. E. Hinton, “Imagenet classification with
deep convolutional neural networks,” in Advances in Neural Information
local optima and thus perform better overall. Processing Systems 25 (NIPS 2012) (F. Pereira, C. J. C. Burges,
L. Bottou, and K. Q. Weinberger, eds.), pp. 1097–1105, 2012.
The recognition results are shown by confusion matrix [5] H. Suk, S. Lee, and D. Shen, “Hierarchical feature representation
achieved on the DSB dataset with 3D CNN as shown in Table and multimodal fusion with deep learning for AD/MCI diagnosis,”
IV. As shown from the Table IV, Accuracy of model is 86.6%, NeuroImage, vol. 101, pp. 569–582, 2014.
Mis-classification rate is 13.4%, False positive rate is 11.9%, [6] G. Wu, M. Kim, Q. Wang, Y. Gao, S. Liao, and D. Shen, “Unsupervised
and False Negative is 14.7%. Almost all patients are classified deep feature learning for deformable registration of mr brain images.,”
correctly. Additionally, there is an enhancement on accuracy Medical Image Computing and Computer-Assisted Intervention, vol. 16,
no. Pt 2, pp. 649–656, 2013.
due to efficient U-Net architecture and segmentation.
[7] Y. Xu, T. Mo, Q. Feng, P. Zhong, M. Lai, and E. I. Chang, “Deep learn-
ing of feature representation with multiple instance learning for medical
image analysis,” in IEEE International Conference on Acoustics, Speech
Table IV: Confusion Matrix of 3D CNN using 30% Testing and Signal Processing, ICASSP, pp. 1626–1630, 2014.
[8] D. Kumar, A. Wong, and D. A. Clausi, “Lung nodule classification using
Predicted deep features in ct images,” in 2015 12th Conference on Computer and
Abnormal Normal Robot Vision, pp. 133–138, June 2015.
Actual [9] Y. Bar, I. Diamant, L. Wolf, S. Lieberman, E. Konen, and H. Greenspan,
Abnormal 0.853 0.147 “Chest pathology detection using deep learning with non-medical train-
Normal 0.119 0.881 ing,” Proceedings - International Symposium on Biomedical Imaging,
vol. 2015-July, pp. 294–297, 2015.
[10] W. Sun, B. Zheng, and W. Qian, “Computer aided lung cancer diagnosis
with deep learning algorithms,” in SPIE Medical Imaging, vol. 9785,
pp. 97850Z–97850Z, International Society for Optics and Photonics,
VIII. C ONCLUSION 2016.
[11] J. Tan, Y. Huo, Z. Liang, and L. Li, “A comparison study on the effect of
In this paper we developed a deep convolutional neural net- false positive reduction in deep learning based detection for juxtapleural
work (CNN) architecture to detect nodules in patients of lung lung nodules: Cnn vs dnn,” in Proceedings of the Symposium on
cancer and detect the interest points using U-Net architecture. Modeling and Simulation in Medicine, MSM ’17, (San Diego, CA,
This step is a preprocessing step for 3D CNN. The deep 3D USA), pp. 8:1–8:8, Society for Computer Simulation International,
2017.
CNN models performed the best on the test set. While we
[12] R. Golan, C. Jacob, and J. Denzinger, “Lung nodule detection in ct im-
achieve state-of-the-art performance AUC of 0.83, we perform ages using deep convolutional neural networks,” in 2016 International
well considering that we use less labeled data than most state- Joint Conference on Neural Networks (IJCNN), pp. 243–250, July 2016.
of-the-art CAD systems. As an interesting observation, the [13] Kaggle, “Data science bowl 2017.” [Link]
first layer is a preprocessing layer for segmentation using science-bowl-2017/data, 2017.
different techniques. Threshold, Watershed, and U-Net are used [14] LUNA16, “Lung nodule analysis 2016.” [Link]
to identify the nodules of patients. [Link]/, 2017.
[15] M. Firmino, A. Morais, R. Mendoa, M. Dantas, H. Hekis, and
The network can be trained end-to-end from raw image R. Valentim, “Computer-aided detection system for lung cancer in
patches. Its main requirement is the availability of training computed tomography scans: Review and future prospects,” BioMedical
database, but otherwise no assumptions are made about the Engineering OnLine, vol. 13, p. 41, 2014.
objects of interest or underlying image modality. [16] S. Hawkins, H. Wang, Y. Liu, A. Garcia, O. Stringfield, H. Krewer,
Q. Li, D. Cherezov, R. A. Gatenby, Y. Balagurunathan, D. Goldgof,
M. B. Schabath, L. Hall, and R. J. Gillies, “Predicting malignant nodules
In the future, it could be possible to extend our current from screening ct scans,” Journal of Thoracic Oncology, vol. 11, no. 12,
model to not only determine whether or not the patient has pp. 2120–2128, 2016.
cancer, but also determine the exact location of the cancerous [17] M. S. AL-TARAWNEH, “Lung cancer detection using image processing
nodules. The most immediate future work is to use Watershed techniques,” Leonardo Electronic Journal of Practices and Technolo-
segmentation as the initial lung segmentation. Other oppor- gies, pp. 147–158, June 2012.
tunities for improvement include making the network deeper, [18] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional net-
and more extensive hyper parameter tuning. Also, we saved works for biomedical image segmentation,” CoRR, vol. abs/1505.04597,
our model parameters at best accuracy, but perhaps we could 2015.
have saved at other metrics, such as F1. Other future work [19] M. D. Zeiler, M. Ranzato, R. Monga, M. Mao, K. Yang, Q. V. Le,
P. Nguyen, A. Senior, V. Vanhoucke, J. Dean, and G. E. Hinton, “On
include extending our models to 3D images for other cancers. rectified linear units for speech processing,” in IEEE International
The advantage of not requiring too much labeled data specific Conference on Acoustics, Speech and Signal Processing, pp. 3517–
to our cancer is it could make it generalizable to other cancers. 3521, May 2013.

[Link] 416 | P a g e
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 8, No. 8, 2017

[20] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities

improve neural network acoustic models,” in Proc. ICML, vol. 30, 2013.
[21] L. Bottou, Large-Scale Machine Learning with Stochastic Gradient
Descent, pp. 177–186. Augest 2010.
[22] I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance
of initialization and momentum in deep learning,” in Proceedings of the
30th International Conference on International Conference on Machine
Learning, ICML’13, pp. 1139–1147, [Link], 2013.
[23] H. Han, L. Li, H. Wang, H. Zhang, W. Moore, and Z. Liang, “A novel
computer-aided detection system for pulmonary nodule identification
in ct images,” in Proceedings of SPIE Medical Imaging Conferenc,
vol. 9035, 2014.

[Link] 417 | P a g e

View publication stats