Major Challenges and State-of-the-Art Solutions
The medical image segmentation field has gained advantage from deep learning, but still it is a
challenging task to employ deep neural networks due to the following.
The different challenges related to the dataset include the following:
Limited Annotated Dataset. Deep learning network models require large amount of data. The
data required for training is well annotated. The dataset plays an important role in various DL
based medical procedures [84]. In medical image processing, the collection of large amounts of
annotated medical images is tough [85]. Also, performing annotation on fresh medical images is
tedious and expensive and requires expertise. Several large-scale datasets are publicly available.
A list of few such datasets is provided in Table 2. There is still a need of more challenging
datasets which can enable better training of DL models and are capable of handling dense
objects. Typically, the existing 3D datasets [86] are not so large and few of them are synthetic,
so more challenging datasets are required.
The size of the existing medical image datasets can be increased by (a) application of image
augmentation transformations like rotating image by different angles, flipping image vertically or
horizontally, cropping, and shearing image. These augmentation techniques can boost the
system performance. (b) The application of transfer learning from efficient models can provide
solution to the problem of limited data [87]. (c) Finally comes synthesizing data collected from
various sources [87].
Class Imbalance in Datasets. Class imbalance is intrinsic in various publicly available medical
image datasets. A highly imbalanced data poses great difficulty in training DL model and makes
model accuracy misleading, for example, in a patient data, where the disease is relatively rare
and occurs only in 10% of patients screened. The overall designed model accuracy would be high
as most of the patients do not have the disease and will reach local minima [88, 89].
The problem of class imbalance can be solved by (a) oversampling the data; the amount of
oversampling depends on the extent of imbalance in the dataset. (b) Second, by changing the
evaluation or performance metric, the problem of dataset imbalance can be handled. (c) Data
augmentation techniques can be applied to create new data samples. (d) By combining minority
classes, dataset class imbalance problem can also be handled.
Sparse Annotations. Providing full annotation for 3D images is a time-consuming task and is not
always possible. So, partial labelling of information slices in 3D images is done. It is really
challenging to train DL model based on these sparsely annotated 3D images [85]. In case of
sparsely annotated dataset, weighted loss function can be applied to the dataset. The weights for
the unlabeled data in the available dataset are all set to zero, so as to learn only from the pixels
which are labelled.
Intensity Inhomogeneities. In pathology images, colour and intensity inhomogeneities [90] are
common. Intensity inhomogeneities cause shading over the image. It is more specific in the
segmentation of MR images. Also, the TEM images have brightness variations due to presence of
nonuniform support films. The segmentation process becomes tedious due to these variations.
For correcting intensity inhomogeneities [90], different algorithms are employed and many
nonparametric techniques are proposed in the literature. Prefiltering operation can be employed
before segmentation to remove inhomogeneities. Also, intensity inhomogeneities are taken care
of by improvement in scanning devices.
Complexities in Image Texture. In medical images, there may be different artifacts present
during manipulation of images. The different sensors and electronic components used for
capturing images create noise in the image [11, 91]. In the captured image, gray levels can be
very close to each other and there may be weak image boundaries. There may be overlap in
tissues and presence of irregularities like skin lines and hair in dermoscopic images. All these
complexities cause difficulty in identification of region of interest in medical images.