Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Contents lists available at ScienceDirect

Chemometrics and Intelligent Laboratory Systems

journal homepage: [Link]/locate/chemometrics

Deep adversarial data augmentation for biomedical spectroscopy:

Application to modelling Raman spectra of bone
Eleftherios Pavlou, Nikolaos Kourkoumelis *
Department of Medical Physics, School of Health Sciences, University of Ioannina, 45110, Ioannina, Greece

A R T I C L E I N F O A B S T R A C T

Keywords: Deep learning algorithms have performed remarkably well to predict state of health. Nevertheless, they typically
Generative adversarial networks rely on ample training data to avoid overfitting. In the biomedical sector, sufficient data are not typically
Raman spectroscopy available due to low availability or accessibility. Data augmentation of physiological recordings can be achieved
Bone
using Generative Adversarial Networks (GAN). GAN is a computational framework for approximating generative
Deep learning
models within an adversarial process, where two neural networks compete against one other while being trained
simultaneously. Despite the widespread use and adoption of deep learning algorithms in life sciences, concerns
have been raised about the lack of biological context. Therefore, to assess a data augmentation workflow, both
computational and physiological quality metrics must be considered. Raman spectroscopy can be effectively used
to study the molecular properties of bone tissue. Both inorganic and organic phases can be analysed simulta­
neously as probes of bone health status. In this work, we describe an easy-to-follow GAN approach for generating
synthetic Raman spectra from a small dataset of ex vivo healthy and osteoporotic bone samples. The model was
applied to raw Raman spectra, while it can be modified accordingly to produce any one-dimensional biomedical
signal. We also introduced a novel unsupervised methodology to evaluate the variability of the synthetic dataset,
based on successive Principal Component Analysis (PCA) modelling. The properties of the synthetic spectra were
scrutinized by Fréchet Distance and difference spectroscopy, as well as by bone quality metrics, like mineral-to-
matrix ratio and crystallinity. Finally, classification studies demonstrated the increased discrimination accuracy
of the augmented dataset.

1. Introduction dimensionality. However, it requires comprehensive training data which

are not always available, especially when biological samples are
Deep learning is a subfield of machine learning techniques that involved. Hence, it is expected that the range of application for machine
assemble a network of sequential processing layers of data representa­ learning techniques is vastly characterized by the training data set
tion. Depending on the complexity of the raw data and the learning limitations. A suitable training dataset should be of adequate size (i.e.,
performance of the layered representations, modern deep learning can total number of samples) and quality which ultimately refers to data
expand to tens or hundreds of layers. The entire procedure is carried out consistency and adequate sample variability for the problem space.
using neural network models based on how the brain processes infor­ Goodfellow et al. [1] introduced Generative Adversarial Networks
mation, using progressive sparse encoding of input data. Despite the (GANs) as a framework for approximating generative models using an
successful application and increased dissemination of deep learning al­ adversarial process in which two neural networks compete against one
gorithms in life sciences, concerns are expressed regarding the absence other while being trained simultaneously. Both networks seek to mini­
of biological context in outcomes and the inherently non-transparent mize a cost function towards Nash equilibrium when neither can opti­
multilayer modelling structure of neural networks. Nonetheless, ma­ mize their loss unilaterally. In this competitive min/max game, GANs
chine learning is a well-established, but still evolving, technique mostly generate new samples which are indistinguishable from the ones in the
aiming to classification and regression problems. In particular, it is well original dataset, making predictions via unsupervised learning. Still,
suited for high-dimensional data, surpassing conventional techniques GANs can be also trained under semi-supervised learning, where
which are often limited by the number of hyperparameters or high adequate generalization can be achieved using both labelled and

* Corresponding author.
E-mail address: nkourkou@[Link] (N. Kourkoumelis).

[Link]
Received 18 March 2022; Received in revised form 26 July 2022; Accepted 27 July 2022
Available online 1 August 2022
0169-7439/© 2022 Elsevier B.V. All rights reserved.
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Fig. 1. The GAN model starts with a random-noise signal and transforms the generated data into new spectra through the loss function.

unlabelled data [2]. energy levels are quantized, Raman spectra contain chemical bond-
Augmentation of physiological data with GANs has some clear ad­ specific bands, providing a "molecular fingerprint" of the measured
vantages, especially in the biomedical sector. Firstly, generating syn­ sample. Raman active molecules, featuring anisotropic polarizability,
thetic medical data can be beneficial when there is low availability or provide spectroscopic signals with significant chemical and composi­
accessibility of original data. Often, even if data records are accessible, tional information. It is also an excellent technique to study biological
they are either expensive, or they require complex procedures to fulfil specimens [21–24] since it does not interfere with water molecules, is
the legal requirements set in place for proper usage. Secondly, robust non-invasive, and requires minimal sample preparation; nonetheless,
deep learning models require test error that decreases with training tissue auto-fluorescence interference and a low SNR are common limi­
error. With augmented data, a comprehensive set of synthetic, but tations. The application of GANs to Raman spectroscopic studies is
realistic, data points will be produced to reduce the risk of overfitting limited to one recent study [25]. Wu et al. designed a data augmentation
while minimizing the distance between the training and test sets [3]. module for skin cancer classification using a convolutional neural
Similarly, GANs can overcome performance obstacles due to bias caused network (CNN) as a classifier of pre-processed Raman spectra. Using
by class imbalance (i.e., when a class label is much more common than data augmentation, the proposed GAN improved the classification ac­
others) [4]. curacy among BCC (basal cell carcinoma), SCC (squamous cell carci­
The in-silico synthesis of biomedical signals is a challenging task as noma) and healthy skin samples when compared to other supervised
each signal possesses distinct characteristics that depend on the mea­ methods, like logistic regression and support vector machines.
surement technique, the probed tissue, signal-to-noise ratio (SNR), the In this work, we propose a simple GAN scheme for generating syn­
tissue state, and the underline pathology. Recently, a growing number of thetic Raman spectra using a limited dataset of ex vivo healthy and
data augmentation studies that are using GANs for the synthesis of osteoporotic bone samples. The model can be applied to raw (unpro­
biomedical signal have been proposed. Most of them reported results on cessed) Raman spectra and can be modified accordingly to produce any
image generation [5–15] and only few studies focused on the generation one-dimensional biomedical signal. It can also be reused to generate a
of low-dimensional data structures, mainly related to electrocardiogram theoretically infinite amount of synthetic data using a pseudo-random
(ECG) and electroencephalogram (EEG) [16–20]. generator. Unlike the supervised evaluation approach that is typically
Raman spectroscopy relies on the inelastic scattering of photons used to ensure the synthetic procedure reliability towards the real data,
exchanging energy via molecule vibrational modes. Because molecular here we propose an unsupervised methodology based on Principal

Fig. 2. The Neural Network architecture showing the G and D models.

2
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Component Analysis (PCA) to assess the variability of the augmented diversity of the synthetic spectra in each run.
dataset. We further scrutinize the spectral outcomes by analysing bone Raman spectra encapsulate a large volume of data that is not directly
quality parameters of the two spectra datasets and we perform classifi­ interpretable, especially when biological samples are involved which
cation studies to examine the discrimination accuracy of the augmented typically feature low SNR and overlapping vibration bands. Hence, the
dataset. Finally, unlike most similar studies on GANs, the source code is most significant barrier to synthetic data adoption is the lack of gener­
publicly available on GitHub at: [Link] alization that stems from the decreased variability of the of the synthetic
data. To overcome this obstacle during training, the weights of the
2. Materials and methods generator model were saved into a checkpoint file (inherently supported
by TensorFlow) every two epochs. The last epochs after the stationary
2.1. Model architecture state between the generator and the discriminator, were subsequently
employed to generate an equal number of synthetic spectra as the
GANs rely on a zero-sum game and consist of the generative model training set. Yet, an arbitrary number of spectra can be generated as a
(G, generator) and the discriminative model (D, discriminator). The pseudo-random seed in the code can produce a theoretically infinite,
input of the generator is a stochastic noise signal Z with Pz data distri­ slightly different, set of spectra each time while, in parallel, a different
bution. Here, the input noise was a low dimensional vector with values number of epochs can be selected to generate the artificial spectra.
from a normal distribution. The output is the generated spectra G(z) The second step of our GANs-centric methodology was to develop a
with a data distribution that resemble the true data distribution Pdata of process to keep fake spectra with adequate noise and variance in the
the raw data. An optimal GAN will transform the discrete distribution of final synthetic dataset and to discard those that don’t meet such re­
the training dataset into a continuous distribution, by simultaneously quirements, having excessive variability (i.e., being an outlier). To that
learning each source of variance in the dataset. The discriminator ac­ end, we implemented a scavenging-PCA methodology where we per­
cepts as input both the generated spectra G(z) and the true spectra (X) formed a successive PCA-based refinement of the synthetic spectra set by
while its output is the probability of the spectra being good or bad sequentially removing the generated spectrum with the longest distance
replicates of the real data. In the simplest approach, the cross-entropy from the 95% confidence ellipse in the PCA scores plot of the real
loss is described by a min/max optimization process. During training, measurements. The ellipse is the empirical description of the variability
the generator attempts to minimize the following function while the of the samples and in principle defines the region that contains 95% of
discriminator to maximize it: all samples that can be drawn from the underlying distribution. At the
{ } end of the “scavenging” iteration, only synthetic spectra having a similar
minmax ​ Ex∼pdata [log D(x)] + Ez∼pz [1 − log D(G(z))] (1) variance with the real spectra remained. To our knowledge, this is the
G D
first time that an unsupervised technique is used for post-processing
where D(x) is the discriminator’s estimate of the probability that real GANs-generated signals without being applied on the signal itself like
data sample x is real, Ex is the expected value over Pdata, G(z) is the post-generation noise reduction methods [29]. In this work, we propose
generator’s output spectra, D(G(z)) is the discriminator’s estimate of the a methodology able to generalize to similar spectra or time-series signals
probability that a fake spectrum is real, and Ez is the expected value over (in a broad sense, Raman spectra can be also considered as a time-series
Pz. The typical GAN structure, that is also used in this work, is presented of frequencies denoted as Raman shifts) without the need of signal pre-
in Fig. 1. or post-processing for data augmentation per se.

2.2. Implementation 2.3. Comparison and validation

The model was implemented and trained in Keras v2.6.0 and Ten­ There is no consensus on which GANs algorithms perform better in
sorFlow v2.6.0 using Python v3.7.11. The overall network architecture specific domains, despite the fact that numerous different types of GANs
is depicted in Fig. 2. In theory, a trained GAN generator is the opposite of have been suggested for either tabular or image data, utilizing unsu­
a classifier and produces a realistic spectrum from a random input latent pervised or conditional learning [30]. The evaluation is problematic due
vector. Therefore, the first step is to input the noise vector and to upscale to the lack of a robust metric consistent across multiple domains. For
the network to an array that can hold the dimensions of each spectrum example, log-likelihood on the test set cannot be approximated for
(i.e., the increasing wavelength). For the generator we used a network of high-dimensional data and the generator’s data distribution cannot be
fully connected layers with LeakyReLU activation function and Batch explicitly computed [31]. Specifically for GANs, apart from the simple
Normalization to speed up the training process and increase the qualitative comparison, two evaluation indices have been proposed:
robustness of the network [26]. The discriminator is a CNN that will take Inception Score (IS) [32] and Fréchet Inception Distance (FID) [33].
a spectrum as an input and will return a binary value as output. We used Both metrics were developed for image generation. The former evalu­
dropout layers after each convolution to avoid overfitting [27] while we ates the entropy of the distribution of the generated images and has
used maxpooling1D to downsample the input to the required dimensions received criticism regarding its reliability [32]. Conversely, FID com­
of the original Raman signal and to extract the extreme features of the pares the distribution of the generated images to that of the generator’s
spectra at the output. Batch size was set to 4. In general, small batch size training set. In this work, we evaluated our results using: (i) discrete FID
training has been shown to yield improved generalization performance (dFID) on the processed spectra, (ii) qualitative metrics by comparing
and allows for a smaller memory footprint [28]. The batch size dictates the spectrum signal using difference spectroscopy and (iii) quantitative
the number of training samples that will be propagated in a pass through metrics, by comparing bone quality parameters like mineral-to-matrix
the network. Hence, the batch size affects the rate of learning and has a ratio and crystallinity of the synthetic spectra, aiming to replicate
major impact on the stability of the learning process. The default batch normal and osteoporotic bones Raman measurements. The third crite­
size in Keras is 32. However, due to the small number of spectra in the rion is probably the most important in the case of biomedical Raman
training set, we performed an initial test grid-search of batch sizes of 2, spectra, since it may be more impactful to generate realistic features
4, 8, 12, 16, and 32, and observed the stability of the model via the than just datasets of similar Raman intensities. This semantic resem­
training loss graph. The Adam optimizer was used with a learning rate of blance provides clearer evidence that our GANs implementation covers
1e-5 and the default beta of 0.9 [29]. Random splitting to test and train enough search space of the training set rather than just memorizing the
data was carried out at 20:80 ratio, respectively. As possible bias is not spectra contour.
an issue in the GAN procedure (given the maintained model stability) we Although the GAN procedure was implemented on raw (unpro­
have chosen the simplest train/test split which can also increase the cessed) spectra, the subsequent analysis of quality metrics was

3
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Fig. 3. Output spectra per thousand epochs up to 104 (left to right; the scientific notation shows the completed number of epochs). (a) healthy bone (b) osteoporotic
bone. After approximately 6,500 epochs, a realistic bone spectrum was obtained.

performed on processed spectra for meaningful results. Therefore, for

evaluation purposes, all spectra (synthetic and real) were smoothed
using Savitzky-Golay transform (11-point moving window width and
second degree polynomial), baseline corrected using the Sensitive
Nonlinear Iterative Peak (SNIP) algorithm [34] and normalized to the ν1
phosphate peak intensity, post GAN run.

2.4. Samples

Raman measurements were performed on ex vivo rabbit bones sam­

ples which were obtained from female New Zealand white rabbits as
previously described [35,36]. Briefly, cortical sections from the diaph­
ysis of femur and tibia were dissected from healthy laboratory rabbits
and from rabbits with induced osteoporosis. In total, 90 spectra from
normal bones and 72 spectra from inflammation-mediated osteoporotic
bone were collected. Spectra were acquired at 30 mW output power,
with 785 nm laser at ~4.5 cm− 1 resolution and 5 s accumulation time, in Fig. 4. Clustering pattern of the normal vs. osteoporotic GAN generated spectra
the 400–1800 cm− 1 spectral range. Stokes Raman photons were detec­ (n = 144) using the t-SNE algorithm.
ted with a 2048 pixel thermoelectrically cooled back-thinned CCD.
be directly evaluated only after spectra pre-processing, as opposed to
3. Results and discussion image generation where plain visual inspection provides early clues. To
verify our model’s initial effectiveness, we plotted the distribution of the
The proposed GAN implementation was trained for 104 epochs generated spectra using the t-Distributed Stochastic Neighbour Embed­
although Nash equilibrium was observed sooner, past 6,500 epochs, ding (t-SNE) [38] method with the following parameters: perplexity =
when the synthetic signal achieved a realistic form. Fig. 3 shows output 30, iterations = 1,000, and learning rate = 15. T-SNE is a nonlinear
spectra generated per thousand epochs for the healthy and osteoporotic dimensionality reduction technique in which high-dimensional data are
bones respectively. To note, that different starting seeds can yield a mapped into a scatter plot of low-dimensional data while trying to
slightly different evolution and consequently a similar, but not identical, preserve the underlying correlation structure. In Fig. 4, 144 generated
set of synthetic spectra. The noisy spectrum observed at 8E3 epochs (8th spectra (72 of each class and equal to population of the training set of
panel, Fig. 3b) was attributed to a temporary instability of the modelling real spectra) were clustered into two groups. As expected, there is an
process, oscillating between spectra sets of variable noise. overlap between the classes since differences in the normal and patho­
GANs are difficult to train due to (i) the inherent instability of two logical bone tissue are not profound and only subtle molecular alter­
neural networks competing each other, (ii) the mode collapse event, ations can be observed by Raman spectroscopy. Similarly, the observed
when the generator produces repetitive samples covering only part of overlap among samples of the same class was also anticipated due to the
the distribution of the real samples and (iii) weak gradients, when the limited diversity in the training data set and the small sample size.
discriminator is by far more successful compared to the generator [37]. Nevertheless, the preliminary objective of the GAN model to produce
With limited training data, mode collapse is a common problem that can two classes of generated spectra, was achieved.

4
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Fig. 5. (a) normal; (b) osteoporotic bone spectra. 95% confidence ellipse de­
notes a probability of 0.95 that a new data observation (i.e., synthetic spec­
trum) will fall inside the ellipse avoiding the chance of being an outlier. In (b),
the distribution of the generated spectra is reduced compared to the total
variability distribution of the real dataset due to spectral differences of a subset
of osteoporotic specimens with high eigenvalues along PC1.

However, we must emphasize that although GANs can generate an

arbitrary number of spectra with sufficient class discrimination, the rigid Fig. 6. (a) normal; (b) osteoporotic bone spectra. The lower part of each sec­
requirement of preserving the features’ variability of the real samples tion is the difference spectra in the bone fingerprint range (400-1800 cm− 1).
must be kept. PCA is frequently used in spectroscopic analysis as it
condenses the variance of complex spectra into a smaller set of variables confirmed the robustness of our implementation with multiple re-runs of
via an unsupervised and tuning-free methodology [39]. Hence, PCA the training that produced similar results.
merely depends on the information within the Raman spectra man­ To infer the success or failure of any GAN infrastructure, apart from
ifested as signal information (peaks, noise, etc.), highly correlated in­ the visual interpretation, we need to set qualitative and quantitative
formation (background, broad bands) or even redundant information metrics. Two objective criteria, based on the FID and spectral subtrac­
(cosmic spikes, etc.). Consequently, it is free from stochastic optimiza­ tion were assessed, while a third one specific to bone quality was also
tion bias while the PCA-constructed model is succinct and based on the evaluated. The discrete FID algorithm tests the similarity between the
available data only. Although there is no theoretical definition of data spectral curves while the difference plot indicates intensity resemblance.
diversity, we can safely correlate data variance to diversity for signals of
low dimension such as spectra. As a result, PCA can (or should) be
3.1. Discrete Fréchet Inception Distance
included in the GANs workflow for signal generation as a means to
provide a bias-free assessment of the generated samples. Fig. 5 (a, b)
We assessed the dFID between the mean real and fake spectra for the
depicts the 95% confidence ellipses of the real and synthetic samples for
normal and osteoporotic samples. The calculated dFID values were 0.02
normal and osteoporotic bones. Each ellipse denotes an area that con­
and 0.03 for the normal and osteoporotic bones respectively. These
tains 95% of all samples that can be described from the underlying
values indicate the good quality and diversity of the generated signal.
normal distribution. Although a true correlation of data can be found
Conversely, the dFID similarity between the normal and osteoporotic
near the intersection of the two ellipses, our scavenging-PCA algorithm
raw samples was evaluated to 0.06. The outcomes suggest that: (i)
allowed only a small subset of synthetic samples. These were within the
Raman spectroscopy can expose minute differences between normal and
confidence ellipse of the real ones projected in the space of the first two
pathological tissue and (ii) the GAN structure was able to detect these
principal components (PC1 vs. PC2) which explain the larger portion of
discrepancies and properly attribute them to the synthetic spectra.
the total variance. Using the scavenging-PCA algorithm we ensure that
the generated data would not include outliers. By successive experi­
mentation we estimated that the acquired synthetic spectra were 3.2. Difference spectroscopy
approximately 10–20% of the training data set for each run of the
randomly initialized GANs scheme. A varying pseudo-random seed used Difference spectroscopy is an accurate tool for monitoring spectral
to modify the starting weights of the generator and discriminator [31] changes [40]. Typically, spectra subtraction is used to remove the
contribution of known components from a mixture spectrum.

5
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

Fig. 7. Processed mean spectrum of normal bone. Characteristic peaks of bone mineral (navy): phosphate ion (PO3− 2−
4 ), carbonate ion (CO3 ); the ν labelling refers to
standard vibration modes. Characteristic peaks of the organic matrix (red): CH2 deformation band, proline (Pro), hydroxyproline (Hyp), phenylalanine (Phe), amide
I, and amide III. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Subtraction works well even on overlapped peaks while it suffers from refers to a collection of structural and compositional factors that
increased noise between the reference and the sample spectrum. Setting determine bone strength and fracture resistance. These are: morphology
the mean real spectra as the reference and the mean synthetic spectra as (spatial distribution of mass), chemical composition, and biomechanical
the sample, we expect an almost flat line of intensity, if the spectra are attributes of bone tissue, such as stiffness, fatigue strength, and fracture
similar, using a subtraction factor of one unit. The corresponding dif­ toughness [41]. Vibrational spectroscopy evaluates the compositional
ference spectra for the two classes are presented in Fig. 6. The difference and chemical properties of the bone extracellular matrix (ECM).
spectra resemble a contour with small intensity variations across the Important “intrinsic determinants” of bone quality reflected on the ECM
wavenumbers range, indicating the successful replication of the real are mineral-to-matrix ratio (MMR; usually correlated to the degree of
samples. mineralization), and crystallinity (defined as the crystallite size/strain
and perfection of bone crystallites; increases with age and osteoporosis
due to the deterioration of structural and mechanical properties). In
3.3. Bone quality
Raman spectroscopy, the above metrics are evaluated by analysing the
corresponding vibrational peaks (Fig. 7). Specifically, MMR can be
Healthy or diseased bone can be characterized in qualitative and
assessed by the integrated area ratio of ν1PO3− 4 to the CH2 protein
quantitative index metrics around the term “bone quality”. Bone quality

Fig. 8. Bar plots of MMR and crystallinity across all samples (N–R: normal-real; N–S: normal-synthetic; O–R: osteoporotic-real; O–S: osteoporotic-synthetic). Sig­
nificant differences are marked with one and four asterisks (p < 0.05 and p < 0.0001, respectively); ns: non-significant. Data presented as the average ± the standard
error of the mean.

6
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

3.4. Classification of synthetic spectra

Using the scavenging-PCA procedure, 9 synthetic normal and 16

synthetic osteoporotic spectra were finally produced that comply with
the required bone quality indices mentioned before. The synthetic
spectra were accurately classified into two groups using PCA combined
with K-Nearest Neighbors (KNN) analysis (Fig. 9). The fact that the two
classes were discriminated despite the small number of samples, pro­
vides further indication that these spectra contain discriminative in­
formation similar to the original spectra.

3.5. Classification of spectra with data augmentation

As GAN data augmentation expands the training dataset, the per­

formance of the model is expected to improve along with its general­
ization. Principal Component (PC)-fed linear discriminant analysis
(LDA) was carried out to assess the classification accuracy of the
augmented dataset. The dataset was randomly split using stratification,
into training and test sets, in 60:40 ratio respectively, and the accuracy
of the classification was evaluated on the test sets, having approximately
99% accuracy for both real and real plus synthetic datasets. After con­
firming the unbiased estimate of the LDA performance, we retrained the
final LDA models on the entire datasets. PCA-LDA confirmed the data
Fig. 9. PCA-KNN analysis of the generated spectra after the scavenging-
augmentation function manifested by the improvement of the correctly
PCA procedure.
classified samples in the augmented dataset. The results are visualized in
the corresponding confusion matrix for each case (Fig. 10). Each row of
deformation band area (960/1450 cm− 1), and crystallinity by the in­ the confusion matrix denotes the predicted class of instances while the
verse of the full width at half maximum (FWHM) of the ν1PO3− 4 peak columns represent the actual class. The elements on the main diagonal of
fitted by a Gaussian curve at ~960 cm− 1 [42]. the matrix (i = j) show correctly classified spectra as either normal or
To interpret variance, we carried out statistical analysis using the osteoporotic. Considering that the dataset of the real spectra featured an
Brown-Forsythe equality of means test (α = 0.05), where variances are initial good classification between the normal vs osteoporotic bones,
not assumed to be equal, and normality is questionable especially for GAN augmentation can be considered as significant, given that it
small datasets (synthetic spectra). Multiple comparisons testing was enhanced the accuracy of the pathologic subset. In this study, the two
performed with unpaired t-test with Welch’s correction. Statistically categories used for the classification i.e., Raman spectra from normal
significant differences were found between the real Raman spectra of and osteoporotic bones, were well-separated. Thus, training a good
normal and osteoporotic samples. Furthermore, analogous differences classifier was feasible even with a small dataset. We expect that a larger
were observed between the synthetic spectra for MMR while no signif­ number of generated samples should be needed when the two original
icant difference was observed between normal and synthetic spectra of classes have significant overlapping features. Sordo and Zeng suggested
each class (Fig. 8). Therefore, synthetic spectra can blend into the that the size of the training set and the classification rate are correlated
dataset of real spectra in each class without being treated as distinct. The [43]. Therefore, the more samples generated, the larger the training
results further validate the capability of the constructed GAN model to data, the better the classification rate.
capture the correct variance of the real spectra in order to produce
synthetic spectra of adequate quality.

Fig. 10. Classification results: The augmented dataset misclassified one spectrum whereas two spectra were incorrectly assigned to the normal class in the original
dataset consisting of real spectra.

7
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

4. Conclusions [9] Ü. Budak, Z. Cömert, Z.N. Rashid, A. Şengür, M. Çıbuk, Computer-aided diagnosis
system combining FCN and Bi-LSTM model for efficient breast cancer detection
from histopathological images, Applied Soft Computing 85 (2019), 105765,
Data augmentation via GANs emerged as an effective technique for [Link]
increased accuracy and generalization of machine learning models. In [10] S. Pandey, P.R. Singh, J. Tian, An image augmentation approach using two-stage
theory, the use of GANs has the advantage of adding different instances generative adversarial network for nuclei image segmentation, Biomedical Signal
Processing and Control 57 (2020), 101782, [Link]
to the original dataset that have the same low-level features. In this bspc.2019.101782.
work, we tackled the challenging problem of performing data augmen­ [11] D. Dirvanauskas, R. Maskeliūnas, V. Raudonis, R. Damaševičius, R. Scherer,
tation to similar (i.e., from the same tissue) yet different (i.e., with HEMIGEN: human embryo image generator based on generative adversarial
networks, Sensors 19 (2019), [Link]
different physiology) biomedical Raman spectra of bone. We showed [12] M. Frid-Adar, I. Diamant, E. Klang, M. Amitai, J. Goldberger, H. Greenspan, GAN-
that two crucial requirements must be fulfilled to attempt data based synthetic medical image augmentation for increased CNN performance in
augmentation of biomedical data: (i) to ensure that the distribution of liver lesion classification, Neurocomputing 321 (2018) 321–331, [Link]
10.1016/[Link].2018.09.013.
the extended dataset agrees with that of the original dataset and (ii) to [13] X. Yi, P. Babyn, Sharpness-aware low-dose CT denoising using conditional
validate that the generated data set is similar but not identical to the generative adversarial network, Journal of Digital Imaging 31 (2018) 655–669,
original one. In algorithmic terms, the former constraint was achieved [Link]
[14] S.D. Wickramaratne, MdS. Mahmud, Conditional-GAN based data augmentation
using a novel scavenging-PCA algorithm, which forces new data to fall for deep learning task classifier improvement using fNIRS data, Front. Big Data. 4
within the 95% confidence ellipse of the original. The second restriction (2021), 659146, [Link]
was tested by evaluating dFID values and difference plots. In terms of [15] S. Motamed, P. Rogalla, F. Khalvati, Data augmentation using Generative
Adversarial Networks (GANs) for GAN-based detection of Pneumonia and COVID-
tissue physiology, the semantic resemblance between real and synthetic
19 in chest X-ray images, Informatics in Medicine Unlocked 27 (2021), 100779,
data was confirmed by established bone quality indicators like MMR and [Link]
bone crystallinity. To our knowledge, this is the first time that quality [16] D. Hazra, Y.-C. Byun, SynSigGAN: generative adversarial networks for synthetic
metrics of physiological parameters were critically assessed in GANs biomedical signal generation, Biology 9 (2020), [Link]
biology9120441.
modelling. The synthetic spectra generated by our small-scale GAN [17] F. Zhu, F. Ye, Y. Fu, Q. Liu, B. Shen, Electrocardiogram generation with a
model, showed good correlation with the respected class of the original bidirectional LSTM-CNN generative adversarial network, Scientific Reports 9
spectra, with adequate diversity. A slight improvement was observed in (2019) 6734, [Link]
[18] E. Piacentino, A. Guarner, C. Angulo, Generating synthetic ECGs using GANs for
the supervised classification of the augmented dataset which is expected anonymizing healthcare data, Electronics 10 (2021), [Link]
to increase further with the number of training samples. With small electronics10040389.
adjustments, the current approach can be applied to a variety of [19] V. Thambawita, J.L. Isaksen, S.A. Hicks, J. Ghouse, G. Ahlberg, A. Linneberg,
N. Grarup, C. Ellervik, M.S. Olesen, T. Hansen, C. Graff, N.-H. Holstein-Rathlou,
biomedical signals or time-series data. I. Strümke, H.L. Hammer, M.M. Maleckar, P. Halvorsen, M.A. Riegler, J.K. Kanters,
DeepFake electrocardiograms using generative adversarial networks are the
Funding beginning of the end for privacy issues in medicine, Scientific Reports 11 (2021),
21896, [Link]
[20] T. Luo, Y. Fan, L. Chen, G. Guo, C. Zhou, EEG signal reconstruction using a
This research is co-financed by Greece and the European Union generative adversarial network with wasserstein distance and temporal-spatial-
(European Social Fund- ESF) through the Operational Programme frequency loss, Front. Neuroinform. 14 (2020) 15, [Link]
fninf.2020.00015.
«Human Resources Development, Education and Lifelong Learning» in [21] N. Kourkoumelis, I. Balatsoukas, V. Moulia, A. Elka, G. Gaitanis, I. Bassukas,
the context of the project “Strengthening Human Resources Research Advances in the in vivo Raman spectroscopy of malignant skin tumors using
Potential via Doctorate Research” (MIS-5000432), implemented by the portable instrumentation, IJMS 16 (2015) 14554–14570, [Link]
ijms160714554.
State Scholarships Foundation (ІΚΥ).
[22] N. Kourkoumelis, G. Gaitanis, A. Velegraki, I.D. Bassukas, Nail Raman
spectroscopy: a promising method for the diagnosis of onychomycosis. An ex vivo
Declaration of competing interest pilot study, Medical Mycology 56 (2018) 551–558, [Link]
myx078.
[23] C. Petrokilidou, E. Pavlou, G. Gaitanis, I.D. Bassukas, M.N. Saridomichelakis,
The authors declare that they have no known competing financial A. Velegraki, N. Kourkoumelis, The lipid profile of three Malassezia species
interests or personal relationships that could have appeared to influence assessed by Raman spectroscopy and discriminant analysis, Molecular and Cellular
Probes 46 (2019), 101416, [Link]
the work reported in this paper. [24] M.Z. Vardaki, K. Seretis, G. Gaitanis, I.D. Bassukas, N. Kourkoumelis, Assessment of
skin deep layer biochemical profile using spatially offset Raman spectroscopy,
References Applied Sciences 11 (2021), [Link]
[25] M. Wu, S. Wang, S. Pan, A.C. Terentis, J. Strasswimmer, X. Zhu, Deep learning data
augmentation for Raman spectroscopy cancer tissue classification, Sci Rep 11
[1] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair,
(2021), 23842, [Link]
A. Courville, Y. Bengio, Generative Adversarial Networks, 2014. ArXiv:1406.2661
[26] S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training by
[Cs, Stat], [Link] (Accessed 14 February 2022).
reducing internal covariate shift, in: Proceedings of the 32nd International
[2] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, Improved
Conference on International Conference on Machine Learning - Volume 37, JMLR.
Techniques for Training GANs, 2016. ArXiv:1606.03498 [Cs], [Link]
Org, 2015, pp. 448–456.
org/abs/1606.03498. (Accessed 14 February 2022).
[27] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a
[3] C. Shorten, T.M. Khoshgoftaar, A survey on image data augmentation for deep
simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15
learning, J Big Data 6 (2019) 60, [Link]
(2014) 1929–1958.
[4] D. Kiyasseh, G.A. Tadesse, L.N.T. Nhan, L. Van Tan, L. Thwaites, T. Zhu, D. Clifton,
[28] D. Masters, C. Luschi, Revisiting Small Batch Training for Deep Neural Networks,
PlethAugment: GAN-based PPG augmentation for medical diagnosis in low-
2018. [Link] (Accessed 24 July 2022).
resource settings, IEEE Journal of Biomedical and Health Informatics 24 (2020)
[29] D.P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, 2017. [Link]
3226–3235, [Link]
[Link]/abs/1412.6980. (Accessed 24 July 2022).
[5] Z. Qin, Z. Liu, P. Zhu, Y. Xue, A GAN-based image synthesis method for skin lesion
[30] A. Aggarwal, M. Mittal, G. Battineni, Generative adversarial network: an overview
classification, Computer Methods and Programs in Biomedicine 195 (2020),
of theory and applications, International Journal of Information Management Data
105568, [Link]
Insights 1 (2021), 100004, [Link]
[6] L. Lan, L. You, Z. Zhang, Z. Fan, W. Zhao, N. Zeng, Y. Chen, X. Zhou, Generative
[31] M. Lucic, K. Kurach, M. Michalski, S. Gelly, O. Bousquet, Are GANs Created Equal?
adversarial networks and its applications in biomedical informatics, Front. Public
A Large-Scale Study, 2018. ArXiv:1711.10337 [Cs, Stat], [Link]
Health 8 (2020) 164, [Link]
711.10337. (Accessed 14 February 2022).
[7] A. Tavakkoli, S.A. Kamran, K.F. Hossain, S.L. Zuckerbrod, A novel deep learning
[32] S. Barratt, R. Sharma, A Note on the Inception Score, 2018. ArXiv:1801.01973 [Cs,
conditional generative adversarial network for producing angiography images
Stat], [Link] (Accessed 14 February 2022).
from retinal fundus photographs, Sci Rep 10 (2020), 21580, [Link]
[33] F. Huszár, How (not) to train your generative model: scheduled sampling,
10.1038/s41598-020-78696-2.
likelihood, adversary?, ArXiv:1511.05101 [Cs, Math, Stat], [Link]
[8] V. Kearney, J.W. Chan, T. Wang, A. Perry, M. Descovich, O. Morin, S.S. Yom, T.
511.05101, 2015. (Accessed 14 February 2022).
D. Solberg, DoseGAN: a generative adversarial network for synthetic dose
[34] C.G. Ryan, E. Clayton, W.L. Griffin, S.H. Sie, D.R. Cousens, SNIP, a statistics-
prediction using attention-gated discrimination and generation, Sci Rep 10 (2020),
sensitive background treatment for the quantitative analysis of PIXE spectra in
11073, [Link]

8
E. Pavlou and N. Kourkoumelis Chemometrics and Intelligent Laboratory Systems 228 (2022) 104634

geoscience applications, Nuclear Instruments and Methods in Physics Research [39] J.R. Beattie, F.W.L. Esmonde-White, Exploration of principal component analysis:
Section B: Beam Interactions with Materials and Atoms 34 (1988) 396–402, deriving principal component analysis visually using spectra, Appl Spectrosc 75
[Link] (2021) 361–375, [Link]
[35] N. Kourkoumelis, A. Lani, M. Tzaphlidou, Infrared spectroscopic assessment of the [40] J. Grdadolnik, Infrared difference spectroscopy: Part I. Interpretation of the
inflammation-mediated osteoporosis (IMO) model applied to rabbit bone, J Biol difference spectrum, Vibrational Spectroscopy 31 (2003) 279–288, [Link]
Phys 38 (2012) 623–635, [Link] org/10.1016/S0924-2031(03)00018-3.
[36] A. Hadjipanteli, N. Kourkoumelis, P. Fromme, J. Huang, R.D. Speller, Evaluation of [41] N. Kourkoumelis, X. Zhang, Z. Lin, J. Wang, Fourier transform infrared
the 3D spatial distribution of the Calcium/Phosphorus ratio in bone using spectroscopy of bone tissue: bone quality assessment in preclinical and clinical
computed-tomography dual-energy analysis, Physica Medica 32 (2016) 162–168, applications of osteoporosis and fragility fracture, Clinic Rev Bone Miner Metab 17
[Link] (2019) 24–39, [Link]
[37] H. Chen, Challenges and corresponding solutions of generative adversarial [42] G.S. Mandair, M.D. Morris, Contributions of Raman spectroscopy to the
networks (GANs): a survey study, J. Phys.: Conf. Ser. 1827 (2021), 012066, understanding of bone strength, BoneKEy Reports 4 (2015), [Link]
[Link] 10.1038/bonekey.2014.115.
[38] L. van der Maaten, G.E. Hinton, Visualizing Data using t-SNE, Journal of Machine [43] M. Sordo, Q. Zeng, On sample size and classification accuracy: a performance
Learning Research 9 (2008) 2579–2605. comparison, in: J.L. Oliveira, V. Maojo, F. Martín-Sánchez, A.S. Pereira (Eds.),
Biological and Medical Data Analysis, Springer Berlin Heidelberg, Berlin,
Heidelberg, 2005, pp. 193–201.