Journal of Manufacturing Systems 68 (2023) 660–669

Contents lists available at ScienceDirect

Journal of Manufacturing Systems

journal homepage: [Link]/locate/jmansys

Surface roughness prediction through GAN-synthesized power signal as a

process signature☆
Clayton Cooper a, Jianjing Zhang a, Y.B. Guo b, c, Robert X. Gao a, *
a
Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, USA
b
Department of Mechanical and Aerospace Engineering, Rutgers University-New Brunswick, Piscataway, NJ, USA
c
New Jersey Advanced Manufacturing Institute, Rutgers University-New Brunswick, Piscataway, NJ, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Predicting machined surface roughness is critical for estimating a part’s performance characteristics such as
Machining susceptibility to fatigue and corrosion. Prior studies have indicated that power consumed at the tool-chip
Surface roughness interface may represent an indicator for the surface integrity of the machining process. However, no quantita­
Process signature
tive association has been reported between the machining power and surface roughness due to a lack of data to
Generative adversarial network
Convolutional neural network
develop predictive models. This paper presents a data synthesis method to address this gap. Specifically, a
Machining power conditional generative adversarial network (CGAN) is developed to synthesize power signals associated with
varying process parameter combinations. The quality of the synthesized signals is evaluated against experi­
mentally measured power signals by examining the consistency in: 1) the spatial pattern of the signals induced by
the cutting process as shown in the frequency domain, and 2) the temporal pattern as shown in the clustering of
the synthesized and measured signals corresponding to the same parameter combination. The synthesized signals
are then used to augment the measured signals and develop a convolutional neural network (CNN) for predicting
the machined surface roughness. Experiments performed using H13 tool steel have shown that data augmen­
tation by CGAN has effectively reduced the error of the surface roughness prediction from 58 %, when no
synthetic data is used for CNN training, to 9.1 % when 250 synthetic samples are used. The results demonstrate
the effectiveness of CGAN as a data augmentation method and CNN for mapping machining power to surface
roughness.

1. Introduction instance, a roughness prediction equation is developed in [9] that relates

roughness to feed and tool nose radius, which does not include other
Surface roughness is one of the most important quality factors to be machining parameters, material properties, or stochastic process dis­
quantified in machining because it determines part functionality in turbances. This contradicts experimental results in [10] which show that
terms of fatigue resistance, corrosion susceptibility, and fluidic drag surface roughness depends on the specific cutting energy, which is a
[1–3]. Traditional roughness quantification relies on post-process in­ function of all process parameters as well as the workpiece condition.
spection after the workpiece is fabricated by means of contact and To comprehensively consider process parameters as well as in-
contactless profilometry, which often constitutes a bottleneck in process data, machine learning (ML) models have been established to
manufacturing in terms of time and cost [4,5]. To eliminate this predict surface roughness as a function of parameters and sensor data
bottleneck, in-process approaches that associate surface roughness with [11,12]. A limitation in parameter-based roughness quantification has
process parameters have been investigated. been that it does not consider the in-process variations caused by process
In recent years, statistically significant influences of process pa­ dynamics such as workpiece material imperfections and changing tool
rameters such as speed, feed, and depth-of-cut (DOC) on surface conditions that are commonly seen in real-world manufacturing envi­
roughness have been studied [6–8]. However, many of the theoretical ronments. To overcome this limitation, the incorporation of process
approaches do not reflect experimentally observed behavior. For signatures into surface roughness prediction models has attracted

☆
51st SME North American Manufacturing Research Conference (NAMRC 51, 2023)
* Corresponding author.
E-mail address: [Link]@[Link] (R.X. Gao).

[Link]

Available online 1 June 2023

0278-6125/© 2023 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved.
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

significant interest [13]. predict the roughness of the machined surfaces. A CNN is chosen as the
Acquired during the machining process, process signatures refer to prediction model given its strong image processing capabilities [26] and
the measured signals that quantify energy conversion as the workpiece previous successes analyzing time-frequency data in manufacturing
responds to applied loads [14,15]. As the energy-driven part modifica­ [27]. An overview of the proposed methodology is shown in Fig. 1.
tions imparted by a machine reflect the compound effect of process The contributions of this study are summarized as follows:
parameters and process dynamics, e.g., tool condition and material
imperfections that are not accounted for by parameter-based models, 1. Established a data-driven mapping from the machining energy pro­
process signatures can be used as complementary input to data-driven cess signature to surface roughness using CNN, requiring only in-
predictive models [10,16]. process data as input, and eliminating the need for post-process
Various process signatures, such as sound, force, and temperature, quantification.
have been investigated. Griffin et al. [17] investigated acoustic emis­ 2. Developed a CGAN-based method for high-fidelity synthesis of
sions as input to a classification tree to predict surface roughness in power signals, which are validated through consistency in signal
micromachining. Wu and Lei [18] used features from vibration signals patterns from both the physical and statistical aspects, thereby
as input to a neural network to predict the roughness of conventionally alleviating the limitations in data collection for surface predictive
machined parts. Similarly, Kong et al. [19] used vibration signal features modeling.
as input to a Bayesian linear regression model for roughness prediction. 3. Achieved good surface roughness prediction accuracy by CNN as
Wang et al. [20] developed a joint convolutional and recurrent deep demonstrated by comparing the synthesized data with experimen­
learning model to fuse spindle power signals with machined surface tally acquired ground truth.
images for simultaneous tool wear inference and surface roughness
prediction. Tian et al. [21] combined and built upon these approaches to The remainder of the paper is organized as follows: Section 2 pro­
develop a surface prediction model with acoustic, vibrational, force, and vides background on surface roughness and CGANs. Section 3 describes
power signal features as input to a neural network. A common limitation the experiment to evaluate the developed data synthesis and prediction
in the process signatures investigated in these studies is that they only methods. In Section 4, power signal synthesis and surface roughness
considered a portion of the energy transferred into the workpiece, e.g., results are presented and discussed. Conclusions and future research
vibrational and acoustic dissipations, thus represent an incomplete directions are summarized in Section 5.
characterization of the total energy input to the workpiece and its effects
on the surface formation. Others, like Moliner-Heredia et al. [22], Wang 2. Background and methodology
et al. [23], and Corne et al. [24], have leveraged process signatures to
predict tool condition but not the corresponding surface roughness. To During machining, material is removed from the workpiece through
establish a direct link between the energy as a process signature and shear deformation as shown in Fig. 2(a) [28]. Energy is delivered to the
surface roughness and fill an existing research gap, this study presents a workpiece via the motion of the cutting tool that removes a small “chip”
data-driven surface roughness predictive model based on energy input of the material. The gross power demand of machining can be modeled
only. as:
Given that specific power measurement instruments are needed to
specifically quantify the amount of energy used for material removal Pcut = MRR(PP)⋅SFC (PP, TC) + ξ(t, TC, MP) (1)
during machining, as well as the time-consuming nature of experimental
where MRR is the material removal rate (mm3/s) as a function of related
data acquisition, only a limited amount of power/energy data is typi­
process parameters (PP), SFC is the specific cutting force (N/mm2) as a
cally available. To address this limitation, a data synthesis approach has
function of PP and tool condition (TC), and ξ represents power fluctu­
been investigated based on the generative adversarial network (GAN)
ations caused by regenerative vibration, etc., which is dependent on
architecture. GANs rely on a data generator and a discriminator as a pair
time (t), TC, and material properties (MP) [25]. For milling operations,
of competing neural networks for the training process to arrive at an
MRR and SFC are defined as:
equilibrium point for high-fidelity data synthesis. However, standard
GANs operate on random input with no consideration of application- MRR = ap ae v (2)
relevant data such as process parameters in machining operations. For
[ ( ]
this reason, standard GANs do not learn the effects of these variables on fz 1 − cosϕs ) β+1
the expected output and the synthesized data is difficult to control and SFC = kap (3)
ϕs
interpret. To overcome this challenge, a conditional generative adver­
sarial network (CGAN) has been developed to synthesize power signals where ap is the axial depth of cut (mm), ae is the radial depth of cut
based on a limited number of measured power data. In the presented (mm), v is the cutting feed (mm/s), fz is the feed per tooth (mm/tooth),
study, the CGAN is regularized by the pertinent machining process pa­ ϕs is the swept angle of removed material (rad), and k and β are
rameters to capture the physical effects of varying parameters on the experimentally-derived constants dependent on the tool geometry [29].
power signal patterns and ensure the output of is physically sound and Three plots are shown at the bottom of Fig. 2 to illustrate variations
meaningful. in the machining power signal as cutting parameters change: feed in (b),
The validity of the synthesized power signals is then verified from speed in (c), and DOC in (d). In each plot, the solid lines represent
both the physical and the statistical aspects. The former involves measured power signals while the dotted lines represent the theoretical
quantifying the consistency in the spatial pattern induced by the specific power signals. As seen in these plots, changes in the cutting parameters
cutting parameters as reflected in the frequency domain of the power shift the power curves (both measured and theoretical) up and down,
signals. The latter reveals the statistical distances among the synthesized indicating the parameters have a pronounced effect on the power de­
intra- and inter-class samples (e.g., samples corresponding to the same mand of machining. Deviations of the measured power from the theo­
or different parameter combinations, respectively), to confirm the retical level can be attributed to process variations, i.e., material
capability of CGAN in generating data that exhibit clustering of intra- imperfections, which induce variations through grain boundaries and
class samples and separation among inter-class ones. inhomogeneities that, in turn, cause the cutting forces and correspond­
Finally, in consideration of previous work relating power signal ing power signal to change. As these imperfections are located randomly
frequencies with tool condition and surface roughness [25], the syn­ throughout the material being machined, their manifestation in the
thesized signals are decomposed into time-frequency images via wavelet power signals (as deviations from theoretical levels) also exhibit
transform and used to train a convolutional neural network (CNN) to randomness.

661
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

Fig. 1. Surface roughness prediction based on augmented power signals and machine learning.

Fig. 2. Chip removal and surface roughness formation during cutting (a); examples of cutting power demand fluctuations over breadth of cutting parameters (b–d);
DOC: radial depth-of-cut.

During the material removal process, variations in the material as

described above as well as dulling along the cutting edge cause plowing
or “ripping” instead of shearing at the microscopic level [30]. This
tensile mode of material removal induces a rough surface profile behind
the cutting tool as shown in Fig. 3. This surface formation is inherent to
all mechanical material removal processes, including milling, turning,
and grinding, and indicates the importance of roughness prediction
across a breadth of metal removal processes.
The surface roughness quantity of interest in this study is the mean
deviation from the profile centerline, Ra:
∫ l1
1
Ra = |h(l) − h|dl (4)
l1 − l0 l0

where l0 and l1 are the starting and stopping point of the stylus used to
trace the surface, respectively, h is the traced surface profile, and h is the Fig. 3. Machined surface and roughness measurement using stylus (contact
profile’s mean value [31]. Because surface roughness is determined by profilometry).

662
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

the energy transfer at the tool-chip interface, which is in turn deter­ responding to a perfect discriminator performance). Such adversarial
mined by the machining power, it is hypothesized that machining power training is envisioned to arrive at a Nash or proximal equilibrium where
can be mapped to surface roughness by way of a data-driven model using the discriminator can no longer distinguish the synthetic data from the
ML methods to allow for in-process roughness inference instead of real ones, and the generator can synthesize high-fidelity data but can no
time-consuming post-process inspection. However, to develop a proper longer improve either, due to the lack of further guidance from the
ML model, an adequate power signal dataset covering a broad range of discriminator [33,34]. In practice, optimization (5) is carried out in an
cutting parameters under various operational conditions is needed, average sense, by using expectations rather than individual samples.
which is time-consuming to collect experimentally [32]. To alleviate this ℓD
limitation and support energy-based data-driven predictive modeling, ⏞̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏟⏟̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏞
(5)
′ ′
G , D = minG maxD Ey [log D(y)] + Ez [log(1 − D(G(z)))]
the development of CGAN for high-fidelity power data synthesis con­ ⏟̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏞⏞̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏟
stitutes the focus of this study. ℓG

where G′ and D′ are optimized G and D, and the denoted loss quantities
2.1. Fundamentals of GAN
(l ∗ ) are the objective functions when training the generator and
discriminator models.
The objective of generative modeling is to approximate the statistical
In recent years, GANs have emerged as effective tools for data syn­
distribution of real-world data (e.g., power signals) using a surrogate
thesis in manufacturing and related fields [35]. For example, Hertlein
model. This distribution can then be sampled to generate new obser­
et al. [36] developed a 2-D CNN GAN for topology optimization in ad­
vations, thereby alleviating data availability constraints. GANs have
ditive manufacturing to improve manufacturability, which was shown
attracted significant attention for this purpose due to their ability to
to be significantly faster than traditional topology optimization
leverage ML to closely learn relevant data distribution and produce
methods. Likewise, Schleich et al. [37] formulated a 2-D CNN GAN for
realistic synthetic output [33].
tolerance analysis that predicted shape-agnostic part deviations and
The basic structure of a GAN is shown in Fig. 4. Generator G receives
enabled expected error stack-up to be calculated before production took
random vector z and outputs synthetic data ̃y. Meanwhile, discriminator
place. Cooper et al. [38] used a 2-D CNN GAN to synthesize milling
D randomly receives ̃y or ground truth data y and distinguishes the input
audio signals as spectrograms and develop a tool condition classification
as either synthetic or real, encoded as 0 and 1, respectively. During the
model using a mix of experimental and synthetic data. Barua et al. [39]
training process, the discriminator’s goal is to minimize the classifica­
proposed a joint FC and 2-D architecture for multi-view object synthesis
tion error whereas the generator’s goal is to maximize the classification
that was shown to be extensible over a breadth of datasets and learning
error by outputting realistic images that mislead the discriminator, i.e.,
problems. Qu et al. [40] and Liu et al. [41] developed GANs to syn­
G attempts to learn a surrogate to the unknown ground truth data dis­
thesize speed, temperature, and vibration data for wind turbines using
tribution Ωy . As a result, G and D are adversaries to one another and GAN
simultaneous, cross-sensor 1-D convolution operations such that a
training is a minimax game represented by the optimization problem as
sensor fusion fault detection system could be developed. By generating
expressed in (5):
outputs based on limited training data, GANs have shown to reduce the
Specifically, the first term of the sum in (5) measures how well the
need for extensive experimentation, conserve resources, and improve
discriminator performs on the ground truth samples. In this scenario, the
the robustness of predictive models by expanding the data availability
output from the discriminator contributes to the loss function only when
for developing such models.
it classifies a real sample as synthetic, or D(y) = 0. This is achieved
through the logarithm of the discriminator’s output such that log(D(y) )
= − c (contribution of -c to the loss function), where c is a constant by 2.2. CGAN for synthesis of cutting power profile
convention. Similarly, the second term measures how well the generator
output can mislead the discriminator. In this case, the output from the Numerous variants to the standard GAN structure have been pro­
discriminator contributes to the loss function only when it classifies a posed over recent years for a breadth of learning tasks. Of these, CGANs
synthetic image as real, or log(1 − D(G(z) ) = − c. are of significant interest to this study [42]. A prevalent issue with the
By minimizing the second term through training the generator, the standard GAN structure is that the generator output cannot be condi­
network weights are adjusted in the direction of reducing log(1 − tioned on the physical application-specific classes or labels. Since
D(G(z) ), thus making the generator produce synthetic samples that are varying process parameters have a direct impact on the formation of
less likely to be detected by the discriminator. In contrast, by maxi­ surface roughness as shown in Fig. 2, the synthesized power signals are
mizing both the first and the second terms through training the required to contain parameter-specific patterns to retain the
discriminator, the network weights are adjusted in the direction of roughness-related information embedded in different process parameter
increased log(D(y) ) and log(1 − D(G(z) ), making both closer to 0 (cor­ combinations (e.g., classes) to ensure the validity of predictive
modeling.
To solve this problem and enable class-dependent data synthesis,
CGAN is investigated in which the generator and discriminator outputs
are conditioned on additional class information u. Consequently, Eq. (5)
is modified as
⏞̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏟⏟̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏞ ℓD
(6)
′ ′
G , D = minG maxD Ey [log D(y|u) ] + Ez [log(1 − D(G(z|u)|u) ) ]
⏟̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏞⏞̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅⏟
ℓG

where conditioning is achieved by passing u as an input to both the

generator and the discriminator in addition to z or y, as shown in Fig. 4
This enables the generator to learn a unique output distribution corre­
sponding to each u while also allowing the discriminator to conduct
class-specific classification. In other words, G learns surrogates of Ω1 (u1 )
,Ω2 (u2 ),…,ΩN (uN ), where Ωn (un ) is the ground truth data distribution of
the nth class and N is the number of classes used to train the CGAN. D
Fig. 4. Fundamental structure of GAN and CGAN. then learns to determine whether the synthesized sample

663
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

G(z|un ) belongs to Ωn (un ). noted that the occurrence of these transient events forms different dis­
Training of a CGAN structure follows the same procedure as that of a tributions under different parameter combinations. For example, the
conventional GAN. Once trained, the generator can be queried with (z, occurrence exhibits a more regular pattern for specimens #1 and #8,
u) pairs to synthesize power signals corresponding to the class input u while appearing more sparsely in specimens #2, 5, 6, and 7. The distinct
and augment the dataset. For this study, u is a vector representing patterns in distributions indicate the underlying influence of process
process speed, feed, and radial depth-of-cut, since these physical attri­ parameters when coupled with process variations, and they can be
butes have been shown to effectively characterize the power signal [43]. verified physically against the ground truth by comparing the measured
and synthetic signals in the frequency domain, as well as statistically, by
3. Experimental evaluation assessing the statistical similarity between the real and synthetic data
using embedding techniques.
To validate the proposed surface roughness prediction method using
CGAN data augmentation, milling experiments have been performed at 3.2. Quantification of generated image quality
various speeds, feeds, and depths-of-cut to collect a set of spindle power
signals. To quantify consistency of the synthesized image signals with pro­
cess physics and statistical faithfulness to the ground truth data, two
metrics are considered: 1) the frequency spectra of the generated signals
3.1. Milling data collection and representation as compared to the ground truth, and 2) inter- and intra-class similarity
as quantified by the method of uniform manifold approximation and
A total of 21 prismatic specimens of AISI H13 tool steel (50 ± 1 HRC) projection (UMAP).
were dry milled on a CNC machining center using a 20 mm, two flute It is known from previous work that process parameter settings
end mill with tungsten carbide inserts. Six cuts were made using each set uniquely determine the frequency content of the machining power sig­
of the cutting parameters listed in Table 1, yielding 126 experimental nals [25]. For instance, the dominant frequency in the power signal
data. Spindle power was recorded during each cut at 341 kHz using a monotonically increases as cutting speed increases. The same relation­
Fluke Norma power analyzer. Following milling, the average Ra value in ship holds for the feed/tooth. Each set of parameters (u) should there­
the stepover direction is determined for each combination of cutting fore have a unique frequency spectrum associated with it. This
parameters using contact profilometry. observation serves as a basis of comparison for experimental and syn­
Previous studies [25] have shown that low-frequency components of thetic power signals, namely the frequency spectra of the real and syn­
machining power signals dissipate as tool wear increases. Since tool thesized data should be similar under the same process parameters. This
wear and surface roughness are closely linked, it is expected that the similarity can be quantified as:
frequency spectrum of the power signals contains surface ( ( ))
roughness-related information. Thus, to ensure the CGAN generates ξfreq = ρ F (P), F P̃ (7)
realistic power data in both the time and frequency domains of the
signal, continuous wavelet transform is first used to represent the power where ξfreq is the spectral similarity metric, ρ is the Pearson correlation
signals as time-frequency images, or scalograms [44]. The Ricker ( )
wavelet was selected as the base wavelet, based on the criterion of best [46] of the Fourier spectra F (P) and F P ̃ , P is a ground truth power
energy-to-entropy ratio as described in [45]. signal, P
̃ is a synthetic signal found via inverse wavelet transform [44] of
Representative wavelet scalograms for specimens #1–9 are shown in the generated scalogram, and both signals share the same u.
Fig. 5 with the corresponding power signals. In each scalogram, the If the generator has correctly established the unique mapping be­
peaks and valleys present along the time axis correspond to sudden tween the frequency distribution of the power signal and the physical
changes in the power demand caused by vibration, material imperfec­ information embedded in the process parameters, correlation between
tion, and tool deterioration. These events manifest themselves in the the real and synthetic frequency spectra will approach 1. This metric,
time-frequency domain as slender spikes reaching into the upper fre­ however, speaks only to each datum and does not assess if unique Ωn (un )
quency bands but quickly disappearing as the transient event ends. It is were learned for each combination of process parameters. For this
purpose, UMAP is investigated.
Table 1 The fundamental concept of UMAP is to visualize the distribution of
Milling parameters and measured surface roughness. high-dimensional data, e.g., scalogram images (or time-frequency
Spec. Speed Feed/tooth (mm/tooth) Radial DOC (mm) decomposition of the power signals) in this study, by learning a low-
Ra (nm)
(mm/min) dimensional surrogate distribution which maintains the distance be­
1 200 0.1 0.3 241 tween the data points proportional to their high-dimensional counter­
2 200 0.1 0.5 520 parts. As a result, statistically similar data clusters together whereas
3 200 0.1 0.5 384 dissimilar data separate. Details of UMAP are provided in [47].
4 200 0.05 0.5 360 To verify the result of power signal synthesis, the synthetic scalo­
5 200 0.1 0.4 646
grams conditioned on the same process parameter combination should
6 100 0.1 0.5 440
7 200 0.1 0.5 480 cluster together while maintaining separability from those conditioned
8 300 0.1 0.5 650 on different parameter combinations. Since different CGAN architec­
9 200 0.05 0.5 372 tures will yield different spectral correlations and UMAP embeddings,
10 200 0.1 0.4 329
several CGAN architectures are investigated.
11 100 0.1 0.5 399
12 300 0.1 0.5 396
13 200 0.1 0.3 426 3.3. CGAN architecture search
14 200 0.05 0.5 374
15 200 0.2 0.5 488
To examine the generator behavior across different CGAN architec­
16 100 0.1 0.5 356
17 200 0.1 0.3 265 tures, experiments have been performed to train fully connected (FC), 1-
18 200 0.1 0.4 399 D CNN, and 2-D CNN CGANs. A hyperparameter search is performed for
19 200 0.2 0.3 584 each model independently over the ranges listed in Table 2 to determine
20 300 0.1 0.5 657 the optimal network structure and training process. These same ranges
21 200 0.2 0.5 1059
are used to optimize the CNN using backpropagation once the generator

664
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

Fig. 5. Experimentally measured power signals and scalograms.

3.4. Surface roughness prediction

Table 2
ML model hyperparameters and search ranges; SGD: stochastic gradient descent,
To predict surface roughness, each CGAN is trained to its equilibrium
PReLU: parametric rectified linear unit.
and its generator is used to generate k images for each specimen, 70 % of
Hyperparameter Range which are used to train a CNN for surface roughness prediction, and the
Optimizer {Adam, RMSprop, SGD} remaining 30 % are held out to validate the model after training. (The
Initial learning rate {1 × 10− 6, 1 × 10− 5, …, 1 × 10− 2} effects of varying k from 0 to 500 images/specimen are shown for the
Activation function {Leaky ReLU, PReLU}
CNN and discussed in Section 4.) A CNN is chosen as the surface
Batch size {20, 22, …, 26}
Dropout probability {0 %, 20 %, …, 80 %} roughness prediction model due to its ability to extract features from the
z length {200, 400, …, 1000} scalograms using sequential convolutions and fuse the features from
Number of layers {2, 3, …, 6} low-level to high-level to yield a surface roughness estimate Râ [49].
Kernel size (CNN) {3, 5, …, 11}
Kernels/layer (CNN) {25, 26, …, 29}
Surface roughness prediction error is quantified as the mean absolute
Layer size (FC) {210, 211, …, 214} percentage error (MAPE), which is iteratively minimized for a training
dataset using gradient descent and assessed after training on a validation
dataset that is held out of the training data [50].
and discriminator are trained, also using backpropagation [48]. The
generator and the discriminator are constrained to the same architecture 4. Results and discussion
category during the search.
4.1. CGAN-synthesized power signal

Exemplary data synthesis results associated with each CGAN are

665
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

shown in Fig. 6. While the synthesized scalograms from both the FC-
CGAN and the 1-D CNN CGAN exhibit a slender spike pattern similar
to the ground truth images, they are much more defined for the FC
model than the 1-D CNN with substantially less noise contamination.
Notably, while the FC model retains qualitative similarity, e.g., spike
shape and color, the spikes are shown to translate left and right along the
time axis as compared to the ground truth, indicating that the GAN is
synthesizing transient events in the power signal as seen in Fig. 5, but at
variable times. Meanwhile, the 2-D CNN failed to converge for any
hyperparameter combination as described by the search ranges
(Table 2). Oscillatory behavior in the 2-D CNN generator and discrimi­
nator loss functions was observed for learning rates between 0.000001
and 0.001. As a result, no meaningful patterns were recognized from the Fig. 7. Frequency spectra correlation with ground truth for different CGAN
generated scalograms. architectures.
The performance of the CGANs is quantitatively evaluated first using
frequency spectra correlation between the synthesized power signal In addition to the established consistency between the synthesized
(obtained using inverse wavelet transform of scalogram from each data and the ground truth in terms of frequency patterns, the capability
CGAN model) and the ground truth, measured signal for each specimen, of the developed CGAN in learning parameter-specific distributions
as depicted in Fig. 7. For the FC CGAN, the average frequency spectra Ωn (un ), n = 1, 2, …21 is also evaluated using the UMAP embedding.
correlation is greater than 0.9 across all specimens, indicating that the
frequencies present in the synthetic signals closely resemble that of the
ground truth. The results from 1-D CNN CGAN yield correlations in the
range of 0.6–0.8. This is consistent with the results shown in Fig. 5 that
the scalogram appearing in the synthetic 1-D CNN images still captures
the frequency patterns associated with different process parameter
combinations but is obscured by noise. Since the 2-D CNN images do not
converge to any frequency pattern, their spectral correlation coefficients
are near zero as expected.
The ineffectiveness of the 2-D CNN can be considered as caused by
the poor performance of the convolutional generators due in part to the
sparse connections within the network. Whereas the FC generator con­
siders every pixel when passing the input through the network layers,
the convolutional models only consider local regions of 3–11 pixels at
once. Thus, the FC network can guide training for the entire image
structure at once whereas the CNNs analyze only local patches. Addi­
tionally, the number of learnable parameters is higher for the FC model
than the 1-D and 2-D CNNs (178 M vs. 1.1 M and 2.8 M, respectively), Fig. 8. UMAP embeddings of FC CGAN images (100/specimen) showing clus­
indicating stronger learning potential. Further investigation of the tering of intra-class samples and separation of inter-class ones; UMAP axes are
learning capability of different CGANs is considered for future studies. dimensionless.

Fig. 6. Synthesized scalograms from different CGAN architectures.

666
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

Fig. 8 shows the distribution of 2100 synthesized images corresponding Table 4

to 21 different process combinations in the UMAP embedding space. Optimal network structure of 2D CNN.
Each sample is color-coded according to the class u at the generator’s Input Size 128 × 128
input. It is seen that intra-class clustering and inter-class separation have
Kernel #1 11 × 11×64
been achieved, confirming a well-trained generator network that is able Kernel #2 7 × 7×128
to generate images unique to each process parameter combination. The Kernel #3 5 × 5×256
structure of the best-performing FC CGAN is shown in Table 3. The Kernel #4 and #5 3 × 3×512
optimizer during training is Adam with a learning rate of 0.01. The FC #1 4608
FC #2 1024
activation function is Leaky ReLU. FC #3 64
Output 1
4.2. Surface roughness prediction results Batch Size 64

The optimized CNN structure for surface roughness prediction is

tabulated in Table 4, trained with Adam optimizer (learning rate 0.01)
and Leaky ReLU activation function. The prediction error (in MAPE) is
shown in Fig. 9 as a function of the size of the synthetic data. During this
experiment, the training dataset size was the only variable while the
CNN architecture and hyperparameters were held constant. With all
other variables fixed, a larger dataset size produced a lower prediction
error because more diverse synthetic training data enabled the CNN to
fully optimize the network weights and learn the unique patterns from
the scalograms. It is seen that the surface roughness prediction error is
reduced from 58 %, when only the ground truth data but no synthetic
data was used to train the CNN, to 9.1 % when 250 or more synthetic
samples are included for CNN training.
In addition to the synthetic data size, the effect of the ground truth
data size is also evaluated on power signal synthesis and surface
roughness prediction. Specifically, the CGAN is retrained using pro­
gressively smaller ground truth datasets, ranging from 6 images/spec­ Fig. 9. Roughness prediction error vs. synthetic data size; insets depict
imen to 2 images/specimen, once again holding constant the CGAN measured vs. CNN-predicted Ra where dashed line is ideal.
architecture and hyperparameters. The resulting MAPE of the roughness
prediction are shown in Fig. 10. It is seen that increasing the number of considered as one of the authors’ research topics in the future.
ground truth samples per specimen significantly improves CGAN image
sharpness as well as the accuracy of the surface roughness prediction. 5. Conclusion
For instance, images synthesized using 3 ground truth samples for CGAN
training are significantly blurrier and more amorphous than the scenario In an effort to establish a direct mapping between power signals and
with 6 ground truth training samples per specimen, which produces surface roughness in end milling that enables in-process roughness
sharper images with more detail in the higher frequency bands. Addi­ quantification and eliminates the bottleneck of post-process inspection,
tionally, MAPE is reduced by 56 % when the ground truth dataset is a data-driven approach based on CGAN and CNN has been developed.
increased from 3 samples/specimen to 6 samples/specimen, indicating The CGAN enables synthesis of high-fidelity power signals under vary­
that the impact of ground truth dataset size is significant in determining ing process parameter combinations to circumvent the limitation of
CGAN and CNN behavior in the developed surface roughness prediction power signal data availability and augment the training dataset. The
method. CNN then takes the augmented dataset and learns the surface roughness
The surface roughness prediction method developed in this work is by way of the power signals’ time-frequency characteristics as extracted
compared to representative work by other researchers on surface via wavelet transform. An experiment has been carried out on milling
roughness prediction, and the result is summarized in Table 5. With the H13 tool steel to evaluate the developed methods.
exception of [21], this study’s technique exhibits a lower prediction Examining both the physical and statistical patterns of the synthe­
error than other comparable studies. However, the CGAN + CNN sized power data using frequency spectra and UMAP, respectively, it is
approach relies on only a single process signature as input rather than confirmed that CGAN is to learn unique data distributions specific to
the four upon which [21] relies, making the former approach more each process parameter combination. The CNN trained on the
feasible than the latter in a production setting. Additionally, the BGWO augmented dataset has achieved a 9.1 % mean error in predicting sur­
model requires extraction of 73 statistical features from the four input face roughness, confirming the power process signature as a viable
signatures, which is computationally more complex than the CGAN + candidate for in-process surface characterization in machining. It is
CNN approach as presented in this study. Nonetheless, low prediction further observed that the prediction error is asymptotic as a function of
error of [21] indicates the effect of multi-sensor fusion, which will be the synthetic dataset size, with a larger ground truth dataset resulting in
smaller prediction error.
Table 3 Industrial application of our method is envisioned to follow a two-
Optimal network structure of FC CGAN. step process: 1) model recalibration for the process and machine of in­
Generator Discriminator
terest, and 2) provisioning of real-world, in-situ production data to
refine our developed model. Regarding point 1, the CGAN and CNN from
Input z: 1000, u: 200 Scalogram: 16,384, u: 200
our method should be recalibrated to the application-specific power
Layer #1 2048 (211) 2048 (211)
Layer #2 4096 (212) 1024 (210) signals and process parameters to avoid extrapolating beyond the range
Layer #3 8192 (213) 512 (29) of the training data, which may yield spurious results. The model
Output 16,384 (1282) 1 recalibration process to avoid these issues would involve collecting a
Dropout % 60 % 40 %
small number of power signals and surface roughness measurements,
Batch Size 64

667
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

Fig. 10. Surface roughness prediction error under reducing CGAN training dataset sizes.

References
Table 5
Comparison of presented study to literature; BP ANN: backpropagation artificial [1] Toloei A, Stoilov V, Northwood D. The relationship between surface roughness and
neural network, BLR: Bayesian linear regression, RNN: recurrent neural corrosion. In: Volume 2B: Advanced Manufacturing, San Diego, California, USA:
network, BGWO: binary grey wolf optimization. American Society of Mechanical Engineers; Nov. 2013, p. V02BT02A054.
〈[Link]
Ref. Process Signature Input Ra Prediction Ra Prediction [2] Gu X, Cegla F. The effect of internal pipe wall roughness on the accuracy of clamp-
Model Error on ultrasonic flowmeters. IEEE Trans Instrum Meas 2019;68(1):65–72. [Link]
org/10.1109/TIM.2018.2834118.
Ours Power profile CGAN + CNN 9%
[3] Zhao B, Song J, Xie L, Hu Z, Chen J. Surface roughness effect on fatigue strength of
[18] Spindle, fixture vibrations BP ANN 18 %
aluminum alloy using revised stress field intensity approach. Sci Rep 2021;11(1):
[19] Spindle, fixture, workpiece BLR 10 % 19279. [Link]
vibrations [4] Quick Guide to Surface Roughness Measurement Mitutoyo; Dec. 2016.
[20] Surface image, tool image, power CNN + RNN 12 % [5] Ghodrati S, Kandi SG, Mohseni M. Nondestructive, fast, and cost-effective image
profile processing method for roughness measurement of randomly rough metallic
[21] Acoustic, spindle vibration, fixture BGWO 1% surfaces. J Opt Soc Am A 2018;35(6):998. [Link]
forces, power profile JOSAA.35.000998.
[6] Chen C-H, Jeng S-Y, Lin C-J. Prediction and analysis of the surface roughness in
CNC end milling using neural networks. Appl Sci 2021;12(1):393. [Link]
such as the 126 observed in our study, and retraining the CGAN and CNN 10.3390/app12010393.
[7] Yeganefar A, Niknam SA, Asadi R. The use of support vector machine, neural
using the procedure described in Section 3.3. Regarding point 2, the network, and regression analysis to predict and optimize surface roughness and
post-calibration CNN from point 1 will be used for surface roughness cutting forces in milling. Int J Adv Manuf Technol 2019;105(1–4):951–65. https://
prediction during production by providing it with time-frequency im­ [Link]/10.1007/s00170-019-04227-7.
[8] Sekulic M, Pejic V, Brezocnik M, Gostimirovic M, Hadzistevic M. Prediction of
ages of in-situ power signals acquired by the end user. This imple­ surface roughness in the ball-end milling process using response surface
mentation scheme represents a low barrier to entry for industrial use of methodology, genetic algorithms, and grey wolf optimizer algorithm. Adv Prod
the developed techniques as presented in this study. Eng Manag 2018;13(1):18–30. [Link]
[9] Groover MP. Fundamentals of Modern Manufacturing: Materials, Processes, and
Future research will explore the potential of generative models in Systems. 4th ed. Hoboken, NJ: J. Wiley & Sons; 2010.
synthesizing additional process signatures. Additionally, tool wear will [10] Sealy MP, Liu ZY, Guo YB, Liu ZQ. Energy based process signature for surface
be included in the CGAN input to more comprehensively account for integrity in hard milling. J Mater Process Technol 2016;238:284–9. [Link]
org/10.1016/[Link].2016.07.038.
process variations in surface characterization. The effects of changing [11] Nasir V, Sassani F. A review on deep learning in machining and tool monitoring:
tool material and geometry on the scalogram synthesis and surface methods, opportunities, and challenges. Int J Adv Manuf Technol 2021;115(9–10):
roughness prediction will also be studied in order to make the developed 2683–709. [Link]
[12] Pimenov DY, Bustillo A, Wojciechowski S, Sharma VS, Gupta MK, Kuntoğlu M.
model more generalizable across tool types. Finally, physical factors of
Artificial intelligence systems for tool condition monitoring in machining: analysis
the machining process such as strong plastic deformation and chatter and critical review. J Intell Manuf 2022. [Link]
will be detected in the power signals and accounted for by future surface 01923-2.
roughness prediction models to further reduce the prediction error. [13] Bhuiyan MSH, Choudhury IA, Dahari M. Monitoring the tool wear, surface
roughness and chip formation occurrences using multiple sensors in turning.
J Manuf Syst 2014;33(4):476–87. [Link]
Acknowledgments [14] Process Signature, Collaborative Research Center 136. German Research
Foundation. [Online]. Available: 〈[Link]
us/glossary/p/process-signature〉 [Accessed: 19 September 2022].
This work is supported by the National Science Foundation under [15] Process Quantities, Collaborative Research Center 136. German Research
grant CMMI-2040288/2040358. Clayton Cooper acknowledges support Foundation. Accessed: Sep. 19, 2022. [Online]. Available: 〈[Link]
from the National Science Foundation Graduate Research Fellowship [Link]/en/about-us/glossary/p/process-quantities〉 [Accessed: 19
September 2022].
under Grant No. 1937968. Robert Gao and Jianjing Zhang acknowledge [16] Ragai I, Abdalla AS, Abdeltawab H, Qian F, Ma J. Toward smart manufacturing:
support from the NSF Engineering Research Center for Hybrid Autono­ Analysis and classification of cutting parameters and energy consumption patterns
mous Manufacturing: Moving from Evolution to Revolution (ERC- in turning processes. S027861252200067X J Manuf Syst 2022. [Link]
10.1016/[Link].2022.04.016.
HAMMER) under award EEC-2133630.

668
C. Cooper et al. Journal of Manufacturing Systems 68 (2023) 660–669

[17] Griffin JM, et al. Control of deviations and prediction of surface roughness from [34] Farnia F, Ozdaglar A. Do GANs always have Nash equilibria? In: Proceedings of the
micro machining of THz waveguides using acoustic emission signals. Mech Syst 37th international conference on machine learning, in Proceedings of Machine
Signal Process 2017;85:1020–34. [Link] Learning Research, 119. PMLR; 2020, pp. 3029–3039.
[18] Wu TY, Lei KW. Prediction of surface roughness in milling process using vibration [35] Ntavelis E, Kastanis I, Van Gool L, Timofte R. Same same but different:
signal analysis and artificial neural network. Int J Adv Manuf Technol 2019;102 augmentation of tiny industrial datasets using generative adversarial networks. In:
(1–4):305–14. [Link] Proceedings of the 2020 7th Swiss conference on data science (SDS), Luzern,
[19] Kong D, Zhu J, Duan C, Lu L, Chen D. Bayesian linear regression for surface Switzerland: IEEE; Jun. 2020, pp. 17–22. 〈[Link]
roughness prediction. Mech Syst Signal Process 2020;142:106770. [Link] 0.00011〉.
10.1016/[Link].2020.106770. [36] Hertlein N, Buskohl PR, Gillman A, Vemaganti K, Anand S. Generative adversarial
[20] Wang P, Liu Z, Gao RX, Guo Y. Heterogeneous data-driven hybrid machine learning network for early-stage design flexibility in topology optimization for additive
for tool condition prognosis. CIRP Ann 2019;68(1):455–8. [Link] manufacturing. J Manuf Syst 2021;59:675–85. [Link]
10.1016/[Link].2019.03.007. jmsy.2021.04.007.
[21] Tian W, et al. Broad learning system based on binary grey wolf optimization for [37] Schleich B, Qie Y, Wartzack S, Anwer N. Generative adversarial networks for
surface roughness prediction in slot milling. IEEE Trans Instrum Meas 2022;71: tolerance analysis. CIRP Ann 2022;71(1):133–6. [Link]
1–10. [Link] cirp.2022.03.021.
[22] Moliner-Heredia R, Peñarrocha-Alós I, Abellán-Nebot JV. Model-based tool [38] Cooper C, Zhang J, Gao RX, Wang P, Ragai I. Anomaly detection in milling tools
condition prognosis using power consumption and scarce surface roughness using acoustic signals and generative adversarial networks. Procedia Manuf 2020:
measurements. J Manuf Syst 2021;61:311–25. [Link] 372–8. [Link]
jmsy.2021.09.001. [39] Barua S, Erfani SM, Bailey J. FCC-GAN: A fully connected and convolutional net
[23] Wang J, Li Y, Zhao R, Gao RX. Physics guided neural network for machining tool architecture for GANs. arXiv, May 27; 2019. [Online]. Available: 〈[Link]
wear prediction. J Manuf Syst 2020;57:298–310. [Link] org/abs/1905.02417〉 [Accessed: 14 October 2022].
jmsy.2020.09.005. [40] Qu F, Liu J, Ma Y, Zang D, Fu M. A novel wind turbine data imputation method
[24] Corne R, Nath C, El Mansori M, Kurfess T. Study of spindle power data with neural with multiple optimizations based on GANs. Mech Syst Signal Process 2020;139:
network for predicting real-time tool wear/breakage during inconel drilling. 106610. [Link]
J Manuf Syst 2017;43:287–95. [Link] [41] Liu J, Qu F, Hong X, Zhang H. A small-sample wind turbine fault detection method
[25] Wang X, Williams RE, Sealy MP, Rao PK, Guo Y. Stochastic modeling and analysis with synthetic fault data using generative adversarial nets. IEEE Trans Ind Inf
of spindle power during hard milling with a focus on tool wear. J Manuf Sci Eng 2019;15(7):3877–88. [Link]
2018;140(11):111011. [Link] [42] Mirza M, Osindero S. Conditional Generative Adversarial Nets. arXiv; Nov. 06,
[26] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep 2014. [Online]. Available: 〈[Link] [Accessed: 02
convolutional neural networks. Commun ACM 2017;60(6):84–90. [Link] September 2022].
10.1145/3065386. [43] Liu ZY, Guo YB, Sealy MP, Liu ZQ. Energy consumption and process sustainability
[27] Jiang X, Scott P, Whitehouse D. Wavelets and their applications for surface of hard milling with tool wear progression. J Mater Process Technol 2016;229:
metrology. CIRP Ann 2008;57(1):555–8. [Link] 305–12. [Link]
cirp.2008.03.110. [44] Gao RX, Yan R. Wavelets: theory and applications for manufacturing. Boston, MA:
[28] Priyadarshini A, Pal SK, Samantaray AK. Finite element modeling of chip formation Springer US; 2011. 〈[Link]
in orthogonal machining. In: Statistical and Computational Techniques in [45] Yan R, Gao RX. Base wavelet selection for bearing vibration signal analysis. Int J
Manufacturing, J. P. Davim, Ed., Berlin, Heidelberg: Springer Berlin Heidelberg; Wavel Multiresolut Inf Process 2009;07(04):411–26. [Link]
2012, pp. 101–144. 〈[Link] S0219691309002994.
[29] Liu Z, Guo Y. A hybrid approach to integrate machine learning and process [46] Everitt B, Skrondal A. The Cambridge dictionary of statistics; 2010. [Online].
mechanics for the prediction of specific cutting energy. CIRP Ann 2018;67(1): Available: 〈[Link] [Accessed: 08
57–60. [Link] December 2021].
[30] Bhokse V, Chinchanikar S, Anerao P, Kulkarni A. Experimental investigations on [47] McInnes L, Healy J, Melville J. UMAP: Uniform Manifold Approximation and
chip formation and plowing cutting forces during hard turning. Mater Today: Proc Projection for Dimension Reduction.” Sep. 17; 2020. [Online]. Available: 〈htt
2015;2(4–5):3268–76. [Link] p://[Link]/abs/1802.03426〉 [Accessed: 11 August 2021].
[31] Oberg E, Jones FD, Horton HL, Ryffel H.H., McCauley CJ, Brengelman L. [48] Rojas R. The backpropagation algorithm. In: Neural Networks, Berlin, Heidelberg:
Machinery’s handbook, 31st ed. South Norwalk: Industrial Press, Inc; 2020. Springer Berlin Heidelberg; 1996, pp. 149–182. 〈[Link]
[32] Lu F, Zhou G, Liu Y, Zhang C. Ensemble transfer learning for cutting energy 642-61068-4_7〉.
consumption prediction of aviation parts towards green manufacturing. J Clean [49] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436–44.
Prod 2022;331:129920. [Link] [Link]
[33] Goodfellow IJ et al., Generative adversarial networks. arXiv; Jun. 10, 2014. [50] Malakooti B. Operations and production systems with multiple objectives.
[Online]. Available: 〈[Link] [Accessed: 06 September Hoboken, New Jersey: John Wiley & Sons Inc; 2014.
2022].

669