See discussions, stats, and author profiles for this publication at: [Link]

net/publication/352657114

GANs Applications

Technical Report · June 2021

DOI: 10.13140/RG.2.2.31108.86405

CITATIONS READS
0 56

2 authors:

Sheikh Badar ud din Tahir Syeda Amna Rizwan

Air University of Islamabad Air University of Islamabad
11 PUBLICATIONS 85 CITATIONS 6 PUBLICATIONS 31 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Automatic Multi-Facial Landmarks Tracking and Synthetic Mask Prediction for Expressions Recognition and Age Estimation Models View project

Intelligent Media-Wearable Smart Home Activities (IM-WSHA) Dataset View project

All content following this page was uploaded by Sheikh Badar ud din Tahir on 22 June 2021.

The user has requested enhancement of the downloaded file.

Contents
Introduction .................................................................................................................................................. 2
GAN and cGAN........................................................................................................................................... 2
Related Issues in GAN Model over cGAN ................................................................................................ 3
Training a Conditional GAN.......................................................................................................................... 3
I. Generator Network .......................................................................................................................... 4
II. Discriminator Network ..................................................................................................................... 5
III. Gradients and Loss Functions ...................................................................................................... 6
IV. Training Options ........................................................................................................................... 6
V. Model Training ................................................................................................................................. 6
References .................................................................................................................................................... 7
 How to Train your Conditional GAN
Introduction
In recent times, generative adversarial networks have gained a lot of attention and been
introduced as an alternate framework for training generative-based models to avoid the issues of
approximating intractable stochastic computations. Typically, Generative Adversarial Networks
(GAN) are a type of deep learning generative model (CNN) that can generate image data with
similar features to the input training data (Xu et al., 2019). GAN models are capable of generating
new stochastic instances for any given dataset.

GANs rely on two components: a generator that learns to generate new data and a
discriminator that learns how to discriminate synthetic images from an original image. In other
words, the generator attempts to fool the discriminator, while the discriminator attempts to avoid
being tricked. While in cGANs, conditional-based rule is implemented, which implies that both
the discriminator and generator are constrained by source information (for example, labels or class
data) from other streams. Consequently, the ideal model which was generated with the inclusion
of diverse contextual information, is capable of learning a multi-modal mapping from inputs to
output. The clear extension from unconditional GANs to cGAN can be achieved by thresholding
both the discriminator and the generator on the signal. However, a simple extension will compel
the models to ignore it (Isola et al., 2017; Pathak et al., 2016). This technique has been broadly
utilized for various tasks such as image to image translation, super-resolution (SR), video
generation, and convolutional face generation (Wang et al., 2018; Maeda et al., 2020; Bulat et al.,
2018).

GAN and cGAN

The fact that GANs are effective in image synthesis is based on the evidence that they are capable
of generating new examples of images for a target class. Few datasets include additional
information, such as a class label. Therefore, it is preferable to make use of this extra information
when possible.
However, there are two reasons why class label information should be used in a GAN model.
Firstly, to improve the GAN and target image generation. Secondly, the additional information
like classes or labels associated with the input images could be used to improve the GAN model.
However, this gain may distinct itself in the form of more consistent training with better training
or improved quality of generated images (Mirza et al., 2014).

Related Issues in GAN Model over cGAN

A constraint of a GAN model is that it generate a random image from the whole region.
Also, the points in the latent space are associated to the randomly generated images, but this
correlation is complex to analyze. However, in cGAN allows the inclusion of an additional input
layer of hot-encoded image labels. This layer assists the generator on which image to generate and
inputs a feature vector generated either from an image class encoding or a set of specific image
features.

Random noise and

Images are transmuted
conditional data are
using convolution into a
fused to generate a
dense code
dense code of output
image
Generator

y
y Discriminator

The dense code is unsampled

using deconvolution into
image space

Figure1. Architecture of Conditional GAN (Source: (Mirza et al., 2014))

Training a Conditional GAN

GAN can be expanded to a conditional model only if both the models including generator and
discriminator are conditioned on auxiliary information y such as class labels from other mode. We
can conduct the conditioning by including y as an additional layer in both discriminator and
generator. In order to train a conditional GAN, both networks simultaneously optimize the
performance. Initially, GAN trains the generator to generate images. Secondly, a trained
discriminator or model separates the real and generated image data.

To improve the effectiveness of the generator, maximize the loss of the discriminator
when disposed the generated labeled data. Thus, the goal of the generator is to generate labeled
data that is classified as real by the discriminator. In addition, the ideal implementation of these
techniques results in a generator that provides impressively reliable data. Also, it correlates to the
input labels and a discriminator that has learned accurate and robust features extraction for each
label from the training data. Following are the steps of training a conditional GAN which is listed
below.

• Download and extract dataset to the desired location. Then, verify the number of classes
in the dataset.
• Then, assume the 5 number of classes in the dataset.
• Fuse the data using random horizontal flipping and resize the all the images equally (for
example, 64-by-64). Then, we need to define the two input networks, which generate the
images from the random size of vectors (such as 100) and their categorical labels.

I. Generator Network
Generator Network converts the labels to encoded vectors and resizes them into an array (for
example 4 by 4). Then, the resultant images are joined together from the two inputs with channel
dimension. The output of a vector size 100 is generated as a 4 by 4 array. The resulting arrays are
scaled to 64-by-64 by three arrays via a set of transposed convolution layers including ReLU layers
and batch normalization. In addition, in order to resize the random noise input, we need to use the
custom layer that scales the input noise via a fully correlated operation and resize the output to a
particular size. Finally, in order to input the labels to the generator network, utilize the feature
input layer object and specify one feature. Then, embed and reshape layers object transform a
categorical label data to a one-channel image data of the particular size by embedding and a
completely connected operation.
 Labels

Noise

Defining the architecture of Generator Network (Source: ([Link], 2021))

II. Discriminator Network

The next phase is to define the two-input network, which classifies real and generated image data
(for example, 64 by 64) and the relevant labels. In order to create a network, convolutional layers,
ReLU layers, and batch normalization are required to build a network that takes as an input a 64-
by-64 by 1 image and the relevant labels generate a scaler prediction score. The dropout is used to
introduce random noise to the input images.

Defining the architecture of Discriminator Network(Source:([Link], 2021))

 III. Gradients and Loss Functions
Create a gradient function which takes two data inputs of mini batch and array values, and returns
the gradients of the loss function w.r.t the learnable parameters (weights and biases) in the
networks including and an array of images.

IV. Training Options

Training can only be done with a mini-batch sizes (for example, 128 for 500 epochs). However, If
the discriminator learns too quickly to distinguish between the original and generated images, the
generator may not be able to train properly and may perform poorly. In order to properly balance
the generator’s and discriminator’s learning, randomly flip (flip factor 0.5) the labels of a
percentage of the original images.

V. Model Training
Firstly, create a custom training loop for model utilization. Then, iteratively loop through the
training data, and update the parameters with each iteration. In order to track the training progress,
a batch of images is shown via held-out random array values (for example, 25) to be input into the
network scores and generator.

Finally, to training the cGAN for each epoch we need to change the data, and loop over the
mini-batches of image data. However, for each mini-batch, determine the gradients of the model
using modelGradient functions. Then adjust the network parameters using the Adam optimization
(Kingma et al., 2014) function and plot the scores of the two input networks. In this way, the
discriminator will learn an optimal extracted features that assists in the detection of the real images
among the generated images. Additionally, the generator has also learned a similar optimal
composition of features that allows it to generate similar images to the training data. Where each
column correlates to a single class.
 References

[1] Xu, L., Skoularidou, M., Cuesta-Infante, A., & Veeramachaneni, K. (2019). Modeling
tabular data using conditional gan. arXiv preprint arXiv:1907.00503.
[2] Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with
conditional networks. In Proceedings of the IEEE conference on computer vision and
pattern recognition (pp. 1125-1134).
[3] Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016). Context
encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on
computer vision and pattern recognition (pp. 2536-2544).
[4] Wang, T. C., Liu, M. Y., Zhu, J. Y., Tao, A., Kautz, J., & Catanzaro, B. (2018). High-
resolution image synthesis and semantic manipulation with conditional gans.
In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.
8798-8807).
[5] Maeda, S. (2020). Unpaired image super-resolution using pseudo-supervision.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition (pp. 291-300).
[6] Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint
arXiv:1411.1784.
[7] (2021). Retrieved from [Link]:
[Link]
[Link]
[8] Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6980.

View publication stats