AM3602 Gen AI UNIT IV

602-Generative AI

UNIT IV
APPLICATIONS OF GENERATIVE AI
Syllabus: Image generation and manipulation – Text generation and natural language
processing – Anomaly detection and data augmentation – Style transfer and artistic
applications – Real-world use cases (Art & Design, Medical Imaging, Content creation,
Chatbots, Virtual Assistants, Cybersecurity, etc.) and industry examples-Advanced
applications of CNNs in Image Processing-Applications of RNNs and LSTMs in Sequential
Data- Guest Lectures by Industry Experts, and Researchers
Image Generation and Manipulation
Image generation and manipulation involve using generative models like Generative
Adversarial Networks (GANs), Variational Autoencoders (VAEs), and diffusion models to
create or edit visual content. GANs, for instance, are widely used in high-resolution image
synthesis, style transfer, and editing tasks. Applications include creating photorealistic
images, modifying facial features, or generating artistic variations of existing artwork.
Techniques like Deep Dream and Neural Style Transfer enable blending artistic styles into
photographs. In practical scenarios, tools such as DALL·E, Midjourney, and RunwayML
demonstrate the capability of AI in generating unique and customized visual content.
Manipulation tasks also extend to object removal, inpainting (filling missing parts of an
image), and image super- resolution, which have applications in fields such as design,
forensics, and entertainment.
Image generative AI models are a subset of generative models that aim to create realistic and
coherent images from scratch. These models use complex algorithms and deep learning
techniques to learn patterns and features from a vast amount of training data. They can be
broadly classified into two categories: Variational Autoencoders (VAEs) and Generative
Adversarial Networks (GANs).
Variational Autoencoders (VAEs): VAEs are probabilistic models that encode images into a
latent space, where they are represented as vectors. The decoder then reconstructs the images
from the encoded vectors, enabling the model to generate new images by sampling from the
latent space.
Generative Adversarial Networks (GANs): GANs consist of two neural networks, a
generator, and a discriminator, engaged in a competitive process. The generator creates
synthetic images to fool the discriminator, which, in turn, aims to distinguish between real
and fake images. This back-and-forth battle results in the generation of highly realistic
images.
2. Text-to-Image Synthesis
Text-to-image synthesis, a fascinating application of image generative AI models, enables the
conversion of textual descriptions into corresponding visual representations. This process
involves combining natural language processing (NLP) techniques with image generative
models to achieve impressive results.
Conditional GANs for Text-to-Image: Researchers have successfully integrated text
embeddings with GANs, allowing for the generation of images conditioned on specific
textual input. This means that by providing a detailed textual description, the model can
generate highly specific and accurate images.
1
AM3602 Gen AI UNIT IV
602-Generative AI

Applications of Text-to-Image Synthesis: Text-to-image technology finds applications in

diverse fields, such as e-commerce, content creation, and virtual reality. For instance, in e-
commerce, this technology can generate product images from textual descriptions, aiding in
faster product design and development.
3. State-of-the-Art Text-to-Image Models
Over the years, several state-of-the-art models have emerged, showcasing the impressive
capabilities of text-to-image synthesis:
DALL-E: Developed by OpenAI, DALL-E is a prominent text-to-image model capable of
creating unique and creative images based on textual prompts. It has the ability to generate
art, objects, and even fictional creatures with remarkable precision.
CLIP: Another innovation by OpenAI, CLIP enables the model to understand and generate
images based on natural language descriptions. It combines vision and language pre-training
to achieve impressive cross-modal capabilities.
CM3leon: One of the latest and most promising additions to the world of image generative
AI models is Meta CM3leon. This groundbreaking multimodal model, introduced by Meta
AI, has garnered significant attention for its unparalleled text-to-image generation
capabilities, coupled with unmatched compute efficiency.
TEXT GENERATION AND NATURAL LANGUAGE PROCESSING
Text generation and natural language processing (NLP) leverage advanced language models
like GPT (Generative Pre-trained Transformer) and BERT to create coherent and
contextually relevant text. These models can generate essays, write poetry, summarize
documents, or simulate human-like conversations. Text generation finds applications in
creative writing, chatbots, automated journalism, and code generation. Furthermore, NLP
extends to tasks such as sentiment analysis, translation, and question-answering systems.
Real-world tools like ChatGPT and Google Translate are prime examples of these
capabilities. Recent advancements in large language models (LLMs) have further enabled
complex tasks like summarizing legal documents, writing marketing copy, and generating
code snippets in programming.
Text generation is the process of automatically producing coherent and meaningful text,
which can be in the form of sentences, paragraphs or even entire documents. It involves
various techniques, which can be found under the field such as natural language processing
(NLP), machine learning and deep learning algorithms, to analyze input data and generate
human-like text. The goal is to create text that is not only grammatically correct but also
contextually appropriate and engaging for the intended audience.
Natural language generation (NLG) and natural language understanding (NLU) are 2
essential components of a robust natural language processing (NLP) system, but they serve
different purposes.
Natural language understanding (NLU) is the ability of a machine to comprehend, interpret
and extract meaningful information from human language in a valuable way. It involves tasks
like sentiment analysis, named entity recognition, part-of-speech tagging and parsing. NLU
helps machines understand the context, intent and semantic meaning of human language
inputs.

2
AM3602 Gen AI UNIT IV
602-Generative AI

Natural language generation (NLG) is the ability of a machine to produce human-like text or
speech that is clear, concise and engaging. It involves tasks like text summarization,
storytelling, dialogue systems and speech synthesis. NLG helps machines generate
meaningful and coherent responses in a way that is easily understood by humans.
NLU focuses on understanding human language, while NLG focuses on generating human-
like language. Both are crucial for building advanced NLP applications that can effectively
communicate with humans in a natural and meaningful way.
Text generation techniques
Statistical models: These models typically use a large dataset of text to learn the patterns and
structures of human language, and then use this knowledge to generate new text. Statistical
models can be effective at generating text that is similar to the training data, but they can
struggle to generate text that is both creative and diverse. N-gram models and conditional
random fields (CRF) are popular statistical models.
N-gram models: These are a type of statistical model that uses the n-gram language model,
which predicts the probability of a sequence of "n-items" in a given context.
Conditional random fields (CRFs): These are a type of statistical model that uses a
probabilistic graphical model to model the dependencies between words in a sentence. CRFs
can be effective at generating text that is both coherent and contextually appropriate, but this
type of text generation model can be computationally expensive to train and might not
perform well on tasks that require a high degree of creative language generation.
Neural networks: These are machine learning algorithms that use artificial neural networks
to identify data patterns. Through APIs, developers can tap into pretrained models for
creative and diverse text generation, closely mirroring the training data's complexity. The
quality of the generated text heavily relies on the training data. However, these networks
demand significant computational resources and extensive data for optimal performance.
Recurrent neural networks (RNNs): These are a foundational type of neural network
optimized for processing sequential data, such as word sequences in sentences or paragraphs.
They excel in tasks that require understanding sequences, making them useful in the early
stages of developing large language models(LLMs). However, RNNs face challenges with
long-term dependencies across extended texts, a limitation stemming from their sequential
processing nature. As information progresses through the network, early input influence
diminishes, leading to the "vanishing gradient" problem during backpropagation, where
updates shrink and hinder the model's ability to maintain long-sequence connections.
Incorporating techniques from reinforcement learning can offer strategies to mitigate these
issues, providing alternative learning paradigms to strengthen sequence memory and
decision- making processes in these networks.
Long short-term memory networks (LSTMs): This is a type of neural network that uses a
memory cell to store and access information over long periods of time. LSTMs can be
effective at handling long-term dependencies, such as the relationships between sentences in
a document, and can generate text that is both coherent and contextually appropriate.

3
AM3602 Gen AI UNIT IV
602-Generative AI

Transformer-based models: These models are a type of neural network that uses self-
attention mechanisms to process sequential data. Transformer-based models can be effective
at generating text that is both creative and diverse, as they can learn complex patterns and
structures in the training data and generate new text that is similar to the training data. Unlike
historical approaches such as RNNs and LSTMs, transformer-based models have the distinct
advantage of processing data in parallel, rather than sequentially. This allows for more
efficient handling of long-term dependencies across large datasets, making these models
especially powerful for natural language processing applications such as machine translation
and text summarization.
Generative pretrained transformer (GPT): GPT is a transformer-based model that is trained
on a large dataset of text to generate human-like text. GPT can be effective at generating text
that is both creative and diverse, as it can learn complex patterns and structures in the training
data and generate new text that is similar to the training data.
Bidirectional encoder representations from transformers (BERT): BERT is a transformer-
based model that is trained on a large dataset of text to generate bidirectional representations
of words. That means it evaluates the context of words from both before and after a sentence.
This comprehensive context awareness allows BERT to achieve a nuanced understanding of
language nuances, resulting in highly accurate and coherent text generation. This
bidirectional approach is a key distinction that enhances BERT's performance in applications
requiring deep language comprehension, such as question answering and named entity
recognition (NER), by providing a fuller context compared to unidirectional models.
Thus, text generation techniques, especially those implemented in Python, have
revolutionized the way we approach generative AI in the English language and beyond.
Using trained models from platforms like Hugging Face, developers and data scientists can
access a plethora of open source tools and resources that facilitate the creation of
sophisticated text generation applications. Python, being at the forefront of AI and data
science, offers libraries that simplify interacting with these models, allowing for
customization through prefix or template adjustments, and the manipulation of text data for
various applications. Furthermore, the use of metrics and benchmarks to evaluate model
performance, along with advanced decoding strategies, ensures that the generated text meets
high standards of coherence and relevance.
Examples of text generation
Text generation is a versatile tool that has a wide range of applications in various domains.
Here are some examples of text generation applications:
 Blog posts and articles:
It can be used to automatically generate blog posts and articles for websites and blogs. These
systems can automatically generate unique and engaging content that is tailored to the
reader's interests and preferences.
 News articles and reports:
It can be used to automatically generate news articles and reports for newspapers, magazines
and other media outlets. These systems can automatically generate timely and accurate
content that is tailored to the reader's interests and preferences.

4
AM3602 Gen AI UNIT IV
602-Generative AI

 Social media posts:

It can be used to automatically generate social media posts for Facebook, Twitter and other
platforms. These systems can automatically generate engaging and informative content that is
tailored to the reader's interests and preferences.
 Product descriptions and reviews:
It can be used to automatically generate product descriptions and reviews for e-commerce
websites and online marketplaces. These systems can automatically generate detailed and
accurate content that is tailored to the reader's interests and preferences.
 Creative writing:
It can be used to automatically generate creative writing prompts for writers with powerful
AI models. These systems can automatically generate unique and inspiring ideas that are
tailored to the writer's interests and preferences.
 Language translation:
It can be used to automatically translate text between different languages. These systems can
automatically generate accurate and fluent translations that are tailored to the reader's
interests and preferences.
 Chatbot conversations:
It can be used to automatically generate chatbot conversations for customer service and
support. These systems can automatically generate personalized and engaging conversations
that are tailored to the reader's interests and preferences.
 Text summaries:
It condenses lengthy documents into concise versions, preserving key information through
advanced natural language processing and machine learning algorithms. This technology
enables quick comprehension of extensive content, ranging from news articles to academic
research, enhancing information accessibility and efficiency.
 Virtual assistant interactions:
Text generation can be used to automatically generate virtual assistant interactions for home
automation and personal assistance. These systems can automatically generate personalized
and convenient interactions that are tailored to the reader's interests and preferences.
 Storytelling and narrative generation:
Text generation can be used to automatically generate stories and narratives for entertainment
and educational purposes. These systems can automatically generate unique and engaging
stories that are tailored to the reader's interests and preferences.

5
AM3602 Gen AI UNIT IV
602-Generative AI

ANOMALY DETECTION AND DATA AUGMENTATION

Anomaly detection uses generative models to identify unusual patterns or outliers in data,
making it invaluable for industries like finance, cybersecurity, and healthcare. For instance,
autoencoders and VAEs can model normal data patterns and flag deviations as anomalies.
This is crucial for fraud detection, network intrusion monitoring, and detecting manufacturing
defects. Data augmentation, on the other hand, enhances the quality and quantity of datasets
by artificially generating variations. Techniques such as flipping, rotating, or cropping
images, and using GANs to create synthetic data, improve model robustness and
generalization. In NLP, data augmentation involves paraphrasing, synonym substitution, and
back-translation to diversify training datasets. These techniques are particularly critical in
scenarios with limited labeled data.
Data augmentation is a powerful technique used to enhance the performance of machine
learning models, particularly in scenarios where the available training data is limited. In the
context of anomaly detection, it plays a crucial role in improving the model's ability to
identify rare or unexpected events.
Common Data Augmentation Methods
Traditional data augmentation methods often involve simple transformations such as:
 Rotation: Rotating the image by a certain angle.
 Translation: Shifting the image horizontally or vertically.
 Scaling: Resizing the image.
 Flipping: Flipping the image horizontally or vertically.
 Cropping: Extracting a portion of the image.
While these methods can increase the diversity of the training data, they may sometimes
generate samples that are significantly different from the original image, potentially
misleading the model.
Advanced Data Augmentation Techniques
To address the limitations of traditional methods, more sophisticated techniques have been
developed:
 AugMix: This method combines the original image with multiple augmented
versions, creating a diverse set of synthetic images that preserve the original image's
features.
 Mask Pool: This technique involves masking out portions of the image, forcing the
model to focus on different regions and improving its ability to generalize to unseen
anomalies.
 Learnable Data Augmentation: This approach automatically learns the optimal
augmentation parameters for a specific dataset and task, leading to more effective and
efficient training.
Anomaly Detection Methods
Several approaches can be used for anomaly detection:

6
AM3602 Gen AI UNIT IV
602-Generative AI

 Embedding-based Methods: These methods utilize pre-trained models to extract

high-level features from images. Anomalies are identified as data points that are
significantly different from the normal data distribution in the feature space.
 Reconstruction-based Methods: These methods aim to reconstruct the input data
from a compressed representation. Anomalies are detected as data points that cannot
be accurately reconstructed.
 Local Outlier Factor (LOF): LOF is a density-based method that calculates the local
density of each data point based on its nearest neighbors. Data points with
significantly lower density than their neighbors are considered outliers.
Anomaly Detection in Industries
Anomaly detection has numerous applications across various industries, including:
 Manufacturing: Identifying defective products, predicting equipment failures, and
optimizing production processes.
 Healthcare: Detecting medical abnormalities in images, monitoring patient health, and
identifying potential outbreaks.
 Finance: Detecting fraudulent transactions, identifying market anomalies, and assessing
credit risk.
 Security: Detecting intrusions in computer networks, identifying suspicious activities,
and preventing cyberattacks.
By effectively leveraging data augmentation techniques and employing suitable anomaly
detection methods, organizations can significantly improve their ability to identify and
respond to unexpected events, leading to increased efficiency, safety, and productivity.
STYLE TRANSFER AND ARTISTIC APPLICATIONS
Style transfer refers to applying the artistic style of one image to another, often achieved
using convolutional neural networks (CNNs) and GANs. This technology powers creative
tools that allow artists and designers to reimagine their works in the styles of famous painters
or other visual aesthetics. Neural Style Transfer, for example, can blend Van Gogh’s or
Monet’s style with a photograph, creating visually captivating results. Artistic applications
extend to music composition, video editing, and even fashion design, enabling AI to act as a
creative collaborator. Industries such as gaming and entertainment utilize style transfer to
enhance the visual appeal of their products and create immersive experiences.
Neural Style Transfer
NST is a deep learning technique that allows us to merge the content of one image with the
artistic style of another, resulting in visually stunning and unique artworksAt its core, Neural
Style Transfer is a process of taking two images: a content image and a style image, and
combining them to create a new image. The resultant image retains the content of the content
image while adopting the artistic style of the style image. This technique is inspired by the
way our brains perceive and interpret art. When we look at a painting, for instance, we can
recognize the objects in the scene (content) and the unique artistic strokes and colors used by
the artist (style).

7
AM3602 Gen AI UNIT IV
602-Generative AI

Here’s a simple breakdown of the steps involved in Neural Style Transfer:

Input Images: As mentioned, you need two images — a content image and a style image.
Neural Network: You use a pre-trained deep neural network, typically a Convolutional
Neural Network (CNN), to extract both the content and style features from these images.
Loss Functions: You define two loss functions, one for content and one for style. The
content loss measures the difference in content between the generated image and the content
image. The style loss measures the difference in style between the generated image and the
style image.
Optimization: The objective is to minimize the content and style loss simultaneously. You
update the generated image iteratively until you achieve a satisfactory result.
Neural style transfer is an optimization technique used to take two images, a content image
and a style reference image (such as an artwork by a famous painter), and blend them so the
output image looks like the content image, but “painted” in the style of the style reference
image. This technique is used by many popular Android iOS apps such as Prisma,
DreamScope, and PicsArt.
VGG-19 Architecture overview
VGG-19 is a convolutional neural network (CNN) architecture from the VGG family of
models. The Visual Graphics Group (VGG) at the University of Oxford introduced the VGG
models, which are known for their simplicity and uniform architecture. VGG-19 has 19
layers, including 16 convolutional layers and 3 fully connected layers. The following are the
key features of the VGG-19 architecture:
Input Layer:
 Accepts 224×224-pixel images with three color channels (RGB) as input.
Convolutional Blocks (Blocks 1–5):
 VGG-19 is made up of five convolutional block sets. Each block is made up of
several convolutional layers followed by max-pooling layers.
 Convolutional layers commonly employ small 3×3 filters with a stride of one and
rectified linear unit (ReLU) activation functions.
 To reduce spatial dimensions, max-pooling layers with 2×2 filters and a stride of 2 are
used.
Layers that are fully connected (FC6, FC7, and FC8):
 There are three fully connected layers (FC6, FC7, and FC8) following the
convolutional blocks.
 The FC6 and FC7 layers each contain 4096 neurons and employ ReLU activation
functions.
 The FC8 layer (output layer) contains 1000 neurons with SoftMax activation, which
correspond to the 1000 classes in the ImageNet dataset on which VGG-19 was
trained.
Parameters:
8
AM3602 Gen AI UNIT IV
602-Generative AI

 Although VGG-19 is known for its simplicity, it has a large number of parameters,
owing to the fully connected layers.
 There are approximately 143.7 million trainable parameters in total.
Pre-trained Model:
 VGG-19 is a popular pre-trained model for a variety of computer vision tasks.
Researchers pre-trained it on large datasets such as ImageNet, allowing it to capture a
diverse set of features from various categories.
The neural style transfer paper uses feature maps generated by intermediate layers of VGG-
19 network to generate the output image. This architecture takes style and content images as
input and stores the features extracted by convolution layers of VGG network.

9
AM3602 Gen AI UNIT IV
602-Generative AI

Losses in Neural Style Transfer

REAL-WORLD USE CASES

 Art & Design: AI-generated art and design tools, such as Adobe’s AI-powered
applications and platforms like Artbreeder, enable creators to generate and customize
digital artwork. AI can also assist in product design by generating prototypes or iterating
on existing ideas.
 Medical Imaging: Generative models are transforming medical imaging by enhancing
image resolution, filling gaps in incomplete scans, and simulating realistic data for
training purposes. GANs are used to generate synthetic MRI or CT images for disease
detection, while style transfer techniques improve scan visualization.
 Content Creation: AI-powered tools like Jasper and Write sonic automate content writing
for blogs, advertisements, and social media. Video generation platforms like Synthesia
create lifelike avatars for educational or marketing content.
 Chatbots and Virtual Assistants: AI-driven systems like ChatGPT, Alexa, and Google
Assistant use NLP to deliver seamless conversational experiences. These tools help
businesses provide 24/7 customer support and improve user engagement.

1
0
AM3602 Gen AI UNIT IV
602-Generative AI

 Cybersecurity: Generative AI aids in simulating attack scenarios for training, detecting

malware, and generating realistic phishing email templates for security awareness
training. Models like VAEs help identify anomalous behavior in network traffic,
enhancing intrusion detection systems.

1
1