Generative AI for Image

Synthesis and Processing

Tech Talk – AY 2024 – 25
10 – 10 - 2025
Presented by,
Dr. V N Manju,
Associate Professor,
Dept of CSE, CMRIT
Generative AI
1
 What is Generative AI?
Definition of Generative AI
• Generative AI refers to a category of artificial intelligence models that can
create new, original content by learning from existing data. These models
generate data that resembles the original dataset, including images, text, music,
or even video.
• Key Concept: Unlike traditional AI that only recognizes patterns or
makes predictions, generative AI creates something new.
• Example: A generative AI model trained on a dataset of landscapes can
generate entirely new, realistic-looking landscape images that do not exist in
reality.

Tech Talk – Generative AI for Image Synthesis 2

 Overview of Creating New Content
• Types of Content Generated by AI:
• Images: Creating photorealistic or artistic images from scratch.
• Text: Generating coherent, contextually relevant text (e.g., GPT models).
• Music: Composing melodies or entire songs in various styles.
• Videos and Animation: Generating videos frame-by-frame or interpolating
video frames for smoother transitions.
• Process:
• The AI model is trained on large amounts of data (e.g., images, text, etc.).
• The model learns underlying patterns, structures, and features of the data.
• Once trained, the model can produce new content that is similar to the training data but is
unique and original.

Tech Talk – Generative AI for Image Synthesis 3

 Key Methods Used in Generative AI
Generative Adversarial Networks (GANs)
• Structure: GANs consist of two models:
• Generator: Attempts to create new content (e.g., images).
• Discriminator: Tries to differentiate between real content and the generated content.
• Training Process:
• The generator creates new data, while the discriminator evaluates it.
• The generator learns from the discriminator’s feedback to improve the quality of the
generated content.
• Famous Application: GANs have been used to create realistic images (e.g., "This Person
Does Not Exist" website).

Tech Talk – Generative AI for Image Synthesis 4

Tech Talk – Generative AI for Image Synthesis 5
Tech Talk – Generative AI for Image Synthesis 6
Tech Talk – Generative AI for Image Synthesis 7
 Variational Autoencoders (VAEs)
• Purpose:
• VAEs encode data into a compressed form and then decode it back, learning to
represent the underlying data distribution.
• Structure: VAEs consist of two parts:
• Encoder: Compresses the input data into a latent space (a lower-dimensional
space).
• Decoder: Reconstructs the data from the latent space.
• Benefit: VAEs can generate new content by sampling from the latent
space.
• Example: Generating new faces or objects by sampling and decoding
latent features.

Tech Talk – Generative AI for Image Synthesis 8

 Diffusion Models
• How They Work:
• These models generate images by starting with random noise and iteratively
refining it to create a realistic image.
• Process:
• The model learns to reverse a degradation process (adding noise to images)
and applies it in reverse to generate images.
• Use Case:
• High-quality image synthesis, often used in creative design and content
generation.

Tech Talk – Generative AI for Image Synthesis 9

 Image Synthesis with Generative AI
• Definition:
• Creating New Images from Scratch or Noise
• Image Synthes is the process where AI models generate entirely new
images from scratch, often starting from random noise or very basic
inputs.
• Example: Generating a portrait of a person who doesn’t exist, synthesizing
landscapes, or creating abstract art.
• From Noise to Image: The AI model takes random noise (like static)
and, through a process of learning, generates a high-quality image,
which could be anything from a realistic face to an artistic landscape.

Tech Talk – Generative AI for Image Synthesis 10

 Examples of AI-Generated Content
• Portraits: AI models can generate hyper-realistic portraits of people who do
not exist by learning from vast datasets of real faces.
• Example: The website "This Person Does Not Exist" uses GANs to create faces of
people that are indistinguishable from real photographs.
• Landscapes: AI can synthesize completely new natural scenes like forests,
beaches, or mountains by blending elements it has learned from real-world
images.
• Example: AI-generated environments for virtual worlds or video games.
• Art Creation: AI can generate entirely new works of art, including abstract
compositions or imitations of artistic styles like Van Gogh or Picasso.
• Example: DeepArt or RunwayML allows users to generate art by mimicking famous
styles.

Tech Talk – Generative AI for Image Synthesis 11

 Generating Images Pixel by Pixel or Via
Features
• Step-by-Step Generation:
• Pixel-by-Pixel Generation (GANs): In some methods, the image is generated
pixel by pixel, where the model gradually learns to arrange pixels in ways that
form realistic images.
• How It Works: The generator starts by creating an image from random noise (a
cloud of pixels) and refines this noise to become a coherent image. The
discriminator then evaluates this image and provides feedback, helping the
generator improve over time.
• Result: Over many iterations, the AI refines the image until it becomes
nearly indistinguishable from a real image.

Tech Talk – Generative AI for Image Synthesis 12

 Feature-Based Generation (Diffusion
Models/VAEs)
• In other methods, the model doesn’t work directly with pixels but
generates key features first (e.g., eyes, mouth, hair) and then
synthesizes the final image by piecing these features together.
• How It Works:
• The model learns higher-level features of the image (such as shape, color,
texture) and generates images by assembling these features in a meaningful
way.
• Example: A diffusion model starts with a noisy image and gradually removes
the noise, reconstructing the image feature by feature until a sharp, detailed
picture is created.

Tech Talk – Generative AI for Image Synthesis 13

 How Generative AI for Images Works
1. Data Collection:
The process begins with gathering a large dataset of images. This dataset should be diverse and representative of
the types of images the model will eventually generate (e.g., landscapes, portraits, objects).
2. Data Preprocessing:
The collected images are preprocessed to ensure uniformity. This may include resizing, normalization (scaling
pixel values), and augmentation (creating variations of the images).
3. Model Selection:
Various generative models can be used, with the most common being Generative Adversarial Networks
(GANs) and Variational Autoencoders (VAEs). Each model has its unique architecture and training process.
4. Training the Model:
The chosen model is trained on the preprocessed dataset. During training, the model learns to recognize patterns,
features, and structures in the images. For GANs, two networks (generator and discriminator) compete against
each other; the generator creates images, while the discriminator evaluates their authenticity.
5. Loss Function:
A loss function measures how well the model is performing. For GANs, it quantifies the difference between real
and generated images. The model aims to minimize this loss through iterative adjustments.

Tech Talk – Generative AI for Image Synthesis 14

 Cont…
6. Feedback Loop:
In GANs, the generator receives feedback from the discriminator. If the discriminator correctly
identifies an image as fake, the generator adjusts its parameters to improve. This competition
continues until the generator produces high-quality images indistinguishable from real ones.
[Link] Generation:
Once the model is trained, it can generate new images by sampling from its learned distribution. In
the case of GANs, the generator takes random noise as input and transforms it into a coherent image.
[Link]-to-Image Synthesis:
For models like DALL·E, text prompts guide image generation. The model encodes the textual input
and decodes it into an image, effectively linking language to visual representation.
[Link]-Processing:
After image generation, additional processing may be applied. This can include enhancing image
quality, adjusting colors, or filtering artifacts.
[Link] and Fine-tuning:
The generated images are evaluated for quality. If they do not meet desired standards, the model may
undergo further training or fine-tuning on additional data.

Tech Talk – Generative AI for Image Synthesis 15

 11. Deployment:
11. Finally, the trained model is deployed as a web service or application,
allowing users to generate images based on text prompts or other inputs.
[Link] Interaction:
End-users can input their prompts, and the model generates images in real time.
This interactive feature showcases the practical applications of generative AI.
[Link] Learning:
Some systems allow for continuous learning, where the model improves over
time as it processes more data and receives feedback from users.

Tech Talk – Generative AI for Image Synthesis 16