Interactive Generative Adversarial Network
(iGAN)
Deep Learning
COE6340
Presented By: AKMAL AHMAD
Table of Content
[Link] (GAN & iGAN)
[Link] of GAN
[Link] of iGAN
[Link] Mathematical Function
[Link] Mathematical Function
[Link] Functions
[Link] Features
[Link]
[Link]
[Link]
[Link] Work
[Link]
Introduction
What is GAN?
Generative Adversarial Network. It is a class of machine learning frameworks
introduced by Ian Goodfellow in 2014.
GANs are used for generating new data that resembles a given dataset, such as
images, audio, or video.
What is iGAN?
iGAN stands for Interactive Generative Adversarial Network. It is a specific use of
GANs focused on interactive image editing.
Developed by researchers at MIT (including Jun-Yan Zhu et al.).
Allows interactive image manipulation using GANs in real time.
Users can edit images in real-time by providing constraints or sketches, and the GAN
adjusts the image accordingly.
Architecture of GAN
[Link]
generation-using-gans/
Architecture of iGAN
Architecture of iGAN
1. Generator
Neural networks.
Takes random noise as input.
Tries to produce realistic fake image/data.
The Generator receives the user input and maps it into a latent space
Goal: Fool the discriminator into believing its output is real.
2. Discriminator
Takes real and fake data as input
Evaluates the generated image against real training images.
Provides feedback to the generator to improve its output.
It helps refine the generator’s output by providing feedback during the training phase
Example: It compares generated cat image with real cat photos and tells the generator
how close it was.
3. Latent Optimizer
During training, the discriminator’s feedback is used to improve the generator’s
performance
This component refines the latent vector to better match user input while keeping the
image realistic.
It optimizes the internal representation (latent space) used by the generator.
Example: If the ears in sketch look more like a fox, it adjusts the internal features to
produce more accurate cat-like features.
4. Image Output
The generator’s output is a high-quality image that reflects the user's editing
instructions.
This image is shown to the user for interactive editing and real-time feedback.
GAN Mathematical Function
Discriminator (D): maximize the probability of assigning correct labels (real vs. fake).
Generator (G): minimize the probability that the discriminator correctly identifies its
output as fake.
iGAN Mathematical Function
iGAN builds on GAN by incorporating user interactions and constraining the image
generation to the natural image manifold.
It starts from an initial latent vector z, and updates it so that the corresponding generated
image G(z) satisfies the user's edits.
iGAN Loss Functions
The iGAN loss is usually composed of several sub-losses, depending on the type of user
interaction. Here's a breakdown:
iGAN Loss Functions
Total Loss
Key Features of iGAN
1. Real-Time Feedback and Updates
It enhances user engagement.
It allows trial-and-error without long waiting times.
It supports iterative improvements with immediate visual validation.
2. Semantic Image Editing
Adjust shapes, like bending or resizing objects.
Modify textures, such as changing material appearance or style.
Rearrange layouts, including positioning and composition of objects.
3. High-Quality, Realistic Outputs
Highly detailed and photorealistic.
Often indistinguishable from real-world images.
Capable of preserving natural-looking textures and structures, even after editing.
Applications of iGAN
Interactive art tools
Game design and character creation
Medical imaging and simulation
Fashion and interior design
Storyboarding and Animation
Virtual Try-On for E-Commerce
Education and Training
Forensics and Law Enforcement
Personalized Avatars and Digital Identity
Advantages of iGAN
Real-Time Image Generation
User-Friendly and Intuitive Interface
High-Quality and Realistic Output
Semantic Control Over Image Features
Efficient Use of Latent Space
Reduces Need for Large Labeled Datasets
Interactive Learning Tool
Supports Iterative Design and Refinement
Conclusion
1. Revolutionizes Image Editing
iGAN introduces a user-friendly and interactive approach to image manipulation, allowing
users to sketch or modify images.
2. iGAN Combines Power of GANs with User Input
By integrating the strengths of GANs with intuitive user input (sketch, color, shape), iGAN
bridges the gap between AI-based generation and human creativity.
3. Real-Time Feedback and High-Quality Output
The system generates high-quality, realistic images in real-time, making it suitable for rapid
prototyping and iterative design workflows
4. Latent Space Optimization Enhances Control
iGAN uses latent vector interpolation and optimization techniques to ensure that user inputs
map to valid images on the natural image manifold, maintaining visual coherence
5. Foundation for Future Interactive AI Tools
iGAN lays the groundwork for more advanced, user-guided generative models, pushing
forward the field of human-AI interaction in content creation.
.
Future Work on iGAN
1. Extension to 3D and Video Generation: Expanding iGAN to handle 3D objects or
interactive video editing would significantly broaden its capabilities in animation and
virtual reality applications.
2. Multimodal Input Support: Incorporating other input types like voice commands, text
descriptions, or gesture inputs can make the system even more intuitive and accessible.
3. Support for Higher Resolution Images: Future versions of iGAN can focus on
generating ultra-high-resolution outputs to meet the growing demands of industries like
film, gaming, and digital art.
4. Dual-Generator Architecture: Investigate a two-generator framework to improve
adversarial balance and enhance recommendation diversity.
5. Integration of Auxiliary Data: Develop hybrid models combining iGAN with
knowledge graphs or social network data.
References
[Link]-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis. Sauer, A., Chitta,
K., Müller, J., & Geiger, A. (2023).
[Link]: Interactive Point-Based Manipulation on the Generative Image Manifold Pan, X., Tewari, A.,
Leimkühler, T., Liu, L., & Theobalt, C. (2023).
3.EG3D: Efficient Geometry-Aware 3D GANs Chan, E., Lin, C. Z., Chan, M., Nagano, K., et al. (2022).
[Link]-XL: Scaling StyleGAN to Large and Diverse Datasets Sauer, A., Schwarz, K., & Geiger, A. (2022).
[Link] Inversion: A Survey Xia, W., Yang, Y., Xue, J.-H., & Wu, B. (2021). GAN inversion: A survey. IEEE
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(3), 3121–3138.
[Link]-Free GAN Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., & Aila, T. (2021).
[Link]: Interpreting the Disentangled Face Representation Learned by GANs Shen, Y., Yang, C.,
Tang, X., & Zhou, B. (2020).
[Link] GANs Meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
Jang, W., & Agapito, L. (2020).
[Link] Generative Adversarial Networks with Limited Data Karras, T., Aittala, M., Hellsten, J., Laine, S., &
Aila, T. (2020).
[Link]: Interactive Image Generation via Generative Adversarial Networks Zhu, J.-Y., Krähenbühl, P.,
Shechtman, E., & Efros, A. A. (2016).
[Link]: A collaborative filtering model based on Improved Generative Adversarial
Networks for recommendation Xiaoyuan Song, Jiwei Qin ∗ , Qiulin Ren, Jiong Zheng(2023)
THANK
AKMAL
YOU!!!
AHMAD