JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY KAKINADA

KAKINADA – 533 003, Andhra Pradesh, India

B. Tech CSE (AI) (R23) COURSE STRUCTURE & SYLLABUS
(Applicable from the academic year 2023-24 and onwards)

Generative AI Lab L T P C
III Year II Semester
0 0 3 1.5

Course Objectives:

1. To learn Python and TensorFlow skills for Generative AI.

2. To study techniques for cleaning and preparing data for Generative AI tasks.

3. To implement generative AI models

4. To develop innovative applications using generative AI tools and techniques.

Course Outcomes:

After learning the course, students will be able to:

1. Implement Python and TensorFlow basics, including data handling and preprocessing techniques.

2. Implement Generative AI models such as GANs, VAEs, LSTM networks, and Transformer
models for image text, and music generation tasks.

3. Evaluate model performance and experiment with hyperparameters and optimization techniques to
enhance Generative AI outcomes.

4. Develop innovative applications in image, text, and music generation, showcasing practical skills

List of Experiments:

1. Write Python scripts to implement basic operations and TensorFlow 2 tensors

2. Implement a Generative Adversarial Network (GAN) architecture using TensorFlow 2. Train
theGAN model on a dataset such as MNIST or CIFAR-10 for image generation tasks.
3. Train a GAN model on a custom dataset for image generation. Experiment with
hyperparameters,loss functions, and optimization techniques to optimize GAN training.
4. Explore advanced techniques such as Wasserstein GANs, Progressive GANs, or StyleGANs
forimage generation. Implement and compare these techniques for generating high-quality
images.
5. Develop applications for image and video generation using trained Generative AI models.
Use themodels to generate art, create deep fakes, or synthesize video content.
6. Text Generation: Implement a Long Short-Term Memory (LSTM) network using
TensorFlow 2 fortext generation tasks. Train the LSTM model on a dataset of text sequences
and generate new textsamples.

Page 54 of 61
 JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY KAKINADA
KAKINADA – 533 003, Andhra Pradesh, India
B. Tech CSE (AI) (R23) COURSE STRUCTURE & SYLLABUS
(Applicable from the academic year 2023-24 and onwards)

7. Text generation: Implement a Transformer-based language model (e.g., GPT) using

TensorFlow 2for text generation. Fine-tune the model on a text corpus and generate coherent
and contextuallyrelevant text.
8. Text generation: Fine-tune a pre-trained language model (e.g., GPT, BERT) using transfer
learningtechniques. Fine-tune the model on a domain-specific dataset and evaluate its
performance for textgeneration tasks.
9. Text generation: Develop applications for text generation tasks such as story generation,
dialoguegeneration, or code generation using trained Generative AI models.
10. Music Generation:Preprocess music data and represent it in a suitable format for music
generation tasks. Explore MIDIor audio representations for training Generative AI models.
11. Music Generation:Implement a Long Short-Term Memory (LSTM) network using
TensorFlow 2 for music [Link] the LSTM model on a dataset of music sequences
and generate new musical compositions.
12. Generate Novel Music Compositions:Transformer-based Music Generation: Implement a
Transformer-based architecture (e.g.,MusicBERT, MusicGPT) using TensorFlow 2 for music
generation. Fine-tune the model on a musicdataset and generate novel music compositions.
References:

1. Responsible AI: Implementing Ethical and Unbiased Algorithms, by Shashin Mishra and
Sray Agarwal
2. Generative AI in Practice: 100+ Amazing Ways Generative Artificial Intelligence is
Changing Business andSociety, Bernard Marr
3. “Generative AI with Python and TensorFlow 2: Create images, text, and music with VAEs,
GANs, LSTMs,Transformer models”, Joseph Babcock and Raghav Bali
4. "Generative Adversarial Networks: An Overview" by Vinod Nair and Geoffrey E. Hinton.
5. "Hands-On Generative Adversarial Networks with PyTorch 1.x" by Stefano Bosisio and
Vijayabhaskar J.

Page 55 of 61
LAB [Link] Script to Implement Basic
Operations and TensorFlow 2 Tensors
Basic TensorFlow 2 Tensor Operations
🔹 Step 1: Import TensorFlow
import tensorflow as tf

print("TensorFlow Version:", tf.__version__)

Creating Tensors
# Scalar (0-D tensor)
a = [Link](10)

# Vector (1-D tensor)

b = [Link]([1, 2, 3, 4])

# Matrix (2-D tensor)

c = [Link]([[1, 2], [3, 4]])

# 3-D Tensor
d = [Link]([[[1, 2], [3, 4]],
[[5, 6], [7, 8]]])

print("Scalar:", a)
print("Vector:", b)
print("Matrix:\n", c)
print("3D Tensor:\n", d)

Basic Arithmetic Operations

x = [Link]([10, 20, 30])
y = [Link]([1, 2, 3])

# Addition
add = [Link](x, y)

# Subtraction
sub = [Link](x, y)

# Multiplication
mul = [Link](x, y)

# Division
div = [Link](x, y)

print("Addition:", add)
print("Subtraction:", sub)
print("Multiplication:", mul)
print("Division:", div)

Matrix Operations
m1 = [Link]([[1, 2], [3, 4]])
m2 = [Link]([[5, 6], [7, 8]])

# Matrix Multiplication
mat_mul = [Link](m1, m2)

# Transpose
transpose = [Link](m1)

print("Matrix Multiplication:\n", mat_mul)

print("Transpose:\n", transpose)

Tensor Properties
tensor = [Link]([[1, 2, 3], [4, 5, 6]])

print("Shape:", [Link])
print("Data Type:", [Link])
print("Rank:", [Link](tensor))
print("Size:", [Link](tensor))

Reshaping and Casting

t = [Link]([1, 2, 3, 4, 5, 6])

# Reshape
reshaped = [Link](t, (2, 3))

# Type Casting
casted = [Link](t, dtype=tf.float32)

print("Reshaped Tensor:\n", reshaped)

print("Casted Tensor:\n", casted)

Random Tensors
# Random uniform tensor
rand1 = [Link]((2, 2))

# Random normal tensor

rand2 = [Link]((2, 2))

print("Random Uniform:\n", rand1)

print("Random Normal:\n", rand2)

Converting Tensor to NumPy

tensor = [Link]([10, 20, 30])
numpy_array = [Link]()

print("NumPy Array:", numpy_array)

Sample Output (Example)

TensorFlow Version: 2.x.x
Addition: [Link]([11 22 33], shape=(3,), dtype=int32)
Matrix Multiplication:
[[19 22]
[43 50]]
Shape: (2, 3)
Rank: 2

LAB [Link] Required Libraries

import tensorflow as tf
from [Link] import layers
import numpy as np
import [Link] as plt

Load and Preprocess MNIST Dataset

We will use:

MNIST

# Load dataset
(x_train, _), (_, _) = [Link].load_data()

# Normalize to [-1, 1]
x_train = (x_train - 127.5) / 127.5
x_train = np.expand_dims(x_train, axis=-1)

BUFFER_SIZE = 60000
BATCH_SIZE = 256

dataset = [Link].from_tensor_slices(x_train)\
.shuffle(BUFFER_SIZE)\
.batch(BATCH_SIZE)
 Define Generator Model
def build_generator():
model = [Link]()

[Link]([Link](7*7*256, use_bias=False, input_shape=(100,)))

[Link]([Link]())
[Link]([Link]())

[Link]([Link]((7, 7, 256)))

[Link](layers.Conv2DTranspose(128, (5,5), strides=(1,1),

padding='same', use_bias=False))
[Link]([Link]())
[Link]([Link]())

[Link](layers.Conv2DTranspose(64, (5,5), strides=(2,2),

padding='same', use_bias=False))
[Link]([Link]())
[Link]([Link]())

[Link](layers.Conv2DTranspose(1, (5,5), strides=(2,2), padding='same',

use_bias=False, activation='tanh'))

return model

generator = build_generator()

Define Discriminator Model

def build_discriminator():
model = [Link]()

[Link](layers.Conv2D(64, (5,5), strides=(2,2), padding='same',

input_shape=[28,28,1]))
[Link]([Link]())
[Link]([Link](0.3))

[Link](layers.Conv2D(128, (5,5), strides=(2,2), padding='same'))

[Link]([Link]())
[Link]([Link](0.3))

[Link]([Link]())
[Link]([Link](1))

return model

discriminator = build_discriminator()

Define Loss Functions & Optimizers

cross_entropy = [Link](from_logits=True)

def generator_loss(fake_output):
 return cross_entropy(tf.ones_like(fake_output), fake_output)

def discriminator_loss(real_output, fake_output):

real_loss = cross_entropy(tf.ones_like(real_output), real_output)
fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)
return real_loss + fake_loss

generator_optimizer = [Link](1e-4)
discriminator_optimizer = [Link](1e-4)

Training Step
@[Link]
def train_step(images):
noise = [Link]([BATCH_SIZE, 100])

with [Link]() as gen_tape, [Link]() as disc_tape:

generated_images = generator(noise, trainingTrue)

real_output = discriminator(images, training=True)

fake_output = discriminator(generated_images, training=True)

gen_loss = generator_loss(fake_output)
disc_loss = discriminator_loss(real_output, fake_output)

gradients_of_generator = gen_tape.gradient(gen_loss,
generator.trainable_variables)
gradients_of_discriminator = disc_tape.gradient(disc_loss,

discriminator.trainable_variables)

generator_optimizer.apply_gradients(zip(gradients_of_generator,
generator.trainable_variables))

discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator,

discriminator.trainable_variables))

Training Loop
EPOCHS = 50
noise_dim = 100
num_examples_to_generate = 16

seed = [Link]([num_examples_to_generate, noise_dim])

def train(dataset, epochs):

for epoch in range(epochs):
for image_batch in dataset:
train_step(image_batch)

print(f'Epoch {epoch+1} completed')

train(dataset, EPOCHS)
 Generate Images After Training
def generate_and_plot(model, test_input):
predictions = model(test_input, training=False)

fig = [Link](figsize=(4,4))
for i in range([Link][0]):
[Link](4,4,i+1)
[Link](predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')
[Link]('off')
[Link]()

generate_and_plot(generator, seed)

Output
After ~50 epochs, the GAN will generate realistic handwritten digits similar to MNIST dataset.

If You Want CIFAR-10 Instead

You can replace MNIST with:

CIFAR-10

Just modify:

 Input shape to (32,32,3)

 Generator final layer filters to 3
 Normalize images to [-1,1]

LAB 3. Train a GAN model on a custom dataset for image generation. Experiment with
hyperparameters,loss functions, and optimization techniques to optimize GAN training.

Aim
To implement and train a GAN model on a custom image dataset and experiment with:

 Hyperparameters
 Loss functions
 Optimization techniques
to improve GAN training stability and output quality.
Software Requirements
 Python 3.8+
 TensorFlow 2.x
 NumPy
 Matplotlib
 OpenCV / PIL

Procedure
1. Load custom dataset
2. Preprocess images
3. Build Generator
4. Build Discriminator
5. Define loss functions
6. Train GAN
7. Tune hyperparameters
8. Generate sample images

Program
import tensorflow as tf
from [Link] import layers
import numpy as np
import [Link] as plt
import os
from glob import glob
import cv2

# ==========================
# 1. Hyperparameters
# ==========================

IMG_SIZE = 64
BATCH_SIZE = 32
LATENT_DIM = 100
EPOCHS = 100
LEARNING_RATE = 0.0002
BETA_1 = 0.5

DATASET_PATH = "custom_dataset/" # Folder containing images

# ==========================
# 2. Load Custom Dataset
# ==========================

def load_images(path):
 images = []
files = glob([Link](path, "*"))
for file in files:
img = [Link](file)
img = [Link](img, (IMG_SIZE, IMG_SIZE))
img = [Link](img, cv2.COLOR_BGR2RGB)
img = (img / 127.5) - 1.0 # Normalize to [-1,1]
[Link](img)
return [Link](images)

images = load_images(DATASET_PATH)
dataset = [Link].from_tensor_slices(images)\
.shuffle(1000)\
.batch(BATCH_SIZE)

# ==========================
# 3. Build Generator
# ==========================

def build_generator():
model = [Link]()
[Link]([Link](8*8*256, use_bias=False,
input_shape=(LATENT_DIM,)))
[Link]([Link]())
[Link]([Link]())

[Link]([Link]((8, 8, 256)))

[Link](layers.Conv2DTranspose(128, 5, strides=2, padding='same',

use_bias=False))
[Link]([Link]())
[Link]([Link]())

[Link](layers.Conv2DTranspose(64, 5, strides=2, padding='same',

use_bias=False))
[Link]([Link]())
[Link]([Link]())

[Link](layers.Conv2DTranspose(3, 5, strides=2, padding='same',

use_bias=False, activation='tanh'))

return model

# ==========================
# 4. Build Discriminator
# ==========================

def build_discriminator():
model = [Link]()

[Link](layers.Conv2D(64, 5, strides=2, padding='same',

input_shape=[IMG_SIZE, IMG_SIZE, 3]))
[Link]([Link]())
[Link]([Link](0.3))

[Link](layers.Conv2D(128, 5, strides=2, padding='same'))

[Link]([Link]())
[Link]([Link](0.3))
 [Link]([Link]())
[Link]([Link](1))

return model

generator = build_generator()
discriminator = build_discriminator()

# ==========================
# 5. Loss Functions
# ==========================

cross_entropy = [Link](from_logits=True)

def discriminator_loss(real_output, fake_output):

real_loss = cross_entropy(tf.ones_like(real_output)*0.9, real_output) #
label smoothing
fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)
return real_loss + fake_loss

def generator_loss(fake_output):
return cross_entropy(tf.ones_like(fake_output), fake_output)

# ==========================
# 6. Optimizers
# ==========================

generator_optimizer = [Link](LEARNING_RATE, beta_1=BETA_1)

discriminator_optimizer = [Link](LEARNING_RATE,
beta_1=BETA_1)

# ==========================
# 7. Training Step
# ==========================

@[Link]
def train_step(images):
noise = [Link]([BATCH_SIZE, LATENT_DIM])

with [Link]() as gen_tape, [Link]() as disc_tape:

generated_images = generator(noise, training=True)

real_output = discriminator(images, training=True)

fake_output = discriminator(generated_images, training=True)

gen_loss = generator_loss(fake_output)
disc_loss = discriminator_loss(real_output, fake_output)

gradients_of_generator = gen_tape.gradient(gen_loss,
generator.trainable_variables)
gradients_of_discriminator = disc_tape.gradient(disc_loss,

discriminator.trainable_variables)

generator_optimizer.apply_gradients(zip(gradients_of_generator,
generator.trainable_variables))
discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator,

discriminator.trainable_variables))
# ==========================
# 8. Training Loop
# ==========================

def train(dataset, epochs):

for epoch in range(epochs):
for image_batch in dataset:
train_step(image_batch)

print(f"Epoch {epoch+1}/{epochs} completed")

train(dataset, EPOCHS)

Hyperparameter Experiments
Parameter Experiment 1 Experiment 2 Experiment 3
Learning Rate 0.0002 0.0001 0.00005
Batch Size 32 64 128
Beta1 0.5 0.4 0.9
Loss BCE Hinge WGAN-GP

Advanced Improvements
Hinge Loss
def discriminator_loss(real, fake):
return tf.reduce_mean([Link](1. - real)) + \
tf.reduce_mean([Link](1. + fake))

def generator_loss(fake):
return -tf.reduce_mean(fake)

WGAN (Wasserstein GAN)

 Remove sigmoid
 Use weight clipping
 Use RMSprop optimizer

Optimization Techniques

 Spectral Normalization
 Gradient Penalty
 Label smoothing
 Batch Normalization
 Dropout tuning
Results
 Generated realistic synthetic images.
 Training stabilized using:
o Learning rate = 0.0001
o Beta1 = 0.5
o Label smoothing
o Batch Normalization

LAB 5.

Develop applications for image and video generation using

trained Generative AI models. Use themodels to generate art,
create deep fakes, or synthesize video content.

Aim

To develop applications for image and video generation using trained Generative AI models
such as GANs, Stable Diffusion, and deepfake frameworks.

EXPERIMENT 1: Image Generation using

GAN (MNIST Dataset)
Objective
To train a Generative Adversarial Network (GAN) for generating handwritten digit images
using the MNIST dataset.

🔹 System Requirements
 Python 3.8+
 TensorFlow 2.x
 NumPy
 Matplotlib
🔹 Architecture Diagram

Program Code (TensorFlow 2)

import tensorflow as tf
from [Link] import layers
import numpy as np
import [Link] as plt

# Load MNIST dataset

(x_train, _), (_, _) = [Link].load_data()
x_train = x_train / 127.5 - 1.0
x_train = np.expand_dims(x_train, axis=-1)

BUFFER_SIZE = 60000
BATCH_SIZE = 256

train_dataset = [Link].from_tensor_slices(x_train)\
.shuffle(BUFFER_SIZE).batch(BATCH_SIZE)

# Generator Model
def build_generator():
model = [Link]([
[Link](256, input_dim=100),
[Link](),
[Link](512),
[Link](),
[Link](1024),
[Link](),
[Link](28*28*1, activation='tanh'),
[Link]((28,28,1))
])
return model

# Discriminator Model
def build_discriminator():
model = [Link]([
[Link](input_shape=(28,28,1)),
[Link](512),
[Link](),
[Link](256),
 [Link](),
[Link](1, activation='sigmoid')
])
return model

generator = build_generator()
discriminator = build_discriminator()

cross_entropy = [Link]()
g_optimizer = [Link](1e-4)
d_optimizer = [Link](1e-4)

@[Link]
def train_step(images):
noise = [Link]([BATCH_SIZE, 100])

with [Link]() as gen_tape, [Link]() as disc_tape:

generated_images = generator(noise, training=True)

real_output = discriminator(images, training=True)

fake_output = discriminator(generated_images, training=True)

gen_loss = cross_entropy(tf.ones_like(fake_output), fake_output)

disc_loss = cross_entropy(tf.ones_like(real_output), real_output) + \
cross_entropy(tf.zeros_like(fake_output), fake_output)

gradients_of_generator = gen_tape.gradient(gen_loss,
generator.trainable_variables)
gradients_of_discriminator = disc_tape.gradient(disc_loss,
discriminator.trainable_variables)

g_optimizer.apply_gradients(zip(gradients_of_generator,
generator.trainable_variables))
d_optimizer.apply_gradients(zip(gradients_of_discriminator,
discriminator.trainable_variables))

EPOCHS = 50

for epoch in range(EPOCHS):

for image_batch in train_dataset:
train_step(image_batch)
print(f"Epoch {epoch+1} completed")

Output

Result
The GAN model successfully generated synthetic handwritten digit images similar to MNIST
dataset samples.

EXPERIMENT 2: Text-to-Image Generation

using Stable Diffusion
Objective
To generate AI art using pre-trained diffusion models like Stable Diffusion.

Theory
Stable Diffusion is a latent diffusion model that generates high-quality images from textual
prompts.

Model Workflow

Python Code (Using Hugging Face Diffusers)

from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"

pipe = StableDiffusionPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16
).to("cuda")
prompt = "A futuristic city in cyberpunk style at sunset"

image = pipe(prompt).images[0]
[Link]("generated_art.png")

Output

Result
The model generated high-quality artistic images based on text prompts.

EXPERIMENT 3: Deepfake / Face Swap

Application
Objective
To perform face swapping using pre-trained deepfake frameworks such as DeepFaceLab.

Basic Command Steps

1. Extract frames
2. Detect & align faces
3. Train autoencoder model
4. Merge faces into video
5. Render final output

⚠ Ethical Note:
Deepfake technology must be used only for educational, research, and authorized purposes.

EXPERIMENT 4: Video Generation using

Text-to-Video Models
Objective
To generate short videos using generative models such as ModelScope Text-to-Video.

Code Example
from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained(
"damo-vilab/text-to-video-ms-1.7b",
torch_dtype=torch.float16
)
pipe = [Link]("cuda")

prompt = "A dog running on the beach during sunset"

video_frames = pipe(prompt).frames

# Save frames as GIF

import imageio
[Link]("generated_video.gif", video_frames, fps=8)

Output
LAB-6
Generation: Implement a Long Short-Term Memory (LSTM) network using TensorFlow 2 fortext
generation tasks. Train the LSTM model on a dataset of text sequences and generate new
textsamples.

Aim
To implement a Long Short-Term Memory (LSTM) network using TensorFlow 2 for text
generation. Train the model on a text dataset and generate new text sequences.

Requirements
 Python 3.x
 TensorFlow 2.x
 NumPy
 Matplotlib (optional)

Install:

pip install tensorflow numpy

Program: LSTM Text Generation using

TensorFlow 2
import tensorflow as tf
import numpy as np
import os

# Load sample text dataset (Shakespeare dataset)

path_to_file = [Link].get_file(
'[Link]',

'[Link]
)

# Read text
text = open(path_to_file, 'rb').read().decode(encoding='utf-8')
print("Length of text:", len(text))

# Create vocabulary
vocab = sorted(set(text))
vocab_size = len(vocab)

# Character to index mapping

char2idx = {u:i for i, u in enumerate(vocab)}
idx2char = [Link](vocab)

# Convert text to integer

text_as_int = [Link]([char2idx[c] for c in text])

# Create training sequences

seq_length = 100
examples_per_epoch = len(text)//(seq_length+1)

char_dataset = [Link].from_tensor_slices(text_as_int)
sequences = char_dataset.batch(seq_length+1, drop_remainder=True)

def split_input_target(chunk):
input_text = chunk[:-1]
target_text = chunk[1:]
return input_text, target_text

dataset = [Link](split_input_target)
BATCH_SIZE = 64
BUFFER_SIZE = 10000

dataset = [Link](BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)

# Build LSTM Model

def build_model(vocab_size, embedding_dim, rnn_units, batch_size):
model = [Link]([
[Link](vocab_size, embedding_dim,
batch_input_shape=[batch_size, None]),
[Link](rnn_units,
return_sequences=True,
stateful=True,
recurrent_initializer='glorot_uniform'),
[Link](vocab_size)
])
return model

embedding_dim = 256
rnn_units = 1024

model = build_model(vocab_size, embedding_dim, rnn_units, BATCH_SIZE)

# Compile model
[Link](optimizer='adam',
loss=[Link](from_logits=True))

# Train model
EPOCHS = 10
history = [Link](dataset, epochs=EPOCHS)

# Save model
[Link]("text_generation_model.h5")

Text Generation Code

# Rebuild model with batch size 1
model = build_model(vocab_size, embedding_dim, rnn_units, batch_size=1)
model.load_weights("text_generation_model.h5")
[Link]([Link]([1, None]))

def generate_text(model, start_string):

num_generate = 500
input_eval = [char2idx[s] for s in start_string]
input_eval = tf.expand_dims(input_eval, 0)

text_generated = []
temperature = 1.0

model.reset_states()
for i in range(num_generate):
predictions = model(input_eval)
predictions = [Link](predictions, 0)

predictions = predictions / temperature

predicted_id = [Link](predictions, num_samples=1)[-
1,0].numpy()

input_eval = tf.expand_dims([predicted_id], 0)

text_generated.append(idx2char[predicted_id])

return start_string + ''.join(text_generated)

print(generate_text(model, start_string="ROMEO: "))

Expected Output
 Model trains over epochs
 Loss gradually decreases
 Generated Shakespeare-like text

Example output:
ROMEO:
I shall not speak of thee again,
For thou art fair and full of grace...

Result
Successfully implemented and trained an LSTM model using TensorFlow 2 for text generation
and generated new text sequences.

LAB-7

Text generation: Implement a Transformer-based language model (e.g., GPT) using

TensorFlow 2for text generation. Fine-tune the model on a text corpus and generate coherent and
contextuallyrelevant text.

Aim

To implement a Transformer-based language model (GPT-style) using TensorFlow 2, fine-tune

it on a custom text corpus, and generate coherent, contextually relevant text

Requirements
pip install tensorflow datasets transformers

Step 1: Load and Prepare Dataset

Example: Using a small text corpus (Shakespeare-style text)

import tensorflow as tf
import numpy as np
import os

# Load text file

with open("[Link]", "r", encoding="utf-8") as f:
text = [Link]()

# Tokenization
tokenizer = [Link]()
tokenizer.fit_on_texts([text])
total_words = len(tokenizer.word_index) + 1

# Create input sequences

input_sequences = []
for line in [Link]('\n'):
token_list = tokenizer.texts_to_sequences([line])[0]
for i in range(1, len(token_list)):
n_gram_seq = token_list[:i+1]
input_sequences.append(n_gram_seq)
# Pad sequences
max_seq_len = max([len(x) for x in input_sequences])
input_sequences = [Link].pad_sequences(
input_sequences, maxlen=max_seq_len, padding='pre'
)

X = input_sequences[:, :-1]
y = input_sequences[:, -1]
y = [Link].to_categorical(y, num_classes=total_words)

Step 2: Build Transformer (GPT-like) Model

from [Link] import layers

def transformer_block(embed_dim, num_heads, ff_dim, rate=0.1):

inputs = [Link](shape=(None, embed_dim))
attention = [Link](
num_heads=num_heads, key_dim=embed_dim
)(inputs, inputs, use_causal_mask=True)

attention = [Link](rate)(attention)
out1 = [Link](epsilon=1e-6)(inputs + attention)

ffn = [Link](ff_dim, activation="relu")(out1)

ffn = [Link](embed_dim)(ffn)
ffn = [Link](rate)(ffn)

return [Link](inputs, [Link](epsilon=1e-

6)(out1 + ffn))

embed_dim = 128
num_heads = 4
ff_dim = 128

inputs = [Link](shape=(max_seq_len-1,))
embedding_layer = [Link](total_words, embed_dim)(inputs)

transformer = transformer_block(embed_dim, num_heads, ff_dim)

x = transformer(embedding_layer)

x = layers.GlobalAveragePooling1D()(x)
outputs = [Link](total_words, activation="softmax")(x)

model = [Link](inputs=inputs, outputs=outputs)

[Link](
optimizer="adam",
loss="categorical_crossentropy",
metrics=["accuracy"]
)

[Link]()
Step 3: Train the Model
history = [Link](X, y, epochs=30, batch_size=64)

Step 4: Text Generation Function

def generate_text(seed_text, next_words=20):
for _ in range(next_words):
token_list = tokenizer.texts_to_sequences([seed_text])[0]
token_list = [Link].pad_sequences(
[token_list], maxlen=max_seq_len-1, padding='pre'
)
predicted = [Link]([Link](token_list), axis=-1)

output_word = ""
for word, index in tokenizer.word_index.items():
if index == predicted:
output_word = word
break

seed_text += " " + output_word

return seed_text

print(generate_text("Once upon a time", 30))

Expected Output
Example generated text:

Once upon a time there was a king who ruled the land with wisdom and courage and the people
loved him for his kindness and strength...

LAB-8

Text generation: Fine-tune a pre-trained language model (e.g., GPT, BERT) using transfer
learningtechniques. Fine-tune the model on a domain-specific dataset and evaluate its
performance for textgeneration tasks.

Aim

To fine-tune a pre-trained language model using transfer learning techniques on a domain-

specific dataset and evaluate its performance for text generation tasks.

Requirements
 Python 3.8+
  PyTorch
 Transformers library (Hugging Face)
 Datasets library
 GPU (recommended)

Install dependencies:

pip install transformers datasets torch evaluate

EXPERIMENT PROCEDURE
Step 1: Import Libraries
import torch
from datasets import load_dataset
from transformers import GPT2Tokenizer, GPT2LMHeadModel
from transformers import Trainer, TrainingArguments
from transformers import DataCollatorForLanguageModeling

Step 2: Load Pre-trained Model

model_name = "gpt2"

tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name)

tokenizer.pad_token = tokenizer.eos_token

Step 3: Load Domain-Specific Dataset

dataset = load_dataset("text", data_files={"train": "[Link]"})

Example [Link]:

Diabetes is a chronic disease that affects blood sugar regulation.

Hypertension is a major risk factor for cardiovascular disease.

Step 4: Tokenization
def tokenize_function(examples):
return tokenizer(
examples["text"],
truncation=True,
padding="max_length",
max_length=128
)
tokenized_dataset = [Link](tokenize_function, batched=True,
remove_columns=["text"])

Step 5: Data Collator

data_collator = DataCollatorForLanguageModeling(
tokenizer=tokenizer,
mlm=False # GPT uses causal language modeling
)

Step 6: Define Training Arguments

training_args = TrainingArguments(
output_dir="./gpt2-domain",
overwrite_output_dir=True,
num_train_epochs=3,
per_device_train_batch_size=8,
save_steps=500,
save_total_limit=2,
logging_steps=100,
prediction_loss_only=True
)

Step 7: Trainer Setup

trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset["train"],
data_collator=data_collator
)

Step 8: Fine-Tune Model

[Link]()
trainer.save_model("./fine_tuned_model")

Text Generation
from transformers import pipeline

generator = pipeline("text-generation", model="./fine_tuned_model")

prompt = "Diabetes treatment involves"

output = generator(prompt, max_length=50)

print(output[0]["generated_text"])
 Evaluation
Perplexity
import math
eval_results = [Link]()
perplexity = [Link](eval_results["eval_loss"])
print("Perplexity:", perplexity)

Lower perplexity → Better model.

BLEU Score (Optional)

import evaluate

bleu = [Link]("bleu")

Expected Output
 Domain-specific vocabulary adaptation
 Lower perplexity compared to base GPT2
 Improved coherence in generated domain text

LAB-9

Text generation: Develop applications for text generation tasks such as story generation,
dialoguegeneration, or code generation using trained Generative AI models.
.

Lab Program: Text Generation Using Generative AI Models

Aim

To develop applications for text generation tasks such as:

 Story Generation
 Dialogue Generation
 Code Generation
using trained Generative AI models.

Introduction
Text generation uses Generative AI models like Transformer-based architectures to produce
human-like text. Popular pretrained models include:

 OpenAI (GPT models)

 Google (PaLM, Gemini)
 Meta (LLaMA models)
 Hugging Face (Transformers library)

These models are trained on large datasets and can generate coherent paragraphs, conversations,
and even programming code.

Experiment 1: Story Generation

Objective

Generate creative stories using a pretrained language model.

Tools Required

 Python 3.x
 Transformers library
 PyTorch or TensorFlow

Sample Implementation (Using Hugging Face Transformers)

from transformers import pipeline

# Load text generation pipeline

generator = pipeline("text-generation", model="gpt2")

prompt = "Once upon a time in a futuristic city,"

output = generator(prompt, max_length=150, num_return_sequences=1)

print(output[0]['generated_text'])

Output

The model generates a short story continuation based on the given prompt.

Applications

 Creative writing assistants

 Content creation tools
 Automated storytelling systems
 Experiment 2: Dialogue Generation
Objective

Develop a simple conversational AI system.

Sample Code
from transformers import pipeline

chatbot = pipeline("conversational", model="microsoft/DialoGPT-medium")

from transformers import Conversation

conversation = Conversation("Hello! How are you?")
response = chatbot(conversation)

print(response)

Output

Model responds conversationally.

Applications

 Customer support bots

 Virtual assistants
 FAQ automation

Experiment 3: Code Generation

Objective

Generate programming code automatically from natural language prompts.

Sample Code
from transformers import pipeline

code_generator = pipeline("text-generation", model="Salesforce/codegen-350M-

mono")

prompt = "Write a Python function to calculate factorial of a number:"

output = code_generator(prompt, max_length=120)

print(output[0]['generated_text'])

Output

Generates Python function for factorial.