Scribd
Upload
24 views10 pages

Deep Learning Lab Manual with Code

Uploaded by

ruchithakanakanallu
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
24 views10 pages

Deep Learning Lab Manual with Code

Uploaded by

ruchithakanakanallu
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Deep Learning Laboratory Manual – With Code Skeletons

This manual contains objectives, steps, and Python/PyTorch starter code for each lab.

Experiment 1: Word Embeddings (Neural Word2Vec)


Objective: Design and implement a neural network to generate word embeddings for a document corpus.
Steps:
1. Collect & clean corpus; tokenize and build vocabulary.
2. Create training pairs for Skip-gram (center→context) or CBOW (context→center).
3. Define embedding and output layers; train with negative sampling.
4. Evaluate using nearest neighbors; visualize with PCA/t-SNE.
Starter Code (skeleton):
# Word2Vec (Skip-gram with Negative Sampling) – PyTorch Skeleton
import math, random, collections
import torch, [Link] as nn
from [Link] import Dataset, DataLoader

# 1) Toy corpus and preprocessing (replace with your corpus)


corpus = "we love deep learning and we love neural networks".split()
vocab = sorted(set(corpus))
stoi = {w:i for i,w in enumerate(vocab)}
itos = {i:w for w,i in [Link]()}

# 2) Build training pairs for Skip-gram


window = 2
pairs = []
for i, w in enumerate(corpus):
center = stoi[w]
for j in range(max(0, i-window), min(len(corpus), i+window+1)):
if i == j:
continue
context = stoi[corpus[j]]
[Link]((center, context))

# Negative sampling table (uniform for skeleton)


unigram = [1]*len(vocab)

class SkipGramNegDataset(Dataset):
def __init__(self, pairs, num_neg=5):
[Link] = pairs; self.num_neg = num_neg
def __len__(self): return len([Link])
def __getitem__(self, idx):
c, ctx = [Link][idx]
negs = []
while len(negs) < self.num_neg:
n = [Link](len(vocab))
if n != ctx:
[Link](n)
return [Link](c), [Link](ctx), [Link](negs)

ds = SkipGramNegDataset(pairs, num_neg=5)
dl = DataLoader(ds, batch_size=32, shuffle=True)

# 3) Model
embed_dim = 50
class SGNS([Link]):
def __init__(self, vocab_size, d):
super().__init__()
self.in_embed = [Link](vocab_size, d)
self.out_embed = [Link](vocab_size, d)
def forward(self, center, pos, neg):
v = self.in_embed(center) # [B, d]
u_pos = self.out_embed(pos) # [B, d]
u_neg = self.out_embed(neg) # [B, K, d]
pos_score = [Link](v*u_pos, dim=1) # [B]
neg_score = [Link](u_neg, [Link](2)).squeeze() # [B, K]
loss = -[Link]([Link](pos_score) + 1e-9).mean() \
-[Link]([Link](-neg_score) + 1e-9).mean()
return loss
def get_embeddings(self):
return self.in_embed.[Link]

device = 'cuda' if [Link].is_available() else 'cpu'


model = SGNS(len(vocab), embed_dim).to(device)
opt = [Link]([Link](), lr=1e-3)

# 4) Train (toy loop)


for epoch in range(5):
for center, pos, neg in dl:
center, pos, neg = [Link](device), [Link](device), [Link](device)
loss = model(center, pos, neg)
opt.zero_grad(); [Link](); [Link]()
print(f"Epoch {epoch+1}, loss={[Link]():.4f}")

emb = model.get_embeddings()
print("Embedding for 'we':", emb[stoi['we']][:5])
Experiment 2: Deep Neural Network for Classification (Tabular)
Objective: Build a feedforward DNN classifier for a tabular dataset.
Steps:
1. Load dataset (e.g., UCI, Kaggle). Split into train/val/test.
2. Standardize numeric features; encode categorical features.
3. Define DNN with ReLU and dropout; train with cross-entropy.
4. Report accuracy, precision/recall/F1; show confusion matrix.
Starter Code (skeleton):
# DNN Classifier – PyTorch Skeleton (synthetic data for structure)
import torch, [Link] as nn
from [Link] import make_classification
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from [Link] import classification_report, confusion_matrix
import numpy as np

X, y = make_classification(n_samples=2000, n_features=20, n_informative=10, n_classes=3, random_


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler().fit(X_train)
X_train = [Link](X_train).astype(np.float32)
X_test = [Link](X_test).astype(np.float32)

X_train, y_train = [Link](X_train), [Link](y_train, dtype=[Link])


X_test, y_test = [Link](X_test), [Link](y_test, dtype=[Link])

class MLP([Link]):
def __init__(self, in_dim, hidden=[128,64], out_dim=3):
super().__init__()
layers = []
dims = [in_dim]+hidden
for a,b in zip(dims[:-1], dims[1:]):
layers += [[Link](a,b), [Link](), [Link](0.2)]
layers += [[Link](hidden[-1], out_dim)]
[Link] = [Link](*layers)
def forward(self, x): return [Link](x)

device = 'cuda' if [Link].is_available() else 'cpu'


model = MLP(20).to(device)
opt = [Link]([Link](), lr=1e-3)
crit = [Link]()

for epoch in range(20):


[Link]()
opt.zero_grad()
loss = crit(model(X_train.to(device)), y_train.to(device))
[Link](); [Link]()
if (epoch+1)%5==0:
print("epoch", epoch+1, "loss", float(loss))

[Link]()
with torch.no_grad():
logits = model(X_test.to(device)).cpu().numpy()
y_pred = [Link](1)
print(classification_report(y_test, y_pred))
print("Confusion:\n", confusion_matrix(y_test, y_pred))
Experiment 3: CNN for Image Classification
Objective: Design and implement a CNN for an image dataset (e.g., CIFAR-10).
Steps:
1. Load dataset with torchvision; apply normalization and augmentation.
2. Build CNN (Conv-BN-ReLU-Pool × N; FC → Softmax).
3. Train with Adam; use early stopping; track accuracy.
4. Evaluate; visualize sample predictions.
Starter Code (skeleton):
# CNN for CIFAR-10 – PyTorch Skeleton
import torch, [Link] as nn, [Link] as F
from [Link] import DataLoader
import torchvision
import [Link] as T

device = 'cuda' if [Link].is_available() else 'cpu'

transform = [Link]([
[Link](),
[Link](32, padding=4),
[Link](),
[Link]((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
])
train_set = [Link].CIFAR10(root='./data', train=True, download=True, transform=tra
test_set = [Link].CIFAR10(root='./data', train=False, download=True,
transform=[Link]([[Link](),
[Link]((0.4914,0.4822,0.4465),(0.2023,0.1994,0.20
train_loader = DataLoader(train_set, batch_size=128, shuffle=True, num_workers=2)
test_loader = DataLoader(test_set, batch_size=256, shuffle=False, num_workers=2)

class SimpleCNN([Link]):
def __init__(self, num_classes=10):
super().__init__()
[Link] = [Link](
nn.Conv2d(3, 32, 3, padding=1), nn.BatchNorm2d(32), [Link](),
nn.Conv2d(32, 32, 3, padding=1), nn.BatchNorm2d(32), [Link](),
nn.MaxPool2d(2),
nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), [Link](),
nn.Conv2d(64, 64, 3, padding=1), nn.BatchNorm2d(64), [Link](),
nn.MaxPool2d(2),
)
[Link] = [Link](
[Link](),
[Link](64*8*8, 256), [Link](), [Link](0.5),
[Link](256, num_classes)
)
def forward(self, x): return [Link]([Link](x))

model = SimpleCNN().to(device)
opt = [Link]([Link](), lr=1e-3)
crit = [Link]()

for epoch in range(5):


[Link]()
for x,y in train_loader:
x,y = [Link](device), [Link](device)
opt.zero_grad(); loss = crit(model(x), y); [Link](); [Link]()
print("epoch", epoch+1, "loss", float(loss))

# Eval
[Link](); correct=total=0
with torch.no_grad():
for x,y in test_loader:
x,y = [Link](device), [Link](device)
pred = model(x).argmax(1)
correct += (pred==y).sum().item(); total += [Link]()
print("Test Acc:", correct/total)
Experiment 4: Autoencoder for Image Compression
Objective: Build an autoencoder and demonstrate compression on an image dataset (MNIST).
Steps:
1. Load grayscale images (MNIST).
2. Define encoder → bottleneck → decoder (Conv or Dense).
3. Train with MSE or BCE loss; visualize reconstructions.
4. Explore latent dim effect on quality.
Starter Code (skeleton):
# Convolutional Autoencoder – PyTorch Skeleton (MNIST)
import torch, [Link] as nn
import torchvision
import [Link] as T
from [Link] import DataLoader

device = 'cuda' if [Link].is_available() else 'cpu'

train_set = [Link](root='./data', train=True, download=True, transform=[Link]


test_set = [Link](root='./data', train=False, download=True, transform=[Link]
train_loader = DataLoader(train_set, batch_size=128, shuffle=True)
test_loader = DataLoader(test_set, batch_size=128)

class ConvAE([Link]):
def __init__(self, latent=16):
super().__init__()
[Link] = [Link](
nn.Conv2d(1, 16, 3, stride=2, padding=1), [Link](),
nn.Conv2d(16, 32, 3, stride=2, padding=1), [Link](),
)
[Link] = [Link](
nn.ConvTranspose2d(32, 16, 4, stride=2, padding=1), [Link](),
nn.ConvTranspose2d(16, 1, 4, stride=2, padding=1), [Link]()
)
def forward(self, x): return [Link]([Link](x))

model = ConvAE().to(device)
opt = [Link]([Link](), lr=1e-3)
crit = [Link]()

for epoch in range(5):


[Link]()
for x,_ in train_loader:
x = [Link](device)
opt.zero_grad(); out = model(x)
loss = crit(out, x); [Link](); [Link]()
print("epoch", epoch+1, "loss", float(loss))

# After training, visualize a few reconstructions (in notebook use matplotlib).


Experiment 5: Text Document Classification (LSTM)
Objective: Design and implement an LSTM classifier for textual documents.
Steps:
1. Load dataset (e.g., 20 Newsgroups/AG News).
2. Tokenize and pad sequences; build embeddings (pretrained optional).
3. Define LSTM/GRU classifier; train with cross-entropy.
4. Report accuracy/F1; show sample predictions.
Starter Code (skeleton):
# LSTM Text Classifier – PyTorch Skeleton (toy tokenization)
import torch, [Link] as nn
from [Link] import Dataset, DataLoader

texts = ["good movie", "bad plot", "awesome film", "terrible acting"]


labels = [1,0,1,0]
vocab = {"<pad>":0}
for t in texts:
for w in [Link]():
if w not in vocab: vocab[w] = len(vocab)

def encode(text, maxlen=4):


ids = [[Link](w,0) for w in [Link]()]
ids = ids[:maxlen] + [0]*(maxlen-len(ids))
return ids

X = [Link]([encode(t) for t in texts])


y = [Link](labels, dtype=[Link])

class TextDS(Dataset):
def __len__(self): return len(X)
def __getitem__(self, i): return X[i], y[i]

dl = DataLoader(TextDS(), batch_size=2, shuffle=True)

class LSTMClassifier([Link]):
def __init__(self, vocab_size, emb=32, hid=64, num_classes=2):
super().__init__()
[Link] = [Link](vocab_size, emb, padding_idx=0)
[Link] = [Link](emb, hid, batch_first=True)
[Link] = [Link](hid, num_classes)
def forward(self, x):
e = [Link](x)
o,(h,c) = [Link](e)
return [Link](h[-1])

model = LSTMClassifier(len(vocab)).train()
opt = [Link]([Link](), lr=1e-3)
crit = [Link]()
for epoch in range(20):
for xb,yb in dl:
opt.zero_grad(); out = model(xb)
loss = crit(out, yb); [Link](); [Link]()
print("Trained. Predict:", model(X).argmax(1))
Experiment 6: Time Series Forecasting (LSTM)
Objective: Design and implement an LSTM for forecasting univariate time series.
Steps:
1. Prepare sliding windows (lookback → target).
2. Define LSTM regressor; train with MSE.
3. Evaluate with RMSE/MAE; plot forecast vs actual.
4. Experiment with window size and hidden units.
Starter Code (skeleton):
# LSTM for Time Series Forecasting – PyTorch Skeleton
import torch, [Link] as nn
from [Link] import Dataset, DataLoader
import math

# Synthetic sine data


import numpy as np
t = [Link](0, 400, 0.1)
series = [Link](0.05*t) + 0.1*[Link](len(t))

def make_windows(data, lookback=50, horizon=1):


X, y = [], []
for i in range(len(data)-lookback-horizon):
[Link](data[i:i+lookback])
[Link](data[i+lookback:i+lookback+horizon])
X = [Link]([Link](X), dtype=torch.float32).unsqueeze(-1)
y = [Link]([Link](y), dtype=torch.float32)
return X, y

X, y = make_windows(series, lookback=60, horizon=1)


train_sz = int(0.8*len(X))
Xtr, Xt, ytr, yt = X[:train_sz], X[train_sz:], y[:train_sz], y[train_sz:]
dl = DataLoader(list(zip(Xtr,ytr)), batch_size=64, shuffle=True)

class LSTMReg([Link]):
def __init__(self, hid=64):
super().__init__()
[Link] = [Link](1, hid, batch_first=True)
[Link] = [Link](hid, 1)
def forward(self, x):
o,(h,c) = [Link](x)
return [Link](o[:,-1,:])

model = LSTMReg()
opt = [Link]([Link](), lr=1e-3)
crit = [Link]()

for epoch in range(10):


for xb,yb in dl:
opt.zero_grad(); out = model(xb)
loss = crit(out, [Link](1)); [Link](); [Link]()
print("epoch", epoch+1, "loss", float(loss))

with torch.no_grad():
pred = model(Xt).squeeze().numpy()
print("Test RMSE:", float([Link]((([Link]().numpy())**2).mean())))
Experiment 7: Transfer Learning for Images (Pre-trained CNN)
Objective: Enable pre-trained models (e.g., ResNet18) to classify a custom dataset.
Steps:
1. Organize dataset in ImageFolder structure: data/train/class_i, data/val/class_i.
2. Load [Link].resnet18(pretrained=True); freeze backbone or fine-tune.
3. Replace final FC to match num_classes; train with small LR.
4. Evaluate and save best model.
Starter Code (skeleton):
# Transfer Learning with ResNet18 – PyTorch Skeleton
import torch, [Link] as nn
from [Link] import DataLoader
import torchvision
import [Link] as T
from [Link] import resnet18

device = 'cuda' if [Link].is_available() else 'cpu'

train_tfms = [Link]([[Link](256), [Link](224), [Link](),


[Link](), [Link]([0.485,0.456,0.406],[0.229,0.224,0.225])])
val_tfms = [Link]([[Link](256), [Link](224),
[Link](), [Link]([0.485,0.456,0.406],[0.229,0.224,0.225])])

train_ds = [Link]("data/train", transform=train_tfms)


val_ds = [Link]("data/val", transform=val_tfms)
train_dl = DataLoader(train_ds, batch_size=32, shuffle=True, num_workers=2)
val_dl = DataLoader(val_ds, batch_size=64, shuffle=False, num_workers=2)

num_classes = len(train_ds.classes)
model = resnet18(weights="IMAGENET1K_V1").to(device) # torchvision>=0.13
# Freeze backbone (optional)
for p in [Link](): p.requires_grad = False
[Link] = [Link]([Link].in_features, num_classes).to(device)

opt = [Link]([Link](), lr=1e-3)


crit = [Link]()

for epoch in range(5):


[Link]()
for x,y in train_dl:
x,y = [Link](device), [Link](device)
opt.zero_grad(); loss = crit(model(x), y); [Link](); [Link]()
print("epoch", epoch+1, "loss", float(loss))

# Validation
[Link](); correct=total=0
with torch.no_grad():
for x,y in val_dl:
x,y = [Link](device), [Link](device)
pred = model(x).argmax(1)
correct += (pred==y).sum().item(); total += [Link]()
print("Val Acc:", correct/total)
Experiment 8: Sentiment Analysis on Reviews (LSTM or BERT)
Objective: Read a dataset of text reviews and classify as positive or negative.
Steps:
1. Option A (classic): Tokenize → Embedding → LSTM/GRU → Linear.
2. Option B (modern): Use a pretrained transformer (e.g., distilbert-base-uncased).
3. Train/evaluate with accuracy and F1; show confusion matrix.
4. Demonstrate single-sentence inference.
Starter Code (skeleton):
# Sentiment Analysis – Hugging Face Transformers Skeleton (DistilBERT)
!pip -q install transformers datasets --upgrade

from datasets import load_dataset


from transformers import AutoTokenizer, DataCollatorWithPadding, AutoModelForSequenceClassificat
import numpy as np
from [Link] import accuracy_score, f1_score, precision_score, recall_score

dataset = load_dataset("imdb")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

def tokenize_fn(batch):
return tokenizer(batch["text"], truncation=True)
tokenized = [Link](tokenize_fn, batched=True)
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels

def compute_metrics(eval_pred):
logits, labels = eval_pred
preds = [Link](logits, axis=-1)
return {
"accuracy": accuracy_score(labels, preds),
"f1": f1_score(labels, preds),
"precision": precision_score(labels, preds),
"recall": recall_score(labels, preds),
}

args = TrainingArguments(
output_dir="out",
evaluation_strategy="epoch",
per_device_train_batch_size=8,
per_device_eval_batch_size=8,
num_train_epochs=1,
fp16=True if [Link].is_available() else False,
logging_steps=50
)

trainer = Trainer(
model=model,
args=args,
train_dataset=tokenized["train"].shuffle(seed=42).select(range(4000)), # subset for speed
eval_dataset=tokenized["test"].select(range(1000)),
tokenizer=tokenizer,
data_collator=data_collator,
compute_metrics=compute_metrics
)

[Link]()
metrics = [Link]()
print(metrics)

# Inference
pred = [Link](tokenized["test"].select(range(1)))
print("Sample prediction logits:", [Link])

Common questions

Powered by AI

The primary objective of training a Word2Vec model with negative sampling in a deep learning laboratory setting is to design and implement a neural network to generate dense word embeddings from a document corpus. The main methods involve collecting and cleaning the corpus, tokenizing it, and building a vocabulary. Training pairs are created for the Skip-gram model using a center→context approach, and negative sampling is utilized to optimize training. Embedding and output layers are defined, and the model is trained to minimize loss by calculating the difference between observed and sampled negative instances. Finally, evaluation is performed using nearest neighbors and visualization techniques such as PCA or t-SNE .

In a convolutional autoencoder for image compression, the encoder reduces the input image to a lower-dimensional latent space (bottleneck), effectively compressing the data by capturing its essential features. The decoder then reconstructs the compressed image back to its original dimensions, aiming to closely match the original input. The quality of the reconstructed image is highly influenced by the dimensionality of the latent space; a smaller latent dimension can result in higher compression but potentially poorer reconstruction accuracy. Conversely, larger latent spaces may retain more detail, leading to higher fidelity reconstructions. This trade-off must be managed depending on the application's tolerance for loss of detail vs. compression efficiency .

The tokenizer function plays a critical role in preparing text data for transformer models like distilBERT by transforming raw text into a format that the model can process. It breaks down sentences into tokens that are consistent with the vocabulary used during the model's pre-training phase. The tokenizer also handles padding and truncation to ensure uniform input shapes. This detailed pre-processing increases the textual representation's accuracy within the model's embeddings, crucially impacting the model's ability to understand context and semantics in sentiment analysis. By ensuring token consistency and input format compatibility, the tokenizer facilitates making accurate predictions and effectively leveraging pre-trained model strengths .

Standardizing numeric features ensures that each feature contributes equally to the model's learning process by bringing them to a common scale, which is particularly important in gradient-based learning methods. Encoding categorical features transforms discrete categories into numeric formats that the neural network can process. Together, these preprocessing steps help improve the stability and performance of the DNN, as they prevent any single feature from dominating the learning process and improve convergence during optimization. This preparation is necessary before defining a DNN with layers such as ReLU and dropout, enabling effective model training with cross-entropy loss .

Early stopping is a technique used during the training of CNNs to enhance performance on validation data by preventing overfitting. It involves monitoring the validation loss throughout the training process and halting training once the loss ceases to decrease for a specified number of epochs. This stopping criterion indicates that further training may lead to the model capturing noise and overfitting the training data rather than improving generalization. Early stopping helps ensure the model maintains its ability to perform well on unseen data, achieving an optimal balance between underfitting and overfitting .

Using a pre-trained model like ResNet18 for transfer learning in custom image classification tasks offers several advantages, including reduced training time, as the model has already learned effective feature representations from a large benchmark dataset (e.g., ImageNet). Additionally, fine-tuning a pre-trained model often results in better generalization compared to training a model from scratch, especially for small datasets. The existing learned parameters serve as a robust initialization point, allowing the model to adapt quickly to new tasks while needing only the final layers to be retrained to match the specific number of classes in the custom dataset. This approach leverages the pre-existing feature extraction capabilities of ResNet18, which can significantly improve performance in resource-constrained scenarios .

The sequence length (lookback window) and hidden unit size are pivotal in defining the performance of an LSTM model used for time series forecasting. The sequence length determines the amount of past information the model leverages to make predictions, with longer sequences potentially capturing more relevant data patterns but also increasing the computational cost and risk of overfitting. The hidden unit size dictates the model's capacity to learn complex temporal patterns, where larger sizes can model intricate relationships but again raise the risk of overfitting and higher computational demand. Balancing these parameters is crucial; optimal settings depend on the specific dataset's characteristics and the need for accuracy versus computational efficiency .

The key design components for implementing a CNN for image classification using a dataset like CIFAR-10 include a series of convolutional layers followed by batch normalization, ReLU activations, and pooling layers to reduce spatial dimensions while capturing important patterns. The network typically culminates in fully connected (FC) layers leading to a softmax output for class probabilities. Training strategies involve using data augmentation techniques such as random cropping and horizontal flipping to increase robustness, employing optimization methods like Adam, and utilizing early stopping to prevent overfitting. Tracking training accuracy and loss metrics further guides the adjustments needed for model refinement .

Dropout layers are employed in deep neural networks, such as the MLP architecture for classification, as a form of regularization to prevent overfitting during training. By randomly setting a subset of neuron outputs to zero during forward passes, dropout layers prevent co-adaptation of neurons—forcing the network to learn more robust features that are useful in conjunction with different subsets of other neurons. This technique helps the model generalize better to new, unseen data by reducing the dependency on any single neuron or layer configuration, thus promoting redundancy and variability in learning representations .

The sliding window approach in univariate time series forecasting with LSTM models facilitates the process by allowing the model to learn temporal dependencies over a fixed-length input window. This method involves creating segments (windows) of consecutive data points from the series as input sequences, where each window's target is the value following the last point in the sequence. It effectively captures local patterns and trends, enabling the LSTM to predict future values based on learned short-term dependencies. The sliding window minimizes the assumption of linearity present in traditional forecasting methods, providing a flexible framework to capture non-linear dynamics in time series data .

You might also like