Q.1 Explain Support Vector Machines.

Explain the working, types and Python code

 Support Vector Machine (SVM) is a supervised machine learning algorithm used for
classification and regression tasks.
 It works by finding the best separating hyperplane that divides data points of different
classes with the maximum possible margin.
 The data points that lie closest to this hyperplane are called support vectors, and they
determine the position and orientation of the hyperplane.
 SVM tries to create a clear boundary between different categories of data so that new data
points can be classified accurately.
 Very useful for BINARY CLASSIFICATION
 Hyperplane: The decision boundary that separates classes in feature space. For linear SVM, it
is w x+ b = 0.
 Support Vectors: The closest data points to the hyperplane. They define the margin and the
final boundary.
 Margin: The distance between the hyperplane and the support vectors. SVM tries to
MAXIMIZE this for better generalization
 Kernel: A function that maps data to a higherdimensional space so non-linear data becomes
linearly separable
 Hard Margin: Assumes perfect separation (no misclassification). Works only when data is
clean and separable; noise sensitive.
 Soft Margin: Allows some errors using slack variables; balances wide margin vs
misclassification when data overlaps.
 C (Regularization): Controls the trade-off. High C = strict penalty (fewer training errors, risk of
overfit). Low C = wider margin, more tolerance.
 Types of SVM
o Linear SVM:
 Finds a straight boundary (HYPERPLANE) that separates classes.
 Works well when data is roughly linearly separable
 Fast to train and simple to understand.
 Needs feature scaling
 Best for: Text classification, high-dimensional data where a straight split is
good.

o Non-Linear SVM:
 Uses the KERNEL TRICK to handle curvy boundaries.
 Maps data to a higher dimension where a straight hyperplane can separate
it.
 Common kernels: RBF (Gaussian), Polynomial, Sigmoid.
 Best for: data with complex patterns where а straight line cannot separate
classes.
  Needs careful tuning (use cross-validation) to avoid overfitting.

 Working of SVM
o Input Data:
 Plots the training data in n-dimensional space (n = number of features).
o Hyperplane
 SVM finds the hyperplane that best separates the classes.
o Support Vectors
 The closest data points to the hyperplane that help define it.
o Maximum Margin
 SVM selects the hyperplane with the maximum margin for better accuracy.
o Non-linear Cases
 For complex data, kernel functions (like RBF or polynomial) map data to a
higher dimension to make it separable.
 Python Code

# (i) Import suitable modules

import [Link] as plt

from sklearn.model_selection import train_test_split

from [Link] import SVC

from [Link] import make_classification

# (ii) Create dataset

X, y = make_classification(

n_samples=200, # number of samples

n_features=2, # only 2 features for easy visualization

random_state=42

# (iii) Visualize dataset

[Link](X[:, 0], X[:, 1], c=y, cmap='coolwarm')

[Link]('Dataset Visualization')

[Link]('Feature 1')

[Link]('Feature 2')

[Link]()

# (iv) Splitting data into training and testing set

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.3, random_state=42)

# (v) Support Vector Machine classifier implementation

svm_classifier = SVC(kernel='linear') # Using linear kernel

svm_classifier.fit(X_train, y_train)

# (vi) Prediction for Support Vector Machine classifier

y_pred = svm_classifier.predict(X_test)

 Advantages of SVM
o SVM classifiers offer good accuracy and perform faster predictions compared to the
Naive Bayes algorithm.
o They also use less memory because they use a subset of training points in the
decision phase
o SVM works well with a clear margin of separation and with high-dimensional space.
 Disadvantages of SVM
o SVM is not suitable for large datasets because of its high training time, and it also
takes more time in training compared to Naive Bayes
o It works poorly with overlapping classes and is also sensitive to the type of kernel
used.

Q.2 Explain K-Mediod Algorithm, working and implementation

 It is a clustering technique similar to K-Means, but instead of using the mean of data points,
it uses an actual data point called a medoid as the center of each cluster.
 A medoid is the most centrally located point within a cluster.
 It is the point whose average dissimilarity (or distance) from all other points in that cluster is
the minimum.
 Cost Function:
o The total cost (or total dissimilarity) for the K-Medoids algorithm is the sum of all
distances between each point and its respective medoid, across all clusters
 Working of K-Mediod Clustering Technique
o Initialization
 Choose k random data points from the dataset as initial medoids.
 Each selected medoid represents one cluster center initially
o Assignment of Points
 Assign every data point to the nearest mediod based on a distance measure
  This forms k clusters where each point belongs to the cluster with the
nearest medoid.
o Calculate the Total Cost
 Compute the total cost of the clustering
o Update Medoids
 For each cluster, check if replacing the current medoid with any other data
point in that cluster reduces the total cost.
 If a better medoid is found, replace it.
o Repeat Until Convergence
 Repeat Steps 2-4 until the medoids stop changing or the cost cannot be
minimized further.
 The algorithm then converges to the final set of medoids and clusters.
 Python Code to implement
# (i) Import libraries
import numpy as np
from sklearn_extra.cluster import KMedoids

# (ii) Prepare sample data

X = [Link]([[2, 6],
[3, 4],
[3, 8],
[4, 7],
[6, 2],
[7, 3],
[7, 4],
[8, 5]])

# (iii) Choose number of clusters (K)

k=2

# (iv) Fit data to K-Medoids algorithm

kmedoids = KMedoids(n_clusters=k, random_state=0)
[Link](X)

# (v) Get cluster centers (medoids)

centers = kmedoids.cluster_centers_
print("Cluster Centers (Medoids):")
print(centers)

# (vi) Get labels

labels = kmedoids.labels_
print("Cluster Labels:")
print(labels)

Q.3 Difference between K-Means Clustering and K-Mediod Algorithm

 Basis of
K-Means Clustering K-Medoids Algorithm
Comparison
Cluster Centroid (mean of data points, Medoid (an actual data point from the
representative not necessarily a real point) dataset)
Sensitivity to Highly sensitive to outliers as Less sensitive to outliers since
outliers mean is affected medoid is a real data point
Can use any distance measure
Distance metric used Mainly Euclidean distance
(Euclidean, Manhattan, Cosine, etc.)
Suitable for numerical and Suitable for numerical, categorical,
Type of data
continuous data and mixed data
Computational
Lower, faster for large datasets Higher, slower compared to K-Means
complexity
Robustness and Less robust due to shifting
More robust and stable clustering
stability centroids

Q.4 Explain Dimensionality Reduction

 Dimensionality reduction is the technique used to reduce the number of input variables in a
dataset while retaining as much relevant information as possible
 Dimensionality reduction helps to improve model performance, speed and visualization
 Types of Dimensionality Reduction are as follows :-
o Feature Extraction: Creating new features from existing ones using mathematical
transformations. Examples: Principal Component Analysis (PCА), Linear Discriminant
Analysis (LDA), t-SNE, Autoencoders.
o Feature Extraction: Creating new features from existing ones using mathematical
transformations. Examples: Principal Component Analysis (PCА), Linear Discriminant
Analysis (LDA), t-SNE, Autoencoders.
 Need for Dimensionality reduction are as follows:-
o Avoids the Curse of Dimensionality
o Reduces Overfitting
o Improves Computational Efficiency
o Removes Redundant & Irrelevant Features
o Enables Visualization

Q.5 Explain Subset Selection

 Subset Selection is a feature selection technique where the goal is to identify and retain the
most relevant subset of features from the original set.
 It helps eliminate redundant, irrelevant, or highly correlated features.
 Steps in Subset Selection:
o Start with all available features.
o Evaluate the performance of the model using different subsets of features.
o Select the subset that gives the best performance according to an evaluation
criterion (For example, accuracy, R², AIC, or BIC).

Q.6 Explain Forward Selection

  Begin with no features.
 Add one feature at a time that improves the model performance the most.
 Stop when no significant improvement occurs.

Q.7 Explain Backward Elimination

 Start with all features.

 Remove one feature at a time that has the least impact on model performance.
 Continue until performance drops or a stopping criterion is met.

Q.8 Explain Stepwise Selection

 A combination of forward and backward selection.

 Features are added or removed at each step based on their contribution to model
performance.

Q.9 Explain advantages and Disadvantages of Subset Selection

 Advantages of Subset selection

o Reduces model complexity.
o Helps avoid overfitting.
o Increases interpretability of models.
o Improves training speed
 Disadvantages of Subset Selection
o Might lose some minor but important information.
o Computationally expensive for large datasets.
o proper evaluation metrics to avoid bias.

Q.10 Explain PCA (Principal Component Analysis)

 Principal Component Analysis (PCA) is a widely used unsupervised dimensionality reduction

technique in machine learning that transforms high-dimensional data into a lower-
dimensional form while preserving as much variability as possible.
 It helps uncover patterns and relationships among variables, making it valuable for
understanding complex datasets.
 PCA is primarily used to:
o Identify hidden patterns and correlations between variables,
o Reduce noise and redundancy in data,
o Improve visualization of high-dimensional datasets,
o Enhance the efficiency of machine learning algorithms.
 Working of PCA
o PCA identifies a new set of variables known as Principal Components (PCs) - these
are linear combinations of the original features that capture the maximum variance
in the data.
o The first principal component (PC1) captures the largest amount of variance (most
significant pattern).
o The second principal component (PC2) captures the next highest variance,
orthogonal to the first, and so on.
o By keeping only the top few principal components, PCA effectively reduces the
dimensionality while retaining most of the important information.
Q.11 Explain Association Rule Learning

1. Association Rule Learning (ARL) is an unsupervised machine learning technique used to identify
relationships and dependencies among variables in large datasets.

2. The main goal of ARL is to discover interesting patterns, relationships, or associations among items
in transactional or relational databases.

3. Advantages of Association Rule Learning:

 Identifies hidden patterns in large datasets.

 Supports decision-making and business intelligence.
 Easy to understand and interpret results.
 Useful in multiple domains like retail, healthcare, and web analytics.

4. Disadvantages of Association Rule Learning

 May produce too many rules, many of which are trivial or redundant.
 Computationally expensive for large or dense datasets.
 Requires careful selection of support and confidence thresholds to get meaningful results.

5. Types of Association Rule Learning Algorithms:

1. Apriori Algorithm:

 The Apriori Algorithm uses a breadth-first search and a hash tree structure to identify
frequent itemsets efficiently.
 It works on the principle that all subsets of a frequent itemset must also be frequent.

2. Eclat Algorithm

 Eclat stands for Equivalence Class Transformation.

 It uses a depth-first search (DFS) approach to discover frequent itemsets.
 Instead of scanning the entire dataset multiple times (as in Apriori), it works on transaction
ID (TID) sets, which makes it faster and more memory-efficient.

3. FP-Growth Algorithm

 FP-Growth (Frequent Pattern Growth) is an improved version of the Apriori algorithm.

 It represents the dataset using a tree structure (FP-tree) instead of generating candidate sets.
 The algorithm recursively extracts the most frequent patterns directly from the FP-tree.
 It is faster and more scalable for large datasets.

Q.12 Explain Apriori Algorithm

 Apriori is an algorithm designed to extract frequent itemsets from transactional databases

and generate association rules.
 It is based on the principle that if an itemset is frequent, all its subsets must also be frequent.
 A dataset for Apriori typically consists of transactions, where each transaction is a collection
of items purchased together.
 Step-by-Step Process:
o Generating candidate itemsets: The algorithm starts by identifying individual items
and counting their occurrences to determine frequent items.
 o Pruning based on minimum support: Itemsets that appear less than the minimum
support threshold are removed.
o Generating frequent itemsets: The algorithm generates larger itemsets by
combining frequent smaller itemsets, iterating until no more frequent itemsets can
be formed.
o Deriving association rules: It extracts rules based on confidence and lift values to
determine meaningful relationships.
 Support: The frequency with which an item appears in the dataset. It is calculated as:

 Confidence: The likelihood that item B is purchased when item A is purchased, given by:

 Lift: The strength of a rule, measuring how much more likely item B is bought when item A is
bought compared to when bought independently:

 A lift value greater than 1 suggests a strong positive association between items.

Q.13 Explain Eclat Algorithm

 The Eclat Algorithm (short for Equivalence Class Clustering and bottom-up Lattice Traversal)
is a frequent itemset mining algorithm used in Association Rule Learning (ARL).
 It is an improvement over the Apriori Algorithm, designed to be faster and more memory-
efficient by using a depth-first search approach instead of the breadth-first search used by
Apriori.
 The main purpose of Eclat is to find frequent itemsets in transactional datasets - groups of
items that often appear together.
 The core concept behind Eclat is using the TID (Transaction ID) list approach. Instead of
scanning the entire dataset multiple times (like Apriori), Eclat keeps track of the transaction
IDs where each item appears. The intersection of these TID lists helps find how often items
occur together.
 Working Steps of the Eclat Algorithm:
o Input the Transaction Dataset
 Each transaction in the dataset contains a list of items purchased together.
 o Create TID Sets for Each Item
 List all items and record the transaction IDS (TIDS) where each appears.

o Generate Frequent Itemsets

 To find the frequency of combinations:
 Take intersections of TID sets.
 The size of the intersection tells how many times those items appear
together.
 Example:

 If minimum support = 2, then these pairs are frequent.

o Recursive Depth-First Search
 Start from single items.
 Combine items to form larger itemsets.
 Continue until no more frequent itemsets can be found.
o Generate Association Rules
 Once frequent itemsets are found, use support, confidence, and lift
measures to derive rules such as:

 Advantages of Eclat Algorithm

o Faster than Apriori: Uses set intersections instead of multiple database scans.
o Less Memory Usage: Stores transaction IDs (TID sets) instead of entire transaction
data.
o Efficient for Dense Datasets: Performs well when many items frequently occur
together.
o Depth-First Search: Reduces redundant computations and speeds up frequent
itemset discovery.
  Disadvantages:
o Memory-Intensive for Sparse Data: If many items appear infrequently, storing TID
sets can become inefficient.
o Not Suitable for Extremely Large Datasets: Intersection operations may be
computationally heavy for very high-dimensional data.
o Requires Preprocessing: Data must transformed into TID format before processing.

Q.14 What is Generative Models

Generative Models are a class of machine learning models that learn to understand the underlying
patterns and distributions in data so they can generate new, similar data samples.

Q.15 Explain Generative Adversarial Networks (GANs)

 GAN are one of the most powerful and widely used generative models in deep learning
 GANs consist of two neural networks — a Generator and a Discriminator - that compete
against each other in a game-like setup, improving through mutual feedback.
 This setup is inspired by a two-player minimax game, where each player tries to outsmart the
other.
 Architecture of GAN
o Generator (G):
 The Generator takes random noise as input (usually a vector of random
numbers) and generates synthetic data (For example, fake images).
 Its goal is to fool the Discriminator into thinking the fake data is real.
 Over time, it learns the true data distribution.
o Discriminator (D):
 The Discriminator is a binary classifier that takes both real and fake data as
input and tries to distinguish between them.
 It outputs a probability:
 1 (real) → if the sample comes from the real dataset
 0 (fake) → if the sample is generated by the Generator
 Working Principle of GAN
o The Generator creates fake samples using random noise.
o The Discriminator evaluates both real and fake samples and provides feedback.
o The Generator updates its parameters to produce more realistic data that can fool
the Discriminator.
o The Discriminator updates its parameters to better distinguish between real and fake
samples.
o This process continues iteratively until the Generator produces data that is almost
indistinguishable from real data.
 Training Process
  Туpes of GANS
o DCGAN (Deep Convolutional GAN): Uses convolutional layers — ideal for image
generation.
o CGAN (Conditional GAN): Generates data based on conditions like class labels (For
example, "generate a cat image").
o CycleGAN: Used for image-to-image translation (For example, converting a photo
into a painting style).
o Pix2Pix: Converts input images into target outputs (For example, sketch → colored
image).
o StyleGAN: Creates high-quality human-like faces with fine-grained control over style
and features.
 Advantages of GANs
o Produces highly realistic data samples.
o Learns complex data distributions without explicit supervision.
o Useful when real data is limited or costly to collect.
 Challenges of GANs
o Training Instability: Generator and Discriminator must remain balanced.
o Mode Collapse: Generator produces limited types of outputs.
o Sensitive to Hyperparameters: Requires careful tuning.
o Ethical Concerns: Potential misuse in fake media or misinformation
 Application of GANs

Q.16 Explain Variational AutoEncoders (VAE)

 Variational Autoencoder (VAE) is a type of generative model and neural network architecture
that learns to represent complex data (like images or text) in a compressed latent space and
can generate new, similar data from that space.
 VAE can Compress input data (encoding), Reconstruct or generate new data, Sample new,
realistic data by learning the underlying probability distribution of the dataset.
 A Variational Autoencoder (VAE) is a generative neural network model that consists of three
main parts:
o Encoder, Latent Space, and Decoder.
 It learns the probability distribution of data and generates new data samples similar to the
training data.
 Encoder Network
o Takes input data x (e.g., image, text, features).
o Converts it into two vectors:
o Mean (μ)
o Standard deviation (σ)
o These represent a probability distribution of latent variables.
o Mathematically:-

 Latent Space (z)

o A low-dimensional hidden space where data is represented in compressed form.
o A random sample z is generated using:

o where ϵ is random noise from normal distribution.

o This step is called reparameterization trick.
 Decoder Network
o Takes latent vector z as input.
o Reconstructs the original data x ' from z .
o Decoder(z) → x’
 Loss Function:
o The VAE Loss Function has two parts:
L = Reconstruction Loss + KL Divergence Loss
o Reconstruction Loss:
 Measures how well the output matches the original input.
o KL Divergence Loss:
 Ensures that the learned latent space follows a normal distribution.
 Helps regularize the network to make sampling meaningful
 Intuitive Explanation:
o Think of the Encoder as a "compressor" that encodes each input into a cloud (a
distribution) in the latent space, not just a single point.
o The Decoder then learns how to "draw samples" from this cloud to recreate data
similar to the input.
o This enables the VAE not just to reconstruct input data but also to generate new,
unseen samples by sampling from the latent space.
 Advantages
o Smooth Latent Space: Latent variables are continuous, allowing smooth interpolation
between points.
 o Generative Capability: Can generate new and meaningful data samples.
o Efficient Representation Learning: Learns a compact, meaningful latent space
representation of the data.
 Limitations
o Blurry Outputs: Generated images can be less sharp compared to GAN outputs.
o Complex Training: Balancing reconstruction and KL losses is sensitive.
o Limited Realism: VAEs generate more "averagelooking" samples compared to GANs.

Q.17 Compare VAE and GAN