Machine Learning - II[BAI702] Module- 4

14.2 VECTOR QUANTISATION

Vector quantisation is a data compression technique based on competitive learning.

The core idea is to replace each input vector with the prototype (cluster centre) it is
assigned to. This reduces storage or transmission cost because only the index of the
prototype needs to be sent or stored, not the full vector.

Vector quantisation is closely related to noise reduction. In both cases, the input is
replaced by the cluster centre it belongs to. In noise reduction, this provides a
cleaner version of the input; in data compression, it reduces the number of data
points transmitted.

To compress data, both sender and receiver agree on a codebook of prototype

vectors. Instead of transmitting actual datapoints, the sender transmits the index of
the nearest prototype. More frequent datapoints can be assigned shorter indices to
improve efficiency. This is a standard idea in information theory, and forms the
basis of many sound and image compression methods.

The codebook cannot include every possible datapoint. When an unseen datapoint
is encountered, it is replaced by the closest prototype. This introduces loss in the
representation and is known as vector quantisation.

Prototype vectors divide the input space into Voronoi sets. Every input within a cell
is represented by the prototype at its centre. The collection of these cells forms a
Voronoi tessellation. Connecting prototype vectors whose cells share edges yields
the Delaunay triangulation, which provides an optimal arrangement for function
approximation.

Figure 14.5 shows an interpretation of prototype vectors in two dimensions. The

dots at the centre of each cell are the prototype vectors, and any datapoint that lies

Dept. of AI, VCET Puttur 1

Machine Learning - II[BAI702] Module- 4

within a cell is represented by the dot. The name for each cell is the Voronoi set of a
particular prototype. Together, they produce the Voronoi tesselation of the space. If
you connect together every pair of points that share an edge, as is shown by the
dotted lines, then you get the Delaunay triangulation, which is the optimal way to
organise the space to perform function approximation.

Selecting good prototype vectors is essential. Competitive learning algorithms are

used for this purpose. The k-means algorithm can be applied if the desired
codebook size is known. However, the Self-Organising Feature Map (SOM) offers a
more flexible and often more effective approach for determining prototype vectors.

14.3 THE SELF-ORGANISING FEATURE MAP

The Self-Organising Feature Map (SOM) is the most widely used competitive
learning algorithm. Proposed by Teuvo Kohonen (1988), it maps high-dimensional
input data onto a usually 1D or 2D grid of neurons while preserving topological
order: neighbouring neurons respond to similar inputs.

Motivation
SOM models inspired by biological sensory maps, e.g., the auditory cortex, where
neurons responding to similar stimuli are located close together.
Two important differences from earlier models:
1. The physical arrangement of neurons matters (feature mapping).
2. Neurons in the map have lateral interactions (not layered connections only).

Dept. of AI, VCET Puttur 2

Machine Learning - II[BAI702] Module- 4

Figure 14.6: SOM structure

 Inputs connect fully to a grid of neurons.

 When one neuron wins (best match), it influences its spatial neighbours.
 Only a neighbourhood around the winner updates its weights.

Topology Preservation
 Inputs close together in input space should activate neurons close together on
the map.
 Exact preservation is only possible when input space and map have the same
intrinsic dimensionality.
 Mismatches (e.g., mapping 3D or 2D to 2D grid) cause distortions.
Figure 14.7
Shows how a 1D line, 2D grid, and 3D cube distort when represented on a 2D map.
Relative ordering is not perfectly preserved due to dimensionality reduction.

Dept. of AI, VCET Puttur 3

Machine Learning - II[BAI702] Module- 4

Mexican Hat Lateral Interaction

To achieve ordered mapping, the winner should:

 Excite nearby neurons (positive interaction).

 Repel farther neurons (negative interaction).
 Ignore distant ones.
Figure 14.8:
Graph of lateral connection strength: high positive in the centre, dips negative at
medium distances, then zero — resembling a “Mexican hat”.

Dept. of AI, VCET Puttur 4

Machine Learning - II[BAI702] Module- 4

14.3.1 The SOM Algorithm

Using the full Mexican hat lateral interactions between neurons is fine, but it isn’t
essential. In Kohonen’s SOM algorithm, the weight update rule is modified instead,
so that information about neighbouring neurons is included in the learning rule,
which makes the algorithm simpler. The algorithm is a competitive learning
algorithm, so that one neuron is chosen as the winner, but when its weights are
updated, so are those of its neighbours, although to a lesser extent. Neurons that
are not within the neighbourhood are ignored, not repelled. We will now look at
the SOM algorithm before examining some of the details further.

Dept. of AI, VCET Puttur 5

Machine Learning - II[BAI702] Module- 4

14.3.2 Neighbourhood Connections

In a Self-Organising Map (SOM), the neighbourhood defines which neurons around
the winning neuron will also have their weights updated.
This neighbourhood size is an important parameter.
Need of neighbourhood:
 At the start of training, weights are random.
→ Neurons close on the map may be far apart in weight space.
→ So neighbourhoods must be large initially to establish the global ordering of
the map (ordering phase).
 After some learning, the SOM roughly reflects the data’s topology.
→ Now learning needs to fine-tune local regions.
→ Neighbourhood must become small (convergence phase).
Neighbourhood size reduction:
 The neighbourhood radius is gradually decreased each iteration.
 Learning rate η is reduced the same way—large initially, smaller later.
→ Both help the SOM shift from rough ordering to fine adjustments.
Implementation detail
 Since neighbourhood size changes over time, explicit connections are not used.
 Instead, a map-distance matrix is precomputed:
 It stores distances between every pair of neurons on the 2D grid.
 Neurons within the current neighbourhood radius are selected and
updated.

Visualization - FIGURE 14.9

 Initially: neighbourhood is large; many neurons around the winner are
updated.
 Later: neighbourhood is small; only very close neighbours are updated.

Dept. of AI, VCET Puttur 6

Machine Learning - II[BAI702] Module- 4

Neighbourhood Connections:
Creating the map-distance matrix:

There is another way to initialise the weights in the network, which is to use
Principal Components Analysis (which is described in Section 6.2) to find the two
(assuming that the map is two-dimensional) largest directions of variation in the

Dept. of AI, VCET Puttur 7

Machine Learning - II[BAI702] Module- 4

data and to initialise the weights so that they increase along these two
directions:

This means that the ordering part of the training has already been done in the
initialisation, and so the algorithm can be trained with small neighbourhood size
from the start. Obviously, this is only possible if the training of the algorithm is in
batch mode, so that you have all of the data available for training right from the
start. This should be true for the SOM anyway—it is not designed for on-line
learning. This can be a bit of a limitation, because there are many cases where we
would like to do unsupervised on-line learning. There are a couple of different
things that we can do. One is to ignore that constraint and use the SOM anyway.
This is fairly common. However, the size of the map really starts to matter, and
there is no guarantee that the SOM will converge to a solution unless batch
learning is applied. The alternative is to use one of a variety of networks that are
designed to deal with exactly this situation.

14.3.3 Self-Organisation

Self-organisation refers to the remarkable ability of the Self-Organising Map (SOM)

to form a globally ordered structure even though only local interactions between
neurons occur during learning.

The Idea

 In SOM, neurons adjust their weights based on:

 the winning neuron (best match to the input), and
 the neurons near the winner in the map (neighbourhood).
 Only these local neighbourhood updates occur.
 Despite this purely local updating, the entire map eventually becomes globally
ordered.

Dept. of AI, VCET Puttur 8

Machine Learning - II[BAI702] Module- 4

 Neurons that are far apart never influence each other directly.
 Yet, after training, the whole grid becomes organised so that:
 Neurons physically close in the map represent similar inputs.
 Neurons far apart represent very different inputs.

The system therefore “organises itself” without any external supervision or any
global control mechanism.

Example Analogy

 A flock of birds flying in formation:

 Birds do not know where the whole flock is.
 Each bird follows simple local rules (e.g., staying diagonally behind its
neighbour).
 A globally coordinated flock pattern emerges naturally.
 Similarly, in SOM, each neuron only reacts to its local neighbourhood, but the
network still forms a global topological ordering.

Summary:
 Global order emerging from local interactions.
 No external supervisor or global controller.

14.3.4 Network Dimensionality and Boundary Conditions

Network Dimensionality
 Although SOMs are usually shown as a 2D rectangular grid of neurons, the
algorithm does not require 2D specifically.
 A SOM may be 1D, 2D, or 3D, depending on what best represents the data.
 What matters is the intrinsic dimensionality of the data:
 Intrinsic dimensionality = the minimum number of dimensions needed to
represent the true structure of the data.
 Example from the text:
Even if inputs occupy a 3D room, if all lie on a 2D plane, then the intrinsic
dimensionality is 2, not 3.

Dept. of AI, VCET Puttur 9

Machine Learning - II[BAI702] Module- 4

 Real datasets often appear high-dimensional due to noise, but the true
structure may require fewer dimensions.
SOM dimensionality should ideally match this intrinsic dimensionality.
Boundary Conditions
The SOM grid has edges. Whether these edges should “exist” depends on the
problem.
Rectangular boundary (normal 2D SOM)
 The map has fixed edges.
 Suitable when the data naturally has meaningful limits (e.g., pitches from
lowest to highest — endpoints are meaningful).
Removing boundary effects
Sometimes edges are artificial and introduce distortions.
To avoid this, the grid edges can be wrapped around:
 1D map → wrap ends to form a circle.
 2D map → wrap top–bottom and left–right edges to form a torus (doughnut
shape).(Illustrated in Figures 14.10 and 14.11.)

Dept. of AI, VCET Puttur 10

Machine Learning - II[BAI702] Module- 4

Effect of using toroidal boundaries

 Eliminates “special” edge neurons.
 Often leads to better topology preservation than a rectangle.
 But distance calculation becomes more complex, because distances must allow
wrap-around.
 Implemented by imagining multiple copies of the map around the original
(Figure 14.11).
 Distance = the smallest distance between a neuron and any copy of another
neuron.
Network Size and Overfitting
 SOM size (total number of neurons) is fixed before training.
 Too small → only coarse generalisations.
 Too large → memorizes individual points (overfitting).
 Usual practice: try several sizes (e.g., 5 x 5, 10 x 10) and compare performance.

14.3.5 Examples of Using the SOM

Purpose of the Examples

Dept. of AI, VCET Puttur 11

Machine Learning - II[BAI702] Module- 4

This section demonstrates how the Self-Organising Map (SOM) behaves on:
 Random 2-D data
 Real datasets (Iris, E. coli)
It highlights topological ordering, cluster formation, and how initialisation affects
training.
Example 1: Uniform 2-D Random Data (Figure 14.12)
Dataset:
 Input vectors sampled uniformly from [−1, 1] × [−1, 1].
Behaviour:
1. Initial Map (top-left of Fig. 14.12):
 Random weights → the map is completely disordered.
2. After Training for 10 iterations (bottom-left):
 Even with random initialisation, SOM quickly orders itself.
 Neighbouring map nodes correspond to input vectors that lie close together in
the plane.
3. Using PCA initialisation (top-right):
 Weights are initialised along the first two principal components of the data
(although PCA is not very meaningful for random data).
 Gives a more ordered map from the start.
4. After only 5 iterations (bottom-right):
 PCA initialisation speeds up training, producing an ordered map faster than
random initialisation.

Dept. of AI, VCET Puttur 12

Machine Learning - II[BAI702] Module- 4

Example 2: Iris Dataset (Figure 14.13)

Dataset:
 Classic 3-class iris dataset (Setosa, Versicolor, Virginica).
 SOM size: 5 × 5 map, trained for 100 iterations.
 Important: SOM receives no class labels — it is fully unsupervised.
Result (Fig. 14.13):
 Plot shows which map node was the best-matching unit (BMU) for each test
sample.
 Different shapes indicate different true iris classes.
 Cluster formation is clearly visible:
 Samples belonging to the same class tend to map to neighbouring nodes.
 SOM has discovered structure consistent with the true class labels, even
though labels were not used in training.
Observations:

Dept. of AI, VCET Puttur 13

Machine Learning - II[BAI702] Module- 4

 The map acts like a 2-D clustering visualisation.

 Researchers sometimes use methods (e.g., LDA-based post-processing) to
colour the map regions according to class clusters.

Example 3: E. coli Protein Localization Dataset (Figure 14.14)

Dataset:
 Protein measurements from UCI repository.
 Classes represent localization sites of proteins in bacteria.
Training Result:
 Left of Fig. 14.14: Training data BMU plot
→ Clearer clusters
 Right of Fig. 14.14: Test data BMU plot
→ Clusters still visible but less distinct.
Observations:
 The problem is more complex → SOM does not separate classes as cleanly as in
iris.
 Still, topological cluster patterns emerge.
 Compared to an MLP (which gets ~50% accuracy on this dataset), SOM
performs reasonably despite having no labels.
 Boundary conditions can affect cluster appearance, because clusters may wrap
around map edges (torus effect).

Dept. of AI, VCET Puttur 14

Machine Learning - II[BAI702] Module- 4

Key Takeaways from the Examples

 SOM effectively orders even random high-dimensional data into smooth
topological maps.
 Shows strong ability for visual clustering and structure discovery.
 PCA initialisation can dramatically speed up ordering.
 Works well for simple datasets (Iris) and reasonably for complex ones (E. coli),
even without labels.
 SOM’s plots are often used to inspect data clusters visually and interpret
patterns.

Dept. of AI, VCET Puttur 15

Machine Learning - II[BAI702] Module- 4

Markov Chain Monte Carlo (MCMC) Methods

In this chapter, we study a method that completely transformed how computers
handle complicated statistical problems.
The main idea behind this method has existed since 1953, but it became truly
powerful only after computers became fast enough to use it on real-world problems.
This method—Markov Chain Monte Carlo (MCMC)—is now considered one of the
most influential algorithms ever.

What kinds of problems does MCMC solve?

Throughout the book, we have been dealing with two main problems:
1. Finding the best (optimal) solution to some objective function.
(Example: finding the minimum error or maximum likelihood.)
2. Computing the posterior distribution for a statistical learning problem.
(Example: Bayesian learning where we want p(parameters | data).)

The challenge is that the state space (all possible solutions) is usually very large,
and we don’t care about all states—we only want the important ones (the best or
the most probable).

We already studied many methods to solve such problems. MCMC is another

powerful method that helps with this.

Core idea behind MCMC

As we explore the state space (move from one possible solution to another:
 We also collect samples
 These samples are more likely to come from the most probable regions of the
space

This means MCMC automatically spends more time exploring the “important” parts
of the distribution.

To understand this idea clearly, the chapter first explains:

Dept. of AI, VCET Puttur 16

Machine Learning - II[BAI702] Module- 4

 What Monte Carlo sampling means (random sampling to estimate

distributions)
 What Markov chains are (a sequence where each step depends only on the
previous one)

These two ideas together form Markov Chain Monte Carlo.

15.1 SAMPLING
Sampling refers to generating random values from a probability distribution. We
have already used sampling in many algorithms (e.g., weight initialisation). Most
commonly, we sample from:
 Uniform distribution using [Link]()
 Gaussian distribution using [Link]()

15.1.1 Random Numbers

 Computers cannot generate truly random numbers; they generate pseudo-
random numbers, determined by mathematical formulas.
 The simplest pseudo-random generator is the Linear Congruential Generator
(LCG):

 Here, parameters a, c, m and the seed x0 determine the sequence.

 Since the output is computed only from the last value, eventually the sequence
repeats → this length is the period.
 A good random generator has a long period. Example good parameters:
 m = 232, a=1,664,525, c= 1,013,904,223
 Modern libraries (like NumPy) use a higher-quality generator called the
Mersenne Twister, based on Mersenne primes.
 Von Neumann’s quote warns that testing randomness is harder than
generating it.
Note: Random numbers produced by computers are deterministic, but good
generators approximate randomness sufficiently well.

Dept. of AI, VCET Puttur 17

Machine Learning - II[BAI702] Module- 4

15.1.2 Gaussian Random Numbers

Computers natively generate uniform random numbers (e.g., via the Mersenne
Twister). But often we need Gaussian (normal) distributed samples.
To convert uniform random numbers into Gaussian ones, the standard method used
is the Box–Muller scheme.

Goal
Generate two independent Gaussian-distributed numbers with:
 Mean = 0
 Variance = 1
Key Idea
Start with two independent uniform random numbers U1, U2 ∼ Uniform(0,1).
Transform them into Gaussian variables using a mathematical trick involving polar
coordinates.

Dept. of AI, VCET Puttur 18

Machine Learning - II[BAI702] Module- 4

Dept. of AI, VCET Puttur 19

Machine Learning - II[BAI702] Module- 4

15.2 MONTE CARLO OR BUST

If you generate independent and identically distributed samples x(i) from a high-
dimensional distribution p(x), then as the number of samples increases, the
distribution of your samples will converge to the true distribution.
In other words,
If you take enough random samples from a distribution, the samples themselves
represent the true distribution.
Mathematical Form:
If pN(x)is the empirical distribution of N samples:

where δ is the Dirac delta function, which “marks” sample locations.

Computing Expectations Using Samples:

If you want the expected value of a function f(x):

The average of the function values at sample points converges to the true expected
value.
Why Monte Carlo Sampling Is Useful?
 Samples naturally appear more frequently in regions of high probability of p(x).
 This means you automatically focus computation where it matters.
 You avoid wasting time on unlikely regions.

Dept. of AI, VCET Puttur 20

Machine Learning - II[BAI702] Module- 4

This is especially important in high-dimensional spaces where uniform sampling

becomes inefficient.
Using Samples to Find Maximum Likelihood:
Because more samples appear in high-probability regions, you can estimate the
most likely value (mode) as:

Thus Monte Carlo sampling can help locate peaks of the distribution.

Historical Note – Why “Monte Carlo”?

The name comes from the Monte Carlo casino, because the technique is based on
probability and randomness.
It originated when physicist Stanislaw Ulam was analyzing card games.
Just like drawing cards many times reveals winning probabilities, sampling many
times reveals the shape of a distribution.

Example:

 For a card game like Patience:

 Total possible arrangements of a deck = 52! ≈ 8 x 1067
 Impossible to compute exact probability analytically.
 But if you play many games (sample the state space), you can estimate the
probability of winning.

This demonstrates why sampling works for huge state spaces.

Note:

 Monte Carlo principle: Large random samples approximate the true

probability distribution.
 Expectations and maxima can be computed from samples.
 Sampling focuses on high-probability areas automatically.
 Inspired by gambling and card-game probability.
 Foundation for more advanced methods like rejection sampling and MCMC.

Dept. of AI, VCET Puttur 21

Machine Learning - II[BAI702] Module- 4

15.3 THE PROPOSAL DISTRIBUTION

The problem addressed here is: what if the target distribution p(x) is difficult to
sample from directly? The solution is to use a simpler distribution q(x), called the
proposal distribution, from which sampling is easy. Then we “correct” the samples
using rejection.

Main Idea:

We assume that although we cannot sample from p(x)p(x)p(x) directly, we can

evaluate a related unnormalized distribution:

Here, Zp is an unknown normalisation constant. We only need , not the exact

value of Zp.

Condition for Using a Proposal Distribution:

We choose a constant M such that:

This means the “scaled-up” proposal distribution covers the target distribution
everywhere. This concept is shown in Figure 15.2, where the curve Mq(x)forms an
envelope around p(x).

Dept. of AI, VCET Puttur 22

Machine Learning - II[BAI702] Module- 4

Rejection Sampling Procedure:

Once we have q(x)q(x)q(x) and M:

Step 1 — Draw a sample

Pick

Step 2 — Generate a uniform random number

Pick

Step 3 — Accept or Reject

Accept x* if

Otherwise, reject x*and try again.

This works because of the envelope principle: all points under the curve Mq(x)are
equally likely, and we keep only the points lying under p(x).

Example - Figure 15.3:

Figure 15.3 shows how rejection sampling is used to sample a mixture of two
Gaussians, using a uniform distribution as the proposal (shown as a dotted line).
Results:

Dept. of AI, VCET Puttur 23

Machine Learning - II[BAI702] Module- 4

 With M=0.8, around 50% of samples are rejected.

 With M=2, around 85% of samples are rejected.

This shows that if M is chosen poorly, the rejection rate can become very high,
making the method inefficient.

Challenge — The Curse of Dimensionality:

As dimension increases, the probability mass of distributions becomes more spread

out:

 It becomes harder to find a good envelope

 The rejection rate becomes extremely high.

Thus, rejection sampling becomes inefficient in high dimensions. This motivates

more advanced methods (importance sampling, MCMC).

Suppose that we want to compute the expectation of a function f(x) for a

continuous random variable x distributed according to unknown distribution p(x).

Dept. of AI, VCET Puttur 24

Machine Learning - II[BAI702] Module- 4

Starting from the expression of the expectation that we wrote out earlier, we can
introduce another distribution q(x):

An implementation of this in Python is shown next, and the results of using sampling
importance-resampling on the example in Figure 15.3 are given in Figure 15.4. Note
that this method does not reject any samples, but it does involve two separate
sampling steps and a relatively expensive loop. Like the other algorithms we have
seen, it is sensitive to the quality of the match between the proposal distribution q(x)
and the actual distribution p(x).

Dept. of AI, VCET Puttur 25

Machine Learning - II[BAI702] Module- 4

Dept. of AI, VCET Puttur 26

Machine Learning - II[BAI702] Module- 4

In Section 16.4.2 we will see a method that uses sampling-importance-resampling

in an on-line application, known as a particle filter or sequential Monte Carlo
method. However, we will first turn our attention to how we can find out more
about the sample space. The basic idea is to keep track of the sequence of samples
and modify the proposal distribution to take advantage of this, for which we will
have to use some more complicated machinery.

15.4 MARKOV CHAIN MONTE CARLO

15.4.1 Markov Chains

In probabilistic terms a chain is a sequence of possible states, where the probability

of being in state s at time t is a function of the previous states. A Markov chain is a
chain with the Markov property, i.e., the probability at time t depends only on the
state at t − [Link] set of possible states are linked together by transition
probabilities that say how likely it is that you move from the current state to each
of the others, and they are generally written as a matrix [Link] might be constant,
or functions of some other variables, but here we will assume that they are
constant.

Given a chain, we can perform a random walk on the chain by choosing a

start state and randomly choosing each successive state according to the transition
probabilities. The link to sampling that we need is that if the transition
probabilities reflect the distribution that we wish to sample from, then a random
walk will explore that distribution. One problem with this is that random walks are
very inefficient at exploring space, since they move back towards the start as often
as they move away, which means the distance they move from the start scales as √t,

Dept. of AI, VCET Puttur 27

Machine Learning - II[BAI702] Module- 4

where t is the number of samples. We therefore want to explore more efficiently

than just using a random walk.

We do this by setting up our Markov chain so that it reflects the distribution

we wish to sample from, and we want the distribution p(x(i)) to converge to the
actual distribution p(x) no matter what state we start from. Since we can start
from any state, this tells us that every state is reachable from every other state,
which means that the chain is irreducible so that the transition matrix can’t be cut
up into smaller matrices. The chain also has to be ergodic, which means that we
will revisit every state, so that the probability of visiting any particular state in the
future never goes to zero, but is not periodic, which means that we can visit at any
time, not just every k iterations for some constant k.

We also want the distribution p(x) to be invariant to the Markov chain,

which means that the transition probabilities don’t change the distribution:

Finding the transition probabilities to make this true requires that we can
move backwards and forwards along the chain with equal probability, so that the
chain is reversible. This says that the probability of being in an unlikely state s`
(sampling datapoint x`), but heading for a likely state s, so that:

This is known as the detailed balance condition and the fact that it leaves the
distribution p(x) alone is fairly obvious with a little calculation. If the chain satisfies
the detailed balance condition, then it must be ergodic, since ∑y T(x, y) = 1, since
you must have come from some state, and so:

Dept. of AI, VCET Puttur 28

Machine Learning - II[BAI702] Module- 4

which means that p(x) must be an invariant distribution of T. So if we can work out
how to construct a Markov chain with detailed balance we can sample from it in
order to sample from our distribution. This is known as Markov Chain Monte Carlo
(MCMC) sampling, and the most popular algorithm that is used for MCMC is the
Metropolis–Hastings algorithm after the two people who were directly involved in
its creation.

15.4.2 The Metropolis–Hastings Algorithm

Dept. of AI, VCET Puttur 29

Machine Learning - II[BAI702] Module- 4

Dept. of AI, VCET Puttur 30

Machine Learning - II[BAI702] Module- 4

15.4.3 Simulated Annealing (Again)

Dept. of AI, VCET Puttur 31

Machine Learning - II[BAI702] Module- 4

15.4.4 Gibbs Sampling

Dept. of AI, VCET Puttur 32

Machine Learning - II[BAI702] Module- 4

Dept. of AI, VCET Puttur 33

Machine Learning - II[BAI702] Module- 4

Question Bank

Chapter 1: Vector Quantisation & Self-Organising Maps

L2 – Understanding Level

1. Explain how Vector Quantisation relates to clustering. How does it differ from k-
means?
2. Describe the basic architecture of a Self-Organising Feature Map (SOM). What
are its two layers and their roles?
3. Explain the competitive learning rule used in SOM. Why is the “winner-takes-all”
neuron important?
4. What is a neighbourhood function in SOM? Explain its purpose with an example.
5. Describe the concept of Self-Organisation in SOM. How does the map evolve
during training?
6. Explain the effect of neighbourhood radius on SOM learning. Why should it
shrink over time?
7. Describe the role of lattice structures (rectangular/hexagonal) in SOM.
8. Explain the importance of network dimensionality in SOM. When do we use 1D vs
2D maps?

L3 – Application/Analysis Level

1. Given a dataset of handwritten digits, explain how SOM can be used for
visualising high-dimensional data.
2. Suppose your SOM is not forming clear clusters. Analyse possible reasons and
suggest corrective steps.
3. Compare the behaviour of SOM with and without neighbourhood connections.
What difference will you observe in the final map?
4. Analyse how boundary conditions (toroidal vs non-toroidal) influence the learnt
topology of SOM.
5. Explain with an example how SOM preserves topological ordering compared to
k-means.

Dept. of AI, VCET Puttur 34

Machine Learning - II[BAI702] Module- 4

6. A SOM is trained on RGB colour data. Interpret how the SOM grid visualises
smooth colour transitions.

Chapter 2 - MCMC Methods

L2 – Understanding Level

1. Explain why computers cannot generate true random numbers. What is meant
by pseudo-random?
2. Describe the Box–Muller method for generating Gaussian random numbers. Why
are two uniform samples needed?
3. Explain the Monte Carlo principle. Why do sample averages converge to true
expectations?
4. What is a proposal distribution in rejection sampling? Explain the envelope
principle.
5. Describe the limitations of rejection sampling in high-dimensional spaces.

L3 – Application/Analysis Level

1. Given a mixture of two Gaussians, explain how importance sampling can

approximate expectations effectively.
2. Analyse why choosing a poor proposal distribution q(x) drastically reduces
MCMC efficiency.
3. Explain the detailed balance condition in Markov Chain Monte Carlo. Why is it
necessary?
4. Compare random walk sampling and MCMC sampling. Which explores the space
more efficiently and why?
5. Given a posterior distribution that is hard to sample from, explain how
Metropolis–Hastings can approximate it.
6. Analyse the behaviour of the acceptance ratio in Metropolis–Hastings when the
proposal distribution is too narrow or too wide.

Dept. of AI, VCET Puttur 35