Summary of the "Feature Extractor" Architecture

Now you can tell the full story of the first half of your model:

1. Input (): The raw image.

2. Conv1 (, 32 Filters):
o Action: Finds basic edges.
o Math: Downsamples by 2 (Stride 2). Uses 32 filters because that's a sufficient
"alphabet" for edges.

Why 32? Think of these filters like the Alphabet.

 In English, you only need 26 letters to write any word in the universe (from "Cat" to
"Supercalifragilistic...").
 You don't need 1,000 letters. That would be confusing and unnecessary.
 Similarly, 32 filters are enough to cover all the basic edge types (rotations and curves)
needed to build any object.

The "Cost" Reason:

 Conv1 works on the Biggest Image (). This is where the image is heaviest (most pixels).
 If you put 256 filters here, the math would be massive (). Your computer would freeze.
 We use a small number (32) at the start to keep the "heavy lifting" manageable.

2. Why does it shrink to ? (The "Stride" Logic)

You asked: "Size: Image shrinks to 64x64. Why?"

This happens because of a setting called Stride.

When the "Flashlight" (Filter) slides over the image, it has a choice:

1. Stride 1 (Baby Steps): Move 1 pixel at a time.

o Result: You keep the same size ().
o Problem: It's slow and you have too much data.
2. Stride 2 (Giant Steps): Move 2 pixels at a time (Jump!).
o Result: You cut the size exactly in half ().

Why do we do this?

 Reason 1: Throw away redundancy. Neighboring pixels are usually almost identical (a
black pixel is usually next to another black pixel). We don't need to measure every single
millimeter. Jumping allows us to summarize the area.
  Reason 2: Widen the View. By stepping back (shrinking the map), the next layer can see
a "bigger picture."
o At 128x128, you see a line.
o At 64x64, you see that the line is part of a square.

3. Conv2 (, 256 Filters):

o Action: Finds complex shapes and pre-bakes the data for capsules.
o Math (Size): Downsamples by 2 again to reach the "Goldilocks" grid size ().
o Math (Filters): Uses exactly 256 filters because we need to build 32 capsules of
length 8 ().

The goal of Conv2 is to provide the raw material to build the Primary Capsules.

 Requirement 1: We want 32 distinct types of capsules (32 different "viewpoints" or

"feature types" at every pixel).
 Requirement 2: We want each capsule to be a Vector of Length 8.

So, how many raw numbers (filters) do we need to output?

We choose 32 types to ensure we have a complete vocabulary of basic shapes. We want to be

ready for anything that might appear in that pixel.

At every single spot (10,10), we have a team of 32 specialists:

 Capsule Type 1: Specializes in Vertical Edges ().

 Capsule Type 2: Specializes in Horizontal Edges ().
 Capsule Type 3: Specializes in Diagonal Slashes ().
 Capsule Type 4: Specializes in Curves/Arcs ().
 Capsule Type 5: Specializes in Corners ().
 Capsule Type 6: Specializes in Blobs/Dots ().
 ...
 Capsule Type 32: Specializes in Textures/Fuzziness.

The Goal: By having 32 different "detectors" running simultaneously at the same spot, we
guarantee that no matter what shape the Partial Discharge takes (a weird spark, a fuzzy
cloud, a sharp spike), at least one of those 32 capsules will recognize it and activate.

Why did the architects (Hinton et al.) choose 8? Why not 4? Why not 16?

The number 8 represents the "Degrees of Freedom" (or complexity) of the shapes you are
trying to detect.
Think of the Length 8 Vector as a Form to fill out describing the object.

1. The Logical Reason: "The List of Properties"

If you want to describe a simple visual shape (like a curve, a line, or a Partial Discharge spark)
completely, how many questions do you need to ask?

Let's count the necessary physical properties of a visual object:

1. X-Position: Where is it horizontally? (Shift left/right)

2. Y-Position: Where is it vertically? (Shift up/down)
3. Orientation: What is the angle? (Rotated 45°?)
4. Scale/Size: How big is it? (Tiny spark vs. Big spark)
5. Deformation: Is it stretched or squashed? (Aspect Ratio)
6. Thickness: Is the line bold or thin? (Stroke width)
7. Intensity: Is it bright white or faint gray?
8. Skew: Is it slanted?

That’s 8 items.

If you chose a Length of 8, you are giving the neural network exactly enough "slots" in memory
to store all these critical details.

2. Why not other numbers? (The Trade-off)

Why not Length 2? (Under-fitting / "Blindness")

If you only gave the capsule a vector of length 2:

 Slot 1: X-Position.
 Slot 2: Y-Position.
 The Problem: The network has run out of space! It sees a Rotated object, but it has
nowhere to write down "Rotation."
 Result: It treats a vertical line and a horizontal line as the exact same thing (because they
are in the same position). It becomes "blind" to orientation.

Why not Length 100? (Over-fitting / "Hallucination")

If you gave the capsule a vector of length 100:

 Slots 1-8: Used for the real properties (Size, Angle, etc.).
 Slots 9-100: Empty space.
 The Problem: Neural networks hate empty space. It will start using those empty slots to
memorize Noise in your training data (e.g., "There is a speck of dust at pixel 5").
 Result: Your model becomes massive, slow, and "Overfits" (it memorizes the training
data but fails on real test data).
3. The Scientific Standard (Hinton's Choice)

In the original Capsule Network paper (Dynamic Routing Between Capsules, 2017), Geoffrey
Hinton used Length 8 for the Primary Capsules on the MNIST dataset.

 MNIST: Handwritten digits (0-9). Simple shapes (loops, lines).

 Your Thesis (Partial Discharge): PD signals (Corona, Surface, Void). Simple shapes
(spikes, clouds, blobs).

The Justification: Since the visual complexity of Partial Discharge patterns is roughly similar to
handwritten digits (they are simple 2D geometric shapes, not complex 3D faces), Length 8 is the
scientifically established "sufficient capacity" to capture the pose without overfitting.

NEXT: PRIMARY Capsule

Primary Capsule Layer: Mathematical and
Logical Explanation
This document details the complete, end-to-end explanation of the Primary Capsule Layer in the
RS-DACN architecture. It covers the transformation from 'Raw Features' (Conv2) to 'Intelligent
Entities' (Capsules).

Step 1: The Input (Receiving the "Raw Evidence")

We start immediately after the second Convolution layer (Conv2) finishes its job.

The Input Data: A 3D block of data coming from Conv2. The Dimensions: 32 × 32 × 256

 32 × 32: The spatial grid (The map).

 256: The number of filters (The raw feature detectors).

The Logic: At any specific pixel location (i, j) on the grid, we have a stack of 256 scalar
numbers. These are just independent 'loudness' scores from different filters. They are not yet
grouped.

Step 2: Reshaping (The "Grouping" Logic)

The network needs to convert these 'dumb scalars' into 'smart vectors'.

The Action: We logically slice the 256 depth channels into 32 groups, where each group
contains 8 numbers.
The Math Calculation:

256 filters ÷ 8 (vector length) = 32 capsules

The New Dimensions: The data shape changes from [32, 32, 256] to [32, 32, 32, 8].

What just happened? At pixel (i, j), we no longer see a pile of 256 numbers. We now see 32
Distinct Vectors, each having 8 slots:

 Capsule 1 (Vector 1): Contains numbers [x1 ... x8]

 Capsule 2 (Vector 2): Contains numbers [x9 ... x16]

Step 3: The Squashing Function (The "Activation" Math)

This is the critical calculation. The vectors from Step 2 are Raw Vectors (). Their numbers are
wild (e.g., -50, +200). We process every single vector through the Squashing Function.

Sub-step 3A: Calculate the Magnitude (Strength)

First, we find out how 'long' the raw vector is using the Pythagorean Theorem.

Logic: This combines all 8 properties into one single 'Strength Score'.

Sub-step 3B: Calculate the Unit Vector (Pure Direction)

We strip away the strength to find the pure orientation (Pose).

Logic: By dividing the vector by its own length, we force the new length to be exactly 1. This
preserves the ratios between the numbers (which encodes the rotation/thickness) but removes the
intensity.

Sub-step 3C: Calculate the Probability (The "Dimmer Switch")

We calculate a scaling factor based on the original raw length.

Logic: This is a non-linear function. If raw length is small (e.g., 0.1), Scale becomes ≈ 0 (Kill the
noise). If raw length is large (e.g., 50), Scale becomes ≈ 1 (Keep the signal).

Sub-step 3D: The Final Multiplication (Recombination)

We merge the Probability back with the Direction.

Logic: We now have a vector that points in the correct direction (Pose) but has a length that
equals the Probability of existence.

Step 4: The Output (Ready for Routing)

The Primary Capsule Layer is now finished. It outputs a structured grid of 'Smart Vectors'.

Final Output Shape: [32, 32, 32, 8]

Meaning at grid point (10, 10):

 We have 32 Vectors.
 Vector 1: Might have length 0.99 (High confidence 'Vertical Edge'). Its internal values
describe angle/thickness.
 Vector 2: Might have length 0.01 (Low confidence 'Horizontal Edge'). It is effectively
turned off.

The Class Capsule Process (Step-by-Step)

1. The Input (From Primary Layer)

 Data: We receive 32 Primary Capsules (let's say to ).

 Shape: Each is a vector of Length 8 ().
 Meaning: These are the "Parts" (e.g., vertical edge, arc, small circle).

2. The Prediction (Transformation)

 Action: Every Primary Capsule makes a guess for Every Class ().
 Math (Matrix Multiplication):
o Multiply Input () by Weight Matrix ().
 Result: A Prediction Vector of Length 16.
o "If I am a vertical line, the Corona Discharge is at [12, 12, 45...]."
3. The Routing Loop (finding Consensus)

 We repeat this cycle 3 times.

A. weighted Sum (The Ballot Box)

o The Class Capsule collects all predictions, weighted by the current trust ().
o Math:
o Result: The Raw Consensus Vector.

B. Squashing (Normalization)

o We shrink the raw vector so its length is between 0 and 1.

o Math:
o Result: The Current Group Consensus ().

C. Agreement Check (The Audit)

o We check if the Individual Prediction matches the Group Consensus.

o Math (Dot Product):
o Result: A scalar score (High Positive = Match, Negative = Mismatch).

D. Update Trust (The Learning)

o We update the "Logit" score () and recalculate the percentages.

o Math:
o Math:
o Result: The "Liars" (noise) get lower (trust), the "Truth-tellers" get higher .

4. The Final Decision (Classification)

 Action: After 3 loops, we take the final Vector .

 Math (Norm): We measure the length.
 Result:
o Corona: Length 0.1
o Surface: Length 0.95 (Winner!)
o Void: Length 0.3
Step-by-Step Architecture Walkthrough

Here is the breakdown of the RS-DACN (Robust Spatial-Deep Adaptive Capsule Network).
Use this structure for your "Methodology" chapter.

Stage 1: Feature Extraction (The "Eyes")

 Component: Convolutional Layer 1 & 2.

 Input: Image (Simulated or Real).
 Math: Standard Convolution operation .
 Function: Detects primitive visual features (edges, bright spots, curves).
 Status: At this point, the model only sees "pixels." It doesn't know what an object is.

Stage 2: Primary Capsules (The "Vectorization")

 Component: Primary Capsule Layer.

 Action: It groups the neurons from Stage 1 into Vectors.
 The Difference:
o CNN (Competitors): Says "There is a 90% chance of a curve here." (Scalar).
o RS-DACN (You): Says "There is a curve here, rotated 30 degrees, scale 1.2,
intensity 0.8." (Vector).
 Why it handles Switchgear Noise: Real switchgear data is messy. If a signal is rotated
or shifted by sensor noise, a CNN loses it. Your Capsule Vector just "rotates" its arrow
but keeps the signal strong. This is called Equivariance.

Stage 3: The Squash Function (The "Filter")

 Component: Non-linear Vector Activation.

 Math:
 Function: This is your "Automatic Denoising."
o Background white noise produces short, random vectors. The formula crushes
them to 0.
o PD Faults produce long, coherent vectors. The formula keeps them.
 Unique Selling Point: You don't need a separate denoising algorithm (like Wavelet
Transform). Your network architecture is the denoiser.

Stage 4: Dynamic Routing (The "Brain")

 Component: Routing-by-Agreement Algorithm.

 Logic:
o Low-level capsules (e.g., "Arc shape") vote for High-level capsules (e.g., "Corona
Fault").
o If many low-level shapes agree ("I see an arc," "I see a glow," "I see a pulse"), the
"Corona" capsule becomes active.
 o Noise Rejection: Random noise pixels do not agree with each other. They vote
randomly. The routing algorithm ignores them because there is no consensus.
 Result: The model is robust against the chaotic environment of Air-Insulated Switchgear.

Stage 5: The Decision (The Output)

 Component: Class Capsules (Corona, Surface, Internal).

 Math: Calculate the Length (Norm) of the final vectors.
o
o (Winner)
 Result: High confidence classification.

Summary for your Thesis:

"Unlike methods that rely on GAN-based data augmentation [Cite Paper 1], this study employs
a Sim-to-Real Domain Adaptation strategy. By leveraging the vector-based equivariance of
Capsule Networks, the RS-DACN learns the invariant physical geometry of partial discharges
from simulation, allowing it to generalize to noisy real-world switchgear environments where
standard CNNs fail."