Scribd
Upload
3 views21 pages

Pooling

The document discusses pooling in the context of neural networks, focusing on its goal to sample feature spaces, reduce feature map sizes, and prevent overfitting. It provides detailed explanations of max-pooling, including its hyper-parameters such as type, window size, and stride, along with visual examples. Additionally, it briefly covers other pooling types like average and min pooling.

Uploaded by

Thanh Hùynh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views21 pages

Pooling

The document discusses pooling in the context of neural networks, focusing on its goal to sample feature spaces, reduce feature map sizes, and prevent overfitting. It provides detailed explanations of max-pooling, including its hyper-parameters such as type, window size, and stride, along with visual examples. Additionally, it briefly covers other pooling types like average and min pooling.

Uploaded by

Thanh Hùynh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Pooling

Dr. Thanh-Sach LE
LTSACH@[Link]

GVLab: Faculty of Computer Science and Engineering,


Graphics and Vision Laboratory HCMUT
2 Contents
❖ Goal of pooling
❖ Max-pooling
❖ Other types of pooling
3 Goal of pooling layer
❖ Sampling feature spaces to remove redundant features
❖ Reducing size of feature maps
❖ Avoid overfitting (minor)
4 Max-pooling
Pooling’s hyper-parameters
✴ (1) Type of pooling
✴ (2) Window size
✴ (3) Stride

∂b
5 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8
8 0 2 1 1 7
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
6 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8 6
8 0 2 1 1 7
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
7 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8 6 6
8 0 2 1 1 7
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
8 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8 6 6 6
8 0 2 1 1 7
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
9 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8 6 6 6
8 0 2 1 1 7
1 2 2 4 6 2 8

1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
10 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
Max

1 3 6 1 3 2
∂b
8 6 6 6
8 0 2 1 1 7
1 2 2 4 6 2 8 5 6 7

1 4 3 5 2 7 4 5 6 8
1 2 1 5 6 8
2 2 7 4 2 2 7 7 7 7

Input feature map Output feature map


11 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1

∂b
i1
1 3 6 1 3 2 8 6 6 6
8 0 2 1 1 7
8 5 6 7
i2 1
1
2
4
2
3
4
5
6
2
2
7 4 5 6 8
1 2 1 5 6 8
2 2 7 4 2 2 7 7 7 7

Input feature map Output feature map


12 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1

1st valid position


∂b

1 3 6 1 3 2
8 0 2 1 1 7

i2 1
1
2
4
2
3
4
5
6
2
2
7
1 2 1 5 6 8
2 2 7 4 2 2

i1
13 Max-pooling

Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 1
i1 − k1 + 1
1st valid position last valid position
=6−3+1
i1 − k1 + 1 (k1 − 1) =4
∂b

1 3 6 1 3 2 8 6 6 6
8 0 2 1 1 7
i2 − k2 + 1 8 5 6 7
i2 1
1
2
4
2
3
4
5
6
2
2
7 =6−3+1
4 5 6 8
=4
1 2 1 5 6 8
2 2 7 4 2 2 7 7 7 7

Output feature map


i1
14
Max-pooling
Non-unit strides
Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b Max

1 3 6 1 3 2
8 0 2 1 1 7 8
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
15
Max-pooling
Non-unit strides
Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b Max
stride =2

1 3 6 1 3 2
8 0 2 1 1 7 8 6
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
16
Max-pooling
Non-unit strides
Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b Max
stride =2

stride =2 1 3 6 1 3 2
8 0 2 1 1 7 8 6
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8 4
2 2 7 4 2 2
Input feature map Output feature map
17
Max-pooling
Non-unit strides
Pooling’s hyper-parameters
✴ (1) Type = max-poling
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b Max
stride =2

stride =2 1 3 6 1 3 2
8 0 2 1 1 7 8 6
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8 4 6
2 2 7 4 2 2
Input feature map Output feature map
18
Max-pooling
Non-unit strides

i1 − k1
⌊ s ⌋+1
(i1 − k1)
s1 k1
∂b

1 3 6 1 3 2
8 0 2 1 1 7 8 6
i2 − k2
⌊ s ⌋+1
1 2 2 4 6 2
1 4 3 5 2 7 2
1 2 1 5 6 8 4 6
2 2 7 4 2 2
Output feature map
19
Max-pooling
Non-unit strides

6−3
⌊ 2 ⌋+1=2
(i1 − k1)
s1 k1
∂b

1 3 6 1 3 2
8 0 2 1 1 7 8 6
1 2 2 4 6 2
2
1 4 3 5 2 7
1 2 1 5 6 8 4 6
2 2 7 4 2 2
Output feature map
20 Other types of pooling

Pooling’s hyper-parameters
✴ (1) Type = average
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b avg

1 3 6 1 3 2
8 0 2 1 1 7 2.78
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map
21 Other types of pooling

Pooling’s hyper-parameters
✴ (1) Type = min
✴ (2) Window size = 3x3
✴ (3) Stride = 2

∂b min

1 3 6 1 3 2
8 0 2 1 1 7 0
1 2 2 4 6 2
1 4 3 5 2 7
1 2 1 5 6 8
2 2 7 4 2 2
Input feature map Output feature map

You might also like