Scribd
Upload
14 views9 pages

Convolutional Layer Techniques

The document provides an overview of convolution layers in deep learning, including the implementation of a cross-correlation operator and a convolutional layer class in MXNet. It demonstrates edge detection in images using convolutional kernels and discusses the learning process for a kernel through backpropagation. The document also includes code snippets and results from various tests, showcasing the effectiveness of the convolutional operations.

Uploaded by

wen zhou
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views9 pages

Convolutional Layer Techniques

The document provides an overview of convolution layers in deep learning, including the implementation of a cross-correlation operator and a convolutional layer class in MXNet. It demonstrates edge detection in images using convolutional kernels and discusses the learning process for a kernel through backpropagation. The document also includes code snippets and results from various tests, showcasing the effectiveness of the convolutional operations.

Uploaded by

wen zhou
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

2/23/2019 conv-layer slides

Convolution Layers
In [1]: from mxnet import autograd, nd
from [Link] import nn

[Link] 1/9
2/23/2019 conv-layer slides

The Cross Correlation Operator


In [2]: def corr2d(X, K):
h, w = [Link]
Y = [Link](([Link][0] - h + 1, [Link][1] - w + 1))
for i in range([Link][0]):
for j in range([Link][1]):
Y[i, j] = (X[i: i + h, j: j + w] * K).sum()
return Y

[Link] 2/9
2/23/2019 conv-layer slides

Sanity Test
In [3]: X = [Link]([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
K = [Link]([[0, 1], [2, 3]])
corr2d(X, K)

Out[3]: [[19. 25.]


[37. 43.]]
<NDArray 2x2 @cpu(0)>

[Link] 3/9
2/23/2019 conv-layer slides

Convolutional Layers
In [4]: class Conv2D([Link]):
def __init__(self, kernel_size, **kwargs):
super(Conv2D, self).__init__(**kwargs)
[Link] = [Link]('weight', shape=kernel_size)
[Link] = [Link]('bias', shape=(1,))

def forward(self, x):


return corr2d(x, [Link]()) + [Link]()

[Link] 4/9
2/23/2019 conv-layer slides

Object Edge Detection in Images


In [5]: X = [Link]((6, 8))
X[:, 2:6] = 0
X

Out[5]: [[1. 1. 0. 0. 0. 0. 1. 1.]


[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]]
<NDArray 6x8 @cpu(0)>

[Link] 5/9
2/23/2019 conv-layer slides

Detect Vertical Edges


In [6]: K = [Link]([[1, -1]])
Y = corr2d(X, K)
print(X, Y)

[[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]
[1. 1. 0. 0. 0. 0. 1. 1.]]
<NDArray 6x8 @cpu(0)>
[[ 0. 1. 0. 0. 0. -1. 0.]
[ 0. 1. 0. 0. 0. -1. 0.]
[ 0. 1. 0. 0. 0. -1. 0.]
[ 0. 1. 0. 0. 0. -1. 0.]
[ 0. 1. 0. 0. 0. -1. 0.]
[ 0. 1. 0. 0. 0. -1. 0.]]
<NDArray 6x7 @cpu(0)>

[Link] 6/9
2/23/2019 conv-layer slides

Can't Detect Horizon Edges


In [7]: Z = corr2d(X.T, K)
print(X.T,Z)

[[1. 1. 1. 1. 1. 1.]
[1. 1. 1. 1. 1. 1.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1. 1.]
[1. 1. 1. 1. 1. 1.]]
<NDArray 8x6 @cpu(0)>
[[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]]
<NDArray 8x5 @cpu(0)>

[Link] 7/9
2/23/2019 conv-layer slides

Learning a Kernel
In [8]: conv2d = nn.Conv2D(1, kernel_size=(1, 2))
[Link]()

X = [Link]((1, 1, 6, 8))
Y = [Link]((1, 1, 6, 7))

for i in range(10):
with [Link]():
Y_hat = conv2d(X)
l = (Y_hat - Y) ** 2
[Link]()
[Link]()[:] -= 3e-2 * [Link]()
if (i + 1) % 2 == 0:
print('batch %d, loss %.3f' % (i + 1, [Link]().asscalar()))

batch 2, loss 4.949


batch 4, loss 0.831
batch 6, loss 0.140
batch 8, loss 0.024
batch 10, loss 0.004

[Link] 8/9
2/23/2019 conv-layer slides

Learned Kernel
In [9]: [Link]().reshape((1, 2))

Out[9]: [[ 0.9895 -0.9873705]]


<NDArray 1x2 @cpu(0)>

[Link] 9/9

You might also like