See discussions, stats, and author profiles for this publication at: [Link]

net/publication/333191474

Classification of Image using Convolutional Neural Network (CNN)

Article · May 2019

DOI: 10.34257/GJCSTDVOL19IS2PG13

CITATIONS READS

64 4,206

2 authors, including:

Md. Anwar Hossain

Pabna University of Science and Technology
30 PUBLICATIONS 205 CITATIONS

SEE PROFILE

All content following this page was uploaded by Md. Anwar Hossain on 18 May 2019.

The user has requested enhancement of the downloaded file.

 Global Journal of Computer Science and Technology: D
Neural & Artificial Intelligence
Volume 19 Issue 2 Version 1.0 Year 2019
Type: Double Blind Peer Reviewed International Research Journal
Publisher: Global Journals
Online ISSN: 0975-4172 & Print ISSN: 0975-4350

Classification of Image using Convolutional Neural Network (CNN)

By Md. Anwar Hossain & Md. Shahriar Alam Sajib
Pabna University of Science & Technology
Abstract- Computer vision is concerned with the automatic extraction, analysis, and understanding of
useful information from a single image or a sequence of images. We have used Convolutional Neural
Networks (CNN) in automatic image classification systems. In most cases, we utilize the features from the
top layer of the CNN for classification; however, those features may not contain enough useful information
to predict an image correctly. In some cases, features from the lower layer carry more discriminative
power than those from the top. Therefore, applying features from a specific layer only to classification
seems to be a process that does not utilize learned CNN’s potential discriminant power to its full extent.
Because of this property we are in need of fusion of features from multiple layers. We want to create a
model with multiple layers that will be able to recognize and classify the images. We want to complete our
model by using the concepts of Convolutional Neural Network and CIFAR-10 dataset. Moreover, we will
show how MatConvNet can be used to implement our model with CPU training as well as less training
time. The objective of our work is to learn and practically apply the concepts of Convolutional
Neural Network.
Keywords: convolutional neural network, CIFAR-10 dataset, MatConvNet, relu, softmax.
GJCST-D Classification: I.2.6

ClassificationofImageusingConvolutionalNeuralNetworkCNN

Strictly as per the compliance and regulations of:

© 2019. Md. Anwar Hossain & Md. Shahriar Alam Sajib. This is a research/review paper, distributed under the terms of the
Creative Commons Attribution-Noncommercial 3.0 Unported License [Link] permitting
all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
 Classification of Image using Convolutional
Neural Network (CNN)
Md. Anwar Hossainα & Md. Shahriar Alam Sajibσ

Abstract- Computer vision is concerned with the automatic We use this dataset to train machine learning and
extraction, analysis, and understanding of useful information computer vision algorithms. CIFAR-10 database is the
from a single image or a sequence of images. We have used contribution of Alex Krizhevsky and Geoffrey Hinton. This

2 019
Convolutional Neural Networks (CNN) in automatic image
dataset has 60,000 colored images. It has ten classes,
classification systems. In most cases, we utilize the features
and they are an airplane, automobile, bird, cat, deer,

Year
from the top layer of the CNN for classification; however, those
features may not contain enough useful information to predict dog, frog, horse, ship, truck. The images are of size
an image correctly. In some cases, features from the lower 32x32 pixels. The dataset consists of 50,000 training 13
layer carry more discriminative power than those from the top. and 10,000 testing examples. It is a database for people
Therefore, applying features from a specific layer only to who want to try learning techniques and pattern

Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I
classification seems to be a process that does not utilize recognition methods on real-world data while spending
learned CNN’s potential discriminant power to its full extent. minimal efforts on preprocessing and formatting. We will
Because of this property we are in need of fusion of features use this database in our experiment.
from multiple layers. We want to create a model with multiple
layers that will be able to recognize and classify the images. b) Convolutional Neural Networks
We want to complete our model by using the concepts of Convolutional neural networks are deep artificial
Convolutional Neural Network and CIFAR-10 dataset. neural networks. We use CNN to classify images, cluster
Moreover, we will show how MatConvNet can be used to them by similarity (photo search), and perform object
implement our model with CPU training as well as less training
recognition within scenes. It can be used to identify
time. The objective of our work is to learn and practically apply
faces, individual, street signs, tumors, platypuses and
the concepts of Convolutional Neural Network.
Keywords: convolutional neural network, CIFAR-10 many other aspects of visual data. The convolutional
dataset, MatConvNet, relu, softmax. layer is the core building block of a CNN. The layer’s
parameters consist of a set of learnable filters (or
I. Introduction kernels) which have a small receptive field but extend
through the full depth of the input volume. During the

C
onvolutional Neural Networks (CNN) becomes forward pass, each filter is convolved across the width
one of the most appealing approaches recently and height of the input volume, computing the dot
and has been an ultimate factor in a variety of product, and producing a 2-dimensional activation map
recent success and challenging applications related to of that filter. As a result, the network learns about the
machine learning applications such as challenge filters. The filter activates when they see some specific
ImageNet object detection, image classification, and type of feature at some spatial position in the input.
face recognition. Therefore, we consider CNN as our Then the activation maps are fed into a downsampling
model for our challenging tasks of image classification. layer, and like convolutions, this method is applied one
We use CNN for segmentation and classification of the patch at a time. CNN has also fully connected layer that
images in academic and business transactions. We use classifies output with one label per node.
image recognition in different areas for example
automated image organization, stock photography, face II. Related Works
recognition, and many other related works.
Image recognition has an active community of
a) CIFAR-10 Database academics studying it. A lot of important work on
The CIFAR-10 database (Canadian Institute for convolutional neural networks happened for image
Advanced Research database) is a collection of images. recognition [1,2,3,4]. The most dominant recent works
achieved using CNN is a challenging work introduced
Author α: Assistant Professor, Department of Information &
by Alex Krizhevsky [5], who used CNN for challenge
Communication Engineering, Faculty of Engineering & Technology, classification ImageNet. Active areas of research are:
Pabna University of Science & Technology, Pabna, Bangladesh. object detection [14,15,16], scene labeling [17],
e-mail: [Link]@[Link] segmentation [18,19], face recognition, and variety of
Author σ: Student, Department of Information & Communication
Engineering, Faculty of Engineering & Technology, Pabna University of
other tasks [20,21,22].
Science & Technology, Pabna, Bangladesh.
e-mail: sajibpust130639@[Link]

© 2019 Global Journals

 Classification of Image using Convolutional Neural Network (CNN)

III. Methodology
Deep Learning has emerged as a main tool for
self-perception problems like understanding images, the
voice from humans, robots exploring the world. We aim
to implement the concept of the Convolutional Neural
Network for the recognition of images. Understanding
CNN and applying it to the image recognition system is
Fig. 1: Convolution Operation
the target of the proposed model. Convolutional Neural
Network extracts the feature maps from the 2D images When constructing CNN, it is common to insert
by using filters. The Convolutional neural network pooling layers after each convolution layer, so that we
considers the mapping of image pixels with the can reduce the spatial size of the representation. This
2 019

neighborhood space rather than having a fully layer reduces the parameter counts, and thus reduces
connected layer of neurons. The Convolutional neural the computational complexity. Also, pooling layers help
Year

network has been proved to be a very dominant and with the overfitting problem. We select a pooling size to
potential tool in image processing. Even in the fields of reduce the amount of the parameters by selecting the
14
computer vision such as handwriting recognition, natural maximum, average, or sum values inside these pixels.
object classification, and segmentation, CNN has Fig.2 shows the max pooling and average pooling
Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I

become a much better tool compared to all other operation.

previously implemented tools.
a) The architecture of the Proposed Model
When one starts learning deep learning with the
neural network, he realizes that one of the most
supervised deep learning techniques is the
Convolutional Neural Network. We design Convolutional
Neural Network to recognize visual patterns directly from
pixel images with minimal preprocessing. Almost all
CNN architectures follow the same general design
principles of successively applying convolutional layers
Fig. 2: Max pooling and Average pooling operation
to the input, periodically downsampling (Max pooling)
the spatial dimensions while increasing the number of A fully connected network is in any architecture
feature maps. Moreover, there are also fully connected where each parameter is linked to one another to
layers, activation functions and loss function (e.g., cross determine the relation and effect of each parameter on
entropy or softmax). However, among all the operations the labels. We can vastly reduce the time-space
of CNN, convolutional layers, pooling layers, and fully complexity by using the convolution and pooling layers.
connected layers are the most important ones. We can construct a fully connected network in the end
Therefore, we will quickly introduce these layers before to classify our images.
presenting our proposed model.
The Convolutional layer is the very first layer
where it can extract features from the images. Because
pixels are only related to the adjacent and close pixels,
convolution allows us to preserve the relationship
between different parts of an image. Convolution is
filtering the image with a smaller pixel filter to decrease
the size of the image without losing the relationship
between pixels. When we apply convolution to a 7x7
image by using a filter of size 3x3 with 1x1 stride (1-pixel
Fig. 3: Fully connected layer
shift at each step), we will end up having a 5x5 output.
Fig.4 shows the overview look of our proposed
convolutional neural network. It is very much similar to
the other image recognition architectures [1,2,3,4] but
has changed in the number of filters, neurons and
activation functions for better performance. We can
divide our model into six sequences of layers.

© 2019 Global Journals

 Classification of Image using Convolutional Neural Network (CNN)

2 019
Fig. 4: The architecture of our proposed CNN

Year
b) Explanation of the Model Layer-4 is the average pooling layer. This layer
A simple convolutional network is a sequence of gets the input of size 32@10*10 from the previous layer.
15
layers. The layer transforms one volume of activations to The pooling size is 2*2; padding is 0 and stride is 2.
another through a differentiable function. We use three After this max pooling operation, we get a feature map

Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I
main types of layers to build network architecture. They of size 32@5*5.
are a convolutional layer, pooling layer, and fully Layer-5 is the third convolutional layer with ReLu
connected layer. We will stack these layers to form six activation function. This layer gets the input of size
layers of network architecture. We will go into more 32@5*5 from the previous layer. The filter size is 4*4;
details below. padding is 0, the stride is 1, and the number of filters is
Fig.4 shows the architecture of our proposed 64. After this convolution operation, we get feature maps
CNN model. At first, we need some pre-processing on of size 64@1*1. This layer acts as a fully connected
the images like resizing images, normalizing the pixel layer and produces a one-dimensional vector of size 64
values, etc. After the necessary pre-processing, data is by being flattened.
ready to be fed into the model. Layer-6 is the last layer of the network. It is a
Layer-1 consists of the convolutional layer with fully connected layer. This layer will compute the class
ReLu (Rectified Linear Unit) activation function which is scores, resulting in a vector of size 10, where each of
the first convolutional layer of our CNN architecture. This the ten numbers corresponds to a class score, such as
layer gets the pre-processed image as the input of size among the ten categories of CIFAR-10 dataset. For final
n*n=32*32. The convolutional filter size (f*f) is 5*5, outputs, we use the softmax activation function.
padding (p) is 0(around all the sides of the image), In this way, CNN transforms the original image
stride (s) is 1, and the number of filters is 32. After this layer by layer from the main pixel values to the final
convolution operation, we get feature maps of size class scores. Note that some layers contain parameters,
32@28*28 where 32 is the number of feature maps and others don’t. In particular, the convolution/fully
which is equal to the number of filters used, and 28 connected layers perform transformations that are a
comes from the formula ((n+2p-f)/s) +1= ((32+2*0- function of not only the activations in the input volume
5)/1) +1=28. Then the ReLu activation is done in each but also of the parameters (the weights and biases of
feature map. the neurons). On the other hand, the Relu/pooling layers
Layer-2 is the max pooling layer. This layer gets will implement a fixed function. We train the parameters
the input of size 32@28*28 from the previous layer. The in the convolutional/fully connected layers with
pooling size is 2*2; padding is 0 and stride is 2. After stochastic gradient descent. By this process, we will
this max pooling operation, we get feature maps of size prepare the trained model which will be used to
32@14*14. Max pooling is done in each feature map recognize the image present in the test data. Thus, we
independently, so we get same number feature maps as can classify the images as Class- airplanes, cars, birds,
the previous layer, and 14 comes from the same formula cats, deer, dogs, frogs, horses, ships, trucks.
((n+2p-f)/s) +1. This layer has no activation function.
Layer-3 is the second convolutional layer with IV. Implementation
ReLu activation function. This layer gets the input of size To implement our CNN architecture, we will use
32@14*14 from the previous layer. The filter size is 5*5; MatConvNet. MatConvNet is an implementation of
padding is 0, the stride is 1, and the number of filters is Convolutional Neural Networks (CNN) for MATLAB [23].
32. After this convolution operation, we get feature maps We built our model by using MatConvNet so that our
of size 32@10*10. Then ReLu activation is done in each model has greater simplicity and flexibility. It exposes
feature map. the building blocks of CNN as easy-to-use MATLAB
functions, providing routines for computing linear

© 2019 Global Journals

 Classification of Image using Convolutional Neural Network (CNN)

convolutions with filter banks, feature pooling and many initialization parameters for example batch size, number
more. In this manner, MatConvNet allows fast of epochs, learning rate, etc.
prototyping of new CNN architectures; at the same time, The batch size determines the number of
it supports efficient computation on CPU and GPU samples for the training phase of the CNN. The CNN will
allowing to train complex models on large datasets such process all the training data, but only in increments of
as ImageNet ILSVRC. the specified batch size. We can use batch size for
Convolutional Neural Networks (CNN) is the computational efficiency, and its value will be dependent
current state-of-art architecture for the image on the user’s available hardware. An epoch is a
classification task. As shown in Fig.4 our proposed 2-D successful forward pass and a backward pass through
Convolutional Neural Network (CNN) model is designed the network. It’s usually beneficial to set its value high
using MatConvNet backend for the well known CIFAR- and then to reduce it once if one is satisfied with the
10 image recognition task. The whole workflow can be convergence at a particular state (chosen epoch) in the
2 019

to preparing the data, building and compiling the model, network. Learning rate is a very sensitive parameter that
Year

training and evaluating the model and saving the model pushes the model towards convergence. Finding its
to disk for reuse. best value will be an experimental process unless one
16 Preparing the data is the first step of our invokes more powerful techniques such as batch
approach. Before we build the network, we need to set normalization. In our experiment, we use batch size 60,
up our training and testing data, combine data, combine several epochs 300 and learning rate 0.0001 for
Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I

labels and reshape into the appropriate size. We save maximum accuracy.
the dataset of normalized data (single precision and Now we can build our CNN by creating each
zero mean), labels, and miscellaneous (meta) layer individually as shown in fig 5. Afterward, we will
information. invoke objective and error layers that will provide a
Building and compiling the model is the second graphical visualization of the training and validation
step. To create the CNN, we must initialize MatConvNets convergence after completing each epoch. MatconvNet
SimpleNN network and then define important initializes the weights by using Gaussian distribution.

Fig. 5: CNN layers in MatConvNet

The third step is the training and evaluating the error in the CNN. We feed the training data through the
model. Training a CNN requires computing the network for the forward pass and backward pass. The
derivative of the loss concerning the network validation data is just used to see how the CNN
parameters. We calculate the derivatives using an responds to new similar data. We do not use the
algorithm called back propagation which is a memory- validation data to train the network. Afterward, we save
efficient implementation of the chain rule for derivatives. the trained CNN and prepare for the testing phase.
We built the model and performed a random gradient During the training phase of the CNN, the
descent training according to the Stochastic Gradient simple network will produce three plots (Objective,
Descent (SGD) training algorithm. We have used SGD Top1error, and Top5error) for each epoch. The top1
training algorithm to adjust the weight of the connection error is the chance that class with the highest probability
between neurons so that the loss reaches a minimum is the correct target. In other words, CNN guesses the
value or stops after several epochs. We have used CPU target correctly. The top5error is the chance that the true
training. It is important to note that GPU training will target is one of the top five probabilities. The objective
dramatically help training time for CNN. for the simple network should mirror the form of the top1
Lastly, we can begin the training of CNN by and top5 error. In all the plots, we represent the training
supplying the training data, the constructed model and error and validation error by blue and orange
the current batch of data. When training the CNN, only respectively.
the data specified for training play a role in minimizing

© 2019 Global Journals

 Classification of Image using Convolutional Neural Network (CNN)

2 019 Year
17
Fig. 6: Objective, Top1error, and Top5error during training

Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I
Finally, by using the testing data, we can
evaluate our model. The following are an example of V. Results and Discussion
classification outputs from the simple network on the Among 10,118 test cases, our model
CIFAR-10 data. misclassifies total of 661 images after three hundred
epochs which correspond to 93.47% recognition rate
shown in Table1. The results are pretty good for three
hundred epochs and for such a simple model with CPU
training and less training time (about 3 hours).

Fig. 7: Some correct recognized outputs Fig. 9: Error rate and accuracy of our model.
Although there are some images which are
difficult to identify, our model will be able to classify
them correctly. For example, our model can recognize
the following image and classify it as ‘deer’:

Fig. 8: Some wrong recognized outputs

Fig. 10: Correct recognition of the bad image
We can determine the test cases that show
failed classifications. The model can’t identify some Table 1: Summary of the experiment
images because of limitations in the input of standard Batch size No. of epochs Testing accuracy
data images. Moreover, missing pixels caused by image 100 250 76.82%
compression and image sharpness problems are also 70 300 82.28%
reasons for misclassification. 60 300 93.47%
The fourth and final step is to save the model in Test accuracy 93.47% implies that the model is
the disk for reuse. We store the trained model in a trained well for prediction. Training set size affects the
MATLAB file format. Hence the saved model can be accuracy increases as the number of data increases.
reused later or easily ported to other environments too. The more data in the training set, the smaller the impact

© 2019 Global Journals

 Classification of Image using Convolutional Neural Network (CNN)

of training error and test error and ultimately the neural networks. arXiv preprint arXiv:1301.3557,
accuracy can be improved. 2013.
8. Ian J Goodfellow, David Warde-Farley, Mehdi Mirza,
VI. Conclusion and Future Work Aaron Courville, and Yoshua Bengio. Maxout
Here we demonstrate a model which can networks. arXiv preprint arXiv:1302.4389, 2013.
recognize and classify the image. Later it can be 9. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky,
extended for object recognition, character recognition, Ilya Sutskever, and Ruslan Salakhutdinov‖ Dropout:
and real-time object recognition. Image recognition is an A Simple Way to Prevent Neural Networks from
important step to the vast field of artificial intelligence Overfitting‖ Journal of Machine Learning Research
and computer vision. As seen from the results of the 15 (2014) 1929-1958.
experiment, CNN proves to be far better than other 10. Fabien Lauer, Ching Y. Suen, and G´erard Bloch
―A trainable feature extractor for handwritten digit
2 019

classifiers. The results can be made more accurate by

increasing the number of convolution layers and hidden recognition‖ Journal Pattern Recognition 2007.
Year

neurons. People can recognize the object from blurry 11. Chen-Yu Lee, Saining Xie, Patrick Gallagher,
images by using our model. Image recognition is an Zhengyou Zhang, Zhuowen Tu, ― Deeply-
18 excellent prototype problem for learning about neural Supervised Nets ―NIPS 2014.
networks, and it gives a great way to develop more 12. M. Fischler and R. Elschlager, ―The representation
and matching of pictorial structures,‖ IEEE
Global Journal of Computer Science and Technology ( D ) Volume XIX Issue II Version I

advanced techniques of deep learning. In the future, we

are planning to develop a real-time image recognition Transactions on Computer, vol. 22, no. 1, 1973.
system. 13. Kevin Jarrett, Koray Kavukcuoglu, Marc’Aurelio
Ranzato and Yann LeCun ‖ What is the Best Multi-
References Références Referencias Stage Architecture for Object Recognition?‖
ICCV’09, IEEE, 2009.
1. Kuntal Kumar Pal, Sudeep K. S.(2016).” 14. Kaiming, He and Xiangyu, Zhang and Shaoqing,
Preprocessing for Image Classification by Ren and Jian Sun ―Spatial pyramid pooling in
Convolutional Neural Networks”, IEEE International deep convolutional networks for visual recognition‖
Conference on Recent Trends in Electronics European Conference on Computer Vision, 2014.
Information Communication Technology, May 20- 15. Ross Girshick, ―Fast R-CNN ―arXiv preprint
21, 2016, India. arXiv:1504.08083, 2015
2. Hayder M. Albeahdili, Haider A. Alwzwazy, Naz E. 16. X. Wang, M. Yang, S. Zhu, and Y. Lin. Regionlets for
Islam (2015).”Robust Convolutional Neural generic object detection. In ICCV, 2013. 8.
Networks for Image Recognition”, (IJACSA) 17. Karen Simonyan and Andrew Zisserman ―VERY
International Journal of Advanced Computer DEEP CONVOLUTIONAL NETWORKS FOR LARGE-
Science and Applications, Vol. 6, No. 11, 2015. SCALE IMAGE RECOGNITION‖ arXiv:1409.1556v5
3. Jingkun Qin, Haihong E, Meina Song and Zhijun [[Link]] 23 Dec 2014.
Ren(2018).”Image Retrieval Based on a Hybrid 18. C. Couprie, C. Farabet, L. Najman, and Y. LeCun.
Model of Deep Convolutional Encoder”, 2018 the Indoor semantic segmentation using depth
International Conference of Intelligent Robotic and information. Internatinal Conference on Learning
Control Engineering. Representation, 2013. 2.
4. K Sumanth Reddy, Upasna Singh, Prakash K 19. R. Girshick, J. Donahue, T. Darrell, and J. Malik.
Uttam(2017).” Effect of Image Colourspace on Rich feature hierarchies for accurate object
Performance of Convolution Neural Networks”, 2017 detection and semantic segmentation. CoRR,
2nd IEEE International Conference on Recent abs/1311.2524, 2013. 4.
Trends in Electronics Information & Communication 20. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L.
Technology (RTEICT), May 19-20, 2017, India. FeiFei. Imagenet: A large-scale hierarchical image
5. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, database. In CVPR, 2009. 2.
Geoffrey. ImageNet classification with deep 21. L. N. Clement Farabet, Camille Couprie and Y.
convolutional neural networks. In Advances in LeCun. Learning hierarchical features for scene
Neural Information Processing Systems 25 labeling. PAMI, 35(8), 2013. 1, 2.
(NIPS’2012). 2012. 22. Honglak Lee, Roger Grosse, Rajesh Ranganath,
6. Marc’Aurelio Ranzato, Fu-Jie Huang, Y-Lan and Andrew Y. Ng ―Convolutional Deep Belief
Boureau, Yann LeCun, ―Unsupervised Learning of Networks for Scalable Unsupervised Learning of
Invariant Feature Hierarchies with Applications to Hierarchical Representations‖.
Object Recognition‖ CVPR, 2007. 23. “MatConvNet Convolutional Neural Networks for
7. Matthew D Zeiler and Rob Fergus. Stochastic MATLAB” Andrea Vedaldi, Karel Lenc, Ankush
pooling for regularization of deep convolutional Gupta.

© 2019 Global Journals

View publication stats