Automatic Detection and Classification of Defective Areas on
Metal Parts by Using Adaptive Fusion of Faster R-CNN and
Shape from Shading
Proposed topic
DS-ResNet: Systematic analysis of tyre defect detection through
Deep Stacked Resilient Network
ABSTRACT
Defect detection is an interesting area of research where identification of various products
and its defects are analysed through localizing the surface area that get impacted and evaluate
the region size, area, texture to validate. The proposed system is developed using deep
learning reinforced architecture using Stacked Resilient Network (DS-ResNet). The proposed
structure read the input tyre images and enhance the image pixels using median filter. The
features are extracted using RBFE region based feature extraction. The images are further
classified into defect image and non-defect image etc. the performance of the system is
evaluated through accuracy, precision, sensitivity and Mathew’s correlation constant (MCC)
Exiting system
In the existing system, to detect and classify metal surface defects, such as cracks, scratches,
inclusion, etc. The type and location of defects were detected by the Faster Regional
Convolutional Neural Network (Faster R-CNN), combined with the Shape from Shading
(SFS) method, which can extract surface characteristics. The proposed algorithm has also
been tested on an unlabelled dataset (KolektorSDD2/KSDD2) to show labelling performance.
Problem statement
Accuracy should be improved
False negative rate should be reduced
Proposed system
A New dataset is created by collecting various websites, google pages, dataset hub and
publicly available repositories. Defect detection in tyre surface is focused here. The proposed
system is developed using region-based texture analysis (RBTA), with Deep stacked ResNet
(DS-ResNet) for classification of defect area.
Solution statement
Deep learning is induced to detect the defect area
Region based analysis provides multi-modality learning
BLOCK DIAGRAM
SOFTWARE REQUIRED
MATLAB 2017A
TOOL BOX IMAGE PROCESSING
NEURAL NETWORKS, COMPUTER VISION
IMAGE TOOL
Image Tool in matlab, imtool( RGB ) displays the truecolor image RGB in the Image Viewer
app. imtool( BW ) displays the binary image BW in the Image Viewer app. Pixel values of 0
display as black; pixel values of 1 display as white. imtool( X , cmap ) displays the indexed
image X with colormap cmap in the Image Viewer app
impixelinfo creates a Pixel Information tool in the current figure. impixelinfo( h ) creates a
Pixel Information tool in the figure specified by the handle h . impixelinfo( hparent , himage )
creates a Pixel Information tool in hparent that provides information about the pixels in
himage.
The imtool function opens the Image Viewer app. ... imtool( I , range ) displays the grayscale
image I in the Image Viewer app, specifying the display range as range . The app clips pixel
values outside the display range to black or white and displays pixel values within the display
range as intermediate shades of gray.
IMPOINT SELECTION
An impoint object encapsulates an interactive point over an image.
You can adjust the position of the point by using the mouse. The point also has a context
menu that controls aspects of its appearance and behavior. For more information, see Usage.
h = impoint begins interactive placement of a point on the current axes, and returns an
impoint object.
h = impoint(hparent) begins interactive placement of a point on the object specified by
hparent.
h = impoint(hparent,position) creates a draggable point with coordinates defined by position.
example
h = impoint(hparent,x,y) creates a draggable point with (x, y) coordinates defined by x and y.
h = impoint(___,Name,Value) specifies name-value pairs that control the behavior of the
point.
Usage
When you call impoint with an interactive syntax, the pointer changes to a cross hairs when
over the image. Click and drag the mouse to specify the position of the point. The point
supports a context menu that you can use to control aspects of its appearance and behavior.
The table describes the interactive behavior supported by impoint.
Interactive Description
Behavior
Moving the point. Move the mouse pointer over the point. The mouse pointer changes
to a fleur shape . Click and drag the mouse to move the point.
Changing the color Move the mouse pointer over the point. Right-click and select Set
used to display the Color from the context menu and specify the color you want to
point. use.
Retrieving the Move the mouse pointer over the point. Right-click and
coordinates of the select Copy Position from the context menu to copy a 1-by-2 array
point. to the clipboard specifying the coordinates of the point [X Y].
Deleting the point Move the pointer on top of the point. Right-click and
select Delete from the context menu. To remove this option from
the context menu, set the Deletable property to false: h = impoint();
[Link] = false;
ROI Creation Overview
You can create ROI objects that represent ROIs of various shapes, including circles, ellipses,
polygons, lines, polylines, rectangles, and hand-drawn shapes. You can also draw freehand
shapes that get "assistance" from the underlying image, automatically following the contour
of edges. In this figure, you see a polygonal ROI drawn over an image.
You can control aspects of the ROI position and appearance. You can create masks from ROIs
and perform other operations. You can also specify how the ROI responds to events that
occur within the ROI, such as mouse clicks and movement.
DRAW LINES
line(x,y) plots a line in the current axes using the data in vectors x and y. If either x or y, or
both are matrices, then line draws multiple lines. Unlike the plot function, line adds the line
to the current axes without deleting other graphics objects or resetting axes properties.
Create x and y as vectors. Then call the low-level version of the line function by specifying
the data as name-value pair arguments. When you call the function this way, the resulting line
is blac
DISTANCE MEASUREMENT
Find the distance between the two perpendicular points in the flank wear, also calculate other
distances.
The calculated distance might be in constant that is measured in mm. depends on the pixel
size, as per our input test image, mentioned as 200mm / Pixel the value is normalized.
Test & Validation
Finally the tested input image and the measurement is compared with the theoretical
measurements and validated.
WORKING OF PROPOSED NOVEL METHOD
PREDICTION OF TYRE DAMAGE
For detecting the tyre damage, RBFE feature extraction is studied. After the preprocessing
stage, RBFE feature extraction is applied.
The feature points detect the strongest corner points that differ in pixels of normalized values.
Based on the strongest pixel points extracted by the RBFE features, the normal and abnormal
images of the Tool wear from the data base is compared.
Deep pattern network is used to analyze the matching feature patterns present with the test
image (DS-ResNet).
Higher the correlation, then the match score will be updated.
Pattern Network
Pattern recognition is a process of finding regularities and similarities in data using machine
learning data. Now, these similarities can be found based on statistical analysis, historical
data, or the already gained knowledge by the machine itself. A pattern is a regularity in the
world or in abstract notions.
ABOUT NEURAL COMPUTING
Features of Pattern Recognition
1. Speed and accuracy for the familiar is high
2. It can recognize an unfamiliar object
3. It has the ability to recognize different shapes and object from all
angles.
4. It identifies the patterns and objects when partly hidden.
5. During analysis quickly catch the patterns with automaticity
Training Set: The training set plays an important part to train the model.
Program process this dataset by using training rules. To get the better
result one need to collect quite a large dataset because the program will
always give better results with a handful of training data. But it may not
give the same results in the case of the test dataset. If someone is
building a masked face recognizer then he/she will need a lot of images of
people wearing a mask. From that dataset, the necessary information will
be gathered by the program. Generally, 80% of the total dataset is used
as the training dataset.
Validation Set: Fine-tuning helps to train the model. If for the training
dataset the accuracy is increasing then a certain portion of data from the
training dataset which is unknown to the model is selected to check that
for that dataset also the accuracy is increasing. If accuracy is not
increasing for the validation set then the program is overfitting the model.
In that case, the developer needs to check the value of the parameters or
he/she may have to reconsider the model.
Test Set: The test set is used to take the output from the model. After the
training, it is used to check how accurate the model is. The rest of the
20% of the dataset is used as a test set.
How does it works
Fig (2): Step by step process
A pattern recognition system will perceive some input from the real world
with sensors. Such a system can work with any type of data: images,
texts, videos, or numbers.
After receiving some information as the input, the algorithm starts to pre-
process the data. That is segmenting something interesting from the
background. For example, when you are given a photo of a park and a
familiar face or any object that attracts the user’s attention, this is pre-
processing.
While the data is in the pre-processing phase it is important to filter the
noise from the main dataset. Depending on the working function of the
application, the filter algorithm will change. For example, consider a face
recognition system where the system is collecting the images for training
purposes.
In order to process the data, it will first convert the images from RGB to
greyscale. Also, the system doesn’t need other areas than the face. So to
filter out unwanted portions of the images and replace them with white or
black background some filter mechanisms are required. Once those filter
mechanisms are used on the data it will be easier for the system to
extract features from the filtered images.
Feature extraction is a process of uncovering some characteristic traits
that are similar to more than one data sample. The derived information
may be general features, which are evaluated to ease further processing.
For example, in image recognition, the extracted features will contain
information about grey shade, texture, shape, or context of the image.
This is the main information used in image processing. The methods of
feature extraction and the extracted features are application dependent.
After extracting the features from the processed data the result of a
pattern recognition system will be either a class assignment (labeled
dataset), or cluster assignment (dataset without labels), or predicted
values (where regression is applied).
Neural Approach
There are quite a few approaches for pattern recognition like Statistical,
Syntactical, and Neural. The statistical approach is nothing but to collect
historical data and based on the observations and analyses from those
data new patterns are recognized. The syntactical approach is also known
as the structural approach as it mainly relies upon sub-patterns called
primitives like words.
The pattern recognition approaches discussed so far are based on direct
computation through machines. Direct computations are based on math
and stats related techniques. Other than those techniques another one is
the neural approach, neural networks related topics are discussed here to
recognize the patterns. As it is known to all neuron is the basic unit of
brain cells and together these neurons create networks to control the
specific tasks. This neural network is implemented in systems. The
outcome of this effort is the invention of artificial neural networks. [3]
An artificial neural network is a computing system that tries to stimulate
the working function of a biological neural network of human brains. In
this network, all the neurons are well connected and that helps to achieve
massive parallel distributing. The input units receive various forms and
structures of information based on an internal weighting system and the
neural network attempts to learn about the information presented to
produce one output report [4].
The advantages of neural networks are their adaptive-learning, self-
organization, and fault-tolerance capabilities. For these outstanding
capabilities, neural networks are used for pattern recognition applications.
An ANN initially goes through a training phase where it learns to recognize
patterns in data, whether visually, aurally, or textually [4]. Some of the
best neural models are back-propagation, high-order nets, time-delay
neural networks, and recurrent nets.
Fig (3): Basic structure of a feed-forward neural network
Normally, only feed-forward networks are used for pattern recognition.
Feed-forward means that there is no feedback to the input. Similar to the
way that human beings learn from mistakes, neural networks also could
learn from their mistakes by giving feedback to the input patterns. This
kind of feedback would be used to reconstruct the input patterns and
make them free from error; thus increasing the performance of the neural
networks. Of course, it is very complex to construct such types of neural
networks. These kinds of networks are called auto-associative neural
networks [3]. This complexity of constructing the network can be avoided
by using back-propagation algorithms. During this supervised phase, the
network compares its actual output produced with what it was meant to
produce—the desired output.
The difference between both outcomes is adjusted using backpropagation.
This means that the network works backward, going from the output unit
to the input units to adjust the weight of its connections between the units
until the difference between the actual and desired outcome produces the
lowest possible error [4]. Local minima is one of the main problems
associated with back-propagation algorithms. In addition, neural networks
have issues associated with hyper-parameters like learning rate,
architecture selection, feature representation, modularity, and scaling.
Though there are problems and obstacles, the application of neural
networks has spread