See discussions, stats, and author profiles for this publication at: [Link]

net/publication/252026828

Feature extraction algorithm for fill level and cap inspection in bottling
machine

Article · June 2011

DOI: 10.1109/ICPAIR.2011.5976910

CITATIONS READS

32 601

3 authors, including:

Anton Satria Prabuwono

King Abdulaziz University
152 PUBLICATIONS 1,080 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Robotic Project View project

All content following this page was uploaded by Anton Satria Prabuwono on 12 April 2022.

The user has requested enhancement of the downloaded file.

 2011 International Conference on Pattern Analysis and Intelligent Robotics
28-29 June 2011, Putrajaya, Malaysia

Feature Extraction Algorithm for Fill Level and Cap

Inspection in Bottling Machine
Leila Yazdi#1, Anton Satria Prabuwono#2, Ehsan Golkar#3
#
Center for Artificial Intelligence Technology (CAIT), Faculty of Information Science and Technology,
Universiti Kebangsaan Malaysia, 43600 UKM Bangi, Selangor D.E., Malaysia
1
[Link]@[Link]
2
antonsatria@[Link]
3
egolkar@[Link]

3
egolkar@[Link]
Abstract—Automated Visual Inspection Systems (AVIS) have a the other hand, especially in the manufacturing environment,
strong ability to improve bottle manufacturing quality control by it is necessary to improve quality control and productivity in
means of inspecting products automatically instead of through order to maintain an advantage over their competitors.
manual inspections. AVIS automatically tends to make a suitable Intelligent visual inspection systems (IVIS) are related to
decision in process and results classification according to the
different subjects such as pattern recognition, image
images of the products via image processing and Artificial
Intelligence techniques. Since bottling is one of the most common processing, machine learning, image analysis, signal
packaging styles in the food and medical industries, in this paper processing and software and hardware behind artificial vision
we will concentrate on the visual inspection of bottles. Checking systems as well as Artificial Intelligent (AI) techniques.
the quality of the cap closure and over-filling/under-filling checks Therefore, extensive research has been done on AVIS to
for the level of the liquid in the bottle have been investigated to propose high levels of visual inspection through merging of
reach an optimized bottle product. Therefore, in this research these areas [4].
general hardware and modules for these systems are In this paper we will concentrate on the bottle industry
investigated. Besides, new techniques of bottle inspection are (bottling machine) which is one of the major and applicable
reviewed along with presenting previous work of other
packaging methods in the packaging industries. Beverage and
researchers. Subsequently we will propose a feature extraction
algorithm to inspect cap closure and level of the liquid in the food industries, milk industries, medicine industries and other
bottle together, in the same system. According to the new chemical product industries are the foremost samples of this
proposed method, our system classifies three situations for cap work. In this area, accurate filling, inspecting cap closure,
condition and three situations for the condition of the level of the sorting recycling plastic bottle, recognition between glass
liquid. As a result the system has investigated 9 situations. The bottle and pet bottles, inspection for over-fill or under-fill,
algorithm of the system will accept its system when the liquid verification of label quality and detection of defect products
level is in the correct position and the cap is in the normal are some significant parameters which seem to be necessary to
condition. Other situations will be rejected. The proper be inspected. Industrial vision systems are not able to handle
algorithm which is proposed here using bottle visual inspection
techniques leads our system to reach an optimized liquid level
all tasks in every field of application [2]. It means that each
with a high quality of the cap closure. system due to its specific requirements and its limitations
needs particular techniques that should be taken into account.
Thus, for any application, particular requirements need to be
Keywords— Feature extraction, bottle inspection, cap closure,
level of content detection. defined reaching to an optimized system subject to its
parameters. Therefore principle issues for any application
I. INTRODUCTION should be discussed separately.
Computer vision systems generally refer to the systems which The basic components for all bottle visual inspection as for
extract desired features from digital images. Increasing the any other AVIS are revealed in Fig. 1. The first step of
scene comprehension from input images is the main objective designing a visual inspection system is the image acquisition
of these systems [1]. Application of the vision systems in which is concerned about hardware equipment to increase
industries in order to automate manufacturing process is quality of the initial image for further processing [5]. As we
considered as Machine Vision [2]. Also it is defined as know, camera features (number, kind and position of camera),
Automated Visual Inspection (AVI) when attempts are made colour of background and speed of the conveyor belt have
to inspect, control products and recognize the defects by using important consequences on the image quality. Additionally the
only product images (quality control) [3]. In fact, humans as position and brightness of the light are significant factors in
inspectors are slower and their efficiency is affected by their the inspection system design (e.g. in reflectance especially
states of exhaustion, illness or other human shortcomings. In when bottle is transparent) [9]. After image capturing, there is
some applications they need training to attain proper skills. the necessary use of computational techniques to filter,
Moreover, there are sometimes special environments which analyze, restore, compress and reconstruct the images to
are dangerous and not conducive for human operation [2]. On remove various types of noise in the image and enhance its

978-1-61284-406-0/11/$26.00 ©2011 IEEE

CVIP-09 47
quality. Also, to simplify the digital image, pixels need to be horizontal projection has been used. Author’s technique has
partitioned and labelled. Lines or curves inside of the image 99% accuracy when light is changing and 100% when
should be identified properly. In fact relevant information infusion bottles are tilted.
from the image is required to be extracted to reduce the data The performance of Canny, Log and ISEF edge detection
sets. Based on the feature vector from the previous step, the methods have been compared to measure levels of liquid in
identity of the object will be verified in order to decide in bottles. It is assumed that these methods are much more
which class the inspected object belongs. The classes should accurate compared to traditional methods [7], since another
be distinct and exclusive. Supervised classification can be powerful technique in the image segmentation is thresholding.
used in this step. General classification algorithms have both Two-dimensional Otsu algorithms proposed which indicate
training and testing parts for bottling machines. Neural that one dimensional has more speed and resistance frontage
networks, Fuzzy, SVM and Neural-fuzzy are some of the of noise but do have some disadvantages that should be taken
applicable classifiers. into account such as its need to use lots of space and having
low accuracy when it is not enough [8].
In the case of bottle classification, there are different kinds
of methods which are applicable in the bottle inspection
technique. Chink defects in glass are detected bottles in order
to recycle bottles by using low-level and high-level neural
networks as in [10]. Moreover they have continued their work
by Fuzzy Support Vector Machine (FSVM) and reached up to
98% accuracy [11]. Reference [12] in investigating lids of
glass bottles have applied FSVM and later have classified
their system by FSVMNN (Fuzzy Support Vector Machine
Neural Networks) [13]. Their classifier is accurate up to 97%.
Reference [14] in a case of improving training in neural
network as a classifier, have proposed districted neural
network which it is more powerful in pattern recognition.
Fig. 1 General component for a typical bottling AVIS Reference [15] has used shapes and morphological operations
to distinguish between different plastic bottles. In fact the
In this paper, we have investigated basic principles of bottle morphological method has been applied for the feature
visual inspection. Moreover, we have presented different extraction and obtaining of histograms. Besides they have
methods and techniques for bottling machines and most recent considered neural network as a classifier. Their method
algorithms have been stated. In this research, we have focused reduces feature data sets properly.
on investigating two quality inspections in bottles: cap closure In the specific case of the bottling machine a
and level of content detection. We have proposed AVIS which classification system is developed which is able to identify
inspects these two parameters together. Besides, we have bottles from their colour and synthesis [16]. They applied a
presented a Feature Extraction algorithm for detection of cap CCD camera along with a quadratic classifier for colour
condition and level of content and it has been explained recognition. Another developed system for sorting plastic
completely through figures. bottle composition was investigated as in [17]. They have
presented automatic ROIS for extraction of features from
II. PREVIOUS WORK
plastic bottles in order to classify bottles into pet and non pet.
In 2007, a low-end webcam within distance of 15 cm from The results show 80% accuracy.
cap of bottle is used for detection of cap closure. Also a dark
background of bottle was selected and speed of the conveyor III. PROPOSED METHOD
was 106 rpm. The accuracy of image acquisition has been As we have explained before, in the bottle visual inspection
reached to 94.264 % for a moving bottle. In order to enhance systems, detection of the liquid level and cap closure have
the image quality, they chose a proper threshold value to been inspected separately. In this paper, we will propose an
increase image contrast. The Sobel’s algorithm was applied as Automatic Visual Inspection System (AVIS) which is capable
an edge detection technique. Moreover a back propagation of investigating both detection of liquid level and cap closure
neural network system was applied to determine three bottle together in the bottling machine, something which has not
cap situations: without-cap and misplaced-cap situations will been done yet. First we present the general modules and
be rejected and a normal situation will gain acceptance [5]. image processing methods which are essential in our systems.
In order to inspect level of content in a fusion bottle, HIK Then we will state our Feature Extraction (FE) technique. The
Vision DS-2CD812PF is used as IP Camera, Intranet, figures are captured from plastic bottles.
Computer and Alarm [6]. Also the Sobel’s algorithm as the
edge detection is applied and it is improved into two direction
vertical and horizontal edges. It is said that image bineryzation
has been applied to find out the threshold value and after that
the filtering method is performed. To find horizontal lines,

48
 unpacked cap and fixed cap. An acceptable condition is a
sealed cap. Besides this, for the liquid level investigation, we
A. Proposed Modules will determine that the liquid level of the bottle is under-filled,
In this part, essential modules for making this system have over-filled or filled to the desired liquid level. The just-desired
been explained step by step. There are seven modules, for level is the acceptable condition in this case. Consequently
which the relations between them are shown in a diagram (Fig. this system has 9 different situations which are obtained by 3
2). In fact this diagram gives a general view of the proposed conditions of liquid level multiplied by 3 conditions of cap
system. investigations. If the cap is fixed and the liquid level is in the
correct range of the bottle, the algorithm accepts its output,
1) Image acquisition condition: To reduce the light
otherwise the output is rejected. Therefore, there are 8
reflection problems in the bottles especially in glass bottles,
rejection conditions. The diagram for the proposed AVIS is
we suppose LED as a lighting source. Also we assume our
demonstrated in the Fig. 2. All steps to generate such system
bottles are without any label. The background of the bottles
will be light and the camera captures the image from one side are shown step by step.
of the bottles.
2) Image enhancement (filtering): In this step, for the
removal of noise, filtering techniques are necessary.
3) Image segmentation: This step shows the most
effectiveness in our study. Since this system should be able to
identify horizontal lines as well, the edge detections such as
Canny, Log and two dimensional Sobel will be applicable. We
suggest applying the appropriate threshold for better edge
detection.
4) Morphological technique: After using edge detection
methods, morphological closing can be used to remove noise
reaching to the edges and lines more noticeably.
5) Feature extraction: To detect the level of liquid content
in the bottles and to investigate the cap condition, we will
propose an appropriate algorithm in which two sub-algorithms
introduced. Therefore, our system is separated into two main
categories of cap checking and liquid level detection subject
to their sub-algorithm. To extract the desired features, the
first step in this algorithm is to find the region of interests. It
means that the regions of the image which are relevant to get
the desired information in the case of cap and level of liquid
have to be selected. According to the Fig. 2, for the cap
closure detection and the liquid level measurement, the
algorithm should find two regions for each of the detections.
Based on the main algorithm, one of the regions is determined
by the reference line which is a line at the bottom of the bottle
cap. Another important parameter in our algorithm is the
prototype line which should be a defining former as a static
value for result comparisons. The algorithm has been
completely explained in the following part along with figures
which demonstrate all regions and lines properly. Fig. 2 Diagram for detect cap closure and level of content

6) Image classification: In this step, the average line from B. Technique

the previous step should be compared with the prototype line. As has been mentioned above, the proposed system in the
The prototype line is a determined line which is a constant Fig. 2 has the potential to detect cap closure and the level of
criterion value for measuring expected liquid level and cap the liquid content together by an appropriate algorithm, which
closure. The value of the prototype line will be considered has not been done yet. Table1 and Table2 show the main
separately subject to the case. Classification of liquid level algorithm procedure for both liquid level detection and cap
and cap closure will be prepared separately. Different Neural situation clearly. The main algorithm is included in two parts:
Network techniques (NN) such as Back Propagation (BP) can part A and B. Part A belongs to the inspection of cap
be applied properly in this work. The output of our algorithm condition and part B is for level of content detection.
will be 3 cap conditions as the final result which are: no cap,

49
 Level detection: to measure level of liquid contents we need inspection Reg1 starts from the reference line to the top of the
to find two regions of interest (ROIs). We call the first one cap. The Reg2 will be determined by means of the edge
Reg1 (yellow box). This region starts from the bottom of the detection algorithm (Table 2) and vertical lines should be
cap downward to the determined distance from the reference illustrated and an average obtained from them.
line. The reference line is a fixed horizontal line in the bottom
of the cap (blue line in Fig. 3, 5, 6 and 7). Another region of TABLE 2
interest (Reg2) that is essential in our system is determined ALGORITHM FOR DETECTION OF CAP CLOSURE CONDITION
from horizontal line detection (illustrated in Fig. 2).Then
vertical lines from the image pixels in Reg2 which are Algorithm 2
positioned inside of the Reg. 1 should be considered to be 1. Capturing image
illustrated. Next, the obtained lines of the previous step should 2. Apply smoothing filter
be averaged in the whole domain when reaching towards the 3. Using edge detection
final result (Fig. 3, 5, 6 and 7). 4. Apply binarization technique on previous step
5. Finding horizontal lines
6. Distinct the horizontal line at the bottom of the
TABLE 1 cap as a Reference line ( R[i,j] )
ALGORITHM FOR LEVEL OF CONTENT DETECTION
7. Draw a line as parallel as Reference line and the
Algorithm 1 cap (L [i.j]) as a Reg1.
1. Capturing image
2. Apply smoothing filter For each i and j on the L
3. Using edge detection Define Prototype Line as P[ i,
4. Apply binarization technique on previous step Distance_Pixel ( L(i,j] AND R( i,j) ) / 2 ].
5. Finding horizontal lines End For
6. Distinct the horizontal line at the bottom of the cap
as a Reference line ( R[i,j] ) 8. Define area between reference line and longest
7. Draw a line as parallel as Reference line and level horizontal line on top ( H[i,j] )in top of image as
of content ( L[i.j]) as a Reg1 a Reg2
For each i AND j on the H
For each i and j on the L Define Average Line as A[ i,
Define Prototype Line as P[ i, Distance_Pixel ( H(i,j] AND R( i,j) ) / 2 ]
Distance_Pixel ( L(i,j] AND R( i,j) ) / 2 ] End For
End For 9. For Each i And j in A[i,j] and P[i,j]
Dis_Value = Distance (A[i,j] , P[i,j])+
8. Define area between reference line and longest Dis_Value
horizontal line on top (H[i,j] )in top of the image End For
as a Reg2. 10. IF ( Dis_Value > Threshold_Value )
For each i AND j on the H The bottle is without cap(Fig.5)
Define Average Line as A[ i, Distance_Pixel ELSE IF ( Dis_Value < Threshold_Value)
( H(i,j] AND R( i,j) ) / 2 ] The cap of bottle is not fixed(Fig.6)
End For ELSE
9. For Each i And j in A[i,j] and P[i,j] the cap is in appropriate situation (Fig. 3)
Dis_Value = Distance (A[i,j] , P[i,j])+
Dis_Value
End For To detect rejected conditions as well as the accepted one,
we require an enhancing image in order to obtain clear
10. IF ( Dis_Value > Threshold_Value ) horizontal lines and an abstract image from the background.
The level of liquid is over fill (Fig.6) Fig. 3 (a) is the original image and Fig. 3 (b) is the enhanced
ELSE IF ( Dis_Value < Threshold_Value) image which displays only accepted conditions while the level
The bottle is under filled(Fig.5) is in the correct position and the cap is fixed. In the Fig. 3 (b)
ELSE regions of interests (Reg1 and Reg2) according to the
the level of liquid is in appropriate situation algorithm (Fig. 2) are drawn. Next, average lines are
(Fig. 3) calculated to be compared with prototypes lines.

Cap condition: the algorithm of the detection location of the

cap is the same as the liquid level detection. Fig. 3 (b), 5, 6
and 7 show the following explanations. In this type of

50
 Regarding Fig. 4, even if any rejected condition of level
detection or cap closure happens, our system should recognize
it as a rejection condition. For example if the cap is in the
fixed condition but the liquid level is under-filled (Fig. 4 (e)),
the system rejects this bottle. The algorithm for identifying
cap condition is shown in Table 1. To make clear the
situation of the rejected conditions, the Fig. 5 has been
illustrated.
(a) Original image
Fig. 5 reveals an enhanced image of Fig. 4 (b). It is
illustrated in the case of a no-cap condition but the level is in a
good situation. In this case the average line from vertical lines
in the cap area is less than the prototype line, so according to
Table 1, this feature demonstrates a no-cap situation. However
the average line for the level is in a proper range.

(b) Enhanced image with horizontal and vertical lines

Fig. 3 Accept situation, cap and level are normal

There are eight reject situations for cap and level. All of them
are demonstrated in Fig. 4 (a)-(h).
Fig. 5 Enhanced image of Fig. 4 (b) within Reg1 and Reg2 and average
lines

In the Fig. 6, the cap is not fixed but the level is correct.
Since Reg2 around the cap region is not rectangular, the
average line will be at the top of the prototype line. So
according to the Table 1, this situation demonstrates that the
cap is not fixed.
(a) No cap, liquid is over fill (b) Unfixed cap, liquid is over fill

(c) No cap, normal level (d) Unfixed cap, normal level Fig. 6 Enhanced image of Fig. 4 (c) within Reg1 and Reg2 and average lines

According to the Fig. 7 for liquid level detection there is no

average line. So this state will be considered as an under-filled
condition which will be rejected.

(e) Normal cap, over fill liquid (f) Normal cap, under fill liquid

Fig. 7 Enhanced image of Fig. 4 (g) within Reg1 and Reg2 and average
lines

(g) Unfixed cap, under fill liquid (h) No cap, under fill liquid

Fig.4 Eight reject situations

51
C. Hardware Setup REFERENCES
As we explained before, the experimental setup has a [1] S. Ramli, M. M. Mustafa, D. A. Wahab, and A. Hussain, “Plastic bottle
vital role in all AVI systems in order to capture the best shape classification using partial erosion-based approach,” in Proc. 6th
image. So in Table 3, we have cleared some experimental International Colloquium on Signal Processing & Its Applications
(CSPA'10), 2010.
conditions like source lights, kind and number of cameras, [2] E. N. Malamas, E. G. M. Petrakis, M. Zervakisa, L. Petit, and J. Legat,
distances between bottle and cameras and speed of moving “A survey on industrial vision systems, applications and tools,” Image
bottle on the conveyor. Besides we have suggested using and Vision Computing., vol. 21, pp. 171–188, 2003.
the C++ programming language. [3] T. Petkovi´cy, J. Krapacz, S. Loncari´cy, and M. Sercerz,
“Automated visual inspection of plastic products,” in Proc. 11th
TABLE 3 International Electrotechnica (ERK'02), 2002, pp. 283-286 .
EXPERIMENT SETUP [4] Y. Cheng, M. Jafari and I. Bakhadyrov, “An online 3-D visual
inspection and process control”, Univ. of Rutgers, NJ, USA, Rep.
Light source One white LED MURI project #N-0014-96-1-1175.
Background Black [5] A. S. Prabuwono, R. Sulaiman, A. R. Hamdan, and Hasniaty A.,
“Development of intelligent visual inspection system (IVIS) for
Speed of 10 m/Second bottling machine”, in Proc. IEEE Tencon'07, 2007.
conveyor belt [6] H. Zhu, “New algorithm of liquid level of infusion bottle based on
camera One Webcam image processing,” in Proc. International Conference on Information
Engineering and Computer Science, 2009.
Distance of 15-20 cm from cap of bottle
[7] K. J. Pithadiya, C. K. Modi, and J. D. Chauhan, “Comparison of
camera from optimal edge detection algorithms for liquid level inspection in bottles,”
bottle in Proc. Second International Conference on Emerging Trends in
Kind of bottle Plastic bottle without label Engineering and Technology, 2009, pp. 447-452.
[8] C. Lu, P. Zhu, and Y. Cao, “The segmentation algorithm of
Programming C++ improvement a two-dimensional Otsu and application research,” in
language Proc. 2nd International Conference on Software Technology and
Engineering (ICSTE'10), 2010, pp. v1-76-v1-79.
[9] J. Honec, P. Honec, P. Petyovský, S. Valach, and J. Brambor,
IV. DISCUSSION “Transparent materials optical inspection methods,” in Proc. 13th Int.
Conference on Process Control, 2001.
An accurate level of the content and detection of the cap [10] F. Duan, Y. Wang, and H. Lin, “A Real-time Machine Vision System
closure are two main challenges in the bottling packaging for Bottle Finish Inspection,” in Proc. 8th International Conference on
quality. So in this paper, we have proposed a new Feature Control, Automation, Robotics and Vision (IEEE), 2004.
[11] Y. N. Wang, H. J. Liu, and F. Duan, “A bottle finish inspection method
Extraction (FE) technique to investigate these two parameters based on fuzzy support vector machine and wavelet transform,” in
in one AVIS. The advantage of our system is to improve Proc. 4th Fourth International Conference on Machine Learning and
inspection by one algorithm which simplifies this system. For Cybernetics(IEEE), 2005, vol.8, pp. 4588-4592.
each of the features, there are 3 situations. So our techniques [12] H. Liu, Y. Wang, and F. Duan, “An empty bottle intelligent inspector
based on support vector machines and fuzzy theory,” in Proc. 6th World
should be able to classify the results in 9 situations: no cap- Congress on Intelligent Control and Automation, 2006, pp. 9739-9743.
liquid is over-filled, unfixed cap-liquid is over-filled, no cap- [13] H. Liu and Y. Wang, “Development of a computerized method to
normal level, unfixed cap-normal level, normal cap-over-filled inspect empty glass bottle,” in Proc. International Symposium on
liquid, normal cap-under-filled liquid, unfixed cap-under- Computer Science and Computational Technology, 2008, pp. 118-123.
[14] L. Chen, “Pattern classification by assembling small neural networks,”
filled liquid, no cap, under-filled liquid and normal cap- in Proc. International Joint Conference on Neural Networks, 2005,
normal level of content. However, the AVIS presented here vol.3, pp. 1947-1952.
seems to be potentially powerful to detect cap and level of [15] S. Ramli, M. M. Mustafa, and A. Hussain, “A morphological-based
content situations by the FE algorithm; we predicted the approach for plastic bottle shape classification,” in Proc. International
Conference on Electrical Engineering and Informatics, 2007, pp. 214-
system will have difficulty while the level is too low or empty. 217.
It means that it may baffle two cases of the empty bottle with [16] Y. Tachwali, Y. Al-Assaf, and A. R. Al-Ali, “Automatic multistage
the full ones. classification system for plastic bottles recycling,” Resources,
Conservation and Recycling, vol. 52, pp. 266–285, Nov. 2007.
V. CONCLUSIONS [17] S. Ramli, M. M. Mustafa, A. Hussain, and D. A. Wahab, “Automatic
Detection of ‘ROIs’ for Plastic Bottle Classification,” in Proc. 5th IEEE
Since one of the most popular topics in industry is the Student Conference on Research and Development, 2007, pp.205-209.
automated visual inspection system and bottles are applicable .
in the food and medical industries, we have investigated
general issues in the bottle inspection systems. Besides
increasing the quality of the bottle packaging style we have
concentrated on inspection of the cap closure and level of the
liquid content in the bottle. We have proposed a system which
will be able to inspect these two parameters together at the
same time in an appropriate algorithm. The expected results
from the system have been discussed in detail.

52

View publication stats