Pattern analysis deals with the automatic detection of patterns in data, and plays
a central role in many modern artificial intelligence and computer science
problems. By patterns we understand any relations, regularities or structure
inherent in some source of data. By detecting significant patterns in the
available data, a system can expect to make predictions about new data coming
from the same source. In this sense the system has acquired generalisation
power by ‘learning’ something about the source generating the data. There are
many important problems that can only be solved using this approach, problems
ranging from bioinformatics to text categorization, from image analysis to web
retrieval. In recent years, pattern analysis has become a standard software
engineering approach, and is present in many commercial products.
Pattern Recognition has attracted the attention of researchers in last few
decades as a machine learning approach due to its wide spread application
areas.
The various application areas of pattern recognition are like bioinformatics,
document classification, image analysis, data mining, industrial automation,
biometric recognition, remote sensing, handwritten text analysis, medical
diagnosis, speech recognition, GIS and many more. Similarity between all these
applications is that for a solution-finding approach features have to be extracted
and then analyzed for recognition and classification purpose. Three processes
take place in pattern recognition task. First step is data acquisition. Data
acquisition is the process of converting data from one form (speech, character,
pictures etc.) into another form which should be acceptable to the computing
device for further processing. Data acquisition is generally performed by
sensors, digitizing machine and scanners. Second step is data analysis. After
data acquisition the task of analysis begins. During data analysis step the
learning about the data takes place and information is collected about the
different events
and pattern classes available in the data. This information or knowledge about
the data is used for further processing. Third step used for pattern recognition is
classification. Its purpose is to decide the category of new data on the basis of
knowledge received from data analysis process. Data set presented to a Pattern
Recognition system is divided into two sets: training set and testing set. System
learns from training set and efficiency of system is checked by presenting
testing set to it. The performance of the pattern recognition techniques is
influenced by mainly three elements (i) amount of data (ii) technology
used(method) (iii) designer and the user. The challenging job in pattern
recognition is to develop systems with capability of handling massive amounts
of data. The various models opted for pattern recognition are:
Statistical Techniques, Structural Techniques, Template Matching, Neural
Network based techniques, Fuzzy models and Hybrid Models.
Statistical Model
In Statistical method of Pattern Recognition each pattern is described in terms
of features. Features are chosen in such a way that different patterns occupy
non-overlapping feature space. It recognizes the probabilistic nature both of the
information we seek to process, and of the form in which we should express it
It works well when the selected features lead to feature spaces which cluster in
a recognizable manner, i.e. there is proper interclass distance.
After analyzing the probability distribution of a pattern belonging to a class,
decision boundary is determined
Here patterns are projected to pre-processing operations to make them suitable
for training purposes.
Features are selected upon analyzing training patterns. System learns and
adapts itself for unknown patterns as shown in Fig. 1.
Fig1. Statistical Pattern Recognition Model
Test patterns are applied to check suitability of system to recognize patterns.
Feature measurement is done while testing, then these feature values is
presented to learned system and in this way classification is performed.
When conditional probability density distribution is known, parametric
classification schemes are used otherwise non parametric classification scheme
need to be used. Various decision rules are there to determine decision
boundary like, Bayes Decision Rule, Optimal Bayes Decision Rule, The
Maximum Likelihood Rule, Neyman-Pearson rule and MAP rule Depending upon
whether the method opted is supervised or unsupervised statistical technique can be categorized as:
Discriminant Analysis and Principal Component Analysis [1].
Discriminant Analysis is a supervised technique in which we approach for
dimensionality reduction. Here linear combination of features is utilized to
perform the classification operation. For each pattern class, a discriminant
function is defined which performs the classification function [5] - [8]. There is
not a well defined rule regarding the form of discriminant function like
minimum distance classifier uses one reference point for each class, and
discriminant function computes minimum distance from unknown vectors to
these points, on the other hand nearest neighbor classifier uses set of points for
each class. There are various kinds of Discriminant Analysis methods that are
used based upon the application and system requirement such as: Linear
Discriminant Analysis (LDA), Null-LDA (N-LDA), Fisher Discriminant
Analysis (FDA), Two Dimensional Linear Discriminant Analysis (2D-LDA),
Two Dimensional Fisher Discriminant Analysis (2D-FDA). In LDA feature set
is obtained by linear combination of original features. Intra-class distance is
minimized as well as inter-class distance is maximized to obtain the optimum
[Link] suffers from small sample size (SSS) problem.
Principal Component Analysis (PCA) or Karhunen-Loeave expansion is a multi-element unsupervised
technique in which we approach for dimensionality reduction [10]. Using PCA, patterns are detected in the data
and these patterns determine the similarity measure [11]. In PCA Eigen vectors with largest Eigen values are
computed to form the feature space. PCA is closely related to Factor Analysis [11].
B. Structural Model
When we came across patterns with strong inherent structures, statistical methods give ambiguous results,
because feature extraction destroys vital information concerning the basic structure of pattern. Therefore, in
complex pattern recognition problems, like recognition of multidimensional objects it is preferred to adopt a
hierarchical system, where a pattern is considered to be made up of simple sub-patterns, which are further
composed of simpler sub patterns [16], [17]. In structural approach of pattern recognition a collection of
complex patterns are described by a number of sub-patterns and the grammatical rules with which these sub
patterns are associated with each other. This model is concerned with structure and attempts to recognize a
pattern from its general form. The language which provides structural description of patterns in terms of pattern
primitives and their composition is termed as pattern description language. Increased descriptive power of a
language leads to increased complexity of syntax analysis system
C. Template Matching Model
Template matching is simplest and most primitive among all pattern recognition models. It is used to determine
the similarity between two samples, pixels or curves. The pattern to be recognized is matched with the stored
templates while assuming that template can be gone through rotational or scalar changes. The efficiency of this
model depends upon the stored templates. Correlation function is taken as recognition function and is optimized
depending on the available training set. The shortcoming of this approach is that, it does not work efficiently in
the presence of distorted patterns [29].
D. Neural Network Based Model
Neural networks are the massively parallel structures composed of “neuron” like subunits [29]. Neural networks
provide efficient result in the field of classification. Its property of changing its weight iteratively and learning
[10], [30], give it an edge over other techniques for recognition process. Perceptron is a primitive neuron model.
It is a two layer structure. If output function of perceptron is step, then it performs classification problems, if it is
linear than it perform regression problems. The most commonly used family of neural networks for pattern
classification is the feed forward networks like MLP and RBF networks [31]. Different types of neural networks
are used depending upon the requirement of the application.
E. Fuzzy Based Model
The importance of fuzzy sets in Pattern Recognition lies in modeling forms of uncertainty that cannot be fully
understood by the use of probability theory [37], [38]. Kandel [39] states, “In a very fundamental way, the
intimate relation between theory of fuzzy sets and theory of Pattern Recognition and classification rests on the
fact that most real world classes are fuzzy in nature”, Kandel defined various techniques of fuzzy pattern
recognition. Syntactic techniques are utilized when the pattern sought is related to the formal structure of
language. Semantic techniques are used when fuzzy partitions of data sets are to be produced. Then a similarity
measure based on weighted distance is used to obtain similarity degree between the fuzzy description of
unknown shape and reference shape.
F. Hybrid Model
In most of the emerging applications, it is clear that a single model used for classification doesn’t behave
efficiently, so multiple methods have to be combined together giving result to hybrid models. Primitive
approaches to design a Pattern Recognition system which aims at utilizing a best individual classifier have
some drawbacks [40]. It is very difficult to identify a best classifier unless deep prior knowledge is available at
hand [40], [41]. Statistical and Structural models can be combined together to solve hybrid problems. In such
cases statistical approach is utilized to recognize pattern primitives and syntactic approach is then used for the
recognition of sub-patterns and pattern itself. Fu [28] gave the concept of attributed grammars which unifies
statistical and structural pattern recognition approach. To enhance system performance one can use a set of
individual classifiers and combiner to make the final decision. Tumer and Ghosh [42] experimentally proved
that using a linear combiner or order statistics combiner minimize the variance of actual decision boundaries
around the optimal boundary. Multiple classifiers can be used in several ways to enhance the system
performance. Each classifier can be trained in a different region of feature space or in other way, each classifier
can provide probability estimate and decision can be made upon analyzing individual results. Methods utilizing
classifier ensemble design [43], [44] generate a set of mutually complementary classifiers that achieve optimal
accuracy using a fixed decision function. Those methods which utilize combination function design tend to find
an optimal combination of decisions from a set of classifiers. To achieve optimum results, a large set of
combination functions of increasing complexity, ranging from simple voting rules through trainable
combination functions is available to designer [45] - [47].
Pattern Recognition (PR)
• Statistical PR
• Syntactic PR
• Fuzzy logic PR
• Neural PR
Example - basketball players and
jockeys
We will keep practical applicability in
mind:
• Are PR techniques suitable to the
problem?
• Can we develop useful models and
determine
model parameters?
• Are there available formal tools and
applicable
heuristics?
• Does a computationally practical
solution exist?
Applications
• Image processing, analysis, machine
vision
• Seismic analysis
• Radar signal analysis
• Face recognition
• Speech recognition
• Fingerprint identification
• Character recognition
• Medical diagnosis
PATTERN RECOGNITION: OPEN PROBLEMS AND THE ROLE OF
FUZZY LOGICPATTERN RECOGNITION: OPEN PROBLEMS AND THE ROLE OF
FUZZY LOGICPATTERN RECOGNITION: OPEN PROBLEMS AND THE ROLE OF
FUZZY LOGICPATTERN RECOGNITION: OPEN PROBLEMS AND THE ROLE OF
FUZZY LOGIC