Pattern Recognition 38 (2005) 2270 – 2285

[Link]/locate/patcog

Score normalization in multimodal biometric systems夡

Anil Jaina , Karthik Nandakumara , Arun Rossb,∗
a Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA
b Department of Computer Science and Engineering, West Virginia University, Morgantown, WV 26506, USA

Received 23 December 2003; received in revised form 18 January 2005; accepted 18 January 2005

Abstract
Multimodal biometric systems consolidate the evidence presented by multiple biometric sources and typically provide
better recognition performance compared to systems based on a single biometric modality. Although information fusion in a
multimodal system can be performed at various levels, integration at the matching score level is the most common approach
due to the ease in accessing and combining the scores generated by different matchers. Since the matching scores output by the
various modalities are heterogeneous, score normalization is needed to transform these scores into a common domain, prior
to combining them. In this paper, we have studied the performance of different normalization techniques and fusion rules in
the context of a multimodal biometric system based on the face, fingerprint and hand-geometry traits of a user. Experiments
conducted on a database of 100 users indicate that the application of min–max, z-score, and tanh normalization schemes
followed by a simple sum of scores fusion method results in better recognition performance compared to other methods.
However, experiments also reveal that the min–max and z-score normalization techniques are sensitive to outliers in the data,
highlighting the need for a robust and efficient normalization procedure like the tanh normalization. It was also observed that
multimodal systems utilizing user-specific weights perform better compared to systems that assign the same set of weights to
the multiple biometric traits of all users.
䉷 2005 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Biometrics; Multibiometrics; Score normalization; Face; Fingerprint; Hand-geometry; Tanh; Min–max; z-score; Sigmoid;
Parzen; User-specific weights

1. Introduction biometric systems) are often affected by several practical

problems like noisy sensor data, non-universality and/or
Biometric systems make use of the physiological and/or lack of distinctiveness of the biometric trait, unacceptable
behavioral traits of individuals, for recognition purposes [1]. error rates, and spoof attacks [2]. Multimodal biometric
These traits include fingerprints, hand-geometry, face, voice, systems overcome some of these problems by consolidat-
iris, retina, gait, signature, palm-print, ear, etc. Biometric ing the evidence obtained from different sources [3]. These
systems that use a single trait for recognition (i.e., unimodal sources may be multiple sensors for the same biometric
(e.g., optical and solid-state fingerprint sensors), multiple
夡 This research was supported by the Center for Identification instances of the same biometric (e.g., fingerprints from dif-
ferent fingers of a person), multiple snapshots of the same
Technology Research (CITeR), a NSF/IUCRC program.
∗ Corresponding author. Tel.: +1 304 293 0405; biometric (e.g., four impressions of a user’s right index
fax: +1 304 293 8602. finger), multiple representations and matching algorithms
E-mail addresses: jain@[Link] (A. Jain), for the same biometric (e.g., multiple face matchers like
nandakum@[Link] (K. Nandakumar), ross@[Link] PCA and LDA), or multiple biometric traits (e.g., face and
(A. Ross). fingerprint).

0031-3203/$30.00 䉷 2005 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/[Link].2005.01.012
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2271

The use of multiple sensors addresses the problem of tion schemes on the performance of a multimodal biomet-
noisy sensor data, but all other potential problems associ- ric system based on the face, fingerprint and hand-geometry
ated with unimodal biometric systems remain. A recogni- modalities.
tion system that works on multiple instances of the same The rest of the paper is organized as follows: Section 2
biometric can ensure the presence of a live user by ask- presents a brief overview of the various approaches used for
ing the user to provide a random subset of biometric mea- information fusion in multimodal biometrics and motivates
surements (e.g., left index finger followed by right middle the need for score normalization prior to matching score
finger). Multiple snapshots of the same biometric, or mul- fusion. Section 3 describes different techniques that can be
tiple representations and matching algorithms for the same used for the normalization of scores obtained from different
biometric may also be used to improve the recognition per- matchers. The experimental results are presented in Section
formance of the system. However, all these methods still suf- 4 and we have outlined our conclusions in Section 5.
fer from many of the problems faced by unimodal systems.
A multimodal biometric system based on different traits
is expected to be more robust to noise, address the prob- 2. Fusion in multimodal biometrics
lem of non-universality, improve the matching accuracy, and
provide reasonable protection against spoof attacks. Hence, A biometric system has four important modules. The sen-
the development of biometric systems based on multiple sor module acquires the biometric data from a user; the fea-
biometric traits has received considerable attention from ture extraction module processes the acquired biometric data
researchers. and extracts a feature set to represent it; the matching module
In a multimodal biometric system that uses different bio- compares the extracted feature set with the stored templates
metric traits, various levels of fusion are possible: fusion at using a classifier or matching algorithm in order to generate
the feature extraction level, matching score level or decision matching scores; in the decision module the matching scores
level (as explained in Section 2). It is difficult to consoli- are used either to identify an enrolled user or verify a user’s
date information at the feature level because the feature sets identity. Sanderson and Paliwal [5] have classified informa-
used by different biometric modalities may either be inac- tion fusion in biometric systems into two broad categories:
cessible or incompatible. Fusion at the decision level is too pre-classification fusion and post-classification fusion (see
rigid since only a limited amount of information is available Fig. 1).
at this level. Therefore, integration at the matching score Pre-classification fusion refers to combining information
level is generally preferred due to the ease in accessing and prior to the application of any classifier or matching algo-
combining matching scores. rithm. In post-classification fusion, the information is com-
In the context of verification, fusion at the matching score bined after the decisions of the classifiers have been ob-
level can be approached in two distinct ways. In the first tained.
approach the fusion is viewed as a classification problem,
while in the second approach it is viewed as a combination
problem. In the classification approach, a feature vector is 2.1. Pre-classification fusion
constructed using the matching scores output by the indi-
vidual matchers; this feature vector is then classified into Prior to classification/matching, integration of informa-
one of two classes: “Accept” (genuine user) or “Reject” (im- tion can take place either at the sensor level or at the feature
postor). In the combination approach, the individual match- level. The raw data from the sensors are combined in sensor
ing scores are combined to generate a single scalar score level fusion. For example, the face images obtained from
which is then used to make the final decision. Both these several cameras can be combined to form a single face image
approaches have been widely studied in the literature. Ross [6]. In sensor level fusion, the data obtained from the differ-
and Jain [4] have shown that the combination approach per- ent sensors must be compatible, and this may not always be
forms better than some classification methods like decision possible (e.g., it may not be possible to fuse face images ob-
tree and linear discriminant analysis. However, no single tained from cameras with different resolution). Feature level
classification or combination scheme works well under all fusion refers to combining different feature vectors that are
circumstances. In this paper, we use the combination ap- obtained by either using multiple sensors or employing mul-
proach to fusion and address some of the issues involved tiple feature extraction algorithms on the same sensor data.
in computing a single matching score given the scores of When the feature vectors are homogeneous (e.g., multiple
different modalities. Since the matching scores generated fingerprint impressions of a user’s finger), a single resultant
by the different modalities are heterogeneous, normaliza- feature vector can be calculated as a weighted average of the
tion is required to transform these scores into a common individual feature vectors. When the feature vectors are non-
domain before combining them. While several normaliza- homogeneous (e.g., feature vectors obtained using different
tion techniques have been proposed, there has been no de- feature extraction techniques, or feature vectors of different
tailed study of these techniques. In this work, we have biometric modalities like face and hand geometry), we can
systematically studied the effects of different normaliza- concatenate them to form a single feature vector. Concate-
2272 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

Information Fusion

Pre-classification Post-classification

Sensor level Feature level Dynamic Classifier

Classifier Fusion
i) Weighted Summation Selection
ii) Concatenation

Measurement level Rank level Abstract level

Classification Combination i) Highest Rank i) Majority

Approach Approach ii) Borda Count Voting
iii) Logistic ii) Behavior
i) Neural Regression Knowledge
Networks i) Normalization + Space
ii) k-NN Linear combination of iii) Dempster
iii) Decision scores + Thresholding Shafer Theory
Trees ii) Normalization + of Evidence
iv) SVM {Sum, Product, Max, Min} iv) AND Rule
Rules +Thresholding v) OR Rule

Fig. 1. Approaches to information fusion.

nation is not possible when the feature sets are incompatible 2.2. Post-classification fusion
(e.g., fingerprint minutiae and eigenface coefficients).
Biometric systems that integrate information at an early Schemes for integration of information after the classi-
stage of processing are believed to be more effective than fication/matcher stage can be divided into four categories:
those systems which perform integration at a later stage. dynamic classifier selection, fusion at the abstract level, fu-
Since the features contain richer information about the input sion at the rank level and fusion at the matching score level.
biometric data than the matching score or the output deci- A dynamic classifier selection scheme chooses the results of
sion of a classifier/matcher, integration at the feature level that classifier which is most likely to give the correct deci-
should provide better recognition results than other levels sion for the specific input pattern [8]. This is also known as
of integration. However, integration at the feature level is the winner-take-all approach and the device that performs
difficult to achieve in practice because of the following rea- this selection is known as an associative switch [9].
sons: (i) The relationship between the feature spaces of dif- Integration of information at the abstract or decision level
ferent biometric systems may not be known. In the case can take place when each biometric matcher individually
where the relationship is known in advance, care needs to decides on the best match based on the input presented to
be taken to discard those features that are highly correlated. it. Methods like majority voting [10], behavior knowledge
This requires the application of feature selection algorithms space [11], weighted voting based on the Dempster–Shafer
prior to classification. (ii) Concatenating two feature vec- theory of evidence [12], AND rule and OR rule [13], etc.
tors may result in a feature vector with very large dimen- can be used to arrive at the final decision.
sionality leading to the ‘curse of dimensionality’ problem When the output of each biometric matcher is a subset of
[7]. Although, this is a general problem in most pattern possible matches sorted in decreasing order of confidence,
recognition applications, it is more severe in biometric ap- the fusion can be done at the rank level. Ho et al. [14] de-
plications because of the time, effort and cost involved in scribe three methods to combine the ranks assigned by the
collecting large amounts of biometric data. (iii) Most com- different matchers. In the highest rank method, each possi-
mercial biometric systems do not provide access to the fea- ble match is assigned the highest (minimum) rank as com-
ture vectors which they use in their products. Hence, very puted by different matchers. Ties are broken randomly to
few researchers have studied integration at the feature level arrive at a strict ranking order and the final decision is made
and most of them generally prefer post-classification fusion based on the combined ranks. The Borda count method uses
schemes. the sum of the ranks assigned by the individual matchers to
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2273

calculate the combined ranks. The logistic regression ment structurally noise-resistant classifiers like a piece-wise
method is a generalization of the Borda count method where linear classifier and a modified Bayesian classifier. Ross
the weighted sum of the individual ranks is calculated and and Jain [4] use decision tree and linear discriminant clas-
the weights are determined by logistic regression. sifiers for combining the scores of face, fingerprint, and
When the biometric matchers output a set of possible hand-geometry modalities.
matches along with the quality of each match (matching
score), integration can be done at the matching score level. 2.2.2. Combination approach to measurement level fusion
This is also known as fusion at the measurement level or Kittler et al. [18] have developed a theoretical frame-
confidence level. Next to the feature vectors, the matching work for consolidating the evidence obtained from mul-
scores output by the matchers contain the richest information tiple classifiers using schemes like the sum rule, product
about the input pattern. Also, it is relatively easy to access rule, max rule, min rule, median rule and majority vot-
and combine the scores generated by the different matchers. ing. In order to employ these schemes, the matching scores
Consequently, integration of information at the matching are converted into posteriori probabilities conforming to a
score level is the most common approach in multimodal genuine user and an impostor. They consider the problem
biometric systems. of classifying an input pattern Z into one of m possible
In the context of verification, there are two approaches classes (in a verification system, m = 2) based on the evi-
for consolidating the scores obtained from different match- dence provided by R different classifiers. Let xi be the fea-
ers. One approach is to formulate it as a classification prob- ture vector (derived from the input pattern Z) presented to
lem, while the other approach is to treat it as a combination the ith classifier. Let the outputs of the individual classi-
problem. In the classification approach, a feature vector is fiers be P (
j | xi ), i.e., the posterior probability of the pat-
constructed using the matching scores output by the individ- tern Z belonging to class
j given the feature vector xi .
ual matchers; this feature vector is then classified into one Let c{1, 2, . . . , m} be the class to which the input pattern
of two classes: “Accept” (genuine user) or “Reject” (impos- Z is finally assigned. The following rules can be used to
tor). Generally, the classifier used for this purpose is capa- determine c:
ble of learning the decision boundary irrespective of how Product Rule: This rule is based on the assumption of sta-
the feature vector is generated. Hence, the output scores of tistical independence of the representations x1 , x2 , . . . , xR .
the different modalities can be non-homogeneous (distance The input pattern is assigned to class c such that
or similarity metric, different numerical ranges, etc.) and no
R


processing is required prior to feeding them into the clas-
c = argmax P (
j |
xi ).
sifier. In the combination approach, the individual match- j i=1
ing scores are combined to generate a single scalar score
which is then used to make the final decision. To ensure In general, different biometric traits of an individual (e.g.,
a meaningful combination of the scores from the different face, fingerprint and hand-geometry) are mutually indepen-
modalities, the scores must be first transformed to a common dent. This allows us to make use of the product rule in a
domain. multimodal biometric system based on the independence as-
sumption.
2.2.1. Classification approach to measurement level fusion Sum Rule: Apart from the assumption of statistical indepen-
Several classifiers have been used to consolidate the dence of the multiple representations used in the product
matching scores and arrive at a decision. Wang et al. [15] rule, the sum rule also assumes that the posteriori probabil-
consider the matching scores at the output of face and iris ities computed by the individual classifiers do not deviate
recognition modules as a two-dimensional feature vector, much from the prior probabilities. This rule is applicable
and use Fisher’s discriminant analysis and a neural net- when there is a high level of noise leading to ambiguity in
work classifier with radial basis function for classification. the classification problem. The sum rule assigns the input
Verlinde and Chollet [16] combine the scores from two pattern to class c such that
face recognition experts and one speaker recognition expert
R

using three classifiers: k-NN classifier using vector quanti-
c = argmax P (
j |
xi ).
zation, decision-tree based classifier and a classifier based j i=1
on a logistic regression model. Chatzis et al. [17] use fuzzy
k-means and fuzzy vector quantization, along with a me- Max Rule: The max rule approximates the mean of the pos-
dian radial basis function neural network classifier for the teriori probabilities by the maximum value. In this case, we
fusion of scores obtained from biometric systems based on assign the input pattern to class c such that
visual (facial) and acoustic features. Sanderson and Pali-
wal [5] use a support vector machine classifier to combine c = argmax max P (
j |
xi ).
j i
the scores of face and speech experts. They show that the
performance of such a classifier deteriorates under noisy Min Rule: The min rule is derived by bounding the product
input conditions. To overcome this problem, they imple- of posteriori probabilities. Here, the input pattern is assigned
2274 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

to class c such that (2) The weighted sum of scores (sw ) for user i is computed
as sw = w1,i s1 + w2,i s2 + w3,i s3 , where s1 , s2 , and s3
c = argmax min P (
j |
xi ). are the scores provided by the three biometric matchers
j i
for user i.
Prabhakar and Jain [19] argue that the assumption of sta- (3) The set of weights that minimizes the total error rate
tistical independence of the feature sets may not be true (sum of the false accept and false reject rates) at some
in a multimodal biometric system that uses different fea- specified threshold () is chosen. If more than one set
ture representations and different matching algorithms on of weights minimize the total error rate, then the set
the same biometric trait. They propose a scheme based on of weights that assigns almost equal weights to all the
non-parametric density estimation for combining the scores modalities is chosen. The threshold  is set to a common
obtained from four fingerprint matching algorithms and use value for all users. In [22], user-specific thresholds had
the likelihood ratio test to make the final decision. They been suggested.
show that their scheme is optimal in the Neyman–Pearson
decision sense, when sufficient training data is available to
estimate the joint densities. 3. Score normalization
The use of Bayesian statistics in combining the scores of
different biometric matchers was demonstrated by Bigun et Consider a multimodal biometric verification system that
al. [20]. They proposed a new algorithm for the fusion mod- utilizes the combination approach to fusion at the match
ule of a multimodal biometric system that takes into account score level. The theoretical framework developed by Kittler
the estimated accuracy of the individual classifiers during et al. in [18] can be applied to this system only if the out-
the fusion process. They showed that their multimodal sys- put of each modality is of the form P (genuine|Z) i.e., the
tem using image and speech data provided better recognition posteriori probability of user being “genuine” given the in-
results than the individual modalities. put biometric sample Z. In practice, most biometric systems
The combined matching score can also be computed as output a matching score s, and Verlinde et al. [23] have pro-
a weighted sum of the matching scores of the individual posed that the matching score s is related to P (genuine|Z)
matchers [4,15]. In this paper, we use the weighted sum as follows:
of the scores of the individual matchers (assuming equal
weights) and compare it to a threshold in order to decide if s = f {P (genuine|Z)} + (Z), (1)
the user is a genuine user or an impostor. We refer to this
method as the simple sum of scores method. We also use two where f is a monotonic function and  is the error made by
other simple fusion methods called the max-score and min- the biometric system that depends on the input biometric
score methods [21]. In the max-score method, the combined sample Z. This error could be due to the noise introduced
score is merely the maximum value among the scores of the by the sensor during the acquisition of the biometric signal
individual matchers. The minimum value among the scores and the errors made by the feature extraction and matching
of the individual matchers is assigned as the combined score processes. If we assume that  is zero, it is reasonable to
in min-score fusion. approximate P (genuine|Z) by P (genuine|s). In this case,
Jain and Ross [22] have proposed the use of user-specific the problem reduces to computing P (genuine|s) and this re-
weights for computing the weighted sum of scores from quires estimating the conditional densities P (s|genuine) and
the different modalities. The motivation behind this idea P (s|impostor). Snelick et al. [21] assumed a normal distri-
is that some biometric traits cannot be reliably obtained bution for the conditional densities of the matching scores
from a small segment of the population. For example, we (p(s|genuine)˜N (g , g ) and p(s|impostor)˜N (i , i )),
cannot obtain good quality fingerprints from users with and used the training data to estimate the parameters g ,
dry fingers. For such users, assigning a lower weight to g , i , and i . The posteriori probability of the score being
the fingerprint score and a higher weight to the scores that of a genuine user was then computed as,
of the other modalities reduces their probability of being p(s|genuine)
falsely rejected. This method requires learning of user- P (genuine|s) = .
p(s|genuine) + p(s|impostor)
specific weights from the training scores available for
each user. Consider a multimodal biometric system with The above approach has two main drawbacks. The as-
three modalities. Let w1,i , w2,i , and w3,i be the weights sumption of a normal distribution for the scores may not be
assigned to the three modalities for user i. The process true in many cases. For example, the scores of the fingerprint
of learning these weights for user i can be described as and hand-geometry matchers used in our experiments do not
follows: follow a normal distribution. Secondly, the approach does
not make use of the prior probabilities of the genuine and
(1) The weights w1,i , w2,i , and w3,i are varied over the impostor users that may be available to the system. Due to
range [0, 1] in steps of 0.02, such that the constraint these reasons, we have proposed the use of a non-parametric
w1,i + w2,i + w3,i = 1 is satisfied. technique, viz., Parzen window density estimation method
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2275

[7], to estimate the actual conditional density of the gen- output of the matchers may follow different statistical dis-
uine and impostor scores. After estimating the conditional tributions. Due to these reasons, score normalization is es-
densities, the Bayes formula can be applied to calculate the sential to transform the scores of the individual matchers
posteriori probability of the score being that of a genuine into a common domain prior to combining them. Score nor-
user. Thus, malization is a critical part in the design of a combination
scheme for matching score level fusion. Fig. 2 shows the
p(s|genuine) ∗ P (genuine)
P (genuine|s) = , conditional distributions of the face, fingerprint and hand-
p(s) geometry matching scores used in our experiments. The
where p(s) = {p(s|genuine) ∗ P (genuine) + p(s|impostor) scores obtained from the face and hand-geometry matchers
∗ P (impostor)} and P (genuine) and P (impostor) are the are distance scores and those obtained from the fingerprint
prior probabilities of a genuine user and an impostor, re- matcher are similarity scores. One can easily observe the
spectively. non-homogeneity in these scores and the need for normal-
Although the Parzen window density estimation tech- ization prior to any meaningful combination.
nique significantly reduces the error in the estimation of Score normalization refers to changing the location and
P (genuine|s) (especially when the conditional densities are scale parameters of the matching score distributions at the
not Gaussian), the error is still non-zero due to the finite outputs of the individual matchers, so that the matching
training set and the problems in choosing the optimum win- scores of different matchers are transformed into a common
dow width during the density estimation process. Further, domain. When the parameters used for normalization are de-
the assumption that the value of  in Eq. (1) is zero is not termined using a fixed training set, it is referred to as fixed
valid in most practical biometric systems. Since  depends score normalization [24]. In such a case, the matching score
on the input biometric sample Z, it is possible to estimate  distribution of the training set is examined and a suitable
only if the biometric system outputs a confidence measure model is chosen to fit the distribution. Based on the model,
(that takes into account the nature of the input Z) on the the normalization parameters are determined. In adaptive
matching score along with the matching score itself. In the score normalization, the normalization parameters are es-
absence of this confidence measure, the calculated value of timated based on the current feature vector. This approach
P (genuine|s) is a poor estimate of P (genuine|Z) and this has the ability to adapt to variations in the input data such
can lead to poor recognition performance of the multimodal as the change in the length of the speech signal in speaker
system (see Fig. 10). Hence, when the outputs of individual recognition systems.
modalities are just matching scores without any measures The problem of score normalization in multimodal
quantifying the confidence on those scores, it would be bet- biometric systems is identical to the problem of score
ter to combine the matching scores directly using an appro- normalization in metasearch. Metasearch is a technique for
priate method without converting them into probabilities. combining the relevance scores of documents produced by
The following issues need to be considered prior to com- different search engines, in order to improve the perfor-
bining the scores of the matchers into a single score. The mance of document retrieval systems [25]. Min–max and
matching scores at the output of the individual matchers z-score normalization are some of the popular techniques
may not be homogeneous. For example, one matcher may used for relevance score normalization in metasearch. In
output a distance (dissimilarity) measure while another may metasearch literature [26], the distribution of scores of rel-
output a proximity (similarity) measure. Furthermore, the evant documents is generally approximated as a Gaussian
outputs of the individual matchers need not be on the same distribution with a large standard deviation while that of
numerical scale (range). Finally, the matching scores at the non-relevant documents is approximated as an exponential

0.25 1 0.35
Genuine Scores Genuine Scores Genuine Scores
Impostor Scores 0.9 Impostor Scores
0.3 Impostor Scores
0.2 0.8
0.7 0.25
0.15 0.6
0.2
p(x)

p(x)
p(x)

0.5
0.1 0.4 0.15

0.3 0.1
0.05 0.2
0.05
0.1
0 0 0
0 50 100 150 200 250 300 0 200 400 600 800 1000 0 200 400 600 800 1000
(a) Raw Face Score (x) (b) Raw Fingerprint Score (x) (c) Raw Hand-Geometry Score (x)

Fig. 2. Conditional distribution of genuine and impostor scores: (a) face (distance score); (b) fingerprint (similarity score); and (c)
hand-geometry (distance score).
2276 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

distribution. In our experiments, the distributions of the minimum and maximum values for a set of matching scores
genuine and impostor fingerprint scores closely follow the and then apply the min–max normalization. Given a set of
distributions of relevant and non-relevant documents in matching scores {sk }, k = 1, 2, . . . , n, the normalized scores
metasearch. However, the face and hand-geometry scores are given by
do not exhibit this behavior.
For a good normalization scheme, the estimates of the sk − min
sk = .
location and scale parameters of the matching score distri- max − min
bution must be robust and efficient. Robustness refers to in-
When the minimum and maximum values are estimated
sensitivity to the presence of outliers. Efficiency refers to
from the given set of matching scores, this method is
the proximity of the obtained estimate to the optimal es-
not robust (i.e., the method is highly sensitive to outliers
timate when the distribution of the data is known. Huber
in the data used for estimation). Min–max normalization
[27] explains the concepts of robustness and efficiency of
retains the original distribution of scores except for a
statistical procedures. He also explains the need for statisti-
scaling factor and transforms all the scores into a com-
cal procedures that have both these desirable characteristics.
mon range [0, 1]. Distance scores can be transformed
Although many techniques can be used for score normal-
into similarity scores by subtracting the min–max nor-
ization, the challenge lies in identifying a technique that is
malized score from 1. Fig. 3 shows the distribution of
both robust and efficient.
face, fingerprint and hand-geometry scores after min–max
normalization.
3.1. Normalization techniques
Decimal scaling can be applied when the scores of dif-
ferent matchers are on a logarithmic scale. For example, if
The simplest normalization technique is the Min–max
one matcher has scores in the range [0, 1] and the other has
normalization (Table 1). Min–max normalization is best
scores in the range [0, 1000], the following normalization
suited for the case where the bounds (maximum and min-
could be applied.
imum values) of the scores produced by a matcher are
known. In this case, we can easily shift the minimum and sk
sk = ,
maximum scores to 0 and 1, respectively. However, even if 10n
the matching scores are not bounded, we can estimate the
where n = log10 max(si ). The problems with this approach
Table 1 are lack of robustness and the assumption that the scores
Summary of normalization techniques of different matchers vary by a logarithmic factor. In our
experiments, the matching scores of the three modalities
Normalization Robustness Efficiency
are not distributed on a logarithmic scale and, hence, this
technique
normalization technique cannot be applied.
Min–max No N/A The most commonly used score normalization technique
Decimal scaling No N/A is the z-score that is calculated using the arithmetic mean
z-score No High and standard deviation of the given data. This scheme can
(optimal for Gaussian data) be expected to perform well if prior knowledge about the
Median and MAD Yes Moderate average score and the score variations of the matcher is
Double sigmoid Yes High
available. If we do not have any prior knowledge about
tanh-estimators Yes High
Biweight estimators Yes High
the nature of the matching algorithm, then we need to es-
timate the mean and standard deviation of the scores from

0.25 1 0.35
Genuine Scores
Genuine Scores Impostor Scores Genuine Scores
Impostor Scores 0.9 Impostor Scores
0.3
0.2 0.8
0.7 0.25
0.15 0.6
0.2
p(y)
p(y)

p(y)

0.5
0.1 0.4 0.15

0.3 0.1
0.05 0.2
0.05
0.1
0 0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
(a) Min-max Normalized Face Score (y) (b) Min-max Normalized Fingerprint Score (y) (c) Min-max Normalized Hand-Geometry Score (y)

Fig. 3. Distribution of genuine and impostor scores after min–max normalization: (a) face; (b) fingerprint; and (c) hand-geometry.
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2277

a given set of matching scores. The normalized scores are that z-score normalization fails to transform the scores of
given by the different modalities into a common numerical range and
also does not retain the original distribution of scores in the
sk − 
sk = , case of fingerprint modality.
 The median and median absolute deviation (MAD) are
where  is the arithmetic mean and  is the standard devi- insensitive to outliers and the points in the extreme tails
ation of the given data. However, both mean and standard of the distribution. Hence, a normalization scheme using
deviation are sensitive to outliers and, hence, this method median and MAD would be robust and is given by
is not robust. Z-score normalization does not guarantee a sk − median
common numerical range for the normalized scores of the sk = ,
MAD
different matchers. If the input scores are not Gaussian dis-
tributed, z-score normalization does not retain the input dis- where MAD =median(|sk −median|). However, the median
tribution at the output. This is due to the fact that mean and the MAD estimators have a low efficiency compared to
and standard deviation are the optimal location and scale the mean and the standard deviation estimators, i.e., when
parameters only for a Gaussian distribution. For an arbi- the score distribution is not Gaussian, median and MAD are
trary distribution, mean and standard deviation are reason- poor estimates of the location and scale parameters. There-
able estimates of location and scale, respectively, but are fore, this normalization technique does not retain the input
not optimal. distribution and does not transform the scores into a common
The distributions of the matching scores of the three numerical range. This is illustrated by the distributions of
modalities after z-score normalization are shown in Fig. 4. the normalized face, fingerprint, and hand-geometry scores
The face and hand-geometry scores are converted into sim- in Fig. 5.
ilarity scores by subtracting from a large number (300 for Cappelli et al. [28] have used a double sigmoid func-
face and 1000 for hand-geometry in our experiments) be- tion for score normalization in a multimodal biometric
fore applying the z-score transformation. It is clearly seen system that combines different fingerprint classifiers. The

0.25 0.9 0.35

Genuine Scores Genuine Scores
Impostor Scores Genuine Scores
0.8 Impostor Scores Impostor Scores
0.3
0.2
0.7
0.25
0.6
0.15
0.5 0.2
p(y)

p(y)
p(y)

0.1 0.4 0.15

0.3
0.1
0.05 0.2
0.1 0.05

0 0 0
-4 -3 -2 -1 0 1 2 3 -6 -5 -4 -3 -2 -1 0 1 2
-5 0 5 10 15 20
(a) Z-Score Normalized Face Score (y) (b) Z-score Normalized Fingerprint Score (y) (c) Z-score Normalized Hand-Geometry Score (y)

Fig. 4. Distribution of genuine and impostor scores after z-score normalization: (a) face; (b) fingerprint; and (c) hand-geometry.

0.25 1 0.35
Genuine Scores Genuine Scores Genuine Scores
Impostor Scores 0.9 Impostor Scores Impostor Scores
0.3
0.2 0.8
0.7 0.25
0.15 0.6 0.2
p(y)

p(y)
p(y)

0.5
0.1 0.15
0.4
0.3 0.1
0.05 0.2
0.05
0.1
0 0 0
-6 -4 -2 0 2 4 -200 0 200 400 600 800 1000 -12 -10 -8 -6 -4 -2 0 2 4
(a) Median-MAD Normalized Face Score (y) (b) Median-MAD Normalized Fingerprint Score (y) (c) Median-MAD Normalized Hand-Geometry Score (y)

Fig. 5. Distribution of genuine and impostor scores after median-MAD normalization: (a) face; (b) fingerprint; and (c) hand-geometry.
2278 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

normalized score is given by transformed non-linearly. The double sigmoid normalization

 is very similar to the min–max normalization followed by the
1  application of two-quadrics (QQ) or logistic (LG) function
 if sk < t,
1 + exp(−2((sk − t)/r1 )) as suggested by Snelick et al. [29]. When r1 and r2 are
sk =

 1 large, the double sigmoid normalization closely resembles
otherwise,
1 + exp(−2((sk − t)/r2 )) the QQ-min–max normalization. On the other hand, we can
make the double sigmoid normalization tend toward LG-
where t is the reference operating point and r1 and r2 denote
min–max normalization by assigning small values to r1 and
the left and right edges of the region in which the function
r2 .
is linear, i.e., the double sigmoid function exhibits linear
Fig. 7 shows the face, fingerprint and hand-geometry
characteristics in the interval (t − r1 , t − r2 ). Fig. 6 shows
score distributions after double sigmoid normalization. The
an example of the double sigmoid normalization, where the
face and hand-geometry scores are converted into similar-
scores in the [0, 300] range are mapped to the [0, 1] range
ity scores by subtracting the normalized scores from 1.
using t = 200, r1 = 20 and r2 = 30.
The parameters of the double sigmoid normalization were
This scheme transforms the scores into the [0, 1] interval.
chosen as follows: t is chosen to be the center of the over-
But, it requires careful tuning of the parameters t, r1 , r2 to
lapping region between the genuine and impostor score dis-
obtain good efficiency. Generally, t is chosen to be some
tribution, and r1 and r2 are made equal to the extent of over-
value falling in the region of overlap between the genuine
lap between the two distributions toward the left and right
and impostor score distribution, and r1 and r2 are made
of the center, respectively. A matching score that is equally
equal to the extent of overlap between the two distributions
likely to be from a genuine user and an impostor is cho-
toward the left and right of t, respectively. This normalization
sen as the center (t) of the region of overlap. Then r1 is the
scheme provides a linear transformation of the scores in the
difference between t and the minimum of genuine scores,
region of overlap, while the scores outside this region are
while r2 is the difference between the maximum of impos-
tor scores and t. In order to make this normalization ro-
bust, approximately 2% of the scores at the extreme tails of
1
the genuine and impostor distributions were omitted when
0.9 calculating r1 and r2 . It must be noted that this scheme
0.8 cannot be applied as described here if there are multiple
0.7 intervals of overlap between genuine and impostor distri-
butions. Although this normalization scheme transforms all
Normalized Score

0.6
the scores to a common numerical range [0, 1], it does not
0.5
retain the shape of the original distribution of the fingerprint
0.4
scores.
0.3 The tanh-estimators introduced by Hampel et al. [30] are
r2
0.2 robust and highly efficient. The normalization is given by
r1   

0.1 t 1 sk − GH
sk = tanh 0.01 +1 ,
0
0 50 100 150 200 250 300 2 GH
Original Score
where GH and GH are the mean and standard deviation
Fig. 6. Double sigmoid normalization (t =200, r1 =20, and r2 =30). estimates, respectively, of the genuine score distribution as

0.4 0.7 0.45

Genuine Scores Genuine Scores Genuine Scores
0.35 Impostor Scores 0.6 Impostor Scores 0.4 Impostor Scores

0.3 0.35
0.5
0.3
0.25
0.4 0.25
p(y)

p(y)
p(y)

0.2
0.3 0.2
0.15
0.15
0.2
0.1 0.1
0.05 0.1
0.05
0 0 0
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.2 0.4 0.6 0.8 1 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
(a) Sigmoid Normalized Face Score (y) (b) Sigmoid Normalized Fingerprint Score (y) (c) Sigmoid Normalized Hand-geometry Score (y)

Fig. 7. Distribution of genuine and impostor scores after double sigmoid normalization: (a) face; (b) fingerprint; and (c) hand-geometry.
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2279

0.25 1 0.35
Genuine Scores
Genuine Scores 0.9 Impostor Scores Genuine Scores
Impostor Scores Impostor Scores
0.3
0.2 0.8
0.7 0.25
0.15 0.6
0.2

p(y)

p(y)
p(y)

0.5
0.1 0.4 0.15

0.3 0.1
0.05 0.2
0.05
0.1
0 0 0
0.3 0.35 0.4 0.45 0.5 0.55 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6
(a) Tanh Normalized Face Score (y) (b) Tanh Normalized Fingerprint Score (y) (c) Tanh Normalized Hand-geometry Score (y)

Fig. 8. Distribution of genuine and impostor scores after tanh normalization: (a) face; (b) fingerprint; and (c) hand-geometry.

given by Hampel estimators.1 Hampel estimators are based and 38.9, respectively. We observe that the genuine finger-
on the following influence ()-function: print scores have a standard deviation that is approximately
 10 times the standard deviation of the genuine face and hand-
 u 0
|u| < a,

 geometry scores. Hence, using the same constant, 0.01, for
 a sign(u)  a
|u| < b,
(u) = c − |u| the fingerprint modality is entirely inappropriate. To avoid

 a sign(u) b
|u| < c, this problem, the constant factor in the tanh normalization

 c−b
0 |u|  c. for fingerprint modality was set to 0.1. Therefore, the stan-
dard deviation of the tanh normalized genuine fingerprint
The Hampel influence function reduces the influence of the scores is roughly 0.1, which is about 10 times that of the
points at the tails of the distribution (identified by a, b, and face and hand-geometry modalities. This modification re-
c) during the estimation of the location and scale parame- tains the information contained in the fingerprint scores even
ters. Hence, this method is not sensitive to outliers. If the after the normalization, resulting in better performance.
influence of a large number of tail-points is reduced, Mosteller and Tukey [31] introduced the biweight loca-
the estimate is more robust but not efficient (optimal). tion and scale estimators that are robust and efficient. But,
On the other hand, if many tail-points influence the esti- the biweight estimators are iterative in nature (an initial
mate, the estimate is not robust but the efficiency increases. estimate of the biweight location and scale parameters is
Therefore, the parameters a, b, and c must be carefully chosen, and this estimate is updated based on the training
chosen depending on the amount of robustness required scores), and are applicable only for Gaussian data. The bi-
which in turn depends on the estimate of the amount of weight location and scale estimates of the data used in our
noise in the available training data. experiments were very close to the mean and standard devi-
In our experiments, the values of a, b and c were chosen ation. Hence, the results of this scheme were quite similar to
such that 70% of the scores were in the interval (m − a, m + those produced by the z-score normalization. Therefore, we
a), 85% of the scores were in the interval (m−b, m+b), and have not included the results for biweight normalization in
95% of the scores were in the interval (m − c, m + c), where this paper.
‘m’ is the median score. The distributions of the scores of Snelick et al. have developed a general testing framework
the three modalities after tanh normalization are shown in [21] that allows system designers to evaluate multimodal
Fig. 8. The distance to similarity transformation is achieved biometric systems by varying different factors like the bio-
by subtracting the normalized scores from 1. The nature of metric traits, matching algorithms, normalization schemes,
the tanh distribution is such that the genuine score distri- fusion methods and sample databases. To illustrate this
bution in the transformed domain has a mean of 0.5 and testing methodology, they evaluated the performance of
a standard deviation of approximately 0.01. The constant a multimodal biometric system that used face and finger-
0.01 in the expression for tanh normalization determines print classifiers. Normalization techniques like min–max,
the spread of the normalized genuine scores. In our exper- z-score, median and MAD, and tanh estimators were used
iments, the standard deviation of genuine scores of face, to transform the scores into a common domain. The trans-
fingerprint and hand-geometry modalities are 16.7, 202.1, formed scores were then combined using fusion methods
like simple sum of scores, maximum score, minimum score,
1 In [21,24], the mean and standard deviation of all the train- sum of posteriori probabilities (sum rule), and product of
ing scores (both genuine and impostor) were used for tanh nor- posteriori probabilities (product rule). Their experiments
malization. However, we observed that considering the mean and conducted on a database of more than 1000 users showed
standard deviation of only the genuine scores results in a better that the min–max normalization followed by the sum of
recognition performance. scores fusion method generally provided better recognition
2280 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

performance than other schemes. The results of Snelick et 100

al. [29] also show a similar trend. However, the reasons for
90
such a behavior have not been presented by these authors. In

Genuine Acceptance Rate − GAR (%)

this paper, we have tried to analyze the reasons for the dif- 80
ferences in the performance of the different normalization
schemes. We have tried to systematically study the different 70

normalization techniques to ascertain their role in the per-

60
formance of a multimodal biometric system consisting of
face, fingerprint and hand-geometry modalities. In addition 50
to the four normalization techniques employed in [21,29],
we have also analyzed the double sigmoid method of 40 Face
Fingerprint
normalization. Handgeometry
30

20 -3 -2 -1 0 1
4. Experimental results 10 10 10 10 10
False Acceptance Rate − FAR (%)

The multimodal database used in our experiments was

Fig. 9. ROC curves for individual modalities.
constructed by merging two separate databases (of 50 users
each) collected using different sensors and over different
time periods. The first database (described in [4]) was con-
structed as follows: Five face images and five fingerprint ing the independence of the three modalities, we create 43
impressions (of the same finger) were obtained from a set of “virtual” users from each real user, by considering all possi-
50 users. Face images were acquired using a Panasonic CCD ble combinations of scores of the three modalities. Thus, we
camera (640 × 480) and fingerprint impressions were ob- have 64,000 (100 × 43 ) genuine score vectors and 313,600
tained using a Digital Biometrics sensor (500 dpi, 640×480). (100×43 ×49) impostor score vectors to analyze the system
Five hand-geometry images were obtained from a different performance. This separation of the database into training
set of 50 users (some users were present in both the sets) and test sets was repeated 40 times and we have reported the
and captured using a Pulnix TMC-7EX camera. The mutual average performance results. Fingerprint matching was done
independence assumption of the biometric traits allows us using the minutiae features [32] and the output of the finger-
to randomly pair the users from the two sets. In this way, a print matcher was a similarity score. Eigenface coefficients
database consisting of 50 users was constructed, each user were used to represent features of the face image [33]. The
having five biometric templates for each modality. The bio- Euclidean distance between the eigenface coefficients of the
metric data captured from every user is compared with that template and that of the input face was used as the match-
of all the users in the database leading to one genuine score ing score. The hand-geometry images were represented by a
vector and 49 impostor score vectors for each distinct in- 14-dimensional feature vector [34] and the matching score
put. Thus, 500 (50 × 10) genuine score vectors and 24,500 was computed as the Euclidean distance between the input
(50 × 10 × 49) impostor score vectors were obtained from feature vector and the template feature vector.
this database. The second database also consisted of 50 users
whose face images were captured using a Sony video cam- 4.1. Performance results
era (256 × 384) and fingerprint images were acquired using
an Identix sensor (500 dpi, 255 × 256). The Pulnix TMC- The recognition performance of the face, fingerprint, and
7EX camera was used to obtain hand-geometry images. This hand-geometry systems when operated as unimodal systems
database also gave rise to 500 genuine and 24,500 impostor is shown in Fig. 9. From Fig. 2(a), we observe that there
score vectors. Merging the scores from the two databases is a significant overlap between the genuine and impostor
resulted in a database of 100 users with 1000 genuine score distributions of the raw face scores, and this explains the
vectors and 49,000 impostor score vectors. A score vector poor recognition performance of the face module. Fig. 2(b)
is a 3-tuple s1 , s2 , s3 , where s1 , s2 , and s3 correspond to shows that most of the impostor fingerprint scores are close
the matching scores obtained from the face, fingerprint and to zero and that the genuine fingerprint scores are spread over
hand-geometry matchers, respectively. Of the 10 genuine a wide range of values. Moreover, the overlap between the
and 10 × 49 impostor score vectors available for each user, two conditional densities is small and, hence, the fingerprint
6 genuine and 6 impostor score vectors were randomly se- system performs better than the face and hand-geometry
lected and used for training (for calculating the parameters modules. The overlap between the genuine and impostor
of each normalization technique or for density estimation distributions of the hand-geometry system is the highest
by the Parzen window method). The remaining 4 genuine among all the three modalities as shown in Fig. 2(c). Hence,
and 4 × 49 impostor score vectors of each user were used the hand geometry based recognition performance is low
for testing the performance of the system. Again assum- compared to the fingerprint and face matchers.
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2281

Table 2 100
Genuine acceptance rate (GAR) (%) of different normalization and
Minmax
fusion techniques at the 0.1% false acceptance rate (FAR) Tanh
95 ZScore

Genuine Acceptance Rate − GAR (%)

Normalization Fusion techniques
STrans Parzen
techniques 90
Sigmoid
Sum of scores Max-score Min-score

STrans 98.3 (0.4) 46.7 (2.3) 83.9 (1.6) 85

Min–max 97.8 (0.6) 67.0 (2.5) 83.9 (1.6) Median

z-score 98.6 (0.4) 92.1 (1.1) 84.8 (1.6) 80

Median 84.5 (1.3) 83.7 (1.6) 68.8 (2.2)
Sigmoid 96.5 (1.3) 83.7 (1.6) 83.1 (1.8)
Tanh 98.5 (0.4) 86.9 (1.8) 85.6 (1.5) 75

Parzen 95.7 (0.9) 93.6 (2.0) 83.9 (1.9)

70 -3
Note that the values in the table represent average GAR, and the 10 10
-2
10
-1 0
10
1
10
values indicated in parentheses correspond to the standard deviation False Acceptance Rate − FAR (%)
of GAR.
Fig. 10. ROC curves for sum of scores fusion method.
The performance of the multimodal biometric system has
been studied under different normalization and fusion tech-
niques. The simple sum of scores, the max-score, and the
min-score fusion methods were applied on the normalized just a linear transformation of the score vector s1 , s2 , s3 ,
scores. The normalized scores were obtained by using one i.e., s =(a1 s1 −b1 )+(a2 s2 −b2 )+(a3 s3 −b3 ), where s1 , s2 ,
of the following techniques: simple distance-to-similarity and s3 correspond to the matching scores obtained from the
transformation with no change in scale (STrans), min–max face, fingerprint and hand-geometry matchers, respectively.
normalization (Minmax), z-score normalization (ZScore), The effect of different normalization techniques is to deter-
median-MAD normalization (Median), double sigmoid nor- mine the weights a1 , a2 , and a3 , and the biases b1 , b2 , and
malization (Sigmoid), tanh normalization (Tanh), and Parzen b3 . Since the MAD of the fingerprint scores is very small
normalization (Parzen).2 Table 2 summarizes the average compared to that of face and hand-geometry scores, the
genuine acceptance rate (GAR) of the multimodal system median-MAD normalization assigns a much larger weight
along with the standard deviation of the GAR (shown in to the fingerprint score (a2 ?a1 , a3 ). This is a direct con-
parentheses) for different normalization and fusion schemes, sequence of the moderate efficiency of the median-MAD
at a false acceptance rate (FAR) of 0.1%. estimator. The distribution of the fingerprint scores devi-
Fig. 10 shows the recognition performance of the sys- ates drastically from the Gaussian assumption and, hence,
tem when the scores are combined using the sum of scores median and MAD are not the right measures of location
method. We observe that a multimodal system employing and scale, respectively. In this case, the combined score is
the sum of scores method provides better performance than approximately equal to the fingerprint score and the per-
the best unimodal system (fingerprint in this case) for all formance of the multimodal system is close to that of the
normalization techniques except median-MAD normaliza- fingerprint module. On the other hand, min–max normaliza-
tion. For example, at a FAR of 0.1%, the GAR of the finger- tion, z-score normalization, tanh and distance-to-similarity
print module is about 83.6%, while that of the multimodal transformation assign nearly optimal weights to the three
system is high as 98.6% when z-score normalization is used. scores. Therefore, the recognition performance of the mul-
This improvement in performance is significant and it un- timodal system when using one of these techniques along
derscores the benefit of multimodal systems. with the sum of scores fusion method is significantly better
Among the various normalization techniques, we observe than that of the fingerprint matcher. The difference in per-
that the tanh and min–max normalization techniques outper- formance between the min–max, z-score, tanh and distance-
form other techniques at low FARs. At higher FARs, z-score to-similarity transformation is relatively small. However, it
normalization provides slightly better performance than tanh should be noted that the raw scores of the three modalities
and min–max normalization. In a multimodal system using used in our experiments are comparable and, hence, a simple
the sum of scores fusion method, the combined score (s) is distance-to-similarity conversion works reasonably well. If
the scores of the three modalities are significantly different,
2 Conversion of matching scores into posteriori probabilities by then this method will not work.
the Parzen window method is really not a normalization technique.
The performance of the multimodal system using max-
However, for the sake of convenience we refer to this method as score fusion is shown in Fig. 11. Here, z-score and Parzen
Parzen normalization. In this case, the simple sum of scores, max normalization provide better recognition performance com-
score, and min score fusion schemes, respectively, reduce to the pared to that of the fingerprint matcher. In a max-score fusion
sum rule, max rule, and min rule described in [18]. method that uses only the distance-to-similarity transforma-
2282 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

100 100
Parzen
ZScore

Genuine Acceptance Rate − GAR (%)

90 90
Tanh
Genuine Acceptance Rate − GAR (%)

Median
80 Sigmoid
Tanh STrans
80

70
Minmax
Minmax 70 ZScore
Parzen
60 Sigmoid
60 Median
50 STrans

40 50

30 40
-4 -3 -2 -1 0 1
10 10 10 10 10 10
20 -2 False Acceptance Rate − FAR (%)
-1 0 1
10 10 10 10
False Acceptance Rate − FAR (%)
Fig. 12. ROC curves for min-score fusion method.

Fig. 11. ROC curves for max-score fusion method.

estimates of the location and scale parameters. The scores

tion, the hand-geometry scores begin to dominate. There- produced by the matchers in our experiments are unbounded
fore, the performance is only slightly better than the hand- and, hence, can theoretically produce any value. Also, the
geometry module. For min–max normalization, the face and statistics of the scores (like average or deviation from the
hand-geometry scores are comparable and they dominate the average) produced by these matchers is not known. There-
fingerprint score. This explains why the performance of the fore, parameters like the minimum and maximum scores in
multimodal system is close to that of the face recognition min–max normalization, and the average and standard devi-
system. When median-MAD, tanh, and double sigmoid nor- ation of scores in z-score normalization have to be estimated
malization are used, the fingerprint scores are much higher from the available data. The data used in our experiments
compared to the face and hand-geometry scores. This limits does not contain any outliers and, hence, the performance of
the performance of the system close to that of the fingerprint the non-robust normalization techniques were not affected.
module. In z-score normalization, the scores of the three In order to demonstrate the sensitivity of the min–max and
modules are comparable and, hence, the combined score z-score normalization techniques to the presence of outliers,
depends on all the three scores and not just the score of we artificially introduced outliers in the fingerprint scores.
one modality. This improves the relative performance of the For min–max normalization, a single large score whose
max-score fusion method compared to other normalization value is 125% or 150% or 175% or 200% of the original
methods. Finally, Parzen normalization followed by max- maximum score is introduced into the fingerprint data. Fig.
score fusion accepts the user even if one of the three modal- 13 shows the recognition performance of the multimodal
ities produces a high estimate of the posteriori probability system after the introduction of the outlier. We can clearly
and rejects only if all the three modalities make errors in see that the performance is highly sensitive to the maximum
the probability estimation process. Hence, this method has score. A single large score that is twice the original maxi-
a high genuine acceptance rate. mum score can reduce the recognition rate by 3–5% depend-
Fig. 12 shows the performance of the multimodal system ing on the operating point of the system. The performance
when min-score fusion method is employed. For median- degradation is more severe at lower values of FAR.
MAD normalization, most of the face scores have smaller In the case of z-score normalization, a few large scores
values compared to the fingerprint and hand-geometry were introduced in the fingerprint data so that the standard
scores. Therefore, for median-MAD normalization the deviation of the fingerprint score is increased by 125% or
performance of the min-score method is close to the perfor- 150% or 175% or 200% of the original standard deviation.
mance of the face-recognition system. On the other hand, In one trial, some large scores were reduced to decrease the
the fingerprint scores have smaller values for all other nor- standard deviation to 75% of the original value. In the case
malization schemes and, hence, their performance is very of increase in standard deviation, the performance improves
close to that of the fingerprint matcher. after the introduction of outliers as indicated in Fig. 14.
For sum of scores fusion, we see that the performance of a Since the initial standard deviation was small, fingerprint
robust normalization technique like tanh is almost the same scores were assigned a higher weight compared to the other
as that of the non-robust techniques like min–max and z- modalities. As the standard deviation is increased, the domi-
score normalization. However, the performance of such non- nation of the fingerprint scores was reduced and this resulted
robust techniques is highly dependent on the accuracy of the in improved recognition rates. However, the goal of this
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2283

98 100
1.25*TrueMax
99
96 TrueMax
Genuine Acceptance Rate − GAR (%)

Genuine Acceptance Rate−GAR (%)

0.75*TrueMax 98

94 97
1.5*TrueStd
96 2*TrueStd
92 1.50*TrueMax
95 1.75*TrueStd
1.75*TrueMax
90 94
2.00*TrueMax
93 1.25*TrueStd
88
92 TrueStd

91 0.75*TrueStd
86 -3 -2 -1
10 10 10
False Acceptance Rate − FAR (%) 90 -3 -2 -1
10 10 10
False Acceptance Rate−FAR (%)
Fig. 13. Robustness analysis of min–max normalization.

Fig. 15. Robustness analysis of tanh normalization.

100 100

98 1.50*TrueStd
99
Genuine Acceptance Rate − GAR (%)

96 2.00*TrueStd
Genuine Acceptance Rate − GAR (%)

98
94 1.75*TrueStd

92 97

90 TrueStd 96
0.75*TrueStd
88
95
86 Minmax
1.25*TrueStd User-specific minmax
94 Tanh
84
User-specific tanh

82 3 2 1 93
10 10 10 -2 -1 0 1
10 10 10 10
False Acceptance Rate − FAR (%)
False Acceptance Rate FAR − (%)

Fig. 14. Robustness analysis of z-score normalization.

Fig. 16. Improvement in recognition rates using user-specific
weights.

experiment is to show the sensitivity of the system to those 5. Conclusion and future work
estimated parameters that can be easily affected by outliers.
A similar experiment was done for tanh-normalization tech- This paper examines the effect of different score normal-
nique and, as shown in Fig. 15, there is no significant vari- ization techniques on the performance of a multimodal bio-
ation in the performance after the introduction of outliers. metric system. We have demonstrated that the normalization
This result highlights the robustness of the tanh normaliza- of scores prior to combining them improves the recognition
tion method. performance of a multimodal biometric system that uses the
Finally, the weighting of the three modalities during the face, fingerprint and hand-geometry traits for user authen-
computation of the combined score was varied for each user. tication. Min–max, z-score, and tanh normalization tech-
The weights for each user were computed based on the train- niques followed by a simple sum of scores fusion method
ing scores as explained in Section 2.2.2. The scores were result in a superior GAR than all the other normalization
normalized using min–max and tanh normalization before and fusion techniques. We have shown that both min–max
calculating weighted combined score. We find that the in- and z-score methods are sensitive to outliers. On the other
troduction of user-specific weights result in a significant im- hand, tanh normalization method is both robust and efficient.
provement in the recognition performance. The ROC curves If the location and scale parameters of the matching scores
summarizing this improvement are shown in Fig. 16. (minimum and maximum values for min–max, or mean and
2284 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285

standard deviation for z-score) of the individual modalities [8] K. Woods, K. Bowyer, W.P. Kegelmeyer, Combination of
are known in advance, then simple normalization techniques multiple classifiers using local accuracy estimates, IEEE
like min–max and z-score would suffice. If these parame- Trans. Pattern Anal. Mach. Intell. 19 (4) (1997) 405–410.
ters are to be estimated using some training scores, and if [9] K. Chen, L. Wang, H. Chi, Methods of combining multiple
classifiers with different features and their applications to
the training scores are noisy, then one should choose a ro-
text-independent speaker identification, Int. J. Pattern Recogn.
bust normalization technique like the tanh normalization.
Artif. Intell. 11 (3) (1997) 417–445.
We have also explored the use of non-parametric approaches [10] L. Lam, C.Y. Suen, Application of majority voting to pattern
like the Parzen window density estimation method to convert recognition: an analysis of its behavior and performance, IEEE
the matching scores into posteriori probabilities and com- Trans. Systems Man Cybernet. Part A: Systems Humans 27
bining the probabilities using fusion rules. The advantage (5) (1997) 553–568.
of this method is that it avoids the need for any knowledge [11] L. Lam, C.Y. Suen, Optimal combination of pattern classifiers,
about the distribution of matching scores. However, the per- Pattern Recogn. Lett. 16 (1995) 945–954.
formance of this method is dependent on the type and width [12] L. Xu, A. Krzyzak, C.Y. Suen, Methods for combining
multiple classifiers and their applications to handwriting
of the kernel used for the estimation of density. Computa-
recognition, IEEE Trans. Systems Man Cybernet. 22 (3)
tion of the weighted sum of scores based on user-specific
(1992) 418–435.
weights results in further improvement in the recognition [13] J. Daugman, Combining Multiple Biometrics, Available at
performance. [Link]
Some normalization schemes work well if the scores fol- combine/[Link].
low a specific distribution. For example, z-score normal- [14] T.K. Ho, J.J. Hull, S.N. Srihari, Decision combination in
ization is optimal if the scores of all the modalities fol- multiple classifier systems, IEEE Trans. Pattern Anal. Mach.
low a Gaussian distribution. Therefore, we need to develop Intell. 16 (1) (1994) 66–75.
rules that would allow a practitioner to choose a normal- [15] Y. Wang, T. Tan, A.K. Jain, Combining face and iris
biometrics for identity verification, in: Proceedings of Fourth
ization scheme after analyzing the genuine and impostor
International Conference on AVBPA, Guildford, UK, 2003,
score distributions of the individual matchers. The possi-
pp. 805–813.
bility of applying different normalization techniques to the [16] P. Verlinde, G. Cholet, Comparing decision fusion paradigms
scores of different modalities must be explored. Guidelines using k-NN based classifiers, decision trees and logistic
for choosing the design parameters of some normalization regression in a multi-modal identity verification application,
techniques (e.g., the values of the constants a, b, and c in in: Proceedings of Second International Conference on
tanh normalization, the type and width of the kernel func- AVBPA, Washington, DC, USA, 1999, pp. 188–193.
tion to be used in Parzen normalization) need to be de- [17] V. Chatzis, A.G. Bors, I. Pitas, Multimodal decision-level
veloped. Alternative normalization and fusion techniques fusion for person authentication, IEEE Trans. Systems Man
such as those based on Bayesian statistics also need to be Cybernet. Part A: Systems Humans 29 (6) (1999) 674–681.
[18] J. Kittler, M. Hatef, R.P. Duin, J.G. Matas, On combining
explored.
classifiers, IEEE Trans. Pattern Anal. Mach. Intell. 20 (3)
(1998) 226–239.
[19] S. Prabhakar, A.K. Jain, Decision-level fusion in fingerprint
References verification, Pattern Recogn. 35 (4) (2002) 861–874.
[20] E.S. Bigun, J. Bigun, B. Duc, S. Fischer, Expert conciliation
[1] A.K. Jain, A. Ross, S. Prabhakar, An introduction to biometric for multimodal person authentication systems using Bayesian
recognition, IEEE Trans. Circuits Systems Video Technol. 14 statistics, in: Proceedings of First International Conference on
(1) (2004) 4–20 (special issue on image- and video-based AVBPA, Crans-Montana, Switzerland, 1997, pp. 291–300.
biometrics). [21] R. Snelick, M. Indovina, J. Yen, A. Mink, Multimodal
[2] A.K. Jain, A. Ross, Multibiometric systems, Commun. ACM biometrics: issues in design and testing, in: Proceedings
47 (1) (2004) 34–40 (special issue on multimodal interfaces). of Fifth International Conference on Multimodal Interfaces,
[3] L. Hong, A.K. Jain, S. Pankanti, Can multibiometrics Vancouver, Canada, 2003, pp. 68–72.
improve performance? in: Proceedings of IEEE Workshop on [22] A.K. Jain, A. Ross, Learning user-specific parameters in
Automatic Identification Advanced Technologies, NJ, USA, a multibiometric system, in: Proceedings of International
1999, pp. 59–64. Conference on Image Processing, New York, USA, 2002, pp.
[4] A. Ross, A.K. Jain, Information fusion in biometrics, Pattern 57–60.
Recogn. Lett. 24 (13) (2003) 2115–2125 (special issue on [23] P. Verlinde, P. Druyts, G. Cholet, M. Acheroy, Applying
multimodal biometrics). Bayes based classifiers for decision fusion in a multi-modal
[5] C. Sanderson, K.K. Paliwal, Information fusion and person identity verification system, in: Proceedings of International
verification using speech and face information, Research Paper Symposium on Pattern Recognition “In Memoriam Pierre
IDIAP-RR 02-33, IDIAP, September 2002. Devijver”, Brussels, Belgium, 1999.
[6] S.S. Iyengar, L. Prasad, H. Min, Advances in Distributed [24] R. Brunelli, D. Falavigna, Person identification using multiple
Sensor Technology, Prentice-Hall, Englewood Cliffs, NJ, cues, IEEE Trans. Pattern Anal. Mach. Intell. 12 (10) (1995)
1995. 955–966.
[7] R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, [25] M. Montague, J.A. Aslam, Relevance score normalization for
Wiley, New York, 2001. metasearch, in: Proceedings of Tenth International Conference
 A. Jain et al. / Pattern Recognition 38 (2005) 2270 – 2285 2285

on Information and Knowledge Management, Atlanta, USA, [30] F.R. Hampel, P.J. Rousseeuw, E.M. Ronchetti, W.A. Stahel,
2001, pp. 427–433. Robust Statistics: The Approach Based on Influence Functions,
[26] R. Manmatha, T. Rath, F. Feng, Modeling score distributions Wiley, New York, 1986.
for combining the outputs of search engines, in: Proceedings [31] F. Mosteller, J.W. Tukey, Data Analysis and Regression: A
of 24th International ACM SIGIR Conference on Research and Second Course in Statistics, Addison-Wesley, Reading, MA,
Development in Information Retrieval, New Orleans, USA, 1977.
2001, pp. 267–275. [32] A.K. Jain, L. Hong, S. Pankanti, R. Bolle, An identity
[27] P.J. Huber, Robust Statistics, Wiley, New York, 1981. authentication system using fingerprints, Proc. IEEE 85 (9)
[28] R. Cappelli, D. Maio, D. Maltoni, Combining fingerprint (1997) 1365–1388.
classifiers, in: Proceedings of First International Workshop on [33] M. Turk, A. Pentland, Eigenfaces for recognition, J. Cognitive
Multiple Classifier Systems, 2000, pp. 351–361. Neurosci. 3 (1) (1991) 71–86.
[29] R. Snelick, U. Uludag, A. Mink, M. Indovina, A.K. Jain, Large [34] A.K. Jain, A. Ross, S. Pankanti, A prototype hand
scale evaluation of multimodal biometric authentication using geometry-based verification system, in: Proceedings of Second
state-of-the-art systems, IEEE Trans. Pattern Anal. Mach. International Conference on AVBPA, Washington, DC, USA,
Intell. (to appear). 1999, pp. 166–171.

About the Author—ANIL JAIN is a University Distinguished Professor in the Departments of Computer Science and Engineering and
Electrical and Computer Engineering at Michigan State University. He was the Department Chair between 1995–99. His research interests
include statistical pattern recognition, exploratory pattern analysis, texture analysis, document image analysis and biometric authentication.
Several of his papers have been reprinted in edited volumes on image processing and pattern recognition. He received the best paper awards
in 1987 and 1991, and received certificates for outstanding contributions in 1976, 1979, 1992, 1997 and 1998 from the Pattern Recognition
Society. He also received the 1996 IEEE Transactions on Neural Networks Outstanding Paper Award. He was the Editor-in-Chief of the
IEEE Transactions on Pattern Analysis and Machine Intelligence between 1991–1994. He is a fellow of the IEEE, ACM, and International
Association of Pattern Recognition (IAPR). He has received a Fulbright Research Award, a Guggenheim fellowship and the Alexander von
Humboldt Research Award. He delivered the 2002 Pierre Devijver lecture sponsored by the International Association of Pattern Recognition
(IAPR). He holds six patents in the area of fingerprint matching.

About the Author—KARTHIK NANDAKUMAR received his B.E. degree in Electronics and Communication Engineering from Anna
University, Chennai, India in 2002. He is now a Ph.D. student in the Department of Computer Science and Engineering, Michigan State
University. His research interests include statistical pattern recognition, biometric authentication, computer vision and machine learning.

About the Author—ARUN ROSS is an Assistant Professor in the Lane Department of Computer Science and Electrical Engineering at West
Virginia University. Ross received his B.E. (Hons.) degree in Computer Science from the Birla Institute of Technology and Science, Pilani
(India), in 1996. He obtained his M.S. and Ph.D. degrees in Computer Science and Engineering from Michigan State University in 1999
and 2003, respectively. Between July 1996 and December 1997, he worked with the Design and Development group of Tata Elxsi (India)
Ltd., Bangalore. He also spent three summers (2000–2002) with the Imaging and Visualization group at Siemens Corporate Research, Inc.,
Princeton, working on fingerprint recognition algorithms. His research interests include statistical pattern recognition, image processing,
computer vision and biometrics.