Van Looy and Shafagatova SpringerPlus (2016)5:1797

DOI 10.1186/s40064-016-3498-1

REVIEW Open Access

Business process performance

measurement: a structured literature review
of indicators, measures and metrics
Amy Van Looy* and Aygun Shafagatova

Abstract
Measuring the performance of business processes has become a central issue in both academia and business, since
organizations are challenged to achieve effective and efficient results. Applying performance measurement models
to this purpose ensures alignment with a business strategy, which implies that the choice of performance indicators
is organization-dependent. Nonetheless, such measurement models generally suffer from a lack of guidance regard-
ing the performance indicators that exist and how they can be concretized in practice. To fill this gap, we conducted a
structured literature review to find patterns or trends in the research on business process performance measurement.
The study also documents an extended list of 140 process-related performance indicators in a systematic manner by
further categorizing them into 11 performance perspectives in order to gain a holistic view. Managers and scholars
can consult the provided list to choose the indicators that are of interest to them, considering each perspective. The
structured literature review concludes with avenues for further research.
Keywords: Business process, Performance measurement, Indicator, Measure, Metric, Structured literature review,
Systematic literature review

Background challenges of the rapidly changing society and technol-

Since organizations endeavor to measure what they ogy; this approach resulted in multi-dimensional models.
manage, performance measurement is a central issue Perhaps the best known multi-dimensional performance
in both the literature and in practice (Heckl and Moor- measurement model is the Balanced Scorecard (BSC)
mann 2010; Neely 2005; Richard et al. 2009). Perfor- developed by Kaplan and Norton (1996, 2001), which
mance measurement is a multidisciplinary topic that is takes a four-dimensional approach to organizational
highly studied by both the management and informa- performance: (1) financial perspective, (2) customer per-
tion systems domains (business process management or spective, (3) internal business process perspective, and
BPM in particular). Different performance measurement (4) “learning and growth” perspective. The BSC helps
models, systems and frameworks have been developed translate an organization’s strategy into operational per-
by academia and practitioners (Cross and Lynch 1988; formance indicators (also called performance measures
Kaplan and Norton 1996, 2001; EFQM 2010; Kueng 2000; or metrics) and objectives with targets for each of these
Neely et al. 2000). While measurement models were ini- performance perspectives. Even today, the BSC is by far
tially limited to financial performance (e.g., traditional the most used performance measurement approach in
controlling models), a more balanced and integrated the business world (Bain Company 2015; Sullivan 2001;
approach was needed beginning in the 1990s due to the Ulfeder 2004).
Equally important for measuring an organization’s per-
formance is process-oriented management or business
*Correspondence: [Link]@[Link] process management (BPM), which is “about managing
Faculty of Economics and Business Administration – Department entire chains of events, activities and decisions that ulti-
of Business Informatics and Operations Management, Ghent University,
Tweekerkenstraat 2, 9000 Ghent, Belgium
mately add value to the organization and its customers.

© 2016 The Author(s). This article is distributed under the terms of the Creative Commons Attribution 4.0 International License
([Link] which permits unrestricted use, distribution, and reproduction in any medium,
provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license,
and indicate if changes were made.
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 2 of 24

These ‘chains of events, activities and decisions’ are process performance, as well as to ensure a clear scope
called processes” (Dumas et al. 2013: p. 1). In particular, (specifically targeting an organization’s way of working).
an organization can do more with its current resources Accordingly, the study addresses the following research
by boosting the effectiveness and efficiency of its way of questions.
working (i.e., its business processes) (Sullivan 2001). In
this regard, academic research also suggests a strong link ••  RQ1. What is the current state of the research on
between business process performance and organiza- business process performance measurement?
tional performance, either in the sense of a causal rela- ••  RQ2. Which indicators, measures and metrics are
tionship (Melville et al. 2004; Smith and Reece 1999) used or mentioned in the current literature related to
or as distinctive indicators that co-exist, as in the BSC business process performance?
(Kaplan and Norton 1996, 2001).
Nonetheless, performance measurement models tend The objective of RQ1 is to identify patterns in the cur-
to give little guidance on how business (process) per- rent body of knowledge and to note weaknesses, whereas
formance indicators can be chosen and operationalized RQ2 mainly intends to develop an extended list of meas-
(Shah et al. 2012). They are limited to mainly defining urable process performance indicators, categorized into
performance perspectives, possibly with some exam- recognized performance perspectives, which can be tai-
ples or steps to derive performance indicators (Neely lored to diverse purposes. This list could, for instance,
et al. 2000), but without offering concrete indicators. serve as a supplement to existing performance measure-
Whereas fairly large bodies of research exist for both ment models. Practitioners can use the list as a source
performance models and business processes, no struc- for best practice indicators from academic research to
tured literature review of (process) performance meas- find and select a subset of performance indicators that fit
urement has been carried out thus far. To the best of our their strategy. The study will thus not address the devel-
knowledge, existing reviews cover one or another aspect opment of specific measurement systems but rather the
of performance measurement; for instance, reviews on indicators to be used within such systems. To make our
measurement models or evaluation criteria for perfor- intended list system-independent, we will begin with the
mance indicators (Heckl and Moormann 2010; Neely BSC approach and extend its performance perspectives.
2005; Richard et al. 2009). Despite the considerable Given this generic approach, the research findings can
importance of a comprehensive and holistic approach also be used by scholars when building and testing theo-
to business (process) performance measurement, little retical models in which process performance is one of the
is known regarding the state of the research on alterna- factors that must be concretized.
tive performance indicators and their operationaliza- The remainder of this article is structured as follows.
tion with respect to evaluating the performance of an “Theoretical background” section describes the theo-
organization’s work routines. To some extent, this lack retical background of performance measurement models
of guidance can be explained by the fact that perfor- and performance indicators. Next, the methodology for
mance indicators are considered organization-depend- our structured literature review is detailed in “Methods”
ent, given that strategic alignment is claimed by many section. The subsequent sections present the results for
measurement models such as the BSC (Kaplan and Nor- RQ1 (“Results for RQ1” section) and RQ2 (“Results for
ton 1996, 2001). Although the selection of appropriate RQ2” section). The discussion of the results in provided
performance indicators is challenging for practitioners in “Discussion” section, followed by concluding com-
due to the lack of best practices, it is also highly relevant ments (“Conclusion” section).
for performance measurement.
The gap that we are studying is the identification and, in Theoretical background
particular, the concretization/operationalization of pro- This section addresses the concepts of performance
cess-related performance indicators. This study enhances measurement models and performance indicators sepa-
the information systems literature, which focuses on the rately in order to be able to differentiate them further in
design and development of measurement systems with- the study.
out paying much attention to essential indicators. To fill
this gap, our study presents a structured literature review Performance measurement models
in order to describe the current state of business process According to overviews in the performance literature
performance measurement and related performance (Heckl and Moormann 2010; Neely 2005; Richard et al.
indicators. The choice to focus on the business process 2009), some of the most cited performance measurement
management (BPM) discipline is motivated by the close models are the Balanced Scorecard (Kaplan and Norton
link between organizational performance and business 1996, 2001), self-assessment excellence models such as
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 3 of 24

the EFQM (2010), and the models by Cross and Lynch (2) customer results, (3) society results, and (4) key
(1988), Kueng (2000) and Neely et al. (2000). A distinc- results], and a feedback loop for learning, creativity and
tion should, however, be made between models focusing innovation.
on the entire business (Kaplan and Norton 1996, 2001; Since the BSC is the most used performance measure-
EFQM 2010; Cross and Lynch 1988) and models focus- ment model, we have chosen it as a reference model to
ing on a single business process (Kueng 2000; Neely et al. illustrate the function of an organizational performance
2000). measurement model (Kaplan and Norton 1996, 2001).
The BSC is designed to find a balance between finan-
Organizational performance measurement models cial and non-financial performance indicators, between
Organizational performance measurement models typi- the interests of internal and external stakeholders, and
cally intend to provide a holistic view of an organization’s between presenting past performance and predicting
performance by considering different performance per- future performance. The BSC encourages organizations
spectives. As mentioned earlier, the BSC provides four to directly derive (strategic) long-term objectives from
perspectives for which objectives and performance indi- the overall strategy and to link them to (operational)
cators ensure alignment between strategies and opera- short-term targets. Concrete performance measures or
tions (Fig. 1) (Kaplan and Norton 1996, 2001). Other indicators should be defined to periodically measure the
organizational performance measurement models pro- objectives. These indicators are located on one of the four
vide similar perspectives. For instance, Cross and Lynch performance perspectives in Fig. 1 (i.e., ideally with a
(1988) offer a four-level performance pyramid: (1) a top maximum of five indicators per perspective).
level with a vision, (2) a second level with objectives per Table 1 illustrates how an organizational strategy can
business unit in market and financial terms, (3) a third be translated into operational terms using the BSC.
level with objectives per business operating system in During periodical measurements using the BSC, man-
terms of customer satisfaction, flexibility and productiv- agers can assign color-coded labels according to actual
ity, and (4) a bottom level with operational objectives for performance on short-term targets: (1) a green label if
quality, delivery, process time and costs. Another alterna- the organization has achieved the target, (2) an orange
tive view on organizational performance measurement is label if it is almost achieved, or (3) a red label if it is not
given in business excellence models, which focus on an achieved. Orange and red labels thus indicate areas for
evaluation through self-assessment rather than on stra- improvement.
tegic alignment, albeit by also offering performance per- Furthermore, the BSC assumes a causal or logical rela-
spectives. For instance, the EFQM (2010) distinguishes tionship between the four performance perspectives.
enablers [i.e., (1) leadership, (2) people, (3) strategy, (4) An increase in the competences of employees (i.e., per-
partnerships and resources, and (5) processes, prod- formance related to “learning and growth”) is expected
ucts and services] from results [i.e., (1) people results, to positively affect the quality of products and services
(i.e., internal business process performance), which in
turn will lead to improved customer perceptions (i.e.,
customer performance). The results for the previous per-
An established view on organizaonal performance measurement spectives will then contribute to financial performance to
ultimately realize the organization’s strategy, mission and
Financial performance vision (Kaplan and Norton 1996, 2001). Hence, indica-
Customer performance tors belonging to the financial and customer perspectives
Internal business process performance are assumed to measure performance outcomes, whereas
Performance related to learning and growth indicators from the perspectives of internal business pro-
cesses and “learning and growth” are considered as typi-
Fig. 1 An overview of the performance perspectives in Kaplan and
Norton (1996, 2001)
cal performance drivers (Kaplan and Norton 2004).

Table 1 An example of translating an organizational strategy into operational terms using the BSC
Perspective Strategy Objective Indicator, measure or Target Initiative
metric
Year 1 (%) Year 2 (%) Year 3 (%)

Customer Operational Industry-leading Customer satisfaction 80 85 90 Mystery shopper program

excellence customer loyalty rating Customer loyalty program
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 4 of 24

Despite its widespread use and acceptance, the BSC is to organizational performance, given the central role
also criticized for appearing too general by managers who of business processes in an organization. He does so by
are challenged to adapt it to the culture of their organiza- focusing more on the different stakeholders involved in
tion (Butler et al. 1997) or find suitable indicators to cap- certain business processes.
ture the various aspects of their organization’s strategy
(Shah et al. 2012; Vaivio 1999). Additionally, researchers Performance indicators
question the choice of four distinct performance perspec- Section “Performance measurement models” explained
tives (i.e., which do not include perspectives related to that performance measurement models typically dis-
inter-organizational performance or sustainability issues) tinguish different performance perspectives for which
(EFQM 2010; Hubbard 2009, Kueng 2000). Further, the performance indicators should be further defined. We
causal relationship among the BSC perspectives has been must, however, note that we consider performance meas-
questioned (Norreklit 2000). To some degree, Kaplan and ures, performance metrics and (key) performance indi-
Norton (2004) responded to this criticism by introducing cators as synonyms (Dumas et al. 2013). For reasons of
strategy maps that focus more on the causal relationships conciseness, this work will mainly refer to performance
and the alignment of intangible assets. indicators without mentioning the synonyms. In addi-
tion to a name, each performance indicator should also
Business process performance measurement models have a concretization or operationalization that describes
In addition to organizational models, performance meas- exactly how it is measured and that can result in a value
urement can also focus on a single business process, such to be compared against a target. For instance, regard-
as statistical process control, workflow-based monitoring ing the example in Table 1, the qualitative statements to
or process performance measurement systems (Kueng measure customer satisfaction constitute an operation-
2000; Neely et al. 2000). The approach taken in business alization. Nonetheless, different ways of operationaliza-
process performance measurement is generally less holis- tion can be applied to measure the same performance
tic than the BSC. For instance, in an established BPM indicator. Since organizations can profit from reusing
handbook, Dumas et al. (2013) position time, cost, qual- existing performance indicators and the related opera-
ity and flexibility as the typical performance perspectives tionalization instead of inventing new ones (i.e., to facili-
of business process performance measurement (Fig. 2). tate benchmarking and save time), this work investigates
Similar to organizational performance measurement, which performance indicators are used or mentioned in
concrete performance measures or indicators should be the literature on business process performance and how
defined for each process performance perspective. In this they are operationalized.
sense, the established perspectives of Dumas et al. (2013) Neely et al. (2000) and Richard et al. (2009) both pre-
seem to further refine the internal business process per- sent evaluation criteria for performance indicators (i.e.,
formance perspective of the BSC. in the sense of desirable characteristics or review impli-
Neely et al. (2000), on the other hand, present ten cations), which summarize the general consensus in
steps to develop or define process performance indica- the performance literature. First, the literature strongly
tors. The process performance measurement system of agrees that performance indicators are organization-
Kueng (2000) is also of high importance, which is visual- dependent and should be derived from an organization’s
ized as a “goal and performance indicator tree” with five objectives, strategy, mission and vision. Secondly, con-
process performance perspectives: (1) financial view, (2) sensus in the literature also exists regarding the need to
customer view, (3) employee view, (4) societal view, and combine financial and non-financial performance indi-
(5) innovation view. Kueng (2000) thus suggests a more cators. Nonetheless, disagreement still seems to exist
holistic approach towards process performance, similar in terms of whether objective and subjective indicators
need to be combined, with objective indicators preferred
by most advocates. Although subjective (or quasi-objec-
tive) indicators face challenges from bias, their use has
An established view on business process performance measurement some advantages; for instance, to include stakeholders in
an assessment, to address latent constructs or to facilitate
Time-related process performance
benchmarking when a fixed reference point is missing
Cost-related process performance
(Hubbard 2009; Richard et al. 2009). Moreover, empirical
Quality-related process performance
research has shown that subjective (or quasi-objective)
Flexibility-related process performance
indicators are more or less correlated with objective indi-
Fig. 2 An overview of the performance perspectives in Dumas et al. cators, depending on the level of detail of the subjective
(2013) question (Richard et al. 2009). For instance, a subjective
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 5 of 24

question can be made more objective by using clear defi- Formulating the research questions and search strategy
nitions or by selecting only well-informed respondents to A comprehensive and unbiased search is one of the fun-
reduce bias. damental factors that distinguish a systematic review
from a traditional literature review (Kitchenham 2007).
Methods For this purpose, a systematic search begins with the
We conducted a structured literature review (SLR) to identification of keywords and search terms that are
find papers dealing with performance measurement in derived from the research questions. Based on the
the business process literature. SLR can be defined as “a research questions stipulated in the introduction, the
means of evaluating and interpreting all available research SLR protocol (Boellt and Cecez-Kecmanovic 2015) for
relevant to a particular research question, topic area, or our study was defined, as shown in Table 2.
phenomenon of interest” (Kitchenham 2007: p. vi). An The ISI Web of Science (WoS) database was searched
SLR is a meta study that identifies and summarizes evi- using predetermined search terms in November 2015.
dence from earlier research (King and He 2005) or a way This database was selected because it is used by many
to address a potentially large number of identified sources universities and results in the most outstanding pub-
based on a strict protocol used to search and appraise lications, thus increasing the quality of our findings.
the literature (Boellt and Cecez-Kecmanovic 2015). It is An important requirement was that the papers focus
systematic in the sense of a systematic approach to find- on “business process*” (BP). This keyword was used in
ing relevant papers and a systematic way of classifying combination with at least one of the following: (1) “per-
the papers. Hence, according to Boellt and Cecez-Kec- formance indicator*”, (2) “performance metric*”, (3) “per-
manovic (2015), SLR as a specific type of literature review formance measur*”. All combinations of “keyword in
can only be used when two conditions are met. First, the topic” (TO) and “keyword in title” (TI) have been used.
topic should be well-specified and closely formulated Table 3 shows the degree to which the initial sample
(i.e., limited to performance measurement in the con- sizes varied, with 433 resulting papers for the most per-
text of business processes) to potentially identify all rel- missive search query (TOxTO) and 19 papers for the
evant literature based on inclusion and exclusion criteria. most restrictive one (TIxTI). The next stage started with
Secondly, the research questions should be answered by the most permissive search query in an effort to select
extracting and aggregating evidence from the identified and assess as many relevant publications as possible.
literature based on a high-level summary or bibliometric-
type of content analysis. Furthermore, King and He (2005) Filtering and extracting data
also refer to a statistical analysis of existing literature. Figure 3 summarizes the procedure for searching and
Informed by the established guidelines proposed by selecting the literature to be reviewed. The list of papers
Kitchenham (2007), we undertook the review in distinct found in the previous stage was filtered by deleting 35
stages: (1) formulating the research questions and the duplicates, and the remaining 398 papers were further
search strategy, (2) filtering and extracting data based narrowed to 153 papers by evaluating their title and
on inclusion and exclusion criteria, and (3) synthesizing abstract. After screening the body of the texts, 76 full-
the findings. The remainder of this section describes the text papers were considered relevant for our scope and
details of each stage. constituted the final sample (“Appendix 1”).

Table 2 The structured literature review protocol for this study, based on Boellt and Cecez-Kecmanovic (2015)
Protocol elements Translation to this study

1/Research question RQ1. What is the current state of the research on business process performance measurement?
RQ2. Which indicators, measures and metrics are used or mentioned in the current literature related to business process
performance?
2/Sources searched Web of science database (until November 2015)
3/Search terms Combining “business process*” and “performance indicator*”/“performance metric*”/“performance measur*”
4/Search strategy Different search queries, with keywords in topic and title (Table 3)
5/Inclusion criteria Include only papers containing a combination of search terms, defined in the search queries
Include only papers indexed in the Web of Science from all periods until November 2015
Include only papers written in English
6/Exclusion criteria Exclude unrelated papers, i.e., if they do not explicitly claim addressing the measurement of business process performance
7/Quality criteria Only peer-reviewed papers are indexed in the web of science database
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 6 of 24

Table 3 The number of papers in the web of science also synthesized by combining similar indicators and
per search query (until November 2015) rephrasing them into more generic terms.
(1) “Performance (2) “Performance (3) “Performance TOTAL The next phase was a comparative study to categorize
indicator*” metric*” measur*” the output of phase 1 into the commonly used measure-
ment models in the performance literature (see “Theo-
Column keywords in TO
retical background” section). For the purpose of this
BP-TO 153 30 250 433
study, we specifically looked for those organizational
BP-TI 31 4 64 99
performance models, mentioned in “Theoretical back-
Column keywords in TI
ground” section, that are cited the most and that suggest
BP-TO 19 2 62 83
categories, dimensions or performance perspectives that
BP-TI 5 0 14 19
can be re-used (Kaplan and Norton 1996, 2001; EFQM
2010; Cross and Lynch 1988; Kueng 2000). Since the BSC
More specifically, studies were excluded if their main (Kaplan and Norton 1996, 2001) is the most commonly
focus was not business process performance measure- used of these measurement models, we began with the
ment or if they did not refer to indicators, measures or BSC as the overall framework to categorize the observed
metrics for business performance. The inclusion of stud- indicators related to business (process) performance,
ies was not restricted to any specific type of intervention supplemented with an established view on process per-
or outcome. The SLR thus included all types of research formance from the process literature (Dumas et al. 2013).
studies that were written in English and published up to Subsequently, a structured list of potential performance
and including November 2015. Furthermore, publica- indicators was obtained.
tion by peer-reviewed publication outlets (e.g., journals In the third and final phase, an evaluation study was
or conference proceedings) was considered as a qual- performed to validate whether the output of phase 2 is
ity criterion to ensure the academic level of the research sufficiently comprehensive according to other perfor-
papers. mance measurement models, i.e., not included in our
sample and differing from the most commonly used
Synthesizing the findings performance measurement models. Therefore, we inves-
The analysis of the final sample was performed by means tigated the degree to which our structured list cov-
of narrative and descriptive analysis techniques. For RQ1, ers the items in two variants or concretizations of the
the 76 papers were analyzed on the basis of bibliometric BSC. Hence, a validation by other theoretical models is
data (e.g., publication type, publication year, geography) provided. We note that a validation by subject-matter
and general performance measurement issues by pay- experts is out of scope for a structured literature review
ing attention to the methodology and focus of the study. but relates to an opportunity for further research.
Details are provided in “Appendix 2”.
For RQ2, all the selected papers were screened to iden- Results for RQ1
tify concrete performance indicators in order to generate The final sample of 76 papers consists of 46 journal
a comprehensive list or checklist. The latter was done in papers and 30 conference papers (Fig. 4), indicating a
different phases. In the first phase, the structured litera- wide variety of outlets to reach the audience via opera-
ture review allowed us to analyze which performance tions and production-related journals in particular or
indicators are mainly used in the process literature and in lower-ranked (Recker 2013) information systems
how they are concretized (e.g., in a question or mathe- journals.
matical formulation), resulting in an unstructured list of When considering the chronological distribution of the
potential performance indicators. The indicators were sampled papers, Fig. 5 indicates an increase in the uptake

Exclusion and
Search database with Exclusion and
inclusion based on
ISI Web of Science the most permissive inclusion based on
detailed reading and
search query tle and abstract
data extracon

With duplicates: 433 With or without full text: 153

76 arcles
Without duplicates: 398 Only access to full text: 136
Fig. 3 Exclusion of papers and number of primary studies
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 7 of 24

of the topic in recent years, particularly for conference

80
papers but also for journal publications since 2005.
70
This uptake seems particularly situated in the Western
60 world and Asia (Fig. 6). The countries with five or more
Number of papers

50 46
papers in our sample are Germany (12 papers), the US (6
40 papers), Spain (5 papers), Croatia (5 papers) and China
30 (5 papers). Figure 6 shows that business process perfor-
20 mance measurement is a worldwide topic, with papers
30 across the different continents. Nonetheless, a possible
10
explanation for the higher coverage in the Western world
0
Publicaon type
could be due to its long tradition of measuring work (i.e.,
BSC origins).
Conference paper (39,47%) Journal paper (60,53%)
The vast majority of the sampled papers address arti-
Fig. 4 The distribution of the sampled papers per publication type facts related to business (process) performance measure-
(N = 76)
ment. When looking at the research paradigm in which
the papers are situated (Fig. 7), 71 % address design-sci-
ence research, whereas 17 % conduct research in behav-
11
ioral science and 12 % present a literature review. This
10 could be another explanation for the increasing uptake
9 in the Western world, as many design-science research-
8
Number of papers

7
ers are from Europe or North America (March and Smith
6 1995; Peffers et al. 2012).
5 Figure 8 supplements Fig. 7 by specifying the research
4
3 methods used in the papers. For the behavioral-science
2 papers, case studies and surveys are equally used. The
1 54 papers that are situated within the design-science
0
1990 1995 2000 2005 2010 2015 paradigm explicitly refer to models, meta-models,
frameworks, methods and/or tools. When mapping
All papers Conference papers Journal papers
these 54 papers to the four artifact types of March and
Fig. 5 The chronological distribution of the sampled papers per
publication type (N = 76) Smith (1995), the vast majority present (1) methods in
the sense of steps to perform a task (e.g., algorithms or

Europe
(incl. Turkey):
46 Asia: 17
North
America: 8

Africa: 2
South
America: 1
Australia: 2

Fig. 6 The geographical distribution of the sampled papers per continent, based on a paper’s first author (N = 76)
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 8 of 24

focus on the process level, organizational level or even

cross-organizational level. Nonetheless, the indicators
Literature
review
mentioned in those papers are illustrative rather than
12% comprehensive. An all-inclusive list of generic perfor-
mance indicators seems to be missing. Some authors pro-
BH pose a set of indicators, but those indicators are specific
17% to a certain domain or sector instead of being generic. For
instance, Table 4 shows that 36 of the 76 sampled papers
are dedicated to a specific domain or sector, such as tech-
nology-related aspects or supply chain management.
DSR Furthermore, the reviewed literature was analyzed with
71% regard to its (1) scope, (2) functionalities, (3) terminol-
ogy, and (4) foundations.
Starting with scope, it is observed that nearly two-
thirds of the sampled papers can be categorized as deal-
Fig. 7 The distribution of the sampled journal papers per research ing with process-oriented performance measurement,
paradigm (N = 76) whereas one-third focuses more on general performance
measurement and management issues. Nonetheless,
most of the studies of process performance also include
80 general performance measurement as a supporting con-
60 cept. A minor cluster of eight research papers specifically
40
45 focuses on business process reengineering and meas-
30 27
14 15 urement systems to evaluate the results of reengineer-
20 9 6 6
1
9 5 3 ing efforts. Furthermore, other researchers focus on the
0
measurement and assessment of interoperability issues
Expert validaon
Method
Both

Technical experiment
Case study

Model
Construct

Case study
Survey

Illustrave scenario
Instanaon

and supply chain management measurements.

Secondly, while analyzing the literature, two groups
of papers were identified based on their functionalities:
(1) focusing on performance measurement systems or
Review BHS DSR arfacts DSR evaluaon frameworks, and (2) focusing on certain performance
Fig. 8 The distribution of the sampled journal papers per research indicators and their categorization. Regarding the first
method (N = 76)

Table 4 The number of sampled papers dedicated to a

guidelines for performance measurement) and/or (2) specific domain or sector (N = 76)
models to describe solutions for the topic. The number of Domain or sector Number of papers
papers dealing with (3) constructs or a vocabulary and/
IS/IT 7
or (4) instantiations or tools is much more limited, with
14 construct-related papers and 9 instantiations in our Supply chain 5
sample. We also looked at which evaluation methods, Business network 3
defined by Peffers et al. (2012), are typically used in the Manufacturing 3
sampled design-science papers. While 7 of the 54 design- Services 3
science papers do not seem to report on any evaluation Automobile 2
effort, our sample confirms that most papers apply one or Banking/financial 2
another evaluation method. Case studies and illustrative Government 2
scenarios appear to be the most frequently used methods Health 2
to evaluate design-science research on business (process) Helpdesk/maintenance 2
performance measurement. Construction 1
The sampled design-science research papers typically HR 1
build and test performance measurement frameworks, SME 1
systems or models or suggest meta-models and generic Strategic planning 1
templates to integrate performance indicators into the Telecom 1
process models of an organization. Such papers can Total 36
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 9 of 24

group, it should be mentioned that while the process less precise. Hence, a great degree of ambiguity exists in
of building or developing a performance measurement the formulation of performance objectives among to the
system (PMS) or framework is well-researched, only a reviewed papers.
small number of papers explicitly address process per- Finally, regarding the papers’ foundations, “Perfor-
formance measurement systems (PPMS). The papers in mance measurement models” section already indicated
this first group typically suggest concrete steps or stages that the BSC plays an important role in the general litera-
to be followed by particular organizations or discuss the ture on performance management systems (PMS), while
conceptual characteristics and design of a performance Kueng (2000) also offers influential arguments on process
measurement system. Regarding the second group of performance measurement systems (PPMS). In our liter-
performance indicators, we can differentiate two sub- ature review, we observed that the BSC was mentioned in
groups. Some authors focus on the process of defining 43 of the 76 papers and that the results of 19 papers were
performance indicators by listing requirements or quality mainly based on the BSC (Fig. 9). This finding provides
characteristics that an indicator should meet. However, additional evidence that the BSC can be considered the
many more authors are interested in integrating perfor- most frequently used performance model in academia as
mance indicators into the process models or the whole well. However, the measurement model of Kueng (2000)
architecture of an organization, and they suggest con- was also mentioned in the sampled papers on PPMS,
crete solutions to do so. Compared to the first group of though less frequently (i.e., in six papers).
papers, this second group deals more with the categori- Interestingly, the BSC is also criticized by the sampled
zation of performance indicators into domains (financial/ papers for not being comprehensive; for instance, due
non-financial, lag/lead, external/internal, BSC dimen- to the exclusion of environmental aspects, supply chain
sions) or levels (strategic, tactical, operational). management aspects or cross-organizational processes.
Thirdly, regarding terminology, different terms are used In response, some of the sampled papers also define sec-
by different authors to discuss performance measure- tor-specific BSC indicators or suggest additional steps
ment. Performance “indicator” is the most commonly or indicators to make the process or business more sus-
used term among the reviewed papers. For instance, it is tainable (see Table 4). Nonetheless, the majority of the
frequently used in reference to a key performance indi- papers agree on the need for integrated and multidimen-
cator (KPI), a KPI area or a performance indicator (PI). sional measurement systems, such as the BSC, and on the
The concept of a process performance indicator (PPI) is importance of directly linking performance measurement
also used, mainly in the process-oriented literature. Per- to an organization’s strategy. However, while these papers
formance “measure” is another prevalent term in the mention the required link with strategy, the prioritization
papers. The least-used term is performance “metric” (i.e., of indicators according to their strategic importance has
in only nine papers). Although the concepts of perfor- been studied very little thus far.
mance indicators, measures and metrics are used inter-
changeably throughout most of the papers, the concepts Results for RQ2
are sometimes defined in different ways. For instance, For RQ2, the sampled papers were reviewed to dis-
paper 17 defines a performance indicator as a metric, tinguish papers with performance indicators from
and paper 49 defines a performance measure as an indi- papers without performance indicators. A further dis-
cator. On the other hand, paper 7 defines a performance tinction was made between indicators found with
indicator as a set of measures. Yet another perspective is
taken in paper 74, which defines a performance measure
as “a description of something that can be directly meas-
70
ured (e.g., number of reworks per day)”, while defining
a performance indicator as “a description of something 60

that is calculated from performance measures (e.g., per- 50

43
centage reworks per day per direct employee” (p. 386). 40
Inconsistencies exist not only in defining indicators
30
but also in describing performance goals. For instance, 19
some authors include a sign (e.g., minus or plus) or a 20

verb (e.g., decrease or increase) in front of an indicator. 10

Other authors attempt to describe performance goals in 0
a SMART way—for instance, by including a time indica- Papers men oning the BSC Papers based on the BSC
tion (e.g., “within a certain period”) and/or target (e.g., Fig. 9 The importance of the BSC according to the sampled papers
“5 % of all orders”)—whereas most of the authors are (N = 76)
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 10 of 24

Table 5 A description of the observed performance perspectives, linked to the Balanced scorecard (Kaplan and Norton
1996, 2001)
Initial BSC perspectives Observed perspectives based on target groups and focus Scope of the performance indicators

1. Financial performance 1.1 Financial performance for shareholders and top Strategic financial data
management
2. Customer-related perfor- 2.1 Customer performance Outcomes of external quality or meeting end user
mance needs
2.2 Supplier performance External collaboration and process dependencies
2.3 Society performance Outcomes for other stakeholders and the environ-
ment during process work
3. Internal business process 3.1 General process performance Descriptive data of process work, not related to
performance time, costs, quality or flexibility
3.2 Time-related process performance Time-related data of process work
3.3 Cost-related process performance Operational financial data
3.4 Process performance related to internal quality Capability of meeting end user needs and internal
user needs
3.5 Flexibility-related process performance Data of changes or variants in process work
4. Performance related to “learn- 4.1 (Digital) innovation performance Innovation of processes and innovation projects
ing and growth” 4.2 Employee performance Staff contributions to process work and personal
development

operationalization (i.e., concretization by means of a For reasons of objectivity, the observed performance
question or formula) and those without operationaliza- indicators were assigned to a single perspective starting
tion. We note that for many indicators, no operationali- from recognized frameworks (Kaplan and Norton 1996,
zation was available. We discovered that only 30 of the 2001; Dumas et al. 2013). Bias was further reduced by
76 sampled papers contained some type of performance following the definitions of Table 5. Furthermore, the
indicator (namely 3, 5, 6, 7, 11, 16, 17, 18, 20, 22, 26, 27, authors of this article first classified the indicators indi-
30, 35, 37, 40, 43, 46, 49, 51, 52, 53, 55, 57, 58, 59, 60, 66, vidually and then reached consensus to obtain a more
71, 73). In total, approximately 380 individual indicators objective categorization.
were found throughout all the sampled papers (including Additional rationale for the identification of 11 perfor-
duplicates), which were combined based on similarities mance perspectives is presented in Table 6, which com-
and modified to use more generic terms. This resulted in pares our observations with the perspectives adopted
87 indicators with operationalization (“Appendix 3”) and by the most commonly used performance measurement
48 indicators without operationalization (“Appendix 4”). models (see “Theoretical background” section). This
The 87 indicators with operationalization were then comparison allows us to highlight similarities and differ-
categorized according to the four perspectives of the BSC ences with other respected models. In particular, Table 6
(i.e., financial, customer, business processes, and “learn- shows that we did not observe a dedicated perspective
ing and growth”) (Kaplan and Norton 1996, 2001) and the for strategy (EFQM 2010) and that we did not differen-
four established dimensions of process performance (i.e., tiate between financial indicators and market indicators
time, cost, quality, and flexibility) (Dumas et al. 2013). In (Cross and Lynch 1988). Nonetheless, the similarities
particular, based in the identified indicators, we revealed in Table 6 prevail. For instance, Cross and Lynch (1988)
11 sub-perspectives within the initial BSC perspectives to also acknowledge different process dimensions. Further,
better emphasize the focus of the indicators and the dif- Kueng (2000) and the EFQM (2010) also differentiate
ferent target groups (Table 5): (1) financial performance employee performance from innovation performance,
for shareholders and top management, (2) customer- and they both add a separate perspective for results
related performance, (3) supplier-related performance, related to the entire society.
(4) society-related performance, (5) general process per- Figure 10 summarizes the number of performance
formance, (6) time-related process performance, (7) cost- indicators that we identified in the process literature per
related process performance, (8) process performance observed performance perspective. Not surprisingly,
related to internal quality, (9) flexibility-related process the initial BSC perspective of internal business process
performance, (10) (digital) innovation performance, and performance contains most of the performance indica-
(11) employee-related performance. tors: 29 of 87 indicators. However, the other initial BSC
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 11 of 24

perspectives are also covered by a relatively high num- cost-related performance indicator (i.e., process cost),
ber of indicators: 16 indicators for both financial per- two indicators of process performance related to inter-
formance and customer-related performance and 26 nal quality (i.e., quality of internal outputs and deadline
indicators for “learning and growth”. This result confirms adherence), and one indicator of employee performance
the close link between process performance and organi- (i.e., perceived employee satisfaction).
zational performance, as mentioned in the introduction. Consistent with “Performance indicators” section, the
A more detailed comparison of the perspectives pro- different performance perspectives are a combination
vides interesting refinements to the state of the research. of financial or cost-related indicators with non-financial
More specifically, Fig. 10 shows that five performance data. The latter also take the upper hand in our sam-
perspectives have more than ten indicators in the sample, ple. Furthermore, the sample includes a combination of
indicating that academic research focuses more on finan- objective and subjective indicators, and the vast major-
cial performance for shareholders and top management ity are objective indicators. Only eight indicators explic-
and performance related to customers, process time, itly refer to qualitative scales; for instance, to measure the
innovation and employees. On the other hand, fewer than degree of satisfaction of the different stakeholder groups.
five performance indicators were found in the sample For all the other performance indicators, a quantifiable
for the perspectives related to suppliers, society, process alternative is provided.
costs and process flexibility, indicating that the literature It is important to remember that a distinction was
focuses less on those perspectives. The latter remains made between the indicators with operationalization and
largely overlooked by academic research, possibly due to those without operationalization. The list of 87 perfor-
the newly emerging character of these perspectives. mance indicators, as given in “Appendix 3”, can thus be
We must, however, note that the majority of the per- extended with those indicators for which operationali-
formance indicators are mentioned in only a few papers. zation is missing in the reviewed literature. Specifically,
For instance, 59 of the 87 indicators were cited in a sin- we found 48 additional performance indicators (“Appen-
gle paper, whereas the remainder are mentioned in more dix 4”) that mainly address supplier performance, pro-
than one paper. Eleven performance indicators are fre- cess performance related to costs and flexibility, and the
quently mentioned in the process literature (i.e., by five employee-related aspects of digital innovation. Conse-
or more papers). These indicators include four indica- quently, this structured literature review uncovered a
tors of customer-related performance (i.e., customer total of 135 performance indicators that are directly or
complaints, perceived customer satisfaction, query time, indirectly linked to business process performance.
and delivery reliability), three indicators of time-related Finally, the total list of 135 performance indicators was
process performance (i.e., process cycle time, sub-pro- evaluated for its comprehensiveness by comparing the
cess turnaround time, and process waiting time), one identified indicators with other BSC variants that were

Table 6 The comparison of our observed performance perspectives with the perspectives taken in the most commonly
used performance measurement models in the literature (Kaplan and Norton 1996, 2001; EFQM 2010; Kueng 2000; Cross
and Lynch 1988)
Balanced scorecard (Kaplan EFQM (2010) Kueng (2000) Cross and Lynch Our observed performance perspec-
and Norton 1996, 2001) (1988) tives

Financial perspective Key results Financial view Financial measures Financial performance for shareholders
Market measures and top management
Customer perspective Customer results Customer view Customer satisfac- Customer performance
tion Supplier performance
Society performance
Internal business processes Enablers (processes/prod- Overall process perfor- Flexibility General process performance
perspective ucts/services, people, mance based on the Productivity Time-related process performance
strategy, partnerships/ other views as driving Quality Cost-related process performance
resources, leadership) forces Delivery Process performance related to internal
Process time quality
Cost Flexibility-related process performance
“Learning and growth” per- People results Employee view – (Digital) innovation performance
spective Learning, creativity and Innovation view Employee performance
innovation
– Society results Societal view – Society performance as a sub-perspec-
tive of customer performance (see
above)
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 12 of 24

20
the BSC. Most of the sampled papers mentioned or used
16
15 14 the BSC as a starting point and basis for their research
11 12 12
10 and analysis. The literature study also showed a variety
7
5 3
2
4 5
1
of research topics, ranging from behavioral-science to
0 design-science research and from a focus on performance

Internal quality
Suppliers
Customers

Society

Employee
Time

Flexibility
General process info
Shareholders and top

Cost

(Digital) innovaon
measurement models to a focus on performance indica-
management

tors. In addition to inconsistencies in the terminology

used to describe performance indicators and targets, the
main weakness uncovered in this literature review deals
with the concretization of performance indicators sup-
Financial Customer Internal business processes Learning & Growth
plementing performance measurement systems. The SLR
Fig. 10 The number of performance indicators with operationaliza-
tion per performance perspective results suggest that none of the reviewed papers offers
a comprehensive measurement framework, specifically
one that includes and extends the BSC perspectives, is
not included in our sample. More specifically, based on process-driven and encompasses as many concrete per-
a random search, we looked for two BSC variants in the formance indicators as possible. Such a comprehensive
Web of Science that did not fit the search strategy of framework could be used as a checklist or a best practice
this structured literature review: one that did not fit the for reference when defining specific performance indi-
search term of “business process*” (Hubbard 2009) and cators. Hence, the current literature review offers a first
another that did not fit any of the performance-related step towards such a comprehensive framework by means
search terms of “performance indicator*”, “performance of an extended list of possible performance indicators
metric*” or “performance measur*” (Bronzo et al. 2013). bundled in 11 performance perspectives (RQ2).
These two BSC variants cover 30 and 17 performance Regarding RQ2 on process performance indicators,
indicators, respectively, and are thus less comprehensive the literature study revealed that scholars measure per-
than the extended list presented in this study. Most of the formance in many different ways and without shar-
performance indicators suggested by the two BSC vari- ing much detail regarding the operationalization of the
ants are either directly covered in our findings or could measurement instruments, which makes a comparison of
be derived after recalculations. Only five performance research results more difficult. As such, the extended list
indicators could not be linked to our list of 135 indica- of performance indicators is our main contribution and
tors, and these suggest possible refinements regarding (1) fills a gap in the literature by providing a detailed over-
the growth potential of employees, (2) new markets, (3) view of performance indicators mentioned or used in
the social performance of suppliers, (4) philanthropy, or the literature on business process performance. Another
(5) industry-specific events. novel aspect is that we responded to the criticism of
missing perspectives in the original BSC (EFQM 2010;
Discussion Hubbard 2009; Kueng 2000) and identified the narrow
This structured literature review culminated in an view of performance typically taken in the process litera-
extended list of 140 performance indicators: 87 indicators ture (Dumas et al. 2013). Figures 1 and 2 are now com-
with operationalization, 48 indicators without operation- bined and extended in a more exhaustive way, namely
alization and 5 refinements derived from two other BSC by means of more perspectives than are offered by other
variants. The evaluation of our findings against two BSC attempts (Table 6), by explicitly differentiating between
variants validated our work in the sense that we present performance drivers (or lead indicators) and perfor-
a more exhaustive list of performance indicators, with mance outcomes (or lag indicators), and by considering
operationalization for most, and that only minor refine- concrete performance indicators.
ments could be added. However, the comprehensiveness Our work also demonstrated that all perspectives in the
of our findings can be claimed only to a certain extent BSC (Kaplan and Norton 1996, 2001) relate to business
given the limitations of our predefined search strategy process performance to some degree. In other words,
and the lack of empirical validation by subject-matter while the BSC is a strategic tool for organizational per-
experts or organizations. Notwithstanding these limita- formance measurement, it is actually based on indicators
tions, conclusions can be drawn from the large sample of that originate from business processes. More specifi-
76 papers to respond to the research questions (RQs). cally, in addition to the perspective of internal business
Regarding RQ1 on the state of the research on business processes, the financial performance perspective typi-
process performance measurement, the literature review cally refers to sales or revenues gained while doing busi-
provided additional evidence for the omnipresence of ness, particularly after executing business processes. The
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 13 of 24

customer perspective relates to the implications of prod- The eleven performance perspectives (Fig. 11) can be
uct or service delivery, specifically to the interactions used by organizations and scholars to measure the per-
throughout business processes, whereas the “learning formance of business processes in a more holistic way,
and growth” perspective relates to innovations in the way considering the implications for different target groups.
of working (i.e., business processes) and the degree to For each perspective, performance indicators can be
which employees are prepared to conduct and innovate selected that fit particular needs. Thus, we do not assert
business processes. The BSC, however, does not present that every indicator in the extended list of 140 perfor-
sub-perspectives and thus takes a more high-level view mance indicators should always be measured, since
of performance. Hence, the BSC can be extended based “Theoretical background” section emphasized the need
on other categorizations made in the reviewed litera- for organization-dependent indicators aligned with an
ture; for instance, related to internal/external, strategic/ organization’s strategy. Instead, our extended list can be
operational, financial/non-financial, or cost/time/quality/ a starting point for finding and using appropriate indi-
flexibility. cators for each performance perspective, without los-
Therefore, this study refined the initial BSC perspec- ing much time reflecting on possible indicators or ways
tives into eleven performance perspectives (Fig. 11) by to concretize those indicators. Similarly, the list can be
applying three other performance measurement models used by scholars, since many studies in both the process
(Cross and Lynch 1988; EFQM 2010; Kueng 2000) and literature and management literature intend to measure
the respected Devil’s quadrangle for process performance the performance outcomes of theoretical constructs or
(Dumas et al. 2013). Additionally, a more holistic view of developed artifacts.
business process performance can be obtained by meas- Consistent with the above, we acknowledge that the
uring each performance perspective of Fig. 11 than can observed performance indicators originate from differ-
be achieved by using the established dimensions of time, ent models and paradigms or can be specific to certain
cost, quality and flexibility as commonly proposed in the processes or sectors. Since our intention is to provide
process literature (Dumas et al. 2013). As such, this study an exhaustive list of indicators that can be applied to
demonstrated a highly relevant synergy between the dis- measure business process performance, the indica-
ciplines of process management, organization manage- tors are not necessarily fully compatible. Instead, our
ment and performance management. findings allow the recognition of the role of a business
We also found out that not all the performance per- context (i.e., the peculiarities of a business activity, an
spectives in Fig. 11 are equally represented in the studied organization or other circumstances). For instance, a
literature. In particular, the perspectives related to sup- manufacturing organization might choose different
pliers, society, process costs and process flexibility seem indicators from our list than a service or non-profit
under-researched thus far. organization (e.g., manufacturing lead time versus
friendliness, or carbon dioxide emission versus stake-
holder satisfaction).
Another point of discussion is dedicated to the dif-
A holisc view on ference between the performance of specific processes
business process performance (known as “process performance”) and the performance
measurement
of the entire process portfolio (also called “BPM per-
Financial performance for
formance”). While some indicators in our extended list
lag indicators
Outcomes,

Customer
shareholders and top
performance
management clearly go beyond a single process (e.g., competence-
Society / related indicators or employee absenteeism), it is our
Supplier
environmental
performance
performance opinion that the actual performance of multiple pro-
cesses can be aggregated to obtain BPM performance
Internal business process performance
(e.g., the sum of process waiting times). This distinction
General process performance
between (actual) process performance and BPM perfor-
Time-related process Cost-related process
mance is useful; for instance, for supplementing models
lead indicators

performance performance
Drivers,

Process performance Flexibility-related process that try to predict the (expected) performance based on
related to internal quality performance capability development, such as process maturity models
Learning and growth (e.g., CMMI) and BPM maturity models (Hammer 2007;
(Digital) innovaon Employee
performance performance
McCormack and Johnson 2001). Nonetheless, since this
study has shown a close link between process perfor-
Fig. 11 An overview of the observed performance perspectives in mance, BPM performance, and organizational perfor-
the business process literature mance, it seems better to refer to different performance
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 14 of 24

perspectives than to differentiate between such perfor- practitioners and researchers can select appropriate indi-
mance types. cators depending on their needs. In total, the structured
In future research, the comprehensiveness of the literature review resulted in 140 possible performance
extended list of performance indicators can be empiri- indicators: 87 indicators with operationalization, 48 addi-
cally validated by subject-matter experts. Additionally, tional indicators that need further concretization, and 5
case studies can be conducted in which organizations refinements based on other Balanced Scorecard (BSC)
apply the list as a supplement to performance measure- variants. As such, the 11 performance perspectives with
ment models in order to facilitate the selection of indica- related indicators can be considered a conceptual frame-
tors for their specific business context. The least covered work that was derived from the current process literature
perspectives in the academic research also seem to be and theoretically validated by established measurement
those that are newly emerging (namely, the perspectives approaches in organization management.
related to close collaboration with suppliers, society/ Future research can empirically validate the conceptual
sustainability and process flexibility or agility), and these framework by involving subject-matter experts to assess
need more attention in future research. Another research the comprehensiveness of the extended list and refine the
avenue is to elaborate on the notion of a business con- missing concretizations, and by undertaking case stud-
text; for instance, by investigating what it means to have ies in which the extended list can be applied by specific
a strategic fit (Venkatraman 1989) in terms of perfor- organizations. Other research avenues exist to investi-
mance measurement and which strategies (Miller and gate the link between actual process performance and
Friesen 1986; Porter 2008; Treacy and Wiersema 1993) expected process performance (as measured in maturity
are typically associated with which performance indica- models) or the impact of certain strategic or environmen-
tors. Additionally, the impact of environmental aspects, tal aspects on the choice of specific performance indica-
such as market velocity (Eisenhardt and Martin 2000), on tors. Such findings are needed to supplement and enrich
the choice of performance indicators can be taken into existing performance measurement systems.
account in future research.
Abbreviations
Conclusion BH: behavioral science; BPM: business process management; BSC: balanced
scorecard; DS: design-science; RQ: research question; SLR: structured literature
Business quotes such as “If you cannot measure it, you review; TO: keyword in topic; TI: keyword in title.
cannot manage it” or “What is measured improves” (P.
Drucker) are sometimes criticized because not all impor- Authors’ contributions
AVL initiated the conception and design of the study, while AS was respon-
tant things seem measurable (Ryan 2014). Nonetheless, sible for the collection of data (sampling) and identification of performance
given the perceived need of managers to measure their indicators. The analysis and interpretation of the data was conducted by
business and the wide variety of performance indica- both authors. AVL was involved in drafting and coordinating the manuscript,
and AS in reviewing it critically. Both authors read and approved the final
tors (i.e., ranging from quantitative to qualitative and manuscript.
from financial to non-financial), this structured literature
review has presented the status of the research on busi- Acknowledgements
We thank American Journal Experts (AJE) for English language editing.
ness process performance measurement. This structured
approach allowed us to detect weaknesses or inadequa- Competing interests
cies in the current literature, particularly regarding the The authors declare that they have no competing interests.
definition and concretization of possible performance Availability of data and materials
indicators. We continued by taking a holistic view of the The datasets supporting the conclusions of this article are included within the
categorization of the observed performance indicators article (and its additional files).
(i.e., measures or metrics) into 11 performance perspec- Consent for publication
tives based on relevant performance measurement mod- Not applicable.
els and established process performance dimensions.
Ethical approval
The identified performance indicators within the 11 This article does not contain any studies with human participants or animals
perspectives constitute an extended list from which performed by any of the authors.
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 15 of 24

Appendix 1
See Table 7.

Table 7 The final list of sampled papers (N = 76)

1 Huang SY, Lee CH, Chiu AA, Yen DC (2015) How business process reengineering affects information technology investment and employee perfor-
mance under different performance measurement. Inf Syst Front 17(5):1133–1144. doi: 10.1007/s10796-014-9487-4
2 Padua, SID, Jabbour CJC (2015) Promotion and evolution of sustainability performance measurement systems from a perspective of business
process management: From a literature review to a pentagonal proposal. Bus Process Manag J 21(2):403–418. doi:10.1108/BPMJ-10-2013-0139
3 Rinaldi M, Montanari R, Bottani E (2015) Improving the efficiency of public administrations through business process reengineering and simulation:
A case study. Bus Process Manag J 21(2):419–462. doi:10.1108/BPMJ-06-2014-0054
4 Camara MS, Ducq Y, Dupas R (2014) A methodology for the evaluation of interoperability improvements in inter-enterprises collaboration based
on causal performance measurement models. Int J Comput Integr Manuf 27(2):103–119
5 Lehnert M, Linhart A, Röglinger M (2014) Chopping down trees versus sharpening the axe—Balancing the development of BPM capabilities with
process improvement. In: Sadiq S, Soffer P, Völzer H (Eds) BPM 2014. LNCS 8659. Springer, Switzerland, pp 151–167
6 del-Rio-Ortega A, Resinas M, Cabanillas C, Ruiz-Cortes A (2013) On the definition and design-time analysis of process performance indicators. Inf
Syst 38(4): 470–490
7 Balaban N, Belic K, Gudelj M (2011) Business process performance management: theoretical and methodological approach and implementation.
Manag Inf Syst 6(4):003–009
8 Glykas M (2013) Fuzzy cognitive strategic maps in business process performance measurement. Expert Syst Appl 40(1):1–14. doi:10.1016/j.
eswa.2012.01.078
9 Hernaus T, Bach MP, Bosilj-Vuksic V (2012) Influence of strategic approach to BPM on financial and non-financial performance. Balt J Manag
7(4):376–396. doi:10.1108/17465261211272148
10 Akyuz GA, Erkan TE (2010) Supply chain performance measurement : a literature review. Int J Prod Res 48(17):5137–5155.
doi:10.1080/00207540903089536
11 Han KH, Choi SH, Kang JG, Lee G (2010) Performance-centric business activity monitoring framework for continuous process improvement. AIKED
Proceedings of WSEAS, pp 40–45. Available via [Link] Accessed Apr 2016
12 Han KH, Kang JG, Song M (2009) Two-stage process analysis using the process-based performance measurement framework and business process
simulation. Expert Syst Appl 36(3):7080–7086. doi:10.1016/[Link].2008.08.035
13 Cheng MY, Tsai HC, Lai YY (2009) Construction management process reengineering performance measurements. Autom Constr 18(2):183–193.
doi:10.1016/[Link].2008.07.005
14 Alfaro JJ, Rodriguez–Rodriguez R, Verdecho MJ, Ortiz, A (2009) Business process interoperability and collaborative performance measurement. Int J
Comput Integr Manuf 22(9):877–889. doi:10.1080/09511920902866112
15 Pakseresht M, Seyyedi MA, Zade MM, Gardesh H (2009) Business process measurement model based on the fuzzy multi agent systems. AIKED
Proceedings of WSEAS, pp 501–506
16 Bosilj-Vuksic V, Milanovic L, Skrinjar R, Indihar-Stemberger M (2008) Organizational performance measures for business process management: A
performance measurement guideline. Tenth International Conference on Computer Modeling and Simulation (UKSIM Proceedings), pp 94–99.
doi:10.1109/UKSIM.2008.114
17 Wetzstein B, Ma Z, Leymann F (2008) Towards measuring key performance indicators of semantic business processes. In: Abramowicz W, Fensel D
(Eds) BIS 2008, LNBIP vol 7. Springer, Berlin Heidelberg, pp 227–238. doi:10.1007/978-3-540-79396-0_20
18 Glavan LM (2012) Understanding process performance measurement systems. Bus Sys. Res J 2(2):25–38. doi:10.2478/v10305-012-0014-0
19 vom Brocke J (2007) Service portfolio measurement: evaluating financial performance of service-oriented business processes. Int J Web Serv Res
4(2):1–33
20 Korherr B, List B (2007a) Extending the EPC with performance measures. ACM Symposium on Applied Computing, pp 1265–1266
21 Korherr B, List B (2007b) Extending the EPC and the BPMN with business process goals and performance measures. ICEIS Proceedings, pp 287–294
22 Herzog NV, Polajnar A, Pizmoht P (2006) Performance measurement in business process re-engineering. J Mech Eng 52(4):210–224
23 Korherr B, List B (2006) Extending the UML 2 activity diagram with business process goals and performance measures and the mapping to BPEL. In:
Roddick JF et al. (Eds) ER Workshops 2006. LNCS, vol 4231. Springer, Berlin Heidelberg, pp 7–18. doi:10.1007/11908883_4
24 Lenz K, Mevius M, Oberweis A (2005) Process-oriented business performance management with Petri nets. IEEE Proceedings, pp 89–92
25 Kuwaiti ME (2004) Performance measurement process: definition and ownership. International Journal of Operations & Production Management,
24(1):55–78
26 Kutucuoglu KY, Hamali J, Sharp JM, Irani Z (2002) Enabling BPR in maintenance through a performance measurement system framework. Int J Oper
Prod Manag 14(1): 33–52. doi:10.1023/A:1013870802492
27 Jagdev H, Bradley P, Molloy O (1997) A QFD based performance measurement tool. Comput Ind 33(2–3):357–366. doi:10.1016/S0166-
3615(97)00041-9
28 Bititci US, Carrie AS, McDevitt L (1997) Performance management: A business process view. IFIP WG 5.7 Proceedings, pp 284–297
29 del-Rio-Ortega A, Cabanillas C, Resinas M, Ruiz-Cortes A (2013) PPINOT tool suite: a performance management solution for process-oriented
organisations. In: Basu S et al. (Eds) ICSOC Proceedings. LNCS, vol 8274. Springer, Berlin Heidelberg, pp 675–678. doi:10.1007/978-3-642-45005-
1_58
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 16 of 24

Table 7 continued
30 Mirsu DB (2013) Monitoring help desk process using KPI. In: Balas VE et al. (Eds) Soft Comput Appl 195:637–647
31 Koetter F, Kochanowski M (2012) Goal-oriented model-driven business process monitoring using ProGoalML. In: Abramowicz W et al. (Eds) BIS
2012. LNBIP, vol 117. Springer, Berlin Heidelberg, pp 72–83. doi:10.1007/978-3-642-30359-3_7
32 del-Rio-Ortega A, Resinas M, Duran A, Ruiz-Cortes A (2012) Defining process performance indicators by using templates and patterns. In: Barros A,
Gal A, Kindler E (Eds) BPM 2012. LNCS, vol 7481. Springer, Berlin Heidelberg, pp 223–228. doi:10.1007/978-3-642-32885-5_18
33 Arigliano F, Bianchini D, Cappiello C, Corallo A, Ceravolo P, Damiani E, De Antonellis V, Pernici B, Plebani P, Storelli D, Vicari C (2012) Monitoring busi-
ness processes in the networked enterprise. In: Aberer K, Damiani E, Dillon T (Eds) SIMPDA 2011. LNBIP, vol 116. Springer, Berlin Heidelberg, pp
21–38
34 Wetzstein B, Leitner P, Rosenberg F, Dustdar S, Leymann F (2011) Identifying influential factors of business process performance using dependency
analysis. Enterp Inf Syst 5(1):79–98. doi:10.1080/17517575.2010.493956
35 Shamsaei A, Pourshahid A, Amyot D (2011) Business process compliance tracking using key performance indicators. In: zur Muehlen M, Su J (Eds)
BPM 2010 Workshops. LNBIP, vol 66. Springer, Berlin Heidelberg, pp 73–84
36 del-Rio-Ortega A, Resinas M, Ruiz-Cortes A (2010) Defining process performance indicators: An ontological approach. In: Meersman R et al. (Eds)
OTM 2010, Part 1. LNCS, vol 6426. Springer, Berlin Heidelberg, pp 555–572
37 Pourshahid A, Amyot D, Peyton L, Ghanavati S, Chen P, Weiss M, Forster A J (2009) Business process management with the user requirements nota-
tion. Electron Commer Res 9(4):269–316. doi:10.1007/s10660-009-9039-z
38 Wetzstein B, Leitner P, Rosenberg F, Brandic I, Dustdar S, Leymann F (2009) Monitoring and analyzing influential factors of business process perfor-
mance. IEEE EDOC Proceedings, pp 141–150. doi:10.1109/EDOC.2009.18
39 Liu B, Fan Y, Huang S (2008) A service-oriented business performance evaluation model and the performance-aware service selection method.
Concurr Comput Pract Exp 20(15):1821–1836
40 Longo A, Motta G (2006) Design processes for sustainable performances: a model and a method. In: Bussler C et al. (Eds) BPM 2005 Workshops.
LNCS, vol 3812. Springer, Berlin Heidelberg, pp 399–407
41 Zakarian A, Wickett P, Siradeghyan Y (2006) Quantitative model for evaluating the quality of an automotive business process. Int J Prod Res
44(6):1055–1074. doi:10.1080/00207540500371949
42 Wieland U, Fischer M, Pfitzner M, Hilbert A (2015) Process performance measurement system—towards a customer-oriented solution. Bus Process
Manag J 21(2):312–331. doi:10.1108/BPMJ-04-2014-0032
43 Vernadat F, Shah L, Etienne A, Siadat A (2013) VR-PMS: a new approach for performance measurement and management of industrial systems. Int J
Prod Res 51(23–24):7420–7438
44 Zutshi A, Grilo A, Jardim-Goncalves R (2012) The business interoperability quotient measurement model. Comput Ind 63(5):389–404. doi:10.1016/j.
compind.2012.01.002
45 Ciemleja G, Lace N (2011) The model of sustainable performance of small and medium-sized enterprise. Eng Econ 22(5):501–509. doi: 10.5755/j01.
ee.22.5.968
46 Chimhamhiwa D, van der Molen P, Mutanga O, Rugege D (2009) Towards a framework for measuring end to end performance of land administra-
tion business processes—A case study. Comput Environ Urban Syst 33(4):293–301. doi: 10.1016/[Link].2009.04.001
47 Albayrak CA, Gadatsch A, Olufs D (2009) Life cycle model for IT performance measurement: a reference model for small and medium enterprises
(SME). In: Dhillon G, Stahl BC, Baskerville R (Eds) CreativeSME 2009. IFIP AICT, vol 301, pp 180–191. Available via [Link]
ter/10.1007/978-3-642-02388-0_13
48 Hinrichs N, Barke E (2008) Applying performance management on semiconductor design processes. IEEE IEEM Proceedings, pp 278–281.
doi:10.1109/IEEM.2008.4737874
49 Adams TM, Danijarsa M, Martinelli T, Stanuch G, Vonderohe A (2003) Performance measures for winter operations. Transp Res Rec J Transp Res
Board 1824:87–97. doi: 10.3141/1824-10
50 Kueng P (2000) Process performance measurement system: a tool to support process-based organizations. Total Qual Manag 11(1):67–85. doi:
10.1080/0954412007035
51 Kueng P, Krahn AJW (1999) Process performance measurement system: some early experiences. J Scien Ind Res 58(3–4):149–159
52 Walsh P (1996) Finding key performance drivers: some new tools. Total Quality Management. 7(5):509–519. doi: 10.1080/09544129610612
53 Fogarty DW (1992) Work in process: performance measures. Int J Prod Econ 26(1–3):169–172. doi:10.1016/0925-5273(92)90059-G
54 Gunasekaran A, Patel C, McGaughey RE (2004) A framework for supply chain performance measurement. Int J Prod Econ 87(3):333–347.
doi:10.1016/[Link].2003.08.003
55 Gunasekaran A, Kobu B (2007) Performance measures and metrics in logistics and supply chain management : a review of recent literature
(1995—2004) for research and applications. Int J Prod Res 45(12):37–41. doi:10.1080/00207540600806513
56 Wang CH, Lu IY, Chen CB (2010) Integrating hierarchical balanced scorecard with non-additive fuzzy integral for evaluating high technology firm
performance. Int J Prod Econ 128(1):413–426. doi:10.1016/[Link].2010.07.042
57 Wu HY (2012) Constructing a strategy map for banking institutions with key performance indicators of the balanced scorecard. Eval Program Plann
35(3):303–320. doi:10.1016/[Link].2011.11.009
58 Martinsons M, Davison R, Tse D (1999) The balanced scorecard: a foundation for the strategic management of information systems. Decis Support
Syst 25(1):71–88. doi: 10.1016/S0167-9236(98)00086-4
59 Grigoroudis E, Orfanoudaki E, Zopounidis C (2012) Strategic performance measurement in a healthcare organisation: A multiple criteria approach
based on balanced scorecard. Omega 40(1):104–119. doi:10.1016/[Link].2011.04.001
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 17 of 24

Table 7 continued
60 Bhagwat R, Sharma MK (2007) Performance measurement of supply chain management: a balanced scorecard approach. Comput Ind Eng
53(1):43–62. doi:10.1016/[Link].2007.04.001
61 Al-Mashari M, Al-Mudimigh A, Zairi M (2003) Enterprise resource planning: a taxonomy of critical factors. Eur J Oper Res 146(2):52–364. doi:10.1016/
S0377-2217(02)00554-4
62 Jalali NSG, Aliahmadi AR, Jafari EM (2011) Designing a mixed performance measurement system for environmental supply chain management
using evolutionary game theory and balanced scorecard: a case study of an auto industry supply chain. Resour Conserv Recycl 55(6):593–603.
doi: 10.1016/[Link].2010.10.008
63 Huang HC (2009) Designing a knowledge-based system for strategic planning: a balanced scorecard perspective. Expert Syst Appl 36(1):209–218.
doi:10.1016/[Link].2007.09.046
64 Bosilj-Vuksic V, Glavan LM, Susa D (2015) The role of process performance measurement in BPM adoption outcomes in Croatia. Econ Bus Rev
17(1):117–143. Available via [Link]
65 Jahankhani H, Ekeigwe JI (2005) Adaptation of the balanced scorecard model to the IT functions. IEEE ICITA Proceedings, pp 784–787. doi:10.1109/
ICITA.2005.52
66 Spremic M, Zmirak Z, Kraljevic K (2008) IT and business process performance management: case study of ITIL implementation in finance service
industry. ITI Proceedings, pp 243–250. doi:10.1109/ITI.2008.4588415
67 Li S, Zhu H (2008) Generalized stochastic workflow net-based quantitative analysis of business process performance. IEEE ICINFA Proceedings, pp
1040–1044. doi:10.1109/ICINFA.2008.4608152
68 Cardoso ECS (2013) Towards a methodology for goal-oriented enterprise management. IEEE EDOC Proceedings, pp 94–103. doi:10.1109/
EDOCW.2013.17
69 Tung A, Baird K, Schoch HP (2011) Factors influencing the effectiveness of performance measurement systems. Int J Oper Prod Manag
31(12):1287–1310. doi:10.1108/01443571111187457
70 Koetter F, Kochanowski M (2015) A model-driven approach for event-based business process monitoring. Inf Syst E-bus Manag 13(1):5–36.
doi:10.1007/s10257-014-0233-8
71 Banker RD, Chang H, Janakiraman SN, Konstans C (2004) A balanced scorecard analysis of performance metrics. Eur J Oper Res 154(2):423–436.
doi:10.1016/S0377-2217(03)00179-6
72 Peng Y, Zhou L (2011) A performance measurement system based on BSC. In: Zhu M (Ed) ICCIC 2011, Part V. CCIS, vol 235. Springer, Berlin Heidel-
berg, pp 309–315
73 van Heck G, van den Berg J, Davarynejad M, van Duin R, Roskott B (2010) Improving inventory management performance using a process-oriented
measurement framework. In: Quintela Varajao JE et al. (Eds) CENTERIS 2010, Part I. CCIS, vol 109. Springer, Berlin Heidelberg, pp 279–288
74 Caputo E, Corallo A, Damiani E, Passiante G (2010) KPI modeling in MDA Perspective. In: Meersman R et al. (Eds) OTM 2010 Workshops. LNCS, vol
6428. Springer, Berlin Heidelberg, pp 384–393. doi:10.1007/978-3-642-16961-8_59
75 Behrouzi F, Shaharoun AM, Ma’aram A (2014) Applications of the balanced scorecard for strategic management and performance measurement in
the health sector. Aust Heal Rev 38(2):208–217. doi:10.1071/AH13170
76 Skrinjar R, Indihar-Stemberger M (2009) Improving organizational performance by raising the level of business process orientation maturity:
empirical test and case study. In: Barry C et al. (Eds) Information Systems Development: Challenges in Practice, Theory and Education. Springer,
Heidelberg, pp 723–740. doi:10.1007/978-0-387-78578-3_11

Appendix 2: The mapping of the structured

literature review
The mapping details per sampled paper can be found
here.
[Link]
d / 0 B _ 2 Vp j w s R L r l R H h f R H J 4 Z F B Wd E E /
view?usp=sharing.
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 18 of 24

Appendix 3
See Table 8.

Table 8 The list of performance indicators with operationalization

Perspectives Indicators/measures/metrics Operationalization Papers

1/Financial performance
Sales performance [Achieved total sales]/[planned sales] * 100 7
Inventory turnover [Annual total sales]/[average inventory] * 100 59
Market share % of growth in the last years [Sales volumes of products and 16, 57
services]/[total market demands] * 100
Earnings per share (EPS) [After-tax net earnings − preferred share dividends]/ 57
[weighted average nr of shares outstanding]
Average order value [Aggregated monthly sales]/[monthly nr of orders] 7
Order growth [Number of orders in the current month]/[total nr of orders] 7
Revenue growth [Revenue from new sources]/[total revenue] * 100 16
Operating revenue Sales revenues 57
Return on investment (ROI) [After-tax profit or loss]/[total costs] 57, 55
[Revenue − cost]/[cost]
Return on assets (ROA) [After-tax profit or loss]/[average total assets] 57, 16
Circulation of assets [Operating revenues]/[assets] * 100 59
Current ratio [Current assets]/[current liabilities] * 100 59
Net profit margin [After-tax profit or loss]/[total operating revenues] [Total 16, 57, 59
operating revenues − operating expenses − non-operating
expenses]/[total operating revenues]
Profit per customer [After-tax earnings]/[total nr of online, offline or all customers] 57
Management efficiency [Operating expenses]/[operating revenues] * 100 59
Debt ratio, leverage level [Debts]/[assets] 57, 59
2/Customer performance
2.1/Customer performance
Customer complaints, return rate Nr of complaints, criticisms or notifications due to dissatis- 27, 30, 37, 40, 51,
faction about or non-compliance of orders, products and 57, 59
services
Nr or % of orders returned, rework or services to be redone
(e.g., incorrect deliveries, incorrect documentation)
Perceived customer satisfaction Qualitative scale on general satisfaction (e.g., Likert), possibly 5, 16, 22, 40, 46, 11,
indexed as the weighted sum of judgements on satisfaction 55 57, 59, 58, 60
dimensions (e.g., satisfaction with products and services,
perceived value, satisfying end-user needs, being the
preferred suppliers for products or services, responsiveness,
appearance, cleanliness, comfort, friendliness, communica-
tion, courtesy, competence, availability, security)
Perceived customer easiness Qualitative scale (e.g., Likert) on the degree of easiness to find 40
information and regulations, to fill out applications, and to
understand the presentation of bureaucratic language
Customer retention Nr of returning customers 57
Customer growth Nr of new customers 57
Customer query time, resolution Average time between issuing and addressing a customer 30, 40, 46, 58, 59, 60
time, response time problem or inquiry for information
Customer waiting time [Time for information about a product or service] + [time for 3, 40, 52, 59
following status updates] + [time for receiving the product
or service]
Max nr of customers in the queue or waiting room
[Handled requests]/[total requests]
Punctuality, delivery reliability [Late deliveries or requests]/[total nr of deliveries or requests] 16, 18, 26, 27, 40,
% of On-time deliveries according to the planning or schedule 51, 55, 60, 73
Payment reliability [Nr of collected orders paid within due date]/[total nr of 7
orders] * 100
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 19 of 24

Table 8 continued
Perspectives Indicators/measures/metrics Operationalization Papers
Information access cost, informa- Information provided/not provided 40
tion availability Time spent in asking for information about a product or
service (in days)
Time required to get updated about the status of a product
or service
Cost of information (euro)
Customer cost Product cost or the cost of using a service (euro) 40
2.2/Supplier performance
External delays Nr of delayed deliveries due to outage or delays of third-party 26, 73
suppliers
External mistakes % of Incorrect orders received 27
Transfers, partnerships % of Cases transferred to a partner 59
2.3/Society performance
Perceived society satisfaction Qualitative scale on general satisfaction (e.g., Likert), possibly 46
indexed as the weighted sum of judgements on satisfaction
dimensions
% of Society satisfied with the organization’s outcomes
Societal responsibility, sustainabil- Number of realized ecology measures (e.g., waste, carbon 51
ity, ecology, green dioxide, energy, water)
Quantity of carbon dioxide emitted per man month
3/Business process perfor-
mance
3.1/General process perfor-
mance
Process complexity Number of elementary operations to complete the task 40
General process information Nr of orders received or shipped per time unit 6, 27, 52
Nr of incoming calls per time unit
Nr of process instances
Order execution [Nr of executed orders]/[total nr of orders] * 100 7
Perceived sales performance Qualitative scale (e.g., Likert) on the successful promotion of 57
both efficiency and effectiveness of sales
Perceived management perfor- Qualitative scale (e.g., Likert) on the improvement of effective- 57
mance ness, efficiency, and quality of each objective and routine
tasks
Surplus inventory % of current assets 59
Value of surplus inventory (e.g., pharmaceutical material) to
total assets ratio
Occupancy rate Average % occupancy, e.g., of hospital beds 59
3.2/Time-related process
performance
Throughput Nr of processed requests/time unit 46
Process duration, efficiency [Σ(finish date − start date) of all finished business objects]/ 17
[number of all finished business objects]
Process cycle time, order cycle Time for handling a process instance end-to-end 5, 6, 11, 37, 40, 43,
time, process duration, average Aggregated time of all activities associated with a process (per 46, 60, 73
lifetime, completion time, process instance)
lead time [Application submission time] − [application response time]
Average sub-process turnaround [Sub-process start time] − [Sub-process finish time] 6, 37, 40, 52, 60
time, task time, activity time
Processing time Time that actual work is performed on a request 46
Average order execution time, [Σ(Dispatch time − creation time)]/[total number of orders] 7, 46, 60, 73
order fulfillment time, order lead [order entry time] + [order planning time] + [order sourcing,
time assembly and follow-up time] + [finished goods delivery
time]
Average order collection time [Σ(Collection time − creation time)]/[number of collected 7
orders]
Average order loading time [Σ(Final distribution time − distribution creation time)]/[num- 7
ber of loaded orders]
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 20 of 24

Table 8 continued
Perspectives Indicators/measures/metrics Operationalization Papers
Process waiting time, set-up time Average time lag between sub-processes, when a process 3, 5, 20, 37, 46, 52
instance is waiting for further processing
Time between the arrival of a request and the start of work on
it (=time spent on hold)
Average waiting time for all products and services
Manufacturing cycle efficiency [setup time + (nr of parts * operation time)]/[manufacturing 53
lead time]
Manufacturing lead time [setup time + (nr of parts * operation time) + queue 18, 53, 55
time + wait time + movement time]
Value added efficiency [Operation time]/[manufacturing lead time] 53
3.3/Cost-related process
performance
Activity cost Cost of carrying out an activity 46
Process cost, cost of quality, cost Sum of all activity costs associated with a process (per 5, 11, 16, 18, 20, 22,
of producing, customer order instance) 26, 27, 40, 43, 46
fulfilment cost
Unit cost Nr of employees (headcount) per application, product or 40
service
Information sharing cost [Time for system data entry] + [time for system delivery 40
output]
3.4/Process performance
related to internal quality
Quality of internal outputs, external % of instance documents processed free of error 5, 16, 18, 20, 22,
versus internal quality, error Number of mistakes 37, 40, 43, 46, 55,
prevention [Nr of tasks with errors]/[Total nr of tasks per process] 60, 66
Nr of syntactic errors
Nr of repeated problems
Presence of non-technical anomaly management (yes/no)
Deadline adherence, schedule % of Activity cycle times realized according to the planning 16, 17, 18, 26, 43
compliance, due date perfor- or schedule
mance effectiveness, responsive- [Number of finished business objects on time]/[number of all
ness finished business objects] * 100
Process yield Multiply the yield per process steps, e.g., (1 − scrap parts/total 43
parts)step 1 * (1 − scrap parts/total parts)step 2
Rework time, transaction efficiency Time to redo work for an incident that was solved partially or 30, 43, 57
totally incorrect the first time
Average time spent on solving problems occurring during
transactions
Integration capability Time to access and integrate information 40
3.5/Process performance
related to flexibility
Special requests Nr of special cases or requests 40
4/“Learning and growth”-
performance
4.1/(Digital) innovation
performance
Degree of digitalization % Reduction in processing time due to computerization 40, 46, 71
[Nr of process steps replaced by computer systems]/[Total nr
of steps in the entire process]
Nr of digital products or services
Degree of rationalization % of Procedures and processes systemized by documenta- 57
tion, computer software, etc.
Time for training on the procedure Measured in hours 40
Novelty in output Nr of new product or service items 57
Customer response Nr of suggestions provided by customers about products and 57
services
Third-party collaboration Nr of innovation projects conducted with external parties 59
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 21 of 24

Table 8 continued
Perspectives Indicators/measures/metrics Operationalization Papers
Innovation projects Nr of innovations proposed per quarter year 51
Nr of innovations implemented per quarter year
IS development efficiency Nr of change requests (+per type of change or per project) 6, 58, 66
Time spent to repair bugs and finetune new applications
Time required to develop a standard-sized new application
% of Application programming with re-used code
Relative IT/IS budget [Total IT/IS budget]/[Total revenue of the organization] * 100 58
Budget for buying IT/IS [Budget of IT/IS bought]/[Total budget of the organiza- 59
tion] * 100
Budget for IS training [IS training budget]/[overall IS budget] * 100 58
Budget for IS research [IS research budget]/[overall IS budget] * 100 58
Perceived management compe- Qualitative scale (e.g., Likert) on the improvement in project 57
tence management, organizational capability, and management
by objectives (MBO)
Perceived relationship between IT Qualitative scale (e.g., Likert) on the perceived relationship, 58
management and top manage- time spent in meetings between IT and top management,
ment and satisfaction of top management with the reporting
on how emerging technologies may be applicable to the
organization
4.2/Employee performance
Perceived employee satisfaction Qualitative scale on general satisfaction (e.g., Likert), possibly 16, 43, 11, 57, 58, 59
indexed as the weighted sum of judgements on satisfaction
dimensions
Qualitative scale (e.g., Likert) on satisfaction about hardware
and software provided by the organization
Average employee saturation, [Time spent daily on working activities]/[total working 3, 40, 46
resource utilization for process time] * 100
work [Work time]/[available time]
% of operational time that a resource is busy
Resource utilization for (digital IS expenses per employee 58
innovation) % of Resources devoted to IS development
% of Resources devoted to strategic projects
Process users Nr of employees involved in a process 37
Working time Actual time a business process instance is being executed by 20
a role
Workload Nr of products or services handled per employee 71
Staff turnover % of Employees discontinuing to work and replaced, com- 16, 57, 58
pared to the previous year
Employee retention, employee % of Employees continuing to work in the organization, com- 16, 57, 58, 59
stability pared to the previous year
Employee absenteeism [Total days of absence]/[total working days for all staff ] * 100 59
Motivation of employees Average number of overtime hours per employee 16
Professional training, promotion % of Employees trained 57, 59, 22
and personal development % of Employees participated in a training program per year
Nr of professional certifications or training programs per
employee
Professional conferences % of Employees participating in conferences 59
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 22 of 24

Appendix 4
See Table 9.

Table 9 Additional list of performance indicators without operationalization

Perspectives Performance indicators/measures/metrics Papers

1/Financial performance
Selling price 18, 55
Cash flow 22
2/Customer performance
2.1/Customer performance
Customer relationship management, direct customer cooperation, efficiency of 11, 22, 58
customer cooperation, establishing and maintaining relationships with the user
community
Warranty cost 55
Delivery cost 27
Delivery frequency 18, 60, 73
2.2/Supplier performance
Efficiency of cooperation with vendors, buyer–supplier partnership level, degree of 11, 60, 73
collaboration and mutual assistance, nr of supplier contracts
Information carrying costs, level and degree of information sharing 60
Supplier rejection rate 60
Buyer-vendor cost saving initiatives 60
Delivery frequency 60
Supplier ability to respond to quality problems 60
Supplier’s booking in procedures 60
Supplier lead time against industry norms 60
3/Business process performance
3.3/Cost-related process performance
Cost of risks 58
Cost per operating hour, running cost 18, 60
Material cost 22
Service cost 18, 22
Inventory cost (e.g., incoming stock level, work-in-progress, scrap value, finished 22, 55, 60
goods in transit)
Overhead cost 55
Obsolescence cost 55
Transportation cost 55
Maintenance cost 26
3.4/Process performance related to internal quality
Conformance to specifications 55
Compliance with regulation 18, 43, 55
Verification mismatches 73
Forecasting accuracy, accuracy of scheduling 55, 60, 73
3.5/Process performance related to flexibility
Process flexibility 22, 58
General flexibility 5, 22, 40
Product or service variety 55
Range of products or services 60
Modification of products or services, volume mix, resource mix 18, 22, 55
Flexibility of service systems to meet particular customer needs 60
Effectiveness of delivery invoice methods 60
Payment methods 52
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 23 of 24

Table 9 continued
Perspectives Performance indicators/measures/metrics Papers
Order entry methods 60
Responsiveness to urgent deliveries 60
4/“Learning and growth”-performance
4.1/(Digital) innovation performance
R&D performance, investment in R&D and innovations 11, 16
New product or service development costs 22
Knowledge base 16
4.2/Employee performance
Productivity 11, 22, 40
Labor efficiency 55
Labor cost 22
Employee availability 22, 26, 40, 52
Expertise with specific existing technologies 58
Expertise with specific emerging technologies 58
% of multi-skilled workforce 26
Age distribution of IS staff 58

Received: 17 June 2016 Accepted: 10 October 2016 Kitchenham B (2007) Guidelines for performing systematic literature reviews in
software engineering (version 2.3) (technical report EBSE-2007-01). Keele
University and University of Durham
Kueng P (2000) Process performance measurement system: a tool to support
process-based organizations. Total Qual Manag 11(1):67–85
March ST, Smith GF (1995) Design and natural science research on information
References technology. Decis Support Syst 15(4):251–266
Bain Company (2015) Management tools and trends 2015. [Link] McCormack K, Johnson WC (2001) Business process orientation. St. Lucie Press,
com/publications/articles/[Link]. Florida
Accessed Apr 2016 Melville N, Kraemer K, Gurbaxani V (2004) Review: information technology and
Boellt SK, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature organizational performance: an integrative model of IT business value.
reviews in IS. J Inf Technol 30:161–173 MIS Q 28(2):283–322
Bronzo M, de Resende PTV, de Oliveira MP, McCormack KP, de Sousa PR, Fer- Miller D, Friesen PH (1986) Porter’s (1980) Generic strategies and performance:
reira RL (2013) Improving performance aligning business analytics with an empirical examination with American data part I: testing porter. Organ
process orientation. Int J Inf Manag 33(2):300–307 Stud 7(1):37–55
Butler A, Letza SR, Neale B (1997) Linking the balanced scorecard to strategy. Neely A (2005) The evolution of performance measurement research. Int J
Long Range Plann 30(2):242–253 Oper Prod Manag 5(12):1264–1277
Cross KF, Lynch RL (1988) The “SMART” way to define and sustain success. Natl Neely A, Mills J, Platts K, Richards H, Gregory M, Bourne M, Kennerley M (2000)
Product Rev 8(1):1–23 Performance measurement system design: developing and testing a
Dumas M, La Rosa M, Mendling J, Reijers HA (2013) Fundamentals of business process-based approach. Int J Oper Prod Manag 20(10):1119–1145
process management. Springer, Berlin Norreklit H (2000) The balance on the balanced scorecard. A critical analysis of
EFQM (2010) EFQM—the official website. [Link] Accessed Apr some of its assumptions. Manag Accoun Res 11(1):65–88
2015 Peffers K, Rothenberger M, Tuunanen T, Vaezi R (2012) Design science research
Eisenhardt KM, Martin JA (2000) Dynamic capabilities: what are they? Strateg evaluation. In: Peffers K, Rothenberger M, Kuechler B (eds) DESRIST 2012.
Manag J 21(10–11):1105–1121 LNCS 7286. Springer, Berlin, pp 398–410
Hammer M (2007) The process audit. Harv Bus Rev 4:111–123 Porter ME (2008) The five competitive forces that shape strategy. Harv Bus Rev
Heckl D, Moormann J (2010) Process performance management. In: Rose- 86(1):78–93
mann M, vom Brocke J (eds) Handbook on business process manage- Recker J (2013) Scientific research in information systems. A beginner’s guide.
ment 2. Springer, Berlin, pp 115–135 Springer, Berlin
Hubbard G (2009) Measuring organizational performance: beyond the triple Richard PJ, Devinney TM, Yip GS, Johnson G (2009) Measuring organiza-
bottom line. Bus Strateg Environ 18(3):177–191 tional performance: towards methodological best practice. J Manag
Kaplan RS, Norton DP (1996) The balanced scorecard. Translating strategy into 35(3):718–804
action. Harvard Business School Press, Boston Ryan L (2014) ‘If you can’t measure it, you can’t manage it’: not true. http://
Kaplan RS, Norton DP (2001) The strategy-focused organization. How balanced [Link]/sites/lizryan/2014/02/10/if-you-cant-measure-it-you-
scorecard companies thrive in the new business environment. Harvard cant-manage-it-is-bs/#aca27e3faeda. Accessed Apr 2015
Business School Press, Boston Shah L, Etienne A, Siadat A, Vernadat F (2012) (Value, Risk)-Based performance
Kaplan RS, Norton DP (2004) Strategy maps. Converting intangible assets into evaluation of manufacturing processes. In: INCOM proceedings of the
tangible outcomes. Harvard Business Press, Massachusetts 14th symposium on information control problems in manufacturing,
King WR, He J (2005) Understanding the role and methods of meta-analysis in 23–25 May 2012. Bucharest, Romania, pp 1586–1591
IS research. Commun Assoc Inform Sys 16:665–686
Van Looy and Shafagatova SpringerPlus (2016)5:1797 Page 24 of 24

Smith TM, Reece JS (1999) The relationship of strategy, fit, productivity, and Vaivio J (1999) Exploring a non-financial management accounting change.
business performance in a services setting. J Oper Manag 17(2):145–161 Manag Acc Res 10(4):409–437
Sullivan T (2001) Scorecards ease businesses’ balance act. Infoworld, 8 Jan, p 32 Venkatraman N (1989) The concept of fit in strategy research: toward verbal
Treacy M, Wiersema F (1993) Customer intimacy and other value disciplines. and statistical correspondence. Acad Manag Rev 14(3):423–444
Harv Bus Rev 71(1):84–93
Ulfeder S (2004) The new imperative. Enterprise leadership. CIO advertising
supplements, 15 Feb, p S5