Introduction to Key Performance Indicators
Chapter 2
Introduction to
Key Performance
Indicators
As mentioned in the introduction, key performance indicators are a response to a general
organizational fear of big, ugly spreadsheets and complex applications. The big idea
behind KPIs is that you’re taking technical data and presenting it using business-relevant
language. Key performance indicators:
• Use rates, ratios, percentages and averages instead of raw numbers
• Leverage tachometers and thermometers and stoplights instead of pie charts and
bar graphs
• Provide temporal context and highlight change instead of presenting tables of data
• Drive business-critical action
The last point is the most important, that all good key performance indicators drive
action. I’ll say it again since it’s worth repeating: All good key performance indicators
drive action. This is the polite way of saying, “Any KPI that, when it changes suddenly
and unexpectedly does not inspire someone to send an email, pick up the phone or take a
quick walk to find help, is not a KPI worth reporting.”
What is a Key Performance Indicator?
Keeping the description above firmly in mind, let’s get to the nuts and bolts. Key
performance indicators are numbers designed to succinctly convey as much information
as possible. Good key performance indicators are well defined, well presented, create
expectations and drive actions.
Introduction to Key Performance Indicators 7
What is a Key Performance Indicator?
Definition
Key performance indicators are always rates, ratios, averages or percentages; they are
never raw numbers. Raw numbers are valuable to web analytics reporting to be sure, but
because they don’t provide context, are less powerful than key performance indicators.
Consider the following …
Say you take 10,000 orders on Monday. Great, right? Not if you took 100,000 orders on
the previous Monday. And not if you took those 10,000 orders from 1,000,000 people
you’d paid good money to bring to your site, especially when you took 100,000 orders
the previous Monday …
See what I mean? Without context 10,000 is just a number. Not good, not bad, but not
really informative. That’s why I insist to the chagrin of my respected peers that KPIs are
always rates, ratios, averages or percentages. It’s not to say that you should exclude raw
numbers from your KPI report—quite the opposite! Raw numbers are necessary to
provide context to these reports and to promote conversation. All I’m saying is that raw
numbers are not key performance indicators.
Key performance indicators are designed to summarize meaningfully compared data.
Prior to writing this book, many people spent a great deal of time discussing which data
were meaningfully compared. Now you can just read the definitions in this book and
save yourself the time.
Presentation
I’m tempted to say that presentation is the most important aspect of any key performance
indicator—how you choose to highlight changes over time, alert based on thresholds,
etc.—but that wouldn’t be right. Whether the KPI drives valuable action is the most
important aspect. Still, I’ve observed that companies that use colors, visual cues and
appropriate visual elements to present their KPIs usually see greater interest on the part of
the reader. Consider the following images:
Figure 1: A standard key performance indicator report showing values for the current and previous
reporting period
Introduction to Key Performance Indicators 8
What is a Key Performance Indicator?
Figure 2: A standard key performance indicator report showing values for the current and previous
reporting period plus a visual indicator of directional change, percent change, target value, percent
of goal and any relevant warnings to quickly call out problem metrics
Hopefully you’ll see that in the second example it is much easier to quickly identify the
problem areas on the site. Even without any warning messages, the use of downward
arrows and the color red (nature’s universal “oh shit!” color) draws the readers eye
towards the metrics that demand attention.
Consider the following presentation cues when you build your own key performance
indicator reports:
• Indicators always show comparison over time. You should never present a
single, static key performance indicator unless the people you’re presenting to
know the number like they know their age or phone number. Never assume that
people will remember these numbers from day-to-day or week-to-week. Show
them how they were doing, considering temporal comparisons like “this day last
week”, “yesterday”, “last week”, “this week last month”, etc. and combinations
thereof.
• Green is good, red is bad, yellow is getting bad. If you’re using Microsoft
Excel, use the conditional formatting option to color-code your indicators for easy
reading. Oh, and bold and red is really bad.
• Indicators trending up have up-arrows; indicators trending down have
down-arrows. Even if you’re color coding your numbers, providing simple
arrows to show whether the indicator is improving or declining over time gives
the reader additional context (for example, bold and red with a red down-arrow
indicating that the trend is getting worse over time is really, really bad.)
• Always show the percent change from reporting period to reporting period.
Because key performance indicators are designed to set expectations, you need to
let your reader know where they are regarding those expectations. Plus, if you’re
going to bother showing comparison over time, you might as well go the extra
mile and do the math. Remember: (this period minus last period) divided by last
period equals percent change from last period to this period.
Introduction to Key Performance Indicators 9
What is a Key Performance Indicator?
• Set thresholds and show warnings. While you’re color coding your indicators,
take the time to compare either the numbers or the percent change calculations to
a pre-set threshold and show a warning if that threshold is exceeded. For
example, if your order conversion rate drops by 5 percent, show a “MILD
CONCERN” warning, if it drops by 10 percent, show a “MEDIUM CONCERN”
warning, and if it drops by more than 20 percent show a “RUN SCREAMING!”
warning.
• Set targets for improvement and report against those targets. Since setting
expectation is critical to the use of key performance indicators, you may as well
report and measure against those expectations. That way you can show a warning
if you’re dangerously far from your target.
Sounds complicated, huh? That’s why I coded all of the key performance indicators
described in this book into a companion Excel spreadsheet, to save you the time having to
build spreadsheets (see an example in Figure 2). All you have to do is drop the necessary
data in, set your thresholds, add definitions that will be understood by your audience and
you’re off and running. Hopefully you’ll look like a genius.
No need to thank me.
Expectation
A big part of presentation is setting expectations and then communicating how close you
are to your set targets. Don’t simply track your indicators; challenge yourself and your
organization to improve upon them.
Put another way, you won’t get the full value out of your investment in key performance
indicators (and this book) until you use them as the reporting input into the continual
improvement process—measure, report, analyze, optimize—using them week-over-week,
month-over-month. The only reason you optimize the site is to drive improvement
(hence the name, continual improvement process.) Trying to do so in a vacuum is
wasteful. I strongly recommend that you set a target for improvement and diligently
work towards that goal.
Even if you don’t meet your targets and expectations, by setting them you force people to
keep KPIs under consideration. If you want to take it to the next level, consider setting
high (but reasonable) expectations for improvement in key performance indicators and
then paying bonuses each quarter based on successful attainment of those goals. The
promise of free money usually gets people intensely interested in the numbers.
Action
Key performance indicators should either drive action or provide a warm, comforting
feeling to the reader; they should never be met with a blank stare. Ask yourself “If this
number improves by 10 percent who should I congratulate?” and “If this number declines
Introduction to Key Performance Indicators 10
What is a Key Performance Indicator?
by 10 percent who should I scream at?” If you don’t have a good answer for both
questions, likely the metric is interesting but not a key performance indicator.
There is enough data in the world already. What most people need is data that helps them
make decisions. If you’re only providing raw data, you’re part of the problem. If you’re
providing clearly actionable data, you’re part of the solution. If you discover you’re
already doing the latter, give yourself a hug.
Most of the indicators outlined in this book are really good and useful metrics. I
specifically describe what action you might take based on the indicator in a variety of
contexts so that you’ll have those possible actions in mind. If you think of other actions
you might take based on a specific key performance indicator I would love to hear from
you. E-mail me at eric@[Link].
What is not a Key Performance Indicator?
Raw numbers are not key performance indicators. I know that many smart people
disagree with me on this point but, well, they’re wrong. I’m not saying that key
performance indicators and raw numbers cannot be used in the same context, presented
side-by-side even. In fact, in many instances it’s not a bad idea to present a few raw
numbers—data like number of visitors, visits and page views to the site, revenue, orders
taken, etc.—to further contextualize the reports. But don’t go overboard, if you put a
bunch of raw numbers in a spreadsheet you don’t have a KPI report, you have the exact
same spreadsheet that nobody understood and nobody used.
If you want to argue about whether raw numbers are key performance indicators, please
e-mail me at yourletterwillgostraightintothetrash@[Link].
How Should Key Performance Indicators Be
Presented?
Considering everything you’ve already read in this book about presentation you might be
surprised that there’s more! You know how to construct and present a KPI, now you
need to deliver it.
Format
There are a variety of ways you can deliver KPI reports throughout your organization:
email, spreadsheets, slides, documents, dashboards and the like. I tend to favor
spreadsheets like Microsoft Excel because they provide most of the functionality
necessary to achieve the presentation goals for key performance indicators described
above. Additionally, many web analytics application vendors provide direct data access
from Microsoft Excel that can dramatically simplify the report generation process.
Hopefully you’ll be able to automate data into the Excel spreadsheet provided with this
book to save yourself a bunch of time generating reports so you can dedicate time to
analyzing the metrics.
Introduction to Key Performance Indicators 11
How Should Key Performance Indicators Be Presented?
If you plan on using slides, you may be better off providing an annotated spreadsheet and
using slides to highlight indicators of note and drive the presentation. If you build a
spreadsheet and copy the table into a slide, you’re only making a hard-to-read slide. The
harder to read your slides are, the less likely your audience is to pay attention to the
message.
Dashboards are a format that nearly all analytics vendors recommend for your KPI
reporting but the use of dashboards in most cases assumes that your audience is going to
log into the analytics application, an assumption that is often false. Also, dashboards
often don’t provide enough flexibility in terms of which metrics and indicators can be
presented. While dashboards do usually allow for interesting visualizations
(thermometers, tachometers and so on), these visualizations often do not convey enough
information and thusly become impediments to the actual use of the data.
Timeliness of Delivery
If you take the necessary time to build a key performance indicator report but either only
distribute the report once a quarter or worse, don’t distribute the report at all, you’re
wasting your time. Every organization is different but KPIs are only effective if people
see them frequently enough to actually keep them in mind when making business
decisions. In general, I recommend that retailers deliver their KPI reports on a daily basis
and all other business models deliver reports on a weekly basis.
Even if you’re unable to meet every day or every week to discuss the ramifications of the
report, make sure your indicators are being generated, annotated and delivered. Doing so
will keep the recipients up to date and hopefully make any conversation about the metric
more productive. Fight the temptation to only send out reports just prior to any meeting
on the subject of KPIs; this practice is the same as hoping that people will log into the
analytics application frequently enough to maintain any sense of relationship with the
data. It’s great in theory but usually fails to produce the desired results.
Annotation
As I’ve alluded to several times, annotating your KPI reports is perhaps one of the most
important things your web data analysis staff can do to promote the proper use of these
metrics. While KPIs are designed to promote action, providing relevant notes alongside
indicators that are in decline often helps promote the “right” action. If nothing else,
adding a note to any metric exceeding set thresholds stating that “the web data team is
already exploring the problem and hopes to have a recommendation very soon” will cut
down on unnecessary phone calls and meetings (Figure 3).
Introduction to Key Performance Indicators 12
How Should Key Performance Indicators Be Presented?
Figure 3: Annotation included as a top-line summary of indicators that are changing or under
investigation.
Who Gets What?
The topic of who in the organization should get which KPI reports is important enough to
explore in depth in Chapter 4
Key Performance Indicators by Business Type, addressing which indicators are
appropriate to each job type for each business model. In general, I strongly recommend
that you adopt a hierarchical model when deciding which indicators should be sent to
which employees; the alternative, sending every KPI to every internal stakeholder, only
creates more unnecessary work for everyone. The general model I advocate is as
follows:
• Senior strategists: Senior stakeholders should get two to five KPIs depending on
the breadth of their direct responsibility in the organization. An example would
be the CEO of a retail web site who should see order conversion rate, average cost
per conversion and average revenue per visitor along with whatever
measurements he or she needed to do her job.
• Mid-tier strategists: Junior strategic stakeholders should get five to seven KPIs
that include the KPIs senior stakeholders receive plus strategic indicators relevant
to their particular department or line of business. An example would be the Vice
President of Marketing who would get the same indicators as the CEO plus top-
line KPIs reporting conversion rate for each campaign type currently deployed.
• Tactical resources: Tactical stakeholders get seven to ten KPIs including the
same indicators their managers get plus detailed KPIs reporting on individual
campaigns, promotions or pages. An example would be the Director of Online
Marketing who would get the same indicators as the Vice President of Marketing
plus KPIs describing conversion rates for top active campaigns.
Introduction to Key Performance Indicators 13
How Should Key Performance Indicators Be Presented?
Below the level of tactical stakeholder I usually advise that people get comfortable using
the actual analytics application. While they should get the appropriate KPI report,
tactical managers are usually responsible for all campaigns, products or pages and should
thusly be familiar with how to gather necessary data from the application directly. The
primary reason I make this recommendation is that any action a KPI instigates generally
falls downstream to the appropriate tactical resource for diagnosis and correction.
Seriously, Don’t Send Everyone 50 Key
Performance Indicators!
Despite guidance about “Who Gets What?” and my attempt to break down indicator
usage by organizational role and business model in Chapter 4
Key Performance Indicators by Business Type of this book, experience tells me that it is
worthwhile to emphasize the following:
NO KEY PERFORMANCE INDICATOR REPORT SHOULD HAVE MORE
THAN A HANDFUL OF METRICS, TWO HANDSFUL AT MOST!
Sorry for yelling but it seems that no matter how many times I bring this up, someone
always asks me to review a KPI report with 30 different metrics on it. When I ask the
response is almost always, “Well, someone in accounting needs that metric” or “We
always tracked that number so everyone is used to seeing it.”
Auuugh!
Key performance indicators exist because there is already too much data available to any
business of any size. Given this, how can providing long spreadsheets of irrelevant data
possibly provide any value? Simply, it cannot. You need to follow these three easy
guidelines when determining which KPIs you should distribute throughout your
organization:
1. Be hierarchical. Follow my recommendation for hierarchical delivery of
indicators, making sure that people are only tasked with understanding indicators
that directly impact the performance of their group, division, department or line of
business.
2. Be focused. Per recommendation #1, always fight the temptation to simplify the
process into a single spreadsheet. Trust me on this one—more relevant data
garners more attention than generic data.
3. Be open to suggestions. If there is any doubt about the relevance and utility of a
metric, ask the potential recipient what action they would take if the indicator
increased or decreased by ten percent. If they don’t have a pretty good answer,
don’t send them the indicator.
You’re welcome to ignore this sage advice but please don’t come whining to me six
months later when nobody is paying any attention to your beautiful KPI reports because
Introduction to Key Performance Indicators 14
Seriously, Don’t Send Everyone 50 Key Performance Indicators!
they’re really long and only appear to have a little data that is actually relevant to their
job. If you do, expect to hear me say “I told you so.”
How Should Key Performance Indicators Be
Used?
The best use for a key performance indicator can best be explained using two examples:
1. Example #1: Senior executive responsible for the web site arrives at work
Monday morning, opens an email containing his key performance indicator report
comparing the previous week to the week prior and the same week last month.
She examines the metrics, noting that all of the critical KPIs are improving and
that and problems being reported are all known issues. She closes the report and
goes on with her busy day.
2. Example #2: Senior executive responsible for the web site arrives at work
Monday morning, opens an email containing his key performance indicator report
comparing the previous week to the week prior and the same week last month.
He sees red everywhere. His conversion rate has tanked, his revenue per visitor is
down 23%. He notes that it looks like most of the problems are associated with
recently launched marketing campaigns so he picks up the phone and calls his
direct reports in for a “nice chat” about their jobs.
In both cases, a manager was able to make a quick decision about how the day or week
was going to go regarding the web site as a business channel. The first executive went on
with her day, knowing that her downstream people had their own reports and that she
would hear from them if necessary. The second needed to be more proactive, calling for
an immediate meeting to discuss how the problem would be researched and resolved.
Assuming his team is seeing the same reports those folks should not be at all surprised to
get the call and will hopefully already be working on a response.
Key performance indicators are tools designed to simplify people’s relationship with web
data and guide action. Because non-data analysts will only be getting the information
they need to do their jobs in a format that they’re comfortable with, they can more
quickly assess performance and respond appropriately. This is another way of saying that
if you keep sending them huge, complex reports, eventually most people will stop paying
attention.
KPIs also help improve data sharing throughout the organization and in meetings.
Because everyone should have access to the same set of reports, people won’t come to
meetings with the “wrong” data (wrong being a function of the metrics used, the
timeframes examined or the calculations made.) If people are using the same reports
week over week they become more ingrained, ideally becoming common knowledge.
That way everyone is thinking about the same problem, working together towards a
solution.
Introduction to Key Performance Indicators 15
How Should People Respond to Key Performance Indicators?
How Should People Respond to Key Performance
Indicators?
The only reason you should use key performance indicators for reporting web analytic
data is because the organization is motivated to optimize the online channel. If the
company is not motivated to use the available data, repackaging it into key performance
indicators and forcing it on people isn’t going to help. Trust me on this one.
In the course of my research I recently uncovered something surprisingly simple about
how companies need to think about an investment in web analytics: the idea that it takes
more than just an investment in measurement technology to be successful. If you’re not
willing to invest in technology and people to use that technology you’re doomed from the
get-go. But if you’re not willing to establish process around that investment you’re still
more likely than not to fail. The use of key performance indicators is an excellent way to
establish process. Still, without organizational interest, all you have are reports and
reports by themselves are unlikely to have any profound impact on your online business.
So given this framework, how should people respond to the key performance indicators
they receive? It’s hard to say. I know that whenever one of the KPIs I use to measure
my online business (selling books, right?) declines I immediately try to figure out what
went wrong and why. Depending on which KPI is in decline I focus more or less quickly
on the problem. Revenue KPIs I address immediately, other KPIs I address more slowly
but still I make sure to address them. Why? Because I’m really interested in making my
online business as successful as possible; the KPIs are about my business objectives and
their successful attainment.
That’s the most critical thing to keep in mind: your key performance indicators are about
your business success. If your personal success is tied to your business success, as it is
for so many employees, all the more reason to pay special attention to your business
KPIs. Perhaps all you need to do is occasionally remind people that these indicators are
not just numbers, they’re numbers that describe how successful the organization truly is.
Hopefully that will create the level of interest your organization needs to work diligently
to optimize the online channel.
About Business Specific Key Performance
Indicators
One of the comments I got from Bob Page, a really smart guy who knows his metrics,
was that my really big list of key performance indicators didn’t include any business
specific KPIs. Metrics like “percent completed streams” for online media properties or
“percent successfully executed trades” for online brokerage houses. Bob makes an
excellent point.
Introduction to Key Performance Indicators 16
About Business Specific Key Performance Indicators
Please don’t treat the calculations described in this book as the end-all-be-all list of
metrics for your specific business. Treat them as guides to help you get started with
using key performance indicators and to help you understand how your own business-
specific KPIs should be defined and reported. If you build out an appropriate list of key
performance indicators and people comment, “Yeah, that list looks good but it’s missing
the number I need to do my job” then you should ask them what that number is, where it
comes from and how you can include it in their specific KPI report. As long as it is
widely understood within the organization and actionable it is probably a pretty good key
performance indicator.
Oh, and I’d absolutely love to hear about your business specific KPIs as I plan on
updating this book as frequently as is reasonable and will gladly include the metrics that
work for you. Please email me directly at eric@[Link] and let me
know what your business specific KPI is, how you define it and how it helps you run
your online business.
Introduction to Key Performance Indicators 17
