Benchmarking Follies
By: Alfred L. Angelici, MSIM
May 21, 2009
While the benefits of Benchmarking have been well documented, little has been shared publicly on the
“dark side” of Benchmarking.
In the process of running a consulting business there is a natural conflict designed into the system. The
consultant’s discord between acquiring work and generating a business income versus the realization that a
buyer’s company may not able or willing to substantially gain from the results of a Benchmarking study.
While this may not be crystal clear at the outset of a new client relationship, it is this consultant’s opinion
that once identified one must clearly and tactfully convey such news to the Buyer poste-haste; regardless
of the phase of any ongoing engagement. Failure to do so will surely lead to long-term, irrevocable harm
in the consultant / buyer relationship. After all, the consultant’s greatest contribution to a buyer is her/his
personal integrity and objective view no matter how potentially unwelcome the message.
When is Benchmarking not appropriate for a client?
There are three fundamental circumstances when Benchmarking is likely to be of little or no value to a
buyer and her/his firm.
#1. The buyer’s organization is not ready for Benchmarking
¾ Operationally, there is a lack of standard, defined
business processes
¾ The company lacks a coherent business vision or
strategy plans to benchmark against
¾ Management commitment and follow-through for
change is poor or non-existent
Summary #1: A firm that is “lost”, tripping over its own feet, or too hung up on internal infighting and
power struggles is very likely not ready to make good use of Benchmarking. The firm
would likely do much better to focus on healing itself before looking at its competition for
information it cannot make credible use of.
1
Example: Early in my career, I was employed with a company that that was extremely successful;
despite itself. A young CEO had taken over from the founder and he was a gifted visionary.
Over time he replaced the founder’s senior team and surrounded himself with a team that was
just like him: relatively young, inexperienced, and visionary. Within a year, there was almost
no experienced “execution” talent remaining on his team. Suddenly everyone wanted to be a
“visionary”, and this culture permeated itself down the organization hierarchy.
Near the zenith of this company’s existence the firm was so top heavy that those with the
title of “Director” and above numbered themselves for every three (3) staff-level positions.
During this company’s heyday, the firm employed many consultants to produce
Benchmarking studies and most delivered valuable competitive information. Regardless,
the organization could not effectively deliver and unlock the value of these study results
under the weight of an inverted staffing structure and a deficiency of solid execution skills.
Eventually the company’s cash cow revenue source gave out before its promising new
replacement could support the organizational mountain tittering above it.
The CEO had a major blind spot. He failed to recognize that he needed to surround himself
with a top-notch senior management team that brought a diverse set of proven skills to
complement his own exceptional visionary talent. Instead, there was little diversity of
thought and skill. Without a fundamental change in the senior management team
composition, the company was not capable of improving from any informational gems it
obtained through Benchmarking.
Conclusion: Management did not need consultants for Benchmarking. It needed a consultant
that could have recognized these core deficiencies and proposed ways to assist in shifting the
organization to a position where it could successfully execute and manage itself within a
realistic financial framework. At the time I was not aware of any consulting group that ever
presented these issues and opportunities to the executive team. Either these consultants did
not see the problem or ignored it and conducted their benchmarking studies and got paid.
#2. Buyer / Management only seeks results that will support their
preconceived conclusions and decisions
¾ If Benchmarking results do not support a personal agenda or
the institutional imperative, it is repudiated and summarily
ignored regardless of the true potential benefits exposed.
Summary #2: I cannot tell you how many times I have witnessed this. Sometimes it can exist within one
key senior manager, and other times it is ingrained across the entire senior management team.
In either event opposing evidence and information is typically a “show-stopper”.
2
Example: I was working with an overseas firm. My assignment was to create a multi-year operational
support systems (OSS) plan that would closely align the company’s future systems investment
with their long-term strategic business plans. During the engagement, the CIO commented that
his budget funding was very tight and that his historical attempts to increase funding were usually
unsuccessful. I told him that as part of my assignment I would conduct benchmark study to
compare his organization’s OSS spending levels with that of some of his industry competitors.
The results of this benchmarking study revealed with reasonable clarity that the company was
more than likely under investing in the OSS area; especially given the firm’s accelerated growth.
Initially the Benchmarking study was well received, until it reached the hands of a key senior
executive who had recently returned from an extended business trip. After reviewing the
benchmark data, he and I met separately and he explained to me that he did not believe the
Benchmark study results and my assertion that additional funding was required. He then shared
his own opposing view. His rationale was based on knowledge he possessed of a comparable size
firm that was spending significantly less on OSS development than his own and achieving
superior results and return on investment. He felt his company could and should do the same.
Despite this executive’s argument, the facts simply did not support his viewpoint. Firstly, his
firm’s system group was not world-class. In fact, they were no better or worse than their
competitor’s. Secondly, there was no use or movement towards leading edge, accelerated
development and measurement techniques (e.g.: Agile with Scrum) to significantly improve time
to market and to lever up the business contribution return. Thirdly, the vast majority of the
annual systems budget was pegged to “maintenance” (historical) work, not to “build” or
“transform” the business work (in the future).
Additionally, this company was well into a program of outsourcing the majority of their
system development and support to large offshore technology firms. The outsource firm was
well-known as a body-shop that was typically populated with mediocre talent with limited
experiential knowledge.
Despite the total picture and ongoing performance evidence, this same influential executive fully
expected to continue to cost cut and outsource his way to world-class performance.
In the meantime, his company continued to suffer from significant business process
bottlenecks, scalability limitations, unnecessary human resource growth, & management
information gaps.
3
#3. Benchmarking results are not viewed from a proper perspective
¾ Perhaps Warren Buffet summed it up best, saying "It is better to be
approximately right than precisely wrong."
¾ And Misters Graham, Dodd also note: “It is extremely difficult to
tell if a stock is fairly priced, but it is usually easy to tell when a
stock is significantly under or over-priced.” Therefore, one should
ALWAYS require a “margin of safety” in making investment
decisions.
Summary #3: Benchmarking results do not represent perfect information. However, if used as a
“guidepost” with a healthy dose of realism and additional corroboration, they can provide
invaluable competitiveness insight and a possible call to action.
Example: Expanding a little more on the prior story, the system spending Benchmarking study showed
that all the competitors were investing more than my client was. Several of these competitors
were experiencing low, single digit or slightly negative annual revenue growth, and one was
achieving nearly double-digit revenue growth and spending nearly twice as much as my client.
Meanwhile, my client’s revenue growth rate had been in the double-digit growth range annually
for the past four years. This accelerated growth coupled with low systems funding had led
many standard business tasks to become partially or fully supported by “Spreadmarts”1 and
other standalone solutions outside of the core enterprise systems.
When I presented the benchmarking data to the senior management team, I was asked “What is
the ideal spending level?” I told the senior team that Benchmarking is not an exact science and
will not provide an exact answer. However, it seemed quite safe to conclude from the study and
local evidence that the company had been under investing in its core systems.
I recommended that they consider modestly increasing their OSS funding level for the next year
and be sure that 100% of the increase was targeted to strategic business projects. Next, they
needed to inventory and review all legacy systems to determine which were supporting core or
critical business functions. Those that failed to measure up needed an end-of-life plan and
timeline. The company could reinvest those maintenance costs into business growth areas.
Over time they could measure results and progress to the “right” investment level.
1
Spreadmarts are data shadow systems in which individuals collect and massage data on an ongoing basis to
support their information requirements or for their immediate workgroup. These shadow systems ‐‐ usually built
on spreadsheets ‐‐ exist outside of approved, IT‐managed corporate data repositories, such as data warehouses,
data marts, or ERP systems, and contain data and logic that often conflict with corporate data.
[Link]
4
Wrap up:
For the Buyer
1) When not to use Benchmarking?
a) If the corporate house is a mess. Clean it up, first.
b) If the senior management team is looking for one perfect answer. Benchmarking will
not provide it. (Remember Buffett’s quote: “It is best to be approximately right than
precisely wrong!”)
2) When NEVER to use Benchmarking?
c) When it is only considered valuable if and when the information obtained will support
management’s preconceived beliefs, conclusions, and decisions. Save money, and just
make the decision!
For the Consultant
A) Know the Buyer’s organizational environment early!
During the initial contact meetings and engagement / proposal scoping phase, be sure to
ask appropriate questions about the organization, its people and their past successes with
using external consultants and implementing their recommendations.
- Have they had a positive working / implementation track record?
- Have they achieved expected value (ROI) from their past engagements?
- If not, why not?
i. Has one or more powerful / critical organizational figures derailed the efforts?
ii. Lack of solid execution skills? (Weak project managers or project staff?)
iii. Lack of senior management commitment?
o Underfunded implementation plans?
o Unrealistic delivery timelines?
o Constantly changing strategies, priorities, and goals?
o Frequent reorganizations and movement of senior staff?
Conducting this simple step at the outset of the engagement process, you will be doing yourself and
your prospective Buyer a service by:
♦ Better knowing the organizational environment you may be entering into,
♦ Identifying potentially dysfunctional organizational behavior and single points of
failure in advance
At the very least, this knowledge may prevent a consultant from ruining their good name, and at
most it may enable a consultant to offer a better, more valuable proposal to the buyer. Assisting the
buyer in this way will enable the consultant to overcome the buyer’s internal hurdles, create greater
trust with the buyer, and develop invaluable goodwill for future work opportunities.
5