THE ROLE OF BIG DATA

Why Big Data is the new competitive advantage

Data are now woven into every sector and function in the global economy, and, like other essential
factors of production such as hard assets and human capital, much of modern economic activity
simply could not take place without them. The use of Big Data large pools of data that can be
brought together and analyzed to discern patterns and make better decisions will become the
basis of competition and growth for individual firms, enhancing productivity and creating
significant value for the world economy by reducing waste and increasing the quality of products
and services.

Until now, the torrent of data flooding our world has been a phenomenon. But we are now at an
inflection point. According to research from the McKinsey Global Institute (MGI) and McKinsey
& Companys Business Technology Office, the sheer volume of data generated, stored, and mined
for insights has become economically relevant to businesses, government, and consumers.

The history of previous trends in IT investment and innovation and its impact on competitiveness
and productivity strongly suggest that Big Data can have a similar power, namely the ability to
transform our lives. The same preconditions that allowed previous waves of IT-enabled innovation
to power productivity, i.e., technology innovations followed by the adoption of complementary
management innovations, are in place for Big Data, and we expect suppliers of Big Data
technology and advanced analytic capabilities to have at least as much ongoing impact on
productivity as suppliers of other kinds of technology.

All companies need to take Big Data and its potential to create value seriously if they want to
compete. For example, some retailers embracing big data see the potential to increase their
operating margins by 60%.1

1
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
 Big Data: A new competitive advantage

The use of Big Data is becoming a crucial way for leading companies to outperform their peers. In
most industries, established competitors and new entrants alike will leverage data-driven strategies
to innovate, compete, and capture value. Indeed, we found early examples of such use of data in
every sector we examined. In healthcare, data pioneers are analyzing the health outcomes of
pharmaceuticals when they were widely prescribed, and discovering benefits and risks that were
not evident during necessarily more limited clinical trials. Other early adopters of Big Data are
using data from sensors embedded in products from childrens toys to industrial goods to determine
how these products are actually used in the real world. Such knowledge then informs the creation
of new service offerings and the design of future products

In addition to the sheer scale of Big Data, the real-time and high-frequency nature of the data are
also important. For example, nowcasting, the ability to estimate metrics such as consumer
confidence, immediately, something which previously could only be done retrospectively, is
becoming more extensively used, adding considerable power to prediction. Similarly, the high
frequency of data allows users to test theories in near real-time and to a level never before possible.

We studied five domains in depthhealthcare and retail in the United States, the public sector in
Europe, and manufacturing and personal location data (the location data generated by the smart
mobile devices many of us now carry) globallyand some broadly applicable ways of leveraging
big data emerged.2

Big Data will help to create new growth opportunities and entirely new categories of companies,
such as those that aggregate and analyse industry data. Many of these will be companies that sit in
the middle of large information flows where data about products and services, buyers and suppliers,
consumer preferences and intent can be captured and analysed. Forward-thinking leaders across
sectors should begin aggressively to build their organisations Big Data capabilities.

Big data analytics

2
[Link]
Theres no single technology that encompasses big data analytics. Of course, theres advanced
analytics that can be applied to big data, but in reality several types of technology work together to
help you get the most value from your information. Here are the biggest players:

Data management. With data constantly flowing in and out of an organization, it's important
to establish repeatable processes to build and maintain standards for data quality. Once data
is reliable, organizations should establish a master data management program that gets the
entire enterprise on the same page.
Data mining. Data mining technology helps you examine large amounts of data to discover
patterns in the data and this information can be used for further analysis to help answer
complex business questions.
Hadoop. This open source software framework can store large amounts of data and run
applications on clusters of commodity hardware. It has become a key technology to doing
business due to the constant increase of data volumes and varieties, and its distributed
computing model processes big data fast. An additional benefit is that Hadoop's open source
framework is free and uses commodity hardware to store large quantities of data.
In-memory analytics. By analyzing data from system memory (instead of from your hard
disk drive), you can derive immediate insights from your data and act on them quickly. This
technology is able to remove data prep and analytical processing latencies to test new
scenarios and create models.
Predictive analytics. Predictive analytics technology uses data, statistical algorithms and
machine-learning techniques to identify the likelihood of future outcomes based on
historical data.3
Some of the most common applications of predictive analytics include fraud detection, risk,
operations and marketing, which helps organizations feel more confident while making new
decisions.
Text mining. With text mining technology, you can analyze text data from the web,
comment fields, books and other text-based sources to uncover insights you hadn't noticed
before. Text mining uses machine learning or natural language processing technology to
comb through documents emails, blogs, Twitter feeds, surveys, competitive intelligence

3
[Link]
 and more to help you analyze large amounts of information and discover new topics and
term relationships.4

THE ROLE OF BIG DATA IN MEDICINE

Technology is revolutionizing our treatment of disease. Most companies make a conscious and deliberate
decision to embrace digitization and the information revolution. Yet the role of big data in medicine
seems almost to compel organizations to become involved.

The role of big data in medicine is one where we can build better health profiles and better predictive
models around individual patients so that we can better diagnose and treat disease.

One of the main limitations with medicine today and in the pharmaceutical industry is our understanding
of the biology of disease. Big data comes into play around aggregating more and more information
around multiple scales for what constitutes a diseasefrom the DNA, proteins, and metabolites to cells,
tissues, organs, organisms, and ecosystems.

There is great maturity in the information sciences beyond medicine.

The life sciences are not the first to encounter big data. Information-power companies like Google and
Amazon and Facebook, and a lot of the algorithms that are applied thereto predict what kind of movie
you like to watch or what kind of foods you like to buyuse the same machine-learning techniques.
Those same types of methods, the infrastructure for managing the data, can all be applied in medicine.

How wearables are poised to transform medicine

Wearable devices and engagement through mobile health apps represent the futurenot just of the
research of diseases, but of medicine. I can be confident in saying that, because today in medicine, a
normal individual who is generally healthy spends maybe ten minutes in front of a physician every year.
What that physician can possibly score you on to assess the state of your health is very minimal. Unless
something catastrophic is going on within youlipid levels that are way off the charts or glucose levels
or something extremetheyre not doing much to assess what your state of well-being is, and the
information stored in medical records is not extensive enough.

4
[Link]
What the wearable-device revolution provides is a way to longitudinally monitor your statewith
respect to many different dimensions of your healthto provide a much better, much more accurate
profile of who you are, what your baseline is, and how deviations from that baseline may predict a
disease state or sliding into a disease state.5

That means well be able to intervene sooner to prevent you from that kind of slide. That sort of modeling
would be impossible unless you could phenotype individuals on a longitudinal and long-term basis. And
while the wearable devices today are in this more recreational-grade state, theyre changing incredibly
rapidly into research grade and ultimately clinical grade. There are already glucose monitors that are
FDA approved that individuals can wear and that interface with digital apps, which then connect directly
with healthcare providers based on what theyre seeing with your glucose profiles. Youll see that kind
of sensoring get better and better, providing higher and higher grades and better and better profiles on
individuals over time. I estimate that in five to ten years, accurate information about your health will
exist more outside the health system than inside the health system. And that will force the engagement
of that information by the medical community.

What big data means for patients, payers, and pharma

What I see for the future for patients is engaging them as a partner in this new mode of understanding
their health and wellness better and understanding how to make better decisions around those elements.
Most of their data collection will be passive, so individuals wont have to be active every daylogging
things, for examplebut theyll stay engaged because theyll get a benefit from it. Theyll agree to have
their data used in this way because they get some perceived benefit. Ultimately, thatll be the number of
doctor visits you require, the number of times you were sick, the number of times you progressed into a
given disease state. All should diminish. And theres a benefit from being presented with the
information, so theyre looking at dashboards about themselvestheyre not blind to the information or
dependent on a physician to interpret it for them, theyre able to see it every day and understand what it
means.

I believe payers are perhaps among the top of the chain as far as who can benefit from this. Because,
ultimately, payers want to constrain the cost of each patient. They care about the health of the patient,

5
Schadt, E., (2015), The role of Big Data in medicine, Interview.
[Link]
medicine
but they want to do whatever they can to motivate both the patients and the medical systems that treat
them to minimize the cost through better preventative measures, better targeted therapies, and increased
compliance for medication usage. 6

So now, payers are getting a better benefit from drugs being taken, because theyre able to see that the
drug is being taken as prescribed or that its not having the effect on the patient so the patient can be
switched earlier to a more effective treatment. If youre able to intervene sooner in the course of a
patients health, before they slide into a disease state, then youre going to save money on those
unexpected hospitalizations or emergency-room visits or even physician visits.

Then theres just the general risk profiling of patients. Of course, payers care a lot about understanding
the overall risk of a patient and what theyre likely to cost year over year. For example, say were able
to generate genomic information that tells us what the heritable cancer risk of every patient is; you dont
need to wait until a lump is felt or the persons at a later stage of cancer, when its much more expensive.
Those better risk profiles will be an incentive for payers to pay attention and to actually be involved in
that development.

For device makers, I just see this as a revolution thats theirs to lose if they dont embrace the
development of consumer wearable devices or sensors, more generally, in environments where every
person in the US or on the planet is buying a device versus one of a handful of medical systems. Thats
a better business model thats going to generate lots of revenue. And so its up to the device maker to
embrace that revolution and even start transforming some of the devices theyre already making into
consumer-grade devices that can be not just recreation grade but higher grade, on toward the clinical
grade.

Finally, from the pharmaceutical standpoint, I think its major. I mean, just look at Regeneron
Pharmaceuticals and Geisinger engaging the Geisinger Health System and sequencing everybody in that
population to create a better understanding of disease and protections against disease to do therapeutics.
What youre seeing, at some level, is some embracing of this sort of information revolution by the
pharmaceutical companies. Its doing it mainly from the genomics arena, but its also approaching it

6
Schadt, E., (2015), The role of Big Data in medicine, Interview.
[Link]
medicine
from the standpoint of better understanding disease, having a better understanding of the causal players
of disease, and using that or the causal protectants against disease to directly develop therapeutics.7

Five ways to leverage Big Data

1. Big Data can unlock significant value by making information transparent. There is still a
significant amount of information that is not yet captured in digital form, e.g., data that are on
paper, or not made easily accessible and searchable through networks. We found that up to 25
percent of the effort in some knowledge worker workgroups consists of searching for data and then
transferring them to another (sometimes virtual) location. This effort represents a significant source
of inefficiency.

2. As organisations create and store more transactional data in digital form, they can collect more
accurate and detailed performance information on everything from product inventories to sick days
and therefore expose variability and boost performance. In fact, some leading companies are using
their ability to collect and analyse big data to conduct controlled experiments to make better
management decisions.

3. Big Data allows ever-narrower segmentation of customers and therefore much more precisely
tailored products or services.

4. Sophisticated analytics can substantially improve decision-making, minimise risks, and unearth
valuable insights that would otherwise remain hidden.

5. Big Data can be used to develop the next generation of products and services. For instance,
manufacturers are using data obtained from sensors embedded in products to create innovative
after-sales service offerings such as proactive maintenance to avoid failures in new products.8

Value created by the use of Big Data

7
Schadt, E., (2015), The role of Big Data in medicine, Interview.
[Link]
medicine
8
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
If the U.S. healthcare system were to use big data creatively and effectively to drive efficiency and
quality, the sector could create more than $300bn in value every year. Two-thirds of that would be
an 8 percent reduction in U.S. healthcare expenditure. In the developed economies of Europe,
government administrators could create more than 100bn ($123bn) in operational efficiency
improvements alone by using Big Data and thats not including employing advanced analytic
tools to reduce fraud and errors and boost the collection of tax revenues.

But its not just companies and organisations that stand to gain from the value that Big Data can
create. Consumers can also reap highly significant benefits. For instance, users of services enabled
by personal-location data can capture $600bn in consumer surplus.

Take smart routing using real-time traffic information, which is one of the most heavily-used
applications of personal-location data. As the penetration of smart phones increases, and free
navigation applications are included in these devices, the use of smart routing is likely to grow. By
2020, more than 70 percent of mobile phones are expected to have a GPS capability, up from 20
percent in 2010. All told, we estimate that the potential global value of smart routing in the form
of time and fuel savings will be about $500bn by 2020. This is equivalent to saving drivers 20bn
hours on the road, or 10 to 15 hours every year for each traveller, and about $150bn on fuel
consumption.

Some of the most significant potential to generate value from Big Data will come from combining
separate pools of data. The U.S. healthcare system, for instance, has four major pools of data
clinical; activity (claims) and cost; pharmaceutical and medical products R&D; and data about
patient behaviour and sentiment each of which is primarily captured and managed by a different
constituency. MGI estimates that if U.S. healthcare fully used all the available techniques that can
be enabled by Big Data, such as analyzing records of real-world medical treatments, their costs and
health outcomes to guide physicians on which treatments provide the best outcomes at the best
cost, the annual productivity of the sector could grow by an additional 0.7 per cent. But achieving
this boost in productivity will require the combination of data from different sources often from
organizations that have no history of sharing data at scale. Sets of data such as patient records and
clinical claims would have to be integrated.9

Doing so would create benefits not just for the various industry players but for patients, who would
have broader, clearer access to a wider variety of healthcare information, making them more
informed. Patients would be able to compare not only the prices of drugs, treatments, and
physicians, but also their relative effectiveness, enabling them to choose more effective, better-
targeted medicines, potentially even customised to their personal genetic and molecular make-up.

To obtain those broad benefits, healthcare consumers may have to accept a slightly different trade-
off between their privacy and the benefits that wider pooling of data would bring. Sensitivities
around privacy and data security are just one hurdle that companies and governments need to
overcome if the economic benefits of big data are to be realised. One of the most pressing
challenges is a significant shortage of people with the skills to analyse big data. By 2018, the United
States alone could face a shortage of 140,000 to 190,000 people with deep analytical training (in
statistics or machine learning) and another 1.5m people with the managerial and quantitative skills
to be able to frame and interpret analyses effectively enough to base decisions on them.

There are also many technological issues that need to be resolved to make the most of big data.
Legacy systems and incompatible standards and formats often prevent the integration of data and
the application of the more sophisticated analytics that create value. Ultimately, making use of
large digital datasets will require the assembly of a technology stack from storage and computing
through analytical and visualisation software applications.

Above all, access to data needs to broaden. Increasingly, companies will need to access data from
third parties, e.g., business partners or customers, and integrate them with their own. A vital
competency for data-driven organizations in the future will be the ability to create compelling value
propositions for others, including consumers, suppliers and potentially even competitors, to share
data. If it looks unlikely that data sharing will occur despite the potential for societal benefits (a
market failure), legislators may then have to step in.

9
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
As long as companies and governments understand the power of Big Data to deliver higher
productivity, better value for consumers, and the next wave of growth in the global economy, there
should be a strong enough incentive for them to act robustly to overcome the barriers to its use. By
doing so they will unleash avenues to new competitiveness among companies, higher efficiency in
the public sector that will enable better services, even in constrained fiscal times, and enable firms
and even whole economies to be more productive.10

Big data is a big deal

The era of Big Data could yield new management principles. In the early days of professionalized
corporate management, leaders discovered that minimum efficient scale was a key determinant of
competitive success. Likewise, future competitive benefits are likely to accrue to companies that
can not only capture more and better data but also use that data effectively at scale. We hope that
by reflecting on such issues and the five questions that follow, executives will be better able to
recognize how big data could upend assumptions behind their strategies, as well as the speed and
scope of the change thats now under way.

1. What happens in a world of radical transparency, with data widely available?

As information becomes more readily accessible across sectors, it can threaten companies that have
relied on proprietary data as a competitive asset. The real-estate industry, for example, trades on
information asymmetries such as privileged access to transaction data and tightly held knowledge
of the bid and ask behaviour of buyers. Acquiring both requires a significant expense and effort. In
recent years, however, online specialists in real-estate data and analytics have started to bypass
agents, permitting buyers and sellers to exchange perspectives on the value of properties and
creating parallel sources for real-estate data.

Cost and pricing data are becoming more accessible across a spectrum of industries. Another swipe
at proprietary information is the assembly by some companies of readily available satellite imagery
that, when processed and analyzed, contains clues about competitors physical facilities. These

10
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
satellite sleuths glean insights into expansion plans or business constraints as revealed by facility
capacity, shipping movements, and the like.11

One big challenge is the fact that the mountains of data many companies are amassing often lurk
in departmental silos, such as R&D, engineering, manufacturing, or service operations
impeding timely exploitation. Information hoarding within business units also can be a problem:
many financial institutions, for example, suffer from their own failure to share data among diverse
lines of business, such as financial markets, money management, and lending.

Often, this prevents these companies from forming a coherent view of individual customers or
understanding links among financial markets.

Some manufacturers are attempting to pry open these departmental enclaves: they are integrating
data from multiple systems, inviting collaboration among formerly walled-off functional units, and
even seeking information from external suppliers and customers to co-create products. In
advanced-manufacturing sectors such as automotive, for example, suppliers from around the world
make thousands of components. More integrated data platforms now allow companies and their
supply chain partners to collaborate during the design phasea crucial determinant of final
manufacturing costs.

2. If you could test all of your decisions, how would that change the way you compete?

Big Data ushers in the possibility of a fundamentally different type of decision making. Using
controlled experiments, companies can test hypotheses and analyze results to guide investment
decisions and operational changes. In effect, experimentation can help managers distinguish
causation from mere correlation, thus reducing the variability of outcomes while improving
financial and product performance.

Robust experimentation can take many forms. Leading online companies, for example, are
continuous testers. In some cases, they allocate a set portion of their Web page views to conduct
experiments that reveal the factors that drive higher user engagement or sales gains. Companies
selling physical goods also use experiments to aid decisions, but Big Data can push this approach

11
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
to a new level. McDonalds, for example, has equipped some stores with devices that gather
operational data as they track customer interactions, traffic in stores, and ordering patterns.
Researchers can model the impact of variations in menus, restaurant designs, and training, among
other things, on productivity and sales.12

Where such controlled experiments arent feasible, companies can use natural experiments to
identify the sources of variability in performance. One government organization, for instance,
collected data on multiple groups of employees doing similar work at different sites. Simply
making the data available spurred lagging workers to improve their performance.

3. How would your business change if you used Big Data for widespread, real-time customization?

Customer-facing companies have long used data to segment and target customers. Big Data permits
a major step beyond what until recently was considered state of the art, by making real-time
personalization possible. A next-generation retailer will be able to track the behavior of individual
customers from Internet click streams, update their preferences, and model their likely behavior in
real time. They will then be able to recognize when customers are nearing a purchase decision and
nudge the transaction to completion by bundling preferred products, offered with reward program
benefits. This real-time targeting, which would also leverage data from the retailers rewards
program, will increase purchases of higher-margin products by its most valuable customers.

Retailing is an obvious industry for data-driven customization because the volume and quality of
data available from Internet purchases, social-network conversations, and, more recently, location-
specific smart phone interactions have mushroomed. But other sectors, too, can benefit from new
applications of data, along with the growing sophistication of analytical tools for dividing
customers into more revealing microsegments.13

4. How can Big Data augment or even replace management?

Big data expands the possible domains of application for algorithms and machine-mediated
analysis. At some manufacturers, for example, algorithms analyze sensor data from production

12
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
13
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
lines, creating self-regulating processes that cut waste, avoid costly (and sometimes dangerous)
human interventions, and ultimately lift output. In advanced, digital oil fields, instruments
constantly read data on wellhead conditions, pipelines, and mechanical systems. That information
is analyzed by clusters of computers, which feed their results to real-time operations centers that
adjust oil flows to optimize production and minimize downtimes. One major oil company has cut
operating and staffing costs by 10 to 25 percent, while increasing production by 5 percent. Products
ranging from copiers to jet engines can now generate data streams that track their usage.
Manufacturers can analyze the incoming data and, in some cases, automatically remedy software
glitches or dispatch service representatives for repairs.

Some enterprise computer hardware vendors are gathering and analyzing such data to schedule
preemptive repairs before failures disrupt customers operations. The data can also be used to
implement product changes that prevent future problems or to provide customer-use inputs that
inform next-generation offerings.

The bottom line is improved performance, better risk management, and the ability to unearth
insights that would otherwise remain hidden. As the price of sensors, communications devices, and
analytic software continues to fall, more and more companies will be joining this managerial
revolution.

5. Could you create a new business model based on data?

Big Data is spawning new categories of companies that embrace information-driven business
models. Many of these businesses play intermediary roles in value chains where they find
themselves generating valuable exhaust data produced by business transactions. One transport
company, for example, recognized that in the course of doing business, it was collecting vast
amounts of information on global product shipments. Sensing opportunity, it created a unit that
sells the data to supplement business and economic forecasts.

Another global company learned so much from analyzing its own data as part of a manufacturing
turnaround that it decided to create a business to do similar work for other firms. Now the company
aggregates shop-floor and supply-chain data for a number of manufacturing customers and sells
software tools to improve their performance. This service business now outperforms the companys
manufacturing business.14

THE 8 MOST IN-DEMAND BIG DATA ROLES

As more organizations begin to get on the big data bandwagon, demand for IT professionals with
the skills to collect, organize, analyze and architect disparate sources of data is on the rise.

Demand For Big Data Skills On the Rise

In today's digital world there is no shortage of data. In fact, many times there seems to be too much.
Companies are investing in cloud technologies, mobile technologies and social media. Combine
that with everything else it takes to run an organization and you start to see how much data is being
collected. Along with the rise of this data is an increase in demand for people who can collect,
organize and make sense of it, according to recent data from Kforce, a staffing firm headquartered
in Tampa, Florida. The salaries listed here are from and are based on a typical IT professional
working 40 hours a week.

ETL (Extract, Transfer and Load) Developers

With the eruption of data and the variety of data types that companies are seeking to take advantage
of, there has been a significant increase in the need for professionals with the skills to acquire and
integrate big data. ETL developers mainly work with different sources of data an organization may

14
McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive advantage, Ivey
Business Journal [Link]
have and creates ways to extract data from sources, as well as, import it and modify it to fit the
needs of the organization and then add it into a data warehouse.

"Given that the ETL software industry is rather mature, these positions are likely to have some of
the longest tenures in the Big Data resource pool, and are often a mix of employees and contract
resources," says Greg Jones, CTO of Kforce.15

Hadoop Developers

Hadoop is a Java-based, open source framework that supports the processing of large data sets.
According to Kforce, data within the framework of Hadoop and various technologies are in high
demand -- Hive, HBase, MapReduce, Pig and so on. This is due to the data volume demand, and
the fact that the cost to process terabytes/petabytes with conventional business intelligence tools
would be too cost prohibitive as well as take too long without massive distributed processing.
"Individuals with experience in the Hadoop framework are the most sought after resources in
today's big data landscape. These positions tend to be primarily contract resources as organizations
mature their long term big data strategy," says Greg Jones, CTO of Kforce.

Visualization Tool Developers

Massive amounts of data can prove challenging to analyze. New types of visualization toolsets like
Spotifre, Qlikview and Tableau allow for intuitive and speedy data probing. While these positions
could fall close to a generic business intelligence developer category, Hadoop is really hot now and
is a new breed of a specialized skill set, according to KForce.

"These skills are highly sought after in the short run as contract resources. As the supply of these
resources catches up to the demand, and the toolsets mature, it is likely that the rates for these

15
Hein, R., (2014), The 8 most in-demand Big Data roles, CIO. [Link]
organization/[Link]#slide9
positions will moderate, and therefore more of these resources will converts to full time employee
positions," says Greg Jones, CTO of Kforce.

Data Scientists

Previously referred to as data architects who were a part of IT, the data scientist is a new breed of
technology professional able to tie their data organization techniques into business value
propositions. They must also have great communication skills to explain data findings to both IT
leadership as well as business leaders. These data scientists typically have their own sandbox in
which to explore and examine an organization's data and help drive innovation.16"Part analyst, part
artist, a data scientist is somebody who is inquisitive, who can stare at data and spot trends. It's
almost like a Renaissance individual who really wants to learn and bring change to an
organization," says Anjul Bhambhri, vice president of big data products at IBM.

OLAP Developer

On-Line Analytical Processing developers are experts in the optimization of data organization to
empower what Kforce calls, "slice and dice" analytics. This is the process of taking data from
relational or nonstructured data sources and creating dimensional models - often referred to as
"Star" and "Snowflake" Schemas, and then building the User Interface to access the data via high
performance predefined queries.

Data Warehouse Appliance Specialist

"These individuals specialize in appliances such as Teradata, Neteeza and Exadata, according to
Kforce. The core responsibilities associated with this role include data integration, administration
and optimization of performance associated with these high end machines. These special
appliances are used in an organization to provide Massive Parallel Processing (MPP), by using
optimized memory, disk and data storage architectures specific to an analytics processing
environment." - Kforce's Big Data Team.

Predictive Analytics Developer

16
Hein, R., (2014), The 8 most in-demand Big Data roles, CIO. [Link]
organization/[Link]#slide9
"Predictive analytics are used heavily in marketing organizations to predict consumer behavior and
target product audiences," according to Kforce. This role can at times seem somewhat similar to
the data scientist in the exploratory nature of what they do, examining many "what if? " scenarios
associated with an organizations data. These highly skilled IT workers are experts in building
potential business scenarios, and utilizing assumptions based on historical data performance to test
thresholds and predict future performance.17

Information Architect

"Big Data has created a renewed interest in Data Mastery," according to Kforce's Big Data Team.
To take full advantage and build an actionable plan using the company's data takes a special skill
set. Information architects must know how to define and document key elements and ensure that
data is being organized and interpreted in the most impactful way. Master data management,
business knowledge and data modeling are all key skills you'll need if you'd like to occupy this
role.18

REPORT: THE ROLE OF BIG DATA IN THE MARKETING INDUSTRY

Marketers are moving from the information-gathering stage to the analytics phase of big data
adoption. But a downturn in hiring could stall big data implementation.

17
Hein, R., (2014), The 8 most in-demand Big Data roles, CIO. [Link]
organization/[Link]#slide9
18
Hein, R., (2014), The 8 most in-demand Big Data roles, CIO. [Link]
organization/[Link]#slide9
A new study from Infogroup Targeting Solutions found that companies will continue spending
heavily on big data marketing initiatives in 2014, but fewer companies plan to hire for data-related
positions.

"The survey findings also indicate that marketers are moving from the information-gathering stage
to the analytics phase of big data adoption. But a downturn in hiring could stall big data
implementation, as the need for human capital is greatest during the analysis and action stages",
said David McRae, president of Infogroup Targeting Solutions.

The survey, which was conducted at the Direct Marketing Association's 2013 Annual Conference,
also found that while some DMA2013 survey respondents acknowledged barriers to big data
adoption ranging from limited budgets to fragmented systems, half of the marketers surveyed by
ITS are enthusiastic about the role of big data in the marketing industry.19

Marketers Budget for Big Data Solutions

The report, based on a survey of almost 400 marketers at the DMA13 Annual Conference, found
for the second straight year that more than 60 percent of companies expect their big data marketing
budgets to increase. But the majority of marketers do not plan on adding new employees to handle
their data efforts in 2014, a reversal from a year ago when most companies said they did expect to
hire for big data positions.

"Big data is meaningless without manpower," McRae said. "While it's exciting that most companies
are making bigger investments in big data, marketers should not forget that it takes people to make
sense of the information. Hiring before reaching the analytics stage enables companies to become
data-led and act on the data."

Heading into 2013, almost 70 percent of marketers said they expected data-related spending to
increase in the year ahead. The spending spree will continue in 2014, with 62 percent of marketers

19
Beal, V., (2014), Report: The role of Big Data in the marketing industry, Webopedia.
[Link]
[Link]
saying their big data budgets will increase. The slight decline could be an indication that fewer
marketers are budgeting for data solutions for the rst time.

Big Data ROI

According to the Infogroup Targeting Solutions report, 54 percent of marketers have already
invested in big data solutions, a number that is expected to rise to almost 90 percent in the next ve
years.20

BIG DATAS ROLE IN INFORMATION-CENTRIC ORGANIZATIONS

Big data. Everyone is talking about it; in fact, it seems to be the buzzword du jour. But, what is big
dataor, more to the point, whats the big deal about big data? What does big data mean for
information professionals, and what impact will it have on us? Gartner, a leading information
technology research and consulting firm, defines big data as high-volume, highvelocity and high-
variety information assets that demand cost-effective, innovative forms of information processing
for enhanced insight and decision making21 (Gartner 2014). Thomas H. Davenport, author of the
recently published book Big Data @ Work, puts it more simply: big data, he says, is perhaps the
most sweeping change in what we do to get value from data since the 1980s22 (Davenport 2014).

20
Beal, V., (2014), Report: The role of Big Data in the marketing industry, Webopedia.
[Link]
[Link]
21
Gartner Inc. 2012. Gartner Says Big Data Creates Big Jobs: 4.4 Million IT Jobs Globally to Support Big Data By
2015. Press release. 22 October.
22
Davenport, Thomas H. 2014. Big Data @ Work: Dispelling the Myths, Uncovering the Opportunities. Boston:
Harvard Business Press.
Just how big is big data? Enormous and getting more so, according to a report published by IDC,
a provider of market intelligence to IT and telecommunications firms. The digital universe in the
United Statesthe digital bits created, replicated, and consumed each year in the countryis
expected to grow from 898 exabytes to 6.6 zettabytes between 2012 and 2020, or more than 25
percent a year, which means it will double about every three years (Gantz and Reinsel 2013). IDC
also forecasts that the market for big data technology and services will grow at a 27 percent
compound annual rate, to $32.4 billion through 2017 (IDC 2013).

While there is no disputing that the quantity of data is immense (be it text, voice, images, or video),
the real revolution is that we can now do something with this data using analytics and thereby
extract some meaningful value from it. A recent Bain & Company report revealed that companies
that use analytics insights perform better financially than their counterparts that do not use
analytics, are three times more likely to execute decisions as intended, and are five times more
likely to make decisions faster23 (Wegener and Sinha 2013). When you get down to brass tacks,
data without context is just data.

So, how are organizations using big data analytics? Retailers are analyzing customer activity to
guide product placement decisions; colleges and universities are using predictive analytics to
determine the success rate of students; and governments are developing models to predict criminal
activity.

Big Data and Your Operations In addition to putting context and meaning around big data for your
organization as a whole, you can examine and analyze the big data in your library or information
center. You might be surprised to realize that you are already collecting and using big data in your
operations. Think about the Gartner definition of big data at the beginning of this article: assets
that demand cost-effective, innovative forms of information processing for enhanced insight and
decision making. Then think about the massive amounts of data that libraries and information
centers gather on a daily basis, such as circulation, acquisitions, cataloging, and other types of data

23
Wegener, Rasmus, and Velu Sinha. 2013. The Value of Big Data: How Analytics Differentiates Winners. Boston,
Mass.: Bain & Company.
from an integrated library system, usage data from online databases, Website visits and
downloaded documents, and research requests and reference questions.24

7 NEW BIG DATA ROLES FOR 2015

The roles, the responsibilities

Data Trader The Data Trader is the high flying market maker of the alternative data universe.
They are essentially the gears and oil of the data market, introducing market providers of data to
market consumers of data. The Data Trader identifies potentially undervalued data, and price and
quality discrepancies in alternative data sources, and then seeks to leverage these discrepancies in
order to monetise their valuable role in keeping the data market healthy. Data Traders also seek
out data instruments on the instructions of a client. They may also issue and buy options and futures
contracts on commoditised data, to be optionally executed and delivered at a later date. Although
it is technically feasible, Data Traders will rarely trade on their own account. 25

Data Hound Although the Data Hound is a special pedigree breed of data management role, the
job of the Data Hound is essential to the work of the Data Trader. When the Data Trader gets a new
requirement for novel, fresh or new data, the Data Hound is charged with searching out the best,
cheapest and most reliable sources for that data, and to identify the owners and vendors of that
data. Essentially, they assist in the data market-making responsibilities of the Data Trader. But
there is more to the role than that. Only a Data Hound can bring infectious enthusiasm to a long
walk on the data landscape. Only a Data Hound can be such a perfect, patient distraction for the
knowledge workers. And only a Data Hound can dispel all gloom, tension and work-stress with a
single explosion of excitement every time you walk through the data portal.

Data Plumber The Data Plumber designs, builds and maintains the infrastructure to ensure that
any validly supplied data reaches the data preparation stage prior to its selection, analysis and

24
Teague, E., Legeros, J., (2014), Big Data's role in Information-Centric Organizations, Information outlook,
Special Libraries Association. [Link]
25
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
consumption. The Data Plumber is charged with ensuring that the required data correctly gets from
the data provider to the data consumer, first time, every time.

Typical responsibilities of the Data Plumber may include:

Reading drawings and specifications to determine the layout of data supply, information
repositories and knowledge systems.
Detecting faults in data plumbing appliances and systems, and correctly diagnosing their
causes.
Locating and marking positions for data pipe adapters, ports and channels, and fixtures in
data centre walls, ceilings and floors.

Data Butcher The Data Butcher works in conjunction with the Data Chef. The Data Butcher
selects and prepares the desired parts of the supplied data which they then pass on to the Data Chef
for data mining, ad-hoc predictive analysis and visualisation. The Data Butcher removes the fat
data from the lean data, and provides quality data that can then be subsequently sliced, diced and
spiced in downstream analytics applications. 26

In years from now, IT archaeologists will marvel at the Tau influences inherent in the role of the
Data Butcher in particular, and data architecture and management, in general. By way of evidence,
the following is a philosophical anecdote from the future: There was once a Data Butcher who was
preparing a piece of Big Data for a customer who had been coming for many years. Pardon me,
Sir the customer asked, But isnt that the same ETL you used last year? Don't you ever need to
upgrade it or maybe go for a more sophisticated and sharper solution?

No replied the kindly Data Butcher Its the same ETL Ive been using for the last 17 years.
He stared wistfully into the distance for a few moments, looking for inspiration, and then continued.
And I havent had to upgrade it, sunset it or change it even once. For, when I select, transform
and integrate raw data, I allow the trusty ETL to find its own way through it without effort or stress.
Just like Bill told me. And when I come to a tricky bit with lots of disconnected, superfluous and

26
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
erroneous data, I just slow down and allow the mystery to solve itself and in no time the good data
comes right through the process. (Adapted from Chuang Tzu: The Basic Writings, 1964)

Data Chef If youve ever seen a great Chef working, up close and intimate (to use a west coast
expression), then you will appreciate the need for the role of Data Chef. First and foremost, the
Data Chef is curator of all the organisations data analytics recipes. They have the data analytics
knowledge. Ideally, the Data Chef has a solid grounding in formal statistical methods and a solid
appreciation of data architecture. This may also be augmented by a wide range of other skills, such
as in Nouvelle analyse des donnes. The Data Chef also works in conjunction with the Data Trader
and the Data Butcher to determine and identify prime data material in the data markets. Based on
the available of prime data the Data Chef is able to determine a menu of data analysis approaches
that will dynamically change depending on what data is in season and available on the data
market.27

Data Taster - A Data Taster is a person that takes data (or information) to be provided to a person
or entity to confirm that it is safe to issue. This is perhaps one of the oldest professions in data,
coming, as it does, from the ancient Roman role of praegustator or data unini. The person or entity
to whom the data is going to be issued is usually an important person or body (for example, a
regulatory reporting body or an organisational strategy group) or any person or body that could
possibly be placed at risk if the data (or information) is erroneous, misleading or compromised. For
example, the Data Taster verifies the outcomes of Big Data Analytics and confirms that the data is
plausible and that the models used are valid so that they do not permit either the accidental or
intentional introduction of data contamination. The Data Taster may also be accountable for the
preparation and provision of data. The hope is that the Data Taster will be conscientious and
meticulous in preventing contamination from being introduced into data, in order to safeguard their
own reputation and that of their organisation.

27
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
Data Server The Data Server is a role that is closely tied to the roles of Data Whisperer and Data
Czar. At a superficial level, the Data Server presents the data menu and takes the data orders, then
serves what has been ordered. The Data Server may also advise data clients on the optimal choices
of data, based on the data that is available and the data preferences of other clients. Because the
role of the Data Server requires that they know a little about everything and a lot about something,
the two most popular career progression paths for Data Servers are moves into the role of Data
Whisperer or Data Shouterer.

Data Whisperer Is integrally associated with the roles of Data Server and Data Czar. This role is
an extremely important key-stone position within an organisation. The Data Whisperer is an
explainer, a storyteller and a stand-up philosopher. The primary responsibility of the Data
Whisperer is to avoid that any senior executive or regulatory body throws a wobbly when they
fail to correctly interpret the data that they are provided. Therefore, the responsibility of the Data
Whisperer is to correctly socialise data analysis outcomes with the intended audiences for those
outcomes, and to jointly present and explain those outcomes in plain and simple language. 28

They are required to have courage, strength and a high degree of empathy both with the data and
also with the consumers of that data.

Data Czar Typically this is a senior board role, comparable to that of CFO. Indeed, the role of
Data Czar (or Data Tsar, for the British) may also be held by the CFO. The role itself is that of
visible figurehead of all data architecture and management activities within an organisation.
Although it bears some striking resemblance with the now widely discredited role of bygone days,
which I will delve into in subsequent articles, its remit goes much further. The Data Czar has the
ability and the empowerment to break down barriers, cut through red tape and knock down walls
that create organisational silos. They can free and easily engage with and involve senior
organisation players in their data campaigns and battles, gaining commitment, trust and willing

28
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
complicity along the way. Naturally, the Data Czar can also call on the skills, talents, knowledge
and experience of the other 9 roles identified here. 29

Data Shouterer (aka Data Shouter) Finally, we come to the last of the roles. The Data Shouterer
is primarily the role of the data evangelist, the extoller of grand data truths, the purveyor of
serendipitous comfort, and the herald of a brave new data world. When things go well the Data
Shouterer is called upon to holler out the successes of data analysis from the rooftops. When
success is reluctant to come forth, they must be there to big up the inherent potential for success,
with brave tales of data buccaneering, ace information pilots and glorious exponents of the Art of
Data.

So there you have a brief explanation of the 7(+3) new big data roles for 2015. So, to conclude
Necessity is the mother of invention and mother is the invention of necessity, and just as the
judicious use of parallel grep, awk and bash could have been reinvented, rebadged and released as
perhaps justifiably the next best thing in data, so too must there be concomitant and relevant roles
to suit the revolutionary data spirit of the times. No?

But, to paraphrase John McEnroe, surely this piece cannot be serious? To which I might reply,
maybe yes, or maybe no, it simply depends. But depend on what?

The English writer George Orwell once mused that The most effective way to destroy people is
to deny and obliterate their own understanding of their history, to which I could add, This may
occur whether the act is intentional, accidental or systemic. I think it is important that when we
look at any new IT industry trend or fad, that we do so with a reasonable knowledge of IT history
and the evolution of IT technology, and with a good understanding of contemporary and legacy
technologies and architectures. This, to my mind, is how we respect both the IT/Information
Architecture and Management profession and those whom we seek to help.

Finally, dear reader, although as I state my intention is quite serious, please do take this piece with
a modicum of sodium chloride.30

29
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
30
Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn. [Link]
roles-2015-martyn-jones
 REFERENCES

1. Beal, V., (2014), Report: The role of Big Data in the marketing industry, Webopedia.
[Link]
[Link]
2. Davenport, Thomas H. 2014. Big Data @ Work: Dispelling the Myths, Uncovering the
Opportunities. Boston: Harvard Business Press.
3. Gartner Inc. 2012. Gartner Says Big Data Creates Big Jobs: 4.4 Million IT Jobs Globally
to Support Big Data By 2015. Press release. 22 October.
4. Hein, R., (2014), The 8 most in-demand Big Data roles, CIO.
[Link]
[Link]#slide9
5. [Link]
insights/the-role-of-big-data-in-medicine
6. [Link]
7. Jones, M. R., (2014), 7 new Big Data roles for 2015, LinkedIn.
[Link]
8. McGuire, T., Manyika, J., Chui. M.,Chui, M., (2012), Why big data is the new competitive
advantage, Ivey Business Journal [Link]
data-is-the-new-competitive-advantage/
9. Schadt, E., (2015), The role of Big Data in medicine, Interview.
10. [Link]
insights/the-role-of-big-data-in-medicine
11. Teague, E., Legeros, J., (2014), Big Data's role in Information-Centric Organizations,
Information outlook, Special Libraries Association. [Link]
content/uploads/2014/12/[Link]
12. Wegener, Rasmus, and Velu Sinha. 2013. The Value of Big Data: How Analytics
Differentiates Winners. Boston, Mass.: Bain & Company.