The New Data Center Imperatives
Demand for new and upgraded data centers will continue to grow, despite the economic
downturn. Planning for new construction or major renovations, however, must reflect the
realities of energy and financial markets and leverage opportunities presented by new
technologies. Careful planning can help IT and finance managers get the most from
data center investments and build a flexible foundation for enterprise growth.

Introduction
Data is the lifeblood of the modern past were inconvenient and potentially
organization. While rapid advances in costly, they disrupted relatively few systems.
technology for personal computers and Failures today can quickly bring operations
mobile devices receive more attention and to a halt in ways that are all too evident to
accolades in the press, the data center customers and investors, and ruin a firm’s
remains a critical element that impacts the reputation. Catastrophic failure – for even a
quality, variety, and reliability of information- short time – is not an option, but that carries
based services that power and differentiate a cost and carries implications for site
an enterprise. A great data center may be selection, design, and operations.
invisible to customers while enabling
services that delight them, but a poor data The traditional concerns for data center
center quickly becomes visible to all. planning – price, performance, and risk
management – have been joined by
“greenness” as a goal and by cloud
The  data  center  has  matured  from  a   computing as a potential business model.
back  office  function  to  become  the   We will see that going green is inextricably
nexus  of  communications,   linked and compatible with the traditional
collaboration,  and  computation.   concerns, and that cloud computing is an
increasingly important part of an enterprise
Early data centers featured the most strategy, but doesn’t eliminate the
powerful computers of their day with often- requirement for internal data centers.
inaccessible enterprise applications,
generally evaluated by raw computational Cloud  computing  is  an  increasingly  
power. As much of this power moved to
departmental and desktop computers,
important  part  of  an  enterprise  
security and reliability were often sacrificed strategy,  but  doesn’t  eliminate  the  
in the interest of flexibility. Data centers requirement  for  data  centers.  
became hubs for distributed computing,
while maintaining their enterprise application In fact, many of the technologies
    and
role and expanding it to become he process we consider – from ambient cooling
communication center for the organization. to low power processors to virtualization –
reduce the carbon footprint of an enterprise
Today, the data center must still support while reducing costs. A green data center
enterprise applications, provide a repository should help an enterprise operate in the
for distributed data, offer bulletproof black.
communications, and increasingly support
collaboration platforms. Customer-facing A  green  data  center  should  help  an  
and supply-chain systems depend on enterprise  operate  in  the  black.  
reliable data centers. While failures in the
 2  
 
 
 
 

Whether one is considering a new or approach, and in the following sections we

replacement data center, the main address the major risks and criteria to be
considerations fall into the three categories considered in site selection, data center
familiar to managers in all industries: design, services for construction and
people, processes, and products. Each commissioning, and technology selection.
category requires a risk management

Site Selection and Data Center Design

Physical site selection requires careful consideration of costs and risks, and recognition
that many factors – including the regulatory environment – may change during the life of
a data center. As power becomes scarce in some regions and governments use
incentives to attract businesses, it is important to have a long-term plan that also meets
short-term objectives. Some design decisions – for example a preference for ambient
cooling for green or economic goals – may influence site selection. Some site choices
will likewise constrain design options (sealed modular solutions required in hostile
environments, for example).

Site Selection A data center must offer reliable operations

for a long time at relatively predictable costs,
Data centers require a physical location, and be flexible enough to grow – or shrink –
even those built to support cloud computing as business needs change. It will likely
environments. In residential and most house critical data about clients, processes,
commercial real estate, it is all about personnel and operations that could
location, but data compromise the
centers do not need business if breached,
Key  Considerations  for  Site  Selection   or open the firm to
curb appeal. Similar
to manufacturing
• Political/governance   legal penalties if
plants that need to • Network   Connectivity   privacy, security, or
be able to get • Energy  Availability  &  Cost   governance
regulations are broken.
materials in and • Water  
products out reliably, • Climate  &  Natural  Disaster    
data centers require Political/governance
a fault tolerant • Labor  Pool   At the national level,
internet connection • Green  Attributes   decisions are made on
to process that data, using   servers that the basis of stability of government,
require power and water (for cooling), and business climate fostered by that
  government (taxes and labor incentives),
people with technical skills.
and regulatory impact of site location. While
Choosing a site requires development or the first two items are self-explanatory, the
adoption of criteria, prioritization of values, regulatory impact is critical and a poor
and ultimately evaluation and selection. The choice has long-term consequences.
actual cost of the real estate will be a factor,
of course, but that is influenced by the other A firm considering construction of a data
considerations in this section. Construction center in a country experiencing or
costs, as measured by the construction price anticipating rapid growth to serve the local
index, is another one time factor to be market may not realize that the government
considered. has a hands off approach to intellectual
property rights, or worse still, fails to keep up
 3  
 
 
 
 

with international leaders in privacy law. financial services firms considered to be part
of the critical infrastructure of the US, there
For example, the European Union has strict are legal requirements for location based on
rules concerning the handling of personal disaster recovery and business continuity
information. The EC Directive on Data assurance needs.
Protection (1998) prohibits the transfer of
certain personal data to non-EU countries Network connectivity
that fail to meet the European standards. It goes without saying that a data center is
That means that an enterprise with useless without connectivity. Government
European customers must factor these agencies or private entities may facilitate
regulations into data center location access to reliable, high-speed Internet
decisions, either by locating in an EC connections. Redundant access is desirable
country, an EC compliant nation, one with for most commercial data centers, and
authorized workarounds (like the US, which provision for access is a critical criterion for
has a Safe Harbor framework through the site selection.
US Department of Commerce and the EC),
or be able to ensure that such data is Energy Availability and Cost
effectively segmented and kept out of the Data centers require uninterrupted access to
non-compliant data center. If the country in power at predictable prices. As capacity is
which the data center is located does not strained in some places, pricing has
legally require compliance, a firm may be in naturally risen to dampen demand.
violation even if all the EU rules are being
followed by the enterprise. Best practice today includes locating at the
intersection of multiple grids or at least
As new regulations emerge, data center separate substations, to reduce the impact
planners must keep current, and operations of local or regional failure, and the use of
must be able to segment data based on uninterruptible power supplies (UPS) to take
regulatory requirements that change after a over for short outages (measured in
center goes live. While the choice of a minutes) and diesel-powered generators for
country may be influenced by existing or longer outages (hours or possibly days).
anticipated regulation, the design must
account for later changes in regulation that Controlling costs long term requires insights
constrains the capture and transfer of into the energy generation plans for
regulated data. alternative providers. Hydroelectric
generation pricing is relatively stable, as has
National governments also become a factor been nuclear power historically in the US.
when they support infrastructure Power generated from fossil fuels has seen
investments, such as broad access to the a dramatic price increase in recent years,
Internet. Finland recently became the first and uncertainty about future supplies needs
country to make broadband Internet access to be factored into site decisions. Alternative
a legal right for its citizens, ensuring further energy sources, from wind to solar to
investment in infrastructure. The US, biomass are still in the early stages so
through the Rural Electrification Agency of historical pricing provides no basis for long-
the US Department of Agriculture, continues term projections.
to invest in bringing connectivity to rural
areas, which benefits firms considering Water
locating data centers in these previously In addition to a potential role for electricity
underserved areas. generation, water is usually a critical
component to data center cooling strategies.
Finally, for some businesses, such as the Even in situations where little or no fresh
 4  
 
 
 
 

water is required for operations (gray water new skills may be demanded by changes in
will often suffice and can be recycled the technology and infrastructure (new
effectively, ensuring some degree of energy management skills, for example),
predictability), data centers require people and where will they come from?
and people require water.
Green Attributes
The increasing global concern over clean Going green – reducing the enterprise
water indicates that long-term availability of carbon footprint – has intangible benefits
fresh water should be a criterion for site beyond the scope of this paper. The tangible
selection. benefits are discussed throughout this
section and the next, as lowered energy
Climate & Natural Disasters costs due to temperate climates, and low
As we will see in a later section, a moderate energy components quickly show up on the
climate can dramatically lower energy costs bottom line.
by enabling new “free cooling” approaches.
It is important to note, however, that some
The likelihood of natural disasters, long a locations, such as the Pacific Northwest,
part of disaster recovery and business which offer lower energy costs due to widely
continuity planning, is also a critical aspect available hydroelectric generation, may also
of site selection. offer more intangible benefits than
comparable locations elsewhere. Less
Considerations include floodplains, tornado obvious than the ongoing public relations
and hurricane paths, and fault-lines. One benefits of locating a corporate
should also consider man-made risks, such headquarters in such a region, perceptions
as proximity to chemical hazards – including of regional sensitivity to sustainable
manufacturing and rail lines that transport operations may improve the ongoing labor
chemicals – and likely terrorist targets. One pool.
firm reported eliminating over 300 sites from
consideration based on a 50-year review of Data Center Design
these risks.
Reliability
Labor Pool Many data center designers use the 4-Tier
As data center operations become more fault tolerance model developed by the
automated, the number of employees Uptime Institute to guide their hardware
required to manage an installation may drop selection process. This model, which
but the expertise required rarely does. The ranges from an acceptable annual outage of
availability of trained or trainable 28.8 hours at Tier 1 to .4 hours at Tier 4 –
professionals should be a consideration in (the highly vaunted 99.995 percent
any site selection exercise. availability) can be used to determine
specific products that may be combined to
This criterion is usually expressed as a give acceptable performance. Of course, in
function of local labor availability though practice, human error is always a factor to
complementary or competing businesses, consider, so relying on hardware measures
universities, and technical training institutes. alone is insufficient.
As an alternative to leveraging existing local
talent, relocation-ready labor from elsewhere The decision to insist on Tier 3 or 4 may be
may be considered if the location has a a make or break one for financial services
reputation as a growth market. institutions, or other enterprises where even
brief outages can be reputation- or life-
The concern should be to determine which threatening. For other markets, the lower
 5  
 
 
 
 

costs associated with Tier 1 or 2 Some containers can now operate in

performance are an acceptable tradeoff. temperature ranges that make outdoor
placement feasible in temperate climates,
Machine-centric Model further reducing construction costs and
Early data centers were built around a single providing more data center design flexibility.
mainframe, designed to expand with new
storage, peripherals, and eventually Containers solve the old issue of spot
replacement processors. Until recently, that cooling and rack spacing, but they still
model simply evolved to account for the require connectivity to power and water, and
emergence of rack-mounted servers as an in most cases will be installed within or
alternative to the mainframe-centric model. adjacent to a conventional building.

For years, data center design has focused Leveraging containers reduces time to
on maximizing capacity and utilization of deploy, much as assembling a pre-fab
rack mounted servers. The major design building reduces construction time and often
issues centered on optimal distribution of cost. Complete prefab data centers are
servers over the available floor space to emerging, but are not expected to replace
most effectively reduce heat buildup and custom data centers in the next few years.
facilitate maintenance. It was the Until the transition, existing centers will
responsibility of the designers to develop an upgrade using conventional rack-centric
appropriate internal electrical system to technology until a cost benefit analysis
permit redistribution of assets within the data indicates that a shift to containers is in order.
center perimeter.
As the choice of containers grows, offering a
While normal considerations about site Lego-like approach to flexibility, data center
selection and physical security remain valid, design will focus more on the specification of
the basic model is changing. a plan for initial operation and final capacity,
with a roadmap for getting from one to the
New Levels of Modularity other by adding containers and
Not long ago, buying individual servers gave incrementally updating them with
way to buying racks full of servers for most replacement or additional racks.
data center designers and mangers. Of
course, that approach is still valid and will
continue to be used in centers constructed
in recent years. The next generation,
however, is already here and it is hard to
imagine a large new data center being
assembled from such low level units.

Now, shipping containers pre-built with racks

holding up to 2,500 servers are appearing,
and for practical purposes some can be
sealed to operate in hostile environments.
Similar to monitors that operate with random
pixel failures, these containers can operate
with some failed servers while still
maintaining an appropriate reliability level.
 6  
 
 
 
 

Choosing Contractors and Commissioning Agents

The selection and management of specialists to build, examine, and verify everything
specified in data center design is critical, and mistakes made here can be costly
immediately or years down the road. Hiring in-house experts or picking the right
vendors, and managing those relationships to a successful conclusion, requires an
understanding of tradeoffs. From single-source vs. general contractor arrangements to
hiring local labor vs. bringing in outsiders, there is no shortage of opportunities for error.
There are, however, some simple guidelines that will help prevent common mistakes.

Data center commissioning is a system- or affiliated with one or more of the vendors.
based form of reliability science that As a general rule, independence is
improves confidence in the performance of a advisable, as much for a fresh viewpoint and
data
center
against Guidelines  for  Construction  &  Commissioning  Support  
its • Familiarity/certification  with  relevant  standards  
• Recent  experience  with  local  regulations/agencies  
• Recent  experience  with  your  major  design  choices  
• Recent  experience  with  your  vendor(s)  of  choice  
 
requirements   by formally specifying and approach – to catch more problems, and
executing tests of individual subsystems and   earlier - as it is for the appearance of
their interaction. propriety.

Commissioning has evolved to keep pace Independence comes at a cost, however,

with the complexities of cascading failures which includes increased communication
when systems such as air conditioning, overhead. Continuing with the auditing
power, and security have interdependencies analogy, it is appropriate to have internal
that may not appear when each is tested staff and/or vendor contractors work with an
individually. When downtime is expensive, independent commissioning agent as we
the cost of commissioning to improve have internal and external audits for the
reliability is easily justified. books.

As most organizations build relatively few Typically, a general contractor selected to

data centers, the focus should be on owning construct the data center will have working
– hiring or training existing staff – the skills relationships with one or more
related to the business that change less commissioning agents, which can simplify
frequently than data center design and the selection process as long as it doesn’t
implementation. compromise objectivity.

The commissioning agent is responsible for The actual process of data center
developing and testing scripts for every commissioning is similar to testing during
system and subsystem, similar to the way application development. It needs to be part
an auditor reviews the processes that of the early plans and scripts need to be
capture and manage financial information executed during development to ensure that
within the enterprise. Similar to the auditor, problems are found as early as possible.
commissioning agents may be independent Commissioning improves quality, and
 7  
 
 
 
 

cannot be an afterthought. As with software Recent experience with local

development, the sooner a problem is regulations/agencies
detected, the lower the cost to correct. Building codes and local labor conventions
must be considered, and it should be
As data centers become more and more determined as a matter of policy if the
modular, it might be natural to assume that enterprise will need to need to have this
the construction and commissioning expertise in house long term or if contracting
processes will be simplified, and to an extent for it is feasible.
that is true. Just as computer designers can Legal issues often arise during construction,
now assemble more integrated components, and local contractors, or those with sufficient
data center designers and contractors reach to deploy teams to the desired site,
working with larger modules, including pre- should know which skills require licensing,
wired and cooled containers, will have fewer union membership, or other legal
low-level concerns. certifications. Again, recent local experience
is priceless.
In this emerging model, then, the major
considerations for selecting a contractor and
commissioning team are based on an Recent experience with your major
understanding of the required expertise, and design choices
the experience paradox: How much Ideally, the contractor will have recently
experience is enough, and can there be too completed one or more similar projects, from
much of a good thing? The general the choice of container or rack-based to the
recommendation is to develop selection cooling approach (free ambient to water).
criteria at design time, and look for relevant When one is at the leading edge, however
recent experience. that may not be possible so start with
experience in the highest-level design
Years of experience and great references criteria first and prioritize your requirements
from firms that have build last-generation based on the likelihood and cost of failure.
data centers are not enough. Here’s what it
takes: Recent experience with your vendor(s) of
choice
Familiarity/certification with relevant From proprietary approaches to knowledge
standards of procurement personnel, there is no
From the Uptime Institute’s reliability tiers to substitute for experience installing the
ANSI/TIA-942, which specifies products upon which your data center
recommendations on topics including media success depends.
and distance for telecom pathways,
distances, etc, anything specified by the
designer should be understood by the team,
and ideally that understanding will have
been demonstrated recently in similar
projects.
 8  
 
 
 
 

Leveraging new data center technologies

Rapid advances in technologies and delivery models will change the way data centers provide
services. Dell and Intel, already trusted providers of leading edge solutions - from thin
provisioning for storage optimization to data center-wide energy management systems and cloud
computing resources - are well positioned to deliver and support the next generation of data
center. This section presents an overview of the new and emerging technologies that will shape
the data center of the future, from energy efficient components to complete modular data centers
that grow with the needs of the business.

Dell, which The

provides fundamental
data Green  Technologies  Underpinning  Modern  Data  Centers   options are
center • Cloud  Computing     public
support • Low-­‐energy  technologies   clouds,
using Intel • Improved  cooling  approaches   private
Xeon clouds, or a
processors
• De-­‐duplication     hybrid
for • Virtualization/increased  density   approach. In
  a pure public
demanding customers including Akamai, cloud model, a third-party vendor provides
Amazon, Facebook and Microsoft, has services on a pay as you go model that
staked out a strong position in this market. generally allows for expansion and
contraction of requirements and fees. A
In this section, we outline five critical areas client’s data and applications are typically
identified by SIG411 where new computing co-located with those of other clients. A
models and technologies can go green and private cloud approach offers similar on-
simultaneously improve the bottom line of an demand services within an enterprise.
enterprise. Each solution is compatible with
a rack or container approach to data center Dell and Intel solutions are at the heart of
design, and in each we find leading edge public cloud solutions offered by leaders
technology from Dell and Intel. including Microsoft and [Link]. The same
technologies that allow public cloud
Software as a Service/Cloud computing providers to offer simplicity and cost control
Cloud computing is essentially a business through the flexible cloud abstraction model
model for delivering IT as a service. In a are available for internal use. The data
cloud, the user can specify what needs to be center of the future is likely to use an
done without needing to know the details internal cloud for critical and legally sensitive
about where the data will reside or where data, and public clouds for other systems to
the processing will take place. Increasing take advantage of the flexible cost model
the level of abstraction this way simplifies and rapid deployment times (especially as
things for users, but requires additional private clouds near capacity).
services that route the work to the right
place at the right time. Cloud computing
providers require much the same hardware Low-energy technologies/Improved
at the core of their data centers, but cooling approaches
specialized infrastructure management Dell offers a range of Intel-powered personal
software. computers that are US EPA Energy Star 5.0
compliant, which assures energy savings at
the desktop. These models are noteworthy
 9  
 
 
 
 

for low power consumption during operation meetings the world over, meetings have
and idle periods. been ruined when two people each produce
their own version of current data, only to
For the data center, where costs have realize that neither knows which version – if
spiraled in recent years, Dell has made a either – is accurate. The quest for “one
significant investment building the Dell truth” is as important as the quest to control
Energy Efficiency Research Center to test storage costs.
new and emerging technologies under real
world conditions. Among other advances, De-duplication, which can occur at multiple
this lab has enabled Dell to optimize a liquid- levels (from file down to block). Energy
to-liquid heat exchanger and pumping unit requirements for de-duped storage may be
that links liquid-cooled racks with a chilled cut by 25x, which frees capital for other
water system in the building. projects or falls to the bottom line.

Within the servers themselves, Dell utilizes For primary storage needs, file sharing
low-flow fans, which vary speed according to devices such as Dell’s NF and NX products
demand, instead of running constantly, or at reduce real estate and energy requirements.
a high speed whenever any cooling is The PowerVault DL2100 goes further with
required, and higher efficiency power block level de-duplication, the state of the art
supplies. As it has been shown that power today.
supplies account for nearly as much energy
consumption as processors in the typical
data center, this provides a significant cost Virtualization
reduction. Virtualization is a technique that permits one
physical machine to appear to a user –
Intel processors, such as the Xeon 5148, human or machine – to have the
run at lower voltages and wattages than the characteristics of multiple machines. For
processors they replace, with no sacrifice in example, using virtualization software, a
throughput. single Dell server can run multiple operating
systems.
Another approach to energy reduction is to
use ambient cooling with an air-side Dell’s HyperV reference architectures
economizer. As previously noted, in provide virtualization and consolidation
temperate climates, such a system can draw (fewer physical servers required, at higher
in cooler outside air and cycle the warm air utilization rates) with Windows Server 2008,
outdoors. Intel has also shown recently that resulting in smaller space requirements and
servers can operate in higher temperatures significantly lower energy costs.
and higher humidity than previously
specified, literally opening the door to “free Consolidation ratios of 9:1 have been
cooling” in a much wider range of climates. attained with the latest generation of Intel
Xeon processors, and the effect scales
De-duplication remarkably. Emerson, for example, reports
The term de-duplication includes a number consolidating over 130 data centers down to
of techniques and technologies designed to four, using Xeon-based Dell PowerEdge
remove redundant copies of data from a servers.
system or enterprise. Of course, most data
requires an external backup copy, but in Virtualization can also lower software-
many organizations copies of data licensing costs in some cases, as work
propagate and create two problems: wasted streams are more efficiently allocated
space, and lost control. In countless among the virtual machines.
 10  
 
 
 
 

Resources:  Key  Organizations  and  Standards  

 
American  National  Standards  Institute  (ANSI):                              [Link]  
American  Society  of  Heating,  Refrigerating  and  Air-­‐Conditioning  Engineers  (ASHRAE):          [Link]    
Lawrence  Berkeley  National  Laboratory  (LBNL)                        [Link]  
UL:  Underwriters  Laboratories                                  [Link]    
US  Department  of  Energy    
  DOE  Saving  Energy  in  Data  Centers  Guide              [Link]/industry/datacenters/      
  Energy  Star                                  [Link]    
Uptime  Institute                                            [Link]    
Developed   the   de   facto   standard   4-­‐Tier   fault   tolerance   model   for   data   centers,   ranging   from   an  
acceptable  annual  outage  of  28.8  hours  at  Tier  1  to  .4  hours  at  Tier  4    (99.995  percent  availability).    
TIA:  Telecommunications  Industry  Association                                  [Link]  
 
ANSI/TIA-­‐942  Telecommunications  Infrastructure  Standard  for  Data  Centers  
ANSI/NECA/BICSI-­‐002  Data  Center  Design  and  Implementation  Best  Practices