Whitepaper

Sponsored by

Rack Power Distribution:

Just the end of the Power Chain, or Key Strategic Advantage?

By Julius Neudorfer

Data centers have become vital to virtually all aspects of our society, and in many cases are
considered an essential part of our critical infrastructure. Ensuring continuous and reliable
power to the IT equipment is one of the most fundamental aspects of a mission critical
system. Professionally engineered data centers are designed to avoid any downtime by
providing various levels of redundancies in the primary power system components.

This requires a significant capital investment for redundant primary switchgear, UPS units and
generators, as well as the facility’s overall downstream power distribution system, which
ultimately supports the IT equipment in the cabinets. However, in some cases, the power
distribution units (PDU) within the IT cabinets are sometimes overlooked or given little
consideration as to their inherent importance as the last, but critical, link in the power chain.

This whitepaper examines the current technical and operational challenges faced in high-
density power distribution, as well as how the various available power distribution and
management solutions can provide economic and strategic advantages.

Page 1 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Rising Power Demands ered and analyzed by monitoring software, which helps
The power demands and size of the data center have ensure accurate power availability and redundancy, as
increased dramatically over the past several years. More- well as capacity planning. Moreover, the need for remote
over, the overall facility power density (per square foot) power monitoring and management has been driven by
and more significantly, the power density at the rack level the change in strategy in enterprise computing and the
has risen quite considerably as well. widespread use of colocation by the enterprise and has
now become an essential requirement for many organiza-
Historically when the typical server cabinets were fed by tions. Some PDUs offer the ability to control individual
120 volt 15-20 amp branch circuits supporting 1-2 kilo- outlets, allowing remote rebooting of hung IT equipment
watts of load, the rack Power Distribution Unit (PDU) was if necessary. This can improve problem resolution re-
referred to as just a “power strip.” However, modern sponse time and, in many cases, reduce the need and cost
server cabinets require the rack PDU to support much for on-site technical support.
greater power requirements. Typically the lower end ranges
between 2-4 kW and many facilities have medium-density The Green Grid - PUE Metric
cabinets operating at 5-9 kW. And while not as widespread, No discussion about power can avoid the issue of energy
10-25 kW cabinets are no longer making headlines. metering and efficiency, and of course PUE. The Green
Grid created the Power Usage Effectiveness “PUE” metric
While several different data center organizations have in 2007, which is now a commonly accepted international
various definitions of power density, AFCOM released a metric. Several whitepapers that provide detailed infor-
whitepaper in 2014 that categorized Rack Power Density mation on the PUE metric and methods of measurement
into four levels. can be found at [Link].

Rack Power Density Level Power Range (kW)

Low 0-4
Moderate 5-8
High 9-15
Extreme 16 and above
(Source: AFCOM Data Center Institute Standards Endorsed Whitepaper DCISE-001 September 2014)

These high and increasing-power densities and the relat- A simplified view depicting PUE measurement is shown
ed higher operating temperature ranges represent myriad below in Figure 1—and while the metric name incorpo-
engineering and operational challenges on multiple levels, rates the word “power,” it is important to note that as
for both the manufacturers of rack-level PDUs, as well as of 2011 Power Usage Effectiveness refers to annualized
data center operators. energy (measured or averaged over 12 months of opera-
tion).
Rise of the Intelligent PDU as a Strategic
Management Asset In order to provide an assessment of facility energy usage
While the locally metered PDU has been available for and efficiency of a data center, energy metering at several
approximately 20 years, it was a fairly basic device initial- key points of the power chain and distribution system as
ly and not widely deployed. The need for better energy shown are the prescribed method to determine a facility
management and efficiency has given rise to the demand PUE—both as a baseline of existing performance and
for the new generation of remotely monitored and man- to examine the results of any changes made. As seen
aged PDUs, also known as the “smart” or “intelligent” in Figure 1, the highest level of accuracy is obtained by
PDU. These later generation PDUs gained popularity measurement at the rack (PUE Category 3, also known as
especially in the last five years and have really come to Level 3 measurement). Regardless of your data center size
the forefront by offering more-sophisticated features with and power density, tracking energy usage and being able
each generation. These products offer additional granular to correlate it to IT asset utilization is an essential element
rack-level information, which can be automatically gath- to data center management and strategic planning.

Footnote: The use of the words “smart” or “intelligent” (as they pertain to a PDU) are meant as a generic reference and not meant to imply or refer to
any manufacturers’ product trademarks that incorporate those words.

Page 2 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

FIGURE 1. PUE Points of Measurement

Energy Cost Strategies: Enterprise vs maximum power load as well as the actual average energy
Colocation used on a monthly basis. For example, if the IT equip-
In a data center owned and operated by the enterprise, ment specified for your rack requires a maximum of 20
energy costs are a significant part of the Total Cost of amps at 208 Volts, a 30 amp circuit is required. (Based on
Ownership (TCO). As such, it is always important to track the 80% limitation, the maximum total current that can be
how energy is used in order to optimize efficiency—both drawn safely is 24 amperes.) This means that the maxi-
for the facility (PUE) as well as for the IT equipment. mum power that can be delivered to that rack is approxi-
Nonetheless, the utility invoice to the facility owner is mately 5 kVA. You will still need to provision the circuit for
directly based on the actual energy used (kWh). This is the highest current demand expected. However, in many
not necessarily the case for a colocation customer. In a cases, the IT equipment may not continuously draw this
colocation data center the costs are based on two primary amount of power during normal operation and may aver-
factors: the space (either per rack or caged area) and the age only 50% of the energy of the circuit capacity.
power. However, colocation providers typically offer three As can be seen from Table 1 below, it becomes import-
general types of pricing models for power in their facility, ant to monitor and track IT equipment power (peak vs
as seen in Table 1 below. average) and actual energy usage trends. This is one of
the reasons that many colocation customers are installing
The type of power pricing schemes can be either costly higher level intelligent PDUs in their racks that can mea-
or advantageous to the customer, depending on their sure and record peak power and energy usage, not just
those PDUs that can measure only input amperes.

Page 3 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Colocation Provider Power/Energy Cost Model Description Impact on Customer

A: Per Circuit - Fixed Cost Customer is charged a fixed price—regardless of Can be costly if circuits are
(based on Power capacity kVA—not energy usage) the actual energy consumed. underutilized.
B: Energy Metering Customer is charged for the actual energy used (in The energy charge should be based
(actual energy used kWh) kWh). Circuit charges included in facility overhead. on underlying utility rates, plus the
facility overhead (which is generally
C: Combination Customer is charged a fixed cost for the circuit
related to the PUE).
(fixed per circuit power capacity—kVA, plus actual capacity (which would be lower than option A),
energy used - kWh) plus the charge for the actual energy used. Caveat: Each case is unique and
subject to contractual language.
TABLE 1.

are “snapshots”; they do not accurately represent the ac-

Limitations of Typical Building Management tual total energy used by the circuit (kVA/kW reading are
Systems instantaneous power, which varies—kWH represent the
Facility based power management is generally han- actual energy used over time—i.e. per hour, day, week,
dled by Building Management Systems (BMS), which or month).As each generation of IT equipment increased
typically monitor the main Switch Gear, UPS, and the rack power demands, using a basic unmetered “power
Power Distribution Units (PDU) on the data center strip” in the rack soon became inadequate. Beside need-
floor. However, in most cases, the ability to monitor ing to increase the capacity of the branch circuits feeding
power demand or track overall energy usage of the IT each rack, IT administrators found that without visibility
equipment most commonly occurs at the UPS output. into the amount of power that a rack was using, they had
Alternately, it can occur at the floor-level PDU, if it is no way to know if or how many servers could be installed,
equipped with power or energy monitoring (some units or if a server refresh or upgrade would require an increase
only have input monitoring, while others have branch in the capacity or number of branch circuits supporting
circuit monitoring). the rack. Moreover, without knowing how much current
was being drawn, just adding a single server would be
like playing “Russian Roulette,” since it could result in in a
In addition, when redundant circuits are required (A-B tripped circuit breaker. The metered rack PDU can miti-
feeds), pricing is usually based on the total maximum gate this type of exposure to an outage.
power required. Redundant circuits are also subject to the
limitation of 80% of the individual maximum circuit value The lack of real-time power and energy monitoring at the
requirements, typically shared equally across both circuits. rack can delay or disrupt a technical refresh or, worse yet,
(However, regardless of the ratios, the A+B total cannot expose the rack to failure if the branch circuit protection
exceed 80% of the individual circuit value.) This puts trips when more or new IT equipment is installed. More-
the onus on the end-user customer to avoid exposure over, the problem is exacerbated when redundant (A-B)
to “cascade failure.” This fact is often misunderstood or branch circuits are used to feed racks full of servers with
overlooked by many IT administrators when provisioning dual power supplies (dual corded devices normally share
new IT equipment in racks and when projecting estimated the load equally; however, some vary in how they distrib-
colocation power costs. ute the load across each cord or simply fail-over to the
alternate circuit).
If the floor PDU (or electrical sub-panel) was not equipped
with branch circuit monitoring, the only other way this The chart in Figure 2 below depicts a potential scenario
could have been determined would be a request to facili- wherein the typical manual “clamp-on” ammeter is used
ties for a manual electrical survey of branch circuits in the to measure (A-B) redundant branch circuits to a rack at
PDU, or the sub-panel feeding the rack or group of racks. one point in time, while the plot lines show continuous
This is a labor intensive and intrusive process (opening of current measurements over time for the A and B circuits,
energized panels) that increases the risk of an accident
as well as the sum (A+B) of both.
and potential downtime due to human error. Moreover,
the measurements taken are not 100% indicative of the
In this case, at the time the manual readings were taken,
maximum current drawn over an extended period of time
it would seem as if the total current drawn across (A and
under varying IT computing loads. These measurements
B) circuits were only 14 amps (7A+7A). However as the

Page 4 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Example of Manual Survey vs. Continuous Current Monitoring

20 Amp (A-B) Redundant Branch Circuits

FIGURE 2. Example of Manual Survey vs. Continuous Current Monitoring

current plot over time shows, at multiple times during If the wide swings in IT load current seems like an extreme
the day, the sum of the (A+B) circuits exceed the 16 example to those who are more accustomed to a relatively
amp (80%) threshold (which is the maximum current that flat current draw for servers and other IT hardware, this is
should be safely drawn from a 20 amp branch circuit, per no longer an unlikely or extreme scenario. The quest for
US National Electrical Code “NEC”). While under normal energy efficiency in IT equipment has significantly changed
circumstances when both circuits are active, there would the load profile and the idle to maximum power range for
be no problem in the above example. Should a problem modern servers. Whereas only a few years ago a typical
occur, such as the loss of either one of the branch circuits server would idle at 50-60% of maximum power when idle,
(either accidentally or during a maintenance procedure), the newer servers have a much wider ratio and may idle at
the remaining active circuit could trip during the peak cur- only 20-25% of maximum power (the peak current could
rent excursions, since it would now be carrying the entire be 4-5 times the idle current). As a result, a manual survey
load (slightly above 18 amps). This represents a lurking “snapshot” current reading is nearly meaningless, without a
exposure to cascade failure. valid correlation to the servers’ computing load status.

As can be seen by the above example, these peaks would Computing Performance Energy Benchmarking
be very difficult to discover, even with regular manual sur- The PUE metric is fairly straight forward; however it only
vey snapshot readings. This exposure to cascade failure of measures the efficiency of the facility infrastructure. The
redundant power paths can only be revealed by continu- energy efficiency of computing systems has many aspects
ous monitoring and recording of current on each branch that make it far more difficult to quantify into a singular
circuit (A-B) and then setting threshold alarms when the metric for many reasons, one of which is a universally
sum exceeds the prescribed limits. accepted definition of “useful work.” While there are
rating systems for IT hardware such as SPEC power and

Page 5 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

the US Energy Star program, which helps purchasing de-

cisions, they cannot account for how much actual energy Improving Availability and
will be used once software has been installed, as well as
the actual operational utilization profile. Nonetheless, the Uptime
relative efficiency of any given operational computing
system functioning as a whole (IT hardware and software) Optimizing New Equipment Deployment Process
can be benchmarked against itself. One of the advantag- By utilizing rack PDUs with individual outlet switching,
es of individually metered (or metered/controlled) out- any unassigned outlets can be turned off, preventing
lets in the rack PDU is the ability to measure the energy the unauthorized activation of any new (or replacement)
consumed by the individual IT equipment. This data can equipment in the rack without prior workflow approval
then be used to correlate the performance of IT equip- processes. This could prevent an inadvertent overload of
ment assigned to a computing task against its energy a heavily loaded branch circuit from tripping the breaker
usage during that task. This level of analysis can provide a when unapproved IT equipment is plugged into any open
better assessment of how well the hardware and software outlet at random—resulting in the entire rack going down.
perform collectively, as well as delivering an improved
basis for capacity planning. Color Coding to Help Reduce Human Error
The most common color for rack PDUs and their recep-
Determining the Energy Impact of tacles has been black. While this relieves the manufac-
Software Changes turers from having to worry about colors, it makes it a
As noted above by using energy monitoring, either at bit more difficult for technical staff members who work
the rack level or, ideally, the IT device level, baselines in back of the densely packed, dark cabinets, partic-
can be established and correlated to relative comput- ularly when they need to plug-in multiple cords of IT
ing loads. The impact of any software changes can equipment to the correct PDU and outlet position.
then be compared to actual energy used by comparing
known baseline trends over time. For example, with
per-device monitoring, you may be able to observe One of the problems that sometimes occurs during the
that a software change resulted in slightly lower energy installation or replacement of servers (especially those
used by a group of web servers. However, the energy with multiple power supplies) is when the technical staff
usage of the database servers and storage arrays may may inadvertently put the plugs in the same PDU, negat-
have increased significantly. This type of change would ing redundancy of A-B feeds. Also, in the case of blade
not have been readily detectable at the UPS output. servers fed from 3-phase PDUs, it is difficult to consistent-
ly maintain equal and proper coordination of power cord
distribution across the sets of phase pairs. In response
Identifying IT Energy Waste - Finding Zombies to these issues, some manufacturers have recently begun
Moreover, even the process of identifying and correlating offering PDUs in multiple colors (ideal for identifying A-B
the IT equipment, such as its owner and function, along power) and also color-coded outlets on each phase pair to
with its rack location and which outlet of the PDU it is help balance the loads in 3-phase PDUs.
plugged into, can result in better organization of the IT as-
sets. This is especially true in older enterprise data centers Better Phase Balancing
that have a history of having “zombies”—those servers and One common problem is trying to maintain phase bal-
other devices that are no longer being used, yet are still ancing in the rack. Some manufacturers have recently
drawing power (as well as adding to the cooling load). developed PDUs that feature a 3-phase electrical distribu-
tion scheme related to the receptacle positioning pattern.
The Uptime Institute conservatively estimated that up to It sequentially interleaves alternate phases in a continuous
10% of enterprise servers are running obsolete or unused pattern (as well as color coding the outlets)—rather than
software having no function at all, yet remain in opera- grouping the outlets as a block of outlets per phase. This
tion. Decommissioning a single 1U rack server can result feature makes it more intuitive for IT personnel to evenly
in $500 per year in energy savings, an additional $500 in distribute the IT power cords across all the phases. This
operating system license savings, and $1,500 in hardware can also help reduce excessive power cord length and ca-
maintenance costs. bling congestion in the rack, which can impede IT equip-
ment exhaust airflow, resulting in higher temperatures.

Page 6 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Exposing Hidden Risks Reliability, Availability and Mean Time to Repair

In addition to measuring relative energy efficiency, a Of course, the quality of the PDU is paramount to its
PDU with individually metered outlets can help detect reliability and therefore can affect its ability to deliver
hidden issues such as failed server power supplies (in power to the IT equipment. It is therefore important to
equipment with redundant PS) by setting a threshold look for established manufacturers with proven track
alarm if the power draw decreases to zero on any oc- records. Nonetheless, it is a fact that no matter how
cupied outlet. This can help avoid a scenario that may well built they are, like any other device, PDUs can
have gone undetected by data center personnel, which fail. While they can fail in several ways, some types
has happened many times and leaves the device in a of failures can affect the delivery of power to the IT
non-redundant, unprotected position. equipment; others might only impact the management
functionality. One of the more common modes of
failure is the communications card. In this scenario, the
Cord Retention
power will most likely still be flowing to the IT load so
Regardless of how redundant the upstream power chain
there is no interruption. However, some manufacturers
is, it will not matter if the IT equipment cord falls out of
incorporate the network card as an internal component
the PDU outlet. While this would seem like a simple issue
and to repair the PDU, it must be removed, impacting
not worth much discussion, it has been a long-standing
operational availability. Other manufacturers feature a
quandary, particularly with the imperfect retention of IEC
plug-in network module with all the other connectors
C13 receptacles and the C14 (in some cases also C19/
(temperature, cross linking, USB port, etc.), that can
C20) plug-on cord sets from some vendors. There have
be hot swapped, so the PDU itself does not have to be
been many instances where power cords have just fallen
removed from the rack and the IT equipment remains
out of the PDUs, either over time without any apparent
powered and fully operational.
reason or sometimes if a technician lightly brushes the
dangling cord, or just from the minor vibration of plug-
ging or unplugging an unrelated cord in the PDU. This that prevent many data center operators from raising
has led to several solutions, the oldest typically using a supply temperatures is ensuring that the cabinets that are
cord clip or a comb-like piece of plastic or wireframe cage “running hot” stay within the ASHRAE envelope (or even
that prevents the plugs from falling out. This approach is going into thermal shutdown)—even if it means overcool-
somewhat inconvenient and sometimes cords have still ing some or all of the other cabinets. This is especially
fallen out. Some of these systems also limited the num- true for those sites with insufficient airflow management
ber of receptacles. More recently, as the demand for out- and little or no rack-level temperature monitoring.
let density has increased, other solutions have developed.
By using rack-level intake temperature monitoring (via
Some users have purchased new IT cord sets that have PDU based sensors or other methods), airflow manage-
a locking grip on the plug that must be released. These ment issues can be identified, corrective changes can be
new locking cords can be used with existing PDUs, and made, and results can be immediately monitored and
are a good solution for older PDU models with a history tracked. Once the airflow management issues have been
of retention problems. However the need for a locking resolved or substantially mitigated, then the cooling
cord for each receptacle becomes a substantial added system temperatures can be slowly raised and the rack
cost factor, if they must also be purchased for new PDUs. temperature ranges closely monitored to see the results.
Some PDUs have new types of receptacles with a lever This allows energy savings while reducing the risk of IT
that can hold the plug in place, while other vendors offer equipment failure due to exposure to high temperatures.
high-retention force receptacles on the PDUs, eliminating One of the known issues and concerns about higher IT
the added cost of purchasing new locking cord sets. intake temperatures is that at a certain point IT fans will
speed up, resulting in higher power draw—in some cases
Cooling System Energy Saving negating the cooling system energy savings. While this
Cooling systems are the largest user of energy other improves the PUE, it may not lower the total facility ener-
than the IT load itself. And many studies have shown that gy, or could even increase it in extreme cases.
raising cooling supply air temperature can save a sub-
stantial amount of energy (ranging from 1-4 percent per By using rack-level PDUs with temperature sensors, you
degree Fahrenheit). However, one of the biggest issues will be able to determine the temperature correlation to

Page 7 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

any increased IT fan energy in real time (as well as trend it range for most IT equipment, so that even commodity
via software). You can also continuously analyze the data servers could operate at up to 95°F (35°C) intake tem-
to find and optimize temperature settings based on your peratures. These higher temperature ratings were meant
site conditions for maximum overall total energy efficien- to help improve facility cooling system energy, as well as
cy (not just facility PUE)—even under changing operating promote “free cooling” wherever possible.
conditions, as well as when IT equipment is refreshed
over time. However, these new operating ranges mean that some IT
equipment can create a temperature rise (delta-t) of up to
Environmental Monitoring 40°F (22°C), or higher in some cases. This could gener-
The expected reliability of IT equipment is in part ate exhaust temperatures in the back of the rack that can
related to maintaining proper environmental conditions reach or exceed 122°F (50°C), which could be created by
within the manufacturer’s requirements—which in most high-intake/Delta-T combination conditions. However,
cases fall within the ASHRAE “recommended” tem- older PDUs and some current models on the market are
perature and humidity envelope. However, as noted only rated for 113°F (45°C). Furthermore, 113°F (45°C)
above, it is now well documented that it is no longer conditions can be easily exceeded even with more mod-
simply a matter of monitoring “room” temperature. erate Delta-Ts, if higher “allowable” IT intake tempera-
Environmental monitoring at every rack has become tures are encountered (see A3-A4 below).
extremely critical as power densities have risen signifi-
cantly, especially in the last several years. According The 2011 ASHRAE whitepaper also included the introduc-
to ASHRAE’s Thermal Guidelines for Data Processing tion of A3 and A4 extended “allowable” IT intake tem-
3rd edition (2012), for high-density racks, just taking peratures of 104°F (40°C) and 113°F (45°C) respectively.
average readings in the middle of the cold aisle is no These new classes were added to encourage IT manufac-
longer sufficient. Each high-density rack should have turers to create future products with even greater energy
up to three temperature sensors inside the front face saving opportunities by reducing or even eliminating
of the rack (low, middle and high) to monitor the intake mechanical cooling. Today they are no longer considered
temperatures of the IT equipment and to ensure they future products, since some major IT manufacturers are
remain within the recommended range. currently offering specific A3 and A4 rated servers. Fur-
thermore, the increasing use of Solid State Disks (SSD),
which can operate at up 140°F (60°C), have begun to be
Clearly, there are many independent systems available utilized in place of traditional spinning disks in storage
to monitor aisle and rack-level temperature. While the arrays, allowing higher operating temperatures. In order
primary purpose of a rack PDU is power distribution, the to address these issues, some PDU manufacturers have
advent of the intelligent/remotely monitored PDU has en- introduced units that are rated up to 140°F (60°C). It is
abled adding environmental monitoring at the rack cost therefore important to consider PDUs with high tempera-
effectively, by adding small sensors as accessories that ture ratings for present and future purchasing criteria,
simply plug in to the PDU. The environmental information even if you are not currently experiencing very high back-
is then integrated into the rack PDU information, using of-rack temperatures.
the existing management console. This also saves the
substantial cost of any installation and additional cabling Total Cost of Ownership – PDU Budgeting
for separate environmental sensors and systems. Even if and Relative Costs
wireless environmental sensors are installed, they require There are several studies projecting the cost of building
multiple receivers, located to ensure coverage, and which and operating a data center from different organizations.
must be cabled and powered. While they vary, a commonly cited whitepaper by the
Uptime Institute estimated that costs range between
Higher PDU Temperature Requirements $12,500 (Tier II) to $25,000 (Tier IV) per kW for the power
In addition to requiring proper airflow, it should be noted and cooling infrastructure (plus $300 per square foot for
that the operational temperatures of high density IT the computer room space). In effect, the basic infrastruc-
equipment have increased as well. The 2011 ASHRAE’s ture cost to provide mission critical power and cooling for
Expanded Thermal Guidelines whitepaper (and the later a 10 kW rack can range from $125,000 to $250,000. In
formal release of the 3rd edition of the Thermal Guide- sharp contrast, a 10 kW rack PDU which is the last link in
lines) placed more emphasis on the wider A2 “allowable” the critical power chain, costs approximately one percent

Page 8 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Number of onboard PDU Breakers Number of outlet groups Rated IT power (kVA) Notes
(20 amp breakers per group)
3 Breakers (2 poles) 20 amps 3 12.5 Current draw limited to 35 amps due to
only 3 breakers
6 Breakers (2 poles) 20 amps 6 14.4 Current draw 40 amps
(80% of 50A branch circuit breaker)
TABLE 2A. Example of PDU capacity vs. PDU circuit breakers - for a 208/120 Volt - 50 amp 3-phase branch circuit.

Number of onboard PDU Breakers Number of outlet groups Rated IT power (kVA) Notes
(20 amp breakers per group)
3 Breakers (2 poles) 20 amps 3 25.0 Current draw limited to 35 amps due to
only 3 breakers
6 Breakers (2 poles) 20 amps 6 28.8 Current draw 40 amps
(80% of 50A branch circuit breaker)
TABLE 2B. Example of PDU capacity vs. PDU circuit breakers - for a 415/240 Volt - 50 amp 3-phase branch circuit.

or less of the invested facility cost (a PDU typically costs monitoring capabilities required for high-density racks.
from $500 for an unmanaged basic unit to $1,500 for a They should also be aware of the significance of the num-
fully managed unit). Yet, when it comes to selecting a ber of on-board circuit breakers (3 vs. 6) in 3-phase units or
PDU for a high-power rack, many times the lowest cost other features, such as metering per sub-breaker group or
drives the budgeting and purchasing process, with little individual outlet energy metering and control.
weight given to the importance of management or other
critical performance features. Outlet Density
The number of receptacles per PDU has increased to
Clearly there are many different manufacturers that offer meet rising demands for more servers and other devices
rack PDUs, each with different features and price points. per cabinet. However, matching the power capacity of the
Nonetheless, in addition to comparing the reputation of PDU (and capacity of the branch circuit) to the project-
manufacturer and the product warranty, the person making ed power requirements is critical. Example: a common
the decision may not be completely aware of the strategic high-density rack configuration consists of (40) 1U servers
management advantages of a monitored or fully managed @ 250 watts each (plus 1 kW for Top of Rack switches),
PDU. In some cases, the evaluator may not fully understand total power 11 kW.
some of the basic differences between seemingly compa-
rable, yet different models. They should consider various
PDU specifications, such as the higher temperature rating
and additional features such as integrated environmental

Phase Type Current Amps Power each Number of Number of *Total Power Total Number **Number of
(number of (US circuit / PDU (kVA) Outlets PDUs (kVA) of Outlets PDUs for A-B
CB Poles) 80% de-rated) Per PDU (Single Feed) Redundant
Single-phase
120V (1) 30/24 2.9 24 4 11.2 96 8
208V (2) 30/24 4.9 24-36 3 14.7 72-108 6
240V (1) 30/24 5.7 24-36 2 11.4 48-72 4

3-phase
120/208V 30/24 8.6 24-42 2 17.3 48-84 4
120/208V 60/48 17.3 24-42 1 17.3 24-42 2
240/415V 30/24 17.3 24-42 1 17.3 24-42 2
TABLE 3. Examples of several common PDU configurations to support an 11 kVA rack.

Page 9 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Phase Type Current Amps Power each Number of Number of *Total Power Total Number **Number of
(number of (US circuit / PDU (kVA) 20/16 Amp PDUs (kVA) of Outlets PDUs for A-B
CB Poles) 80% de-rated) receptacles (Single Feed) Redundant
(C19 or L6-20)
Single-phase
208V (2) 30/24 4.9 3-6 4 19.6 12-24 8
240V (1) 30/24 5.7 3-6 2 11.4 12-24 4

3-phase
120/208V 30/24 8.6 6-12 3 17.3 18-36 6
120/208V 60/48 17.3 6-12 2 34.6 12-24 4
240/415V 30/24 17.3 6-12 2 34.6 12-24 4
240/415V 60/48 34.6 12-24 1 34.6 12-24 2
Footnotes: *Total power based on US voltage and de-rated current capacity. Actual capacity may change due to European and Asian localized sys-
tem voltage standards (i.e. 380/220V - 400/230V - 415/240V), as well as codes that can impact branch circuit protection ratings.
** PDU count doubles for redundant A-B power.
TABLE 4. Example of PDUs to support 4 Blade Servers and other High Power IT equipment at approximately 20 kVA rack.

Blade Servers besides the need to provide power to each PDU, network
The other common high-density configuration is for (4) communication is also required. This usually means a port
Blade Servers (at approx. 4-5 kVA each). Typically each on an Ethernet switch (either on the production network
requires a minimum of 2 or 3 power supplies (each PS ne- or a dedicated management network). This can become
cessitating a 208-240V 20/16 Amp receptacle) and double a costly aspect of managed PDUs, and some vendors
the number of power supplies for A-B redundancy. have the ability to interconnect multiple PDUs (typically
Note that the tables above represent a general summary up to 4 units) so that only one network port is required.
of common types of PDUs (0U vertical) in each category Moreover, they can be logically clustered together to be
range, from a variety of manufacturers. Many other mod- viewed as a managed group in each rack. Some manufac-
els exist, as well as custom designed units from various turers even provide cross powering to the communication
vendors. In addition, some vendors also offer PDUs with cards (creating redundant power configurations), so that
higher ratings to support racks above 35 kVA. should either side lose input power, the communications
card can still communicate and send out an alert.
As can be seen from the above tables, 3-phase power
offers the greatest power capacity and therefore minimizes New Builds vs Retrofit Upgrades
the number of PDUs for a given load or application. It also When designing a new facility, the type of power distribution
minimizes the total number of cables and circuit breaker system is critical to its ability to adapt to constantly changing
(CB) poles in the upstream power distribution panel. From IT equipment power requirements. Of course the basis of
a physical perspective, most racks can only accommodate design requirements will need to be ascertained early on,
four vertical PDUs, and it is preferable to only use two in order to specify the critical power capacity, and therefore
PDUs when possible to allow space for network cabling. the capacity of the UPS systems. Thereafter, the downstream
distribution voltage should be discussed. In Europe this is
While a detailed discussion of cooling these high-density racks essentially predetermined at 400/230 volts generically (al-
is beyond the scope of this whitepaper, it should be noted though it will vary by country from 380/220 V to 415/240V).
that it is important to ensure that the size and placement of In North America, the most common configuration for larger
the PDUs and the related server power cords, as well as the facilities uses a 480 volt UPS system, which feeds floor-level
network cabling, does not block the exhaust airflow. PDUs with stepdown transformers that then provide 208/120
volt power for distribution to the IT cabinets.
Clustering and Grouping
As can be seen in tables 3 and 4, multiple PDUs in high- While a detailed discussion of cooling these high-density
er-density racks are a common configuration. In addition, racks is beyond the scope of this whitepaper, it should

Page 10 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

Retrofit Upgrade Challenges Multi-Vendor Compatibility

One of the potential bottlenecks when trying to up- Most vendors have their own management software
grade the power to a cabinet (especially to group of that is optimized for their PDUs, some with proprietary
cabinets) can occur at the upstream power distribution features unique to their PDUs. Many vendors have
panel. While an electrical survey of the input to the also made their PDUs accessible via standards-based
panel may show that there is current capacity still avail- SNMP, and many software packages can communi-
able, all or nearly all the branch circuit breaker posi- cate with most other brands of PDUs (although some
tions may be full. This is very common in older facilities proprietary features may not be fully addressable). This
that used 120V branch circuits to feed the cabinets. As is done out of necessity in order to be competitive and
can be seen in tables 3 and 4, it would be wise to con- functional in the multi-vendor environment common
sider installing higher power 3-phase cabling and PDUs in most data centers. When reviewing the features
to maximize the capacity of existing distribution panels. of the PDUs and the vendor’s own matching (or even
Also, where possible, consider locating the new power competing) software, a review of compatibly is critically
cabling overhead, to improve cooling airflow, especially important.
if the under-floor space is already filled with cabling.
of IT vs. Facilities. Originally, most DCIM products were
be noted that it is important to ensure that the size and fairly basic offshoots of Building Management Systems
placement of the PDUs and the related server power (BMS) and were primarily focused on the facility side,
cords, as well as the network cabling, does not block the with a “PUE” dashboard added on. They have now gone
exhaust airflow. through several generations, and in their current iteration
have become far more sophisticated tool sets.
Some manufacturers have been offering 415/240V power
distribution systems in the US marketplace. This has some DCIM may have started primarily as a BMS/BIS facili-
advantages, especially for a new data center, since it can ty-type platform, with data center-specific features (such
deliver double the power (415/240V vs. 208/120V volts) as a PUE dashboard); however, the nature of entire data
over the same-sized branch circuit cabling, saving initial center-centric computing infrastructure and enterprise
costs and reducing the physical cabling and pathway size. IT architecture itself has changed radically since DCIM’s
This also means that the PDUs can deliver twice the power inception. While the topic of DCIM is a hotly debated
as well, thereby reducing the number of PDUs required for industry subject, a detailed analysis is beyond the scope
high-density cabinets, reducing costs substantially. This also of this whitepaper. Nonetheless, there is a real need for a
saves space in the back of the cabinets (improving airflow). system that can collect and aggregate information from
In addition, the IT equipment power supplies are more facilities and IT systems and deliver a cohesive picture
energy efficient at the higher input voltage, saving energy in a manner that is meaningful and correlates to some
and reducing cooling requirements. In some data centers, actionable items for everyone. However, not every orga-
480/277 volt power distribution to the racks has been used; nization is ready to fully commit to a major investment in
however, this is not yet a commonly deployed scenario. DCIM. Alternately there are some management software
offerings from the PDU manufacturers that can effectively
Management Software meeting your basic management requirements, with a
Of course all the sophisticated features and benefits of much lower initial investment.
this latest generation of smart, remotely managed PDUs
are minimized, without centralized software to track, re- The Bottom Line
cord, analyze and display the information from hundreds The new generation of smart, remotely manageable PDUs
or thousands of PDUs, as well as other information. has offered important management tools to many large
enterprise organizations. However, the costs of owning
Beyond just managing PDUs, an entire new class of and operating data centers have changed the landscape
management software has come into the mission critical for both small and large organizations, giving rise to
realm: Data Center Infrastructure Management (DCIM). outsourcing in several forms. The days of the traditional
The premise behind DCIM is the proverbial “single pane enterprise data centers are diminishing as more organiza-
of glass,” which promises to alleviate the challenges re- tions are moving toward outsourcing data center owner-
lated to the traditionally cultural and management issues ship and operations by moving to colocation providers

Page 11 | Mission Critical Rack Power Distribution Whitepaper

Rack Power Distribution

and cloud services. Colocation in particular has exploded, The PDU may be the last part of the power chain; however,
and has increased the need for better remote visibility like any link in a chain it is just as critical as any other com-
and management at the rack level, since the colocation ponent. From maintaining operational status and mini-
facilities may be situated far away from the IT team. mizing human error, the smart PDU is a key component in
providing visibility within the rack. This granular information
From a strategic management perspective, the days of also allows more accurate management decisions to be
the basic “power strip” are numbered. The need to lower made, such as capacity planning. It can also be a flexible
TCO by C-level management requires real-time accurate and cost effective platform to add environmental monitor-
data that can be distilled into actionable information. The ing at the rack level, which is essential in today’s high-den-
added features of the latest generation of smart PDUs sity data center. Moreover, the minor additional cost to
and management software can provide that insight. One include the remote monitoring or management function
of the ultimate goals of optimizing a data center’s overall into a basic PDU is a fairly small incremental investment, es-
efficiency is to be able to measure various applications’ pecially compared to the total cost of the infrastructure of
utilization rates (and throughput) and correlate that to the entire facility, as well as the cost of the IT equipment it
their energy consumption. That information can then be supports. The direct benefits it provides, such as mitigating
used by IT architects to make more intelligent, holistic the risk of needless overload-related outages and cas-
decisions about IT performance and energy efficiency and cade failures (caused by circuit breaker trips), even in fully
related TCO. This data can then also be used to compare redundant power chains, clearly makes it a very worthwhile
it with cloud-based services and to make capacity plan- investment, if not a mandate.
ning, as well as short- and long-term strategic and finan-
cial decisions that can more than justify the ROI question.

About Julius Neudorfer About Server Technology

Julius Neudorfer is the CTO and founder of North Amer- For over 30 years, Server Technology has been recognized
ican Access Technologies, Inc. (NAAT). Based in West- as the global leader of innovative, intelligent power distri-
chester NY, NAAT’s clients include Fortune 500 firms and bution products and solutions for data centers, telecom-
government agencies. Julius has been designing and munications operations and branch offices. Over 60,000
implementing Data Center Infrastructure and related tech- customers around the world rely on Server Technology’s
nology projects for the last 25 years. He developed and cabinet power distribution units (cabinet-level PDUs) and
holds a US patent for a high efficiency cooling system for power management solutions to help reduce downtime,
rack mounted computer equipment. facilitate capacity planning, improve energy utilization and
drive efficiency.
Julius is a member of AFCOM, ASHRAE, BICSI, IEEE and
The Green Grid. He is an instructor for the US Department With the best quality and longest mean time to failure
of Energy “Data Center Energy Practitioner” “DCEP” (MTTF), Server Technology products provide uncompromis-
program. He writes the “Hot Aisle Insight” column, and ing reliability and value for the datacenter. As a leader in
has also written numerous articles and whitepapers for product innovation, Server Technology holds many of the
various IT and Data Center publications and has presented key patents for providing power in zero-U and horizontal
seminars and webinars on data center power, cooling and PDU configurations compatible with today’s racks and dense
energy efficiency. compute infrastructure requirements.

References:
ASHRAE: [Link] and [Link]
AFCOM: [Link] [Link]
[Link]
The Green Grid: PUE Power Usage Effectiveness [Link]

[Link]
[Link]
Examination%20of%20the%20Metric_v6.pdf?lang=en
[Link]
[Link]
SPEC Power: [Link]
US Energy Star Program: [Link] [Link]
servers_specification_version_2_0_pd
Uptime Institute: www. [Link] Whitepaper “Cost Model: Dollars per kW plus
Dollars per Square Foot of Computer Floor” - By W. Pitt Turner IV, P.E. with Kenneth G. Brill – 2008
[Link]
Page 12 | Mission Critical Rack Power Distribution Whitepaper