Distributed and Cloud Computing

K. Hwang, G. Fox and J. Dongarra

Chapter 4: Cloud Platform Architecture

over Virtualized Datacenters
(suggested for use in 5 lectures in 250 minutes)

Prepared by Kai Hwang

University of Southern California
April 3, 2012

Copyright © 2012, Elsevier Inc. All rights reserved. 4-1

The Architecture of a Small
Server Cluster ( ~ 1000 servers)

interconnected by an Ethernet switch and housed in a

warehouse or in a container environment

(Courtesy of Luiz Barroso and Urs Holzle, Google Inc., 2009)

Copyright © 2012, Elsevier Inc. All rights reserved. 4-2

Warehouse-Scale Computer (WSC)
 Provides Internet services
 Search, social networking, online maps, video sharing,
online shopping, email, cloud computing, etc.
 Differences with HPC “clusters”:
 Clusters have higher performance processors and
network
 Clusters emphasize thread-level parallelism, WSCs
emphasize request-level parallelism
 Differences with datacenters:
 Datacenters consolidate different machines and software
into one location
 Datacenters emphasize virtual machines and hardware
heterogeneity in order to serve varied customers
(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4-3

Design Considerations for WSC:
 Cost-performance
 Small savings add up
 Energy efficiency
 Affects power distribution and cooling
 Work per joule
 Dependability via redundancy
 Network I/O
 Interactive and batch processing workloads
 Ample computational parallelism is not important
 Most jobs are totally independent
 “Request-level parallelism”
 Operational costs count
 Power consumption is a primary constraint when designing system
 Scale and its opportunities and problems
 Can afford customized systems since WSC require volume purchase

(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4-4

Typical Datacenter Layout

Copyright © 2012, Elsevier Inc. All rights reserved. 4-5

Power Consumption in Servers

Copyright © 2012, Elsevier Inc. All rights reserved. 4-6

 (Courtesy of Luiz Andre Barroso and Urs Holzle, Google Inc., 2009)

Copyright © 2012, Elsevier Inc. All rights reserved. 4-7

Copyright © 2012, Elsevier Inc. All rights reserved. 4-8
 Architecture of WSC
 WSC often use a hierarchy of networks for
interconnection
 Each 19” rack holds 48 1U servers connected
to a rack switch
 Rack switches are uplinked to switch higher in the
hierarchy
 Uplink has 48 / n times lower bandwidth, where n =
# of uplink ports
 Goal is to maximize locality of communication
relative to the rack
(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4-9

 (Courtesy of Luiz Andre Barroso and Urs Holzle, Google Inc., 2009)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 10

Storage and Array Switch
 Storage options:
 Use disks inside the servers, or
 Network attached storage through Infiniband
 WSCs generally rely on local disks
 Google File System (GFS) uses local disks and maintains at
least three relicas
 Switch that connects an array of racks
 Array switch should have 10 X the bisection bandwidth of
rack switch
 Cost of n-port switch grows as n2
 Often utilize content addressible memory chips and FPGAs

(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 11

WSC Memory Hierarchy
 Servers can access DRAM and disks on other
servers using a NUMA-style interface

(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 12

Interconnection Networks

Figure 4.12 BCube: A High Performance, server-centric network for

modular datacenters. (Courtesy of C. Guo, et al, ACM SIGCOMM
Computer Communication Review, Oct. 2009. [25]).

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 13

Figure 4.13 A 2-D MDCube is constructed from 9 BCube containers.
(Courtesy of . Wu, et al, ACM CoNEXT’09, Dec. 2009, [77]).
Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 14
 Power and Cooling Requirements
 Cooling system also uses water (evaporation and spills)
 E.g. 70,000 to 200,000 gallons per day for an 8 MW facility

 Power cost breakdown:

 Chillers: 30-50% of the power used by the IT equipment
 Air conditioning: 10-20% of the IT power, mostly due to fans

 How many servers can a WSC support?

 Each server:
 “Nameplate power rating” gives maximum power consumption
 To get actual, measure power under actual workloads
 Oversubscribe cumulative server power by 40%, but monitor
power closely
(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 15

Measuring Efficiency of a WSC
 Power Utilization Effectiveness (PEU)
 = Total facility power / IT equipment power
 Median PUE on 2006 study was 1.69
 Performance
 Latency is important metric because it is seen
by users
 Bing study: users will use search less as
response time increases
 Service Level Objectives (SLOs)/Service Level
Agreements (SLAs)
 E.g. 99% of requests be below 100 ms

(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 16

 Larger Datacenter Growth
 One at a time:
 1 system
 Racking & networking: 14 hrs ($1,330)
 Rack at a time:
 ~ 40 systems
 Install & networking: .75 hrs ($60)
 Container at a time:
 ~1,000 systems
 No packaging to remove
 No floor space required
 Power, network, & cooling only
 Weatherproof & easy to transport
 Datacenter construction takes 24+ months
 Both new build & DC expansion require
regulatory approval

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 17

Cloud Computing
 WSCs offer economies of scale that
cannot be achieved with a datacenter:
 5.7 times reduction in storage costs
 7.1 times reduction in administrative costs
 7.3 times reduction in networking costs
 This has given rise to cloud services such as
Amazon Web Services
 “Utility Computing”
 Based on using open source virtual machine and
operating system software
(Courtesy of Hennessy and Patterson, 2012)

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 18

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 19
Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 20
Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 21
Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 22
Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 23
Cloud Services and Major Providers

Copyright © 2012, Elsevier Inc. All rights reserved. 4 - 24