R
Previous
NOVEMBER 2012
[Link]
Next
Previous
Next
Previous
Next
Previous
Next
Download
The Virtual Network
From SDN to network overlays, emerging technologies promise to reshape the data center for the age of virtualization >> By Ethan Banks
Subscribe
PLUS Facebooks Open Compute project rethinks data centers >>
Previous
Next
MORE NETWORK COMPUTING
November 2012 Issue 13
CONTENTS
Let The News Find You Keep up with all of the latest news and analysis on the fastmoving IT industry with the Network Computing newsletter. [Link]/newsletter
IN-DEPTH REPORTS
Data Center Management As virtualization, automation, and private clouds reshape the data center, many tools are necessary to manage hardware and software efficiently. We invited IT pros to rate products in two areas: overall performance and data center management features. [Link]/reports/dcm IaaS Buyers Guide Infrastructure-as-a-service lets companies focus on their core competencies, instead of on installing and maintaining computer hardware. But with so many vendors in the market, how do you know which one is the best fit? [Link]/reports/iaas
COVER STORY
PRACTICAL ANALYSIS
PREVIOUS ISSUE
Storage Virtualization Guide From thin provisioning to replication to federation, virtualization options let you reclaim idle disks, speed recovery, and avoid lock-in. [Link]/issues/may2012
November 2012 2
The Virtual Network
A host of emerging technologies aims to improve support for virtualization and streamline operations p.4
Meet The New Server
The Open Compute Project rethinks server design for the data center with its Open Rack design p.3
CONTACTS
Editorial and Business Contacts p.11
[Link]
Previous
Next
Table of Contents
PR AC TIC AL AN A LYSIS
Open Compute Rethinks Server Design
Blade servers have proven to be an effective means of packing a lot of computing power into a relatively small space, but they arent the last word in dense server packaging. One competing, and compelling, option is the Facebook-instigated Open Compute Project and its Open Rack design. The goals of the design are simple: Create an open standard for a flexible, modular rack platform that can hold everything from 1U and half-U servers to storage arrays, power supplies, power distribution units, and cabling. Facebook started the project to maximize space utilization and serviceability for systems in its new Prineville, Ore., data center. It decided to make its plans public, in part to create an OEM ecosystem for servers and other components. And the plan appears to be succeeding: Executives from Intel, Arista, and Rackspace sit on the Open Compute board, and motherboard designs from Intel and Advanced Micro Devices are in the works. But the projects Open Rack spec goes far beyond just shoehorning naked servers (that is, motherboards with minimal packaging) into a standard 19-inch rack; it ditches the existing form factor entirely. By creating a new, 21-inch-wide rack specification and designing system boards, modular cable management, and other electrical and mechanical interfaces around whole-rack implementations, the Open Compute specs can achieve high density and make for easy system replacement. For example, second-generation motherboard specs call for two dual-socket, hot-swappable systems with 16 DIMM slots in 1U or 1.5U form factors. With each rack divided into three 13U power zones, each made up of a 10U equipment module sharing a 3U power supply, this yields 20 diskless, hot-swappable servers, or 12 1.5U systems with six drives shared between two servers, in each rack zone, with room left over for a 2U top-of-rack switch. Density aside, the projects real value is in decoupling and modularizing server components that have very different useful life spans. In presentations at last springs third Open Compute Summit, project leaders said that while racks and cables easily last 10 or more years and power supplies six, CPUs and disks
Kurt Marko
are often obsolete or dead in four or five years. By making it easier to connect and swap out infrastructure pieces, such as fans and power supplies, the project aims to maximize the utility of each component in the system and obviate the need for forklift upgrades just to update a server motherboard and CPU. So far, there are few commercially available whole-rack systems. Although VCE sells Vblock systems integrating Cisco UCS blades and switches with EMC VNX storage, and HewlettPackard has its CloudSystem Matrix, these resemble highly integrated mainframes for the virtualization era more than they do modular racks. IBM is moving closer to the Open Compute model with its Flex System, a rack-based product, which comes in either 10U or 42U sizes. If the Open Compute Project can do for hardware what Linux and other open source projects did for software, the day may come when swapping server and switch modules is as easy as replacing a failed hard drive. Kurt Marko is an IT pro with broad experience, from chip design to IT systems. Write to us at comments@[Link].
November 2012 3
Eye On Cisco
Our recent Cisco Outlook Survey of nearly 600 IT pros offers plenty of reasons for the network equipment provider to be satisfied, but some kvetching by a vocal minority shows the company cant get complacent.
Download
[Link]
Previous
Next
COVER STORY
Table of Contents
The Virtual Network
From SDN to network overlays, emerging technologies promise to reshape the data center for the age of virtualization
Just as virtualization upended the server market, now its the networks turn. In the past, network designers built topologies in which traffic traveled in a north-south orientation. Thats an adequate design for client-facing traffic and workloads that dont move. Virtualization changes all that. Virtual
[Link]
By Ethan B ank s
machines now talk to other VMs in other racks and rows in an east-west fashion. And VMs can move to unpredictable physical data center locations, so a designer cant know where a workload is at any given time. Building tidy network islands with neatly parti-
tioned IP schemes where every server talks mostly to its neighbor is no longer an option. At the same time, companies are deploying powerful servers that saturate high-bandwidth Ethernet links with storage, database, and application traffic. The new network demands that a host can show up anytime, anywhere
November 2012 4
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
in the data center and have a predictable level of bandwidth and latency. To make the demands of virtualization viable, networking vendors and standards organizations have been aggressively releasing new products and technologies. This article will look at two emerging protocols, examine why OpenFlow is so closely tied to software-defined networking, and consider APIs as an alternative way to embrace SDN. Two Emerging Techniques: VXLAN And NVGRE Virtualization administrators are sometimes frustrated in their designs, as they wish for the freedom to move a virtual machine wherever and whenever they want without requiring the manual intervention of the networking team to facilitate the move. Network designers have built confined bridging domains as a matter of habit and historical best practice. Server virtualization folks have therefore sought ways that Layer 2 domains can be extended across a partitioned data center. This approach is referred to as a network overlay. Several new protocols exist that make this possible, including VXLAN and NVGRE. The two protocols are similar, but they have different vendors backing them and different ideal use cases. VXLAN offers an overlay tunneling scheme that lets VLANS scale beyond the typical limit of 4,000 VLANs up to 16 million VXLANs, which is useful for large multitenant environments. It also allows for abstrac-
Get This And All Our Reports
Our full report on the virtual network is free with registration. This report includes 20 pages of analysis of emerging network technologies. What youll find: > Overviews of TRILL, SPB, VXLAN and other protocols > Insight into SDNs potential to transform the network
Download
[Link]
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
tion of duplicate MACs and VLAN schemes, virtual server at-will migration to diverse locations, and equal-cost multipath (ECMP) load balancing across the existing Layer 3 infrastructure. To accomplish this, VXLAN encapsulates MAC frames inside of a VXLAN packet that uses UDP with random source ports as a transport and a 24-bit VXLAN tag as an identifier. The VXLAN tag uniquely identifies each LAN segment. The protocol accomplishes broadcast, multicast, and unknown unicast flooding across the VXLAN domain by adding that traffic to a multicast group specific to each VXLAN segment. Cisco, Arista Networks, VMware, and others back the Internet Engineering Task Force draft protocol. VXLAN is implemented in soft switches only so far, and theres more adoption by the virtualization folks than the networking folks. Therefore, if youre considering a VXLAN implementation, youre probably a large shop with unique application demands that youre trying to address. Youre also on the cutting edge, as the IETF draft status for VXLAN is currently listed as experimental. NVGRE tackles the same problem as VXLAN with an IETF draft backed by Microsoft, Arista Networks, Intel, and others. NVGRE uses the well-known Generic Routing Encapsulation
[Link]
SDN Benefits
Which of these data center LAN challenges do you believe SDN can be most helpful in overcoming?
Improve network utilization and efficiency
42%
Automate more provisioning and management
35%
Improve security
32%
Implement network-wide policies
31%
Reduce cost
29%
Get more visibility into applications that are using the network
25%
Data: InformationWeek 2012 Software-Defined Networking Survey of 116 business technology professionals at companies with, or planning to have, SDN in production, July 2012
protocol to encapsulate a MAC frame and create the overlay tunnel between endpoints, and sneaks in a 24-bit identifier (similar to the 24-bit VXLAN identifier) by using the GRE headers key field. Running ECMP with GRE packets is a bit trickier to accomplish when compared with VXLAN, but NVGRE proposes to use various IP addresses as sources and destinations to work around GRE not having source or destination ports on which to calculate a hash value. NVGRE bears watching as hypervisor vendors implement it, but at this point, its diffi-
cult to identify a unique value proposition that makes NVGRE a superior technology. Like VXLAN, NVGRE isnt standardized yet; the IETF status is informational. The designer evaluating VXLAN and NVGRE must know this: These are fledgling protocols, and there are no approved standards. While these are interesting way to resolve Layer 2 extension challenges, and do so largely within the hypervisor and associated soft switch, you will be an early adopter. One key player to watch is VMware, which has the majority of the server virtualization
November 2012 6
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
market. Whichever technology VMware backs (VXLAN at present, though the Nicira acquisition may affect that) starts with a leg up simply due to VMwares market share. Another key consideration of deploying an overlay scheme is the underlying physical network. While tunnels connecting hosts from soft switch to soft switch seemingly diminish the virtualization teams connectivity challenges, an overlay still must ride atop a physical network topology. Physical network design remains critical because the traditional concerns of capacity and resiliency dont disappear just because network switches are abstracted by a software layer. Software-Defined Networking Takes Hold Conventional networks function as a collection of switches, each with its own control plane that interacts with the other control planes in the area. In this way, each switch learns how to forward traffic by sharing the
information it knows with everyone else. To facilitate this, protocols such as OSPF, BGP, and Spanning Tree are used to determine network topologies and exchange state. The point to keep in mind here is that every switch functions autonomously. The central idea of software-defined networking is that the network can be programmed by a controller and managed by an operator as a holistic entity. The SDN vision is to program a whole network, with the individual devices abstracted so administrators dont manage individual devices. The key to an SDN is the controller: the all-seeing, allknowing device that instructs the individual switches how to forward traffic. The switches no longer program their own forwarding tables based on the information gleaned from a routing protocol or bridging mechanism, but instead receive their instructions from the controller. An SDN controller itself is useless without an
application running on it. The SDN application parses the networks physical topology, learns the location of endpoints, reacts to topology changes, discerns paths through the network, and determines how specific traffic types should be forwarded based on a policy programmed by an operator. SDNs are very different from what network vendors have been offering for many years. Vendors accustomed to autonomous devices will be increasingly pressured by buyers to offer more open ways that their switch hardware can be programmed. IT professionals have caught on to the concepts behind SDN. According to the InformationWeek 2012 Software-Defined Networking Survey, 35% of respondents say they understand the basics of SDN, and another 17% say they understand the details. Of those respondents that have or plan to have SDN in production on their networks, 66% say the greatest selling point for SDN is
[Link]
November 2012 7
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
that it will create a more efficient and flexible network that will speed service delivery. OpenFlow Leads SDN Movement Originally conceived as a low-risk way to run an experimental network on the same hardware as a production network, the OpenFlow protocol has emerged from academia to be a cornerstone in some vendors SDN strategies. OpenFlow is used between a controller and a switch to inform the controller of traffic flows, and the switch how to forward those flows. The OpenFlow specification (1.2 is current
as of this writing, with 1.3 going through ratification) is governed by the Open Networking Foundation. The foundation is mostly funded by cloud service providers seeking low-cost, highly flexible ways of running their data center networks. Deutsche Telekom, Facebook, Google, Microsoft, NTT Communications, Verizon, and Yahoo make up the board. Numerous networking hardware and software vendors, including Alcatel-Lucent, Brocade, Cisco, Dell, F5, HP, Juniper, NEC, Plexxi, and VMware, are also members. The Open Networking Foundation hasnt released OpenFlow as an
To What Extent Are You Using These Virtualization Technologies?
Extensive use Limited use Evaluating No use or no plans for use
Server virtualization
72%
Storage virtualization
15% 23% 28% 28% 31% 23% 23% 21% 45% 44% 21%
9%
4%
22% 19%
Desktop virtualization
27% 22% 40% 20% 24%
Application virtualization (e.g., ThinApp, XenApp, App-V)
16%
I/O virtualization
12% 11%
Network virtualization (e.g., SDN, OpenFlow, Cisco Nexus, Nicira, NextIO, HP)
Data: InformationWeek 2013 Virtualization Management Survey of 320 business technology professionals at companies with 50 or more employees, September 2012
[Link]
open source specification. Instead, the ONF lets members license OpenFlow for use in their products. The foundation charges a $30,000 annual fee. OpenFlow paves the way for inexpensive, commoditized Ethernet switches with flexible forwarding functionality. That will be interesting for some highly specialized data center environments; cloud providers such as Google favor this approach because it reduces the cost of running the network. Using OpenFlow requires creating and maintaining a custom application that would drive an OpenFlow controller, so only very large enterprise IT shops will likely have the needed talent on staff. One vendor closely tied to OpenFlow is Big Switch Networks, a startup founded in 2010. It has released an open source OpenFlow controller called Floodlight, as well as an OpenFlow standards compliance testing tool called OFTest. NEC sells its ProgrammableFlow package based on OpenFlow that includes a management console, a controller, and a couple of different switches from which to choose. Pica8 sells a line of inexpensive OpenFlow-capable switches. Pica8 switches run an open source operating system called XorPlus that includes not only OpenFlow capability but also a set of
November 2012 8
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
protocols that most network environments use as their foundation. Therefore, a Pica8 switch can be used with or without OpenFlow. However, Pica8s main appeal is that OpenFlow provides a low-cost hardware switch that can be customized by shops that want to develop their own networking applications. Major vendors such as Juniper and HewlettPackard also have ties to OpenFlow. Most notably, HP has added OpenFlow support to a broad array of its switches. The company is well-positioned to offer enterprises an endto-end SDN option based on OpenFlow, including a controller and software interface, should HP choose to come to market with such products. However, none of the major vendors has embraced OpenFlow as a long-term, committed strategy that would eventually displace its current approach to networking. OpenFlow represents a shift away from a networking vendors traditional revenue streamthat of selling specialized silicon at high marginsand as such, established vendors tend to downplay OpenFlow in their SDN marketing messages. APIs: An OpenFlow Alternative As part of their SDN strategy, some networking vendors are exposing the functions
[Link]
To Rebuild Or Not To Rebuild
Have you rearchitected your data center network, or are you considering it?
Weve implemented a new architecture within the last 12 months
Were rolling out a new architecture into production
14%
15%
Were testing a new architecture
9%
We have no plans at this time to change our network architecture
29% 20%
13%
Were planning a new architecture
We believe well need to rearchitect within the next 24 months
Data: InformationWeek 2012 Data Center Networking Vendor Evaluation Survey of 501 business technology professionals at companies using data center networking products, November 2011
of their operating systems with application programming interfaces. The average network R engineer today programs a network device via a command line interface, Java tool, or Web interface; the problem is that these tools are vendor-provided shells that arent extensible. With an API, engineers can write custom applications that can program a network device using any available interface that the programmer favors. In addition, they could create rich applications to manipulate a network environment holistically. While reminiscent of
OpenFlow, the API approach promises to expose more network device functionality than simply the programming of flow tables. Vendors favoring the API approach tout broad programmability as a selling point when compared to the OpenFlow approach. A common use for a network device API is to let a tool like VMwares vCenter program network switches with appropriate port and VLAN settings. It does so in conjunction with a larger task like provisioning a new virtual machine. More network vendors are offering integration with vCenter, since it helps facilitate orchestraNovember 2012 9
Previous
Next
THE VIRTUAL NETWORK
COVER STORY
Table of Contents
tion and automation in VMware-centric shops. Juniper uses its XML API so extensively that even its Junos command-line shell generates XML behind the scenes to program Juniper hardware. If OpenFlow is critical to you, the Junos XML API facilitates the programming of a Juniper device using an OpenFlow controller via a bolt-on package from the Junos software development kit. F5 Networks has a well-documented API called iControl actively supported by its DevCentral community. While not in the network switching market directly, F5 application delivery controllers are well-positioned to integrate with SDN networks because its API can be leveraged to abstract the physical application delivery controller. Cisco in June announced several libraries of APIs under the banner Open Network Environment. ONE APIs will be offered in several Cisco operating systems, including IOS, IOS XR, and NX-OS, letting customers develop their own customized network applications that could reside on network devices or run remotely. Finally, RFC4741 defines a networking standard called NETCONF, an XML interface to program network devices using the Secure Shell protocol over TCP port 830. Other RFCs describe alternate transports.
[Link]
Whats Holding You Back From Adopting SDN In The Next Two Years?
Immaturity of current products
41%
Lack of resources to evaluate SDN
23%
Concern that the technology will not scale to support enterprise-class networks
22%
Worry that cost to implement will exceed ROI
18%
We dont see a compelling value proposition
18%
Lack of a critical mass of organizations that have deployed SDN
14%
Data: InformationWeek 2012 Software-Defined Networking Survey of 250 business technology professionals, July 2012
APIs show that a network doesnt have to use OpenFlow to deliver the holistic management and physical device abstraction promised by SDN, nor does a switch have to run OpenFlow to be programmed by a remote controller. The use of APIs also lets vendors counteract one notion behind the SDN movement namely, that switches and other network gear would become commodity devices (and thus much cheaper). Network designers face tough choices today as virtualization creates new problems but the technology to solve them is still emerging. Your virtualization team wants the network to be
flexible and easily provisioned, with little human intervention. The marketplace is flooded with new ideas and innovation promising to provide that flexibility and automation, but the choices are neither easy nor obvious. While the right strategy will vary widely based on an environments existing infrastructure, established vendor relationships, and aversion to risk, the wrong answer most assuredly is one where network specialists dig in their heels and refuse to change. Ethan Banks is a network engineer and host of the Packet Pushers podcast. Write to us at comments@[Link].
November 2012 10
Previous
Next
Table of Contents
Network
For IT, By IT
Andrew Conry-Murray Editor acmurray@[Link] 724-266-1310
Rob Preston VP and Editor In Chief rpreston@[Link] 516-562-5692 Lorna Garey Content Director, Reports lgarey@[Link] 978-694-1681 Sek Leung Associate Art Director sleung@[Link] Chris Murphy Editor cjmurphy@[Link] 414-906-5331 Jim Donahue Chief Copy Editor jdonahue@[Link] Stacey Peterson Executive Editor, Quality speterson@[Link] 516-562-5933 Mary Ellen Forte Senior Art Director mforte@[Link]
READER SERVICES
[Link] For IT, By IT Electronic Newsletter Subscribe to our newsletter at [Link]/newsletter Events Get the latest on our live events and Net events at [Link]/events Reports [Link] for original research and strategic advice How to Contact Us [Link]/[Link]
Business Contacts
Chief Sales Officer, TechWeb Media, Martha Schwartz (212) 600-3015, mschwartz@[Link] Sales Assistant, Salvatore Silletti (212) 600-3327, ssilletti@[Link] District Manager, Cori Gordon (516) 562-5181, cgordon@[Link] Inside Sales Manager East, Ray Capitelli (212) 600-3045, rcapitelli@[Link]
TECHWEB
Ed Grossman President, TechWeb Media Martha Schwartz Chief Sales Officer, TechWeb Media David Berlind Chief Content Officer, TechWeb Media Joseph Braue Exec. VP, Light Reading Communications Network Fritz Nelson Sr. VP, Editorial Director, InformationWeek Business Technology Network John Ecke VP of Brand and Product Development, InformationWeek Business Technology Network
Editorial Calendar [Link]/edcal Back Issues E-mail: customerservice@[Link] Phone: 888-664-3332 (U.S.) 847-763-9588 (Outside U.S.) Reprints Wrights Media, 1-877-652-5295 Web: [Link]/reprints/?magid=2196 E-mail: ubmreprints@[Link] List Rentals Specialists Marketing Services Inc. E-mail: PeterCan@[Link] Phone: (631) 787-3008 x3020 Media Kits and Advertising Contacts [Link]/contact-us Letters to the Editor E-mail comments@[Link]. Include name, title, company, city, and daytime phone number. Subscriptions Web: [Link]/magazine E-mail: customerservice@[Link] Phone: 888-664-3332 (U.S.) 847-763-9588 (Outside U.S.)
SALES CONTACTSWEST
Western U.S. (Pacific and Mountain states) and Western Canada (British Columbia, Alberta) Western Regional Sales Director, Kevin Bennett (415) 947-6139, kbennett@[Link] Strategic Account Director, Coretta Wright (415) 947-6245, cwright@[Link] District Manager, Jeremy Cotton (415) 947-6237, kbennett@[Link] Account Manager, Ashley Cohen (415) 947-6349, aicohen@[Link]
Strategic Accounts
District Manager, Mary Hyland (516) 562-5120, mhyland@[Link] Account Manager, Tara Bradeen (212) 600-3387, tbradeen@[Link]
SALES CONTACTSMARKETING AS A SERVICE
Director of Client Marketing Strategy, Jonathan Vlock (212) 600-3019, jvlock@[Link] Director of Client Marketing Strategy, Julie Supinski (415) 947-6887, jsupinski@[Link]
UBM LLC
Pat Nohilly Sr. VP, Strategic Development and Business Admin. Marie Myers Sr. VP, Manufacturing
Strategic Accounts
Account Director, Sandra Kupiec (415) 947-6922, skupiec@[Link]
SALES CONTACTSEVENTS
Senior Director, InformationWeek Events, Robyn Duda (212) 600-3046, rduda@[Link]
SALES CONTACTSEAST
Midwest, South, Northeast U.S. and Eastern Canada (Saskatchewan, Ontario, Quebec, New Brunswick) District Manager, Jenny Hanna (516) 562-5116, jhanna@[Link] District Manager, Michael Greenhut (516) 562-5044, mgreenhut@[Link]
Copyright 2012 UBM LLC. All rights reserved.
MARKETING
VP, Marketing, Winnie Ng-Schuchman (631) 406-6507, wng@[Link] Senior Marketing Manager, Monique Kakegawa (949) 223-3609, mkakegawa@[Link] Promotions Manager, Angela Lee-Moll (516) 562-5803, aleemoll@[Link] November 2012 11
[Link]