Network Virtualization in Data Centers
Network Virtualization in Data Centers
CENTER –
NETWORKING
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 3
Network Virtualization in VDC
• Involves virtualizing physical Physical Server Physical Server
and VM networks
Physical Network
storage systems
Storage Array
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 5
Network Virtualization in VDC (contd.)
VM Network
Physical Server Physical Server
• Resides inside physical server
• Consists of logical switches VNIC VNIC VNIC VNIC VNIC VNIC
• Provides connectivity to
Hypervisor kernel
Physical
• Connects to physical network Network
Client
Storage Array
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 6
For communication between two VMs residing in different physical
servers and between a VM and its clients, the VM traffic must
travel through both the VM and physical networks. Hypervisor
traffic is also required to transfer between the VM and physical
networks. Hence, the VM network must be connected to the
physical network.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 7
• A VM network resides inside a physical server. It includes logical
switches, called ‘virtual switches’, which function similar to
physical switches. The VM network enables communication
among VMs within a physical server. For example, a VM which is
running a business application may need to filter its traffic via a
firewall server which could be another VM within the same
physical server. It is beneficial to connect these VMs internally
through the VM network. Connecting them through a physical
network will add more delay to the VM traffic because it travels
over the external physical network.
• Hypervisor kernels are connected to the VM network. Hypervisor
kernels communicate with the management server and storage
systems using the VM network. The management server could
be a VM hosted in a physical server.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 8
Network Virtualization Contd..
Network virtualization allows an administrator to create multiple
virtual networks in VDC. These virtual networks may span across
both VM and physical networks and share physical and virtual
switches.
A virtual network provides grouping of all the nodes that belong to
the same functional unit in an organization.
In the figure shown on the next slide, two virtual networks are
created on both virtual and physical switches. Virtual network 1
provides connectivity to VM1 and VM3 and enables
communication between them without routing of frames.
Similarly, VM2 and VM4 belong to virtual network 2 and are
allowed to communicate without routing.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 9
Network Virtualization in VDC (contd.)
• VM and physical networks are virtualized to create virtual
networks; for example: virtual LAN, virtual SAN
Virtual Network 1
VM1 VM3
VNIC VNIC
PNIC PNIC
VM2 VM4
VNIC VNIC
Virtual Network 2
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 10
Network Virtualization tools..
• Network virtualization is performed by hypervisor and physical
switch Operating System (OS). These types of software allow an
administrator to create virtual networks on physical and VM
networks.
• A physical switch runs an Operating System which performs
network traffic switching.
• The Operating System must have network virtualization
functionality to create virtual networks on the switch.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 11
Network Virtualization Tools
• Physical switch Operating System (OS)
OS must have network virtualization functionality
• Hypervisor
Uses built-in networking and network virtualization functionalities
To create virtual switch and configuring virtual networks on it
Or, uses third-party software for providing networking and
network virtualization functionalities
Third-party software is installed onto the hypervisor
Third-party software replaces the native networking functionality of
the hypervisor
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 12
Components of VDC Network Infrastructure
• VDC network infrastructure includes both virtual and physical network
components
Components are connected to each other to enable network traffic flow
Component Description
• Connects VMs to the VM network
Virtual NIC
• Sends/receives VM traffic to/from VM network
Virtual HBA • Enables a VM to access FC RDM disk/LUN assigned to the VM
• Is an Ethernet switch that forms VM network
• Provides connection to virtual NICs and forwards VM traffic
Virtual switch
• Provides connection to hypervisor kernel and directs
hypervisor traffic: management, storage, VM migration
Physical adapter: • Connects physical servers to physical network
NIC, HBA, CNA • Forwards VM and hypervisor traffic to/from physical network
• Forms physical network that supports Ethernet/FC/iSCSI/FCoE
Physical switch, • Provides connections among physical servers, between
router physical servers and storage systems, and between physical
servers and clients
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 13
Network Connectivity and Traffic Flow:
Example 1 Physical server
• VM Hypervisor
Kernel
• Management
• IP storage PNIC
• VM migration
Traffic type:
• Management
• IP storage
• VM migration
Physical Switch
(Ethernet)
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 14
Example 1
• The connectivity among VDC network components varies based
on the type of protocol and the physical adapter used to enable
physical server access to the storage system.
• In this example, physical servers are connected to the IP storage
system, such as a NAS or an iSCSI storage array. A physical
Ethernet switch is used to connect physical servers and the
storage system.
• Each physical server hosts multiple VMs that are connected to a
virtual switch. Each VM has at least one virtual NIC which
transfers/receives VM I/Os in the form of Ethernet frames. These
Ethernet frames travel through virtual and/or physical switches
before reaching their destination (Clients and any other VMs
residing in other physical servers.)
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 15
Example 1
• Hypervisor kernel is also connected to the virtual switch.
Hypervisor kernel leverages the virtual and physical switches to
send the IP storage, management, and VM migration traffic.
• A physical server has one or more physical NICs (one NIC in this
example). NICs provide a link between the virtual and physical
switches and forwards VM and hypervisor kernel traffic between
the switches.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 16
Network Connectivity and Traffic Flow:
Example 2 Physical server
Virtual Switch
Traffic type: (Ethernet)
• VM
• Management PNIC FC / iSCSI HBA
• VM migration
Traffic type:
• FC or iSCSI
storage
Physical Switch Physical Switch
(Ethernet) (FC/Ethernet)
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 17
Example 2
• The connectivity shown on this slide is similar to the previous
example. However, in this case, the hypervisor kernel uses an FC
or iSCSI HBA to access FC or iSCSI storage array. A hypervisor
kernel is directly connected to the HBA. The HBA sends or
receives storage traffic via an FC or Ethernet switch (This could
be the same physical Ethernet switch that is connected to the
virtual switch).
• A hypervisor kernel still uses the virtual switch to send/receive
management and VM migration traffic.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 18
Network Connectivity and Traffic Flow:
Example 3 Physical server
• VM Hypervisor • IP storage
Kernel • Management
• VM migration
• VM migration
Traffic type:
• FC storage
Physical Switch
(FCoE)
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 19
Example 3
• In this scenario, a physical server uses a CNA card instead of
separate HBA and NIC. The CNA card provides the connection
between a virtual switch and a physical FCoE switch. CNA has
the capability to converge both FC and Ethernet traffic over an
Ethernet connection. This allows hypervisor kernel to access FC
and IP storage system using a single network adapter.
• Hypervisor kernel recognizes CNA as an NIC and an FC HBA. To
access FC storage, hypervisor kernel directly sends storage traffic
to CNA. To access the IP storage or to forward management and
VM migration traffic, the hypervisor kernel sends traffic through
the virtual switch.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 20
Virtual Network Components: virtual NIC
• A VM can have one or more virtual NICs. The working of a virtual
NIC is similar to a physical NIC, even though the virtual NIC is
used for connecting VMs and virtual switches. The guest
Operating System sends network I/Os to the virtual NIC through
a device driver similar to a physical NIC. Virtual NIC forwards I/Os
in the form of Ethernet frames to a virtual switch for further
transmission to destination. Each virtual NIC has unique MAC
and IP addresses and responds to the standard Ethernet
protocol exactly as how a physical NIC would. Hypervisor
generates these MAC addresses and allocates a MAC address to
a virtual NIC at the time of VM creation.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 21
Virtual Network Component: Virtual NIC
• Connects VMs to virtual switch
• Forwards Ethernet frames to virtual switch
• Has unique MAC and IP addresses
• Supports Ethernet standards similar to physical NIC
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 22
Virtual Network Component: Virtual Switch
• Is a logical OSI layer 2 switch that supports Ethernet protocol
• Resides inside a physical server
• Is created and configured using hypervisor
• Maintains MAC address table for frame forwarding
• Directs network traffic to/from VMs and hypervisor kernel
VM to VM within physical server
VM to physical network
Hypervisor kernel: IP storage, VM migration, and management
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 23
Virtual Switch
A virtual switch is a logical layer 2 (OSI model) Ethernet switch that
resides inside a physical server that uses a hypervisor. Virtual
switches are created and configured using hypervisor.
Virtual switches provide traffic management for VMs and
hypervisor kernel. Each virtual switch maintains a MAC address
table, which includes a list of MAC addresses and corresponding
virtual switch ports for frame forwarding. The virtual switch
forwards frames to a virtual switch port based on the destination
MAC address of the frame.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 24
Virtual Switch
Virtual switches enable communication among VMs within a
physical server and direct VM traffic to a physical network.
Switching of VM traffic to physical network allows VMs to
communicate with their clients or with VMs hosted on another
physical server. A virtual switch also handles the hypervisor kernel
traffic so as to enable the management server access the physical
server, hypervisor kernel access the IP storage, and migrate VMs
from one physical server to another.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 25
Virtual Network Component: Virtual Switch
(contd.)
• May connect to multiple physical NICs
Connection to multiple NICs performs load balancing and failover
Physical Server
Hypervisor
Kernel
Virtual Switch
Load balancing
and failover
Physical Switch
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 26
A virtual switch may be connected to one or more physical NICs. If
a virtual switch is connected to more than one physical NIC, it
allows the virtual switch to distribute outbound traffic across
multiple physical NICs. Some of the physical NICs may be
configured as standby. In the event of an active physical NIC
hardware failure or its network link outage, virtual switches
failover the traffic to a standby physical NIC.
A virtual switch has no control over the inbound traffic. The load
balancing and failover of inbound traffic is performed by
supported physical switches that are connected to the virtual
switch via physical NICs.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 27
Virtual Network Component: Virtual Switch
(contd.)
• May have no connection to any physical NIC
If virtual switch has no connection to physical NIC, it directs VM
traffic within the physical server
Physical Server
VM with Firewall
Application
VM1 VM2
VNIC VNIC VNIC
Hypervisor
Kernel
PNIC
Physical Switch
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 28
Virtual Network Component: Virtual Switch
(contd.)
• No direct connection between virtual switches
• Frames may be transferred between virtual switches via a VM
• Physical NICs are not shared between virtual switches
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 29
Virtual Switch: Ports and Port Group
• Types of ports
Hypervisor kernel port: Provides connectivity to hypervisor kernel
VM port: Provides connectivity to virtual NICs
Uplink port: Provides connectivity to physical NIC
• VM port group: Mechanism for applying uniform network policy
settings to a group of VM ports
Policy example: Security, load balancing, and failover across PNICs
• VMs connected to a VM port group share common configuration
Eliminates configuring policies to VM ports individually
VM port
Hypervisor PG 1 PG 2 PG 3
kernel port VM port groups
Uplink ports
Virtual Switch
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 30
Ports and Port groups of Virtual Switch
Virtual switches direct both hypervisor kernel traffic and VM
traffic. For different types of traffic, different types of virtual ports
are configured on a virtual switch. Alternatively, multiple virtual
switches may be created; each with its own virtual port type.
Virtual ports are classified as hypervisor kernel port, VM port, and
uplink port.
• Uplink ports connect a virtual switch to physical NICs of the
physical server where the virtual switch resides. A virtual switch
can transfer data to a physical network only when one or more
physical NICs are attached to its uplink ports.
• VM ports allow virtual NICs to connect to a virtual switch.
• A hypervisor kernel port enables the hypervisor kernel to
connect to a virtual switch.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 31
• Port group is a mechanism for applying network policy settings
to a group of VM ports and hence to a group of VMs. This allows
an administrator to apply identical network policy settings across
a group of VMs, rather than configuring the policies to VM ports
individually. Examples of network policies are:
• Security
• Load balancing and failover across physical NICs
• Limiting network bandwidth for VMs
• Virtual LAN assignment to a VM port group to transfer the VM
traffic
• A virtual switch may have multiple port groups. VMs that are
connected to a VM port group share a common configuration
applied to the port group. An example of security policy setting
for a port group is to stop receiving frames, if guest Operating
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 32
Ports and port groups
• System attempts to change the MAC address assigned to the
virtual NIC. This helps protect against attacks launched by a
rogue Operating System.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 33
Distributed Virtual Switch
• Aggregation of multiple virtual switches distributed across
multiple physical servers
Benefit
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 34
Distributed Switch
• A distributed virtual switch is an aggregation of multiple virtual
switches distributed across multiple physical servers. It functions
as a single virtual switch and is created at the management
server, which provides a single interface for managing a group of
physical servers hosting VMs. Virtual switches that form the VM
network still exist. However, virtual switch configuration and
control is moved into the management server. The standalone
virtual switches execute the frame forwarding based on the
configuration made at the distributed virtual switch.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 35
Distributed Switch
• A distributed virtual switch brings up the next generation virtual
networking concept. Instead of per physical server configuration,
a distributed virtual switch brings in a centralized point of a VM
network configuration. This simplifies configuration and ongoing
administration of the VM network. Virtual switch port and port
groups are configured at the distributed virtual switch level.
Therefore, network policy settings remain consistent as VMs
within a port group migrate across multiple physical servers. A
distributed virtual switch allows movement of port group
policies with VM.
• awesome
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 36
Physical Network Component: NIC
• Physical NICs are used as inter-switch-links between virtual and
physical Ethernet switches
Transfer VM and hypervisor kernel traffic
• Physical NICs are not addressable from network
• IP address not assigned (prohibits OSI layer 3 access)
MAC addresses not available (prohibits OSI layer 2 access)
• Virtual NIC and hypervisor kernel are addressable from network
Have their own MAC and IP addresses
Are used as source address in Ethernet frames
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 37
PNIC
• A physical NIC provides an inter-switch-link between a virtual
switch and a physical Ethernet switch. It is used to transfer VM
and hypervisor kernel traffic between the VM and physical
networks. It is called a link because it is not addressable from the
network. Physical NICs are neither assigned an IP address (for
OSI layer 3 access), nor are their built-in MAC addresses (for OSI
layer 2 access) available to VMs or physical servers on the
network.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 38
PNIC
Virtual NICs and hypervisor kernel are addressable from a
network. A virtual NIC adds its MAC and IP addresses as source
addresses to the Ethernet frames it transfers.
Similarly, hypervisor inserts its own MAC and IP addresses before
sending an Ethernet frame to a virtual switch. These Ethernet
frames are transferred without modification through physical NICs.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center - Networking 39
Virtual Local Area Network (VLAN)
VLAN
Benefit
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 40
Configuring VLAN
• Define VLAN IDs on physical switch
Each VLAN is identified by a unique number: VLAN ID
• Choose necessary VLAN IDs from hypervisor’s built-in VLAN ID
pool
Required for virtual switches
• Assign VLAN ID to physical and virtual switch ports
To include switch ports to a VLAN
To enable grouping of switch ports into VLANs
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 41
Configuring VLAN (contd.)
• Nodes become VLAN members when connected to VLAN ports
• Switch forwards frames between switch ports that belong to
common VLAN
• VLAN traffic is transferred through routers
During inter VLAN communication
When VLAN spans different IP networks
• VM and storage systems may be members of multiple VLANs
Requires support of respective operating system
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 42
VLAN Trunking
VLAN Trunking
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 43
Benefits of VLAN Trunking
• Eliminates the need for dedicated network link(s) for each VLAN
• Reduces inter-device links when the devices have more than one
VLAN
Reduces the number of virtual NICs, storage ports, and switch
ports
Reduces management complexity
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 44
VLAN Tagging
VLAN Tagging
It is a process of inserting or removing a marker (tag) with VLAN-specific
information (VLAN ID) into the Ethernet frame
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 45
VLAN Trunking Scenario
• Sales group: Includes VM1, VM4, and VM5
• Finance group: Includes VM2 and VM5
• Marketing group: Includes VM3 and VM5
VLAN 10
VM4
VM1
VLAN 10
Frame
VLAN 20 Tr
VLAN 20 u nk
Trunk link Trunk link lin
VM2 k
VLAN 30
VM5
VM3
VLAN 10, 20, 30
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 46
Requirements for Network Traffic
Management
• Load balancing
Distributes workload across multiple IT resources
Prevents over/under utilization of resources, and optimizes
performance
• Policy-based management
Allows using a policy for distribution of traffic across VMs and
network links
Allows using a policy for traffic failover across network links
• Resource sharing without contention
Enables guaranteed service levels when traffic from multiple
virtual networks share physical network resources
Sets priority for bandwidth allocation to different types of traffic
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 47
Key Network Traffic Management Techniques
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 48
Technique 1 – Balancing Client Workload:
Hardware Based
• A device (physical switch/router) distributes client traffic across
multiple servers – physical or virtual machines
• The load balancing device resides between the server cluster
and the Internet. This allows all client traffic to pass through the
load balancing device.
• Clients use IP address (virtual) of the load balancing device to
send requests. This IP address is called virtual because it
abstracts the real IP addresses of all servers in a cluster.
• The real IP addresses of the servers are known only to the load
balancing device, which decides where to forward the request.
• Decision making is typically governed by load balancing policy,
for example: Round robin, Weighted round robin, Least
connections.
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 49
Technique 3 – Storm Control
• Prevents impact of storm on regular LAN/VLAN traffic
Storm: Flooding of frames on a LAN/VLAN creating excessive traffic
and degrading network performance
• Counts frames of a specified type over 1-second and compares
with the threshold
• Switch port blocks traffic if threshold is reached and drops the
subsequent frames over the next time interval
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 50
Technique 2 – Balancing Client Workload:
Software Based
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 51
Technique 4 – NIC Teaming
• Logically groups physical NICs connected to a virtual switch
Creates NIC teams whose members can be active and
standby
Balances traffic load across active NIC team members
Provides failover in the event of an NIC/link failure
Allows associating policies for load balancing and failover at a
virtual switch or a port group
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 52
Technique 5 – Limit and Share
• Are configurable parameters at distributed virtual switch
• Are configured to control different types of network traffic,
competing for a physical NIC or NIC team
• Ensure that business critical applications get required bandwidth
Configur
able
Description
Paramet
er
• Sets limit on maximum bandwidth per traffic type
Traffic type will not exceed limit
Limit
• Is specified in Mbps
• Applies to an NIC team
• Specifies relative priority for allocating bandwidth
to different traffic types
Share
• Is specified as numbers
• Applies to a physical NIC
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 53
Technique 6 – Traffic Shaping
• Controls network bandwidth at virtual/distributed virtual switch or port
group
• Prevents impact on business-critical application traffic by non-critical traffic
flow
Parameter Description
•Data transfer rate allowed over time
Average
•Workload at a switch port can intermittently exceed av. Bandwidth
Bandwidth
• Burst: When the workload exceeds the average bandwidth, it is called burst
Peak Bandwidth • Max data transfer rate without queuing/dropping frames
• Max amount of data allowed to transfer in a burst
Burst Size •Burst size = bandwidth × time
•Bandwidth in a burst can go up to peak bandwidth
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 54
Technique 7 – Multipathing Physical server
Hypervisor Kernel
Multipathing Multipathing
Storage array
Copyright © 2011 EMC Corporation. All Rights Reserved. Virtualized Data Center – Networking 55