University of

Michigan Administrative Information Services

Server Virtualization Technologies: Uses, Comparisons, and Implications

David Sweetman Windows Enterprise Systems Admin Administrative Information Services University of Michigan
dsweetma@[Link]

Presentation Overview
The What and Why of virtualization Comparing Product Features Comparing Product Performance Evaluating Physical Servers for virtualization Costs Questions

Spring, 2005

Windows Virtualization

What is server virtualization? Creating multiple logical server OS instances on one physical piece of hardware All HW drivers are virtualized same virtual HW regardless of physical HW Each virtual machine is completely independent of the others and doesnt realize its virtualized
Spring, 2005 Windows Virtualization 3

Why virtualize?
More efficient HW utilization More efficient staff Long-term matching resources & needs Quick and nimble server provisioning Testing & Troubleshooting More effective redundancy HW maintenance w/o app downtime Simplify system imaging Disaster Recovery
Spring, 2005 Windows Virtualization 4

HW Utilization Facts
Individual ebb and flow of resources Cumulative usage of 28 servers in the MAIS data center evaluated for virtualization: 44GB RAM, 138.15Ghz CPU, and 1323GB HD 45% of RAM not used 99.9% of time. 25% of RAM never used concurrently. 85% of CPU not used 99.9% of time. 81% of CPU never used concurrently. 68% of hard disk space unused
Spring, 2005 Windows Virtualization 5

Server Local Disk

More Efficient Hard SAN Manager Hard Disk Utilization 48 8 40 Disk Utilization IIS app test 68 9 59
68 34 34 68 17 136 34 17 68 68 68 34 68 170 34 13 7 24 31 9 56 6 6 16 11 10 13 6 88 7 55 27 10 37 8 80 28 11 52 57 58 21 62 82 27

Total (GB)

Used (GB)

Free (GB)

TNG Scheduling PeopleSoft 8 HE PeopelSoft 8 FIN IIS / SQL:Research app Small use Citrix File Servers Stat Version Control Stat Version Control SQL: eLearning dev IIS: eLearning dev SQL: eLearning Prod IIS: eLearning Prod Machine Room environ IIS document server Domain Controller

Total: 1323 GB Used: 418 GB Free: 905 GB (68% unused)

SAN in 30GB chunks 1 fibre channel >1 server Virtual HDs more granular Share free space allocate as needed Windows Virtualization
6

Spring, 2005

Virtualization vs. Consolidation Virtualized servers = separate OSes Consolidation = same OS Virtualized servers must each be administered, patched, etc. Consolidated applications can introduce conflicts and support issues

Spring, 2005

Windows Virtualization

Virtual Host Licensing Windows and other Microsoft per-server apps are licensed per virtual server. (1 physical server w/ 6 virtual Windows servers = 6-7 licenses needed) As of 4/1/2005, Microsoft per-processor licenses are per physical processor (1 physical server w/ 3 virtual SQL Servers sharing 1 CPU = 1 per-processor license) Virtualization savings are not in licenses. Check with other vendors.
Spring, 2005 Windows Virtualization 8

Virtualization Software MS Virtual PC 2004 workstation only VMWare Workstation 5 workstation only MS Virtual Server 2005, Standard (4p) MS Virtual Server 2005, Enterprise (32p) VMWare GSX Server 3.1 VMWare ESX Server 2.5
Spring, 2005 Windows Virtualization 9

Common Features
Up to 3.6GB RAM per virtual host Web-based console for administration Host OS sees HT CPU, virtual do not VMs consist of 1 config file & 1 file / HD VMs can mount physical CDs or ISOs VMs can be multi-homed Up to 64 VMs per host server Highly scriptable extensive API Granular permissions for individual VMs Detailed logging
Spring, 2005 Windows Virtualization 10

MS Virtual Server 2005 Targeted to increase efficiency in testing and development, and re-hosting Up to 1 processor per virtual host Windows = underlying host OS Only Windows VMs supported No USB support 2 processor SMP coming soon

Spring, 2005

Windows Virtualization

11

VMWare ESX Server 2.5 Targeted at mission-critical enterprise services Up to 2 processors per host Custom Linux = underlying OS Windows & Linux VMs supported Dedicated NIC for admin (2 total min) USB support 4 proc SMP coming soon
Spring, 2005 Windows Virtualization 12

Do I need to know Linux? VMWare ESX Server is based on Linux All administration is possible through web Dont need any Linux experience for installation or ongoing admin SSH and SFTP access to server Used?
Installed backup software sFTPed ISOs to server
Spring, 2005 Windows Virtualization 13

Managing Virtual Servers Web site is primary interface Attach to VM console

Virtual Server = ActiveX control VMWare = separate application

Reboot, power on, power off Create and manage VMs Allocate hardware resources Mount CDs and floppies View recent performance data
Spring, 2005 Windows Virtualization 14

VS Screenshot

Spring, 2005

Windows Virtualization

15

VMWare Screenshot

Spring, 2005

Windows Virtualization

16

Hyper-threading One physical CPU seen as 2 logical Both products see HT, non-HT VMs Slows virtualization performance 1 HT CPU < 2 Phy CPU 0-20% performance increase over no HT [Link] ead/

Spring, 2005

Windows Virtualization

17

RAM Allocation Virtual Server: Max <= total physical memory VMWare: Max <> total physical
RAM Ballooning RAM pooled across multiple VMs Enables more efficient RAM utilization If max out, goes to paging file
Windows Virtualization 18

Spring, 2005

VS Screenshot

Spring, 2005

Windows Virtualization

19

VMWare Screenshot

Spring, 2005

Windows Virtualization

20

Monitoring MOM (or other host monitoring): Monitors VMs like physical Virtual Server: MOM Management Pack
Integrates into MOM framework Monitor overall host and VM servers

VMWare: vmkusage VMWare: VirtualCenter

Database back-end across all servers
Spring, 2005 Windows Virtualization 21

Virtual Center Central monitoring and management in VMWare environment Manage all VMs from one interface Additional software / license Management application Set thresholds and actions like MOM SQL or Oracle DB backend Assign privileges via NTFS
Spring, 2005 Windows Virtualization 22

Virtual Center Screenshot

Spring, 2005

Windows Virtualization

23

Converting Physical Server

Both MS & VMWare offer tools to create virtual systems from physical Physical HW drivers replaced by VM Ideal for the truly unique server (highly customized) Both vendors recommend loading virtual servers from scratch Slow for both vendors 6h / 4GB image VSMT (Virtual Server Migration Tool)
many prereqs (DHCP, ADS, SQL) Not in one month eval

P2V (Physical 2 Virtual)

Simple boot CD and server piece Licensed per use
Spring, 2005 Windows Virtualization 24

VMotion Enables seamless transition of live virtual host between physical servers Dynamic Resource Allocation across servers respond to load changes HW maintenance

Spring, 2005

Windows Virtualization

25

Best Practices Plan out server allocations Create gold image base OS kept upto-date patches duplicate for new VMs Use ISOs for CD access Use standard backup and restore Take system images as needed

Spring, 2005

Windows Virtualization

26

Summary of VMWare differences More comprehensive web GUI (for example, deleting hosts & HDs) Support for dual processor virtuals Support for Linux virtuals Virtual Center: central management Easy-to-use physical-to-virtual support VMotion: seamlessly move virtual servers between physical hosts
Spring, 2005 Windows Virtualization 27

Testing Environment
One month each was spent evaluating MS Virtual Server & VMWare ESX Server Identical testing was attempted on each. Load and usability testing: Win 2000, 2003, IIS5, IIS6, SQL Server 2000, 3rd party apps Test hardware
1.4Ghz x 4 physical processors (8 w/ HT) 8GB of RAM 60GB fibre-channel connected SAN space
Spring, 2005 Windows Virtualization 28

Performance Comparisons
Automated load test of Aspen 2.5 dev environment (Win 2000/IIS5 & Win 2000/SQL 2000) Citrix / TS load test w/ Helpdesk IIS6-based memory, CPU, disk, and network I/O testing SQL Server add, update, and delete testing Load testing both as isolated server and with other virtual server processing Normal usage w/o issue in all cases
Spring, 2005 Windows Virtualization 29

Performance Comparisons
Windows 2003 IIS6 and SQL 2000 perf compare
Physical CPU Memory Disk&NIC I/O SQL 100% 100% 100% 100% MSVS 94% 91% 101% 57% VMWare 80% 91% 101% 87%

VMWare CPU : hyper-threaded related, ~93% w/o VS SQL : VS 2005 SP1 has performance enhancements

Spring, 2005

Windows Virtualization

30

Performance Comparisons
Previous stats were isolated tests VMs wont be alone on physical host How does system perform w/ other VMs running assorted, intensive tasks?
RAM Virtual Server 2005 -/+ <10% VMWare ESX Server Same CPU -/+ <10% Same Disk - <5% - <5% Network - <5% - <5%

Spring, 2005

Windows Virtualization

31

IIS/SQL Load Test Results Mercury LoadRunner scripted test Overall performance
100@30/min: VM = 60% 1000@12/min: VM = 99%

What made it slow?

CPU queuing Memory, HD, NetIO nearly identical

Spring, 2005

Windows Virtualization

32

Terminal Services / Citrix Load Test Results

Currently 14 servers, 4procs (8HT), 4GB RAM load balancing ~700 concurrent CPU and RAM intensive apps ~60 users max per physical server CPU = bottleneck (logon & BusObj) 1CPU = 7 users max ; 2 CPU = 12 max 100 v 1CPU or 58 v 2CPU to match 14 physicals Recommendation: 2 CPU & only for small use
Spring, 2005 Windows Virtualization 33

Business Objects WebI dev

Win 2000 / IIS5 / 2400MB RAM / 1.4Ghz x 2 (no HT)
24 x 7 sampling RAM (MB) Nic KB HD KB /sec /sec Proc % Usage RAM (MB) Hours of Op sampling Nic KB /sec HD KB /sec Proc % Usage

100%847 99.99%839 99.9%823 99%816 95%813 90%809 Av755 StDev47

61 45 19 17 4 2 2 4

4324 1548 487 76 65 64 55 86

112 55 52 3 3 2 2 3

847 839 821 816 814 812 759 44

61 47 32 17 11 2 2 4

4324 97 85 67 65 63 52 136

34 24 22 7 6 5 1 1

Virtualize? Yes. 900 / 1.4Ghz

Spring, 2005 Windows Virtualization 34

PSoft 8 Fin Crystal/nVision: Dev

Win 2000 / 2300MB RAM / 1.1Ghz x 2 (no HT)
24 x 7 sampling RAM (MB) Nic KB HD KB /sec /sec Proc % Usage RAM (MB) Hours of Op sampling Nic KB /sec HD KB /sec Proc % Usage

100%835 99.99%822 99.9%821 99%814 95%798 90%786 Av644 StDev91

5284 31 4 3 3 2 51

5038 4912 4802 910 61 205 834

106 89 82 67 10 2 4 8

835 829 822 819 809 799 662 106

5284 4542 72 5 3 3 3 87

2529 1555 1494 1015 100 32 49 181

106 92 75 13 3 2 2 3

2973 5005

Virtualize? Yes. 900 / 1.4Ghz Spring, 2005 Windows Virtualization 35

PSoft8 HE Crystal/nVision - Prod

Win 2000 / 1500MB RAM / 2.8Ghz x 1 (w/ HT)
24 x 7 sampling RAM (MB) Nic KB /sec HD KB Proc % /sec Usage RAM (MB) Hours of Op sampling Nic KB /sec HD KB /sec Proc % Usage

100%716 99.99%710 99.9%620 99%534 95%483 90%447 Av378 StDev67

11499 9803 1422 119 8 1 10 162

3421 350 3379 329 2440 244 2304 192 183 50 105 356 34 23 23 7

716 713 691 460 440 437 363 63

9437 3493 674 119 11 1 7 109

3421 3379 779 351 74 51 49 159

348 333 267 220 32 22 21 6

Virtualize? NOT at this time CPU needs too high Spring, 2005 Windows Virtualization 36

sumTotal Aspen 2.5 eLearning

Win 2000 / SQL 2000 / 2358MB RAM / 1.9Ghz x 2 (w/ HT)
24 x 7 sampling RAM (MB) Nic KB /sec HD KB /sec Proc % RAM Usage (MB) Hours of Op sampling Nic KB /sec HD KB /sec Proc % Usage

100%2077 99.99%2075 99.9%2073 99%1984 95%1777 90%1670 Av1628 StDev76

9061 5865 2667 91 68 5 16 173

4477 3682 3673 3626 839 517 236 505

277 233 216 138 101 41 24 5

2077 2075 2073 2064 1684 1665 1636 60

406 404 206 70 67 3 5 20

1047 1039 971 827 623 459 166 183

155 149 138 125 59 30 21 4

Virtualize? Yes 2300MB / 1.4Ghz x 2 Spring, 2005

Note: high NIC=sync ; CPU=imp/exp Windows Virtualization 37

Domain Controllers
Win 2003 / 2000MB RAM / 700Mhz x 4 (no HT)
24 x 7 sampling RAM (MB) Nic KB /sec HD KB /sec Proc % RAM Usage (MB) Hours of Op sampling Nic KB /sec HD KB /sec Proc % Usage

100%776 99.99%771 99.9%768 99%753 95%713 90%707 Av633 StDev74

5677 5326 2131 51 24 15 12 138

4298 3674 3440 1972 140 91 128 302

146 131 78 43 12 10 8 3

767 766 757 753 713 707 646 56

457 98 93 42 27 20 7 11

1237 195 194 180 90 88 78 41

92 51 16 13 11 10 8 1

Virtualize? Yes 850MB / 1.4Ghz Spring, 2005 Windows Virtualization 38

Univ of Michigan - Flint VMWare ESX Server Determining factor: Linux support & MS Virtual Server wasnt available Several years of experience, starting with GSX, public web services, online teaching, real video server, internal file/print, 46v on 5 physical (15 on 1), <10% slower, Dell 2650s & 4600s, 2 proc, 12GB RAM
Spring, 2005 Windows Virtualization 39

NC State University MS Virtual Server 2005 Determining factor: Cost PeopleSoft v8 Crystal/nVision app servers: 18 virtual servers, 7 physical servers, dual Xeon >2GB, physical v. virtual head-to-head, little difference in performance.

Spring, 2005

Windows Virtualization

40

Potential Uses from Previous Presentations

NAP - Remediation Servers Big Red Button for critical fix assign additional resources Keynote - Reliability one of pillars of Trustworthy Computing Boston U Matt - NetReg peak usage first couple weeks of semester WSUS 3Ghz, 1GB RAM recommended sitting idle most of time? Decrease dev system allocation in busy times
Spring, 2005 Windows Virtualization 41

Pricing
MS Virtual Server 2005 (4CPU Server, 8GB RAM)
Win 2003 Std: up to 4 processors, Ent: up to 32 VS Std: 4proc/4GB; Ent: 8proc/32GB 2003 Ent/Std: ~$500+~$500 = ~$1000

VMWare Server ESX (4CPU other pricing scales)

ESX: $4500/phy server + $945/yr support ESX+SMP+V-agents: $6000/phy server + $1764/yr support

VMWare Add-ons
VirtualCenter server: $3000 + $1050/yr P2V Starter kit (25): $2000 + $420/yr
Spring, 2005 Windows Virtualization 42

Cost / Benefit Example VMWare Server ESX

$45K separate HW purchase price $29K + $2K/yr (ESX w/SMP): ~35%

MS Virtual Server Std

$33K separate HW purchase price $30K virtual HW + software: ~10%
Note: In both cases, estimates are conservative

Spring, 2005

Windows Virtualization

43

Summary / take-aways
More effective resource utilization and response to changing needs (5-15% to 60-70%) Virtual Server & VMWare = comparable performance, VMWare more isolated VMWare more feature-rich: SMP, VMotion, manage multiple servers VMWare costs more, but you can do more, virtualize more costly servers Both platforms have limits, active improvement

Spring, 2005

Windows Virtualization

44

Other Resources VMWare: [Link] Virtual Server: [Link]/virtualserver/ Rapid App: [Link]

Spring, 2005

Windows Virtualization

45

David Sweetman University of Michigan dsweetma@[Link]

Questions?

Spring, 2005

Windows Virtualization

46