Chapter 06: Virtual Machines
Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013
Chapter 16: Virtual Machines
Overview
History
Benefits and Features
Building Blocks
Types of Virtual Machines and Their Implementations
Virtualization and Operating-System Components
Examples
Operating System Concepts – 9th Edition 16.2 Silberschatz, Galvin and Gagne ©2013
Chapter Objectives
To explore the history and benefits of virtual machines
To discuss the various virtual machine technologies
To describe the methods used to implement virtualization
To show the most common hardware features that support
virtualization and explain how they are used by operating-
system modules
Operating System Concepts – 9th Edition 16.3 Silberschatz, Galvin and Gagne ©2013
Overview
Fundamental idea – abstract hardware of a single computer into
several different execution environments
Similar to layered approach
But layer creates virtual system (virtual machine, or VM) on which
operation systems or applications can run
Several components
Host – underlying hardware system
Virtual machine manager (VMM) or hypervisor – creates and
runs virtual machines by providing interface that is identical to the
host
4 (Except in the case of paravirtualization)
Guest – process provided with virtual copy of the host
4 Usually an operating system
Single physical machine can run multiple operating systems
concurrently, each in its own virtual machine
Operating System Concepts – 9th Edition 16.4 Silberschatz, Galvin and Gagne ©2013
System Models
processes
processes
processes processes
programming
kernel kernel kernel
interface
VM1 VM2 VM3
kernel
virtual machine
manager
hardware
hardware
(a) (b)
Non-virtual machine Virtual machine
Operating System Concepts – 9th Edition 16.5 Silberschatz, Galvin and Gagne ©2013
Implementation of VMMs
Vary greatly, with options including:
Type 0 hypervisors - Hardware-based solutions that provide
support for virtual machine creation and management via firmware
4 IBM LPARs and Oracle LDOMs are examples
Type 1 hypervisors - Operating-system-like software built to
provide virtualization
4 Including VMware ESX, Joyent SmartOS, and Citrix XenServer
Type 1 hypervisors – Also includes general-purpose operating
systems that provide standard functions as well as VMM functions
4 Including Microsoft Windows Server with HyperV and RedHat Linux
with KVM
Type 2 hypervisors - Applications that run on standard operating
systems but provide VMM features to guest operating systems
4 Includeing VMware Workstation and Fusion, Parallels Desktop, and
Oracle VirtualBox
Operating System Concepts – 9th Edition 16.6 Silberschatz, Galvin and Gagne ©2013
Implementation of VMMs (cont.)
Other variations include:
Paravirtualization - Technique in which the guest operating system is
modified to work in cooperation with the VMM to optimize performance
Programming-environment virtualization - VMMs do not virtualize real
hardware but instead create an optimized virtual system
4 Used by Oracle Java and [Link]
Emulators – Allow applications written for one hardware environment to
run on a very different hardware environment, such as a different type of
CPU
Application containment - Not virtualization at all but rather provides
virtualization-like features by segregating applications from the operating
system, making them more secure, manageable
4 Including Oracle Solaris Zones, BSD Jails, and IBM AIX WPARs
Much variation due to breadth, depth and importance of virtualization
in modern computing
Operating System Concepts – 9th Edition 16.7 Silberschatz, Galvin and Gagne ©2013
History
First appeared in IBM mainframes in 1972
Allowed multiple users to share a batch-oriented system
Formal definition of virtualization helped move it beyond IBM
1. A VMM provides an environment for programs that is
essentially identical to the original machine
2. Programs running within that environment show only minor
performance decreases
3. The VMM is in complete control of system resources
In late 1990s Intel CPUs fast enough for researchers to try
virtualizing on general purpose PCs
Xen and VMware created technologies, still used today
Virtualization has expanded to many OSes, CPUs, VMMs
Operating System Concepts – 9th Edition 16.8 Silberschatz, Galvin and Gagne ©2013
Benefits and Features
Host system protected from VMs, VMs protected from each other
I.e. A virus less likely to spread
Sharing is provided though via shared file system volume,
network communication
Freeze, suspend, running VM
Then can move or copy somewhere else and resume
Snapshot of a given state, able to restore back to that state
4 Some VMMs allow multiple snapshots per VM
Clone by creating copy and running both original and copy
Great for OS research, better system development efficiency
Run multiple, different OSes on a single machine
Consolidation, app dev, …
Operating System Concepts – 9th Edition 16.9 Silberschatz, Galvin and Gagne ©2013
Benefits and Features (cont.)
Templating – create an OS + application VM, provide it to
customers, use it to create multiple instances of that combination
Live migration – move a running VM from one host to another!
No interruption of user access
All those features taken together -> cloud computing
Using APIs, programs tell cloud infrastructure (servers,
networking, storage) to create new guests, VMs, virtual
desktops
Operating System Concepts – 9th Edition 16.10 Silberschatz, Galvin and Gagne ©2013
Building Blocks
Generally difficult to provide an exact duplicate of underlying
machine
Especially if only dual-mode operation available on CPU
But getting easier over time as CPU features and support for
VMM improves
Most VMMs implement virtual CPU (VCPU) to represent
state of CPU per guest as guest believes it to be
4 When guest context switched onto CPU by VMM,
information from VCPU loaded and stored
Several techniques, as described in next slides
Operating System Concepts – 9th Edition 16.11 Silberschatz, Galvin and Gagne ©2013
Building Block – Trap and Emulate
Dual mode CPU means guest executes in user mode
Kernel runs in kernel mode
Not safe to let guest kernel run in kernel mode too
So VM needs two modes – virtual user mode and virtual
kernel mode
4 Both of which run in real user mode
Actions in guest that usually cause switch to kernel mode
must cause switch to virtual kernel mode
Operating System Concepts – 9th Edition 16.12 Silberschatz, Galvin and Gagne ©2013
Trap-and-Emulate (cont.)
How does switch from virtual user mode to virtual kernel mode
occur?
Attempting a privileged instruction in user mode causes an error
-> trap
VMM gains control, analyzes error, executes operation as
attempted by guest
Returns control to guest in user mode
Known as trap-and-emulate
Most virtualization products use this at least in part
User mode code in guest runs at same speed as if not a guest
But kernel mode privilege mode code runs slower due to trap-and-
emulate
Especially a problem when multiple guests running, each
needing trap-and-emulate
CPUs adding hardware support, mode CPU modes to improve
virtualization performance
Operating System Concepts – 9th Edition 16.13 Silberschatz, Galvin and Gagne ©2013
Trap-and-Emulate Virtualization Implementation
User Processes
Privileged Instruction
Guest User Mode
Operating
Return
System
Trap
VMM Kernel Mode
Emulate Action Update VCPU
VMM
Operating System Concepts – 9th Edition 16.14 Silberschatz, Galvin and Gagne ©2013
Building Block – Binary Translation
Some CPUs don’t have clean separation between privileged
and nonprivileged instructions
Earlier Intel x86 CPUs are among them
4 Earliest Intel CPU designed for a calculator
Backward compatibility means difficult to improve
Consider Intel x86 popf instruction
4 Loads CPU flags register from contents of the stack
4 If CPU in privileged mode -> all flags replaced
4 If CPU in user mode -> on some flags replaced
– No trap is generated
Operating System Concepts – 9th Edition 16.15 Silberschatz, Galvin and Gagne ©2013
Binary Translation (cont.)
Other similar problem instructions we will call special instructions
Caused trap-and-emulate method considered impossible until 1998
Binary translation solves the problem
Basics are simple, but implementation very complex
1. If guest VCPU is in user mode, guest can run instructions natively
2. If guest VCPU in kernel mode (guest believes it is in kernel mode)
1. VMM examines every instruction guest is about to execute by
reading a few instructions ahead of program counter
2. Non-special-instructions run natively
3. Special instructions translated into new set of instructions that
perform equivalent task (for example changing the flags in the
VCPU)
Operating System Concepts – 9th Edition 16.16 Silberschatz, Galvin and Gagne ©2013
Binary Translation (cont.)
Implemented by translation of code within VMM
Code reads native instructions dynamically from guest, on demand,
generates native binary code that executes in place of original code
Performance of this method would be poor without optimizations
Products like VMware use caching
4 Translate once, and when guest executes code containing
special instruction cached translation used instead of
translating again
4 Testing showed booting Windows XP as guest caused
950,000 translations, at 3 microseconds each, or 3 second
(5 %) slowdown over native
Operating System Concepts – 9th Edition 16.17 Silberschatz, Galvin and Gagne ©2013
Binary Translation Virtualization Implementation
User Processes
(VMM Reads Instructions)
Special Instruction
Guest User Mode
Operating
System
Return
VMM Kernel Mode
Translate
Update VCPU
Execute Translation
VMM
Operating System Concepts – 9th Edition 16.18 Silberschatz, Galvin and Gagne ©2013
Building Blocks – Hardware Assistance
All virtualization needs some HW support
More support -> more feature rich, stable, better performance of
guests
Intel added new VT-x instructions in 2005 and AMD the AMD-V
instructions in 2006
CPUs with these instructions remove need for binary translation
Generally define more CPU modes – “guest” and “host”
VMM can enable host mode, define characteristics of each guest VM,
switch to guest mode and guest(s) on CPU(s)
In guest mode, guest OS thinks it is running natively, sees devices (as
defined by VMM for that guest)
4 Access to virtualized device, priv instructions cause trap to VMM
4 CPU maintains VCPU, context switches it as needed
HW support for Nested Page Tables, DMA, interrupts as well over
time
Operating System Concepts – 9th Edition 16.19 Silberschatz, Galvin and Gagne ©2013
Types of Virtual Machines and Implementations
Many variations as well as HW details
Assume VMMs take advantage of HW features
4 HW features can simplify implementation, improve performance
Whatever the type, a VM has a lifecycle
Created by VMM
Resources assigned to it (number of cores, amount of memory,
networking details, storage details)
In type 0 hypervisor, resources usually dedicated
Other types dedicate or share resources, or a mix
When no longer needed, VM can be deleted, freeing resouces
Steps simpler, faster than with a physical machine install
Can lead to virtual machine sprawl with lots of VMs, history and
state difficult to track
Operating System Concepts – 9th Edition 16.20 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Type 0 Hypervisor
Old idea, under many names by HW manufacturers
“partitions”, “domains”
A HW feature implemented by firmware
OS need to nothing special, VMM is in firmware
Smaller feature set than other types
Each guest has dedicated HW
I/O a challenge as difficult to have enough devices, controllers to
dedicate to each guest
Sometimes VMM implements a control partition running
daemons that other guests communicate with for shared I/O
Can provide virtualization-within-virtualization (guest itself can be
a VMM with guests
Other types have difficulty doing this
Operating System Concepts – 9th Edition 16.21 Silberschatz, Galvin and Gagne ©2013
Type 0 Hypervisor
Guest Guest Guest Guest Guest
Guest 1 Guest 2 Guest 3 Guest 4
CPUs CPUs CPUs CPUs
memory memory memory memory
Hypervisor (in firmware) I/O
Operating System Concepts – 9th Edition 16.22 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Type 1 Hypervisor
Commonly found in company datacenters
In a sense becoming “datacenter operating systems”
4 Datacenter managers control and manage OSes in new,
sophisticated ways by controlling the Type 1 hypervisor
4 Consolidation of multiple OSes and apps onto less HW
4 Move guests between systems to balance performance
4 Snapshots and cloning
Special purpose operating systems that run natively on HW
Rather than providing system call interface, create run and manage
guest OSes
Can run on Type 0 hypervisors but not on other Type 1s
Run in kernel mode
Guests generally don’t know they are running in a VM
Implement device drivers for host HW because no other component can
Also provide other traditional OS services like CPU and memory
management
Operating System Concepts – 9th Edition 16.23 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Type 1 Hypervisor (cont.)
Another variation is a general purpose OS that also provides
VMM functionality
RedHat Enterprise Linux with KVM, Windows with Hyper-V,
Oracle Solaris
Perform normal duties as well as VMM duties
Typically less feature rich than dedicated Type 1 hypervisors
In many ways, treat guests OSes as just another process
Albeit with special handling when guest tries to execute
special instructions
Operating System Concepts – 9th Edition 16.24 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Type 2 Hypervisor
Less interesting from an OS perspective
Very little OS involvement in virtualization
VMM is simply another process, run and managed by host
4 Even the host doesn’t know they are a VMM running
guests
Tend to have poorer overall performance because can’t take
advantage of some HW features
But also a benefit because require no changes to host OS
4 Student could have Type 2 hypervisor on native host, run
multiple guests, all on standard host OS such as
Windows, Linux, MacOS
Operating System Concepts – 9th Edition 16.25 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Paravirtualization
Does not fit the definition of virtualization – VMM not presenting
an exact duplication of underlying hardware
But still useful!
VMM provides services that guest must be modified to use
Leads to increased performance
Less needed as hardware support for VMs grows
Xen, leader in paravirtualized space, adds several techniques
For example, clean and simple device abstractions
4 Efficient I/O
4 Good communication between guest and VMM about
device I/O
4 Each device has circular buffer shared by guest and VMM
via shared memory
Operating System Concepts – 9th Edition 16.26 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Paravirtualization (cont.)
Xen, leader in paravirtualized space, adds several techniques
(Cont.)
Memory management does not include nested page tables
4 Each guest has own read-only tables
4 Guest uses hypercall (call to hypervisor) when page-
table changes needed
Paravirtualization allowed virtualization of older x86 CPUs (and
others) without binary translation
Guest had to be modified to use run on paravirtualized VMM
But on modern CPUs Xen no longer requires guest modification
-> no longer paravirtualization
Operating System Concepts – 9th Edition 16.27 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Programming Environment Virtualization
Also not-really-virtualization but using same techniques, providing
similar features
Programming language is designed to run within custom-built
virtualized environment
For example Oracle Java has many features that depend on
running in Java Virtual Machine (JVM)
In this case virtualization is defined as providing APIs that define a
set of features made available to a language and programs written
in that language to provide an improved execution environment
JVM compiled to run on many systems (including some smart
phones even)
Programs written in Java run in the JVM no matter the underlying
system
Similar to interpreted languages
Operating System Concepts – 9th Edition 16.28 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Emulation
Another (older) way for running one operating system on a different
operating system
Virtualization requires underlying CPU to be same as guest was
compiled for
Emulation allows guest to run on different CPU
Necessary to translate all guest instructions from guest CPU to native
CPU
Emulation, not virtualization
Useful when host system has one architecture, guest compiled for
other architecture
Company replacing outdated servers with new servers containing
different CPU architecture, but still want to run old applications
Performance challenge – order of magnitude slower than native code
New machines faster than older machines so can reduce slowdown
Very popular – especially in gaming where old consoles emulated on
new
Operating System Concepts – 9th Edition 16.29 Silberschatz, Galvin and Gagne ©2013
Types of VMs – Application Containment
Some goals of virtualization are segregation of apps, performance and
resource management, easy start, stop, move, and management of
them
Can do those things without full-fledged virtualization
If applications compiled for the host operating system, don’t need
full virtualization to meet these goals
Oracle containers / zones for example create virtual layer between
OS and apps
Only one kernel running – host OS
OS and devices are virtualized, providing resources within zone
with impression that they are only processes on system
Each zone has its own applications; networking stack, addresses,
and ports; user accounts, etc
CPU and memory resources divided between zones
4 Zone can have its own scheduler to use those resources
Operating System Concepts – 9th Edition 16.30 Silberschatz, Galvin and Gagne ©2013
Solaris 10 with Two Zones
user programs user programs user programs
system programs system programs system programs
CPU resources network addresses network addresses
memory resources device access device access
CPU resources CPU resources
memory resources memory resources
zone 1 zone 2
virtual platform
global zone device management
zone management
Solaris kernel
network addresses
device … device
Operating System Concepts – 9th Edition 16.31 Silberschatz, Galvin and Gagne ©2013
OS Component – Live Migration
Taking advantage of VMM features leads to new functionality not found on
general operating systems such as live migration
Running guest can be moved between systems, without interrupting user
access to the guest or its apps
Very useful for resource management, maintenance downtime windows, etc
1. The source VMM establishes a connection with the target VMM
2. The target creates a new guest by creating a new VCPU, etc
3. The source sends all read-only guest memory pages to the target
4. The source sends all read-write pages to the target, marking them as
clean
5. The source repeats step 4, as during that step some pages were probably
modified by the guest and are now dirty
6. When cycle of steps 4 and 5 becomes very short, source VMM freezes
guest, sends VCPU’s final state, sends other state details, sends final dirty
pages, and tells target to start running the guest
4 Once target acknowledges that guest running, source terminates guest
Operating System Concepts – 9th Edition 16.32 Silberschatz, Galvin and Gagne ©2013
Live Migration of Guest Between Servers
0 – Running 1 – Establish 2 – Create
Guest Source Guest Target
VMM Source
VMM Target
3 – Send R/O Pages
4 – Send R/W Pages
5 – Send Dirty Pages (repeatedly)
7 – Terminate Guest Target running 6 – Running
Guest Source Guest Target
Operating System Concepts – 9th Edition 16.33 Silberschatz, Galvin and Gagne ©2013
End of Chapter 16
Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013