®
Xen Community Update
Ian Pratt
Chairman of Xen.org and
SVP Products at Bromium 1
® Outline
• Year Review
• Secure Isolation
• Xen Differentiators
• Reference Architecture Proposal
2
® Xen.Org Changes
• Welcome Lars Kurth as new Community
Manager!
– Thanks to Stephen Spector for a great job done
• Lars’ Mission: Encourage more vendor
engagement and co-ordination and co-
operation in the community; Foster closer
links with related OSS communities
3
® Development Activity
4
0
1000
2000
3000
4000
5000
6000
7000
8000
Xen-Devel Mailing List Activity
® Xen.Org Blog Activity
5
® Calendar Review
• Aug 2010: XenDirections in Boston, USA
• Sep 2010: XenDirections in Sao Paulo, Brazil
• Postponed from Nov 2011: XenSummit
Seoul, South Korea
• March 2011: Xen Hackathon, Cambridge UK
• July 2011: OSCON, Portland, USA
• Summer 2011: 6 Google Summer of Code
students working on Xen
6
® Xen 4.1 Release – 21 March 2011
• Key Features
– New “XL” lightweight control stack
– Memory Introspection API
– CPU Pools for partitioning
– Very large system support (>255 CPUs)
– Experimental: credit2 scheduler; Remus FT;
Emergency swap
7
® Community Interactions
• Linux
– Privileged domain support upstream in Linux 3.0
– Guest optimizations: use the optimal
combination of h/w and s/w virtualization
• QEMU
– Xen qemu target now upstream
• OpenStack
– XCP integration with OpenStack 8
® Secure Isolation
• Maintaining isolation between VMs is priority #1 – Essential for Cloud, and for Client
– Spatial and Temporal isolation
• Use good software engineering practice – Thin hypervisor: minimize code running with privilege
– Disaggregate and de-privilege functionality into dedicated Service VMs
– Narrow interfaces between components
– Hypervisors are simpler than OSes, simpler than OS kernels
– Use modern high-level languages where possible
• New hardware technologies help – VT-x, VT-d, EPT: reduce software complexity, enhanced protection
– TPM/TXT: Enable Dynamic Root of Trust
9
® XenClient XT / Qubes OS
• First products configured to take advantage
of the security benefits of Xen’s architecture
• Isolated Driver Domains
• QEMU Emulation Domains
• Service VMs (global and per-guest)
• Xen Security Modules / SElinux
• Measured Launch (TXT) 10
® Typical Xen Configuration
Event Channel Virtual MMU Virtual CPU Control IF
Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
Native
Device
Driver
GuestOS
Device
Manager &
Control s/w
VM0
GuestOS
VM1
Front-End
Device Drivers
GuestOS
Applications
VM2
Device
Emulation
GuestOS
Applications
VM3
Safe HW IF
Xen Virtual Machine Monitor
Back-End
Applications
Front-End
Device Drivers
® Xen Driver Domains
Event Channel Virtual MMU Virtual CPU Control IF
Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
Native
Device
Driver
GuestOS
Device
Manager &
Control s/w
VM0
Native
Device
Driver
GuestOS
VM1
Front-End
Device Drivers
GuestOS
Applications
VM2
Device
Emulation
GuestOS
Applications
VM3
IOMMU
Xen Virtual Machine Monitor
Back-End Back-End
® XenClient XT Architecture
13
Xen
Control
Domain
Intel vPro Hardware
Receiv
er
for
XC
Netw
ork
Iso
lation
VP
N
Isola
tion User VM User VM
Service VMs
SELinux
Xen Security Modules
VT-d TXT
VT-x AES-NI
Policy Granularity Policy Granularity
® Disaggregation
• Unique benefit of the Xen architecture:
• Security – Minimum privilege; Narrow interfaces
• Performance – Lightweight e.g. minios directly on hypervisor
– Exploit locality – service VMs see a subset of the machine,
run close to resources with which they interact
• Reliability – Able to be safely restarted
14
® Isolated Driver VMs for High Availability
• First implemented in 2004
• Detect failure e.g.
– Illegal access
– Timeout
• Kill domain, restart – E.g. Just 275ms outage from
failed Ethernet driver
• New work uses restarts to
enhance security
0
50
100
150
200
250
300
350
0 5 10 15 20 25 30 35 40
time (s)
® Proposal
• We should strive to get all Xen products and
deployments to take full advantage of the
Xen architecture
• We need to make this much easier!
• Proposal: define and maintain a reference
architecture and implementation that
embodies best practice recommendations
16
® Reference Architecture
• Define using new technologies
– Latest stable Xen
– Linux 3.x pvops
• Optimization effort required
– Libxl control stack
• For easy consumption by other vendor tool stacks
17
® Target Features
• Network restart-able driver domains – Integrated OpenFlow vswitch
• Storage restart-able driver domains – Also allows easier deployment of new storage options e.g. vastsky, ZFS
• Qemu emulation domains
• Xen Security Modules
• Measured Launch
• Roadmap for enhanced security and performance
features
– E.g. the SR-IOV network plugin / vswitch architecture
18
® Implementation
• Need an initial reference implementation
– Easily consumable by users
• XCP could fulfil this role
– Showcase latest Xen technologies
– Optimized for OpenStack
• Aim to be as kernel/toolstack etc agnostic to
allow easy adoption by all vendors
19
® Summary
• Xen project continues to thrive!
– Great success in Cloud and Client
• Key architectural security, reliability and
performance benefits that are unique to Xen
– We need to do a better job of getting the
message out!
– We need to do a better job of actually taking
advantage of the benefits 20
®
21
® Xen Today
• ~20% enterprise server market share
• >80% of the Public Cloud is Xen based
– World's largest virtualization deployments are Xen based
• Development Community: over 50 Companies,
25 Universities, from 25 Countries, ~250 developers
– More than 20,000 code submissions
• Used in Severs, Desktops, Laptops, Storage Appliances,
Network Appliances and Smart Phones
– x86, IA64, ARM support
®
Xen is great. It’s powerful
and easy to use. But most
important is the very active
community around it.
That was a very big reason
for us in selecting Xen.
Xen Powers the World’s Infrastructure Clouds
Werner Vogels
CTO, Amazon.com ”
“
® Xen Tops Performance Comparisons
Keith Ward, Virtualization Review
“Xen is the Porsche of hypervisors”
“Xen outperforms VMware ESX 3.5 by 41% in user scalability tests.”
The Tolly Group
®
Xen Hypervisor
First and Best to
support new
CPU, chipset,
and Smart IO
Technologies
Pioneers of
OS Para-virtualization
®
Xen 4.0
26
® Xen 4.0
• Released 12 Apr 2010
• Reliability, Availability, Scalability
– Enhanced MCA support, blktap2, netchannel2
• Memory optimizations
• pvops privileged domain support
• Fault tolerance for VMs
27
® Hardware Fault Tolerance
Restart-HA monitors hosts and VMs to keep apps running
Hardware Fault Tolerance with deterministic replay or checkpointing
Xen’s Software-Implemented Hardware Fault Tolerance enables true
High Availability for unmodified applications and operating systems
® Hardware Fault Tolerance
• University of British Columbia’s “Remus” project is
now in xen 4.0
• Smart checkpointing approach yields excellent
performance – VM executes in parallel with checkpoint transmission, with all externally
visible state changes suppressed until checkpoint receipt acknowledged
– Checkpoints delta compressed
• Checkpointing possible across wide-area, even for multi-
vCPU guests
29
® SR-IOV
• SR-IOV: Single Root IO Virtualization – Virtualization friendly IO devices
• High performance, high efficiency, low latency
• Enables even the most demanding applications to
now be virtualized
• Compatible with live relocation via hotplug
• World First, demonstrated at Intel Developer
Forum in September!
30
® SR-IOV NIC Demonstration
31
Dell 10G Switch
NFS Common Storage w/OpenFiler
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
• Full 20Gb/s bi-directional throughput to VMs
• Low latency, High CPU efficiency
• Live relocation between hosts - Even hosts with different NICs
® Network Performance
Type-0
0
5
10
15
20
25
30
35
CP
U (
%)
usercopy
kern
xen1
grantcopy
kern0
xen0basic smart
NIC
SR-IOV
NIC
native
201%
100% 123% 103%
• New Smart NICs reduce CPU overhead substantially
• Care must be taken with SR-IOV NICs to ensure benefits
of VM portability and live relocation are not lost
• Need for an industry standard for “driver plugins”
s/w only
®
Xen Cloud Platform
33
® Xen Cloud Platform (XCP)
• XCP Expands Xen.org’s scope beyond the core
hypervisor, to create a full virtual infrastructure layer for
Cloud deployments – Simplify and streamline use of Xen by Cloud providers and vendors
– Promote greater standardisation of components between vendors
• Advanced virtual infrastructure to enable Virtual Private
Datacenters rather than just Virtual Private Servers – Multi-tenant hosts, networking, storage, etc
– Promote interoperability between xen-based clouds and other clouds
– Drive standards activities via DMTF
34
® August 2009 XCP Announcement
35
®
Where Xen Cloud Platform Fits
Resource Pool
VM Mgt
State
Mgt
State Mgt
State
Mgt
State
VM VM
VM
VM
VM VM
VM VM
Management API
& OVF Format
® XCP 0.2
• Xen 3.4; Linux 2.6.27; optimized dom0 file system
• xapi toolstack – Resource Pools; VM, host, networking and storage
management; snapshots and checkpoints; live and persistent
performance statistics; status alerting; role-based access
control; OVF/CIM support
• Windows PV Drivers; Full installer etc.
• Open vSwitch
37
®
New Open vSwitch
VM
Hypervisor
VM VM VM VM
Hypervisor
VM VM VM VM
Hypervisor
Isolation · Resource control · Multi-tenancy · Visibility · Security
VM VM
• Open Source Virtual Switch maintained at www.openvswitch.org
• Rich layer 2 feature set
®
Distributed vSwitch
Hypervisor Hypervisor Hypervisor
Built-in policy-based ACLs move with VMs
Distributed Virtual Switch
VM VM VM VM VM VM VM VM VM VM VM
Virtual Interface (VIF) {MAC, IP} ACLs
permit tcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq 123
Virtual Interface (VIF) {MAC, IP} ACLs
permit tcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 192.168.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit udp 172.16.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq domain
permit tcp 10.0.0.0 0.0.0.255 10.20.0.0 0.0.0.255 eq 123
®
Distributed vSwitch
Hypervisor Hypervisor Hypervisor
Isolation · Resource control · Multi-tenancy · Visibility · Security
Distributed Virtual Switch
VM VM VM VM VM VM VM VM VM VM VM
Distributed Virtual Switch
Tenant A
Tenant B
® XCP 1.0 Plans
• New Storage Repository plug-ins – For cloud-optimized storage models
• libxenlight integration
• Enhanced vswitch capabilities
• pvops domain0
• Better integration of OVF support
• Secure boot and attestation
• Cloud orchestration and management APIs
• Easier complete build environment 41
®
Xen Client Initiative
42
® The Xen Client Initiative
• Formed in 2007 to develop Xen for desktop and laptop
• Develop enhanced power management, USB, WiFi,
WWAN, 3D Graphics, fingerprint reader, multi-touch, etc
• Support for latest hardware technologies
• Tiny footprint hypervisor, Embeddable in Flash memory
or small disk partition
• Aiming to make virtualization ubiquitous on client
devices...
43
® Client Hypervisor Benefits
• Security, Manageability, Supportability, Auditability
• Building Multi-Level Secure systems – Run multiple VMs with policy controlled information flow
• E.g. Personal VM; Corporate VM; VM for web browsing; VM for banking
– Trusted hypervisor provides secure isolation
• Enables “out-of-band” management and policy
enforcement via Service VMs – Malware detection, remote access, image update, backup, VPN, etc.
Requires a true type-1 hypervisor architecture
Xen is ideally suited to this!
44
®
Xen Hypervisor
User VM1 User VM2
Audio USB
Disk ACPI
GPU
NIC
Xen Client Architecture
Control
Domain
Service
VM
x86 Hardware TXT
TPM
® “Business” & “Personal” Environments
• Allows Local App Installs
• Minimal Management
– Virus Scanner
– Security Patches
• No SLA
– Self-Service Wipe
Business Personal
• Locked Down
• No Local App Installs
• Tightly Managed
• Self-Service Corporate App Installs
® Conclusions
• The Xen Community continues to grow
from strength to strength
• Xen’s architecture makes it #1 in security,
with great performance
– From Cloud to Client
• Xen.org’s role is broadening to develop
whole reference platforms, promote
standards, interopability
47
®
48