xen.org: the past, the present and exciting future
DESCRIPTION
In this talk, John will explore the technology and architecture introduced in the ARM Cortex-A15 processor in support of virtualization. This is the first of multiple processors from ARM that will support true virtualization, and the ability to host existing operating systems binaries without modification. The hardware extensions were defined following careful analysis to address the key virtualization performance limitations of today's solutions while bringing new technologies to the device to better support a virtualized system.TRANSCRIPT
Sponsored by:
&
1
Xen: the Past, Present and Exciting FutureIan PrattChairman of Xen.org,SVP and Founder of Bromium
Outline• Community Update• Xen 4 Review• Xen and the next wave of virtualization
2
COMMUNITY UPDATE
3
2011 Highlights• Inclusion of Xen into Linux 3 (and distros)• New Initiatives:
– Project Kronos– Xen.org Governance– Renewed focus on Xen for ARM
• Successful Community Initiatives– Documentation Day– Google Summer of Code– Hackathons: Cambridge (Citrix) and Munich (Fujitsu)
• Lars Kurth: (not so) new Community Manager
4
Contribution Statistics
5
By Change Sets Contributors to Xen.org
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011*
0.0
500.0
1000.0
1500.0
2000.0
2500.0
3000.0
3500.0
4000.0
4500.0
5000.0
XenARM**PVOPSXCPXen HV
*) End of Sept 2011**) Activity on Development branch (not yet in xen-unstable)
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011*0
20
40
60
80
100
120
140
160
180
200
Individuals
Orgs
2010 & 2011 Contributors (by KLOC)
6
39%
20%
8%
7%
6%
5%
5%
4%4%
2%
Citrix HVCitrix XCPOracleIntelNovellFujitsuAMDIndividualMiscUniversity
2010** 2011** ***
28%
18%
15%
14%
11%
6%
5%3%
1%
Citrix XCPCitrix HVSamsung*NovellOracleAMDIndividualIntelMisc
*) Activity on Development branch (not yet in xen-unstable)**) Includes PVOPS ***) Until Sept 2011
Developer mailing list traffic
7
Conversations, excluding patches
Oct-0
3
Jan-
04
Apr-0
4
Jul-0
4
Oct-0
4
Jan-
05
Apr-0
5
Jul-0
5
Oct-0
5
Jan-
06
Apr-0
6
Jul-0
6
Oct-0
6
Jan-
07
Apr-0
7
Jul-0
7
Oct-0
7
Jan-
08
Apr-0
8
Jul-0
8
Oct-0
8
Jan-
09
Apr-0
9
Jul-0
9
Oct-0
9
Jan-
10
Apr-1
0
Jul-1
0
Oct-1
0
Jan-
11
Apr-1
1
Jul-1
10
500
1000
1500
2000
2500
xen-devel xen-api
Formalized Governance• How to contribute
(had this for a long time, but was poorly documented)
• Election of Maintainers, Committers & Project Leads– Committer Election in September– Jan Beulich (Novell) : Committer on Xen HV project
• 2009 : 107 patches changing 11746 lines of code • 2010 : 147 patches changing 7613 lines of code • 2011 : 130 patches changing 27377 lines of code (as of Sept)
• Project Lifecycle– Xen HV & XCP migrated to new lifecycle
8
Xen.org Web site Activity
9
Blog Traffic/Month:100% average increaseof traffic compared to one year ago
Xen 4.1
Linux 3
Part of SeptPrediction at tip of arrow
Xen 4.0
Website Traffic/Day:Notable traffic increase sinceSeptember (almost double)
Coincides with changes to contentand new content!
Linux 3
Product and project news roundup• Xen support in Linux 3
– More in Linux 3.1 (and subsequent releases)
• Xen support (DomU and Dom0) back in Linux distros– Debian – Ubuntu 11.10 – Fedora 16
• Recent product releases that distribute Xen– Oracle VM 3.0– XenServer 6.0– XenClient 2.0– Beta’s of QubesOS and RC’s of openSuse 12.1
10
Where do we need to improve?
11
Focus for 2012
• New website• Better media presence• More focus on users
(individual & commercial)• More and better documentation• New benchmarks, feature
comparisons, etc.• Formalize volunteer activities such
as “documentation day”
2011 & 2012 Event Calendar • LinuxCon Brazil, Sao Paulo, Nov 17-18• USENIX Lisa, Boston, Dec 4-9• Planning to co-locate XenSummit NA with LinuxCon
(LinuxCon NA, San Diego, Aug 27-28)– Not yet finalized, but should be soon
• OSCON, Portland, July 16-20
12
Community Summary• The Xen Developer community is healthy for a 10 year old project• The inclusion of Xen support into Linux 3.x has made a big impact
– Getting questions by many new users– Building new and productive relationships with many people in the Linux and BSD communities
• Up to now Xen.org was almost exclusively looking after developers– Successful open source communities bring their users and developers together– Xen.org needs to focus on
• Reconnecting to its users • On making it easy to get started with Xen and engage new users
– Many opportunities in Cloud Projects• On better communicating the Xen advantages
• Everybody can help with this!
13
XEN 4.1 REVIEW
14
Xen 4.1 Release – 21 March 2011• Very large system support
– 4 TB; >255 CPUs– Reliability, Availability, Scalability enhancements
• CPU Pools for system partitioning• Page sharing enhancements• Hypervisor emergency paging / compression• New “xl” lightweight control stack• Memory Introspection API• Enhanced SR-IOV support• Software-implemented Hardware Fault Tolerance
15
Hardware Fault Tolerance
X Restart-HA monitors hosts and VMs to keep apps running
X Hardware Fault Tolerance with deterministic replay or checkpointing
Xen’s Software-Implemented Hardware Fault Tolerance enables true High Availability for unmodified applications and operating systems
Hardware Fault Tolerance• University of British Columbia’s “Remus” project is now
in Xen 4• Smart checkpointing approach yields excellent
performance– VM executes in parallel with checkpoint transmission, with all externally visible
state changes suppressed until checkpoint receipt acknowledged– Checkpoints delta compressed
• Checkpointing possible across wide-area, even for multi-vCPU guests
17
Enhanced SR-IOV• SR-IOV: Single Root IO Virtualization
– Virtualization friendly IO devices
• High performance, high efficiency, low latency• Enables even the most demanding applications to now
be virtualized• Compatible with live relocation via hotplug of
acceleration driver “plugin” module– Retain primary benefit of physical hardware abstraction
18
SR-IOV NIC
Dell 10G Switch
NFS Common Storage w/OpenFiler
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
Dell R710 Server
XenServer and Intel 10G SR-IOV NIC
• 80 Gb/s bi-directional aggregate throughput between 4 VM pairs• Low latency, High CPU efficiency• Live relocation between hosts - Even hosts with different NICs
Network Performance
Type-0
0
5
10
15
20
25
30
35
CP
U (
%)
usercopy
kern
xen1
grantcopy
kern0
xen0basic smart
NICSR-IOV
NICnative
201%
100%123% 103%
• New Smart NICs reduce CPU overhead substantially• Care must be taken with SR-IOV NICs to ensure benefits of VM portability and live relocation are not lost
• Need for an industry standard for “driver plugins”
s/w only
THE NEXT VIRTUALIZATION WAVE
21
Security will drive the Next Wave of Virtualization• Security is key requirement for Cloud• Security is the primary goal of virtualization on the Client
– Desktop, Laptops, Smart Phones, etc
• Maintaining isolation between VMs is critical– Spatial and Temporal isolation– Run multiple VMs with policy controlled information flow
• E.g. Personal VM; Corporate VM; VM for web browsing; VM for banking
• Enables “out-of-band” management and policy enforcement– Malware detection, remote access, image update, backup, VPN, etc.
22
Xen Introspection API• Allows a suitably privileged VM to monitor and control
the execution of another VM– Interpose on disk and network IO path– Mark VM memory as immutable, no-execute etc– Inspect/modify CPU and memory state
• Enables robust anti-malware, anti-root kit– Cannot be disabled/bypassed by guest VM
Virtualized can be more secure than physical!
24
Secure Isolation• Use good software engineering practice
– Thin hypervisor: minimize code running with privilege– Disaggregate and de-privilege functionality into dedicated Service VMs– Narrow interfaces between components– Hypervisors are simpler than OSes, simpler than OS kernels – Use modern high-level languages where possible
• New hardware technologies help– VT-x, VT-d, EPT: reduce software complexity, enhanced protection– TPM/TXT: Enable Dynamic Root of Trust
25
XenClient XT / Qubes OS• First products configured to take advantage of the security benefits of
Xen’s architecture• Isolated Driver Domains• Virtual hardware Emulation Domains• Service VMs (global and per-guest)• Xen Security Modules / SElinux• Measured Launch (TXT)
26
Typical Xen Configuration
Event Channel Virtual MMUVirtual CPU Control IF
Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
NativeDeviceDriver
GuestOS
Device Manager & Control s/w
VM0
GuestOS
VM1
Front-EndDevice Drivers
GuestOS
Applications
VM2
Device Emulation
GuestOS
Applications
VM3
Safe HW IF
Xen Virtual Machine Monitor
Back-End
Applications
Front-EndDevice Drivers
Xen Driver Domains
Event Channel Virtual MMUVirtual CPU Control IF
Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
NativeDeviceDriver
GuestOS
Device Manager & Control s/w
VM0
NativeDeviceDriver
GuestOS
VM1
Front-EndDevice Drivers
GuestOS
Applications
VM2
Device Emulation
GuestOS
Applications
VM3
IOMMU
Xen Virtual Machine Monitor
Back-End Back-End
VM3a
Advanced XenClient Architecture
29
Xen
Intel vPro Hardware
Man
agem
ent
Dom
ain
Net
wor
k Is
olat
ion
User VM
Per host/deviceService VMs
Xen Security Modules
VT-d TXT
VT-x AES-NI
Policy Granularity
User VM
Policy Granularity
Dev
ice
Em
ulat
e
VP
N I
sola
tion
Dev
ice
Em
ulat
ion
VP
N I
sola
tion
Per guestService VMs
Con
trol
D
omai
n
Disaggregation• Unique benefit of the Xen architecture:• Security
– Minimum privilege; Narrow interfaces
• Performance– Lightweight e.g. minios directly on hypervisor– Exploit locality – service VMs see a subset of the machine, run close to
resources with which they interact
• Reliability– Able to be safely restarted
30
Isolated Driver VMs for High Availability
• Detect failure e.g.– Illegal access– Timeout
• Kill domain, restart– E.g. Just 275ms outage from
failed Ethernet driver
• New work uses restarts to enhance security
0
50
100
150
200
250
300
350
0 5 10 15 20 25 30 35 40time (s)
Proposal• We should strive to get all Xen products and deployments to
take full advantage of the Xen architecture• We need to make this much easier!• Proposal: define and maintain a reference architecture and
implementation that embodies best practice recommendations
32
Reference Architecture• Define using new technologies
– Latest stable Xen– Linux 3.x pvops
• Optimization effort required– Libxl control stack
• For easy consumption by other vendor tool stacks
33
Target Features• Network restart-able driver domains
– Integrated OpenFlow vswitch
• Storage restart-able driver domains– Also allows easier deployment of new storage options e.g. vastsky, ZFS
• Qemu emulation domains• Xen Security Modules• Measured Launch via TXT• Roadmap for enhanced security and performance features
– E.g. the SR-IOV network plugin / vswitch architecture
34
Implementation• Need an initial reference implementation
– Easily consumable by users• XCP could fulfil this role
– Showcase latest Xen technologies– Optimized for OpenStack
• Aim to be as kernel/toolstack etc agnostic to allow easy adoption by all vendors
35
Summary• Xen project continues to thrive!
– Great success in Cloud and Client• Key architectural security, reliability and performance benefits that are
unique to Xen– We need to do a better job of getting the message out!– We need to do a better job of actually taking advantage of the
benefits in all Xen products
36