![Page 1: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/1.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH technical deep-dive
George Dunlap
Edinburg – 21-23 October, 2013
![Page 2: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/2.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Intro: PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 2 / 25
![Page 3: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/3.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
![Page 4: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/4.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
![Page 5: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/5.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 3 / 25
![Page 6: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/6.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
![Page 7: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/7.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
![Page 8: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/8.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
![Page 9: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/9.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 4 / 25
![Page 10: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/10.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PV
I PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)
I Qemu processI Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 11: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/11.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in Linux
I 64-bit hypercalls
I Issues with HVM (PVHVM)
I Qemu processI Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 12: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/12.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)
I Qemu processI Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 13: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/13.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)
I Qemu processI Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 14: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/14.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)I Qemu process
I Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 15: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/15.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)I Qemu processI Legacy boot
I Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 16: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/16.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PV and HVM
I Issues with PVI PVMMU in LinuxI 64-bit hypercalls
I Issues with HVM (PVHVM)I Qemu processI Legacy bootI Devices emulated in Xen
Edinburg – 21-23 October, 2013 PVH technical deep-dive 5 / 25
![Page 17: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/17.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH
I “PV in an HVM container”
I Goal: take the best aspects of PV and HVM
I Written by Mukesh Rathor @ Oracle
I Significant revisions by George Dunlap @ Citrix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
![Page 18: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/18.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH
I “PV in an HVM container”
I Goal: take the best aspects of PV and HVM
I Written by Mukesh Rathor @ Oracle
I Significant revisions by George Dunlap @ Citrix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
![Page 19: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/19.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH
I “PV in an HVM container”
I Goal: take the best aspects of PV and HVM
I Written by Mukesh Rathor @ Oracle
I Significant revisions by George Dunlap @ Citrix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
![Page 20: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/20.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH
I “PV in an HVM container”
I Goal: take the best aspects of PV and HVM
I Written by Mukesh Rathor @ Oracle
I Significant revisions by George Dunlap @ Citrix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 6 / 25
![Page 21: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/21.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Xen’s perspective
I Begin with HVM guest
I Disable HVM-specific things not needed
I Start & pin in 64-bit paged mode
I Enable PV path for a handful of things
Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
![Page 22: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/22.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Xen’s perspective
I Begin with HVM guest
I Disable HVM-specific things not needed
I Start & pin in 64-bit paged mode
I Enable PV path for a handful of things
Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
![Page 23: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/23.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Xen’s perspective
I Begin with HVM guest
I Disable HVM-specific things not needed
I Start & pin in 64-bit paged mode
I Enable PV path for a handful of things
Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
![Page 24: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/24.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Xen’s perspective
I Begin with HVM guest
I Disable HVM-specific things not needed
I Start & pin in 64-bit paged mode
I Enable PV path for a handful of things
Edinburg – 21-23 October, 2013 PVH technical deep-dive 7 / 25
![Page 25: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/25.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Things disabled
I Device model (qemu)
I Emulated hardware (APIC, PIT, etc)
I Nested HVM
I MSIX
Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
![Page 26: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/26.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Things disabled
I Device model (qemu)
I Emulated hardware (APIC, PIT, etc)
I Nested HVM
I MSIX
Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
![Page 27: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/27.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Things disabled
I Device model (qemu)
I Emulated hardware (APIC, PIT, etc)
I Nested HVM
I MSIX
Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
![Page 28: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/28.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Things disabled
I Device model (qemu)
I Emulated hardware (APIC, PIT, etc)
I Nested HVM
I MSIX
Edinburg – 21-23 October, 2013 PVH technical deep-dive 8 / 25
![Page 29: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/29.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Pinning in 64-bit
I Set initial values ofr cr0, cr4, EFER
I Certain paging things have to happen at start of day
I Disable writes to EFER
I Don’t allow guest to change paging-related cr0 bits
Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
![Page 30: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/30.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Pinning in 64-bit
I Set initial values ofr cr0, cr4, EFER
I Certain paging things have to happen at start of day
I Disable writes to EFER
I Don’t allow guest to change paging-related cr0 bits
Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
![Page 31: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/31.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Pinning in 64-bit
I Set initial values ofr cr0, cr4, EFER
I Certain paging things have to happen at start of day
I Disable writes to EFER
I Don’t allow guest to change paging-related cr0 bits
Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
![Page 32: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/32.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: Pinning in 64-bit
I Set initial values ofr cr0, cr4, EFER
I Certain paging things have to happen at start of day
I Disable writes to EFER
I Don’t allow guest to change paging-related cr0 bits
Edinburg – 21-23 October, 2013 PVH technical deep-dive 9 / 25
![Page 33: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/33.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PV paths
I Enable PV hypercalls
I PV e820 map
I PVH VCPU boot
I PV CPUID
I PV PIO
Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
![Page 34: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/34.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PV paths
I Enable PV hypercalls
I PV e820 map
I PVH VCPU boot
I PV CPUID
I PV PIO
Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
![Page 35: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/35.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PV paths
I Enable PV hypercalls
I PV e820 map
I PVH VCPU boot
I PV CPUID
I PV PIO
Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
![Page 36: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/36.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PV paths
I Enable PV hypercalls
I PV e820 map
I PVH VCPU boot
I PV CPUID
I PV PIO
Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
![Page 37: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/37.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PV paths
I Enable PV hypercalls
I PV e820 map
I PVH VCPU boot
I PV CPUID
I PV PIO
Edinburg – 21-23 October, 2013 PVH technical deep-dive 10 / 25
![Page 38: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/38.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Linux’s perspective
I xen hvm domain() false, xen pv domain() true
I Act natural (get rid of PV special cases)
I Autotranslate side effects
I Use PVHVM callback vector setup
I VCPU bring-up
Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
![Page 39: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/39.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Linux’s perspective
I xen hvm domain() false, xen pv domain() true
I Act natural (get rid of PV special cases)
I Autotranslate side effects
I Use PVHVM callback vector setup
I VCPU bring-up
Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
![Page 40: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/40.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Linux’s perspective
I xen hvm domain() false, xen pv domain() true
I Act natural (get rid of PV special cases)
I Autotranslate side effects
I Use PVHVM callback vector setup
I VCPU bring-up
Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
![Page 41: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/41.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Linux’s perspective
I xen hvm domain() false, xen pv domain() true
I Act natural (get rid of PV special cases)
I Autotranslate side effects
I Use PVHVM callback vector setup
I VCPU bring-up
Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
![Page 42: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/42.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH from Linux’s perspective
I xen hvm domain() false, xen pv domain() true
I Act natural (get rid of PV special cases)
I Autotranslate side effects
I Use PVHVM callback vector setup
I VCPU bring-up
Edinburg – 21-23 October, 2013 PVH technical deep-dive 11 / 25
![Page 43: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/43.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 44: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/44.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 45: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/45.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 46: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/46.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 47: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/47.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 48: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/48.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 49: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/49.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
I No PV IDT
I No PV irq ops
I No PV CPUID
I Native Syscall/sysenter
I No VM assists
I No event / failsafe callbacks
I No need to set IOPL
Edinburg – 21-23 October, 2013 PVH technical deep-dive 12 / 25
![Page 50: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/50.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
MMUOPS:
I No need to pin pagetables
I No pfn/mfn conversion
I No need to special-case PT protections
I Only pvmmu op: flush tlb others
Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
![Page 51: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/51.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
MMUOPS:
I No need to pin pagetables
I No pfn/mfn conversion
I No need to special-case PT protections
I Only pvmmu op: flush tlb others
Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
![Page 52: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/52.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
MMUOPS:
I No need to pin pagetables
I No pfn/mfn conversion
I No need to special-case PT protections
I Only pvmmu op: flush tlb others
Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
![Page 53: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/53.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Things disabled
MMUOPS:
I No need to pin pagetables
I No pfn/mfn conversion
I No need to special-case PT protections
I Only pvmmu op: flush tlb others
Edinburg – 21-23 October, 2013 PVH technical deep-dive 13 / 25
![Page 54: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/54.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PV
I Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVH
I Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 55: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/55.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by Xen
I Real MFN in pagetablesI “Map this page here” hypercall
I PVH
I Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 56: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/56.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetables
I “Map this page here” hypercall
I PVH
I Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 57: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/57.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVH
I Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 58: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/58.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVH
I Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 59: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/59.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVHI Pagetables controlled by guest
I gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 60: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/60.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVHI Pagetables controlled by guestI gpfn in pagetables
I Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 61: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/61.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I PVI Pagetables controlled by XenI Real MFN in pagetablesI “Map this page here” hypercall
I PVHI Pagetables controlled by guestI gpfn in pagetablesI Every mapped page must be in the p2m
Edinburg – 21-23 October, 2013 PVH technical deep-dive 14 / 25
![Page 62: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/62.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I “Special pages”: grant frame, make a hole in the p2m
I Need to add p2m entries to map foreign pages
I Grant table: similar to HVM (but not the same)
Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
![Page 63: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/63.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I “Special pages”: grant frame, make a hole in the p2m
I Need to add p2m entries to map foreign pages
I Grant table: similar to HVM (but not the same)
Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
![Page 64: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/64.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: Autotranslate
I “Special pages”: grant frame, make a hole in the p2m
I Need to add p2m entries to map foreign pages
I Grant table: similar to HVM (but not the same)
Edinburg – 21-23 October, 2013 PVH technical deep-dive 15 / 25
![Page 65: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/65.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: VCPU bring-up
I Brought up via hypercalls (PV)
I Guest in control of IDT, Xen can’t guarantee loading properly
I Only set GS by default
I All other state must be set in on-cpu boot code
Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
![Page 66: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/66.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: VCPU bring-up
I Brought up via hypercalls (PV)
I Guest in control of IDT, Xen can’t guarantee loading properly
I Only set GS by default
I All other state must be set in on-cpu boot code
Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
![Page 67: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/67.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: VCPU bring-up
I Brought up via hypercalls (PV)
I Guest in control of IDT, Xen can’t guarantee loading properly
I Only set GS by default
I All other state must be set in on-cpu boot code
Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
![Page 68: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/68.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Linux: VCPU bring-up
I Brought up via hypercalls (PV)
I Guest in control of IDT, Xen can’t guarantee loading properly
I Only set GS by default
I All other state must be set in on-cpu boot code
Edinburg – 21-23 October, 2013 PVH technical deep-dive 16 / 25
![Page 69: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/69.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH: Not yet working
I 32-bit
I vtsc
I shadow mode
I vcpu hotplug
Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
![Page 70: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/70.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH: Not yet working
I 32-bit
I vtsc
I shadow mode
I vcpu hotplug
Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
![Page 71: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/71.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH: Not yet working
I 32-bit
I vtsc
I shadow mode
I vcpu hotplug
Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
![Page 72: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/72.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
PVH: Not yet working
I 32-bit
I vtsc
I shadow mode
I vcpu hotplug
Edinburg – 21-23 October, 2013 PVH technical deep-dive 17 / 25
![Page 73: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/73.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: HVM paths
I Original idea: “Lightweight” container
I Reality: Large amount of code necessary for minimumfunctionality
I Very large amount of code duplication
I Current patch: use HVM paths, with special cases for PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
![Page 74: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/74.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: HVM paths
I Original idea: “Lightweight” container
I Reality: Large amount of code necessary for minimumfunctionality
I Very large amount of code duplication
I Current patch: use HVM paths, with special cases for PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
![Page 75: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/75.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: HVM paths
I Original idea: “Lightweight” container
I Reality: Large amount of code necessary for minimumfunctionality
I Very large amount of code duplication
I Current patch: use HVM paths, with special cases for PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
![Page 76: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/76.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: HVM paths
I Original idea: “Lightweight” container
I Reality: Large amount of code necessary for minimumfunctionality
I Very large amount of code duplication
I Current patch: use HVM paths, with special cases for PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 18 / 25
![Page 77: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/77.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config spaceI “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 78: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/78.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config spaceI “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 79: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/79.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config spaceI “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 80: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/80.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config space
I “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 81: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/81.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config spaceI “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 82: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/82.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Direct access permitted
I pv PIT
I cmos idx
I pci config spaceI “Abusing” IO with SMM
I Execute exact instruction with guest GPRs
Edinburg – 21-23 October, 2013 PVH technical deep-dive 19 / 25
![Page 83: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/83.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I Problems
I Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I Possibilities
I Need 2 sets of access controls: For user procs, and for guestI PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 84: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/84.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guests
I Race condition because of double checking
I Possibilities
I Need 2 sets of access controls: For user procs, and for guestI PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 85: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/85.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I Possibilities
I Need 2 sets of access controls: For user procs, and for guestI PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 86: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/86.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I Possibilities
I Need 2 sets of access controls: For user procs, and for guestI PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 87: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/87.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I PossibilitiesI Need 2 sets of access controls: For user procs, and for guest
I PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 88: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/88.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I PossibilitiesI Need 2 sets of access controls: For user procs, and for guestI PV only has one
I PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 89: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/89.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Issues: PIO instructions
I ProblemsI Ugly changes to allow PV paths to emulate for PVH guestsI Race condition because of double checking
I PossibilitiesI Need 2 sets of access controls: For user procs, and for guestI PV only has oneI PVH: Have two, so may not need to emulate anymore
Edinburg – 21-23 October, 2013 PVH technical deep-dive 20 / 25
![Page 90: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/90.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table update
I HAP: Memory writeI Shadow: Trip through Xen
I TLB effectiveness
I HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 91: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/91.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory write
I Shadow: Trip through Xen
I TLB effectiveness
I HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 92: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/92.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectiveness
I HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 93: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/93.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectiveness
I HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 94: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/94.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entries
I Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 95: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/95.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 96: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/96.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement cost
I Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 97: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/97.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement costI Shadow: Worst case, 4 memory reads
I HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 98: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/98.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement costI Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 99: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/99.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement costI Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?
I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 100: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/100.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement costI Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?I Kernel build: Shadow 30% slower than HAP
I SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 101: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/101.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
HAP vs Shadow
I Page table updateI HAP: Memory writeI Shadow: Trip through Xen
I TLB effectivenessI HAP: Superpages allow 2MiB / 1GiB TLB entriesI Shadow: Still only 4k TLB entries
I TLB replacement costI Shadow: Worst case, 4 memory readsI HAP: 9-16 memory reads
I Does this matter?I Kernel build: Shadow 30% slower than HAPI SpecJBB: Shadow 30% faster than HAP
Edinburg – 21-23 October, 2013 PVH technical deep-dive 21 / 25
![Page 102: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/102.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
![Page 103: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/103.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
![Page 104: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/104.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
![Page 105: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/105.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Overview
I PV, HVM, and PVH
I PVH from Xen’s perspective
I PVH from Linux’s perspective
I Issues in PVH
Edinburg – 21-23 October, 2013 PVH technical deep-dive 22 / 25
![Page 106: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/106.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
![Page 107: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/107.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
![Page 108: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/108.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Goal
I Give you a technical overview of PVH, so that...
I you can understand the characteristics, advantages, anddisadvantages...
I and approach the code to improve and fix
Edinburg – 21-23 October, 2013 PVH technical deep-dive 23 / 25
![Page 109: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/109.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Questions
Questions?
Feedback? [email protected]
Check out our blog: http://blog.xen.org/
Edinburg – 21-23 October, 2013 PVH technical deep-dive 24 / 25
![Page 110: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/110.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PVH hypercalls
I memory op
I console io
I vcpu op
I mmuext op
Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
![Page 111: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/111.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PVH hypercalls
I memory op
I console io
I vcpu op
I mmuext op
Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
![Page 112: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/112.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PVH hypercalls
I memory op
I console io
I vcpu op
I mmuext op
Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25
![Page 113: PVH technical deep-dive · 2017. 12. 14. · PVH technical deep-dive George Dunlap Edinburg { 21-23 October, 2013. IntroPV and HVMPVH and XenPVH and LinuxPVH IssuesSpare slides Intro:](https://reader036.vdocument.in/reader036/viewer/2022070214/611308f408c3c20c85544c21/html5/thumbnails/113.jpg)
Intro PV and HVM PVH and Xen PVH and Linux PVH Issues Spare slides
Xen: PVH hypercalls
I memory op
I console io
I vcpu op
I mmuext op
Edinburg – 21-23 October, 2013 PVH technical deep-dive 25 / 25