#vFORUMAU#vFORUMAU
Kit Colbert, VMwareKarim Awan, VMware
Project Pacific Technical Overview:Unifying vSphere and Kubernetes
#vFORUMAU
Disclaimer
This presentation may contain product features or functionality that are currently under development.
This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.
This information is confidential.
The information in this presentation is for informational purposes only and may not be incorporated into any contract. There is no commitment or obligation
to deliver any items presented herein.
#vFORUMAU
BusinessApplications
TOMORROW
IndustrialIoT
Business processautomation
Next-generationstorefronts
Advanced analytics
Data-DefinedBusiness Processes
Self-serviceexperiences
In The Next Five YearsThan In The Last 40 Years
More applications andsolutions will be deployed
Modern Application Deployments are Accelerating
TODAY
#vFORUMAU
Kubernetes Cluster
Node Node Node
Control Plane
Legacy App
VM
Database
VM
VM
VM
A Modern Application Is Complex…
My Application
Serverless
Function 1 Function 2
#vFORUMAU
Kubernetes Cluster
Node Node Node
Control Plane
Existing App
VM
Database
VM
VM
VM
Serverless
Function 1 Function 2
…Which Introduces Many Challenges…
My Application
Developer IT Ops
How do I deploy
this app?
How do I operate it
on day 2?
What tools can I
use with it?
How do I ensure
the availability of this app?
How do I ensure
it’s security?
How do I deliver
quality of service?
How do I control
cost of our infrastructure?
#vFORUMAU
vSphere
VMware as the Platform That Connects Developers and OperationsPower the developer, Assure the admin
Workloads
Operations Developers
Collaboration
DeployManage
WorkloadsWorkloads
Performance
Security
Availability
Cost
Diagnostics
Code
Test
Deploy
Support
#vFORUMAU
Kubernetes Cluster
Node Node Node
Control Plane
Legacy App
VM
Database
VM
VM
VM
Serverless
Function 1 Function 2
Using Kubernetes to Manage Workloads!
My Application
kind: KubernetesCluster
apiVersion: vks.vmware.com/v1
metadata:
name: My Application
spec:
topology:
workers:
count: 3
class: small
distribution: v1.14.1
kind: Pod
apiVersion: v1
metadata:
name: Function 1
spec:
containers:
- name: func1
image: func1
ports:
- containerPort: 80
kind: VirtualMachine
apiVersion: vms.vmware.com/v1
metadata:
name: LegacyApp
spec:
className: large
imageName: my-app.ova
powerState: poweredOn
policy:
restartPolicy: OnFailure
kind: HanaDatabase
apiVersion: hana.sap.com/v1
metadata:
name: ERP database
spec:
nodes: 3
class: extra-large
#vFORUMAU
Namespace
Kubernetes Cluster
Node Node Node
Control Plane
Legacy App
VM
Database
VM
VM
VM
Serverless
Function 1 Function 2
Namespaces as the Unit of Management
My ApplicationSecurity
• Encrypt all persistent data
• Disallow all ports but 443
• Audit developer changes
Availability
• Failures to tolerate: 2
• Disaster recovery site: us-east
• Hourly snapshots to backup
Access controls
• Users in group app-admin: Write
• Users in group ops: Read Only
• Disallow MySQL
Quality of Service
• Priority: High
• Reserved vCPUs: 128
• Reserved Memory: 1 TB
#vFORUMAU
Transforming vSphere into the App Platform of the Future
Project Pacific: Rearchitecting vSphere with Native Kubernetes
vSphere
Supervisor Kubernetes Cluster
vCenter
ESXiCluster
Networking Storage
Kubernetes cluster
Virtual
machinesNative
pods
Developer IT Operator
vSphere with Native Kubernetes | App-focused Management | Dev & IT Ops Collaboration
Improved Economics | Control at Scale | Increased Velocity
App
BETA
#vFORUMAU
vSphere with Native Kubernetes
vSphere
Supervisor Kubernetes Cluster
vCenter
ESXiCluster
Networking StorageDeveloper IT Operations
Application Application Application
Kubernetes Clusters
Virtual Machines
Native Pods
Virtual Machines
Persistent Memory
GPU
Accelerators
• Embed Kubernetes into the control plane of vSphere, unifying control of compute, network and storage resources
• Deliver Kubernetes clusters as a service to developers
• Converge VMs and containers using the new vSphere Native Pods that are high performing, secure and easy to consume
#vFORUMAU
Application-Focused Management
• App level control for applying policies, quota and role-based access to Developers
• Apply vSphere features (HA, vMotion, DRS) at the app level and to the containers
• Unified visibility in vCenter for Kubernetes clusters, containers and existing VMs
Application
Kubernetes Clusters
Virtual Machines
Native Pods
#vFORUMAU
Dev & IT Ops Collaboration
vSphere
Supervisor Kubernetes Cluster
vCenter
ESXiCluster
Networking StorageDeveloper IT Operations
• Developers use Kubernetes APIs to access the SDDC
• IT operators use vSphere tools to deliver Kubernetes clusters to developers
• Consistent view between Dev and Ops via Kubernetes constructs in vSphere
#vFORUMAU
Enable Kubernetes in vSphere with Supervisor Clusters
vCenter
ESXi
VM
ESXi
VM
VM
ESXi
VM
VM
ESXi Cluster
hostd hostd hostd
VI Admin
VM
#vFORUMAU
Enable Kubernetes in vSphere with Supervisor Clusters
vCenter
ESXi
Spherelet
K8s Master VM
ESXi
Spherelet
ESXi
Spherelet
ESXi Cluster
hostd hostd hostd DevOps
VI Admin
Pod
CRX VM VMPod
Pod
Pod
Supervisor Kubernetes Cluster
#vFORUMAU
Workload Platform
On-premises | Hybrid cloud | Public cloud
Supervisor Kubernetes Cluster
ESXi Cluster Networking Storage
vCenterDeveloper
IT Ops
A Kubernetes Control Plane for the SDDC
#vFORUMAU
Workload Platform
On-premises | Hybrid cloud | Public cloud
ESXi Cluster Networking Storage
vCenterDeveloper
IT Ops
Kubernetes
Service
Kubernetes as a Service
Supervisor Kubernetes Cluster
#vFORUMAU
Workload Platform
On-premises | Hybrid cloud | Public cloud
Kubernetes
Service
ESXi Cluster Networking Storage
vCenterDeveloper
IT Ops
Virtual Machine
Service
Virtual Machine Service
Supervisor Kubernetes Cluster
#vFORUMAU
Workload Platform
On-premises | Hybrid cloud | Public cloud
Kubernetes
Service
ESXi Cluster Networking Storage
vCenterDeveloper
IT Ops
Virtual Machine
Service
Application
Services
Ecosystem
Anything as a Service
Supervisor Kubernetes Cluster
#vFORUMAU
Reconciliation in action
What is Kubernetes Controller ?
State of the worldController ManagerAPI ServerDatabase
(etcd)
Foo Controller FooFoo
Foo
#vFORUMAU
What Is a Kubernetes Operator?
Custom Resource Definition (CRD)
Custom Controller
Kubernetes Operator
Custom Resource Definition = Definition of New Object Managed Thru Kubernetes API
Custom Controller = Manages the Lifecycle of Custom Resource defined by the CRD
Operator Pattern
#vFORUMAU
• Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management.
• Requires a running Kubernetes Cluster to provide the Master API
• Uses Custom Resource Definitions (CRDs) to manage Component Objects
• Infrastructure Providers deliver “actuators” that implement VM lifecycle
• Kubeadm (another k8 lifecycle project) to actually bootstrap the individual Nodes
What Is Cluster-API?
Cluster API Controllers
User
Cluster Spec
Cluster: Dev
Machine Class: Small + Large VMs
Machine Control Plane: 0/1/2…
Machine Deployment:
Mix of Small/Large VMs Possible
Cluster API
Kubernetes
Cluster
Kubernetes Management Cluster
#vFORUMAU
Workload platform architecture
Namespace
Guest Cluster
NamespaceNamespace
Pod PodVM VM VM VMVM
SDDC
Supervisor Cluster Supervisor Cluster
Namespace Namespace
Pod Pod Pod Pod PodPod Pod
VM Operator
VM Operator
Cluster API
Guest Clusters Manager
Cluster API
Guest Clusters Manager
VM VM VM VM
Pod
Namespace
Pod
Namespace
Pod
Control Plane
Worker Worker Worker
#vFORUMAU
User Namespace
Guest Cluster
Guest Cluster Manager
Cluster API ControllersPod Pod
Service Namespaces
Cluster API Provider
VM Operator
Pod Pod
Guest Cluster Resource
Cluster Resource
Machine Resources
VirtualMachine ResourcesVM VM VM
AuthCNICSI
vCenter
UI Integration
UI Integration
UI Integration
Supervisor Cluster
ESXi ESXi ESXi ESXi
……
User Namespaces
Layered approach
Guest Kubernetes Clusters
#vFORUMAU
Multi-tenancy with supervisor cluster namespaces
Each Namespace has its own Resource Pool
Resource Isolation with Quota for CPU/Memory/Storage
All Workloads in a Namespace are bounded by Namespace Quota
• Guest Clusters
• Native Pods
• Virtual Machines
SDDC
Supervisor Cluster Supervisor Cluster
Namespace Namespace Namespace Namespace
#vFORUMAU
Supervisor Cluster Network Topology and Isolation
It leverages NSX network capabilities
Supervisor Clusters are isolated with Tier-1 Routers and Distributed Firewalls
Namespaces are isolated with vSwitches and Distributed Firewalls
Inbound Traffic are denied for all namespaces by default
Guest Clusters can use your preferred overlay (Calico by default)
Supervisor Cluster
Namespace
Supervisor Cluster
Namespace Namespace Namespace
Physical Network Fabric
Uplinks
Tier-0 Router
Tier-1 Router
vSwitch
VMPod VM VMVMPod Pod Pod
#vFORUMAU
Cloud Native Storage
Unified management of cloud native storage
vSphere SPBM policies become Storage Classes in Supervisor cluster
Works across local, hyperconverged and shared storage
Leverage full vSphere ecosystem of storage partners
Exposes persistent volumes as paravirtualized drivers in Supervisor Namespace
PV PV PV PV PV PV
Namespace Namespace
Cloud Native Storage Control Plane
First Class DisksStorage Policy Based Mgmt
vSAN File Services
VMFS vSAN NFS vVOL
#vFORUMAU
Linux Node
Pod Pod
Container
Container
Container
Linux Kernel
Memory StorageCPU
Native pods
Better security and resource isolation
Better performance
Provide serverless experience to DevOps
Provide workload visibility to VI Admins
Native Pod
Container
Container
Linux Kernel
Memory StorageCPU
Native Pod
Container
Linux Kernel
Memory StorageCPU
#vFORUMAU
Native pod technical details
Native Pod runtime for ESXi
Secure isolation of Pods
Light weight and fast
Advanced resource management
Micro-segmented Pod network
Container
ImageNative Pod
Container 1 Container 2
Container Engine
Pod Agent Spherelet
hostd
Container Image
Ephemeral Disk
Container ImagePersistent Volume
NSX vSwitch
vNIC
#vFORUMAU
Run
Build Manage
Modern Apps Portfolio
Cloud Native Platform
| Expert Services |
Kubernetes Grid
Project Pacific | VMware PKS
Tanzu Mission Control
Optimization
[powered by CloudHealth]
Observability
[powered by Wavefront]
#vFORUMAU
1. Sign-up for Project Pacific Beta: Link
2. VMworld US & EU Session Recordings: Link
3. Project Pacific Website: Link
4. Project Pacific Architecture @ Tech Field Day: Link
5. Project Pacific Blog: Link
6. Project Pacific HOL – Coming Soon
Next Steps