cloud foundry docker openstack - leading open source triumvirate
Post on 29-Jul-2015
3.698 Views
Preview:
TRANSCRIPT
© IBM Corporation 1
OpenStack, Docker, Cloud Foundry How does the leading open source triumvirate come together?
Animesh Singh Manuel Silveyra Daniel Krook
© IBM Corporation 3
In the top 5 of open source cloud projects
http://analystpov.com/cloud-computing/top-15-open-source-cloud-computing-technologies-2014-24727
© IBM Corporation 4
IBM Cloud and open technologies At all tiers, IBM is committed to building its cloud on an open cloud architecture
OAuth
OSLC Infrastructure as a Service
Platform as a Service
Software as a Service
API Economy
Cloud operating
environment
Software- defined
environment
TOSCA
http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/
Products and services built on open source and open standards benefit IBM and customers
© IBM Corporation 5
IBM Cloud and open technologies At all tiers, IBM is committed to building its cloud on an open cloud architecture
OAuth
OSLC Infrastructure as a Service
Platform as a Service
Software as a Service
API Economy
TOSCA
http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/
Products and services built on open source and open standards benefit IBM and customers
© IBM Corporation 6
Agenda
• OpenStack overview • Cloud Foundry introduction and architectural overview • Docker introduction and architectural overview • How Cloud Foundry and OpenStack intersect • How OpenStack and Docker intersect • How Cloud Foundry and Docker intersect • How they all come together
© IBM Corporation 8
OpenStack overview
Collection of well integrated IaaS modules for compute, network, and storage • Compute (Nova) • Networking (Neutron) • Object Storage (Swift) • Block Storage (Cinder) • Identity(Keystone) • Image Service (Glance) • Dashboard (Horizon)
OpenStack is the ubiquitous open source cloud computing platform for public and private clouds, regardless of size.
© IBM Corporation 10
Cloud Foundry overview Open Cloud Platform • There is an increasing appetite for cloud-based mobile, social
and analytics applications from line-of-business executives - drives the need for a more open cloud development platform.
Meets Developers’ Needs • Focus on app development, not provisioning VMs, databases,
messaging servers, etc. • Agile development model • Deploy and scale in seconds
Compelling Community • Cloud Foundry has an emerging ecosystem as well as a
mature set of capabilities and robustness
Cloud Foundry is the industry’s Open PaaS and provides a choice of clouds, frameworks, and application services.
© IBM Corporation 11
Cloud Foundry architecture
• The platform is abstracted as a set of large-scale distributed services.
• It uses Cloud Foundry BOSH to operate the underlying infrastructure from the IaaS providers.
• Components are dynamically discoverable and loosely coupled.
• Health is exposed through HTTP endpoints so agents can collect state information and act on it.
© IBM Corporation 13
Docker overview
Build Ship Run Application portability • Run Docker containers unchanged in any environment, on any infrastructure Organizational speed & agility via separation of concerns • Development owns the “inside” of the container – the app and its dependencies • Operations owns the “outside” of the container – host, cluster, and policies for deployment • Each team can add their part of the image (e.g. OS, configuration, security, monitoring tools, etc.)
Service reliability • Faithful representation of app with encapsulated dependencies
Docker is an open platform for building distributed applications for developers and operators.
© IBM Corporation 14
Docker architecture • Docker uses a client-server architecture.
• The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers.
• Both the Docker client and the daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon.
• The Docker client and daemon communicate via sockets or through a RESTful API.
© IBM Corporation 15
Docker containers • A Docker container consists of an operating system,
user-added files, and metadata – Basically a way to run mini operating systems in your host operating system with strong guarantees of isolation
• The Docker image is read-only. When Docker runs a container from an image, it adds a read-write layer on top of the image (using a union file system) in which your application can then run.
• Underlying Technology : Written in Go and makes use of several Linux kernel features
• Namespaces - pid, net, mnt, ipc, etc. • Control Groups - cgroups (memory, cpu, blkio, devices) • Union File Systems - UnionFS (AUFS, btrfs, vfs) • Container Format - libcontainer or LXC
© IBM Corporation 16
Docker containers vs. virtual machines Virtual Machines
• Each virtualized application includes not only the application - which may be only 10s of MB - and the necessary binaries and libraries, but also an entire guest operating system - which may weigh 10s of GB.
Docker
• The Docker Engine container comprises just the application and its dependencies. It runs as an isolated process in userspace on the host operating system, sharing the kernel with other containers
Compared with Hypervisors, Docker which is OS-Level Virtualization:
• CPU Performance => native performance
• Memory Performance => few % for (optional) accounting
• Network Performance => small overhead; can be optimized to zero overhead
• creating a new base image takes a few seconds (copy-on-write)
• apps in different containers can share the same binaries / libs
© IBM Corporation 18
Cloud Foundry and OpenStack integration
Cloud Provider Interface
Cloud
DEA Pool Apps
Build Packs
Service Nodes
Cloud Controller
Service Connector
Messaging
Health Manager
UAA
Router PaaS
IaaS
Integration BOSH
© IBM Corporation 19
BOSH deploys Cloud Foundry resources to OpenStack Deployment Manifest • Release name/version • # VMs, job params • Stemcells to use
Stemcell • Base OS • BOSH agent
Release • Name • Software packages • Config templates • Scripts
BOSH
Cloud Foundry
Virtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software packages
© IBM Corporation 22
Docker in OpenStack via the Nova Driver and Glance
• Docker is hypervisor driver for OpenStack Nova Compute. It was introduced with the Havana release, but lives out-of-tree for Icehouse, Juno, and Kilo.
• The Nova driver embeds a tiny HTTP client which talks with the Docker internal REST API through a Unix socket. It uses the HTTP API to control containers and fetch information about them.
• The driver will fetch images from the OpenStack Image Service (Glance) and load them into the Docker file system. Images may be placed in Glance by exporting them from Docker using the 'docker save' command.
• Well integrated with Horizon UI.
© IBM Corporation 23
Heat template plugin for Docker in OpenStack
• Not a replacement for Docker Nova Driver.
• OpenStack Orchestration (Heat) is a solution for providing orchestration of resources inside OpenStack clouds.
• Using the Heat plugin, users may deploy and manage Docker Containers on top of traditional OpenStack deployments, making it compatible with existing OpenStack clouds.
• It allows to use the whole Docker API from a Heat template, and makes linking of containers easier.
• In the example here, multiple containers may be created and linked together by simply adding more sections like “my_docker_container”. They’re not constrained by the OpenStack APIs and may leverage the full power of the Docker Remote API.
© IBM Corporation 24
User interface integration with Horizon
Horizon UI > Orchestration > Stacks Configure the Stack (parameters of the Heat template)
© IBM Corporation 25
OpenStack Kolla runs OpenStack itself in Containers
• Containers used as building blocks for an OpenStack control plane (compute, storage, network, and others) to simplify the operational experience
• Containerized services become independently manageable atomic microservices
• Container sets group related OpenStack services • Database control • Messaging control • High availability control • OpenStack interface • OpenStack control • OpenStack compute operation • OpenStack network operation • OpenStack storage operation
© IBM Corporation 26
OpenStack Magnum runs Containers on OpenStack in VMs
• Magnum is a Platform-as-a-Service (or Containers-as-a-Service) solution for OpenStack. Joined March 2015. Not just Docker images/containers.
• Magnum combines OpenStack with a pluggable container orchestration engine of your choice (Kubernetes or Docker Swarm so far)
• Users leverage the same cloud account and credentials they use to provision other cloud resources, such as Compute, Storage, Network.
• Different users and project groups are properly isolated from each other so that they have no visibility into, or access to, other containers in the system
© IBM Corporation 29
Cloud Foundry runtime architecture
Eclipse IDE
CLI
Browser
cf push
hBp
Cloud Controller (API)
Router
DEA Pool DEA Pool VM Pool
Apps
Apps
Apps
Apps
© IBM Corporation 30
Cloud Foundry runtime architecture
Eclipse IDE
CLI
Browser
cf push
hBp
Cloud Controller (API)
Router
DEA Pool DEA Pool Droplet ExecuGon
Agent (DEA) VM Pool
NATS (Message bus)
Health Manager
© IBM Corporation 31
Warden Containers in Cloud Foundry architecture
Eclipse IDE
CLI
Browser
cf push
hBp
Cloud Controller (API)
Router
DEA Pool DEA Pool Droplet ExecuGon
Agent (DEA) VM Pool
NATS (Message bus)
Health Manager
Warden Server Container Manager
© IBM Corporation 32
Cloud Foundry services architecture – Docker service
Eclipse IDE
CLI
Browser
cf create-‐service Cloud Controller (API)
Docker Service Broker
Provision Instance (PUT /v2/service instances/:id)
Create Binding (PUT /v2/service_instances/:id
Fetch Catalog (GET /v2/catalog
DEA Pool Docker Backend
Docker instance
Docker instance
DEA Pool DEA Pool
App
App
Docker instance
App cf bind-‐service
© IBM Corporation 33
Current Cloud Foundry architecture is being rewritten
Cloud Controller (API)
Router
DEA Pool DEA Pool Droplet ExecuGon
Agent (DEA) VM Pool
NATS (Message bus)
Health Manager
Warden Server Container Manager
© IBM Corporation 34
Cloud Controller (API)
Router
DEA Pool DEA Pool Droplet ExecuGon
Agent (DEA) VM Pool
NATS (Message bus)
Health Manager
Warden Server Container Manager
What’s being rewritten
© IBM Corporation 35
Why rewrite
Cloud Controller (API)
Router
• Tight Coupling Between CC and rest of the components Between Warden Server and backend
• Triangular Dependencies Between CC, Health Monitor and DEAs
• Domain Specific Apps specific Hard to extend to new domains (e.g. cron-like jobs)
• Platform Specific Custom Linux containers only – Warden containers
© IBM Corporation 36
Rewrite Goal : Run loosely coupled one off tasks and LRPs….
Executor Pool
Run Tasks!
Launch Long Running Processes!
Run Task!
Launch LRP!
© IBM Corporation 37
..using generic recipes across various container backends
Run Task!
Launch LRP!
Executor Pool
Run Tasks!
Launch LRPs!
Rep
Exec Recipes!
Exec Garden
Manage Containers!
Garden Backend
Run Containers!
© IBM Corporation 38
Introducing Diego
Brain
BBS
CELLS
Cloud Foundry Diego
Diego is a distributed system that orchestrates containerized workloads
GARDEN
Garden Garden defines an interface to be implemented by container-
runners, and runs Linux, Widnows and Docker
containers BBS
An up-to-date cache of the state of the Diego cluster
(including a picture-in-time of all desired LRPs, running LRP instances, and inflight Tasks) is maintained in the
BBS (Bulletin Board System/Store).
Brain Schedule Deployments
using Auction, and Monitor the health
© IBM Corporation 39
Diego
Cloud Foundry Diego
Diego is a distributed system that orchestrates containerized workloads
https://github.com/cloudfoundry-incubator/diego-design-notes
© IBM Corporation 41
CC Bridge
Cloud Foundry V3
Cloud Controller (API)
Router Router EmiBer
Doppler/ Traffic Controller
Run App! Stager Stage App!
App Mgr
Run Task!Launch LRP!
© IBM Corporation 42
CC Bridge
Cloud Foundry V3
Cloud Controller (API)
Doppler/ Traffic Controller
Run App! Stager
Stage App!
App Mgr
Run Task!Launch LRP!
Eclipse IDE
CLI
Browser hBp
cf push
cf push -‐docker
cf push –stack windows
Router Router EmiBer
© IBM Corporation 43
Same Cloud Foundry developer experience – with added features
Eclipse IDE
CLI
Browser hBp
cf push
cf push -‐docker
cf push –stack windows
© IBM Corporation 45
What’s possible now
Cloud
DEA Pool Apps
Build Packs
Service Nodes
Cloud Controller
Service Connector
Messaging
Health Manager
UAA
Router PaaS
IaaS
Integration
BOSH
Docker Service Broker
© IBM Corporation 46
What’s possible now
Cloud
DEA Pool Apps
Build Packs
Cloud Controller
Service Connector
Messaging
Health Manager
UAA
Router
Docker Service Nodes
PaaS
IaaS
Integration
BOSH
Docker Service Broker
© IBM Corporation 47
What’s coming next
Cloud DIEGO Pool
Apps
Docker Files/Build Packs
Cloud Controller
Service Connector
Messaging
UAA
Router
Docker Service Nodes
PaaS
IaaS
Integration
BOSH
Docker Service Broker
© IBM Corporation 48
What’s coming next
Cloud DIEGO Pool
Apps
Docker Files/Build Packs
Cloud Controller
Service Connector
Messaging
UAA
Router
Docker Service Nodes
PaaS
IaaS
Integration
BOSH
Docker Service Broker
© IBM Corporation 49
What’s possible future
Cloud DIEGO Pool
Apps
Docker Files/Build Packs
Cloud Controller
Service Connector
Messaging
UAA
Router
Docker Service Nodes
PaaS
IaaS
Integration
BOSH
Docker Service Broker
© IBM Corporation 50
IBM Bluemix • IBM Platform as a
Services offering
• IBM and partner cloud services
• Integrated DevOps with both Browser and Eclipse-based tools
Services
Lifecycle Management
IDS
Application Runtime
Runtimes & Frameworks
Middleware Application Operational Mobile External Data
Node Java Ruby Worklight WebSphere Liberty
Eclipse IDE Application
Composition Environment
Create & Manage Services
Test/Run Test/Run
Explore Services
Explore Services
IBM Bluemix Check In Code Check In Code
Web IDE (Eclipse Orion)
© IBM Corporation 55
IBM “Open by design” projects and products Infrastructure-as-a-Service Features OpenStack Self-serve virtual machines, storage, networking, and other infrastructure services Docker Linux container management with host, cluster, and policy management
Platform-as-a-Service Features Cloud Foundry Lifecycle managed application and service management Docker Application and service container encapsulation and management
IBM products Features
Bluemix (instant runtimes) Hosted Cloud Foundry based PaaS to manage scalable application and services lifecycle
Bluemix (containers) Hosted OpenStack-based environment for build, deployment of Docker containers
Bluemix (virtual machines) Hosted OpenStack-based virtual machine environment
Bluemix Local On premises version of Cloud Foundry and OpenStack based Bluemix
Cloud Manager with OpenStack IBM branded distribution of OpenStack
Cloud Orchestrator IBM branded distribution of OpenStack with patterns and automation
Cloud OpenStack Services Hosted dedicated OpenStack environment
Object Storage and Elastic Storage Easily consumable storage services based on OpenStack components
Docker Hub Enterprise IBM resold and integrated version of a Docker image registry
© IBM Corporation 56
Agenda
• OpenStack overview • Cloud Foundry introduction and architectural overview • Docker introduction and architectural overview • How Cloud Foundry and OpenStack intersect • How OpenStack and Docker intersect • How Cloud Foundry and Docker intersect • How they all come together
© IBM Corporation 57
Questions? Reach out at the following Twitter handles
@AnimeshSingh @DanielKrook @Manuel_Silveyra
Meet the IBM Team at these sessions Monday, May 18th
12:05 pm - A Conversation with Cinder Developers - Jay Bryant 4:40 pm - Tales From the Gate: How Debugging the Gate Helps Your Enterprise - Matt Riedemann 4:40 pm - From Archive to Insight: Debunking Myths of Analytics on Object Stores - Dean Hildebrand, Simon Lorenz 5:30 pm - OpenStack, Docker, and Cloud Foundry - How does the Leading Open Source Triumvirate Come Together - Animesh Singh, Daniel Krook, Manuel Silveyra, Kalonji Bankole
Tuesday, May 19th 11:15 am How to Configure your Cloud and Tempest for Interoperability Testing Catherine Diep 12:05 pm - Past, Present and Future of Fibre Channel in OpenStack - Jay Bryant 2:00 pm - Building a Production Grade PaaS platform like Bluemix on OpenStack, leveraging Container based scalable services - Animesh Singh, James Busche 2:00 pm - Standing Tall in the Room - Sponsored by the Women of OpenStack - Radha Ratnaparkhi 5:30 pm New Advances in Federated Identity and Federated Service Provider Support for OpenStack Clouds - Brad Topol, Steve Martinelli
Wednesday, May 20th
9:50 am - Network Connectivity in a Hybrid OpenStack Cloud - John Kasperski, Vinit Jain 1:50 pm - Leveraging open source tools to gain insight into OpenStack Swift - Dmitry Sotnikov, Michael Factor 1:50 pm - Keystone advanced authentication methods - Steve Martinelli, Henry Nash 2:40 pm - Helping Telcos go Green and save OpEx via Policy - Dilip Krishnaswamy
Thursday, May 21st 9:00 am - Big Data Analytics and Docker: The Thrilla in Manila - Bill Owen, Dean Hildebrand, Michael Hines, Nilesh Bhosale 9:50 am - Role of NFV Research in Open Source and Open Standards - Dilip Krishnaswamy 1:30 pm - On-demand Disaster Recovery (DR) service enablement through Software Defined Environments under hybrid clouds - Venkata Jagana, Ramesh Palakodeti, CV Venugopal, Mike Williams, Ann Corrao 1:30pm - OpenStack Networking: It's time to talk Performance - Bengi Karacali, John Tracey,Mohammad Banikazemi, George Almasi 4:10 pm - Beyond the Horizon: Innovating and Customizing Horizon using AngularJS - Cindy Lu, Thai Tran
top related