![Page 1: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/1.jpg)
Infrastructure-as-a-ServiceOpenStack
Adrien LebreSTACK Research Group
![Page 2: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/2.jpg)
A BIT OF HISTORY
John Mc Carthy, Speaking at the MIT centennial in 1961
If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility...
![Page 3: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/3.jpg)
/33
Looking back…
• xxx ComputingMeta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud / Sky …
• A common objective: provide computing resources (both hardware and software) in a flexible, transparent, secure, reliable, ... way
• Challenges
Software/Hardware heterogeneity
Security (Isolation between applications, ...)
Reliability / Resiliency
Data Sharing
Performance guarantees... 3
⇒ xxx as Utility Computing
![Page 4: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/4.jpg)
Frontend(Resource Management System)
Looking back…
4
Alice’ working node
Alice
• Network of Workstations 1990 / 20xx
![Page 5: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/5.jpg)
Frontend(Resource Management System)
Looking back…
4
Alice’ working node
Alice
Compute nodes
Storage nodes(Distributed File System)
• Network of Workstations 1990 / 20xx
![Page 6: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/6.jpg)
Frontend(Resource Management System)
AliceAliceAlice
Alice Alice Alice
Looking back…
4
Alice’ working node
Alice
Compute nodes
Storage nodes(Distributed File System)
• Network of Workstations 1990 / 20xx
![Page 7: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/7.jpg)
Frontend(Resource Management System)
Alice AliceAliceAlice
Alice Alice Alice
Alice
Alice
Looking back…
4
Alice’ working node
Alice
Compute nodes
• Network of Workstations 1990 / 20xx
Network File System
![Page 8: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/8.jpg)
Frontend(Resource Management System)
Alice AliceAliceAlice
Alice Alice Alice
Alice
Alice
Looking back…
4
Alice’ working node
Alice
Compute nodes
• Network of Workstations 1990 / 20xx
Network File System
Long term Storage
![Page 9: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/9.jpg)
Frontend(Resource Management System)
Alice AliceAliceAlice
Alice Alice Alice
Alice
Alice
Looking back…
5
Alice’ working node
Alice
Compute nodes
Storage nodes(Distributed File System)
• Network of Workstations 1990 / 20xx
Network File System
Long term Storage
![Page 10: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/10.jpg)
Frontend(Resource Management System)
Alice AliceAliceAlice
Alice Alice Alice
Alice
Alice
Looking back…
5
Alice’ working node
Alice
Compute nodes
Storage nodes(Distributed File System)
• Network of Workstations 1990 / 20xx
Network File System
Map/Reduce framework (leverage attached storage facilities)
Long term Storage
![Page 11: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/11.jpg)
/33
Looking back…
6
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
Charles
Charles’ working node
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 12: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/12.jpg)
/33
Looking back…
6
Bob
Bob’ working node
Wide Area Network
Alice’ working node
AliceGrid File System
Charles
Charles’ working node
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 13: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/13.jpg)
/33
Looking back…
6
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 14: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/14.jpg)
/33
Looking back…
6
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 15: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/15.jpg)
/33
Looking back…
6Bob
Bob Bob
Bob
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
CharlesCharles
Charles
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 16: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/16.jpg)
/33
Looking back…
6Bob
Bob Bob
Bob
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
CharlesCharles
Charles
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 17: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/17.jpg)
/33
Looking back…
6Bob
Bob Bob
Bob
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
CharlesCharles
Charles
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 18: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/18.jpg)
/33
Looking back…
6Bob
Bob Bob
Bob
Bob
Bob’ working node
Wide Area Network
Alice’ working node
Alice
AliceAlice
Alice Alice Alice
Alice
Alice Alice
Grid File System
Charles
Charles’ working node
CharlesCharles
Charles
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 19: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/19.jpg)
/33
Looking back…
7
Bob
Bob’ working node
Alice’ working node
Alice
Charles
Charles’ working node
Resource booking (based on user’s estimates)Security concerns (job isolation)
Heterogeneity concerns (hardware and software)Scheduling limitations (a job cannot be easily relocated)
Fault tolerance issues...
What a Grid ! ? !
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 20: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/20.jpg)
/33
Looking back…
7
Bob
Bob’ working node
Alice’ working node
Alice
Charles
Charles’ working node
What a Grid ! ? !
A lot of progress has been done since the 90's and
several proposals partially addressed these concerns.
• Network of Workstations 1990 / 20xx
• Desktop 1998 / 201x
• Grid 1998 / 201x
![Page 21: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/21.jpg)
/33
Looking back…
Hypervisor
• System virtualization: One to multiple OSes on a physical node thanks to a hypervisor (an operating system of OSes)
Virtual Machines (VMs)
Physical Machine (PM)
Virtual Machine Monitor
“A virtual machine (VM) provides a faithful implementationof a physical processor’s hardware running in a protectedand isolated environment.Virtual machines are created by a software layer calledthe virtual machine monitor (VMM) that runs as aprivileged task on a physical processor.”
8
![Page 22: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/22.jpg)
/33
Looking back…
Hypervisor
• System virtualization: One to multiple OSes on a physical node thanks to a hypervisor (an operating system of OSes)
Virtual Machines (VMs)
Physical Machine (PM)
Virtual Machine Monitor
“A virtual machine (VM) provides a faithful implementationof a physical processor’s hardware running in a protectedand isolated environment.Virtual machines are created by a software layer calledthe virtual machine monitor (VMM) that runs as aprivileged task on a physical processor.”
8
![Page 23: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/23.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Isolation (‘‘security’’ between each VM)
9
• System virtualization: a great sandbox
![Page 24: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/24.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3Virus / Invasion / Crash
• Isolation (‘‘security’’ between each VM)
9
• System virtualization: a great sandbox
![Page 25: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/25.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3Virus / Invasion / Crash
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 26: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/26.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 27: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/27.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 28: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/28.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 29: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/29.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 30: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/30.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 31: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/31.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 32: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/32.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Live migration (negligible downtime ~ 60 ms) Post/Pre Copy
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 33: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/33.jpg)
/33
VM 3
Hypervisor
Looking back…
VM 1 VM 2 VM 3
Hypervisor
• Suspend/Resume Hypervisor
VM 1 VM 2
• Isolation (‘‘security’’ between each VM)
• Live migration (negligible downtime ~ 60 ms) Post/Pre Copy
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 34: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/34.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3
Hypervisor
• Suspend/Resume Hypervisor
VM 1 VM 2 VM 3
• Isolation (‘‘security’’ between each VM)
• Live migration (negligible downtime ~ 60 ms) Post/Pre Copy
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 35: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/35.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3
Hypervisor
• Suspend/Resume Hypervisor
VM 1 VM 2 VM 3
• Isolation (‘‘security’’ between each VM)
• Live migration (negligible downtime ~ 60 ms) Post/Pre Copy
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 36: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/36.jpg)
/33
Hypervisor
Looking back…
VM 1 VM 2 VM 3
• Suspend/Resume Hypervisor
VM 1 VM 2 VM 3
• Isolation (‘‘security’’ between each VM)
• Live migration (negligible downtime ~ 60 ms) Post/Pre Copy
• Snapshotting (a VM can be easily resume from its latest consistent state)
9
• System virtualization: a great sandbox
![Page 37: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/37.jpg)
/33
• Network of Workstations 1990 / 20xx
• Desktop Computing 1998/201x
• Grid 1998 / 201x
• Cloud 2005 / 201x
10
Bob
Bob’ working node
Alice’ working node
Alice
Infrastructure as a Service model
CharlesCharles’ working node
Looking back …
![Page 38: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/38.jpg)
/33
• Network of Workstations 1990 / 20xx
• Desktop Computing 1998/201x
• Grid 1998 / 201x
• Cloud 2005 / 201x
10
Bob
Bob’ working node
Alice’ working node
Alice
Infrastructure as a Service model
CharlesCharles’ working node
VM ImageRepository
Looking back …
![Page 39: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/39.jpg)
/33
• Network of Workstations 1990 / 20xx
• Desktop Computing 1998/201x
• Grid 1998 / 201x
• Cloud 2005 / 201x
10
Bob
Bob’ working node
Alice’ working node
Alice
Infrastructure as a Service model
CharlesCharles’ working node
WAN
Looking back …
![Page 40: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/40.jpg)
/33
• Network of Workstations 1990 / 20xx
• Desktop Computing 1998/201x
• Grid 1998 / 201x
• Cloud 2005 / 201x
10
Bob
Bob’ working node
Alice’ working node
Alice
Infrastructure as a Service model
CharlesCharles’ working node
WAN
Is it as simple as it looks?
Looking back …
![Page 41: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/41.jpg)
/33
IaaS Resource Management Systems• An Operating System for Cloud infrastructures (aka Cloudkits)
Configuration of Virtual Environments (VEs) (contextualization, network...)
Images management/deployment
‘‘Secure’’ accesses to the VEs
11
Alice
![Page 42: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/42.jpg)
/33
IaaS Resource Management Systems• An Operating System for Cloud infrastructures (aka Cloudkits)
Configuration of Virtual Environments (VEs) (contextualization, network...)
Images management/deployment
‘‘Secure’’ accesses to the VEs
11
Alice
![Page 43: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/43.jpg)
/33
IaaS Resource Management Systems• An Operating System for Cloud infrastructures (aka Cloudkits)
Configuration of Virtual Environments (VEs) (contextualization, network...)
Images management/deployment
‘‘Secure’’ accesses to the VEs
11
Alice
![Page 44: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/44.jpg)
/33
IaaS Resource Management Systems• An Operating System for Cloud infrastructures (aka Cloudkits)
Configuration of Virtual Environments (VEs) (contextualization, network...)
Images management/deployment
‘‘Secure’’ accesses to the VEs
11
Alice
Bob
![Page 45: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/45.jpg)
/17
An Overview of the IaaS Internals
12
Alice
Bob
VM ImagesRepository
Working nodes
command lineWeb Interface
SDK
IaaS Service
![Page 46: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/46.jpg)
/17
An Overview of the IaaS Internals
12
Alice
Bob
VM ImagesRepository
Working nodes
command lineWeb Interface
SDK
IaaS Service
VMM
NodeController
VMVMVM
![Page 47: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/47.jpg)
/17
An Overview of the IaaS Internals
13
Bob
VM ImagesRepository
Working nodes - cluster 1
command lineWeb Interface
SDK
IaaS Service
Working nodes - cluster 2
Cluster controller Cluster
controller
Alice
![Page 48: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/48.jpg)
/17
• Open Nebula
2008-20XX Results of the RESERVOIR project (mainly used in EU) Montero & Llorente, DSA-Research at UCMC++ / set of scripts
• CloudStack
2010-20XX Apache project (in 2011) Java Based
• Open Stack
2010-20XX Supported by several industrialsThe defacto open-source solutionPython
Managing IaaS - OpenSource solutions
14
![Page 49: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/49.jpg)
15
You said OpenStack…
![Page 50: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/50.jpg)
OpenStack
![Page 51: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/51.jpg)
OpenStack
![Page 52: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/52.jpg)
OpenStack
![Page 53: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/53.jpg)
OpenStack
![Page 54: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/54.jpg)
OpenStack
![Page 55: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/55.jpg)
OpenStack
![Page 56: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/56.jpg)
• Open source
• Open development: access to each contribution/ logs of meetings,…
• Open design: the community is listened to set the direction of OpenStack
• Open Community: anyone can raise to leadership position
The OOOO principles
![Page 57: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/57.jpg)
What is OpenStack? • an open-source project • a common goal • a coalition of organizations • a foundation • a trademark • an interoperability standard • a set of events (OpenStack Summit/PTG/OpenStack Days…) • a governance model • a job market • a single project / a set of projects • a set of principles • a development community • a big tent • a bunch of python code • a way to produce software • a very active open-source project • a success story • ok so what it is….
credits: D. Pertin FiCloud 2017 Presentation
A lot of Fun !
![Page 58: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/58.jpg)
A Rich (and Complex) Ecosystem
• 20 Millions of LoC, 164 services, some services are composed of sub-services (e.g. nova-scheduler, nova-conductor, …)
![Page 59: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/59.jpg)
A Rich (and Complex) Ecosystem
• 20 Millions of LoC, 164 services, some services are composed of sub-services (e.g. nova-scheduler, nova-conductor, …)
![Page 60: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/60.jpg)
The User/Admin Viewpoints• Everything goes through the API
(including the HORIZON dashboard)
• APIs: REST / one per service
Through HTTP (curl) Through SDKs and broker librariesThrough Horizon or the command line interface Through HEAT
• You need specific credentials (delivered by KEYSTONE)
R-A Cherrueau / D. Pertin Next Week !
![Page 61: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/61.jpg)
CINDER
The Developer ViewpointBest
NEUTRON
NOVA
GLANCE
SWIFT
HORIZON
KEYSTONE
![Page 62: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/62.jpg)
OpenStack core-services
• Each core-service is divided into several sub-services
• Services communicate through a communication bus (AMQP)
• System states are stored in a SQL DB (MySQL/MariaDB)
• Python for all projects
• APIs: OpenStack and AWS-like
![Page 63: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/63.jpg)
HORIZON Dashboard
• Provides a web based user interface to OpenStack services (a Django web application)
• Three central dashboards, a “User Dashboard”, a “System Dashboard”, and a “Settings” dashboard.
![Page 64: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/64.jpg)
HORIZON Dashboard
• Provides a web based user interface to OpenStack services (a Django web application)
• Three central dashboards, a “User Dashboard”, a “System Dashboard”, and a “Settings” dashboard.
![Page 65: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/65.jpg)
KEYSTONE Authentification
• Keystone provides API client authentication, service discovery, and distributed multi-tenant authorization by implementing OpenStack’s Identity API.
• It supports LDAP, OAuth, OpenID Connect, SAML and SQL.PIs: REST / one per service
![Page 66: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/66.jpg)
Nova Compute Service
• Implement services and associated libraries to provide massively scalable, on demand, self service access to compute instances
• Nova supports creating virtual machines and baremetal servers, through the use of IRONIC.
More details later…
![Page 67: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/67.jpg)
Glance Image Service• VM images contain a virtual disk that holds a bootable
operating system on it.
• Each launched instance runs from a copy of the base image. Any changes made to the instance do not affect the base image. Snapshots capture the state of an instances running disk.
• Users can create a snapshot, and build a new image based on these snapshots
• Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image.
• VM images made available through Glance can be stored in a variety of locations from simple filesystems to object-storage systems like the OpenStack Swift project.
![Page 68: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/68.jpg)
Swift Object Store• Swift is a highly available, distributed,
eventually consistent object/blob store (a S3-like system)
• Organizations can use Swift to store lots of data efficiently, safely, and cheaply. It's built for scale and optimized for durability, availability, and concurrency across the entire data set.
• Swift is ideal for storing unstructured data that can grow without bound.
• The only service that does not leverage a central DB
![Page 69: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/69.jpg)
Cinder Block Storage• You can add and remove additional resources from
running instances, such as persistent volume storage
• The Cinder-volume service provides persistent block storage, instead of the ephemeral storage provided by the base image (i.e. the Glance one)
• Cinder virtualizes the management of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device.
• This is done through the use of either a reference implementation (LVM) or plugin drivers for other storage.
![Page 70: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/70.jpg)
Neutron Networking• OpenStack Neutron is an SDN networking project
focused on delivering networking-as-a-service (NaaS) in virtual compute environments.
• Composed of several sub-services Neutron-server: API service Agent DHCP: DHCP service for instances Agent L3 : Routing service
• In addition to elementary services (L2/L3), Neutron provides additional mechanisms (load-balancing, firewalls, VPN…)
• A lot of plugins (LinuxBridge default one)
![Page 71: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/71.jpg)
Let’s Dive into Details
CINDER
NEUTRON
NOVA
GLANCE
SWIFT
HORIZON
KEYSTONE
![Page 72: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/72.jpg)
Let’s Dive into Details
![Page 73: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/73.jpg)
DBs/Communication Bus
DB Bus
agent 1
agent 2
agent 1
agent 1
agent 2
agent 2
agent 2
agent 1
others
Nova
Neutron
Keystone
Glance
internal database
externalStorage
Authentification
Compute
Network
credits: M. Simonin - IPL Discovery
![Page 74: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/74.jpg)
OpenStack Common Libraries
DBs/Communication Bus
agent2
agent1
Bus olso.messaging
RPC Notification
Rab
bitM
Q
Zero
MQ
AMQ
P1.0
Kafk
a
AMQ
P1.0
Rab
bitM
Q
communication pattern
transport
common library
credits: M. Simonin - IPL Discovery
![Page 75: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/75.jpg)
DBs/Communication Bus
agent4agent2
agent1
Bus
unicast RPCblocking & non blocking
Bus
agent1
Bus
multicast RPCnon blocking only
agent3agent3
1 2 34 1 2 1 2
agent4
agent1
agent3agent3
anycast RPCblocking & non blocking
34
credits: M. Simonin - IPL Discovery
![Page 76: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/76.jpg)
Physically, How This Looks Like?
Alice
OpenStack Controller
…
OpenStack Compute nodes
SQL DBs Message queue
Identity
Image Service
Compute Management
NetworkingManagement
Compute Agent
Network Agent
Block Storage Service
KVM
Linux Network Utilities
Object Storage Service
![Page 77: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/77.jpg)
Physically, How This Looks Like?
Alice
OpenStack Controller
…
OpenStack Compute nodes
SQL DBs Message queue
Identity
Image Service
Compute Management
NetworkingManagement
Compute Agent
Network Agent
Block Storage Service
KVM
Linux Network Utilities
Object Storage Service
A First Simplified View
![Page 78: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/78.jpg)
Physically, How This Looks Like?
Alice
OpenStack Controller
…
OpenStack Compute nodes
Compute Agent
Network Agent
KVM
Linux Network Utilities
Compu
te
Manag
emen
t
Ident
itySQ
L DBs
+
Galera
Networki
ng
Manage
ment
Imag
e Ser
vice
Messag
e que
ue
![Page 79: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/79.jpg)
Walkthrough of a typical Nova boot request
• Make a request to the Identity API (Keystone) to get an access token
• Make a request to the Compute API (Nova) using the aforementioned token and mentioning The flavor (VM properties) The image (a bootable image) The network ID (i.e. where the VM gets its IP)
credits: J. Pipes - https://github.com/jaypipes/articles/blob/master/openstack/walkthrough-launch-instance-request.md
Alice
OpenStack Controller
a./ process request b./ select a host
c./ Send the boot instance order
…
OpenStack Compute nodes
vagrant@enos-node:/opt/enos$ openstack server create\ -—flavor m1.tiny\ --image cirros.uec\ --nic net-id=$(openstack network show private --column id --format value)\ cli-vm
![Page 80: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/80.jpg)
Walkthrough of a typical Nova boot request
• Select a host (scheduler)
Ask Placement for possible hosts (placement API)
Apply filters
Sort hosts with weighers and take best match
credits: J. Pipes - https://github.com/jaypipes/articles/blob/master/openstack/walkthrough-launch-instance-request.md
Alice
OpenStack Controller
…
OpenStack Compute nodes
a./ process requestb./ select a host
c./ Send the boot instance order
![Page 81: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/81.jpg)
Walkthrough of a typical Nova boot request
• When the compute node receives the request : Create a port from NEUTRON Prepare the corresponding block device Ask the hypervisor to boot the VM
credits: J. Pipes - https://github.com/jaypipes/articles/blob/master/openstack/walkthrough-launch-instance-request.md
Alice
OpenStack Controller
…
OpenStack Compute nodes
a./ process requestb./ select a host
c./ Send the boot instance order
![Page 82: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/82.jpg)
Focus on Nova Service
Nova Conductor: a key element
![Page 83: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/83.jpg)
Focus on Nova Service
Cell… by default since Ocata
![Page 84: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/84.jpg)
Focus on Nova Service
Cell…a way to segregate your infrastructure
![Page 85: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/85.jpg)
Focus on Neutron
• Different kinds of networks
Project networks provide connectivity to instances for a particular project/tenant (Private IPs)
External networks provide connectivity to external networks such as the Internet (Public IPs)
Routers typically connect project and external networks.
• Other supporting services (DHCP, ssh keys, …)
![Page 86: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/86.jpg)
Understanding Neutron Flows
• In the example configuration, the management network uses 10.0.0.0/24, the tunnel network uses 10.0.1.0/24, and the external network uses 203.0.113.0/24. The VLAN network does not require an IP address range because it only handles layer-2 connectivity.
![Page 87: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/87.jpg)
Understanding Neutron Flows
• Routing among project and external networks resides completely on the network node.
• While this makes the management of network flows easier, it may lead to SPOF issues
![Page 88: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/88.jpg)
Understanding Neutron Flows
Inside the Network Node
![Page 89: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/89.jpg)
Understanding Neutron Flows
Inside the Compute Node
![Page 90: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/90.jpg)
Understanding Neutron Flows
Inside the Compute NodeInterested to understand two slides, please see
https://docs.openstack.org/kilo/networking-
guide/scenario_legacy_lb.html
![Page 91: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/91.jpg)
Neutron Flows in a Nutshell
Alice
…
OpenStack Compute nodes
SQL DBs
Identity
Image Service
Compute Management
NetworkingManagement
…
Message queue
East/West - Same project network
![Page 92: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/92.jpg)
Neutron Flows in a Nutshell
Alice
…
OpenStack Compute nodes
SQL DBs
Identity
Image Service
Compute Management
NetworkingManagement
…
Message queue
East/West - Different project networks
![Page 93: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/93.jpg)
Neutron Flows in a Nutshell
Alice
…
OpenStack Compute nodes
SQL DBs
Identity
Image Service
Compute Management
NetworkingManagement
…
Message queue
North/South - Private or Public IPs
![Page 94: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/94.jpg)
Focus on Glance
• For each image, one can specify properties Image kind (raw, cqow2, vmdk, iso…) Architecture Distribution Version Storage space requirements RAM minimal size …
• Possible backends Swift/S3 Ceph HTTP Local
![Page 95: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/95.jpg)
Other Services IRONIC bare metal
• Ironic provisions bare metal (as opposed to virtual) machines.
• It may be used independently or as part of an OpenStack Cloud, and integrates with the OpenStack Identity (keystone), Compute (nova), Network (neutron), Image (glance) and Object (swift) services.
• When the Bare Metal service is appropriately configured with the Compute and Network services, it is possible to provision both virtual and physical machines through the Compute service’s API.
• Although the project is mature, it is barely used…
![Page 96: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/96.jpg)
• A lot…
• CEILOMETER: Metering & Data Collection Service
• DESIGNATES: DNS-as-a-Service
• OCTAVIA: load balancer
• TROVE: a Database-as-a-service (SQL and NoSQL)
• SAHARA: Big Data Processing Framework Provisioning
• ZUN: Container Management Service
• …
• and HEAT: Orchestration A few slides later
Other Services…
![Page 97: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/97.jpg)
Segregation Tools
• It is sometimes a key point to segregate a large infrastructure into several subsets. OpenStack provides several ways:
Host aggregates - Nova: Classify compute nodes according to their specifics (Storage, GPU, …)
Availability Zones - Nova, Cinder: Classify resources according to availability aspects (racks, data centers, …)
Cells - Nova (Neutron soon !?): address segregation and scalability needs (one communication bus and one DB per cells)
Regions - a federation like approach, each region has an almost complete OpenStack (keystone/horizone are shared). Equivalent to the AWS region.
![Page 98: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/98.jpg)
Segregation Tools
credits: Kimi Zhang - https://kimizhang.wordpress.com/2013/08/26/openstack-zoning-regionavailability-zonehost-aggregate/
![Page 99: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/99.jpg)
Contextualization of the Instances• Two kind of images
• generic: require to be customised after the boot process
• cloud-init, a tool provided in most ``cloud’’ images: leverages the user-data to customise the instance (i.e., add packages, start services…)
• Golden: have been customised previously, leveraging dedicated tools
• virt-builder, Packer…
vagrant@openstack:~$ cat > /tmp/provision.sh << EOF apt update -q apt install -q -y figlet lolcat EOF vagrant@openstack:~$ openstack server create --image debian-9\ --flavor m1.small --network private\ --key-name admin\ --user-data /tmp/provision.sh cli-vm-provision
![Page 100: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/100.jpg)
• Possibility to use dedicated frameworks to configure and orchestrate your instances
• ANSIBLE “is software that automates software provisioning, configuration management, and application deployment.” wikipedia
• Ansible deploys modules to nodes over SSHAnsible uses an agentless architecture
• JUJU “focuses on reducing the operation overhead of today's software by facilitating quickly deploying, configuring, scaling, integrating, and performing operational tasks” wikipedia
• Provide a modeling language for users that abstracts the specifics of operating complex big software topologies
Contextualization of the Instances
![Page 101: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/101.jpg)
CLI OpenStack
• OpenStack provides a command line interface List OpenStack running services: openstack endpoint list List images: openstack image list List flavors: openstack flavor list List networks: openstack network list List computes: openstack hypervisor list List VMs (running or not): openstack server list Get details on a specific VM: openstack server show <vm-name> Start a new VM: openstack server create --image <image-name> --flavor <flavor-name> --nic net-id=<net-id> <vm-name> View VMs logs: openstack console log show <vm-name>
vagrant@openstack:~$ for vm in $(openstack server list -c Name -f value); do\ echo "Delete ${vm}...";\ openstack server delete "${vm}";\done
![Page 102: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/102.jpg)
CLI OpenStack
• OpenStack provides a command line interface List OpenStack running services: openstack endpoint list List images: openstack image list List flavors: openstack flavor list List networks: openstack network list List computes: openstack hypervisor list List VMs (running or not): openstack server list Get details on a specific VM: openstack server show <vm-name> Start a new VM: openstack server create --image <image-name> --flavor <flavor-name> --nic net-id=<net-id> <vm-name> View VMs logs: openstack console log show <vm-name>
vagrant@openstack:~$ for vm in $(openstack server list -c Name -f value); do\ echo "Delete ${vm}...";\ openstack server delete "${vm}";\done
Is a merge of the
two concepts
available?
![Page 103: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/103.jpg)
Focus on HEAT
• One more service…
• Heat orchestrates the infrastructure resources for a cloud application based on templates (HOT) in the form of text files that can be treated like code.
• Heat provides both an OpenStack-native ReST API and a CloudFormation-compatible Query API.
• Heat also provides an autoscaling service that integrates with the OpenStack Telemetry services, so you can include a scaling group as a resource in a template.
![Page 104: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/104.jpg)
Focus on HEAT
• OpenStack-core services hide lot of complexity by automating the deployment process of a new VM/PM
Nova contacts Neutron for network configuration Nova contacts Glance to fetch an OS image Nova is in charge of booting and managing the VM
• But starting an empty OS is not enough to deliver a service…The hard part is to put the VM in context:
Install the software stack (service + its dependencies) Configure the service Set a floating IP address to be reachable from Internet
• A Cloud application is made of multiple services Multiple machines to boot, and services to deploy and configure to deliver the application
credits: D. Pertin - Introduction to Heat
![Page 105: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/105.jpg)
• DEVOPS Philosophy
• Infrastructure as Code Scale in/out (horizontal scaling) Automation
• Monitoring services/apps (instead of the infrastructure)
• Backup/Restart on demand
Focus on HEAT
Manage Application Life-Cycle
with code !
![Page 106: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/106.jpg)
• Readable format:HEAT Orchestration Template (HOT)
YAML file
• Describe infrastructure and applications
• Declare any OpenStack resource typesInstances, floating IPs, volumes, images, users, …
• Declare relationships between resources
e.g. a VM must be booted before installing software on it
• HEAT Engine
Take a template as input Parse itExecute tasks through OpenStack API calls
HEAT Template
credits: D. Pertin - Introduction to Heat
![Page 107: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/107.jpg)
• Exemple: a 3Tier-Web Application : Wordpress
Understanding HEAT Through an Example
![Page 108: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/108.jpg)
• Boot two virtual machinesVM1 and VM2Ubuntu based OS Connect them to the private network
• Install the servicesInstall a database server (mysql) on VM2Install a HTTP server (apache2) on VM1 Download Wordpress on VM1
• Configure servicesCreate an appropriate database in mysql for wordpressConfigure wordpress to access this databaseConfigure apache2 to serve wordpress
• Assign a floating IP to VM1
credits: D. Pertin - Introduction to Heat
How can I automatise
this process by leveraging HEAT?
Understanding HEAT Through an Example
![Page 109: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/109.jpg)
• The Hello World example - boot a VM
HEAT Template
credits: D. Pertin - Introduction to Heat
$ cat boot.yamlheat_template_version: 2015-04-30
description: Simple template to deploy a single compute instance
resources: # HEAT resources are declared here my_instance: # Name of my resource type: OS::Nova::Server # Type of my resource (this resources defines a VM) properties: # Here we define the properties of this resource type key_name: my_key_name # Name of an SSH key managed by Nova (or Barbican) image: ubuntu-trusty-x86_64 # Name of an image managed by Glance flavor: m1.small # Name of a flavor managed by Nova
$ openstack stack create my_stack -f boot.yaml
![Page 110: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/110.jpg)
• Let’s make it a bit more complex: boot a vm with parameters
HEAT Template
credits: D. Pertin - Introduction to Heat
$ cat boot_with_parameter.yamlheat_template_version: 2015-04-30
description: Simple template to deploy a single compute instance with a parameter
parameters: # Parameters definition for this template key_name: # Name of the parameter type: string # Type of the parameter description: Name of a KeyPair to enable SSH access to the instance
resources: my_instance: type: OS::Nova::Server properties: key_name: { get_param: key_name } # Use an intrinsic function to get the value of a parameter image: ubuntu-trusty-x86_64 flavor: m1.small
$ openstack stack create my_stack -f boot_with_parameters.yaml --parameter key_name=my_key
![Page 111: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/111.jpg)
• Let’s make it a bit more complex: boot a vm and get some outputs
HEAT Template
credits: D. Pertin - Introduction to Heat
$ cat boot_with_outputs.yamlheat_template_version: 2015-04-30
description: Simple template to deploy a single compute instance, outputs its ip address
resources: my_instance: type: OS::Nova::Server properties: key_name: my_key_name image: ubuntu-trusty-x86_64 flavor: m1.small
outputs: # Definition of the outputs of this template instance_ip: # Name of the output description: IP address of the deployed compute instance value: { get_attr: [my_instance, first_address] } # Set the IP address of the # machine as the value of the # output instance_ip
$ openstack stack create my_stack -f boot_with_outputs.yaml
![Page 112: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/112.jpg)
• Even more complex
HEAT Template
credits: D. Pertin - Introduction to Heat
$ cat boot_sql_server.yamlheat_template_version: 2015-04-30description: Template to deploy SQL serverparameters: DBRootPassword: type: stringresources: my_sql_instance: type: OS::Nova::Server properties: # general properties ... user_data: # Definition of a boot script str_replace: # Intrinsic function to replace # strings in the script by parameters template: | # Description of the script #!/bin/bash # do things like install mysql ... mysqladmin -u root password $db_rootpassword # do more things ... params: # Description of the used parameters $db_rootpassword: { get_param: DBRootPassword }…
$ openstack stack create my_sql_server -f boot_sql_server.yaml --parameter DBRootPassword=0p3nSt4cK
![Page 113: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/113.jpg)
• The complete example during the practical session !
Understanding HEAT Through an Example
![Page 114: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/114.jpg)
• The complete example during the practical session !
Understanding HEAT Through an Example
R-A Cherrueau D. Pertin
Practial sessions !
![Page 115: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/115.jpg)
Cloud Application Design Rules
• When you are developing an application for the cloud, there are some guidelines to follow
• Pets vs Cattle, think of your application as components that may crash
![Page 116: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/116.jpg)
72
I want to test it !
![Page 117: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/117.jpg)
DevStack
• A series of extensible scripts used to quickly bring up a complete OpenStack environment based on the latest versions of everything from git master stack.sh + local.conf
• A development environment and as the basis for much of the OpenStack project’s functional testing.
![Page 118: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/118.jpg)
DevStack
• A series of extensible scripts used to quickly bring up a complete OpenStack environment based on the latest versions of everything from git master stack.sh + local.conf
• A development environment and as the basis for much of the OpenStack project’s functional testing.
WARNING: DevStack makes substantial changes to your system
Only launch it inside a VM
![Page 119: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/119.jpg)
Experimental eNvironment for OpenStack
• A dedicated framework to conduct performance analyses of OpenStack at large-scale in a reproducible manner. The framework enables engineers/researchers to conduct experiments in an automize manner on top of different testbeds such as Grid’5000, Chameleon, OpenStack…
• Developed in the context of the Discovery Initiative
• Deploy a real production system by leveraging Kolla (i.e. not DevStack)
🐒
![Page 120: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/120.jpg)
Experimental eNvironment for OpenStack
• A dedicated framework to conduct performance analyses of OpenStack at large-scale in a reproducible manner. The framework enables engineers/researchers to conduct experiments in an automize manner on top of different testbeds such as Grid’5000, Chameleon, OpenStack…
• Developed in the context of the Discovery Initiative
• Deploy a real production system by leveraging Kolla (i.e. not DevStack)
🐒
R-A Cherrueau D. Pertin Practial sessions !
![Page 121: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/121.jpg)
Experimental eNvironment for OpenStack 🐒
enos DEPLOY
enos UP
enos BENCH
enos BACKUP
Get/Check machines and networks
Deploy the cloud
Benchmark the cloud
Analyse the cloudin real time or offline
enos DESTROYRelease resources
Loop until goal is reached
![Page 122: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/122.jpg)
Experimental eNvironment for OpenStack 🐒
enos DEPLOY
enos UP
enos BENCH
enos BACKUP
Get/Check machines and networks
Deploy the cloud
Benchmark the cloud
Analyse the cloudin real time or offline
enos DESTROYRelease resources
Loop until goal is reached
R-A Cherrueau D. Pertin Practial sessions !
![Page 123: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/123.jpg)
You cannot Wait For the Practical Session
• Install VirtualBox Vagrant EnOS
• Prepare your RAM (at least 10GB ;-) )
![Page 124: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/124.jpg)
You cannot Wait For the Practical Session
• Install VirtualBox Vagrant EnOS
• Prepare your RAM (at least 10GB ;-) )
https://enos.readthedocs.io/en/stable/
![Page 125: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/125.jpg)
77
Why diving in such a level of details?!
![Page 126: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/126.jpg)
Fog/Edge Challenges
credits: M. Simonin - IPL Discovery
WAN
WAN
site 1
site 2 to n
site 1 site 2
• Communication Bus Central rabbitMQ and many edge servers Distributed RabbitMQ through federations
![Page 127: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/127.jpg)
Fog/Edge Challenges
• DBsAMQPbus
AMQPbus
AMQPbus
Key/Value Store
NovaController 3
n-schedn-condn-apin-netn-cpuhorizon
NovaController 2n-sched
n-condn-apin-netn-cpuhorizon Nova
ComputeNodes
Nova Compute Nodes
NovaController 1
n-schedn-condn-apin-netn-cpuhorizon
NovaController 5 n-sched
n-condn-apin-netn-cpuhorizon
Nova Controller 4and compute node
n-schedn-condn-apin-netn-cpuhorizon
NovaComputeNode
Site 1
Site 2
Site 3
Site 4
![Page 128: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/128.jpg)
Fog/Edge Challenges
• Locality/Split BrainBus
DB
BusBus
DB
London
Paris
Lyon
DB
! !
! !
! !
![Page 129: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/129.jpg)
Neutron and Fog/Edge challenges
Alice
SQL DBs
Identity
Image Service
Compute Management
NetworkingManagement
Message queue
East/West - Different project networks
![Page 130: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/130.jpg)
Neutron and Fog/Edge challenges
Alice
SQL DBs
Identity
Image Service
Compute Management
NetworkingManagement
Message queue
East/West - Different project networks
![Page 131: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/131.jpg)
83
Takeaway Message
![Page 132: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/132.jpg)
84
Alice
Duke
Distributed Clouds (Fog/Edge)
Fog/Edge Computing
--
Fog/Edge Computing
Paula
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Tom
Fog/Edge ComputingFog/Edge Computing
Charles
Bob Dan
Sam
Rob
Fog/Edge
![Page 133: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/133.jpg)
84
Alice
Duke
Distributed Clouds (Fog/Edge)
Fog/Edge Computing
--
Fog/Edge Computing
Paula
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Tom
Fog/Edge ComputingFog/Edge Computing
Charles
Bob Dan
Sam
Rob
Fog/Edge
![Page 134: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/134.jpg)
84
Alice
Duke
Distributed Clouds (Fog/Edge)
Fog/Edge Computing
--
Fog/Edge Computing
Paula
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Tom
Fog/Edge ComputingFog/Edge Computing
Charles
Bob Dan
Sam
Rob
Fog/Edge
Clouds hide the infrastructure…….by adding more layers !
![Page 135: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/135.jpg)
84
Alice
Duke
Distributed Clouds (Fog/Edge)
Fog/Edge Computing
--
Fog/Edge Computing
Paula
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Fog/Edge Computing
Tom
Fog/Edge ComputingFog/Edge Computing
Charles
Bob Dan
Sam
Rob
Fog/Edge
Clouds hide the infrastructure…….by adding more layers !
and someone else’s network
![Page 136: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/136.jpg)
Cloud Computing technology is changing every day
How developers should develop new applications to benefit from geographically distributed infrastructures.
How to locate hardware/software components?
…
Do not hesitate to push the boundaries
Thanks
85
http://beyondtheclouds.github.io/We have Internship Positions
Utility
![Page 137: Infrastructure-as-a-Service OpenStackenos.irisa.fr/tp-polytech/openstack-slides.pdf · /33 Looking back… • xxx Computing Meta / Cluster / Grid / Desktop / ‘’Hive’’ / Cloud](https://reader033.vdocument.in/reader033/viewer/2022050409/5f8678cbcb79e3138332fb28/html5/thumbnails/137.jpg)
Bibliography
•In a Nutshell - How OpenStack Works A bit deprecated but good entry point to have a first idea in less than 10 minuteshttp://vmartinezdelacruz.com/in-a-nutshell-how-openstack-works/
•OSONES SLIDES Rich (almost up-to-date) https://github.com/Osones/formations
•OpenStack official DocumentationComplete and up-to-date (at least yesterday ;-)) https://www.openstack.org/software/