tecniris@: opennebula tutorial

OpenNebula Tutorial - RedIRIS 2013November 19th, 2013

1/16Creative Commons Attribution-NonCommercial-ShareAlike License

An Introduction to Cloud Computing with OpenNebula

Daniel Molina Aranda

Acknowledgments

The research leading to these results has received funding from the European Union's Seventh Framework Programme ([FP7/2007-2013] ) under grant agreement n° 612053

(CloudCatalyst Project)

dmolina@opennebula.orgProject Engineer

2An Introduction to Cloud Computing with OpenNebula

Agenda

An Introduction to Cloud Computing with OpenNebula

● Infrastructure as a Service●The OpenNebula Model●The Anatomy of the Cloud

Infrastructure as a Service

Types of Cloud ServicesWhat Who

On-demand access to any application

End-user(does not care about hw or sw)

Platform for building and

delivering web applications

Developer (no managing of the underlying

hw & swlayers)

Raw computer infrastructure

System Administrator(complete management of the

computer infrastructure)

Software as a Service

Platform as a Service

Physical Infrastructure

Types of Cloud Deployments

• Simple Web Interface

• Raw Infrastructure Resources

• Pay-as-you-go (On-demand access)

• Elastic & “infinite” Capacity

Public Cloud

A “Public Cloud behind the firewall”

• Simplify internal operations

• Dynamic allocation of resources

• Higher utilization & operational savings

• Security concerns

Private Cloud

Public Cloud

Private Cloud

• Supplement the capacity of the Private Cloud

• Utility Computing dream made a reality!

Hybrid Cloud

Public Cloud

Private Cloud

• Suplement the capacity of the Private Cloud

• Utility Computing dream made a reality!

Hybrid Cloud

Challenges of IaaS Clouds

● How do I provision a new VM?Image Management & Context

● Where do I store the disks?Storage

● How do I set up networking for a multitier service?Network & VLANs

● Where do I put my web server VM?Monitoring & Scheduling

● How do I manage any hypervisor?Virtualization

● Who has access to the Cloud’s resources?User & Role Management

● How do I manage my distributed infrastructure?Interfaces & APIs

● How do I provision a new VM?Image Management & Context

● Where do I store the disks?Storage

● How do I set up networking for a multitier service?Network & VLANs

● Where do I put my web server VM?Monitoring & Scheduling

● How do I manage any hypervisor?Virtualization

● Who has access to the Cloud’s resources?User & Role Management

● How do I manage my distributed infrastructure?Interfaces & APIs

Uniform management layer thatorchestrates multiple technologies

Challenges of IaaS Clouds

The Anatomy of the Cloud

Cloud Architecture - The Internals of the Cloud

Interfaces, Tools & API• CLI & Sunstone (GUI)

• API

• Cloud (EC2,OCCI)

• Service Management & Catalogs

Compute Hosts• Grouped into logical clusters

• Multiple hypervisors

• Monitoring

Storage• VM disks (file & block)

• Image Distribution

• Multiple Backends

Multi-tenancy• AAA Services

• Scheduling

• Permissions & roles

Network• VLAN

• Firewalling

• Multiple Technologies

Different Perspectives of the Cloud – Demands from the Different Communities

Cloud Consumer

Cloud Administrator

Cloud Integrator

Cloud Application Developer

An Uniform Management Layer

StoryA Project Aimed at Building the Industry Standard Open Cloud Management Tool

2008 2009 2010 2011 2012

• Develop & innovate• Support the

community• Collaborate

Large-scale production deployment: 16,000 VMs

5,000 downloads/mont

20142013

Research Project

TP v1.0 v1.2v1.4 v2.0 v2.2 v3.0v3.2v3.4v3.6v3.8v4.0V4.2

Openness• Open Architecture• Open Interfaces & Code

Interoperable• Use of standards• OCCI, AWS EC2

Adaptable• Modular architecture

Very Light• Highly efficient

Enterprise Ready• Single installing & update process• Support for various platforms

Proven• Tested & Mature

Powerful• Features for DC virtualization

No Lock-In• Platform

independant

The OpenNebula ModelDesign principles

Widely Used to Build Enterprise Private Clouds in Medium and Large Data Centers

Reference Users

Survey Q2/Q3 2012 (2,500 users http://c12g.com/resources/survey/)

The OpenNebula Model

OpenNebula Architecture - Infrastructure Agnostic and Highly Customizable

OpenNebula core

Virtualization Images

Storage Network

Monitoring

Scheduler

XML-RPC API

OCA (Ruby, Java)

CLI GUICloud

Servers

Languages

Basic OpenNebula Deployment

• Repository of VM images• Multiple Backends (LVM, Ceph)

Monitoring,Virtualization, Storage and Network

• Provides physical resources for the VMs• Must have a hypervisor installed

We Will Be Happy to Clarify Any Question

Questions?

19/16Creative Commons Attribution-NonCommercial-ShareAlike License

Installation and Basic Usage

Acknowledgments

Daniel Molina Arandadmolina@opennebula.org

Project Engineer

The research leading to these results has received funding from the European Union's Seventh Framework Programme ([FP7/2007-2013] ) under grant agreement n° 612053

(CloudCatalyst Project)

OpenNebula Tutorial - RedIRIS 2013November 19th, 2013

Agenda

Installing and Basic Usage

●Planning the Installation●Virtual Lab Configuration●Basic Usage

●Managing Hosts● Images, Networks, Templates and VMs●Managing Users, Quotas and ACLs●Logging & Debugging●Configuration Files

●Appendix A - Installation

A Typical OpenNebula Environment

Planning the Installation

• Repository of VM images• Multiple Backends (LVM, Ceph)

Monitoring,Virtualization, Storage and Network

• Provides physical resources for the VMs• Must have a hypervisor installed

Required Software

Installing

●Head node●ssh, ruby●OpenNebula: oned, mm_sched, sunstone, …

●Worker nodes●Hypervisor (KVM, Xen or VMWare)●ssh, ruby (Xen & KVM)

●Optional●Storage Backends (LVM, iSCSI, Ceph, …)●Networking systems (VLAN, Open vSwitch, …)●Ganglia, LDAP, Apache, Nginx

Virtual Lab

Planning the Installation

NODE 1 NODE 2

Virtual Lab

Installing

●Start Virtual Box●File > Import Appliance…

● frontend_node1.ova & node2.ova●Create a Host-only network in VirtualBox if you have

not created it yet

Virtual Lab

Installing

●Start frontend_node1 and node2 images●You can access them from:

1. VirtualBox GUI

2. Console/putty● frontend_node1 (password centos)

●node2 (password centos)

$ ssh localhost -l root -p 2222

$ ssh localhost -l root -p 2223

CLI – node1

Basic Usage

Hands on (node1) !●OpenNebula CLI Commands

# su - oneadmin$ one[TAB]

oneuser Manage Users oneimage Manage Images

onegroup Manage Groups onetemplate Manage Templates

oneacl Manage ACLs onevm Manage VMs

onehost Manage Hosts oneacct Accounting Tool

onecluster Manage Clusters onemarket Marketplace Tool

onevnet Manage Networks onedb DB Tool

onedatastore Manage Datastores

CLI – node1

Basic Usage

●Get the Sunstone Login information

●Try out sunstone!

http://localhost:9869

# (as oneadmin)$ cat ~/.one/one_authoneadmin:<password>

Adding Hosts - Sunstone

Basic Usage – Hosts

Hands on! (Sunstone)

●Create one host in Sunstone: node1

●Virtualization: KVM

●Information: KVM

●Network: dummy

●Cluster: none

●Watch transition INIT ON

●Click on the row for more information

●Automatic gathering of monitoring data

●Take a look at the graphs

Adding Hosts - CLI

Basic Usage – Hosts

Hands on! (CLI)

(always as oneadmin in the Frontend – node 1)

$ onehost -help$ onehost create -help

$ ssh node2 ls /var/tmp/one$ onehost create node2 -i kvm -v kvm -n dummy$ onehost list$ onehost top

# Wait for ON ... and then CTRL-C

$ ssh node2 ls /var/tmp/one$ onehost show node2$ onehost show 1$ onehost show -x 1

Adding Images

Basic Usage – Images

●Create an Image in Sunstone

●Name: tty

●Provide a Path: /var/tmp/tutorial/ttylinux.qcow2.img

●Advanced Driver: qcow2

●Datastore: default

●Create!

●Watch transition LOCKED READY

●Ownership and Permissions (ala Unix!), Size, Driver...

Hands on! (CLI)

$ oneimage list$ oneimage show tty

# DO NOT EXECUTE THE FOLLOWING COMMAND$ oneimage create --name tty --driver qcow2 \ --path /var/tmp/tutorial/ttylinux.qcow2.img -d default

System Datastore

Basic Usage – Datastores

●Inspect each Datastore:

●The system datastore:

●Holds images for running VMs

●The TM_MAD (transfer manager driver) is shared which means:

Shared Datastore

Basic Usage – Datastores

●The default datastore:

●Holds images ready to be cloned or linked for VMs

●The DS_MAD is fs because our image is a regular file

●The TM_MAD (transfer manager driver) is shared which means:

Adding Networks

Basic Usage – Networks

●Create a new Network

●Name: private

●Type: Fixed Network

●IP: 192.168.0.1 -> [ENTER] -> repeat ... -> 192.168.0.4

●Network Model: default

●Bridge: br1

Node 1

Node 2

Adding Networks

Basic Usage – Networks

●Network extended information:

●Lease Management Add, Hold and Remove Leases

Hands on! (CLI)

$ cat private2.netNAME = private2TYPE = fixedBRIDGE = br1LEASES = [ IP = 10.0.0.1 ]LEASES = [ IP = 10.0.0.2 ]

$ onevnet create private2$ onevnet list$ onevnet show private$ onevnet addleases private 192.168.0.105$ onevnet hold private 192.168.0.105

Adding Templates

Basic Usage – Template

●A template is a Virtual Machine definition ready to be instantiated

●It has CPU, Memory, Disks, NIC, Graphical Ports, etc...

●Create a new Template:

●Name: ttylinux

●CPU: 0.1

●Memory: 64M

●Storage: tty

●Network: private

●Input/Output: VNC

●Random values in Context Custom Variables

●Create!

Adding Templates

Basic Usage – Template

Hands on! (CLI)

●Try the useful --dry option in the CLI

$ onetemplate create --help$ onetemplate create --name ttylinux --cpu 0.1 \

--memory 64 --disk tty --nic private --vnc --dry

NAME="ttylinux“CPU=0.1MEMORY=64DISK=[ IMAGE="tty“]NIC=[ NETWORK="private“]GRAPHICS=[ TYPE="vnc", LISTEN="0.0.0.0" ]

Instantiating

Basic Usage – VMs

●Instantiate the template

●Deploy 2 VMs

●Leave the name blank

●Watch the transition PENDING RUNNING

●In which host is running each VM?

●vnc (vm1) root / password

●ifconfig configured using context

●ping the other machine (vm2)

●Migrate / live-migrate (vm2)

Main VM actions

Basic Usage – VMs

suspend VM state saved. Kept in the host.

power off (--hard) Powers off a VM. Kept in the host.

stop VM state saved. Taken to the system datastore.

undeploy (--hard) Powers off a VM. Taken to the system datastore.

reboot (--hard) Reboots the VM.

delete --recreate Cleans the VM and moves it to PENDING.

shutdown (--hard) Powers off a VM, cleans host and VM is removed from OpenNebula.

delete VM is immediately destroyed regardless of state. Recommended only for oneadmin.

Other VM actions

Basic Usage – VMs

●VM extended information tab

●Capacity - Resize VM capacity (power off –hard, first halt from vnc)

●Storage

●Network - Attach new nic

●create a new network attach nic reboot from vnc

●Snapshot

●create a file using VNC

●Take snapshot

●Modify the file

●Revert

●Placement

●Actions - Schedule action

●Template

●Log

Managing Users

Basic Usage – Users

●Create new user: testuser / testpass

●Click on new user Update Quotas

●Enforce 1 Max VMs Add/edit quota

●Other possible options

●limit the use of a Datastore

●limit the use of an Image

●limit the use of a Network

●Apply changes!

Basic Usage – Logging and Debugging

●Logs are kept under /var/log/one

●oned.log: all the information related to the oned daemon. Every request, actions and driver errors will be here.

The verbosity is set by DEBUG_LEVEL in /etc/one/oned.conf

●sched.log:has all the information related to the placement of Virtual Machines. If a VM is not being deployed (kept in PENDING state), this log will explain why.

●<id>.log: the log of each VM. Also accessible through Sunstone.

oned.conf

Configuration

/etc/one/oned.conf (open this file and take a look!)

●OpenNebula Daemon:

●LOG, PORT, DB

●Monitoring Intervals:

●MANAGER_TIMER, MONITORING_INTERVAL

●Configuration options for VMs:

●VNC_BASE_PORT

●MAC_PREFIX (MAC IP)

●DEFAULT_DEVICE_PREFIX = "hd“ (or vd, xvd, etc…)

●Drivers:

●IM_MAD, VMM_MAD, TM_MAD, DATASTORE_MAD, HM_MAD, AUTH_MAD

●Resources:

●DEFAULT_UMASK

●VM_RESTRICTED_ATTR, IMAGE_RESTRICTED_ATTR

sched.conf

Configuration

/etc/one/sched.conf (open this file and take a look!)

●Scheduler Daemon:

●ONED_PORT, SCHED_INTERVAL, LOG

●Dispatch Options

●MAX_VM, MAX_DISPATCH, MAX_HOST, LIVE_RESCHEDS

●Policy

●DEFAULT_SCHED (packing, striping, load-aware, custom)

We Will Be Happy to Clarify Any Question

Questions?

Appendix A

Installation

This is just a reference of what have been done to configure the frontend_node1 and node2 images

Installation node 1 - Frontend

Installing

●Activate repo and Install Packages

●Add QEMU drivers

●Configure NFS Server

# cp /var/tmp/tutorial/opennebula.repo /etc/yum.repos.d/# yum clean all # yum install opennebula-server opennebula-sunstone opennebula-node-kvm

# Change VM_MAD type from kvm to qemu

# cat /etc/exports

/var/lib/one *(rw,sync,no_subtree_check,root_squash,anonuid=9869,anongid=9869)

Installation node 1 - Frontend

Installing

●Configure Sunstone

●Start Services

●Quick overview of the CLI

# service nfs start# service libvirtd start# service opennebula start# service opennebula-sunstone start# chkconfig nfs on

# gpasswd -a oneadmin wheel# su - oneadmin$ oneuser show$ oneuser -help

# sed -i 's/127.0.0.1/0.0.0.0/' /etc/one/sunstone-server.conf

Installation node 2 – Worker Node

Installing

●Activate repo and Install Packages

●Configure Network, Hostname, NFS and sudo

# cp /var/tmp/tutorial/opennebula.repo /etc/yum.repos.d/# yum clean all # yum install opennebula-node-kvm

# chkconfig NetworkManager off# service NetworkManager stop# echo HOSTNAME=node2 > /etc/sysconfig/network# hostname node2# sed -i 's/1.1.1.1/1.1.1.2/' /etc/sysconfig/network-scripts/ifcfg-br1# ifconfig br1 1.1.1.2/24 up # mount –t nfs 1.1.1.1:/var/lib/one /var/lib/one# gpasswd -a oneadmin wheel# service libvirtd start

Configure Password-less SSH

Installing

●OpenNebula needs passwordless ssh access to all the nodes from all the nodes:

# (as oneadmin)$ ssh-keyscan node1 node2 > ~/.ssh/known_hosts

# test it!

$ ssh node2$ exit$ ssh node1$ exit

tecniris@: opennebula tutorial

Technology

opennebula 5.6 introduction and release...

opennebula conf 2014 | state and future of opennebula -...

opennebula conf 2014: expanding opennebula´s support for...

opennebula conf 2014 | understanding the opennebula model...

opennebula on ubuntu

opennebula conf 2014 | dynamic virtual private clusters with...

opennebula - the project

opennebula conf 2014 | bootstrapping a virtual...

opennebula experiences outlook

opennebula conf 2014: centos, qa an opennebula - christoph...

opennebula and amazon ec2 integration table of contents ·...

opennebula conf 2014 | opennebula at cenatic - jose angel...

opennebula conf 2014 | opennebula as alternative to...

opennebula at bit.nl

interoperability features in opennebula

opennebula techday boston 2015 - an introduction to...

building a hybrid cloud with opennebula opennebula...

the opennebula standard-based open-source toolkit … ·...

opennebula techday waterloo 2015 - an introduction to...

opennebula interoperability