management framework for amazon ec2

Management Framework for Amazon EC2

Speaker:

Frank Bitzer [email protected]

http://www.just-works.de

2 © 2008 F.BitzerManagement Framework for Amazon EC2

Outline

1. Introduction

1. Cloud Computing

2. Amazon EC2

3. Goals & Motivation

2. ECToo – Concepts

1. Base Functions

2. Remoting

3. File Transfer

4. Bundling AMIs

5. Notification Mechanism

3. ECToo - Implementation

1. Architecture

2. Technology Stack

4. Live-Demo

5. Summary & Outlook


Introduction – What means the „Cloud“

Cloud Computing

• Computing Paradigm

• Tasks accomplished by a collection of services

• Origin: Grid Computing

The „Cloud“ …

• … is highly scalable

• … may be spread over multiple data centers

• … usually is accessed via a Web service interface


Introduction – Amazon EC2

„Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud.“

Source: http://aws.amazon.com/ec2/

Designed for developers

Allocate computing power within minutes

• Allows to quickly scale up and down according to requirements

AMI (Amazom Machine Image)

• Preconfigured virtual server image


Introduction – Goals & Motivation

EC2 offers a lot of possibilities

• Scenarios ranging from executing simple stress tests on EC2 to building huge distributed applications that scale on demand

Amazon‘s Web service interface does only provide base functionalities

an enhanced management framework is desirable

• Wrapper around the EC2 API

• Convienient Web service interface

• Web-based GUI

• Encapsulate enhanced functions

• Idea: orchestrating EC2 from BPEL processes


Outline

1. Introduction

• Cloud Computing

• Amazon EC2

• Goals & Motivation


• Base Functions

• Remoting

• File Transfer

• Bundling AMIs

• Notification Mechanism


• Architecture

• Technology Stack

4. Live-Demo



ECToo – Base Functions

As a management tool for EC2, ECToo supports…

• …registering and deregistering AMIs

• …starting, rebooting and stopping instances

– Including the possibility to pass custom user data or files at startup

• …monitoring instances

• …managing security groups

• …managing keypairs

All functionalities are accessible…

• …via Web service interface

• …from a web-based user interface


ECToo – Remoting

Control server instance on EC2 remotely

• Allow execution of arbitrary commands

• Enable full flexibility

– BPEL process can invoke functions on an instance

• Fundament for other enhanced functionalities


ECToo – Remoting II

Returning results

• Standard Output (StdOut)

• Error Output (ErrOut)

• Exit code

All of these values are returned by ECToo Web service

• Caller can evaluate the result


ECToo – Remoting Request & Response


ECToo – File Transfer

Transfer files to an AMI instance

• From local file system

• From S3

• From any URL

Download files from an AMI instance

• To local file system

• No need to download into a S3 bucket since there are a lot of good tools that can be invoked using ECToo‘s remoting capabilities

SCP is used for file transfer

Web service interface uses MTOM


ECToo – Bundling AMIs

„Bundling“ = Process of creating a new AMI based on an existing one

• Used to persist configuration changes

• Long running process, depending on size of AMI

ECToo support for bundling

• Sequence of remote commands

• Encapsulated in one function

• Accessible via Web service and user interface

• Coupled with the ECToo notification mechanism

– Trigger a notification message when bundling has finished

– See next slides…


ECToo – Notification Mechanism

Goals

• Subscribe to events that happen on an AMI instance

• Possibility to react on events such as

– Load exceeds a particular value

– A certain process finishes its work

– …

• Be generic!

– Allow to subscribe any kind of endpoint

– SOAP endpoint: default implementation

– Possible extensions (example)

– Notifications via SMS

• Use the common publish/subscribe pattern

– Support for topics desired


ECToo – Notification Mechanism II

Architecture


ECToo – Notification Mechanism III

Notification Core


Outline

1. Introduction

• Cloud Computing

• Amazon EC2



• Base Functions

• Remoting

• File Transfer

• Bundling AMIs



• Architecture


4. Live-Demo



ECToo – Architecture I

Goals

• Extensibility

– Provide extension points where developers can trim or enhance ECToo

– Ideally without having to change existing code

• Modularity

– Web application and Web service interface share same business logic

– But it should be possible to use one without the other

• Ease of use

– No complicated installation mechanisms

– Convenient and clean design of both Web service interface and GUI


ECToo – Architecture II

Deployment diagram


ECToo – Technology Stack I

General

• Maven 2

– Not only a build system, but a comprehensive software project management tool

– Dependent libraries are fetched from public repositories and added to the classpath

– Transitive dependencies no need to care for dependencies of dependencies

Web service layer

• Axis2

– Web service engine available in C and Java

– Core concept: Modularity

– Modules available e.g. for

– WS-Adressing

– WS-Security

no limits for further enhancements to ECToo


ECToo – Technology Stack II

User interface / web application

• Java Server Faces (JSF)

– Server side user interface component framework for Java web applications

– Realizes MVC pattern

• Facelets

– Alternative view technology that allows to use JSF without JSP

– Comprehensive templating mechanism

• JBoss Seam

– Powerful application framework for building Web 2.0 applications

– Integration framework, integrates AJAX, JSF, Hibernate and various other technologies

• JBoss RichFaces

– Component library for JSF

– Provides AJAX capabilities out of the box


Outline

1. Introduction

• Cloud Computing

• Amazon EC2



• Base Functions

• Remoting

• File Transfer

• Bundling AMIs



• Architecture


4. Live-Demo



Live-Demo

Be impressed…


Outline

1. Introduction

• Cloud Computing

• Amazon EC2



• Base Functions

• Remoting

• File Transfer

• Bundling AMIs



• Architecture


4. Live-Demo



Summary & Outlook

Cloud Computing is a current topic

• Example: Microsoft announced Windows Azure

• Amazon continues to refine its services

ECToo offers a lot of functionalities

• All of them accessible from BPEL processes

• ECToo can be used as a web-based, graphical management tool, too

However… • ECToo does not implement all of the newest features of EC2

– so there is still some work to do…


Thank you…

…for listening!

Please note: ECToo is about to be published!Visit http://ectoo.net to stay up to date.


Discussion

Any questions?

management framework for amazon ec2

Documents