paremus cloud and osgi beyond the vm - osgi cloud workshop march 2012

Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.

OSGi Dev Con 2012 - Cloud Workshop II March 2012

Cloud & OSGi - Beyond the VM

Richard Nicholson: Paremus CEO President of the OSGi Alliance www.paremus.com

http://www.paremus.com




Virtual Machine Man

Amazon EC2, RackSpace, Azure? They all suffer from the same fundamental design flaw.

Virtual Machine image centric Cloud solutions:• Cause excessive network infrastructure load.• Cause excessive storage consumption - caused by virtual machine

image sprawl.• Fail to address to primary issue - the ongoing cost of maintaining

applications.



Artefact Module Centric Image Centric

Machine Image ~ 1 to 2 Gbytes per node ~ 1 to 2 Gbytes per node

Application ~ 1 TO 100 Mbytes ~ 1 to 2 Gbytes per node

Application Patch ~ 100’s Kbytes ~ 1 to 2 Gbytes per node

Property / Config Changes ~ 1 Kbyte ~ 1 to 2 Gbytes per node

Type of Deployment Artefact

VM images: The Wrong deployment artefact!



Observation

1. The Unit of Deployment is...• The unit of fix / maintenance.• The potential unit of re-use.

Also...2. Time to deploy - directly influences...• Time to recover a failed Business Service.• Time to scale an environment in response to business load.

DetectDecision

Response

Need to Optimise

Is Kirk

in the

Room

?

Patterns of Modular Architecture

(Kirk Knoernschild)



Principle: Modularity and Abstraction

The smaller the deployment artefact the better

➟ BUT MUST AT ALL COST avoid Operational Complexity.

Automatically manage dependenciesAutomatically install required parts.

ABSTRACTION - operations must manage the WHOLE, not the constituent PARTS.

➟ The new OSGi Alliance R5.0 OBR (Resolver & Repository specifications)?



Principle: Modularity and Abstraction

The smaller the deployment artefact the better

➟ BUT MUST AT ALL COST avoid Operational Complexity.

Automatically manage dependenciesAutomatically install required parts.

ABSTRACTION - operations must manage the WHOLE, not the constituent PARTS.

➟ The new OSGi Alliance R5.0 OBR (Resolver & Repository specifications)



circa 2005

Paremus introduce the Service Fabric - a distributed OSGi / OBR based Cloud runtime.



The Paremus Service Fabric

Java / Scala, [Blueprint, Declarative Services, WARs & generic artefacts].

(1..n) compute resource (physical or virtual) may contribute to create a Service Fabric

A highly modular OSGi™ / OSGi Resolver based cloud runtime.

(1..m) ‘Composite Applications’ (a.k.a. Systems) may be dynamically assembled and run upon a single Service Fabric.

IaaS

SaaS



Runtime State

= 1

= 5

= 1

Target State

The Service Fabric is a model driven runtime. To change a running System, simply change its model in the required manner.

Re-Configure

Operational simplicity through Abstraction



= 1

= 5

= 1

M!del

Target State

"Structure#SLA$

Runtime

Entity

Monitor

Provision Delta

Planned Deltase.g. Configuration

changes

Unplanned Deltase.g. Resource

failures

Target State Runtime State

Updating the System

The runtime will respond...



Today - 2012

Building consensus that next generation of cloud solution will be composite and will support dynamically assembly highly

modular applications.

Growing realisation that OSGi, and specifically the new OBR resolver specification, are key enablers.



Dependency management really is everything!



OBR Resolution Process

OSGi Resolver

RESOLVER ENVIRONMENT

Physical Repository

OSGi Repository A

OSGi Repository C

OSGi Repository B

‘Scoped’ Repositories(specific views of one - or more - physical

repositories)




OSGi Resolver

Resource/s + Constraints e.g. ‘Pricer (a,c)’


Physical Repository

OSGi Repository A

OSGi Repository C

OSGi Repository B


repositories)




OSGi Resolver



Physical Repository

OSGi Repository A

OSGi Repository C

OSGi Repository B


repositories)

Part ‘Pricer’




OSGi Resolver



Physical Repository

OSGi Repository A

OSGi Repository C

OSGi Repository B


repositories)

Part ‘Pricer’

WHAT NOW?



Centralised Resolver: push / pull resultant composites?

OSGi Resolver

Part ‘Pricer’

Physical Repository

OSGi Repository A

OSGi Repository C

OSGi Repository B

2

Push / Pull The new software

Artefact?

1



• Better than a fully stacked VM (WHAT COULD BE WORSE?!).

• Heavy resolving load on centralised (one or a few) ‘Cloud‘ resolvers.

• Shipping assembled composites to target nodes loads loads network and source nodes.

• No visibility of local environment on target nodes. Resolution may be incorrect or not optimal.

• Example of approach - Apache ACE?

Centralised Resolver: push / pull resultant composites?



Centralised Resolver: Push / Pull List of Bundles?

OBR Resolver

Part ‘Pricer’ Pricer bundle a bundle b bundle c bundle d bundle e bundle f bundle g bundle h

1

2

Physical Repository

OBR Repository A

OBR Repository C

OBR Repository B

*AR



Centralised Resolver: Push / Pull List of Bundles?

• Better than pushing / pulling composite artefacts.• Avoids some network loading - but not the loading

on the centralised resolvers.• Still no visibility of local environment on target

nodes. As before - resolution may be incorrect or not optimal.

• Example of approach?



Service Fabric approach - OSGi Resolver on each node ✔

OSGi Resolver

An OSGi runtime instance

$ ./poshParemus Service Fabric Evaluation License,expires Wed Jan 09 23:59:59 EST 2013.______________________________________Welcome to the Paremus Service Fabric![paremus-ran-2.local.0]% lsb 0 ACTIVE org.apache.felix.framework:3.2.1 1 ACTIVE com.paremus.core.types:1.4.5 2 ACTIVE com.paremus.core.hooks:1.4.5 3 ACTIVE com.paremus.posh.runtime:1.4.5 4 ACTIVE com.paremus.posh.shell:1.4.5 5 ACTIVE com.paremus.console:1.4.5 6 ACTIVE com.paremus.nimble.core:1.4.5 7 ACTIVE com.paremus.nimble.repos:1.4.5 8 ACTIVE com.paremus.nimble.cli:1.4.5 9 ACTIVE org.apache.felix.configadmin:1.2.810 ACTIVE com.paremus.util.cmds:1.4.5

Resolver centric OSGi runtimes are lightweight runtimes ! (example Paremus

Nimble).



Inject Root Resource & OSGi repository scope

Physical Repository

OBR Repository A

OBR Repository C

OBR Repository B


neg

otia

teA negotiation process selects optimal nodes to host the ‘Pricer’ composite



Inject Root Resource & OSGi Repository scope

Physical Repository

OBR Repository A

OBR Repository C

OBR Repository B


Each node performs a local resolution.



Inject Root Resource & OSGi Repository scope

Physical Repository

OBR Repository A

OBR Repository C

OBR Repository B


RESO

LVE

Each node performs a local resolution.



Each participating node asynchronous pulls artefacts

Physical RepositoryResource/s + Constraints e.g. ‘Pricer (a,c)’ Asynchronous flow of bundles from

one or more physical repositories toparticipating nodes

Each node asynchronously downloadsrequired bundles that are not already local.



‘Persistence (b,c)’

‘Gateway (b)’

Repeat

System Models

Physical Deployment Running System = FX Trading

Pricer (b,c) [1..n] Persistence (a,c) [1..1] Gateway (b) [1..m]

In RFC-133 we defined the notion of a ‘System’: A runtime abstraction comprised of elements, wiring and runtime constraints.

<system name="gateway.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>

 <nature group="demo" />

<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>

<system name="gateway" boundary="fibre"> <system.part category="msf" name="com.example.gateway"> <property name="id" value="foo" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> </system></system>

System Model = FX Trading

http://schema.paremus.com/sf/1

<system name="fx.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>



...

...

... </system>



A ‘System’ Model - (example Paremus Service Fabric)

Defines system name and version. Whether System is distributed.

The OSGi repositories to be used when assembling Parts.






....

....

....

</system>



A ‘System’ Model - (example Paremus Service Fabric)

The Pricer Part. A managed service fabric based built from com.example.pricer. An instance of which is created with attribute ‘firm’






....

....

....

</system>



A ‘System’ Model - (Paremus Service Fabric)

Remote Services using the Paremus implementation of the OSGI Alliance RSA specification.

In this example Essence RMI is the protocol - and SLP isused for discover. Normally we recommend DDS - which is the default.

AVRO?






....

....

....

</system>



A ‘System’ Model - (Paremus Service Fabric)

Replication Handlers control the number of Part instances required in the runtime.

....

1

Many





DEMO TIME

1. Start a client. 2. Start single node Service Fabric & a simple OBR repository.3. Load Repository & System model.4. Deploy System. 5. Apply a property change to the running System.



Control Loops & Hierarchies

Highly agile, stable & robust systems tend to use (fast) inner and (slow) outer provisioning control loops (reference http://www.ist-selfman.org/wiki/images/9/94/SelfmanFACS06talk.pdf).

Cloud Runtime

reactive Inner Control Loop (PaaS)Controls distribution

of software components across population of available resources

automatic or manual Outer Control Loop (IaaS)controlling number of VM resources

in the environment

http://www.ist-selfman.org/wiki/images/9/94/SelfmanFACS06talk.pdf






BNDTools <> Service Fabric Integration

Code Run

An OBR centric end-to-end solution



Modular Systems are Maintainable Systems

Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010

⎫⎬⎭

｝Applicability w.r.t OPEX of VM based (non modular) private Cloud solutions.

Thanks

Copyright © 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.

Service Fabric Overview February 2012

www.paremus.com

Service Fabric - www.paremus.com/psfNimble - www.paremus.com/nimble

Bndtools - http://bndtools.org

[email protected]



http://www.paremus.com/psf

http://www.paremus.com/psf

http://www.paremus.com/nimble

http://www.paremus.com/nimble

http://bndtools.org

http://bndtools.org

mailto:[email protected]

mailto:[email protected]

paremus cloud and osgi beyond the vm - osgi cloud workshop march 2012

Technology

cloud osgi

osgi alliance osgi dev

cloud workshop iicopyright

cloud runtime

express permission

modular osgi osgi resolver

recongure osgi dev

paremus ceo