cloud & osgi - the dawn of composite clouds (now with demo videos)
DESCRIPTION
Presentation by Richard Nicholson (Paremus) from OSGi DevCon / EclipseCon 2011. Now with demo videos. Despite the element of novelty and fashion, there is little doubt that Cloud Computing will have a fundamental and long lasting influence on the technology landscape. Yet virtual machine based Cloud Compute offerings, which attempt to maximise resource utilisation and minimise operational management of those resources, have nothing to say about the dominant contributor to an organisations IT OPEX. Application maintainability accounts for approximately 70% of an applications TCO. To increase application maintainability, one must modularise and preferably modularise using an industry standard. Hence, for organisations with large in-house development teams, it is predicted that OSGi will have equivalent or greater impact than Cloud Computing! This presentation will look at the intersect of Cloud Computing and OSGi based Composite Applications. Areas explored will include the importance of dynamic dependency management, the anatomy of cloud enabled composite applications and the role of the PaaS in an OSGi enabled Cloud. Relevant OSGi standards will be reviewed along with how these may be used to address the configuration and management of distributed Cloud hosted composite applications. The presentation will conclude by demonstrating a distributed Cloud / OSGi runtime that demonstrates the concepts discussedTRANSCRIPT
Transforming the Way the World Runs Applications
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Cloud & OSGi - The Dawn of Composite Clouds
[email protected] Nicholson
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Mythology, marketing...
:-)
:-(
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Cloud Computingaddresses this
?
The Problem is...
Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
✗ Deployment of static software images
✔ Dynamic Assembly composites applications
Modular Systems are Maintainable Systems
Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
IaaS
SaaS
Introducing the Paremus Service Fabric
(1..m) ‘Systems’ may run upon a single Service Fabric
(1..n) compute resource may contribute to a Service Fabric
Fabric ‘EclipseCon’ = ∑ all ‘EclipseCon’ fibres
An OSGi™ technology based cloud runtime
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Off Fabric node
Bootstrap Atlas ManagerSource Repository for Examples
POSH CLIApplication UI
Service Fabric Nodes
Service Fabric InfrastructureCached Examples
Deployed Systems
Demo EnvironmentA ‘blue’ Atlas Agent
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Managing characteristics of population
rather than each individual node has a
dramatic effect on decreasing runtime
complexity -
http://adaptevolve.blogspot.com/
2008/01/complexity-part-ii-it-all-
depends-on.html
Atlas & Resource Target StateA blue !"bre#
Service Fabric $ EclipseCon
Atlas agents:host resource property: colour
nonered
blue
fibres: \ infra; quota=[10..10]; filter=(&(Group=Red)(os.name=Darwin), \ default; quota=[100..1000]; filter=(Group=Blue) \
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Demo: Bootstrapping a Service Fabric
A video of the demo is available on the next page
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
System Description Running System
A Model Driven Runtime
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
System
=System
ElementsScaling Behaviour
(Replication Handlers)
+Resource Contract
= (cost_center=engineering)
= !(os.name=Windows)
= (os.name=linux) & (CPU.speed > 3 Ghz)
= fn(z)
= fm(y)
= fl(x)
A System
A Reference to...an OSGi assembly ora WAR, or EAR or a package name or evena named VM image.
service wires (remote) Service AService BService DService EService F
OSGi Service Registry
Service Propertiesfor E
ExportFabric SystemSystem Element
Topology Manager
Remote Service AdminEssence RMI
Remote Service AdminAVRO
Discovery ProviderSLP
Discovery ProviderDDS
E
E
E
Service E EndpointEssence RMIE
Service Eover Essence RMI
E
!
"
#
$
%
&
'
Export Registration
OSGi - Remote Service / Remote Service Admin
+
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
= 1
= 3
= 1
Target State Runtime State
Deploy
Starting a System
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
M!del
Target State
"Structure#SLA$
Runtime
Entity
Monitor
Provision Delta
Planned Deltase.g. Configuration
changes
Unplanned Deltase.g. Resource
failures
= 1
= 3
= 1
Target State Runtime State
The Service Fabric responds by creating the corresponding System
Starting a System
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Runtime State
= 1
= 5
= 1
Target State
To change a runtime System, change its model in the required manner.
Re-Configure
Updating a System
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
= 1
= 5
= 1
M!del
Target State
"Structure#SLA$
Runtime
Entity
Monitor
Provision Delta
Planned Deltase.g. Configuration
changes
Unplanned Deltase.g. Resource
failures
Target State Runtime State
The Service Fabric responds by modifying the running System
Updating a System
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Demo: Application Agility
A video of the demo is available on the next page
❶
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
JEE
EAR
Each ‘fibre’ dynamically assembles and configures any local infrastructure services in response to the runtime dependencies of each hosted System Element
A self-configuring PaaS Modular & runtime dependencies
WAB
WAR
triggers policy based runtime assembly
triggers policybased runtime assembly
❷
❶
❷
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Robust Systems...
Reddit Went Down: Blame Amazon, the Cloud or Both?
“I am a data storage/data center specialist. It pains me to see the "cloud" illusion come crashing down on a great product like Reddit.”......
“It's no secret among most IT folks that the cloud really isn't cheaper than rolling your own infrastructure for reasons exactly like this.”
http://www.readwriteweb.com/cloud/2011/03/ipad-for-business-round-up-web.php
Financial Services Organization: “How do you stop people rebooting machines....?”
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
•A ‘Robust’ platform is designed to survive cascading failure.
•A ‘Robust’ platform is adaptive & self-repairing: continually attempting to ‘settle’ into the desired state.
•No central or static points of command & control
Adaptive & Recovery Oriented
What do we mean by Robust?
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Whac-A-Mole :)
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Whac-A-Mole the Paremus Way
A video of the demo is available on the next page
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
A
B
C
Management & Monitoring
ProvisionerSystem Managers
RegistryService Advertisements
RepositoryOSGi bundles
System DescriptionsNimble Policies
WAR EAR
General artifacts
Principles:• No ‘special’ management nodes• Source so*ware repository may fail• Population changes over time (nodes come and go)• Systems change over time (upgrades / roll-backs)• Attempting to cache everything / everywhere / is doomed to
failure
Approaches:• All infra services: Dynamic Group formation / re-formation• All infra services : Dynamic leadership election• On-fabric Repository Cache: Eventual Consistency across
self-selecting set of participating members
Robust PaaS Services
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
CDS Behaviour - Form Group & Elect Leader
CDS Group Lead! active members
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
A System is Loaded
= 1= 5= 1
!
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Artefacts are automatically cached in response
Business Group Repository
Business Division Repository
!
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Artefacts propagated around Group
!
"
#
$
Data Flow
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Lead Fails
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
New leader elected - Group closes around failure
!
"#
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
New members join team
!
"
#
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
No Silver Bullets!Consistency, Availability, Partition-ability (CAP) trade-offs still apply. Use most appropriate data service for each business application.
• Unstructured data processing - Hadoop
• Key / Value - Voldemort
• Column - Cassandra
• Graph Database - Neo4j
• Relational - Derby, MySQL
• Distribution to nodes - BitTorrent
What about Application Data?
http://blog.nahurst.com/visual-guide-to-nosql-systems
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Late Break Demos....
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Introducing Zeus
+A video of the demo is
available on the next page
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Lab49 is a strategy, design and technology consulting firm. Lab49 deliver strategic consulting and build advanced solutions for the world’s leading investment banks, hedge funds and exchanges.
“Lab49 deliver world-class solutions to some of the most sophisticated firms in financial services today. Our innovative approach stems from adopting user-centric practices enabling clients to respond quickly to today’s challenges, as well as realize new business opportunities in the modern world of trading.
SDP Foreign Exchange (FX) functionality;
‣Request for Quote (RFQ)
‣Request for Stream (RFS)
‣ Execution blotter
‣Research/News
Lab49 SDP leverages;
‣ Paremus OSGi Service Fabric
‣ Akka Open Source Agent-Actor Framework
‣ Lab49 Design and Engineering
‣HTML5/JavaScript
Copyright © 2011 Lab49 UK Ltd.May not be reproduced by any means without express permission. All rights reserved.
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Demo: Single Dealer Platform
Copyright © 2011 Lab49 UK Ltd.May not be reproduced by any means without express permission. All rights reserved.
For access to demo videoplease contact
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Industry Standards...
• OSGi framework - Equinox, Knopflerfish or Felix
• Paremus OSGi command shell (Posh): OSGi RFC147
• Paremus Nimble Resolver: OSGi OBR resolver (RFC112)
• Paremus implementation of OSGi Remote Services Administration
• OSGi ConfigAdmin
• Paremus implementation of OSGi Web Archive Bundle (RFC66)
• RTI implementation of OMG DDS: service discovery, configuration, management & monitoring
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Industry Standards...
• Cloud / OSGi workshop organised by Peter Kriens - EclipseCon 2011
• Ongoing EEG conference calls throughout 2010 / 2011 chaired by David Bosschaert
• RFP-133 Cloud Computing - Current Status
• To get involved: contact David Bosschaert or Peter Kriens
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
http://dspace.mit.edu/bitstream/handle/1721.1/60085/BBFRFC66.pdf?sequence=1
Modularity is a Must Have not a Maybe!
Patterns of Modular Architecture (Kirk Knoernschild) - http://www.kirkk.com/modularity/chapters/
Diversity and Complexity - Scott E. Page
Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview March 2011
Distributed OSGi - Dr Robert DunneTuesday, 10:40, 20 minutes | Ballroom BC
Any Questions?
& drop by the Paremus stand for a chat!
OSGi DevCon BoFTuesday, 7:30 | Napa