1 towards a generic deployment framework for j2ee, osgi, web services (or everything) tuesday,...
Post on 15-Jan-2016
215 views
TRANSCRIPT
1
Towards a Generic Deployment Framework for J2EE, OSGi, Web Services(or Everything)
Tuesday, February 28th, 2006
Philippe Merle
Email: [email protected] Project - INRIA Futurs,Laboratoire d‘Informatique Fondamentale de Lille (LIFL),Université des Sciences et Technologies de Lille (USTL),Villeneuve d’Ascq, France
ITEA S4ALL Task 3.5 Meeting, INT, Evry, France
2
Un brin d’histoire
• OpenCCM• Plate-forme de composants répartis CORBA
• Automatisation du déploiement
• Tout est spécifique au CCM
• Identifier des abstractions logicielles pour construire des plates-formes de déploiement
• Qu’est ce qui peut être factoriser en concepts/codes entre le déploiement OpenCCM et le déploiement Fractal ?
• Voir articles [DECOR 2004], [LMO 2005]
• Le travail présenté aujourd’hui est dans la poursuite de cette recherche du Graal
3
Agenda
• Context• Deployment of distributed services and systems
• Motivation• No generic deployment approach/tool/framework exists!
• Objectives• Build a generic deployment framework
• Principles• A fine grain component-based framework with Fractal
• Illustrations• Some of the Fractal-based components
• Work plan
• Conclusion and perspectives
4
Context - Multiple Services and Providers
• The user-centric approach considers the users’ services as gathering of the many services provided by the all-around world.
• Within each involved platform, deploying these services is made easy.
Why not everything deployed in once?
OSGi Gateway J2EE Server
Web Server <insert yours>
Directories
Banking Facilities
Extra Stuff
Monitoring
5
We Would Like To Have Something Which…
• Starts various processes on remote hosts
• Enables complex configurations for the launched processes
• Deploys applications’ services inside the hosting platforms
• Adds some test / control over this deployment
• Relays on nothing more than existing facilities• Operating systems, remote shell accesses, etc.
Via an easy to use, automation, extensible, and efficient approach!
6
Dependencies for remote deployment/launching
ServiceTowards
Users
Towards Systems
But Heterogeneity At Every Layer
Operating System Linux Windows Mac OS
SSH Telnet OSGiRemote Access ProActive
JOnAS Oscar JOnASApplication Server JOnAS
Requires a unified approach!
7
Motivation
• Deployment of distributed applications is a complex task, e.g.:
• Connect to remote hosts
• Download application (and middleware) binaries
• Launch middleware services and application servers– Set up environment variables– Start processes
• Install application binaries into application servers/gateways
• Configure and publish application services
• Many other specific tasks!
8
Motivation - Related Work
• Scripting is just a tool instead of a paradigm, e.g.:• How to reuse scripts?
• Many work done to automatically deploy applications• Into application servers/gateways, e.g.:
– J2EE, OSGi, Fractal, etc.
• On distributed systems, e.g.:– OLAN, OpenCCM, ProActive, JADE, etc.
• Via generic frameworks, e.g.– [DECOR 2004], etc.
• Via model driven approaches, e.g.– [LMO 2005], ORYA, OMG DnC, etc.
9
Motivation - Related Work
• No work cover the whole deployment process in a generic and technological independent way, e.g.:
• J2EE/OSGi do not address the launching of J2EE/OSGi gateways
• OSGi does not address deployment of distributed applications
• ProActive only deploys ProActive-based servers and applications
• OpenCCM only deploys CCM-based applications but not OpenCCM servers
• JADE deploys J2EE systems but not JADE servers
• Etc.
10
Motivation - Requirements
• Providing a unified approach• Heterogeneity of technologies
• Independent against granularity
• Automation instead of manual operations
• Distributed synchronizations, e.g.:• Connect to remote hosts before do anything
• Set up environment variables before launch processes
• Launch a transaction service before launch application servers
• Launch a registry service before publish application services
• Etc.
• Describing instead of programming
11
Objectives
• Build a generic deployment framework• Unified approach• Covering any task required to deploy
• Independent against granularity• Distributed systems, applications, services, components, objects, etc.
• Independent against technology• Usable with J2EE, OSGi, Web Services, Fractal, OpenCCM, etc.
• Easy to use by end-users• Describing what must be deployed instead of how to do
• Highly customizable and extensible by developers• Adapting the framework to specific purposes• Adding new concepts/mechanisms easily
12
Principles for a Unified Approach
• A framework instead of a single tool• Build tools from the framework
• Everything is reified as components• From end-user configurations to technical entities
• Fine grain component-based approach• Micro components and interfaces
• Intensive use of composition and sharing
13
Everything is Reified as Components
• A hosting system is a component containing a remote access
• This remote shell provides the interfaces to set variables and execute processes on the hosting system
• An execution platform is a component managing applications and services, with many interfaces such as deployment, monitoring, configuration, and so on. hosting system
remote shell
execution platform
service
service
14
Everything is Reified as Components
Network
hosting system
remote shell
execution platform
serviceservice
Reify and Control
15
Everything Reified as Components• Remote access protocols such as SSH, TELNET, etc.
• File transfer protocols such as SCP, FTP, HTTP, etc.
• Shells such as SH, CSH, WINBAT, etc.• Bound to a protocol
• Shell variables and commands• Bound to a shared shell
• Servers such as registry services and transaction managers (RMI, CORBA, WS), etc.
• Composed of shell variables and commands
• Application servers as JOnAS, OSGi, etc.• Composed of shell variables and commands
• Application beans as J2EE jars, OSGi bundles, etc.• Bound to a shared application server
• Etc.
16
Everything Reified as Components
protocol: SSH SSHd
SHshell: SH
host
Reified System Real System
JOnAS
EJB
server: JOnAS
bean: EJB
17
Technical Approach
This generic deployment framework is
• Designed with the Fractal component model• Each deployment concept/mechanism is a Fractal component
• Implemented with the Fraclet annotation framework• Leverage Component-Based Software Engineering
• Configured with Fractal ADL• End-users’ configurations
• Developers’ components
• Managed with Fractal Explorer
• Distributed with Fractal RMI
• Could benefit from any other Fractal-based work• AOKell, FAC, etc.
18
Technical Architecture
Network
End-UserConfiguration
Fractal ADL
Fractal ADL
ComponentsComponentsDeploymentComponents
hosting system
remote shell
execution platform
serviceservice
Fractal Explorer
Deploy
Translation Manage
Insta
ntiat
e
Fraclet
19
A Simple End-User ConfigurationThe End-User Description
MySystem = {
MyBean = BEAN {
location = URL(http://www.lifl.fr/bean.jar) ;
server = JONAS {
home = JONAS_HOME(/usr/jonas) ;
host = HOST {
hostname = HOSTNAME(host.lifl.fr) ;
protocol = SSH ;
shell = SH ;
}; }; }; };
An easy to use end-users’ configuration languageDescribing what the configuration is instead of how to deploy it
20
A Simple End-User ConfigurationThe Fractal Components
MySystem
MyBean: BEAN
location: URL server: JONAS
home: JONAS_HOME
host: HOST
hostname: HOSTNAME
protocol: SSH
shell: SH
21
A Simple End-User ConfigurationThe Fractal ADL Definition
<?xml version="1.0" encoding="ISO-8859-1" ?><!DOCTYPE definition PUBLIC "-//objectweb.org//DTD Fractal ADL 2.0//EN” "classpath://org/objectweb/fractal/adl/xml/standard.dtd">
<definition name="MySystem">
<component name=“MyBean" definition="BEAN">
<component name="location" definition="URL(http://www.lifl.fr/bean.jar)"/>
<component name="server" definition="JONAS">
<component name="home" definition="JONAS_HOME(/usr/jonas)"/>
<component name="host" definition="HOST">
<component name="hostname" definition="HOSTNAME(host.lifl.fr)"/>
<component name="protocol" definition="SSH"/>
<component name="shell" definition="SH"/>
</component>
</component>
</component>
</definition>
22
A Simple End-User ConfigurationThe Expected Runtime Behavior
Deploying MyBean implies to automatically
1. Connect to host.lifl.fr via the SSH protocol
2. Setup the JONAS_HOME variable to /usr/jonas
3. Add $JONAS_HOME/bin/unix to the PATH variable
4. Launch the command jonas start
5. Download http://www.lifl.fr/bean.jar
6. Launch the command jonas admin –a bean.jar
23
A More Complex End-User Configuration
MySystem = ITEA_S4ALL {
hosts = {
...
};
servers = {
...
};
beans = AUTO_INSTALL {
...
};
};
24
A More Complex End-User Configuration
MySystem = ITEA_S4ALL {
hosts = {
M = HOST {
hostname = HOSTNAME(host.lifl.fr) ;
protocol = SSH ;
shell = SH ;
};
… other host definitions ...
}; };
25
A More Complex End-User Configuration
MySystem = ITEA_S4ALL {
servers = {
jonas_on_M = JONAS {
home = JONAS_HOME(/usr/jonas) ;
host = hosts/M ;
};
oscar_on_M = OSCAR {
home = OSCAR_HOME(/usr/oscar) ;
host = hosts/M ;
};
... Other server definitions …
}; };
Sharing:- The M host- The SSH channel- The SH shell
26
A More Complex End-User ConfigurationMySystem = ITEA_S4ALL {
beans = AUTO_INSTALL {
bean1 = BEAN { location = LOCAL_FILE(bean1.jar) ; server = servers/jonas_on_M ;
};
bean2 = BEAN { location = LOCAL_FILE(bean2.jar) ; server = servers/jonas_on_M ; };
bean3 = BEAN { location = LOCAL_FILE(bean3.jar) ; server = servers/oscar_on_M ; };
bean4 = BEAN { location = LOCAL_FILE(bean4.jar) ; server = servers/oscar_on_M ; };
... Other bean definitions …
}; };
JOnAS on host Mshared by beans
OSCAR on host Mshared by beans
Ask for auto installation
27
A More Complex End-User ConfigurationThe Expected Runtime Behavior
1. Connect to host.lifl.fr via the SSH protocol
2. Setup the JONAS_HOME variable to /usr/jonas
3. Add $JONAS_HOME/bin/unix to the PATH variable
4. Launch the command jonas start
5. Download http://www.lifl.fr/bean1.jar
6. Launch the command jonas admin –a bean1.jar
7. Download http://www.lifl.fr/bean2.jar
8. Launch the command jonas admin –a bean2.jar
9. Setup system variables related to OSCAR
10. Launch OSCAR
11. Execute OSCAR commands to install bean3.jar and bean4.jar
• Other execution sequences and/or parallelization are possible
28
Everything is Reified as Components
Network
hosting system
remote shell
execution platform
serviceservice
Control
29
Everything is a Service(Not a Web Service)
• Each component has a Service interface to control the life cycle of the real entity reified
• interface Service { void start(); void stop();}
• Semantics depends on components/implementations• Protocol.start ~ initializes the network connection
• Protocol.stop ~ shutdowns the network connection
• Server.start ~ launches the server process
• Server.stop ~ stops the server process
• Bean.start ~ launches the bean into the application server
• Bean.stop ~ uninstalls the bean from its server
30
Composition
Composition of Services
Host.inria.fr
ShellSH
ProtocolSSH
JOnAS
OSCAR
Bean
Bean
Bean
Bean
31
Orchestration of Services
• Goal• Address distributed synchronizations of composition of services
• Orchestration• Implements the Service interface
• Schedules execution of services automatically
• Implementation• Introspects the composition and bindings between components to
plan the execution of services
• Various execution strategies– Sequential– Parallel
32
Orchestration
Orchestration of Services
Host.inria.fr
ShellSH
ProtocolSSH
JOnAS
OSCAR
Bean
Bean
Bean
Bean
Service
33
Orchestration
Orchestration of Services
Host.inria.fr
ShellSH
ProtocolSSH
JOnAS
OSCAR
Bean
Bean
Bean
Bean
(1)start (2)
start(3)start
(4)start
(5b)start
(6) start
(5a)start
(6) start
(6) start
(6) start
34
Orchestration
Orchestration of Services
Host.inria.fr
ShellSH
ProtocolSSH
JOnAS
OSCAR
Bean
Bean
Bean
Bean
(1)stop (6)
stop(5)stop
(4)stop
(3)stop
(2) stop
(3)stop
(2) stop
(2) stop
(2) stop
35
Protocol Components
• Reify and encapsulate protocols to access the real system
• interface Protocol { void send(String command); }
• Possible implementations• LOCAL by wrapping an external local shell• SSH by wrapping
– an external local ssh process– the BSD open source JSCH library
– http://www.jcraft.com/jsch/index.html• TELNET by wrapping
– an external local telnet process– the open source JTelnet library
– http:// Source Forge– the Apache open source Jakarta Commons Net library
– http://jakarta.apache.org/commons/net
Protocol
Service
36
Internet Protocol Components
Internet Protocol
Basic Protocolas
SSH, TELNET
IP HostName
IP Port
37
Implementation of the IP Port Component with Fraclet
/** @fractal.itf name=“port” */interface IpPort { int getIpPort();}
class PORT implements IpPort { /** @fractal.ac */ protected int port;
public int getIpPort() { return port; }}
• Fractal AttributeController interface & implementation, and Fractal ADL definitions generated automatically by Fraclet :-)
38
/** @fractal.itf name=“hostname” */interface IpHostName { String getIpHostName();}
class HOST_NAME implements IpHostName { /** @fractal.ac */ protected String hostname;
public String getIpHostName() { return hostname; }}
• Fractal AttributeController interface & implementation, and Fractal ADL definitions generated automatically by Fraclet :-)
Implementation of the IP Host Name Component with Fraclet
39
Implementation of the Basic Internet Protocol Components with Fraclet
abstract class InternetProtocol implements Service, Protocol { /** @fractal.bc */ protected IpHostName hostname; /** @fractal.bc */ protected IpPort port;}
class Telnet extends InternetProtocol { public void start() { . . . hostname.getIpHostName() . . . . . . port.getIpPort() . . . } public void stop() { . . . } public void send(String command) { . . . }}
• Ditto for all possible implementations, e.g.:• TELNET, SSH, etc.• Via an external process or an embedded Java library
40
Shell Components
• Reify shells executed in the real system
• interface Shell { void set_variable(String name, String value); void unset_variable(String name); void execute(String command); }
• Each shell is bound to a single Protocol component
• Behavior• Construct commands according to the shell syntax• Format path separator, i.e. / \ : ;
• Possible implementations• SH for Borne Unix shells• CSH for C Unix shells• WINBAT for Windows command shells
ProtocolShell
41
The HOST Component
• Provide an abstract composition of (Shell, InternetProtocol, Hostname) <definition name=“HOST”> <component name=“hostname” definition=“IpHostName”/> <component name=“protocol” definition=“InternetProtocol”/> <component name=“shell” definition=“ShellComponent”/> <binding client=“shell.protocol” server=“protocol.protocol”/> <binding client=“protocol.hostname” server=“hostname.hostname”/> </definition>
• Simplify the configuration for end-users <component name=“MyHost” definition=“HOST”> <component name=“hostname” definition=“HOSTNAME(host.lifl.fr)”/> <component name=“protocol” definition=“SSH”/> <component name=“shell” definition=“SH”/> </component>
shell protocol hostname
42
The VARIABLE Component
• Reify shell variables
• Each variable is bound to a shared Shell component
• Possible implementation class VARIABLE implements Service {
/** @fractal.bc */protected Shell shell;/** @fractal.ac */protected String name;/** @fractal.ac */protected String value;public void start() { shell.set_variable(name, value); }public void stop() { shell.unset_variable(name); }
}
• Possible instantiations• JONAS_HOME(path) = VARIABLE(JONAS_HOME, path)• OSCAR_HOME(path) = VARIABLE(OSCAR_HOME, path)
VARIABLE
ShellService
43
The Command Component• Reify shell commands on the real system• interface Command { void execute(Map arguments); }
• Each command is bound to a shared Shell component
• Possible implementation class COMMAND implements Command {
/** @fractal.bc */protected Shell shell;/** @fractal.ac */protected String command;private String computeCommand(Map arguments) { . . . }public void execute(Map arguments) { shell.execute(computeCommand(arguments));
}}
• Possible instantiations• COMMAND(echo hello world!)• COMMAND(jonas start)• COMMAND(jonas admin –a %url%)
COMMAND
Shell
Command
44
Server Components
• Reify a server/daemon executed in the real system
• Examples• Middleware servers
– Registry
– Transaction Manager
• Apache servers
• Various implementations• Wrapping a protocol to interact with the remote server
• Bound to a shell
• Etc.
SERVERService
45
Implementation of the SERVER Component by Composition
VARIABLE
shell
COMMANDstart
ServerImpl
SERVER
Service
*
COMMANDstop
SERVER*
depend
46
Implementation of the SERVER Component by Composition (Java code)
class ServerImpl implements Service {
/** @fractal.bc signature=“Service” */ protected Map required-servers;
/** @fractal.bc signature=“Service” */ protected Map variables;
/** @fractal.bc */ protected Command start-command;
/** @fractal.bc */ protected Command stop-command;
public void start() { start-command.execute(null);
}
public void stop() { stop-command.execute(null);} }
48
Other Concerns to Address• Display output from remote hosts
• Various GUI • Also Fractal components• Configurable by end-users
• Take care of system resource management• Threads and Sockets are limited• Study work done on DREAM• Use AOKell to build new controllers for system resources• Quid for deployment of large scale systems
• Distribute Fractal components into several JVM/host• To address system resource limitation• Using Fractal RMI
• Transactional deployment• GoTM framework
• Dynamic reconfiguration• Fractal Explorer to manage components, as done by JADE• Rules for autonomous computing
49
Work Plan
• Proof of concept already done• Areski Flissi and Philippe Merle involved
• Engineer recruitment from March 1st 2006
• Define Fractal interfaces and components
• Develop micro-components with Fraclet
• Compose components with Fractal ADL
• Implement Fractal ADL translator
• Implement management with Fractal Explorer
• Evaluate on various ITEA S4ALL scenarios
50
Conclusion
• A deployment framework• Generic
• Independent against technology and granularity
• Easy to use
• Highly customizable and extensible
• Fine grain component-based approach• Each deployment mechanism is reified as a component
• Intensive use of the Fractal component model and tools• Composition and sharing
• Julia, AOKell, Fraclet, Fractal ADL, Fractal Explorer, etc.
• Will be developed soon• Prototyping already done
• Implementation details could change but not the overall approach
51
Perspectives
• Deployment on Grid’5000 Specific components for OAR and KaDeploy services
• Deployment of OpenCCM Specific components for OpenCCM daemons Automation on PDA prototyped
• Deployment of Fractal Specific components for Fractal RMI Registry and Server
• Autonomous computing• Rules for dynamic reconfiguration – Ph.D. Jérémy Dubus
52
Thank you for your attention…
If you have any questions?
53
TODO
The JRE HOST Component
Counter Start/stop + synchronization sur start
ApplicationServer
JONAS
OSGi
BEAN
File transfer
54
class COMPUTED_HOSTNAMEimplements IpHostName {
/** @fractal.bc */ protected IpHostName computer;
protected String hostname = null;
public String getIpHostName() { if(hostname==null) hostname = computer.getIpHostName(); return hostname; }}
• Fractal BindingController code generated automatically by Fraclet :-)
• Hostname computation could be done by a reservation system like OAR on Grid’5000 :-)
Another Implementation of the IP Host Name Component with Fraclet
COMPUTED_HOSTNAME
COMPUTED_HOSTNAME
HOSTNAME_COMPUTER
55
Automation Protocol Components
PROTOCOL
Do start()at the 1stsend()
AnyProtocol
56
Implementation of Automation Protocol Component with Fraclet
class AutomationProtocol implements Protocol { /** @fractal.bc */ protected Protocol next;
private boolean isInit = false;
public synchronized void init() { if(isInit == false) { next.init(); isInit = true; } }
public void send(String command) { this.init(); next.send(command); }
public void shutdown() { next.shutdown(); }}
57
The JRE Component
• Provides an abstract composition for JRE
VARIABLEJAVA_HOME shell
VARIABLEPATH
JREImpl
JRE
58
Java Implementation of the JRE Component
class JREImpl implements Environment { /** @fractal.bc */ protected Environment home;
/** @fractal.bc */ protected Environment path;
public void set(){ home.set(); path.set(); }
public void unset() { home.unset(); path.unset(); }
}
59
Fractal ADL for the JRE Component• Fractal ADL definition
<definition name=“JRE” extends=“Environment”> <component name=“jre” definition=“JREImpl”/> <component name=“home” definition=“JAVA_HOME()”/> <component name=“path” definition=“VARIABLE(PATH,$JAVA_HOME)”/> <component name=“shell” definition=“Shell”/>
<binding client=“this.env” server=“jre.env”/> <binding client=“jre.java_home” server=“home.env”/> <binding client=“jre.path” server=“path.env”/>
<binding client=“home.shell” server=“shell.shell”/><binding client=“path.shell” server=“shell.shell”/>
</definition>
• Fractal ADL usage
<component name=“MyJre” definition=“JRE”> <component name=“home” definition=“JAVA_HOME(/usr/java)”/> <component name=“shell” definition=“hosts/M/shell”/> </component>