c ompiling h igh-level a ccess i nterfaces for m ulti-site s oftware
Post on 19-Mar-2016
21 Views
Preview:
DESCRIPTION
TRANSCRIPT
June 1998 CHAIMS 1
Compiling High-level AccessInterfaces for Multi-site Software Stanford University
Objective: Investigate revolutionary approaches to large-scale software composition.
Approach: Develop and validate a composition-only language.
Planned contributions: • Asynchrony by splitting up CALL-statement.• Hardware and software platform independence.• Potential for multi-site dataflow optimization.• Performance optimization by invocation scheduling.
CHAIMS: Mega-Programming Research
CHAIMS
June 1998 CHAIMS 2
Participants• Support
– DARPA ISO EDCS program (1996-1999)– Siemens Corporate Research (1996-1998)– DoD AFOSR AASERT student support (1997-1999)– Sloan Foundation - computer industry study (1996-97)
• People - Gio Wiederhold PI -- Marianne Siroker: Administration– Dorothea Beringer (postdoc EPF Lausanne) since Dec.1997– Ron Burback (CS PhD cand.)– Laurence Melloul (CS MS)– Woody Pollack (CS MS)– MS and BS CS graduated: Joshua Hui, Gaurav Bhatia, Prasanna
Ramaswami, Kirti Kwatra, Pankaj Jain, Mehul Bastawala, Catherine Tornabene, Wayne Lim, Connan King
– Louis Perrochon (postdoc ETH Zurich) Fall quarter 1996
June 1998 CHAIMS 3
My Personal Background• Masters in Computer Science: hybrid-monitoring tool
for debugging and software performance analysis for distributed software
• Software engineer: telecommunication systems
• Consultant: software methodologies, quality assurance, project management, CASE-tools
• PhD: Modeling scenarios in object-oriented analysis
• Teaching: Fusion
• Now: CHAIMS -- large-scale software composition, distributed systems
June 1998 CHAIMS 4
Presentation• Motivation and Objectives
– changes in software production– basis for new visions and education
• Concepts of CHAIMS– CHAIMS language– CHAIMS architecture and composition process– Scheduling– Dataflow optimization
• Status, Plans, Conclusions
June 1998 CHAIMS 5
Coding
Integration
1970 1990 2010
Shift in Programming Tasks
June 1998 CHAIMS 6
Typical Scenario: LogisticsA general has to ship troops and/or various material from L.A. to Washington DC:
– different kind of material: criteria for preferred transport differ– not every airport equally suited– congestion, prices– weather constraints– exact due or ready dates
Today: calling different companies, looking up information on the web, reservations by hand
Future: system proposes possibilities that take into account various conditions
» hand-coded systems» composition of processes
June 1998 CHAIMS 7
Scaling alternatives
?
June 1998 CHAIMS 8
C H A I M S
Megamodules
Megaprogram for composition, written by domain programmer
CHAIMS system automates generation of client for
distributed system
Megamodules, provided by various megamodule
providers
CHAIMS
June 1998 CHAIMS 9
Megamodules - DefinitionMegamodules are large, autonomous, distributed,
heterogeneous services or processes.• large: computation intensive, data intensive, ongoing processes
(monitoring services)• distributed: to be used by more than one client• heterogeneous: accessible by various distribution protocols (not
only different languages and systems)• autonomous: maintenance and control over recourses remains
with provider, differing ontologies ( ==> SKC)
Examples:– logistics: “find best transportation route from A to B”,
reservation systems– genomics: easier framework for composing various processing
tools than ad-hoc coding
June 1998 CHAIMS 10
Challenge: Thin Clients (1) Domain expert
Client computer
Control &Computation
Services
I/O
a bcd
e
Wrappers to resolve
differences
I/O
DataResources
June 1998 CHAIMS 11
Challenge: Thin Clients (2) Domain expert
Client workstation
ComputationServices
IO module
MEGA modules
IO module
ab
cd
e
DataResources
Sites RT
S U T
C
June 1998 CHAIMS 12
Challenge: Heavy-weight Services
Services are not free for a client:• execution time of a service• transfer time for data• fees for services
What we need:==> monitoring progress of a service==> possibility to choose among equivalent services
based on estimated waiting time and fees==> parallelism among services==> preliminary overview results, choosing level of
accuracy / number of results for complex processes==> novel optimization techniques
June 1998 CHAIMS 13
Challenge:Non-technical Domain Experts
Company providing services:• domain experts of domain of service (e.g. weather)• technical experts for programming for distribution
protocols, setting up servers in a middleware system• marketing experts
“Megaprogrammer”:• is domain expert of domain that uses these services• is not technical expert of middleware system or
experienced programmer,• wants to focus on problem at hand (=results of using
megaprogram)• e.g. scientist, assistant of a general
June 1998 CHAIMS 14
Challenge: Purely Compositional Language Possible?
Which languages did succeed?– Algol, ADA: integrated composition and computation– C, C++: focus on computation
Why new language?– complexity: not all facilities of a common language (compare
to approach of Java), – inhibiting traditional computational programming (compare C+
+ and Smalltalk concerning object-oriented programming)– focus on issue of composition, parallelism by asynchrony,
and optimization
June 1998 CHAIMS 15
CHAIMS “Logical” Architecture
Customer
Megaprogramclients(in CHAIMS)
Network/Transport(DCE, CORBA,...)
Megamodules(Wrapped or Native)
June 1998 CHAIMS 16
CHAIMS Physical Architecture
NetworkDCE, CORBA, JAVA RMI, DCOM...
MegaprogramClients in CHAIMS
Megamodules (wrapped, native) each supportingsetup, estimate, invoke, examine, extract, and terminate.
June 1998 CHAIMS 17
Decomposing CALL statements
Copying
Code sharing
Parameterized computation
Objects with overloaded method names
Remote procedure calls to distributed modules
Constrained (black box) access to encapsulated data
progressin
scale ofcomputing
ExtractInvokeEstimate ExamineSetup
CHAIMSdecomposes CALL functions
CALL gainedfunctionality
June 1998 CHAIMS 18
CHAIMS PrimitivesPre-invocation:
SETUP: set up the connection to a megamoduleSET-, GETATTRIBUTES: set global parameters in a megamoduleESTIMATE: get estimate of execution time for optimization
Invocation and result gathering:INVOKE: start a specific methodEXAMINE: test status of an invoked methodEXTRACT: extract results from an invoked method
Termination:TERMINATE: terminate a method invocation or a connection to
a megamodule
Control: Utility:WHILE, IF GETPARAM: get default parameters
June 1998 CHAIMS 19
Megaprogram Example: Overview
InputOutput- Input- Output
RouteInfo- AllRoutes- CityPairList- ...
AirGround- CostForGround- CostForAir- ...
Routing- BestRoute- ...
RouteOptimizer- Optimum- ...
General I/O-megamodule» Input function takes as parameter a default
data structure containing names, types and default values for expected input
Travel information:» Computing all possible routes between two
cities» Computing the air and ground cost for each
leg given a list of city-pairs and data about the goods to be transported
Two megamodules that offer equivalent functions for calculating optimal routes
» Optimum and BestRoute both calculate the optimum route given routes and costs
» Global variables: Optimization can be done for cost or for time
June 1998 CHAIMS 20
Megaprogram Example: Codeio_mmh = SETUP ("InputOutput")route_mmh = SETUP ("RouteInfo")...best2_mmh.SETATTRIBUTES (criterion = "cost")
cities_default = route_mmh.GETPARAM(Pair_of_Cities)input_cities_ih = io_mmh.INVOKE ("input”, cities_default)WHILE (input_cities_ih.EXAMINE() != DONE) {}cities = input_cities_ih.EXTRACT()...route_ih = route_mmh.INVOKE ("AllRoutes", Pair_of_Cities = cities)WHILE (route_ih.EXAMINE() != DONE) {}routes = route_ih.EXTRACT() …
IF (best1_mmh.ESTIMATE("Best_Route") < best2_mmh.ESTIMATE("Optimum") ) THEN {best_ih = best1_mmh.INVOKE ("Best_Route", Goods = info_goods, Pair_of_Cities = cities, List_of_Routes = routes, Cost_Ground = cost_list_ground, Cost_Air = cost_list_air)}ELSE {best_ih = best2_mmh.INVOKE ("Optimum", Goods = info_goods, …...best2_mmh.TERMINATE()
// Setup connections to megamodules.
// Set global variables valid for all invocations // of this client.
// Get information from the megaprogram user // about the goods to be transported and about// the two desired cities.
// Get all routes between the two cities.
//Get all city pairs in these routes.//Calculate the costs of all the routes.
// Figure out the optimal megamodule for// picking the best route.
//Pick the best route and display the result.
// Terminate all invocations
June 1998 CHAIMS 21
Operation of one Megamodule
• SETUP
• SETATTRIBUTES provides context
• ESTIMATE serves scheduling
• INVOKE initiates remote computation
• EXAMINE checks for completion
• EXTRACT obtains results
• TERMINATE I / ALL
M handle
M handle
M handle
M handle
I handle
I handle
I handle
M handle
I handle
June 1998 CHAIMS 22
CHAIMS Megaprogr. Language
Purely compositional:– no primitives for arithmetic ==> math megamodules– no primitives for input/output ==> general and problem-
specific I/O megamodules
Splitting up CALL-statement:– parallelism by asynchrony in sequential program– novel possibilities for optimizations– reduction of complexity of invoke statements
• higher-level language (assembler => HLLs, HLLs => composition/megamodule paradigm)
June 1998 CHAIMS 23
Architecture: Runtime
ed
a
b
c
Distribution System (CORBA, RMI…)
CSRT(compiled megaprogram)
MEGA modules
June 1998 CHAIMS 24
Architecture: Composition Process
ed
a
b
cMEGA modules
CHAIMS Repository
adds information to
MegamoduleProvider
wraps non-CHAIMScompliant megamodules
Wrapper Templates
June 1998 CHAIMS 25
writes
Architecture: Composition Process
Megaprogrammer
CSRT(compiled megaprogram)
Megaprogram(in CHAIMS language)
CHAIMS Compiler
generates
CHAIMS Repository
information
information
June 1998 CHAIMS 26
writes
Architecture: Overview
e
Megaprogrammer
d
a
b
c
Distribution System (CORBA, RMI…)
CSRT(compiled megaprogram)
Megaprogram(in CHAIMS language)
CHAIMS Compiler
generates
MEGA modules
CHAIMS Repository
adds information to
MegamoduleProvider
wraps non-CHAIMScompliant megamodules
information
information
Wrapper Templates
June 1998 CHAIMS 27
writes
Architecture: Overview
e
Megaprogrammer
d
a
b
c
Distribution System (CORBA, RMI…)
CSRT(compiled megaprogram)
Megaprogram(in CHAIMS language)
CHAIMS Compiler
generates
MEGA modules
CHAIMS Repository
adds information to
MegamoduleProviders
wraps non-CHAIMScompliant megamodules
information
information
Wrapper Templates
run-time
June 1998 CHAIMS 28
Architecture: CHAIMS-Language and CHAIMS-Protocols
Megaprogram
Megaprogrammer
M e g a m o d u l e s
CHAIMS-language
CHAIMS-protocolsCORBA-idl DCE-idl Java-class
CHAIMS API defines interface between megaprogrammer and megaprogram; the megaprogram is
written in the CHAIMS language.
The CHAIMS protocols define the calls the mega-modules have to understand. These protocols are slightly different for the different distribution protocols, and are defined by an idl for CORBA, another idl for DCE, and a Java class for RMI.
June 1998 CHAIMS 29
Name of Person
Architecture: GentypeA Gentype is a triple of name, type and value, where
value is either a simple type or a list of other gentypes (i.e. a complex type).
Possible simple types: given by ASN.1, the ASN.1-conversion library for C++, our own conversion routines.
Example: Person_Information complex
First Name string Joe Last Name string Smith
Personal Data complex Address
Date of Birth date 6/21/54 Soc.Sec.No string 345-34-345
June 1998 CHAIMS 30
Wrapper: CHAIMS Compliance
• CHAIMS protocol - knowing all CHAIMS primitives
• State management and asynchrony: » clientId (megamodule handle in CHAIMS language)» callId (invocation handle in CHAIMS language)» results must be stored for possible extraction(s) until
termination of the invocation
• Data transformation: » all parameters of type blob (BER-encoded Gentype) must
be converted into the megamodule specific data types (combination hand-coding/decoding routines
June 1998 CHAIMS 31
Architecture: Three Views
Transportation View moving around data blobs and CHAIMS messages
Composition View (megaprogram)
- composition of megamodules
- directing of opaque data blobs
Data View - exchange of data - interpretation of
data - in/between
megamodules
CHAIMS Layer
Distribution Layer
Objective: Clear separation between composition of services, computation of data, and transport
June 1998 CHAIMS 32
execution of a remote method
synchronous
invoke a methodie extract results
setup / set attributes s
s
e
i
time
decomposed(no benefit for one module)
asynchronous
s,i
time
etim
e
available for other methods
e
s,i
Scheduler: Decomposed Execution
June 1998 CHAIMS 33
Scheduler: Optimized Execution
M1 M4(<M1+M2)
M5
M2M3
(>M1+M2)
i1e1
e4e3
e2
i3i4
i5
i2
e5
time
M1
M4
M5
M2
M3
i1
e1
e2
e3
e4
e5
i2
i3
i4
i5
time
data dependenciesexecution of a method
non-optimized, orhandprogrammed
optimized by scheduleraccording to estimates
invoke a methodie extract results
set up / set attributes (not shown)s
June 1998 CHAIMS 34
Scheduling: Simple Example
1 cost_ground_ih = cost_mmh.INVOKE ("Cost_for_Ground", 1 List_of_City_Pairs = city_pairs,Goods = info_goods)
2 WHILE (cost_ground_ih.EXAMINE() != DONE) {} 3cost_list_ground = cost_ground_ih.EXTRACT()
3 cost_air_ih = cost_mmh.INVOKE ("Cost_for_Air", 2 List_of_City_Pairs = city_pairs,Goods = info_good)
4 WHILE (cost_air_ih.EXAMINE() != DONE) {} 4cost_list_air = cost_air_ih.EXTRACT()
order inunscheduledmegaprogram
order in automaticallyscheduled megaprogram
June 1998 CHAIMS 35
Scheduling: Possible ActionsINVOKES: call INVOKE’s as soon as possible
» may depend on other data» moving it outside of an if-block: depending on cost-
function (ESTIMATE of this and following functions concerning execution time, dataflow and fees (resources).
EXTRACT: move EXTRACT’s to where the result is actually needed
» no sense of checking/waiting for results before they are needed
» instead of waiting, polling all invocations and issue next possible invocation as soon as data could be extracted
TERMINATE: terminate invocations that are no longer needed (save resources)
» not every method invocation has an extract (e.g. print-like functions)
June 1998 CHAIMS 36
Compiling into a NetworkMega Program
Module A
Module B
Module CModule E
Module DModule F
current CHAIMS systemMega Program
Module DModule F
control flow data flow
with automatic dataflow optimization
Mega Program
Module A
Module B
Module C Module E
Module DModule F
June 1998 CHAIMS 37
CHAIMS Implementation• Specify minimal language
– minimal functions: CALLs, While, If *– minimal typing {boolean, integer, string, handles, object}
» objects encapsulated using ASN.1 standard– type conversion in wrappers, service modules*
• Compiler for multiple protocols (one-at-time, mixed*)• Wrapper generation for multiple protocols
• Native modules for I/O, simple mathematics*, other• Implement API for CORBA, Java RMI, DCE usage
• Wrap / construct several programs for simple demos
• Schedule optimization *• Demonstrate use in heterogeneous setting *• Define full-scale demonstration * in process
June 1998 CHAIMS 38
Concept Status• Definition of architecture for Megaprogramming
– bottom up assessment of code to be generated• examples: room reservation, shipping
– primitives– handles for parallel operation– heterogeneity -- common features of distribution protocols
• Minimal language that can generate the code– no versus very few types -- ASN.1 for complex types– natural parallelism -- still a major research issue
• Awareness of novel optimizations– information flow constraints -- scheduling– direct data flow between megamodules
June 1998 CHAIMS 39
Focus for Future
• Finishing basic infrastructure and demo examples.
• CHAIMS interpreter instead of CHAIMS compiler.
• Scheduling of invocations and extractions.
• Flexible interaction with megamodules; extracting and handling overview results.
• Direct dataflows between megamodules (future project).
June 1998 CHAIMS 40
CHAIMS
Conclusion: Research Questions• Is a Megaprogramming language focusing only on
composition feasible?• Can it exploit on-going progress in client-server
models and be protocol independent? • Can natural parallelism for distributed services be
effectively scheduled?• Can high-level dataflow among distributed modules
be optimized?• Can CHAIMS express clearly a high-level distributed
SW architecture?• Can the approach affect SW process concepts and
practice?
June 1998 CHAIMS 41
Paying for SW ServicesYou can not run an effective (SW) business and not be
reimbursed for it. How? Four approaches:– Sell Softwaresell oilfield to customer– Lease copy / usage rights lease well– Time / user limited access fill tank– Charge by use instance provide bus
General problems, effects differ– IP protection? – keeping SW updated – billing for est.value – performance effect
poor some fair good
poor ok good good
simple simple awkw. hard
no no little some
Buy Lease Limit Use
protect
update
bill
perform
June 1998 CHAIMS 42
Conclusion: Questions not addressed
• Will one Client/Server protocol subsume all others?– distributed optimization remains an issue
• Synchronization / Concurrency Control– autonomy of sources negates current concepts– if modules share databases, then database locks may
span setup/terminate all for a megaprogram handle.
• Will software vendors consider moving to a service paradigm?
– need CHAIMS demonstration for evaluation
June 1998 CHAIMS 43
Composition of Processes...• versus composition and integration of Data
» data-warehouses» wrapping data available on web
• versus composition of Components» reusing small components via copy/paste or shared
libraries locally installed» large distributed components within same “domain” as
composition, e.g. within one bank or airline
CHAIMS: » processed information
» composing autonomous execution threads
June 1998 CHAIMS 44
Summary• CHAIMS requires rethinking of many common
assumptions– gain understanding via simple examples
• Work focused on CALL statement decomposition– to accomplish integration of large services – exploit inherent asynchrony
• First version of architecture and language drafts are completed; basic infrastructure partially available (compiler, wrapper templates).
• More demos will come soon. Half-way through a four year project.
http://www-db.stanford.edu/CHAIMS
June 1998 CHAIMS 45
CHAIMS
June 1998 CHAIMS 46
Other systems (1)
Darwin/Regis:» configuration language, architecture definition language,
allowing various communication mechanisms (yet all in C++ and under local control)
» local (distributed) architecture, non-autonomous, system not only specifies, generates and controls composition, but also basic components
» only for homogeneous environment (C++), no legacy
Polylith/Polygen:» changes source-files (C, Ada, Pascal, Lisp) according to
annotated design data (architecture information) for distribution
» only for local distribution (access to source code) and only for TCP/IP
June 1998 CHAIMS 47
Other systems (2)Hadas:
» network centric framework with peer-to-peer communication» Java-based (RMI) with Java-wrappers for non-compliant
objects» concept of ambassador objects which are copied to the
peer object» allows administrative and design autonomy for objects
(heterogeneity, changes of methods)
KQML:» protocol for asynchronous communication among agents» allows explicitly different ontologies» offers primitives for getting and updating data, but also for
advertising available knowledge
June 1998 CHAIMS 48
Other systems (3)Differences:
» autonomous megamodules, legacy modules» higher-level (just invoke and extract, no specification of
various kinds of communication; no specification of module content)
» heavy-weight services: ESTIMATE, parallelism by automatic scheduling of invocations according to run-time estimates and cost-functions, EXAMINE, SETPARAM
» heterogeneity in distribution protocols
focus so far: heterogeneity in distribution protocol on client side
focus in future: using ESTIMATE and EXAMINE, automatic optimization and invocationscheduling
June 1998 CHAIMS 49
CHAIMS proves that...• We can do composition in a high-level language.
» same language for Java-RMI-invocations and CORBA-invocations (and DCE, DCOM, TCP/IP protocols)
» (single megaprogram can deal with multiple protocols simultaniously)
» multiple megamodules can run in parallel
• Large-scale composition can be automated.» in contrast to manual non-software composition (e.g. telephone,
cut&paste) » in contrast to fixed programs for one specific problem (e.g.
transporting military goods within US)
• We can do schedulings of programs in a way right now only smart logistics officers can do, avoiding unnecessary waits.
» Scheduling of invocations can be optimized.
June 1998 CHAIMS 50
Long-term Objectives of CHAIMS1 Implementing a system for a simple and purely
compositional language hiding differences of diverse protocols
2 Automatic optimized scheduling of invocations (taking advantage of inherent parallelism and estimate-capabilities of megamodules, hence splitting up of CALL-statement)
3 Decision-making support (direct) interaction with megamodules, based on overview and incremental results (fixed flow, not yet interactive changes to megaprogram)
4 Automatic dataflow optimization (direct dataflows between megamodules), not yet
June 1998 CHAIMS 51
Assumptions, Additional Constraints• Heterogenous legacy modules ==> wrapping of modules, mixing
protocols on client side or in wrappers.• Parallelism of megamodule-methods not through multithreading on
client side but through splitting up CALL-statement (==> sequential program on client side); this leads to useful parallelism because we deal with coarse-grain parallelism.
• CHAIMS-compliancy for megamodules is achieved by wrapper-templates, for new native megamodules as well as for legacy ones (CHAIMS-compliancy is more than just knowing CHAIMS-protocol!).
• No reliance on existence of one specific higher level protocol like CORBA, DCOM, RMI ==> implementing an independent data-encoding and marshalling with ASN.1, instead of using one of them and then having converters in the wrappers.
• Interfaces of megamodules match <==> no investigation into opaque datablobs on client side necessary.
• Thin client, client should be able to run anywhere (not quite fulfilled right now - we need local ORB, DCE, JavaVirtual-machine).
• Clear seperation client - server, minimal repository.
June 1998 CHAIMS 52
Non- (not yet)-Objectives of CHAIMS• No commercial product.• No specific controls over ilities (security, name-
serving, etc.) that they are normally present in distributed systems.
• No sophisticated front-end, no graphical programming/composition, no browser for repository, no higher-level language as input (not yet).
• Not solving all problems of megamodule composition that are mentioned in the various CHAIMS-papers (e.g. differing ontologies, non-matching interfaces of megamodules), only the ones mentioned in objectives and additional conditions.
June 1998 CHAIMS 53
Short-term Objectives of CHAIMSRest of 1998:
» Basic infrastructure (fixing most severe flaws, moving to consistent architecture, all primitives, types, associative lists with handling it, having CORBA) ==> conceptual and implementation work -- CONSOLIDATION
» More examples (descriptions of scenarios as well as implemented demos), wrapping one (maybe two) additional suites of megamodules. ==> implementation work -- CONSOLIDATION
» Mixing of protocols in client (CORBA, RMI) or/and TCP/IP-three-tier architecture
» Preparing for more capable scheduler (examples with current scheduler, reading about other scheduler-problems and implementations, redesigning architecture of compiler (interpreter?), designing scheduler algorithm and architecture, writing paper about all this…) ==> lots of conceptual work, some implementation -- looking ahead for better Scheduler
1999 (depending on where we are at the end of 1998):» Scheduler
June 1998 CHAIMS 54
Upcoming Changes to Architecture:Other Approach to Heterogeneity
Client (megaprogram)
nativeserver
1
nativeserver
3
nativeserver
2
chaimscompliantmodule
chaimsI/O
module
RMIwrapper
TCP/IP socketsCHAIMS protocol
CORBA RMIserver-specific protocols
site
s o
f se
rver
s
clientsite
differentwrapper site
RMIwrapperCORBAwrapper
June 1998 CHAIMS 55
Reasons Alternative ArchitectureOverall:• Simpler architecture: less wrappers, just one protocol on client side
Server-side:• No direct linking with legacy code also for CORBA-wrappers, different sites for wrapper
and legacy megamodule possible• All native CHAIMS-megamodules will be built using wrapper templates ==> no reason for
several protocols, they can all use TCP/IP.• Dataflow-optimization: direct messages between megamodules/their wrappers necessary
(without bridges)
Client-side:• Thin client that could run everywhere (TCP/IP is available everywhere, but not CORBA or
DCE, RMI also is easily available everywhere).• CSRT could be implemented by interpreter instead of compiler, maybe also possible with
current architecture, but more complex.• We use just transport-facility (really true? what about native CHAIMS-types like string,
integer, boolean?) of CORBA, RMI, DCE (for data we have ASN.1); this is already offered by TCP/IP ==> no unnecessary overkill
Drawback: missing one of the current funding objectives (heterogeinity on client side).
June 1998 CHAIMS 56
Upcoming Changes to Architecture:Interpreter instead of Compiler
megaprogramin CHAIMS language
client code in C, C++, Java,
stub code
executableclient (CSRT)
userCompiler:
CHAIMS executionmachine
(interpreter andscheduler)
userInterpreter:
networkCHAIMS-protocol
completemegaprogram
in CHAIMS language
some CHAIMSstatements
user
serves as input to
serve as input to
CHAIMS compiler,simple scheduler
Idl-file generatorand compiler
C++, Java compiler
and linkernetwork
June 1998 CHAIMS 57
Interpreter• Dynamic scheduler:
» Input is parsed and stored in a dependancy graph.» Execution machine (interpreter / scheduler) works through
the graph and makes appropriate calls:– estimate-calls are added in order to get necessary run-time information
for scheduling (cost-function)– every invocation is issued as soon as possible (data-flow) and
reasonable (according to cost-function)– all invocations for which the CSRT waits for results are polled regularly,
and results extracted and new invocations issued as soon as possible
CSRT would still be sequential!
• Overview results, flexible interactions:» megaprogrammer can program statement by statement and get results
immediately; results will influence what he/she does next» like ftp, web
June 1998 CHAIMS 58
Example: parameters of method xz
general default valuesclient-specific settings for client A
method xz, client A:invocation-specific settings
client-specific settings for client B
from deststart-date
end-date number
1JAN1998 1JAN1998
6OCT19984OCT1998
7OCT19984OCT1998
LAS BWI 1
2
actual values used in invocation xz1
SJO ZRH
SFO
1
1
SFO
SJO6OCT19984OCT1998 ZRH
ZRHactual values used in invocation xz2
FRA
7OCT1998
9DEC19981DEC1998
9DEC19981DEC1998 2LAS FRAactual values used in invocation xz1
method xz, client A:invocation-specific settings
method xz, client B:invocation-specific settings
Hierarchical Setting of Parameters
June 1998 CHAIMS 59
Using the General I/O-Megamodulefor Input
Corresponding code in CLAM:ex = SETUP("Example")io = SETUP("io") // io is the general I/O megamodule availabe in CHAIMS(myadress = adress) = ex.GETPARAM() io.INVOKE("ask", value = adress,text =
"Please enter adress:")ih1 = io.INVOKE("ask", value = myadress, text = "Please enter adress:") (myadress = result) = ih1.EXTRACTWHENDONE() ih2 = ex.INVOKE("method", adress = myadress)…..
Megaprogram General I/0Megamodule
MegamoduleExample
2. Invoke method "ask" with default values and type information
3. Extract user-input
4. Invoke desired method with user-input
1. Get default type and value for desired parameter
June 1998 CHAIMS 60
Using the General I/O-Megamodulefor Output
Corresponding code in CLAM:…………... // Setups, Invocations, ih1 is the handle of a
// method of megamodule examples(myadress = adress_result) = ih1.EXTRACTWHENDONE() ih2 = io.INVOKE("write", value = myadress, text ="Currently the adress is:")
Every element of the data structure in parameter adress_result has besides the actual value also type information and descriptive name. Therefore, the general I/O-megamodule can display the parameter in a decent format without any further formatting information.
Megaprogram General I/0Megamodule
MegamoduleExample
2. Invoke method "write" with result to be displayed
1. Extract result to be displayed
June 1998 CHAIMS 61
Primitives in PAM
Pre-invocation:SETUP: set up the connection to a megamoduleSET-, GETPARAM: preset parametersESTIMATE: get cost estimates for a specific method
Invocation and result gathering:INVOKE: start a specific methodEXAMINE: test status of an invoked methodEXTRACT: extract results from an invoked method
Termination:TERMINATE: terminate a method invocationTERMINATEALL: terminate the connection to a megamodule
June 1998 CHAIMS 62
Name of Person
GentypeA Gentype is a triple of descriptive name, type and
value, where value is either a simple type or a list of other gentypes (i.e. a complex type).
Possible simple types: given by ASN.1-library and our own conversion routines (integer, real, string, boolean, date-time, sequence of bytes)
Example: Person_Information complex
First Name string Joe Last Name string Smith
Personal Data complex Address
Date of Birth date 13JUN1934 Soc.Sec.No string 345-34-345
June 1998 CHAIMS 63
INVOKE in various protocolsRMI: public long INVOKE(long clientID, String methodName,
ChaimsAttrValContainer cont) throws RemoteException;
CORBA: void INVOKE ( in CHAIMSCLIENTID clientid,in CHAIMSMETHODNAME methodname, in CHAIMSATTRVALSEQ attrvalseq, out CHAIMSCALLID callid );
DCE: ...local C++: long Invoke (char* methodName, AttrValList * plist) local Java: public long INVOKE (String methodName,
ChaimsAttrValContainer cont);
• AttrValList, CHAIMSATTRVALSEQ and ChaimsAttrValContainer are sequences of name-value pairs, adapted to the various languages and systems.
• INVOKE for local protocols does not need a client ID because for each client there is one instance of the local megamodule.
June 1998 CHAIMS 64
Setting up connection in various protocols (client code RMI, Java)
RMI:System.setSecurityManager(new RMISecurityManager()); ChaimsCompliantObject remote;try{remote=(ChaimsCompliantObject)Naming.lookup(url+mmname);}catch (Exception e){}try {remote.SETUP(clientID);}catch (Exception e){}
local Java: Class cl; //Class to be loaded, for which an object will be instantiatedtry {cl = Class.forName(full_class_name);}catch (Exception e) {}ChaimsCompliantLocal local;try {local = cl.newInstance();}catch (Exception e) {}local.SETUP();
June 1998 CHAIMS 65
Setting up connection in various protocols (client code CORBA)
CORBA (Orbix, C++ client):nameMM_var _pObj0; try {_pObj0=nameMM::_bind(":mmname","site");} catch (...) {} CHAIMSMMID mmid = _pObj0; pObj0->SETUP(mmid); CORBA (Omnibroker, C++ client):CORBA_ORB_var orb = CORBA_ORB_init(argc,argv);// first get reference from a file and then convert it to an objectconst char* refFile_pObj0 = "io.ref"; ifstream in_pObj0;in_pObj0.open(refFile_pObj0); // error handling omitted herechar stre_pObj0[1000]; in_pObj0 >> stre_pObj0;CORBA_Object_var obj_pObj0 = orb -> string_to_object(stre_pObj0);assert(!CORBA_is_nil(obj_pObj0)); // bind now to megamoduleio_var _pObj0 = io::_narrow(obj_pObj0); assert(!CORBA_is_nil(_pObj0));CHAIMSMMID mmid = _pObj0;pObj0->SETUP(mmid);
June 1998 CHAIMS 66
Alternative Architecture:Other Approach to Heterogeneity
Client
nativeserver
1
nativeserver
3
nativeserver
2
chaimscompliantmodule
chaimsI/O
module
RMIwrapper
e.g. TCP/IP socketsPAM
CORBA RMIserver-specific protocols
site
s o
f se
rver
s
clientsite
differentwrapper site
RMIwrapperCORBAwrapper
June 1998 CHAIMS 67
Current Architecture: Several Implementations of PAM
Client
P A M - protocolsCORBA-idl DCE-idl Java-class
M e g a m o d u l e s
W r a p p e r s
June 1998 CHAIMS 68
CHAIMSDomain expert (composer)
writes megaprogram for composition.
CHAIMS automates generation of client for
distributed system and optimized scheduling.
Megamodule provider provides processed
knowledge.
CHAIMS
Megamodules
June 1998 CHAIMS 69
Client program
Domain expert / Composer
DistributedComputation
Services
IO module
Infrastructure: Remote, distributed, parallel computation of a, b, c, d, e
IO module
Wrapper
Legacy Module
ab c
de
Native CHAIMS Module
Control FlowData Flow
MEGA modules
June 1998 CHAIMS 70
e
Composer (Domain Expert)
d
a
b
c
Distribution System (CORBA, RMI…)
CSRT, incl. run-time scheduling
Megaprogram (written in CLAM)
writes
CHAIMS Compiler
generates
MEGA modules
CHAIMS Repository
adds information to
MegamoduleProvider
Wrapper Templates
wraps non-CHAIMScompliant megamodules
information
information
run-time execution
composition process
CPAM protocol
June 1998 CHAIMS 71
Exploiting parallelism of megamodule pieces
M1
M4(<M1+M2)
e
M3 (>M1+M2)
a
d(<a+b)
e
b
c (>a+b)
invoke a method
set up / setattributes
time
prior time
extract results
before piecewisedoptimization
June 1998 CHAIMS 72
Primitives of CLAM• Pre-invocation:• SETUP: set up the connection to a megamodule• SET-, GETPARAM: preset parameters in a megamodule• ESTIMATE: get cost estimation for optimization
• Invocation and result gathering:• INVOKE: start a specific method• EXAMINE: test status and progress of an invoked method• EXTRACT: extract results from an invoked method
• Termination:• TERMINATE: terminate a method invocation• TERMINATEALL: terminate a connection to a megamodule
• Control:• WHILE, IF
June 1998 CHAIMS 73
Exploit parallelisma d
(<a+b)
e
bc
(>a+b)
i1e1
e4e3
e2
i3i4
i5
i2
e5
time
a
d
e
b
c
i1
e1
e2
e3
e4
e5
i2
i3
i4
i5
time
data dependenciesexecution of a method
non-optimized, orhandprogrammed
exploring parallelismaccording to estimates and based on dataflow dependencies
invoke a methodie extract results
June 1998 CHAIMS 74
Dataflow optimizationClient program
DistributedComputation
Services
IO moduleIO module
a
b c
de
Control FlowData Flow
June 1998 CHAIMS 75
ChaimsCallRecord
callID clientID paramContmethodID resCont ObjectWrapperThreadstatus MethodWrapper
~ ~~ ~ ~ ~ ~~~
key
~ ~~ ~ ~ ~ ~~~n-1 ~~ ~ ~ ~ ~~~n
INVOKE( clientID, methodName,attrValContainer)
ChaimsThread
LocalObjectWrapper
ServerObject
MethodXWrapper
createNewCallEntry1
LocalMethodWrapper
start2
callLocalMethod3 callMethod4
methodX5 -> result6
setPartialResult7
setResultContainer8
setResultDone9
setStatus( DONE )10
June 1998 CHAIMS 76
SETUP( clientID )
addClientEntry
ChaimsClientRecord
clientID paramCont (default values)
~
key
~“PA58”
DEFAULT
(A, ) (B, ) (Z, ) (B, ) (C, )
(A, ) (B, ) (C, ) (Z, )
written by SETPARAM, read by GETPARAM
June 1998 CHAIMS 77
What’s the problem?
C++ Client
Java Client
Modules written inC++ or Java
Modules written inJava
CORBA
RMI
June 1998 CHAIMS 78
What’s the problem?
Client
RMI
CORBA
CORBA
Modules
June 1998 CHAIMS 79
Possible Solution: JNI
C++ MasterClient
JavaCode
CORBA
CORBA
RMI
JNI
June 1998 CHAIMS 80
Possible Solution: JNI
JavaMasterClient
C++Code
CORBA
CORBA
RMI
JNI
C++Code
JNI
June 1998 CHAIMS 81
CHAIMS Repository• CHAIMS repository data presently an ASCII file.• Each line conveys information about a specific
megamodule, method, parameter, or “synonym” pair.• C - style commenting ( /* … */ ).• Basically hierarchical model with certain exceptions.
June 1998 CHAIMS 82
CHAIMS Repository Model
June 1998 CHAIMS 83
CHAIMS repository browser: Window Layout #1
June 1998 CHAIMS 84
CHAIMS repository browser: Window Layout #2
top related