1 cca meeting, januray 25th 2007 supporting the master-worker paradigm in the common component...

11CCA Meeting, Januray 25th 2007

Supporting the Master-Worker Paradigm Supporting the Master-Worker Paradigm in the Common Component Architecturein the Common Component Architecture

Hinde Lilia Bouziane,Hinde Lilia Bouziane, Christian PérezChristian Pérez, Thierry Priol, Thierry Priol

PARIS research-teamPARIS research-teamIRISAIRISA / / INRIAINRIA

Rennes – FranceRennes – France

contact: [email protected]: [email protected]


Outline of the talkOutline of the talk

ContextContext– Master-worker applications and environmentsMaster-worker applications and environments– Software component modelsSoftware component models

Limits with existing component modelsLimits with existing component models Our proposition: a generic modelOur proposition: a generic model

– ObjectivesObjectives– Overview of the proposed modelOverview of the proposed model

Master and workers abstract compositionMaster and workers abstract composition Request scheduling integrationRequest scheduling integration

Cases of study: CCACases of study: CCA ConclusionConclusion


IntroductionIntroduction ContextContext

– Parametric applicationsParametric applications– Several kinds of resources Several kinds of resources

multi-core processorsmulti-core processors SMP machinesSMP machines clustersclusters gridsgrids

– Dedicated environmentsDedicated environments BOINC, DIET, NetSolve, BOINC, DIET, NetSolve,

Nimrod/G, Ninf, ...Nimrod/G, Ninf, ...

Challenges Challenges – Simplifying application programmingSimplifying application programming– Independence from resource kindsIndependence from resource kinds– HPCHPC

u


Master-worker environmentsMaster-worker environments

Many environments:Many environments:– BOINC,BOINC, XTremWEB, XTremWEB, DIET, DIET, NetSolve, Nimrod/G, ...NetSolve, Nimrod/G, ...

Characteristics:Characteristics:– ScalableScalable– Request delivery Request delivery

policypolicy– Transparent request Transparent request

delivery management delivery management for the programmerfor the programmer

– Dedicated APIDedicated API– Only master-worker Only master-worker

paradigm is supportedparadigm is supported Reactive transport in Hydrogeology

u

PrecipitationSorption

Relargage

ConvectionDispersionAqueous Reactions

Gaz-liquid exchange Dissolution

Biology

worker

worker

worker

Masterworker

Workers collectionRequests transport

SchedulingFault tolerance


Software component modelsSoftware component models

Black boxBlack box PortsPorts

– Method invocations, events/messages/streamsMethod invocations, events/messages/streams Several component modelsSeveral component models

– Common Component Architecture/CCA Forum (CCA)Common Component Architecture/CCA Forum (CCA)– CORBA Component Model/OMG (CCM)CORBA Component Model/OMG (CCM)– Fractal/ObjectWeb Fractal/ObjectWeb – Grid.it-ASSIST/UNIPIGrid.it-ASSIST/UNIPI– Etc. Etc.

Softwarecomponent

PROVIDEDPORTS

REQUIRED PORTS

(client interfaces)

Softwarecomponent

(server interfaces)


A simple master-worker application A simple master-worker application exampleexample

master

interface Compute { long f (in long value);} ;

Master:ForAll i= 0 to N {

call f (..) on a worker through yellow port;}

worker

Worker: long f (long value) {

// computationreturn res;

}


Limits with existing component Limits with existing component models (1/2)models (1/2)

m w

WWw10

m

Resources infrastructure Resources infrastructure dependencedependence– SMP machine vs ClusterSMP machine vs Cluster

Request delivery Request delivery implementation at the implementation at the burden of the masterburden of the master– No transparencyNo transparency– ComplexComplex


Limits with existing component Limits with existing component models (2/2)models (2/2)

WWw35m

interceptor

WWw111

m proxy

Request delivery policy Request delivery policy outside but fixedoutside but fixed

– Resources infrastructure Resources infrastructure dependence!dependence!

– For grids, complex request For grids, complex request delivery implementationsdelivery implementations

– No transparencyNo transparency


A generic model to support A generic model to support master-worker paradigmmaster-worker paradigm


Features of the modelFeatures of the model

Resources infrastructure independenceResources infrastructure independence TransparencyTransparency

– No dealing with the number of workersNo dealing with the number of workers– No dealing with request delivery concernsNo dealing with request delivery concerns

An initial number of workers depending on An initial number of workers depending on the current resources infrastructurethe current resources infrastructure

Introduction of a request delivery policy Introduction of a request delivery policy depending on the current resources depending on the current resources infrastructureinfrastructure


Abstract master and worker Abstract master and worker composition (1/2): a collectioncomposition (1/2): a collection

Collection Collection definitiondefinitionExposed provided port

worker W1 Wn

W2 Wi

Type1 Type3Type2

Type1 T1x Type2 T2y T2y

Type3Type3 T3z

Collection at execution

Instantiation

Instantiation


Abstract master and worker Abstract master and worker composition (2/2): abstract assemblycomposition (2/2): abstract assembly

Composing components Composing components with collections with collections

≈≈

Component compositionComponent composition

m

Type1 Type3Type2

X


Integration of a request delivery Integration of a request delivery policy (1/2): transformationpolicy (1/2): transformation

Abstract ADL ->concrete ADLAbstract ADL ->concrete ADL

Round-Robin

Deploymentenvironment

Transformationprocess

Round-Robin


Integration of a request delivery Integration of a request delivery policy (2/2): patternspolicy (2/2): patterns

w w w

M

Round-Robin / Random

LA

LALA

M

w w w

MA

MAMA

Simple component based pattern

Hierarchical schedulingpattern

DIETpattern

M

Random

w w

Round-Robin Round-Robin

w w


Overview of the proposalOverview of the proposalProgrammer/designer

view

Collection

binding

master

worker

Exposed provided port

Resourcesinfrastructure

Round-Robin

Framework implementation view

#workers+

Pattern selection

Set of request transport mechanism patterns

1. Random2. Round-Robin3. NetSolve4. Diet5. …


Extending the Extending the Common Component Common Component

ArchitectureArchitecture


Overview of the CCAOverview of the CCA All is done at runtimeAll is done at runtime Examples of specified SIDL Examples of specified SIDL

interfacesinterfaces – BuilderServiceBuilderService

Component creationComponent creation– createInstance(…createInstance(…))

CompositionComposition– connect(…connect(…))

– ServicesServices Port declarationsPort declarations

– addProvidesPort(…)addProvidesPort(…)– registerUsesPort(…registerUsesPort(…))

Getting a port referenceGetting a port reference– getPort(…)getPort(…)

fw

A

createInstance(A)

setServicesregisterUsesPort(aP) aP

connect(aP, bP)

B

createInstance(B)

setServicesaddProvidesPort(bP)

bP


Mapping principleMapping principle

Create new interfaces forCreate new interfaces for– collection managementcollection management

Creation, adding element, binding portsCreation, adding element, binding portsLet the collection be a concrete componentLet the collection be a concrete component

– pattern managementpattern managementDeclare, instantiate itDeclare, instantiate it


Master-worker paradigm in CCAMaster-worker paradigm in CCAuser viewuser view

Collection and binding specification– Services : 1 op– BuilderService : 3 ops– others: 2 interfaces (5 ops)

Pattern specification (optional) – 1 interface for pattern

representation (1 op implemented by scheduling expert )

– BuilderService : 1 op explicit choice of a pattern

Dynamic collection management (optional)– 1 interface for adding/removing

elements (2 ops)– Services : 1 op– BuilderService : 2 ops

interface BuilderService2 { ... CollectionID createInstanceCollection( in string instanceCollName, in string className, in TypeMap properties); BindingID bind(in CollectionID collID, in string collPortName, in ComponentID elem, in string elemPortName); array<BindingID> getBindingIDs(...);}

interface CollectionManagement : Port { bool addElement(in CollectionID collID); bool subElement(in CollectionID collID);}interface Services3 { void setCollManagementPort(in string portName, in collectionManagement port);}interface BuilderService3 { ... bool addElementToColl(in CollectionID collID); bool subElementToColl(in CollectionID collID);}


Master-worker paradigm in CCAMaster-worker paradigm in CCAUser View

interface BuilderService2 : BuilderService { CollectionID createInstanceCollection

(in string instanceCollName, in string className, in TypeMap properties); BindingID bind

(in CollectionID collID, in string collPortName, in ComponentID elem, in string elemPortName); …}Interface PatternInstantiation {…}Interface CollectionManagement {…}

master

createInstance(master)

fw

setServicesregisterUsesPort(mP)

createInstanceCollection(coll)

worker

collsetServices

addProvidesPort(cP)loop : createInstance(wrk) bind(cP, wP)

mP

cP

wP

connect(mP, cP)

1

2

3


Master-worker paradigm in CCA (2/2)Master-worker paradigm in CCA (2/2)

Request delivery Pattern specification (optional) – 1 interface for pattern representation

(1 op implemented by scheduling expert ) Dynamic collection management (optional)

– 1 interface for adding/removing elements (2 ops)– Services : 1 op– BuilderService : 2 ops

addElement(collID)subElement(collID)

collManager

instantiatePattern(collID)

fw

pattern


Impacts on the CCA framework Impacts on the CCA framework implementationimplementation

Re-implementation of the BuilderService interface– connect operation

Identifying a provides collection port to instantiate an associated pattern

– getPort operation

Implement “new” interfaces Framework should provide a default pattern


ConclusionConclusion Improving the support of master-worker paradigm in CCAImproving the support of master-worker paradigm in CCA

– Collections, request delivery policy patternsCollections, request delivery policy patterns– Resources infrastructure independent programmingResources infrastructure independent programming– Adaptation of workers and requests delivery at the Adaptation of workers and requests delivery at the

burden of the frameworkburden of the framework Impact on CCAImpact on CCA

– Additional interfaces and limited runtime modificationAdditional interfaces and limited runtime modification Ongoing feasibility study with MOCCAOngoing feasibility study with MOCCA DiscussionsDiscussions

– How CCA component placement should be handle?How CCA component placement should be handle?

– Multiple/Collection uses port will simplify lifeMultiple/Collection uses port will simplify life

– How to generate the specific code for the “proxies”?How to generate the specific code for the “proxies”?

1 cca meeting, januray 25th 2007 supporting the master-worker paradigm in the common component...

Documents