navarro, marquès & freitag 1 april 2004 on distributed systems and cscl joan manuel marquès -...
Post on 15-Jan-2016
213 views
TRANSCRIPT
Navarro, Marquès & Freitag 1
April 2004
On Distributed Systems and CSCL
Joan Manuel Marquès - [email protected] Oberta de Catalunya
Leandro NavarroFelix Freitag
Universitat Politècnica de Catalunya
Navarro, Marquès & Freitag 2
April 2004
Table of contents
● Introduction● Requirements● Our proposal: LaCOLLA● Architecture of LaCOLLA● Virtual synchronism● Execution of tasks● Prototype and validation● Conclusions
Navarro, Marquès & Freitag 3
April 2004
Introduction
1.Asynchronous collaborative applications have to deal with: – Interaction nature
● Participants dispersed at Internet; many-to-many collaboration; people participate at different times/places (home, work, notebook); ...
– Idiosyncrasy of groups● Flexibility, dynamism, decentralization, autonomous participation, groups
exist while members participate and provide resources, few participants, ...– Technical and administrative issues:
● Guarantee information availability; interoperability among applications; security (authorization, access rights, firewalls), ... participants belong to different organizations or departments with different authorities that impose rules and limitations in order to facilitate administration, internal work and individual use.
Navarro, Marquès & Freitag 4
April 2004
Introduction
● Development of applications that take into account those requirements are costly (in time, resources and economically)
● [Solution]: Simplifications● Centralization, reduction autonomy, dependency on external resources, ...
● [Consequence]:– Development of collaborative applications is a hard work.– Applications include minimal collaborative functionalities– Existing collaborative applications focus on a few of those aspects
(the key aspects for the application) and neglect other aspects.
Navarro, Marquès & Freitag 5
April 2004
Introduction
2.To have groups that are computationally autonomous, flexible and independents ...
... ability to execute tasks and deploy services using resources belonging to group is required:
– Execute task: use computational resources belonging to group to execute tasks.
● Task: application that has an start and an end.– Deploy services: deploy services in resources belonging to group.
Group guarantees that services are available.● Service: application that executes “for ever”.
Navarro, Marquès & Freitag 6
April 2004
Requirements
● Oriented to groups● Internet scale (distributed, dispersed)● Universal and transparent access● Decentralization● Self-organization of the system● Individual autonomy● Self-sufficiency● Allow sharing● Resources availability (replication)● Security
Navarro, Marquès & Freitag 7
April 2004
Our proposal: LaCOLLA
● a fully decentralized infrastructure for building collaborative applications that:– provides general purpose collaborative functionalities.
– Concept of group– Know what is going on in the group (= disseminate information about actions that
take place inside the group to all group members). Awareness.– Capacity to share information generated in the group (= access to objects
belonging to group at any moment; independently whether the member was connected or disconnected when the object was created).
● Will avoid applications deal with complexities derived from groups/members● This simplification will help inclusion collaborative aspects into applications.● To meet flexibility required by groups, LaCOLLA is decentralized,
autonomous, self-organized, uses resources provided by group members...– provides groups computational autonomy and independence:
● ability to execute tasks and deploy services using resources belonging to group
Navarro, Marquès & Freitag 8
April 2004
Functionalities
● “Immediate” and consistent dissemination of events● Consistency in the storage of objects (virtually strong
consistency)● Execute tasks (and deploy services)● Presence (of members and components)● Location transparency (of members and components)● Instant messaging● Management of groups and members
● Not implemented– Security– Disconnected mode
Navarro, Marquès & Freitag 9
April 2004
Architecture of LaCOLLA
● Five kind of components: UA, RA, GAPA, TDA and EA.● Internal mechanisms to coordinate the components.
● Following peer-to-peer paradigm– (decentralized, every peer can be server and client, self-sufficiency,
self-organizing, ad hoc connectivity, scalability, ...)● Use pools of resources (computational, storage) belonging (or
available) to members of the group.– Resources are: geographically distributed, owned by different
members (different authorities), heterogeneous, ...
Navarro, Marquès & Freitag 10
April 2004
Components
Five kind of components: UA (User Agent):
● Represents the user;● Interacts with applications
RA (Repository Agent):● Persistent storage (objects and events)
GAPA (Group Administration and Presence Agent):
● Administration: groups and members● Users' preferences● Authentication of members
TDA (Task Dispatcher Agent):● Distributes tasks to executors
EA (Executor Agent):● Executes tasks
Members in the peer decide to instantiate any number of components.
UA RA GAPA
Transport
...
Applications
Peer LaCOLLA EA TDA
Navarro, Marquès & Freitag 11
April 2004
Internet
Application A
Peer LaCOLLA
Transport
UA
UA
Transport
UA
RAUA GAPA
Peer LaCOLLA
Transport
UA
UA
Peer LaCOLLA
Transport
UA
RAUA GAPA
Peer LaCOLLA
TDA EATransport
UA
RAUAEA
Peer LaCOLLA
GAPA
Transport
UA
UA
Peer LaCOLLA
ApplicationB
Application B
Transport
UA
TDAUA GAPA
Peer LaCOLLA
RA
Application B
Application B
Application B
Application A
Application A
Application A
Application A
Navarro, Marquès & Freitag 12
April 2004
Mechanisms
Categories of Mechanisms UA RA GAPA TDA EAEvents X X - - -Objects X X - - XTasks/services X - - X XPresence X X X X XLocation X X X X XInstant Messaging X - X - -Groups X X X X XMembers X - X - -Security X X X X XDisconnected Operational Mode X - - - -
Navarro, Marquès & Freitag 13
April 2004
Virtual synchronism
● Immediate and consistent dissemination of events– All peers know what is happening in the group:
● All connected peers receive events immediately after they occur.● Not connected members receive events when they connect.
– How is achieved?● Immediate: the component where the event is generated disseminates it.● Consistent: weak consistency algorithms to guarantee that eventually all
connected components will have all events.● Virtually strong consistency of objects
– Immediate and consistent dissemination of events guaranties that all peers know where each object is located.
– Storage components store and reproduce objects in an autonomous way.
● They don't need to worry about consistency. The consistency is achieved because all components know the location of all objects.
Navarro, Marquès & Freitag 14
April 2004
Execution of tasks
● Organized in a decentralized and dynamic manner– TDAs coordinate among them in a decentralized manner to schedule
tasks.– EAs execute tasks– When the task is finished, the result is provide to the user in an
asynchronous way. If user is not connected, the result will be provided to him at his next connection.
● Storage (code and data), presence, ... provided by LaCOLLA● Based on ideas used to design JNGI (<http://jngi.jxta.org>)
– JNGI: a decentralized and dynamic framework for large-scale computations for problems that features coarse-grained parallelization. Components of JNGI communicate using JXTA.
● In our case use communication facilities of LaCOLLA.
Navarro, Marquès & Freitag 15
April 2004
Prototype and validation
● Architecture validated by simulation:– To prove basic functionality (virtual synchronism)
● Members know what is happening at a time that members perceive as immediate (while the actions occur).
● Members have access to generated objects right after they are created.– Simulation using J-Sim <http://www.j-sim.org>
● [Now:] Prototype:– basic functionality (implemented)– execution task capability (being implemented)– language: java (future: extend to other languages)
● Applications:– chat (implemented)– sharing documents (being implemented)– execute java applications– ...
Navarro, Marquès & Freitag 16
April 2004
Conclusions
● An infrastructure that provides general purpose collaborative functionalities (groups, awareness and storage) will facilitate the creation of collaborative applications.
● + infrastructure use resources provided by members: self-sufficiency of groups.
● + infrastructure has the ability to execute tasks (and deploy services): groups will gain in autonomy and independence.
Navarro, Marquès & Freitag 17
April 2004
Conclusions (II)
● LaCOLLA is our proposal for an autonomous and self-organized infrastructure for collaboration support that only requires resources provided by group members.– Refined in terms of components and mechanisms.– Validated by simulation.– Prototype implemented + some application.
● Future work:– Extend/complete the implementation (deploy services,
security, ...).– Extend the range of implemented applications (instant messaging,
blackboard, collaborative editor, ...).– Use applications in real situations in order to improve the
infrastructure.
Navarro, Marquès & Freitag 18
April 2004
On Distributed Systems and CSCL
Joan Manuel Marquès - [email protected] Oberta de Catalunya
Leandro NavarroFelix Freitag
Universitat Politècnica de Catalunya
Navarro, Marquès & Freitag 19
April 2004
On Distributed Systems and CSCL
Leandro NavarroJoan Manuel Marquès - [email protected]
Felix Freitag
Navarro, Marquès & Freitag 20
April 2004
Abstractions
● Concept of group– (= operations at group level; operations involve groups)– Hide complexity of group dispersion; ...
● Know what is going on in the group.– Provide a means to disseminate information about actions that take
place inside the group to all group members in a transparent way (independently of location of members).
● Capacity to share information generated in the group.– Allow access to objects generated inside the group at any moment;
independently whether the member was connected or disconnected when the object was created.
● Execute tasks and deploy services– Allow members of a group user computational resources belonging
to group.
Navarro, Marquès & Freitag 21
April 2004
Abstractions (II).
● Collaborative groups we are interested in are characterized by:
– Few members– Dispersed– Multiplicity (= members belong to one or more groups)– Decentralized organization– Members behave autonomously– Self-sufficient– Not all group members are equal– No fix connection point– Not all peers are equal
Navarro, Marquès & Freitag 22
April 2004
Abstractions (III).
● How can we know what is going on in the group?
– Disseminating events that inform about actions done by group members
– This dissemination should be:● Immediate● Consistent
Navarro, Marquès & Freitag 23
April 2004
Abstractions (IV).
● How should be the storage of information generated inside groups?
– Available: persistent, resilient, reproduced– Decentralized and distributed– Detects conflicts in object handling– Objects located near demand– Dynamic– Storage capacity and bandwidth adapted to group needs
Navarro, Marquès & Freitag 24
April 2004
Mechanisms (II)
● Behaviors– Push: send to other interested components
● disseminate event; I'm still alive; ...– Pull: ask another component
● consistency sessions; ...– Autonomous decisions: decide according to local information
● replication of objects; get object; inactive component; ...
● Mechanisms implemented using:– Diffusion / application level broadcast– Weak-consistency optimistic protocols– Random selection
● Transport: reliable channel (TCP)
Navarro, Marquès & Freitag 25
April 2004
Table of contents
● Introduction● Foundations● Contribution 1: architecture● Contribution 2: virtual synchronism● Validation● Conclusions & future work
Navarro, Marquès & Freitag 26
April 2004
Prototype
● LaCOLLA architecture behaves in a self-organized manner.
● Virtual synchronism guaranties that group members:– Know what is happening at a time they perceive as immediate
(while the actions occur).– Have availability of generated objects right after the object are
created.● All the above is achieved even though members are using
their own resources.● Presence influences self-organization.● Consistent view is achieved even though all components
are not consistent.
Navarro, Marquès & Freitag 27
April 2004
Table of contents
● Introduction● Foundations● Contribution 1: architecture● Contribution 2: virtual synchronism● Validation● Conclusions & future work
Navarro, Marquès & Freitag 28
April 2004
Validation
● By simulation
– Using J-Sim <http://www.j-sim.org>● Provides network simulation until transport layer.● Scripting language (tcl) to organize simulations.● Simulator code in Java.● Generated code requires few changes to be be used in a real infrastructure.
– Simulation variables and parameters● Kind of groups: size and activity level of its members.● Levels of dynamism: connection, disconnection, mobility and failures.● Parameters of mechanisms.● Degree of reproduction (#RA and #GAPA)
Navarro, Marquès & Freitag 29
April 2004
Simulation
● Three cases:– “Normal situation”:
● 10 members; low level of dynamism; different degrees of reproduction.– Effect of dynamism.– Effect of scale:
● small groups, larger groups with little dynamism, very dynamic larger groups.
● Methodology: (for each experiment)– Hypothesis– Definition of the experiment– Results– Conclusions
Navarro, Marquès & Freitag 30
April 2004
Simulation (II)
time
seco
nds
to b
e co
nsis
tent
Phase 1: show tolerance to failures and changes● Events and objects are generated depending on activity degree.● Occur failures and changes (connections, disconnections and mobility) depending on degree of dynamism.● All internal mechanism are operative.
Phase 2: ability to recover automatically● Calculate how much time requires LaCOLLA to be consistent(Consistent: all components have same information about: presence and location information, available objects, events, GAPA belonging to group).● Only are active internal mechanisms.
Navarro, Marquès & Freitag 31
April 2004
Types of analysis used in the validation(influence of dynamism)
● Percentage of experiments that have necessary resources (RA and GAPA) connected to group at the end of first phase
– 10 Members– Low dynamism
– 10 Members– High dynamism
2RA 2GAPA 4RA 4GAPA 5RA 5GAPA 6RA 6GAPA 8RA 8GAPA 10RA 10GAPA
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Percentatge d'experiments que han finalitzat la primera fase amb, com a mínim, un RA i un GAPA(per 10 UA)
No hi ha com a mínim un RA i un GAPA
Mínim un RA i un GAPA
grau de reproducció
perc
enta
tge
2RA 2GAPA 4RA 4GAPA 5RA 5GAPA 6RA 6GAPA 8RA 8GAPA 10RA 10GAPA
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Percentatge d'experiments que han finalitzat la primera fase amb, com a mínim, un RA i un GAPA(per 10 UA)
No hi ha com a mínim un RA i un GAPA
Mínim un RA i un GAPA
grau de reproducció
perc
enta
tge
Navarro, Marquès & Freitag 32
April 2004
Types of analysis used in the validation (II)(automatic recovery)
● 10 Members● Low dynamism
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 320.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Accumulated probability that, in a number of iterations,LaCOLLA is self-organized.
2RA 2GAPA
4RA 4GAPA
5RA 5GAPA
6RA 6GAPA
8RA 8GAPA
10RA 10GAPA
# iterations
pro
ba
bili
ty
Navarro, Marquès & Freitag 33
April 2004
Types of analysis used in the validation
● 10 Members● Low Dynamism
0 1 2 3 4 5 6 7 8 9 10 11 12 13
0
20
40
60
80
100
120
140
160
Distribution of quantity of experiments that arrivedto a consistent state for each number of iteration.
4RA 4GAPA
5RA 5GAPA
6RA 6GAPA
8RA 8GAPA
10RA 10GAPA
# iterations
# e
xpe
rim
en
ts
0 1 2 3 4 5 6 7 8 9 10 11 12 130
1
2
3
4
5
6
7
8
9
10
Distribution of quantity of experiments that arrived to a consistent state
for each number of iteration.
4RA 4GAPA
5RA 5GAPA
6RA 6GAPA
8RA 8GAPA
10RA 10GAPA
# iterations
# ex
peri
men
ts
Navarro, Marquès & Freitag 34
April 2004
Validation summary
● Percentile 90 Low Dynamism # RA and #GAPA: 10
020406080
100120140160180200220
0 20 40 60 80 100
size (#members)
#sec
on
ds
self-organizationvirtual synchronismpresence + location
Navarro, Marquès & Freitag 35
April 2004
Conclusions of validation
● LaCOLLA architecture behaves in a self-organized manner.
● Virtual synchronism guaranties that group members:– Know what is happening at a time they perceive as immediate
(while the actions occur).– Have availability of generated objects right after the object are
created.● All the above is achieved even though members are using
their own resources.● Presence influences self-organization.● Consistent view is achieved even though all components
are not consistent.
Navarro, Marquès & Freitag 36
April 2004
Table of contents
● Introduction● Foundations● Contribution 1: architecture● Contribution 2: virtual synchronism● Validation● Conclusions & future work
Navarro, Marquès & Freitag 37
April 2004
Conclusions
● An infrastructure that provides asynchronous collaborative functionalities in an autonomous manner, requires that the infrastructure has the ability to self-organize.
● Virtual synchronism contributes to autonomy because it guarantees immediate and consistent availability of awareness information and generated objects to all members and components.
● LaCOLLA is my proposal for an autonomous and self-organized infrastructure for collaboration support that only requires resources provided by group members.– Refined in terms of components and mechanisms.– Validated by simulation.
Navarro, Marquès & Freitag 38
April 2004
Future work
● Implementation of a prototype– Prototype and applications [currently in progress]– Implement internal mechanisms:
● Delete objects● Conflict detection● Management of groups and members● Instant messaging● Disconnected operational mode● Handling of anomalous situations
● New mechanisms– Extend internal mechanisms: events transformation; security and anonymity.– Decide best location and optimal deployment of objects– Improve download mechanism (download from many locations)– Allow sharing of resources between groups– Decide best algorithm for each mechanism (improve random techniques)