mod2014-mens-lecture3

Evolving6So9ware6Ecosystems6

Marktoberdorf6Summer6School62014Lecture63

Tom6Mens6

So#ware(Engineering(Lab(University(of(Monsinforma7que.umons.ac.be/genlog

Natural(Ecosystems

65

Standing(on(the(shoulders(of(giants

Lamarck

Lotka

Volterra

Darwin

Pareto

July?August(2014(—(NATO(Marktoberdorf(Summer(School(—(Dependable(So#ware(Systems(Engineering

Main(Research(Ques7ons

Which(control(mechanisms(driving(natural(ecosystems(can(be(used(to(explain(dynamics(of(so#ware(ecosystems?(!

Which(mechanisms(and(measures(can(we(borrow(from(ecology(to(explain(and(predict(how(so#ware(projects(evolve?

66

Biological(evolu7on

AND(BY(A(DUMMY

July?August(2014(—(NATO(Marktoberdorf(Summer(School(—(Dependable(So#ware(Systems(Engineering 68

Terminology Biological(ecosystem

Defini7ons

• Ecology:(the/scien$fic/study/of/the/interac$ons/that/determine/the/distribu$on/and/abundance/of/organisms(

• Ecosystem:/the/physical/and/biological/components/of/an/environment/considered/in/rela$on/to/each/other/as/a/unit/– combines(all(living(

organisms((plants,(animals,(micro?organisms)(and(physical(components((light,(water,(soil,(rocks,(minerals)

Example:(coral(reefs

• High(biodiversity:(polyps,(sea(anemones,(fish,(mollusks,(sponges,(algae


Comparison


Comparison

Two/dual/views


Comparison

software ecosystem - view 1

shared libraries

environment: hardware, OS, external software, developers, users, ...

species

active projects

archived projects

Two/dual/views


Comparison

Two/dual/views

software ecosystem - view 2

developers

environment: source code, tests, commits, bug reports,

species

testers

end-users debuggers


Comparison(Bipar7te(networks

In(ecology(hosts(?(parasites(plants(?(pollinators

73

D6.2 Periodic report - scientific results

Figure 3.3: Example of bi-partite mutualistic relationship

DIVERSIFY aims at defining software evolution rules, which can lead to the emergence of soft-ware diversity in client-server networks. This is illustrated in figure 3.4. There are different kindsof servers that have emerged through software evolution (e.g., through addition/removal of kernelpackages) and client machines are possibly connected to more than one server that can provide therequired service. The figure also illustrates that we target a network with more diversity but also withmore machines. More servers will provide robustness through redundancy, but it will also increasethe cost of the network. The set of software evolution rules, should thus consider a trade-off betweenthe robustness of the network and its global cost.

We adapt state-of-the-art ecological metrics to assess the benefit of diversity of software bi-partiterelationships. First, we adapt biodiversity metrics to quantify the evolution of diversity among servers.Shannon entropy is a a classical metric to quantify the diversity of species inside one level of the graph,while several indices exist to measure the diversity of relations [15]. To demonstrate a beneficialeffect of diversity on robustness, we adapt the robustness metric of Dunne et al. [17]. This metricfor bipartite ecological relations evaluates the ability of one level to survive extinction sequencesof the other level. Our experiments will evaluate robustness according to different server extinctionsequences, e.g., BOBE attack or random failure. Details about these metrics in ecology and theiradaptation in software diversity are provided in D1.1 and D2.1.

Figure 3.4: Emerging software diversity in servers

3.2 Neutral and non-neutral evolution

The EvSD paradigm for software diversification follows an ecological approach to explore the impactof evolution on diversity emergence and its impact on the robustness on a software system. Forour experiments, we analogize a client-server architecture to a plant-pollinator bipartite ecologicalnetwork. Using the EvSD paradigm, we draw upon ideas from evolutionary biology and biodiversitydynamics to create a new model of server-side evolutionary software diversification and present 2 setsof evolutionary strategies to test the paradigm.

As in the case of evolutionary biology, the first set of evolutionary strategies, called the NeutralStrategies are used to designed a neutral model of server evolution. The strategies in this set are neutral

DIVERSIFY - 600654 17

Mutual(dependency(and(func7onal(redundancy:(

disappearance(of(one(species(may(be(compensated(by(other(species(if(there(is(sufficient(diversity(in(both(layers



In(so#ware(ecosystems(contributors(?(projects

74

Projet 1

Projet 2

Projet 3

D6.2 Periodic report - scientific results

Figure 3.3: Example of bi-partite mutualistic relationship

DIVERSIFY aims at defining software evolution rules, which can lead to the emergence of soft-ware diversity in client-server networks. This is illustrated in figure 3.4. There are different kindsof servers that have emerged through software evolution (e.g., through addition/removal of kernelpackages) and client machines are possibly connected to more than one server that can provide therequired service. The figure also illustrates that we target a network with more diversity but also withmore machines. More servers will provide robustness through redundancy, but it will also increasethe cost of the network. The set of software evolution rules, should thus consider a trade-off betweenthe robustness of the network and its global cost.

We adapt state-of-the-art ecological metrics to assess the benefit of diversity of software bi-partiterelationships. First, we adapt biodiversity metrics to quantify the evolution of diversity among servers.Shannon entropy is a a classical metric to quantify the diversity of species inside one level of the graph,while several indices exist to measure the diversity of relations [15]. To demonstrate a beneficialeffect of diversity on robustness, we adapt the robustness metric of Dunne et al. [17]. This metricfor bipartite ecological relations evaluates the ability of one level to survive extinction sequencesof the other level. Our experiments will evaluate robustness according to different server extinctionsequences, e.g., BOBE attack or random failure. Details about these metrics in ecology and theiradaptation in software diversity are provided in D1.1 and D2.1.

Figure 3.4: Emerging software diversity in servers

3.2 Neutral and non-neutral evolution

The EvSD paradigm for software diversification follows an ecological approach to explore the impactof evolution on diversity emergence and its impact on the robustness on a software system. Forour experiments, we analogize a client-server architecture to a plant-pollinator bipartite ecologicalnetwork. Using the EvSD paradigm, we draw upon ideas from evolutionary biology and biodiversitydynamics to create a new model of server-side evolutionary software diversification and present 2 setsof evolutionary strategies to test the paradigm.

As in the case of evolutionary biology, the first set of evolutionary strategies, called the NeutralStrategies are used to designed a neutral model of server evolution. The strategies in this set are neutral

DIVERSIFY - 600654 17



In(so#ware(ecosystems(contributors(?(projects

75

Projet 1

Projet 2

Projet 3

Mutual(dependency(and(func7onal(redundancy:(

disappearance(of(projects(or(contributors(may(be(compensated(by(others


Ecological(theories(of evolu7on(of(species

• Different(evolu7onary(theories(• Natural(selec7on((Darwinism)(• Re7culate(evolu7on(• Hologenome(theory((symbiosis)

76


Evolu7onary(theories(Darwinism

• all(species(of(life(have(descended(over(7me(from(common(ancestors(

• this(branching(patern(of(evolu7on(resulted(from(natural6selec&on,(similar(to(ar7ficial(selec7on(in(selec7ve(breeding(

!• Example(

– 13(types(of(Galapagos(finches,(same(habits(and(characteris7cs,(but(different(beaks

Charles(Darwin (1809–1882)


Evolu7onary(theories(Darwinism

Darwin((1837)• Evolu7on(history(of(species(can(be(represented(by(a(phylogene/c$tree.(

• Describes(the(evolu7onary(rela7onships(among(species(assuming(that(they(share(acommon(ancestor.


Evolu7on(History So#ware

• Type(to(enter(text


Evolu7onary(theories(Re7culate(evolu7on

Re7culate(evolu7on(• Evolu7on(history(is(represented(as(a

graph(structure(!

• Two(or(more(evolu7onary(lineages(can(berecombined(at(some(level(• hybrid/specia$on

(2(lineages(recombine(to(create(a(new(one)((• horizontal/gene/transfer

(genes(are(transferred(across(species)



Re7culate(evolu7on(of(so#ware(?(((

• hybrid/specia$on

Distributed(version(control(systems((e.g.(Git)(promote(re7culate(evolu7on(through(fork(and(merge(In(prac7ce,(few(projects(actually(merge(

Robles/et/al./A/Comprehensive/Study/of/SoCware/Forks:/Dates,/Reasons/and/Outcomes./OSS/Conference/2012,/Best/Paper/Award.



Re7culate(evolu7on(of(so#ware(?((!

• horizontal/gene/transferTransfer/of/technical/resources/(code)/All(large(so#ware((eco)systems(exhibit((a(large(number(of(accidental(or(deliberate(“code(clones”(and(“code(reuse”(Very(ac7ve(research(domain(in(so#ware(engineering(

Mojica et al. “A Large Scale Empirical Study on Software Reuse in Mobile Apps”; IEEE Software 2014



Re7culate(evolu7on(of(so#ware(?((!

• horizontal/gene/transferTransfer/of/human/resources/How/do/project/contributors/join,/leave/and/move/across/projects/in/the/ecosystem?/How/are/newcomers/aQracted/to/an/ecosystem?

Weiss(et(al.(Evolu$on/of/Open/Source/Communi$es./OSS(2006(Bird(et(al.(Open/Borders?/Immigra$on/in/Open/Source/Projects./MSR(2007(Jergensen(et(al.(The/onion/patch:/migra$on/in/open/source/ecosystems./ESEC/FSE(2011(Canfora(et(al.(Who/is/going/to/mentor/newcomers/in/open/source/projects?(FSE(2012


Evolu7onary(theories(Hologenome(theory

Hologenome(theory(• The(unit(of(natural(selec7on(is(the(holobiont:(the(organism(

together(with(its(associated(bacterial(communi7es,(that(live(together(in(symbiosis.(

• The(holobiont(can(adapt(to(changing(environmental(condi7ons(far(more(rapidly(than(by(gene7c(muta7on(and(selec7on(alone.((

• Darwinism(emphasises(compe$$on((survival(of(the(fitest),(hologenome(theory(also(includes(coopera$on((through(symbiosis)


Evolu7onary(theories(Hologenome(theory

Hologenome(theory(in(so#ware(evolu7on?(!Hologenome(theory(may(be(closer(to(what(one(observes(in(open(source(projects(where(coopera$on(plays(a(more(important(role.


Trophic(web((food(chain) in(natural(ecosystems


Trophic(web(in so#ware(ecosystems

Producer?consumer(rela7on

Users

Peripheral(developers

Core(developers

Onion(model

TOP?DOWN(change(requests(&(bug(reports

BOTTOM?UP(changes(in(core(projects(and(architecture(

Nakakoji(et(al.(Evolu$on/paQerns/of/openUsource/soCware/systems/and/communi$es.(IWPSE(2002.(


Ecosystem(Dynamics

PredatorIprey6rela&onship

• An(instance(of(the(consumer?resource(rela7onship(

• Predators((hun7ng(animals)(feed(upon(their(prey((atacked(animals)

Dynamic(model

• Two(mutually(dependent(parametric(differen7al(equa7ons(Lotka?Volterra(1925/1926)


So#ware(EcosystemDynamics

PredatorIprey6rela&onship6in6so9ware6ecosystems?

• Debuggers(are(predators,(so#ware(defects(are(prey(Calzolari/et/al./Maintenance/and/tes$ng/effort/modeled/by/linear/and/nonlinear/dynamic/systems,”/Informa$on/and/SoCware/Technology,/43(8):/477/–/486,/2001


• Stability/• the(capacity(to(maintain(an(equilibrium(over(longer(periods(of(7me(

• Resistance/• the(ability(to(withstand(environmental(changes(without(too(much(

disturbances(of(its(biological(communi7es(• Resilience/

• the(ability(to(return(to(an(equilibrium(a#er(a(disturbance

Desirable(ecosystem(characteris7cs

91


• Stability/• the(capacity(to(maintain(an(equilibrium(over(longer(periods(of(7me(

• Resistance/• the(ability(to(withstand(environmental(changes(without(too(much(

disturbances(of(its(biological(communi7es(• Resilience/

• the(ability(to(return(to(an(equilibrium(a#er(a(disturbance(!

Higher(biodiversity(favours(these(characteris7cs

Desirable(ecosystem(characteris7cs

92


Desirable(ecosystem(characteris7cs(Biodiversity

Biodiversity((of(species)(measures(the(degree(of(varia7on(of(species(within(a(given((natural)(ecosystem(• Many(different(metrics:(• species(richness(• species(evenness(• Shannon(diversity(index(• Simpson(index(• …

Goal:(Use(these(and(related(measures(to(study(maintainability(and(survivability(of(so#ware(projects(within(their(ecosystem

mod2014-mens-lecture3

Education