metaheuristic models for decision support in the software ... · dr. josé raúl romero salguero...
TRANSCRIPT
Metaheuristic models for decision support in the softwareconstruction process
– Ph.D. Thesis –
Aurora Ramírez Quesada
Dept. Computer Science and Numerical AnalysisUniversity of Córdoba
Supervisors:Dr. José Raúl Romero Salguero
Dr. Sebastián Ventura Soto
24th September 2018
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
1 Introduction
2 Evolutionary discovery of software architectures
3 The multi- and many-objective perspectives
4 The human-in-the-loop approach
5 Conclusions and future work
Aurora Ramírez Quesada Ph.D. Thesis 2 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Introduction
Aurora Ramírez Quesada Ph.D. Thesis 3 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Software architectures
A software architecture represents “the fundamental concepts or properties of asystem in its environment embodied in its elements, relationships, and in the
principles of its design and evolution” (ISO/IEC Std. 42010)
A bridge between requirements and implementation
Essential in early software conception and its subsequent evolution
Special focus on non-functional requirements: availability, performance, security...
Software architectures are fundamental for:Understanding
Reuse
Construction
Evolution
Analysis
Management
Aurora Ramírez Quesada Ph.D. Thesis 4 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Metaheuristics
“Intelligent” exploration of the search space
Optimality is not guaranteed
Intensification and diversificationProperties:
1 Iterative
2 Stochastic
3 Memory-based
4 Bio-inspired
Aurora Ramírez Quesada Ph.D. Thesis 5 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Evolutionary computation
“It is not the strongest of the species that survives, nor the most intelligent thatsurvives. It is the one that is most adaptable to change.” (Charles Darwin)
Phenotype/genotype mapping
The fitness function
Aurora Ramírez Quesada Ph.D. Thesis 6 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Optimisation with multiple objectives
Multi-objective optimisation problem:
x = [x1, x2, ..., xn] subject to gj (x) ≤ 0 j = 1, ..., pf (x) = [f1(x), f2(x), ..., fm(x)]
2-3 objectives, often in conflict (≥ 4→ many-objective)
The concept of Pareto dominance
No single optimal solution⇒ Pareto Set (PS)
In the objective space⇒ Pareto Front (PF)
The goals of a multi-/many-objective evolutionary algorithm (MOEA/MaOEA):
1 To find a good approximation of the true PF
2 To generate a well-spread and uniform PF
Aurora Ramírez Quesada Ph.D. Thesis 7 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Interactive optimisation
Full automation is not always realistic:Uncertainty scenarios
Creative tasks
Any optimisation method, includingmetaheuristics, in which the human
actively participates in any step of theprocess to provide feedback
Human is put in the loop:To provide problem knowledge
To increase trust on automatic results
To meet user’s expectations
Design and implementation issues:Selection of solutions
Type of feedback
Frequency of interaction
Information lifetime
Aurora Ramírez Quesada Ph.D. Thesis 8 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Search-based Software Engineering
Origin and characteristicsSBSE was formalised as research field by Harman and Jones in 2001
Software engineering activities can be reformulated as optimisation problems
Examples along the software lifecycle: project planning, requirement priorisation,selection of test cases, refactoring...
Software metrics are adopted to define the fitness function
Evolutionary computation (EC) stands out as the most popular technique
Search-based software design (SBSD)The need of creativity and intuition makes it particularly challenging
Applicable to different paradigms (object-oriented, services...)
Problem encoding and evaluation are hard to define
Aurora Ramírez Quesada Ph.D. Thesis 9 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Background - Software architecture optimisation
Scope and characteristicsAutomatic specification and improvement of architectural models
Often guided by more than one non-functional requirement
Problems can be defined at different levels of abstraction
Current approachesArchitecture synthesis from requirements or use cases
Selection and assembling of black-box components, e.g. COTS
Recovery of software components from code
Architecture deployment (reconfiguration and allocation)
Aurora Ramírez Quesada Ph.D. Thesis 10 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Objectives
General objective:
Development of search models based on metaheuristic algorithms to providesemi-automatic support to software engineers in early stages of the software
development process
Subobjectives:
O1: Analysis of the state of the art in SBSE, especially in the areas of SBSD andarchitecture optimisation
O2: Design and development of a metaheuristic model based on EC to supportthe discovery of component-based software architectures
O3: Design and development of multi-objective approaches and hybrid techniquesto study their suitability to the decision support process
O4: Design and development of an interactive approach to incorporate humanexpertise in the context of software architecture optimisation
Aurora Ramírez Quesada Ph.D. Thesis 11 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Work plan
1 Formulation of software architecture discovery as an optimisation problem
2 A first evolutionary model to validate the idea
3 Exploration of diverse search techniques to cope with additional decision factors
4 Participation of the engineer to complete the scenario
Aurora Ramírez Quesada Ph.D. Thesis 12 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Problem description
Software architecture discovery
To identify the underlying architecture of the system from the analysis informationcontained in a UML class diagram
The elements of the component-based software architecture are obtained as follows:ComponentA cohesive group of classes that work together to satisfy the expected behaviourof the componentInterfaceA directed relationship between classes in the analysis model belonging todifferent componentsConnectorThe linkage between a pair of required / provided interfaces interconnectingdifferent components
Aurora Ramírez Quesada Ph.D. Thesis 13 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Evolutionary discovery of software architectures
Aurora Ramírez Quesada Ph.D. Thesis 14 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Motivation
Current approaches recover components from low-level artefacts, e.g. code
The process cannot be performed until the system is built
The system structure could be different from that originally conceived
Analysis information is omitted
Aurora Ramírez Quesada Ph.D. Thesis 15 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Research questions
RQ: Can single-objective EAs help the software engineer to iden-tify an initial candidate architecture of a system at a high level ofabstraction?
RQ: How does the configuration of the algorithm influence both theevolutionary performance and the quality of the returned solution?
Aurora Ramírez Quesada Ph.D. Thesis 16 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Solution encoding
Solutions are encoded using tree structures:
Hierarchical decomposition
Comprehensive for software engineers
Architectural solutions of different size A_prov_E
E_req_A
Phenotype
Architecture
Components Connectors
Component_1 Component_2 Connector_1
Classes ClassesProvided
interfaces
Required
interfaces
Provided
interfaces
Required
interfaces
Provided
interface
Required
interface
B A_prov_E E_req_A A_prov_EE_req_AA C D FE G H
Genotype
Aurora Ramírez Quesada Ph.D. Thesis 17 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Fitness function
Intra-modular Coupling Density (ICD)
ICDi =#classestotal −#classesi
#classestotal·
CI ini
CI ini + CIout
iICD =
1n·
n∑i=1
ICDi
External Relations Penalty (ERP)
ERP =n∑
i=1
n∑j=i+1
(was · nasij + wag · nagij + wco · ncoij + wge · ngeij )
Groups/Components Ratio (GCR)
GCR =#cgroups
n
fitness(ind) =
{r(ICD(ind)) + r(ERP(ind)) + r(GCR(ind)) if ind is feasible#individuals ·#metrics + 1 if ind is infeasible
Aurora Ramírez Quesada Ph.D. Thesis 18 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Mutation operator
Algorithm 1: Add
1 Choose contributors;2 Extract small groups;3 Create new component;4 Add selected classes;5 Set interfaces;
Algorithm 2: Remove
1 Find highly-coupled components;2 Choose component;3 Distribute classes;4 Set interfaces;
Algorithm 3: Move
1 Choose origin;2 Choose class;3 Choose target;4 Move class;5 Set interfaces;
Algorithm 4: Merge
1 Find highly-coupled components;2 Choose candidate 1;3 if found>2 then4 Choose candidate 2;5 else6 Choose component 2;7 Join classes;8 Set interfaces;
Algorithm 5: Split
1 Find components with > 1 group;2 if found then3 Choose candidate;4 Distribute groups;5 else6 Choose component;7 Distribute classes;8 Set interfaces
Aurora Ramírez Quesada Ph.D. Thesis 19 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Illustrative example
(a) Input class diagram
Component_3
Component_2Component_1
E_req_A
G_req_H
A_prov_E
H_prov_G
ICD=0.17
GCR=2.00
Fitness=7.00
ERP=13.00
(b) Initial population
Fitness=5.00 ERP=5.00ICD=0.35 GCR=1.50
A_prov_E
E_req_A
Component_1 Component_2
(c) 5th Generation
Fitness=3.00 ERP=0.00ICD=0.38 GCR=1.00
A_prov_E
E_req_A
Component_1 Component_2
(d) 10th Generation
Aurora Ramírez Quesada Ph.D. Thesis 20 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Experimental framework
Implementation details
JCLEC as the metaheuristic framework to code the algorithm
SDMetrics Open Core to parse XMI files
Datapro4j to preprocess input data and generate reports
Problem instances
Problem #Classes #Relationships #InterfacesAs De Ag Co GeAqualush 58 69 6 0 0 20 74Borg 167 44 109 36 38 90 300Datapro4j 59 3 4 3 2 49 12Java2HTML 53 20 66 15 0 15 170JSapar 46 7 33 21 9 19 80Marvin 32 5 11 22 5 8 28NekoHTML 47 6 17 15 18 17 46
Aurora Ramírez Quesada Ph.D. Thesis 21 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Results
Parameter study
Selection and replacement strategies: Binary tournament, 10% elitism
Mutation weights (126 combinations):wadd = 0.2, wremove = 0.1, wmerge = 0.1, wsplit = 0.3, wmove = 0.3
Population size and number of evaluations: 150, 20,000-24,000
Final results
Problem ICD (↑) ERP (↓) GCR (↓) Time (ms)Aqualush 0.41240.0604 6.23333.8443 1.08330.1708 116.10198.4891Borg 0.28200.0689 3.93332.4626 1.16670.2274 2, 489.1223171.2028Datapro4j 0.64250.0356 33.600014.7436 2.39890.6744 37.11012.8308Java2HTML 0.25930.0000 0.00000.0000 1.00000.0000 250.45727.4674JSapar 0.37510.0307 9.00001.5492 1.16670.2981 94.88915.1418Marvin 0.50800.0187 3.16671.0980 1.67500.1146 14.11940.6885NekoHTML 0.45940.0345 3.26675.3037 1.23890.3768 57.11503.5524
Aurora Ramírez Quesada Ph.D. Thesis 22 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Lessons learned
I Lack of software metrics suitable for fitness function definition
I Need of domain-specific operators to improve performance
I Ability to obtain solutions close to manual designs
I Influence of the number and type of UML relations in the hardness of the problem
Aurora Ramírez Quesada Ph.D. Thesis 23 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Associated publications
Main publication:
A. Ramírez, J.R. Romero, S. Ventura. “An approach for the evolutionary discovery ofsoftware architectures”. Information Sciences, vol. 305, pp. 234-255, 2015.
Additional publications:
1 A. Ramírez, J.R. Romero, S. Ventura. “Análisis de la aplicabilidad de medidas software para eldiseño semi-automático de arquitecturas”. Proceedings of the XIX Jornadas en Ingeniería delSoftware y Bases de Datos (JISBD), pp. 307-320, 2014.
2 A. Ramírez, J.A. Molina, J.R. Romero, S. Ventura. “Estudio de mecanismos de hibridación parael descubrimiento evolutivo de arquitecturas”. Proceedings of the XXI Jornadas en Ingenieríadel Software y Bases de Datos (JISBD), pp. 481-494, 2016.
3 A. Ramírez, R. Barbudo, J.R. Romero, S. Ventura. “Memetic Algorithms for the Automatic Dis-covery of Software Architectures”. Proceedings of the 16th International Conference on Intelli-gent Systems Design and Applications (ISDA), vol. 557 AISC, pp. 437-447, 2016.
Aurora Ramírez Quesada Ph.D. Thesis 24 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
The multi- and many-objective perspectives
Aurora Ramírez Quesada Ph.D. Thesis 25 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Motivation
Evidence of conflict between ICD and ERP
Additional quality criteria usually guide the discovery process
Each system might require the optimisation of a different number of metrics
The algorithm could provide a set of alternative solutions for engineer’s choice
Aurora Ramírez Quesada Ph.D. Thesis 26 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Research questions
RQ: How do a larger number and combination of metrics influencethe search-based discovery of software architectures?
RQ: How do state-of-the-art MOEAs and MaOEAs perform whenmultiple objectives guide the discovery process?
Aurora Ramírez Quesada Ph.D. Thesis 27 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Objective functions
Maintainability is “the degree to which the software product can be modified", wheremodifications “may include corrections, improvements or adaptation of the software to
changes in the environment, and in requirements and functional specifications"(ISO/IEC Std. 25000)
Modularity: “the degree to which a system is composed of discrete componentssuch that a change to one component has minimal impact on other components"
Reusability: “the degree to which an asset can be used in more than one softwaresystem or in building other assets"
Analysability: “the degree to which the parts of the software to be modified can beidentified"
Metric Min/Max Quality attribute Range of values Design goalsICD max modularity [0, 1] Small components with high cohesionERP min modularity [0, ∗] Large components with low couplingINS min modularity [0, 1] Components with few interactionsENC max modularity [0, 1] Components with hidden classesCS min modularity [0, n] Small or medium-sized componentsCL min modularity [0, n] Components with few provided interfaces
GCR min reusability [1, ∗] Connected classes within each componentABS max reusability [0, 1] Components with abstract classesCB max analisability [0, 1] Equal-sized components
Aurora Ramírez Quesada Ph.D. Thesis 28 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Multi/Many-objective evolutionary algorithms
1 Multi-objective algorithms (Pareto-based)SPEA2
NSGA-II
2 Decomposition algorithmsMOEA/D
3 Algorithms based on reference setNSGA-III
4 Algorithms based on landscape partitionε-MOEA
GrEA
5 Indicator-based algorithmsIBEA
HypE
Aurora Ramírez Quesada Ph.D. Thesis 29 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Experimental framework
Development of JCLEC-MOIndependence between the metaheuristic and the multi-objective techniques
Experimental utilities
Parameters and analysisAll possible combinations of 2, 4, 6, 8 and 9 metrics (256 in total)
The number of software systems is increased up to 10 (32 - 256 classes)
Quality indicators: hypervolume, spacing
Statistical analysis: Friedman&Holm, Cliff’s Delta (effect size)
Parameter ValuePopulation size 100, 120, 126, 330, 495Max. evaluations 10000, 15000, 20000, 66000, 99000Min-max. components 2-8Mutator weights wadd = 0.2, wremove = 0.3, wmerge = 0.2
wsplit = 0.1, wmove = 0.2ERP weights was = 2, wag = 3, wco = 3, wge = 5CS threshold 0.3CL threshold 8Algorithms configured following authors’ recommendations / preliminary experiments.
Aurora Ramírez Quesada Ph.D. Thesis 30 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Results
HypervolumeAlgorithm 2 4 6 8 9SPEA2 4.35± 0.78 4.66± 0.86 5.64± 0.29 6.33± 0.28 6.70± 0.00NSGA-II 2.02± 0.59 1.39± 0.40 1.92± 0.54 2.66± 0.46 2.90± 0.00MOEA/D 3.90± 0.81 4.41± 0.68 3.75± 0.47 3.48± 0.43 3.30± 0.00NSGA-III 6.75± 0.49 6.62± 0.21 6.49± 0.16 6.01± 0.29 5.90± 0.00ε-MOEA 3.88± 1.12 2.78± 1.15 1.51± 0.55 1.08± 0.09 1.00± 0.00GrEA 4.86± 0.79 5.30± 0.75 5.80± 0.41 5.83± 0.11 5.50± 0.00IBEA 7.21± 0.31 7.64± 0.15 7.76± 0.06 7.79± 0.05 7.90± 0.00HypE 3.04± 0.58 3.20± 0.75 3.20± 0.55 2.82± 0.31 2.80± 0.00
SpacingAlgorithm 2 4 6 8 9SPEA2 3.65± 1.25 2.09± 1.08 1.10± 0.17 1.37± 0.37 1.70± 0.00NSGA-II 4.00± 0.82 4.59± 0.91 2.77± 0.85 1.97± 0.34 1.80± 0.00MOEA/D 3.23± 1.10 3.16± 0.79 5.21± 0.50 5.69± 0.23 5.30± 0.00NSGA-III 4.35± 1.45 4.02± 1.09 3.53± 0.51 3.13± 0.16 3.20± 0.00ε-MOEA 5.58± 1.04 3.02± 1.49 3.41± 0.69 4.06± 0.37 4.60± 0.00GrEA 5.37± 0.52 6.78± 0.45 7.29± 0.17 7.28± 0.14 6.60± 0.00IBEA 5.73± 1.51 7.43± 0.42 7.60± 0.04 7.63± 0.07 7.80± 0.00HypE 4.09± 0.72 4.91± 0.95 5.11± 0.82 4.88± 0.57 5.00± 0.00
Aurora Ramírez Quesada Ph.D. Thesis 31 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Results
9-objective Pareto fronts
(a) SPEA2 (b) NSGA-II (c) MOEA/D (d) NSGA-III
(e) ε-MOEA (f) GrEA (g) IBEA (h) HypE
Aurora Ramírez Quesada Ph.D. Thesis 32 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Lessons learned
I Not all many-objective algorithms are effective in this context
I Some metrics do not present real conflict, others are highly opposed
I The number and combination of metrics might depend on the system under study
I Practical implications in decision-making: execution time, number of solutions
Aurora Ramírez Quesada Ph.D. Thesis 33 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Associated publications
Main publication:
A. Ramírez, J.R. Romero, S. Ventura. “A comparative study of many-objectiveevolutionary algorithms for the discovery of software architectures”. Empirical
Software Engineering, vol. 21, no. 6, pp. 2546-2600, 2016.
Additional publications:
1 A. Ramírez, J.R. Romero, S.Ventura. “On the Performance of Multiple Objective EvolutionaryAlgorithms for Software Architecture Discovery”. Proceedings of the 16th Genetic and Evolu-tionary Computation Conference (GECCO), pp. 1287-1294, 2014. Best paper of the SBSEtrack and nominated to best paper of GECCO’14.
2 A. Ramírez, J.R. Romero, S.Ventura. “Estudio preliminar del rendimiento de familias de algo-ritmos multiobjetivo en diseño arquitectónico”. Proceedings of the X Congreso Español sobreMetaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB), pp. 173-180, 2015.
3 A. Ramírez, J.R. Romero, S.Ventura. “An Extensible JCLEC-based Solution for the Implemen-tation of Multi-Objective Evolutionary Algorithms”. Proceedings of the Companion Publicationof 17th Genetic and Evolutionary Computation Conference (GECCO Companion), pp. 1085-1092, 2015.
4 A. Ramírez, J.R. Romero, S. Ventura. “On the effect of local search in the multi-objective evolu-tionary discovery of software architectures”. Proceedings of the IEEE Congress on EvolutionaryComputation (IEEE CEC), pp. 2038-2045, 2017.
5 A. Ramírez, J.R. Romero, C. García-Martínez, S.Ventura. “JCLEC-MO: A Java suite for solvingmany-objective optimization engineering problems”. Under review, 2018. (Indexed journal, Q1).
6 A. Ramírez, J.R. Romero, S.Ventura. “A survey of many-objective optimisation in search-basedsoftware engineering”. Under review, 2018. (Indexed journal, Q1).
Aurora Ramírez Quesada Ph.D. Thesis 34 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
The human-in-the-loop approach
Aurora Ramírez Quesada Ph.D. Thesis 35 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Motivation
Software design is a human-centred task
Qualitative aspects are difficult to quantify
The engineer should be further involved in the optimisation process
Sometimes it is easier to identify a bad solution rather than a good one
Aurora Ramírez Quesada Ph.D. Thesis 36 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Research questions
RQ: How can the qualitative judgement of the engineer be inte-grated into the evolutionary discovery of software architectures?
RQ: Does putting the human in the loop involve a significant im-provement compared with not considering him/her along the opti-misation process?
Aurora Ramírez Quesada Ph.D. Thesis 37 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
The algorithm
Steady-state algorithm: two offspringare produced in each generation
Binary tournament selection frompopulation and archive
Offspring replace population memberswith worst fitness
Archive to store a small set ofrepresentative solutions
Aurora Ramírez Quesada Ph.D. Thesis 38 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
The fitness function
A novel fitness function that combines objective and subjective evaluation criteria
fitness(s) = wobj · fobj (s) + wsub · fsub(s)
Quantitative: Maximin function to quantify bothdominance and diversity (software metrics)
fobj (s) =1 + maxz 6=s(mink (f s
k − f zk ))
2∀z ∈ Z
Qualitative: Engineer’s preferences onphenotypic aspects of the solution(architectural preferences)
fsub(s) = 1−1P·
P∑p=1
wp · prefp(s)
Aurora Ramírez Quesada Ph.D. Thesis 39 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Architectural preferences
Preference DescriptionBest component Similarity to the set of classesWorst component Dissimilarity to the set of classesBest provided interface Similarity to the set of operationsWorst provided interface Dissimilarity to the set of operationsNumber of components Distance to the preferred numberMetric in range Distance to the midrangeAspiration levels Weighted distance to the reference point
Aurora Ramírez Quesada Ph.D. Thesis 40 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Interaction mechanism
Interactions are scheduled at regularintervals
Solutions are selected from the currentpopulation using a clustering method
The software engineer rewards orpenalises some aspect of the solutionby choosing one preference
Additional actions:Freeze one component
Add to the archive
Remove from the population
Stop the search
Aurora Ramírez Quesada Ph.D. Thesis 41 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Empirical study
MethodologyCase study, 9 participants
3 interactions, 3 solutions each
Log files and questionnaires
●
●
●
●
● ● ●
●●
00
01
02
03
04
05
i1s1 i1s2 i1s3 i2s1 i2s2 i2s3 i3s1 i3s2 i3s3Step of interaction
Eva
luat
ion
time
(min
utes
)
Impact of interactionCohesion is indirectly improved
Different number of components
Components similar or equal tothe manual design
Actions (add, freeze) contribute tofind better solutions 0
10
20
30
40
50
60
70
80
2 3 4 5 6Number of components
Num
ber
of s
olut
ions
Without interaction With interaction
Aurora Ramírez Quesada Ph.D. Thesis 42 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Empirical study
Analysis of architectural preferencesMore interest in the internal structure
Negative preferences are morefrequently applied at the beginning
Use of “No preference”0
5
10
15
20
25
Freq
uenc
y
No prefere
nce
Best c
omponent
Best in
terfac
e
Worst co
mponent
Worst in
terfac
e
No. Componen
ts
Measu
re in ra
nge
Aspira
tion le
vels
Architectural Selected Usefulness IntuitivenessPreference (%) [1-8] [1-8]No preference 22.22 6.44 7.33Best component 29.63 7.44 7.44Worst component 23.46 7.22 7.33Best provided interface 2.47 5.29 6.38Worst provided interface 0.00 4.71 6.38Number of components 17.28 7.50 7.33Metric in range 2.47 4.17 5.44Aspiration levels 2.47 5.80 5.22
Aurora Ramírez Quesada Ph.D. Thesis 43 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Lessons learned
I Architecture preferences as a novel mechanism for subjective evaluation
I Design metrics are still needed at the beginning
I Negative opinions are useful too
I Humans are willing to participate (even more) in the optimisation
Aurora Ramírez Quesada Ph.D. Thesis 44 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Associated publications
Main publication:
A. Ramírez, J.R. Romero, and S. Ventura. “Interactive multi-objective optimization ofsoftware architectures”. Information Sciences, vol. 463-464, pp. 92-109, 2018.
Additional publications:
1 A. Ramírez, J.R. Romero, S.Ventura. “Interactividad en el descubrimiento evolutivo de arquitec-turas”. Proceedings of the XX Jornadas en Ingeniería del Software y Bases de Datos (JISBD),2015.
2 A. Ramírez, R. Barbudo, J.R. Romero, S. Ventura. “Herramienta basada en computación evo-lutiva interactiva para arquitectos software”. Proceedings of the XI Congreso Español sobreMetaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB), pp. 387-396, 2016.
3 A. Ramírez, J.R. Romero, S. Ventura. “Búsqueda coevolutiva interactiva aplicada al diseñode software”. Proceedings of the XXII Jornadas en Ingeniería del Software y Bases de Datos(JISBD), 2017.
4 A. Ramírez, J.R. Romero, S. Ventura.“API para el desarrollo de algoritmos interactivos en inge-niería del software basada en búsqueda”. XXIII Jornadas en Ingeniería del Software y Basesde Datos (JISBD), 2018.
5 A. Ramírez, J.R. Romero, C. Simons. “A Systematic Review of Interaction in Search-BasedSoftware Engineering”. IEEE Transactions on Software Engineering, pp. 1-22. 2018. In press.
Aurora Ramírez Quesada Ph.D. Thesis 45 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Conclusions and future work
Aurora Ramírez Quesada Ph.D. Thesis 46 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Concluding remarks
1 Definition of software architecture discovery as an optimisation problem
2 Design of a comprehensible evolutionary model to discover architectures
3 Analysis of the influence of software metrics in a many-objective space
4 Exploration of hybrid approaches to improve the performance
5 Development of a general interactive model to integrate human expertise
Literature Problem Algorithm Experimental Modelanalysis formulation implementation validation integration
O1TSE
UnderReview1O2 JISBD’14 INFSCI
O3
GECCO’14, MAEB’15, EMSE GECCO’15JISBD’16, ISDA’16, CEC’17 UnderReview2
JISBD’16 ESwA
O4
JISBD’15INFSCI MAEB’16
JISBD’17 JISBD’18
Overview of publications derived from the Ph.D. Thesis
Aurora Ramírez Quesada Ph.D. Thesis 47 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Future lines of research
1 Extensions to the problem formulationDesign constraints
Evaluation and semantic aspects
2 Advanced interactive modelsCooperative coevolution + interaction
Learning from user interaction
3 New optimisation problems in SBSDService-oriented architectures
Cloud computing
4 Industrial acceptanceIntegrated process
Software tools
Aurora Ramírez Quesada Ph.D. Thesis 48 / 50
Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions
Thesis quality indicators
Compendium of publications
A minimum of three articles publishedin journals ranked in the
Journal Citation Report (Q1)
International mention
Research stay (three months) in a foreigninstitution: University of the West ofEngland, under the supervision of
Dr. Christopher L. Simons
Positive evaluation reports by tworesearchers from a different institution
Additional international publications
Aurora Ramírez Quesada Ph.D. Thesis 49 / 50
Metaheuristic models for decision support in the softwareconstruction process
– Ph.D. Thesis –
Aurora Ramírez Quesada
Dept. Computer Science and Numerical AnalysisUniversity of Córdoba
Supervisors:Dr. José Raúl Romero Salguero
Dr. Sebastián Ventura Soto
24th September 2018
Mutation operatorAlgorithm 1 Add a component
Require:
Ensure:
1: �
2: for all component in do
3: candidates�
4: if (random(0,1)> Prob(component)) then
5: allGroups� getGroups(component)
6: if (size(groupsOf Classes) > 1) t hen
7: for all groupOf Classes in allGroups do
8: if (size(groupOf Classes) = min) then
9: � groupOf Classes
10: end if
11: end for
12: end if
13: end if
14: newComp � randomGroup( )
15: � component �
16: �
17: end for
18: if (newComp = ) then
19: �
20: for all component in do
21: for all class in component do
22: if (random(0,1) > 0.5) then
23: � class
24: end if
25: end for
26: � component �
27: newComp �
28: �
29: end for
30: end if
31: � newComp
32: )
33: return
offspring
offspring
offspring
offspring
offspring
offspring
candidates
candidates
candidates
candidates
candidates
candidates
candidates
candidates
offspring
parent
parent
parentoffspring
setInterfacesAndConnectors(
Algorithm 2 Split a component
Require:
Ensure:
1: �
2: �
3: for all component in do
4: if (numberOfGroups(component) > 1) then
5: � component
6: end if
7: end for
8: if (size( ) > 0) then
9: � randomComponent( )
10: for all in do
11: if (random(0,1) > 0.5) then
12: component1�
13: else
14: component2�
15: end if
16: end for
17: else
18: � randomComponent( )
19: for all class in do
20: if (random(0,1) > 0.5) then
21: component1� class
22: else
23: component2� class
24: end if
25: end for
26: end if
27: � �
28: � component1
29: � component2
30: )
31: return
parent
parent
parent
parent
offspring
offspring
offspring
offspringoffspring
offspring
offspring
groupOfClasses
groupOfClasses
groupOfClasses compToSplit
compToSplit
compToSplit
candidates
candidates
candidates
candidatescompToSplit
compToSplit
setInterfacesAndConnectors(
Aurora Ramírez Quesada Ph.D. Thesis 1 / 7
Mutation operator
Algorithm 3 Remove a component
Require:
Ensure:
1: �
2: �
3: maxRel � maxNumExtRel( )
4: for all component in do
5: if (numExtRel(component)=maxRel) then
6: � component
7: end if
8: end for
9: compToRemove � randomComponent( )
10: � � compToRemove11: for all class in compToRemove do
12: randomComponent( )� class
13: end for
14: )
15: return
parent
parent
offspring
offspring
parent
offspring
offspring
offspring
candidates
candidates
candidates
parent
offspring
setInterfacesAndConnectors(
Algorithm 5 Move a class
Require:
Ensure:
1: �
2: � randomComponent( )
3: destination� randomComponent( )
4: � randomClass( )
5: removeClass( , )
6: addClass( , )
7: � �
8: �
9: �
10: )11: return
offspring
offspring
offspringoffspring
offspringoffspring
offspring
parent
parent
parent
parent
origin
origin
origin
(origin
origin
destinationdestination)
destination
class
class
class
setInterfacesAndConnectors(
∪
Algorithm 4 Merge two components
Require:
Ensure:
1: �
2: �
3: maxRel � maxNumExtRel( )
4: for all component in do
5: if (numExtRel(component)=maxRel) then
6: � component
7: end if
8: end for
9: component1� randomComponent( )
10: if (size( ) > 1) then
11: component2� randomComponent( )
12: else
13: component2� randomComponent( )
14: end if
15: � � com ponent1 component2
16: � component1∪component2
17: )
18: return
offspring
offspring
offspringoffspring
offspring
offspring
parent
parent
parent
parent
parent
candidates
candidates
candidates
candidates
candidates
( ∪ )
setInterfacesAndConnectors(
Aurora Ramírez Quesada Ph.D. Thesis 2 / 7
Mutation operator
Initialindividual
Architecture
Components Connectors
Component_1 Component_2 Connector_1
Classes ClassesProvided
interfaces
Required
interfaces
Provided
interfaces
Required
interfaces
Provided
interface
Required
interface
A_prov_E
E_req_A A_prov_EE_req_AA C
DE
GF
H
B
Provided
interface
Required
interface
Connector_2
D_prov_BB_req_D
D_prov_B B_req_D
Component_1 Component_2
A_prov_E
E_req_A
D_prov_B
B_req_D
Add acomponent
Architecture
Components Connectors
Component_1
Component_2
Connector_1
Classes
Classes
Provided
interfaces
Required
interfaces
Provided
interfaces
Required
interfacesProvided
interface
Required
interface
A_prov_E
E_req_A
A_prov_EE_req_AA
C D
E
G
F
H
B
Provided
interface
Required
interface
Connector_2
D_prov_BB_req_DD_prov_B
B_req_D
Component_3
Classes
Provided
interfaces
Required
interfaces
B F
B_req_D
A_prov_E
E_req_A
D_prov_B
B_req_D
Component_3
Component_1
Component_2
Move aclass
Architecture
Components Connectors
Component_1 Component_2 Connector_1
Classes ClassesProvided
interfaces
Required
interfaces
Provided
interfaces
Required
interfacesProvided
interface
Required
interface
A_prov_E
E_req_A A_prov_EE_req_AA
C
E
G
F H
B
Provided
interface
Required
interface
Connector_2
D_prov_BB_req_D
D_prov_B B_req_DB
D
Component_1 Component_2
A_prov_E
E_req_A
Aurora Ramírez Quesada Ph.D. Thesis 3 / 7
Objective functions
Instability
INSi =ECi
ECi + ACiINS =
1n·
n∑i=1
INSi
Encapsulation
ENCi =#classesinner
#classestotalENC =
1n·
n∑i=1
ENCi
Abstractness
ABSi =#classesabstract
#classestotalABS =
1n·
n∑i=1
ABSi
Aurora Ramírez Quesada Ph.D. Thesis 4 / 7
Objective functions
Critical size
CCsizei =
{1 if size(i) > threshold0 otherwise
CS =n∑
i=1
CCsizei
Critical link
CC linki =
{1 if #provided interfacesi > threshold0 otherwise
CL =n∑
i=1
CC linki
Component balance
SB(n) =
n−1µ−1 if n < µ
1− n−µω−µ if µ < n < ω
0 if n ≥ ω
CSU(C) = 1− Gini({volume(c) : c ∈ C})CB = SB(|C|) · CSU(C)
Aurora Ramírez Quesada Ph.D. Thesis 5 / 7
Architectural preferences
Best component
prefbc = max{J(classes(c), classes(c+))} ∀c ∈ [1, n] J(A,B) = |A ∩ B|/|A ∪ B|
Worst component
prefwc = max{1− J(classes(c), classes(c−))} ∀c ∈ [1, n]
Best provided interface
prefbi = max{J(operations(interface(c)), operations(p+))} ∀c ∈ [1, n]
Worst provided interface
prefwi = max{1− J(operations(interface(c)), operations(p−))} ∀c ∈ [1, n]
Aurora Ramírez Quesada Ph.D. Thesis 6 / 7
Architectural preferences
Number of components
prefnc =
{(n − nmin)/(n+ − nmin) if n < n+
1− ((n − n+)/(nmax − n)) if n ≥ n+
Metric in a range
prefmr =
0 if ms < mmin
1− (ms −mmid )/mmid if ms ∈ [mmin,mmax ]
0 if ms > mmax
mmid = (mmax −mmin)/2
Aspiration levels
prefal =
{1 if ASF ≤ 01− ASF if ASF > 0
ASF = max{wk · (f sk − z∗k )}
Aurora Ramírez Quesada Ph.D. Thesis 7 / 7