expert mining compsac-2014
DESCRIPTION
To take maximum advantage of open source software (OSS), the understanding, management and mitigation of OSS adoption risks is crucial. We describe the empirical application of the tactical workshops with the purpose of obtaining the domain expert evaluation.TRANSCRIPT
![Page 1: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/1.jpg)
Expert Mining for Evaluating Risk Indicators Scenarios
Oscar Franco-Bedoya, Dolors Costal, Soraya Hidalgo, Ron Ben-Jacob
Monday 21st July 2014
![Page 2: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/2.jpg)
2
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
Outline
![Page 3: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/3.jpg)
3
Bac
kgro
un
d
![Page 4: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/4.jpg)
4
Bac
kgro
un
d
![Page 5: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/5.jpg)
5
Bac
kgro
un
d
Managing risk in
open source adoption
![Page 6: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/6.jpg)
6
Identification
Mitigation
methods
Management
Advanced tools
Provides
Platform
Methods
OSS
adoption
projects To supporta
In
Uses
Ecosystem
modelingStatistical
tools
Risk
Management
i.e. i.e.i.e.
Bayesian
Networks
Social
network
analysis
Expert
scenarios
assessment
e.g. e.g. e.g.
Bac
kgro
un
d
IdentificationRisks
Management
Systematic
protocoluses
![Page 7: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/7.jpg)
7
Bac
kgro
un
d
Project Site
Code Version Repository
Bug Tracker IRC Mailing List
Ecosystem hubs
Project indexes
Social Media
Twitter Facebook
I
Raw
DataSNA
Measures
Risk Indicators:• Project
• Community
• ContextualII
IndicatorsScenario-based
Assessment DomainExpert
Business Analysis• Business goals
III
Business
Goals
3-Layered
RISCOSS approach
Number of
downloads
Number of
event
referencesCentrality
Number of
open bugs
![Page 8: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/8.jpg)
8
Ou
tlin
e
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
Outline
![Page 9: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/9.jpg)
9
The RISCOSS tactical workshop is designed to permit
experts to assess risk indicators
Wo
rksh
op
pro
ced
ure
![Page 10: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/10.jpg)
10
Tactical workshop
protocol
Wo
rksh
op
pro
ced
ure
Pre-Tasks
Part II: Community dinamics
Part I : Community data
TacticalWorkshop Organizer
DomainExpert
6.Make presentation of RISCOSS
project summary.
7.Explain the RISCOSS analytics platform.
8.Explain the tactical
workshop Part I and
Part II
9.Explain the risk driver
selection WS Excel File
Excel File: risk driver selection WS
TabsTimelinessActivenessCommunity
10.Study the use case scenario
11.Assess the use case scenario
overall state
12.Determine the level of
the risk indicator
13.Study the use case scenario
14.Assess the use case scenario
overall state
15.Determine the level of
the community
risk indicator
PDF File:KPA RISCOSS Analytics
[more risk drivers]
[not more risk drivers]
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
16.Send the scenarios
judgement
Populate BN nodes
![Page 11: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/11.jpg)
11
Pre-Tasks
TacticalWorkshop Organizer
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
Correspond to
tasks that must
be done before
conducting the
tactical workshop
Wo
rksh
op
pro
ced
ure
![Page 12: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/12.jpg)
12
Pre-Tasks
TacticalWorkshop Organizer
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
Wo
rksh
op
pro
ced
ure
Risk Driver
Forum posts per day
Forum messages per thread
Mail per day
Overall community size
Number of developers involved
Number of testers (individuals
providing feedback)
Number of companies using the
software
Companies supporting the project
(adding to code)
Risk Indicator Activeness
![Page 13: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/13.jpg)
13
Pre-Tasks
TacticalWorkshop Organizer
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
Wo
rksh
op
pro
ced
ure
![Page 14: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/14.jpg)
14
Pre-Tasks
TacticalWorkshop Organizer
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
Wo
rksh
op
pro
ced
ure
![Page 15: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/15.jpg)
15
Pre-Tasks
TacticalWorkshop Organizer
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
Wo
rksh
op
pro
ced
ure
![Page 16: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/16.jpg)
16
The tactical
workshops begin
with an exposition
about the main
topics that will be
covered
Wo
rksh
op
pro
ced
ure
6.Make presentation of RISCOSS
project summary.
7.Explain the RISCOSS analytics platform.
8.Explain the tactical
workshop Part I and
Part II
PDF File:KPA RISCOSS Analytics
TacticalWorkshop Organizer
![Page 17: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/17.jpg)
17
Wo
rksh
op
pro
ced
ure
9.Explain the risk driver
selection WS Excel File
Excel File: risk driver selection WS
TabsTimelinessActivenessCommunity
TacticalWorkshop Organizer
![Page 18: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/18.jpg)
18
Wo
rksh
op
pro
ced
ure
Part I : Community data
DomainExpert
10.Study the use case scenario
11.Assess the use case scenario
overall state
12.Determine the level of
the risk indicator
![Page 19: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/19.jpg)
19
Wo
rksh
op
pro
ced
ure
Part II: Community dinamics
13.Study the use case scenario
14.Assess the use case scenario
overall state
15.Determine the level of
the community
risk indicator
16.Send the scenarios
judgement
DomainExpert
![Page 20: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/20.jpg)
20
Ou
tlin
e
Outline
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
![Page 21: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/21.jpg)
21
Ge
ne
raliz
atio
n o
f th
e a
pp
roac
h
![Page 22: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/22.jpg)
22
Ou
tlin
e
Outline
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
![Page 23: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/23.jpg)
23
Applications and lessons learnedA
pp
licat
ion
s an
d le
sso
ns
lear
ne
d
We have conducted 10 technical workshops In private and official organizations and academic institutions.
The experts were from different countries.
France, Israel, Italy, Spain, and Netherlands
There are some inconsistencies in the scenarios The scenarios were designed using random number generators
While the domain experts are conducting the tactical workshops, The degree of "calibration" of their judgement improves
![Page 24: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/24.jpg)
24
Ou
tlin
e
Outline
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
![Page 25: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/25.jpg)
25
Re
late
d W
ork
Related Work
Delphi method
(QUELCE)
Quantifying Uncertainty
in Early Cost
Estimation[2]
Reliable consensus of
opinion of a group of
experts [1]
[1]N. Dalkey and O. Helmer, “An experimental application of the Delphi method to the use of experts,”.
[2] R. W. Ferguson, D. Goldenson, J. M. McCurley, R. W. Stoddard, and D. Zubrow, “Quantifying
Uncertainty in Early Lifecycle Cost Estimation ( QUELCE ),”
![Page 26: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/26.jpg)
26
Ou
tlin
e
Outline
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
![Page 27: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/27.jpg)
27
DomainExpert
Conclusions and further workC
on
clu
sio
ns
and
fu
rth
er
wo
rk Pre-Tasks
Part II: Community dinamics
Part I : Community data
TacticalWorkshop Organizer
DomainExpert
6.Make presentation of RISCOSS
project summary.
7.Explain the RISCOSS analytics platform.
8.Explain the tactical
workshop Part I and
Part II
9.Explain the risk driver
selection WS Excel File
Excel File: risk driver selection WS
TabsTimelinessActivenessCommunity
10.Study the use case scenario
11.Assess the use case scenario
overall state
12.Determine the level of
the risk indicator
13.Study the use case scenario
14.Assess the use case scenario
overall state
15.Determine the level of
the community
risk indicator
PDF File:KPA RISCOSS Analytics
[more risk drivers]
[not more risk drivers]
RISCOSS Analytics
Team
1.Determine drivers and
risk indicators
2.Construct Bayesian networks
3.Define scenarios
4.Identify and contact experts
5.Workshops planning and preparation
16.Send the scenarios
judgement
Populate BN nodes
DomainExpert SNA
Project Site
Code Version Repository
Bug Tracker IRC Mailing List
Step-by-step
protocol
Data used to
construct
Bayesian
networks
Future work
Combines opinion
of domain experts
with OSS raw data
Empirical
application
&
lessons
![Page 28: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/28.jpg)
28
Ou
tlin
e
Outline
Background
Workshop procedure
Generalization of the approach
Applications and lessons
learned
Related work
Conclusions and further
work
![Page 29: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/29.jpg)
29
SEe
D4
FI
![Page 30: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/30.jpg)
Thanks for your attention
Comments and Questions
![Page 31: Expert mining compsac-2014](https://reader033.vdocument.in/reader033/viewer/2022060115/557cf92bd8b42a98158b4b55/html5/thumbnails/31.jpg)
31
SEe
D4
FI