software process simulation modeling - kaistse.kaist.ac.kr › ... ›...
TRANSCRIPT
Software Process Software Process Simulation Modelingg
HyeonJeong Kim2009. 2. 252009. 2. 25
ⓒ KAIST SE LAB 2009
Content
Part I: Definition of Software Process Simulation M d li (SPSM)Modeling (SPSM)Part II: Overall procedure to build SPSMPart III: Research issues
2/27ⓒ KAIST SE LAB 2009
Definition of SPSM(1/3)( / )
Software ProcessA set of procedures and methods
• Define the relationship of tasks[1]
Al f l h l i d[2]Also a part of process, people, technology triad[2]
Process
Task, activities
P l TechnologyPeople Technology
People with skills, training, and motivation
Tool and equipment
3/27ⓒ KAIST SE LAB 2009
Definition of SPSM(2/3)( / )
Simulation modelingA method to specify real system at some abstraction level
<Real world> <Simulation model>Mapping
System uncertainty
Characteristics
Stochastic simulation
Corresponding concept
Change over time
Learning mechanism
Dynamic behavior
Feedback loop
4/27ⓒ KAIST SE LAB 2009
Definition of SPSM(3/3)( / )
Software Process Simulation model (SPSM)System specification describes how the process works
• SPSM should contain all three elements of triad
Process
<Project environments
T h l
<Project environments in SPSM>
People Technology
5/27ⓒ KAIST SE LAB 2009
Overall procedures to build SPSMp
D fi th bj ti f i l tiDefine the objectives of simulation
Gather model elements
Requirements analysis
Deriving qualitative information Formal modelingDesign
Implement the simulatorImplementation
Verification
ValidationTesting(V&V) Calibration
Collect the data from simulatorAnalysis
6/27ⓒ KAIST SE LAB 2009
Analyze the collected datay
Define the objectives(1/2)Requirements analysis
j ( / )
Determine why the simulator is neededMain categories of the objectives in SPSM[3]
• Most simulators are made for supporting the manager’s decision makingdecision making
Strategic managementP ss imp m nt nd
Planning
Process improvement and technology adoption
Control and operational management
7/27ⓒ KAIST SE LAB 2009
Define the objectives(2/2)Requirements analysis
j ( / )
Objectives Strategic Planning Control & SPI & tech. j gmanagement
goperational management
adoption
Area that Deciding on the Predicting/forec Project tracking Forecasting the objective deals with
organizational strategies or policies
asting some project parameters across time
and oversight on software project during enactment
impact of a potential process(tech) change before across time enactment change before enactment
Example • Decide development
• Predicting staffing levels
•Predicting which activity
•Analyzingimpacts of the development
site (cross sites vs. centralized at one site)
D id t ff
staffing levels across time and periodical re-planning
which activity threats product quality
impacts of the inspection on product quality•Analyzingi t f • Decide staff
strategies (in-house vs. outsource)
impacts of automation of test design on effortoutsourc ) ffort
8/27ⓒ KAIST SE LAB 2009
Gather the model elements(1/2)Requirements analysis
( / )
Model elementsIdentified information elements
• To be needed for the defined objectives in simulation
Simulation model
Input Output
Model abstractionProcessp
parametersp
Variables
People Technology
Model scope
9/27ⓒ KAIST SE LAB 2009
Gather the model elements(2/2)Requirements analysis
( / )
Categories Model scope Result variables Model Input g pabstraction
pparameters
Definition Boundary of the simulation model
Information elements needed
Key elements of software
Information elements
in organizational view and time-span
to answer the key questions related to the objectives
process and project environments
related to result variables and model abstractionsobjectives abstractions
Example •Organizationalview- less than one
•Effort/cost•Schedule•Quality(# of
•Key process components and their level
•Software size(KSLOC)•Defect less than one,
one, multiple projects(teams)
T
Quality(# of defects)•Productivity•ROI(Return on
)
their level•Sequence of the activities•Available
( ff
Defect detection efficiency•Defect removal
d • Time-span- short,
medium, long
Investment)•Staff utilization rate (staffing needs)
resources(staff, hardware) •Iteration loops
and injection rates•Requirements creeping pattern
10/27
(staffing needs) creeping pattern•Resource constraints
ⓒ KAIST SE LAB 2009
Designing the simulation modelDesign
g g
What to do in designDeriving the qualitative information
• Define a cause-effect relationship from input to result variables
F l d li f h li i i f iFormal modeling for the qualitative information
<Deriving qualitative info.> <Formal modeling>g q g
a
b
Informal description Formal description
“Roof size depends on size of dog house”
Roof size a = 1.2 x dog house size b
11/27ⓒ KAIST SE LAB 2009
Deriving the qualitative information(1/2)
Design
Qualitative informationContain a cause-effect relationship between parameters
It d ib th t i b h i f l t ( )• It describes the atomic behavior of real system(process)
Causal diagram (cause-effect diagram)[4]
• Useful diagram to describe qualitative information• Useful diagram to describe qualitative information
Notation Meaning Example
If X increases (decreases), then Y increases(decreases)X Y
+Effort Schedule
+
If X increases (decreases), then Y d (i )X Y
-Productivity Effort
-
12/27ⓒ KAIST SE LAB 2009
decreases (increases)X Y y
Cause-effect relationshipParameters
Deriving the qualitative information(2/2)
Design
Example of a causal diagram[4]
Time remaining
Schedule –Schedule pressure
Work to be doneOvertimeTime per task
++–
Feedback loop
Completion rateFatigue
Error +
+
+–p
Productivity
Error prevention rate
+
–
––
+
• Useful to identify the important dependencies for entire model• Feedback loopFeedback loop
– Represent learning mechanism from previous value13/27ⓒ KAIST SE LAB 2009
Formal modelingDesign
g
Simulation model needs a formal description of t b h isystem behavior
Mathematical representation is required • Based on observation of real projects or literatures
Qualitative information Formal description
Effort Schedule
+
-Productivity Effort
14/27ⓒ KAIST SE LAB 2009
Implement a simulatorImplementation
p
SimulatorComputer program to interpret the simulation model’s instruction
Main simulation methods to build simulator for ftsoftware processSystem DynamicsDiscrete Event-based SimulationHybrid simulation
15/27ⓒ KAIST SE LAB 2009
Method to build simulator(1/5)Implementation
( / )
System dynamics(SD)A method to enhance understanding dependencies between factors in complex system[4][5]
E lExample
• Project environments which change dynamically and continuously over time and their changes affect other variablescontinuously over time and their changes affect other variables
– In SD, all the parameters are automatically re-calculated on every single time unit
16/27ⓒ KAIST SE LAB 2009
Method to build simulator(2/5)Implementation
( / )
System dynamics (Cont’d)Result of simulator depends on single time unit
Schedule pressurepressure
Calculation Total time Accuracy<Simulator performance>
Calculation # in one run
Total time Simulation
Accuracy
8 times Long High4 times Short Low
Time
Short time unit
4 times Short LowLong time unit
Time-unit
17/27ⓒ KAIST SE LAB 2009
Method to build simulator(3/5)Implementation
( / )
Discrete Event-based Simulation(DES)A method to enhance understanding interactions between components of complex system[6]
St t f ft h d b• States of software process are changed by– Start/end of an activity or reception/release of an artifact
» Irrelevant to time » Events represent interactions between process activities
Analysis Design…
Events
18/27ⓒ KAIST SE LAB 2009
Method to build simulator(4/5)Implementation
( / )
SD vs. DES for software process
SD(System Dynamics) DES(Discrete Event-based Simulation)
Advantages-Dependency bet. parameters-Feedback loops
-Easily describe discrete process steps -Attributes of each
titFeedback loops process entity
Disadvantages-Difficult to model discrete process steps, and -Difficult to make Disadvantages p p ,attributes of each process entity
feedback loops
Project environments Process based on
Provide a hybrid simulation method to combine SD
Project environments based on feedback loop
Process based on events
19/27ⓒ KAIST SE LAB 2009
Provide a hybrid simulation method to combine SD and DES
Method to build simulator(5/5)Implementation
( / )
Hybrid simulation methodCombine DES and SD
• DES is responsible for describing software processSD i ibl f d ibi f db k l• SD is responsible for describing feedback loop
Example• Choi’s work[7]• Choi s work[7]
– Combine SD into DES framework
20/27ⓒ KAIST SE LAB 2009
Verification and validationTesting
Real project SimulatorSimulation
model
Validation Verification
project model
Validation Verification
SimulatorSimulation
model spec CompareSimilar
result?
Calibration
Implementation ResultCalibrationInputs
Similar
Real project
Simulationmodel
Simulator Produce
Similar result?
Calibration
21/27ⓒ KAIST SE LAB 2009
Result analysis(1/2)Analysis
y ( / )
Sensitivity analysisPurpose
• Determining which input parameters have a significant impact on the result[8]on the result[8]
– Identified main parameters should be carefully treated in management
Method• For each input parameter• For each input parameter
– Its minimum and maximum value are simulated under setting other parameters to have a fixed value
22/27ⓒ KAIST SE LAB 2009
Result analysis(2/2)Analysis
y ( / )
Sensitivity analysis(Cont’d)Example
• Effect of input parameters of GSD* on duration to complete software project[9]software project[9]
Dur
atio
n
T 3 i ifi l d i f GSD% overlap Distance Culture Language Distribution Familiarity Meeting
D
– Top 3 significant elements on duration for GSD» Overlapping working hour» Distribution overhead» Distance
23/27ⓒ KAIST SE LAB 2009
*GSD: Global Software Development
Research issues(1/2)( / )
Define the objectives of simulation Developing new simulator and Define the objectives of simulation
Gather model requirements and analyze them
panalyzing the results
•For developing a general
Build qualitative model
Collect quantitative information
F p g gsimulation model- ‘plug and play’ model components
Implement the simulator
p•For new developmentprocess
- GSD, business process i h SW Verification
ValidationCalibration
with SW process, …•For deciding managerialpolicies
Req Creeping
Collect the data from simulator
- Req. Creeping•Re-planning- Resource allocation
24/27ⓒ KAIST SE LAB 2009
Analyze the collected data
Research issues(2/2)( / )
Define the objectives of simulationDefine the objectives of simulation
Gather model requirements and analyze them
Build qualitative model
Collect quantitative information
Implement the simulatorV&V for SPSM with limited data
h d Verification
ValidationCalibration
•With minimum data sets, provide the useful, proven SPSM
Collect the data from simulator Focus on analyzing the results•Combine with statistics in
25/27ⓒ KAIST SE LAB 2009
Analyze the collected data stochastic simulation model
References[1] M.Paulk, et al., “Key practices of the capability maturity model”, Version 1.1,
Technical Report CMU/SEI-93-TR-25. SEI, CMU, Pittsburgh, PAp , , g ,[2] M. Chrissis, et al., “CMMI: Guildlines for Process Integration and Product
Improvement”, Addison-Wesley, 2003[3] M Kellner R Madachy and D Raffo “Software process simulation modeling:[3] M. Kellner, R. Madachy, and D. Raffo, Software process simulation modeling:
Why? What? How?”, JSS, Vol. 46, No. 2/3, 1999[4] J. Sterman, “Business Dynamics”, McGrawHill, 2004[5] P j t D i A I t t d A h” P ti H ll 1991T Abd l[5] Project Dynamics: An Integrated Approach”, Prentice Hall, 1991T. Abdel-
Hamid and S. Madnick, “Software [6] B. Zeigler, H. Pracehofer, and T. Kim, “Theory of Modeling and Simulation”,
A d i P 2000Academic Press, 2000[7] K. Choi, D. Bae, and T. Kim, “An Approach to a Hybrid Software Process
Simulation Formalism”, SPIP, 2006[8] A. Law, “Simulation Modeling & Analysis”, McGrawHill, 2007[9] S. Setamanit, W. Wakeland,and D. Raffo, “Using simulation to evaluate global
software development task allocation strategies”, SPIP,2007p g
26/27ⓒ KAIST SE LAB 2009
ⓒ KAIST SE LAB 2009