towards understanding how developers spend their effort ... · how developers spend their e ort z...
TRANSCRIPT
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Towards Understanding How DevelopersSpend their Effort during
Maintenance Activities
Zephyrin Soh, Foutse Khomh, Yann-Gael Gueheneuc,Giuliano Antoniol
Department of Computer and Software EngineeringEcole Polytechnique de Montreal, Quebec, Canada
October 16, 2013
Pattern Trace Identification, Detection, and Enhancement in JavaSOftware Cost-effective Change and Evolution Research Lab
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Outline
Introduction
Effort vs. Complexity
Factors Affecting Effort
Conclusion
2 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionContext and Example (1/2)
3 / 15
Eclipse bug #188083
Patch #74156
File: 2LOC : 26 + 18 LOC - 8 LOC
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionContext and Example (1/2)
3 / 15
Eclipse bug #134884
Patch #94002
File: 2LOC : 20 + 19 LOC - 1 LOC
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionContext and Example (1/2)
3 / 15
Complexity of the Changes
I Which change is more complex?
vs.
Eclipse bug #134884
Patch #94002
File: 2
LOC : 20
+ 19 LOC
- 1 LOC
Eclipse bug #188083
Patch #74156
File: 2
LOC : 26
+ 18 LOC
- 8 LOC
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionContext and Example (2/2)
4 / 15
Effort Needed to Provide a Patch
Spend a certain effort to:
I Explore the program
I Find relevant program entities
I Understand entities and make changes
1. How to estimate the effort spend to provide a patch?
2. Does a complex patch need more effort?
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionContext and Example (2/2)
4 / 15
Effort Needed to Provide a Patch
Spend a certain effort to:
I Explore the program
I Find relevant program entities
I Understand entities and make changes
1. How to estimate the effort spend to provide a patch?
2. Does a complex patch need more effort?
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
IntroductionData
Need detailed information
I Developers’ programming activities
Interactions histories⇒ developers’ effort
I Changes made to address the tasksPatches⇒ source code before and afterchanges
5 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityResearch Question
Does the complexity of the implementation of atask reflect developers effort?
6 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMetrics
Developers’ effort
I Time Spend: Total duration spent on all files and theircontents
I Cyclomatic complexity: Cyclomatic complexity of theexploration graph
Complexity of the changes
I Entropy: How much the changes are scattered betweenfiles [1]
I Change distance: How much difference between thesource code before the changes and source code after.
[1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE2009
7 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMetrics
Developers’ effort
I Time Spend: Total duration spent on all files and theircontents
I Cyclomatic complexity: Cyclomatic complexity of theexploration graph
Complexity of the changes
I Entropy: How much the changes are scattered betweenfiles [1]
I Change distance: How much difference between thesource code before the changes and source code after.
[1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE2009
7 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (1/2)
How do we match interactions and patches?
2,408 Interactions histories 3,395 Patches?
Assumption: An interaction is matched to a patch (i.e., thepatch is the result of the corresponding interaction) if andonly if both are attached to the same bug report, by thesame developer at the same date (date/hour/minutes).
8 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (1/2)
How do we match interactions and patches?
2,408 Interactions histories 3,395 Patches?
Assumption: An interaction is matched to a patch (i.e., thepatch is the result of the corresponding interaction) if andonly if both are attached to the same bug report, by thesame developer at the same date (date/hour/minutes).
8 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (2/2)
9 / 15
Unbalanced matchings
Developers modify files without interacting with them:
I Changes not requiring much effort, e.g., propagation ofrefactoring
I Interactions are not collected when performing the task
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (2/2)
9 / 15
Unbalanced matchings
Developers modify files without interacting with them:
I Changes not requiring much effort, e.g., propagation ofrefactoring
I Interactions are not collected when performing the task
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (2/2)
9 / 15
Unbalanced matchings
Developers modify files without interacting with them:
I Changes not requiring much effort, e.g., propagation ofrefactoring
I Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (2/2)
9 / 15
Unbalanced matchings
Developers modify files without interacting with them:
I Changes not requiring much effort, e.g., propagation ofrefactoring
I Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
F7
F6
F5
F9
F2
F3
F1
F4F8
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityMatching (2/2)
9 / 15
Unbalanced matchings
Developers modify files without interacting with them:
I Changes not requiring much effort, e.g., propagation ofrefactoring
I Interactions are not collected when performing the task
F6F4
F5F3
F2
F1
F7
F3
F2
F1
F8
F7
F6
F5
F9
F2
F3
F1
F4F8
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityResults
10 / 15
Effort vs. complexity of the changes
I 1028 matchings and 217 unbalanced matchings
I Developers do not necessary spend more effort on tasksrequiring more complex changes
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Effort vs. ComplexityResults
10 / 15
Effort vs. complexity of the changes
I 1028 matchings and 217 unbalanced matchings
I Developers do not necessary spend more effort on tasksrequiring more complex changes
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortAdditional Files
11 / 15
Additional Files
I Exploring files that should not be modified
I Significantly relevant files vs. additional (useful andaccidental) files
I Effort vs. number of additional files: 0.63 (time) and0.82 (cyclomatic complexity)
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortAdditional Files
11 / 15
Additional Files
I Exploring files that should not be modified
I Significantly relevant files vs. additional (useful andaccidental) files
I Effort vs. number of additional files: 0.63 (time) and0.82 (cyclomatic complexity)
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortAdditional Files
11 / 15
Additional Files
I Exploring files that should not be modified
I Significantly relevant files vs. additional (useful andaccidental) files
I Effort vs. number of additional files: 0.63 (time) and0.82 (cyclomatic complexity)
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortBug Severity
12 / 15
Bug severity
I Bug severity indicates how much a bug can affect theperformance and stability of the system [2]
I The resolution time of severe bugs is greater than theresolution time of less severe bugs [3]
I Developers may spent more effort when fixing severebugs wrt. less severe bugs
[2] Lamkanfi et al., Predicting the severity of a reported bug, MSR 2010[3] Panjer. Predicting eclipse bug lifetimes, MSR 2007
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortBug Severity
12 / 15
Bug severity
I Bug severity indicates how much a bug can affect theperformance and stability of the system [2]
I The resolution time of severe bugs is greater than theresolution time of less severe bugs [3]
I Developers may spent more effort when fixing severebugs wrt. less severe bugs
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortBug Severity
12 / 15
Bug severity
I Bug severity indicates how much a bug can affect theperformance and stability of the system [2]
I The resolution time of severe bugs is greater than theresolution time of less severe bugs [3]
I Developers may spent more effort when fixing severebugs wrt. less severe bugs
15
663
132
218
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortDevelopers’ Experience (1/2)
13 / 15
Developers’ Experience
I NB
I NF and NLOC (Overall and relevant)
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortDevelopers’ Experience (1/2)
13 / 15
Developers’ Experience
I NB
I NF and NLOC (Overall and relevant)
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
NB NF NLOC
0 0 (0) 0 (0)
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortDevelopers’ Experience (1/2)
13 / 15
Developers’ Experience
I NB
I NF and NLOC (Overall and relevant)
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortDevelopers’ Experience (1/2)
13 / 15
Developers’ Experience
I NB
I NF and NLOC (Overall and relevant)
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
T2 (Case 2) 1 2 (1) 7 (5)
F6F5
F2
F4
F3
F7F4
F2 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Factors Affecting EffortDevelopers’ Experience (2/2)
Developers’ Experience
I Developers experience does not reduce their effortI When a program evolves, developers may increasingly
perform tasks on parts of the program on which theyhave no previous experience
I Consistent result with [4] (#commits) for Mylyn andPDE project⇒ NB and NF can assess developers’ experience
[4] Robbes et al., Using developer interaction data to compare expertisemetrics, MSR 2013
14 / 15
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Conclusion
15 / 15
Thanks for your attention!
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
15
663
132
218
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Conclusion
15 / 15
Thanks for your attention!
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
15
663
132
218
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Conclusion
15 / 15
Thanks for your attention!
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
15
663
132
218
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Conclusion
15 / 15
Thanks for your attention!
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
15
663
132
218
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3
How DevelopersSpend their Effort
Zephyrin Soh et al.
Introduction
Context and Example
Data
Effort vs.Complexity
Research Question
Metrics
Matching
Results
Factors AffectingEffort
Additional Files
Bug Severity
Developers’Experience
Conclusion
Conclusion
15 / 15
Thanks for your attention!
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
Time (sec.)
Entropy
Entropy
Change distance
Change distance
0.16
0.27
0.31
0.33
F4
F1F2
F5F6
F3
F7F9F8
62% 38%
15
663
132
218
F6F4
F5F3
F2
F1
F7
F2 (5 LOC)
F1 (2 LOC)
Task
T1
T2 (Case 1)
NB NF NLOC
0 0 (0) 0 (0)
1 2 (0) 7 (0)
F6F5
F5
F4
F3
F7F4
F5 F3