patterns for extracting high level information from bug reports
DESCRIPTION
Paper at https://github.com/rodrigorgs/dapse13-analysis/blob/master/preprint/icsews13dapse-id11-p-16145-preprint.pdf?raw=trueTRANSCRIPT
![Page 1: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/1.jpg)
P!""#r$% f&r '"r!(")$* H)*+ L#v#, I$f&r-!")&$ fr&- B.* R#p&r"%
Rodrigo Souza1,*
Christina Chavez1
Roberto Bittencourt2
1 Federal University of Bahia, Brazil 2 State University of Feira de Santana, Brazil
DAPSE’13: International Workshop on Data Analysis Patterns in Software Engineering
* speaker; email: [email protected]
May 21, 2013 San Francisco, USA
![Page 2: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/2.jpg)
Bug reports
![Page 3: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/3.jpg)
provide insight about… - the quality of the software - the quality of the process
Bug reports
![Page 4: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/4.jpg)
are like oysters… Bug reports
If you look inside, you may find something valuable
![Page 5: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/5.jpg)
I$ /)% T!,0 Two patterns to help you extract information about the software verification process
1. Fixers and Verifiers
2. Testing Phases
they’re like recipes for data scientists
![Page 6: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/6.jpg)
12#r% !$3 V#r)4#r% Find the quality engineering team (if it exists).
![Page 7: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/7.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
12#r% !$3 V#r)4#r%
![Page 8: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/8.jpg)
Developers assume specific roles in a team fixer: fixes bugs verifier: verifies if fixes are appropriate
![Page 9: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/9.jpg)
A quality engineering team is formed by verifiers, who perform most of the verifications in the project (among other activities)
![Page 10: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/10.jpg)
The roles should be taken into account in data analysis
You can’t judge a verifier by the number of fixes
![Page 11: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/11.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
12#r% !$3 V#r)4#r%
![Page 12: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/12.jpg)
Find the quality engineering team (if it exists)
![Page 13: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/13.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
12#r% !$3 V#r)4#r%
![Page 14: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/14.jpg)
You’ll need: - For each developer
- Number of times he changed status to VERIFIED (i.e., verifications)
- Number of times he changed resolution to FIXED (i.e., fixes)
I$*r#3)#$"%
![Page 15: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/15.jpg)
1
D)r#(")&$%
For each developer, compute the ratio: verifications / (1 + fixes)
2 Choose a threshold and assume that a developer is a verifier if
ratio > threshold how?
![Page 16: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/16.jpg)
D)r#(")&$%
2.1 For each ratio, use it as the threshold and compute: - the number of verifiers in the project - the % of verifications performed by verifiers
2.2 Plot this data
x y
![Page 17: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/17.jpg)
D)r#(")&$% 90%
80%
70%
60%
50%
40%
30% 2% 4% 6% 8% 10% 12% 14%
1 5 10 15
20 25, 30 35
40
How to choose a threshold?
size of QE team = number of verifiers (%)
% of verifications by verifiers
ratio (threshold candidate)
=
![Page 18: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/18.jpg)
D)r#(")&$% 1
5 10 15
20 25, 30 35
40
fit an arm, find the elbow!
% of verifications by verifiers
2% 4% 6% 8% 10% 12%
90%
80%
70%
60%
50%
40%
30%
2.3
size of QE team = number of verifiers (%)
![Page 19: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/19.jpg)
3
D)r#(")&$%
If % of verifications by verifiers is high*, they form a quality engineering team.
* e.g., > 50% 84%
![Page 20: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/20.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
12#r% !$3 V#r)4#r%
![Page 21: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/21.jpg)
Don’t use the absolute number of verifications, because developers may fix & verify simple bugs
![Page 22: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/22.jpg)
If developers are expected to change roles over time, use sliding windows.
![Page 23: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/23.jpg)
T#%")$* P+!%# Identify testing phases in the software development life cycle.
![Page 24: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/24.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
T#%")$* P+!%#
![Page 25: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/25.jpg)
In mature projects, new features and bug fixes are verified before being released to the public
![Page 26: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/26.jpg)
When are bugs verified?
Fix Verify Fix
Verify Fix
Verify
Fix Fix Fix
Verify Verify Verify
testing phase
![Page 27: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/27.jpg)
Failing to recognize testing phases can mislead your analyses
![Page 28: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/28.jpg)
Fix Verify
Fix bug #5
Verify bug #5 Fix
Verify
time …
time ~ complexity
Fix bug #7 Fix Fix
Verify bug #7 Verify Verify
… time
time ~ …?
![Page 29: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/29.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
T#%")$* P+!%#
![Page 30: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/30.jpg)
Identify testing phases in the software development life cycle
![Page 31: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/31.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
T#%")$* P+!%#
![Page 32: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/32.jpg)
You’ll need: - Time of verifications
- Release dates (optional)
I$*r#3)#$"%
![Page 33: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/33.jpg)
Plot the accum. number of verifications over time 1
D)r#(")&$% (%&,.")&$ #1)
time
accum. num. verif.
![Page 34: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/34.jpg)
If possible, highlight release dates 2
D)r#(")&$% (%&,.")&$ #1)
time
accum. num. verif.
![Page 35: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/35.jpg)
Find cliffs, especially before release dates (they are testing phases)
3
D)r#(")&$% (%&,.")&$ #1)
time
accum. num. verif.
![Page 36: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/36.jpg)
Apply Kleinberg’s algorithm to verification times in order to detect verification bursts
1
D)r#(")&$% (%&,.")&$ #2)
Bursts
![Page 37: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/37.jpg)
There’s no 2. 2
D)r#(")&$% (%&,.")&$ #2)
Bursts (= testing phases)
![Page 38: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/38.jpg)
1. Context 2. Problem 3. Solution 4. Discussion
T#%")$* P+!%#
![Page 39: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/39.jpg)
If the number of verifications on a particular day is too high, they may be mass updates
Look Out For Mass Updates and remove them beforing looking for testing phases
![Page 40: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/40.jpg)
Testing phases are less common in projects with quality engineering teams
![Page 41: Patterns for Extracting High Level Information from Bug Reports](https://reader035.vdocument.in/reader035/viewer/2022062514/5593b99d1a28ab27548b458b/html5/thumbnails/41.jpg)
Thank you!
Go beyond the surface to find pearls in bug reports!