Download - Handling Bugs
-
8/9/2019 Handling Bugs
1/27
Introduction Study Results Concluding Remarks
A Study on Developers Perceptions AboutException Handling Bugs
Felipe EbertFernando Castor
Centro de Informatica Universidade Federal de PernambucoRecife, Pernambuco, Brazil
Some rights reserved
http://find/http://goback/ -
8/9/2019 Handling Bugs
2/27
Introduction Study Results Concluding Remarks
Motivation
Error Handling CodeHappy path is happierOften written in an ad hoc
manner?Rarely tested or documented?Low quality?Source of bugs?
http://find/ -
8/9/2019 Handling Bugs
3/27
Introduction Study Results Concluding Remarks
Motivation
Error Handling CodeHappy path is happierOften written in an ad hoc
manner?Rarely tested or documented?Low quality?Source of bugs?
Developer, what are you thinking?
We conducted a survey to nd out.
http://find/http://goback/ -
8/9/2019 Handling Bugs
4/27
Introduction Study Results Concluding Remarks
Exception Handling Bugs
Bugs whose cause is related to exception handling.
http://find/ -
8/9/2019 Handling Bugs
5/27
Introduction Study Results Concluding Remarks
Exception Handling Bugs
Bugs whose cause is related to exception handling.
More specically, bugs related to the ...denitionpropagationthrowinghandlingdocumentation
... of exceptions.
http://find/http://goback/ -
8/9/2019 Handling Bugs
6/27
I t d ti St d R lt C l di R k
-
8/9/2019 Handling Bugs
7/27
Introduction Study Results Concluding Remarks
Research Questions
RQ1 Do organizations and developers pay attention toexception handling?
RQ2 How commonplace are exception handling bugs?
RQ3 Are exception handling bugs harder to x than otherbugs?
RQ4 What are the main causes of exception handling bugs?
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
8/27
Introduction Study Results Concluding Remarks
About the Survey
24 questionsMore than 4000 emails sent
154 respondents
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
9/27
Introduction Study Results Concluding Remarks
Study Results
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
10/27
Introduction Study Results Concluding Remarks
RQ1 Do organizations anddevelopers pay attention to
exception handling?
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
11/27
Introduction Study Results Concluding Remarks
Do organizations and developers pay attention?
Specications, policies, standards73% no
Tests specic for EH code
70% no
Importance of EH in the design phase61% irrelevant or of little importance16% important or very important
Quality of EH code14% bad or very bad40% good or very good
Introduction Study Results Concluding Remarks
http://find/http://goback/ -
8/9/2019 Handling Bugs
12/27
y g
Do organizations and developers pay attention?
Specications, policies, standards73% no
Tests specic for EH code
70% no
Importance of EH in the design phase61% irrelevant or of little importance16% important or very important
Quality of EH code14% bad or very bad40% good or very good
More experienceddevelopers (10+ years)tend to believe that thequality is worse.
p-value = 0.02478
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
13/27
y g
Why do developers use exception handling?
1. To create ways to tolerate faults 66%2. To improve the quality of a functionality 63%3. Importance of functionality 53%4. Language requirement 43%5. Organizational policies 21%6. To debug a specic part of the code 17%7. Does not use exception handling 2%
Reasons 6 and 7 were the most often cited in a previous study*
* Hina Shah, Carsten Gorg, Mary Jean Harrold: Understanding Exception Handling:
Viewpoints of Novices and Experts. IEEE Trans. Software Eng. 36(2): 150-161 (2010)
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
14/27
y g
RQ3 Are exception handlingbugs harder to x than other
bugs?
Introduction Study Results Concluding Remarks
http://find/http://goback/ -
8/9/2019 Handling Bugs
15/27
Developers Opinions
Level of difficulty to x bugs related to exception handling43% consider them to be easy or very easy to x
Average level of difficulty to x non-EH bugsOnly 7% consider them easy or very easy to x
Priority/Severity of exception handling bugsLess experienced developers consider it to be lower (p-value =0.0474)
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
16/27
RQ4 What are the maincauses of exception handling
bugs?
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
17/27
Main Causes
1. Lack of a handler that should exist2. No exception thrown in a situation of a known error3. Programming error in the catch block4. Programming error in the finally block5. Exception is caught unintentionally6. Catch block where only a nally would be appropriate7. Exception that should not have been thrown8. Wrong encapsulation of exception cause9. Wrong exception thrown
10. Lack of a finally block that should exist11. Error in the exception assertion12. Empty catch block
Can guide inspection and test activities
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
18/27
A Note on Swallowed Exceptions
Have you ever needed to x bugs related toexception handling?
Yes 83%
Please describe some of these situations...19 answers (1 in 6) mentioned empty catch blocks as causes
Sometimes swallowing is considered a good thing!
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
19/27
Future Work
Analysis of bug reports
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
20/27
Thank You!A Study on Developers Perceptions About
Exception Handling Bugs
Felipe Ebert and Fernando CastorContact: [email protected]
Discussion Topic
Should developers worry about exception handling(bugs)?
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
21/27
Backup Slides
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
22/27
This is not (necessarily) an exception handling bug
Examplepublic void buggyMethod() throws IOException {
File f = null;.... // should have initialized f but did notreturn f.getName(); // NullPointerException
}
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
23/27
This is.
Examplepublic void buggyMethod() throws IOException {
File f = null;.... // f is still not initializedreturn f.getName(); // NullPointerException
}
public void handlerMethod() {try { ... } catch (IOException e) {
String fileName = this.buggyMethod() ;// will throw NullPointerException but// handlerMethod expects IOException .
}}
Introduction Study Results Concluding Remarks
http://find/http://goback/ -
8/9/2019 Handling Bugs
24/27
Threats to Validity
Developers may have misundestood the denition of EH bug
Relatively small number of respondents
Questionnaire may lack important questions
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
25/27
Who answered the survey?
Java ExperienceMost respondents have 7+ years
7.14% Less than 2 years
20.78% 2 to 5 years
14.94% 5 to 7 years
20.13% 7 to 10 years
37.01% More than 10 years
Project Size42% currently work on projects with more than 100k LoC
14.03% Less than 20k LoC
20.13% 20k to 50k LoC
13.64% 50k to 100k LoC
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
26/27
RQ2 How commonplace areexception handling bugs?
Introduction Study Results Concluding Remarks
http://find/ -
8/9/2019 Handling Bugs
27/27
Developers Estimates
Percentage of bugs related to exception handling code inprojects
Mean 9.72%Median 5%
Exception handling bugs are reported less oftenp-value < 0.0001
Exception handling bugs are less frequentp-value < 0.0001
http://find/