faulthunter workshop (sourcemeter for sonarqube plugin module)
DESCRIPTION
A workshop presentation about FaultHunter, a programming mistake locator module for SourceMeter source code analysis engine compatible with SonarQube.TRANSCRIPT
2
Coding Rule Violations
Gábor Szőke
PhD Student
University of Szeged
www.sed.hu
2014.04.02. FrontEndART.com
2023.04.13. 3
What do we call Coding Rule Violations?
FrontEndART.com
• Source code patterns which indicate potential faults• These patterns come from actual bugs and include• Common programming mistakes• Bad programming habits• Performance issues
• The patterns are gathered by developer communities• Documented Eclipse bug fixes• Chapters of the Effective Java book• Blog posts on the http://thedailywtf.com website
2023.04.13. 4
Bugs? Come on!
FrontEndART.com
• Developers are smart.
• Smart people don’t make dumb mistakes.
• WRONG!
• Smart people do make dumb mistakes.
• Common issues:• Wrong boolean operator, missing parenthesis, etc.
• Misunderstood class or method invariants
2023.04.13. 5
Can You Find The Bug?
FrontEndART.com
JDK 1.6.0, b105, sun.awt.x11.XMSelection
if (listeners == null)
listeners.remove(listener);
2023.04.13. 6
Control Flow Conventions Performance
Open Stream Suspicious Equals Method Use Arrays As List
Avoid Catching NPE Naming Conventions Unused Constructor
StringBuffer Instead Of
Concat
Npath Complexity Unused Imports
Empty Catch Block Cloneable Implementation Too Many Fields
FrontEndART.com
Some Frequent Anti-Pattern Categories
2023.04.13. 7
Example
FrontEndART.com
• libsecurity_ssl/lib/sslKeyExchange.c
OpenSSL bug
2023.04.13. 8
Example
FrontEndART.com
• libsecurity_ssl/lib/sslKeyExchange.c
IESMUB – If-Else Stmts Must Use
Braces
#gotofail
2023.04.13. 9
Popular Solutions
FrontEndART.com
• CheckStyleCoding standard checkerHighly configurable×Small amount of implemented rules×Rules are mostly for style conventions,
not for fault prevention
• FindBugsLooks for bugs in Java codeFinds NPEs, typos, inconsistencies×A lot of irrelevant warnings×Does not show the roots of the problems
• PMDOpen-source code violation checkerMany rules×A lot of irrelevant warnings×Several significant warning are not found
2023.04.13. 10
Pros and Cons
Relevant warning
Issues are found during development
Trains developers (lots of skills can be learned from following coding rules!)
Reduces project costs
Increases efficiency
Irrelevant warning
Creates work overhead
Developers will start ignoring coding issue warnings
Causes stress
Convinces developers that the code lacks actual coding issues even when there are a lot.
FrontEndART.com
2023.04.13. 11
Can You Find The Bug?
FrontEndART.com
// com.sun.xml.internal.txw2.output.XMLWritertry { … }catch (IOException e) { new SAXException(”Server side Exception:” + e);}
// com.sun.xml.internal.txw2.output.XMLWritertry { … }catch (IOException e) { throw new SAXException(”Server side Exception:” + e);}
2023.04.13. 12
False Positive and True Negative hits
FrontEndART.com
Found FaultsFaults
Not Faults
True Negative
False Positive
Solution?
2023.04.13. 13
False Positive and True Negative hits
FrontEndART.com
Found FaultsFaults
Not Faults
True Negative
False Positive
We need a bigger boat!
2023.04.13. 14
False Positive and True Negative hits
FrontEndART.com
Found Faults
Faults
Not Faults
True Negative
False Positive
We need a bigger boat!
2023.04.13. 15
FaultHunter
SourceMeter plugin for SonarQube
FrontEndART.com
2023.04.13. 16
FaultHunter
FrontEndART.com
• Advanced static analysis engine• Re-implements inaccurate
PMD rule violations• 62 implemented PMD rule definitions
• Defines new rules• 8 new rule definitions
• Precise and efficient• A module of SourceMeter• Supports Ant and Maven integration• Is part of the SourceMeter for SonarQube plugin
Advanced Static Analysis
source code
analysis
source code
abstract syntax tree
coding issue warnings
analysis
modeling
modeling
PMD
FaultHunter
Few warnings found
More true positivewarningsabstract semantic graph
PMD Shared FaultHunter
ArgoUML 484 31% 1,086 531 34%Cobertura 195 5% 4,121 7,689 178%Eclipse 6,867 19% 28,661 65,773 185%GWT 1,166 17% 5,533 2,021 30%
Hibernate 560 17% 2,829 2,170 64%Jenkins 430 24% 1,399 2,649 145%Log4j2 120 22% 431 145 26%PMD 113 6% 1,815 2,112 110%
Spring-Framework 1,206 26% 3,497 1,219 26%Tomcat 859 19% 3,728 4,241 92%
… … … … … …Total 51,107 14% 308,339 358,543 100%
• 217 open source software systems analyzed
Results
2023.04.13. 19FrontEndART.com
ResultsPMD
FH
PMD
FH
PMD
FH
PMD
FH
PMD
FH
PMD
FH
PMD
FH
Hib
ern
ate
Log4j
Tom
cat
-100% -50% 0% 50% 100% 150% 200% 250% 300% 350%
SharedPMDFaultHunter
2023.04.13. 20FrontEndART.com
Rule FP TN Rule FP TN
AvoidCatchingNPE 3% 0% CloseResource 27% 45%
AvoidCatchingThrowable 0% 2% DefaultLabelNotLastInSwitchStmt 0% 2%
AvoidDeeplyNestedIfStmts 99% 21% EmptyCatchBlock 49% 0%
AddEmptyString 0% 0% EmptyFinallyBlock 0% 175%
AvoidInstanceofChecksInCatchClause 1% 1% EmptyIfStmt 14% 6%
ArrayIsStoredDirectly - - EmptySynchronizedBlock 0% 0%
AvoidPrintStackTrace 12% 2% EmptySwitchStatements - -
AvoidRethrowingException 51% 0% EmptyTryBlock 0% 0%
AvoidThrowingNullPointerException 2% 4% EmptyWhileStmt 60% 0%
AvoidThrowingRawExceptionTypes 2% 32% ForLoopsMustUseBraces - -
BooleanGetMethodName 0% 0% IfElseStmtsMustUseBraces 25% 397%
BooleanInstantiation 1% 1% IntegerInstantiation 2% 1%
BigIntegerInstantiation 1% 42% LooseCoupling 16% 134%
ByteInstantiation 0% 0% LongInstantiation 1% 0%
Results by rule violation types
2023.04.13. 21FrontEndART.com
Results by rule violation types Rule FP TN Rule FP TN
MissingBreakInSwitch 88% 67% UseArrayListInsteadOfVector 27% 98%
MethodWithSameNameAsEnclosingClass 0% 0% UncommentedEmptyMethod 28% 3%
MethodNamingConventions 0% 2% UseEqualsToCompareStrings 2% 105%
PositionLiteralsFirstInComparisons 6% 75% UnnecessaryLocalBeforeReturn 8% 55%
PreserveStackTrace 1% 35% UnusedLocalVariable 2% 551%
ReplaceHashtableWithMap 7% 7% UseLocaleWithCaseConversions 12% 77%
ReplaceVectorWithList 5% 30% UselessOverridingMethod 29% 42%
SimplifyConditional 13% 10% UnusedPrivateField 3% 439%
SimpleDateFormatNeedsLocale 6% 2% UnusedPrivateMethod 2% 235%
SuspiciousHashcodeMethodName UseStringBufferForStringAppends 21% 7%
ShortInstantiation 0% 0% UnusedImports 0% 25%
ShortMethodName 24% 0% WhileLoopsMustUseBraces
SwitchStmtsShouldHaveDefault 0% 3%
2023.04.13. 22
Precise and Efficient
FrontEndART.com
Uses 50% more memory than PMD on the same source set
Runs 50% longer than PMD on the same source set
BUT!
• Finds 99.97% more faults than PMD
• Skips 14.92% of false positive warnings
2023.04.13. 23
Examples
FrontEndART.com
• org.eclipse.ui.forms/src/org/eclipse/ui/forms/DetailsPart.java
EIF – Empty If StatementPMD misses it, FaultHunter doesn’t.
True Negative
What was the programmer’s intention? Did they simply forget to code the instructions? Potential
threat!
2023.04.13. 24
Examples
FrontEndART.com
• org.eclipse.ui.workbench/Eclipse UI/org/eclipse/ui/internal/WorkbenchWindow.java
BGMN – Boolean Get Method NamePMD misses it, FaultHunter doesn’t.
True Negative
• org.eclipse.ui.ide/src/org/eclipse/ui/internal/ide/misc/ProjectCapabilitySelectionGroup.java
Issue: JavaEE, SpringFramework searches for getters like
isShellActivated() or isModified() Error!
2023.04.13. 25
Examples
FrontEndART.com
• Tomcat/java/org/apache/catalina/core/ApplicationFilterConfig.java
USBFSA – Use String Buffer For String
AppendsPMD finds it, FaultHunter
doesn’t.False Positive
Totally irrelevant warning
2023.04.13. 26
Examples
FrontEndART.com
• org.eclipse.core.resources/src/org/eclipse/core/internal/events/NotificationManager.java
MBIS – Missing Break In SwitchPMD misses it, FaultHunter doesn’t.
True Negative
Why is the break statement missing? It was intentional, as
the comment indicates.
2023.04.13. 27
Examples
FrontEndART.com
• Log4j\src\main\java\org\apache\log4j\NDC.java
Is it really necessary to use the thread-safe Vector class?
If this is not the case, we can use the ArrayList class which offers a much faster implementation with the
same functionality.(Collections.synchronizedList(…) can be used in
a multi-threaded environment too!) Comment indicates that developers see the problem
as well.
UALIOV – Use Array List Instead Of Vector
PMD misses it, FaultHunter doesn’t.True Negative
2023.04.13. 28
Examples
FrontEndART.com
• openejb-4.5.2/org/apache/openejb/server/cxf/rs/CustomProviderTest.java
ACNPE – Avoid Catching NullPointerExceptionPMD misses it, FaultHunter doesn’t.
True Negative
• xalan-2.7/src/org/apache/xpath/jaxp/XPathImpl.java
SMN – Short Method NamePMD misses it, FaultHunter doesn’t.
True Negative
• Spring-Framework/spring-beans/src/test/java/org/springframework/tests/sample/beans/TestBean.java
ACNPEAvoid Throwing Raw Exception TypesPMD misses it, FaultHunter doesn’t.
True Negative
2023.04.13. 29
Examples
FrontEndART.com
• ApacheDS/core-pi/src/test/java/org/apache/directory/server/core/api/changelog/TagTest.java
Position Literals First In ComparisonsPMD finds it, FaultHunter doesn’t.
False PositiveTotally irrelevant warning.