![Page 1: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/1.jpg)
Lightweight Verification of Array Indexing
Martin Kellogg*, Vlastimil Dort**, Suzanne Millstein*, Michael D. Ernst*
* University of Washington, Seattle** Charles University, Prague
![Page 2: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/2.jpg)
The problem: unsafe array indexing● In unsafe languages (C): buffer overflow!
● In managed languages (Java, C#, etc.): exception, program crashes
2
![Page 3: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/3.jpg)
The state of the art
3
Strength of guarantees
Practical for developers
![Page 4: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/4.jpg)
The state of the art
4
Strength of guarantees
Practical for developers
CoqKeY
Clousot
![Page 5: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/5.jpg)
The state of the art
5
Strength of guarantees
Practical for developers
CoqKeY
Clousot
FindBugsCoverity
![Page 6: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/6.jpg)
The state of the art
6
Strength of guarantees
Practical for developers
CoqKeY
Clousot
FindBugsCoverity
The Index Checker (this talk)
![Page 7: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/7.jpg)
Problems with complex analyses
- false positives
- annotation burden
- complex analyses are hard to predict7
![Page 8: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/8.jpg)
Problems with complex analyses
- false positives● bounds checking is hard → complex analysis● complex analysis → harder to implement● harder to implement → more false positives
- annotation burden
- complex analyses are hard to predict8
![Page 9: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/9.jpg)
Problems with complex analyses
- false positives● bounds checking is hard → complex analysis● complex analysis → harder to implement● harder to implement → more false positives
- annotation burden● complex analysis → complex annotations
- complex analyses are hard to predict9
![Page 10: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/10.jpg)
Problems with complex analyses
- false positives● bounds checking is hard → complex analysis● complex analysis → harder to implement● harder to implement → more false positives
- annotation burden● complex analysis → complex annotations
- complex analyses are hard to predict10
![Page 11: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/11.jpg)
Fundamental problem is complex analyses!Insight:
11
![Page 12: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/12.jpg)
Cooperating simple analysesSolve all three problems:
12
![Page 13: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/13.jpg)
Cooperating simple analysesSolve all three problems:● simpler implementation → fewer false positives
13
![Page 14: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/14.jpg)
Cooperating simple analysesSolve all three problems:● simpler implementation → fewer false positives● simpler abstractions → easier to write annotations
14
![Page 15: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/15.jpg)
Cooperating simple analysesSolve all three problems:● simpler implementation → fewer false positives● simpler abstractions → easier to write annotations● simpler analysis → simpler to predict
15
![Page 16: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/16.jpg)
Proving an array access safe
T[] a = …;int i = …;... a[i] ...
We need to show that:● i is an index for a
16
![Page 17: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/17.jpg)
Proving an array access safe
T[] a = …;int i = …;... a[i] ...
We need to show that:● i is an index for a● i ≥ 0● i < a.length 17
![Page 18: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/18.jpg)
Proving an array access safe
T[] a = …;int i = …;... a[i] ...
We need to show that:● i is an index for a● i ≥ 0 A lower bound on i● i < a.length An upper bound on i 18
![Page 19: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/19.jpg)
A type system for lower bounds
T
↑
i ≥ -1↑
↑
i ≥ 0
i ≥ 1
@LowerBoundUnknown int i
↑
@GTENegativeOne int i
↑
↑
@NonNegative int i
@Positive int i19
![Page 20: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/20.jpg)
A type system for lower bounds
T
↑
i ≥ -1↑
↑
i ≥ 0
i ≥ 1
@LowerBoundUnknown int i
↑
@GTENegativeOne int i
↑
↑
@NonNegative int i
@Positive int i20
![Page 21: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/21.jpg)
A type system for upper bounds
if (i >= 0 && i < a.length) {a[i] = ...
}
21
![Page 22: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/22.jpg)
A type system for upper bounds
if (i >= 0 && i < a.length) {a[i] = ...
}
22
i < a.length @LTLengthOf(“a”) int i
![Page 23: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/23.jpg)
Type systems
Linear inequalitiesi < j
Minimum lengthsa.length > 10
Negative indices| i | < a.length
Lower boundsi ≥ 0
Equal lengthsa.length = b.length
Upper boundsi < a.length
23
![Page 24: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/24.jpg)
Type systems
Linear inequalitiesi < j
Minimum lengthsa.length > 10
Negative indices| i | < a.length
Lower boundsi ≥ 0
Equal lengthsa.length = b.length
Upper boundsi < a.length
24
![Page 25: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/25.jpg)
A type system for minimum array lengths
if (a.length >= 3) {a[2] = ...;
}
25
![Page 26: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/26.jpg)
A type system for minimum array lengths
if (a.length >= 3) {a[2] = ...;
}
26
a.length ≥ i T @MinLen(i) [] a
![Page 27: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/27.jpg)
EvaluationThree case studies:● Google Guava (two packages)● JFreeChart● plume-lib
Comparison to existing tools:● FindBugs, KeY, Clousot
27
![Page 28: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/28.jpg)
Case Studies
Guava JFreeChart plume-lib Total
Lines of code 10,694 94,233 14,586 119,503
Bugs found 5 64 20 89
Annotations 510 2,938 241 3,689
False positives 138 386 43 567
Java casts 222 2,740 219 3,181
28
![Page 29: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/29.jpg)
Comparison to other tools: confirmed bugs
Tool Index Checker FindBugs KeY Clousot
True Positives
False Negatives
Approach Types Bug finder Verif. w/ solver Abs. interpret.
Time (100k LoC)
29
![Page 30: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/30.jpg)
Comparison to other tools: confirmed bugs
Tool Index Checker FindBugs KeY Clousot
True Positives
False Negatives
Approach Types Bug finder Verif. w/ solver Abs. interpret.
Time (100k LoC) ~10 minutes ~1 minute cannot scale ~200 minutes
30
![Page 31: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/31.jpg)
Comparison to other tools: confirmed bugs
Tool Index Checker FindBugs KeY Clousot
True Positives 18/18 0/18 9/18 16/18
False Negatives 0/18 18/18 1/18 2/18
Approach Types Bug finder Verif. w/ solver Abs. interpret.
Time (100k LoC) ~10 minutes ~1 minute cannot scale ~200 minutes
31
![Page 32: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/32.jpg)
Using the Index Checker● Distributed with Checker Framework
www.checkerframework.org
32
![Page 33: Lightweight Verification of Array Indexingkelloggm/papers/issta18-slides.pdf · Comparison to other tools: confirmed bugs Tool Index Checker FindBugs KeY Clousot True Positives 18/18](https://reader033.vdocument.in/reader033/viewer/2022042915/5f52d20e223155588c739bc8/html5/thumbnails/33.jpg)
Contributions● A methodology: simple, cooperative type systems
● An analysis: abstractions for array indexing
● An implementation and evaluation for Java
● Verifying the absence of array bounds errors in real codebases (and finding bugs in the process!)
33