unicorn: a unified approach for localizing non-deadlock concurrency bugs
TRANSCRIPT
UNICORN: A UNIFIED APPROACH FOR LOCALIZING NON-DEADLOCK
CONCURRENCY BUGS
Sangmin Park, Richard Vuduc, and Mary Jean Harrold
College of ComputingGeorgia Institute of Technology
2
Concurrency Bugs Technique Studies Conclusion
Interleaving Space
Motivation
Correct Interleaving
Buggy Interleaving involving multiple
resources
Buggy Interleaving involving single
resource
3
Concurrency Bugs Technique Studies Conclusion
Existing Techniques
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations --- multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs --- single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
4
Concurrency Bugs Technique Studies Conclusion
Existing Techniques
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations --- multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs --- single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
LimitationDon't detect concurrency bugs involving multiple variables
5
Concurrency Bugs Technique Studies Conclusion
Existing Techniques
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations — multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs --- single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
LimitationDon't detect concurrency bugs involving multiple variables
6
Concurrency Bugs Technique Studies Conclusion
Existing Techniques
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations — multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs --- single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
LimitationDon't detect concurrency bugs involving multiple variables
LimitationDon't detect order violations
7
Concurrency Bugs Technique Studies Conclusion
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations — multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs — single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
LimitationDon't detect order violations
Existing Techniques
LimitationDon't detect concurrency bugs involving multiple variables
8
Concurrency Bugs Technique Studies Conclusion
Detect concurrency bugs — single variable Data race detectors [FastTrack, Eraser]
Order violation detectors [Falcon]
Atomicity violation detectors [AVIO, Falcon, CTrigger]
Detect atomicity violations — multiple variables Atomicity violation detectors [ColorSafe, AtomTracker]
Detect concurrency bugs — single and multiple General bug detectors w/ statistical analysis
[CCI, Bugaboo, Recon, DefUse]
LimitationDon't detect order violations
Existing Techniques
LimitationDon't detect concurrency bugs involving multiple variables
LimitationDon't provide bug type information
9
Concurrency Bugs Technique Studies Conclusion
Overview
Concurrency Bugs Order violation Single-variable atomicity violation Multi-variable atomicity violation
UNICORN Technique Detects three types of concurrency bugs Provides bug type information
Empirical Studies Conclusion
10
Concurrency Bugs Technique Studies Conclusion
Bug: Order Violation
void main(…) { … pthread_create(consumer);
…
…
mutex = NULL;
…
}
void * consumer (…) {
…
Lock(mutex);
…}
Thread 1 Thread 2
W1R2
* This example is from PBZip2.
11
Concurrency Bugs Technique Studies Conclusion
Bug: Single-Variable Atomicity
void new_file(…) {
…
saved_type = log_type;
log_type = CLOSED;
…
// update file
…
log_type = saved_type;
…}
int mysql_insert(…) {
// actual insert
…
if (log_type != CLOSED)
{
LOG.write(…);
}
…
}
Thread 1 Thread 2
W3
W1 R2
* This example is from MySQL-791.
12
Concurrency Bugs Technique Studies Conclusion
Bug: Multi-Variable Atomicity
int mysql_delete(…) {
…
Lock(X);
Table.remove ();
Unlock(X);
…
Lock(Y);
LOG.write();
Unlock(Y);
…}
int mysql_insert(…) {
…
Lock(X);
Table.insert();
Unlock(X);
…
Lock(Y);
LOG.write();
Unlock(Y);
…}
Thread 1 Thread 2
W4
W1 W2
W3
* This example is from MySQL-169.
13
Concurrency Bugs Technique Studies Conclusion
Bug: Multi-Variable Atomicity
int mysql_delete(…) {
…
Lock(X);
Table.remove ();
Unlock(X);
…
Lock(Y);
LOG.write();
Unlock(Y);
…}
int mysql_insert(…) {
…
Lock(X);
Table.insert();
Unlock(X);
…
Lock(Y);
LOG.write();
Unlock(Y);
…}
Thread 1 Thread 2
W4
W1 W2
W3
* This example is from MySQL-169.
NoteThese three types of concurrency bugs are the most important types of non-deadlock concurrency bugs [Lu 08].
14
Concurrency Bugs Technique Studies Conclusion
Problematic Access Patterns#Var Type Memory Access Patterns
Single
Order
R1,S(x) W2,S(x)
W1,S(x) R2,S(x)
W1,S(x) W2,S(x)
Single-VariableAtomicity
R1,S(x) W2,S(x) R1,S(x)
W1,S(x) W2,S(x) R1,S(x)
W1,S(x) R2,S(x) W1,S(x)
W1,S(x) R2,S(x) W1,S(x)
W1,S(x) W2,S(x) W1,S(x)
MultiMulti-Variable
Atomicity
W1,S(x) W2,S(x) W2,S(y) W1,S(y)
W1,S(x) W2,S(y) W2,S(x) W1,S(y)
W1,S(x) W2,S(y) W1,S(y) W2,S(x)
W1,S(x) R2,S(x) R2,S(y) W1,S(y)
W1,S(x) R2,S(y) R2,S(x) W1,S(y)
R1,S(x) W2,S(x) W2,S(y) R1,S(y)
R1,S(x) W2,S(y) W2,S(x) R1,S(y)
R1,S(x) W2,S(y) R1,S(y) W2,S(x)
W1,S(x) R2,S(x) W1,S(y) R2,S(y)
* The patterns were identified by previous work [Lu 06, Vaziri 06, Hammer 08].
15
Concurrency Bugs Technique Studies Conclusion
UNICORN: Overview
P
T
ranked-patternsUNICORN
[ Fault-localization technique ]
16
Concurrency Bugs Technique Studies Conclusion
UNICORN: Overview
P
TCollect Pairs
From Executions
pairs,outcomes
Step 1:Combine PairsInto Patterns
patterns,outcomes
Step 2:RankPatterns
ranked-patterns
Step 3:
[ UNICORN: A UNIfied approach for localizing non-deadlock COncuRreNcy Bugs ]
Intuition: Patterns consist of (one or two) pairs
Find both single-variable and multi-variable patterns
17
Concurrency Bugs Technique Studies Conclusion
#Var Type Memory Access Patterns
Single
Order
R1,S(x) W2,S(x)
W1,S(x) R2,S(x)
W1,S(x) W2,S(x)
Single-VariableAtomicity
R1,S(x) W2,S(x) R1,S(x)
W1,S(x) W2,S(x) R1,S(x)
W1,S(x) R2,S(x) W1,S(x)
W1,S(x) R2,S(x) W1,S(x)
W1,S(x) W2,S(x) W1,S(x)
MultiMulti-
VariableAtomicity
W1,S(x) W2,S(x) W2,S(y) W1,S(y)
W1,S(x) W2,S(y) W2,S(x) W1,S(y)
W1,S(x) W2,S(y) W1,S(y) W2,S(x)
W1,S(x) R2,S(x) R2,S(y) W1,S(y)
W1,S(x) R2,S(y) R2,S(x) W1,S(y)
R1,S(x) W2,S(x) W2,S(y) R1,S(y)
R1,S(x) W2,S(y) W2,S(x) R1,S(y)
R1,S(x) W2,S(y) R1,S(y) W2,S(x)
W1,S(x) R2,S(x) W1,S(y) R2,S(y)
Problematic Access Patterns#Var Type Memory Access Patterns Memory Access Pairs
Single
Order
R1,S(x) W2,S(x) R1,S(x) W2,S(x)
W1,S(x) R2,S(x) W1,S(x) R2,S(x)
W1,S(x) W2,S(x) W1,S(x) W2,S(x)
Single-VariableAtomicity
R1,S(x) W2,S(x) R1,S(x) R1,S(x) W2,S(x) W2,S(x) R1,S(x)
W1,S(x) W2,S(x) R1,S(x) W1,S(x) W2,S(x) W2,S(x) R1,S(x)
W1,S(x) R2,S(x) W1,S(x) W1,S(x) R2,S(x) R2,S(x) W1,S(x)
W1,S(x) R2,S(x) W1,S(x) W1,S(x) R2,S(x) R2,S(x) W1,S(x)
W1,S(x) W2,S(x) W1,S(x) W1,S(x) W2,S(x) W2,S(x) W1,S(x)
MultiMulti-
VariableAtomicity
W1,S(x) W2,S(x) W2,S(y) W1,S(y) W1,S(x) W2,S(x) W2,S(y) W1,S(y)
W1,S(x) W2,S(y) W2,S(x) W1,S(y) W1,S(x) W2,S(x) W2,S(y) W1,S(y)
W1,S(x) W2,S(y) W1,S(y) W2,S(x) W1,S(x) W2,S(x) W2,S(y) W1,S(y)
W1,S(x) R2,S(x) R2,S(y) W1,S(y) W1,S(x) R2,S(x) R2,S(y) W1,S(y)
W1,S(x) R2,S(y) R2,S(x) W1,S(y) W1,S(x) R2,S(x) R2,S(y) W1,S(y)
R1,S(x) W2,S(x) W2,S(y) R1,S(y) R1,S(x) W2,S(x) W2,S(y) R1,S(y)
R1,S(x) W2,S(y) W2,S(x) R1,S(y) R1,S(x) W2,S(x) W2,S(y) R1,S(y)
R1,S(x) W2,S(y) R1,S(y) W2,S(x) R1,S(x) W2,S(x) W2,S(y) R1,S(y)
W1,S(x) R2,S(x) W1,S(y) R2,S(y) W1,S(x) R2,S(x) R2,S(y) W1,S(y)
18
Concurrency Bugs Technique Studies Conclusion
Step 1: Collect Pairs in Executions
Collect pairs using sliding window [Falcon, ICSE 2010]
Maintain a window per shared variable (pair window) Collect WR, RW, WW access-pairs within window
P
TCollect Pairs
From Executions
pairs,outcomes
Step 1:Combine PairsInto Patterns
patterns,outcomes
Step 2:RankPatterns
Step 3:ranked-patterns
19
Concurrency Bugs Technique Studies Conclusion
Step 1: Collect Pairs in Executions
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2
Tbl:
LOG:
[Run 1] Failing
Accesses
Windows
Pairs
R0 R1 W0R0 R1 W0 W3 W4W1 W2
R0-W0, R0-W1
W1-W2
W3-W4
R0 R1 W0R0 R1 W0 W3 W4W1 W2
, R1-W0, R1-W1R0-W0, R0-W1
…
…
pairs for the code in the slide
20
Concurrency Bugs Technique Studies Conclusion
Step 1: Collect Pairs in Executions
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2
ID Pair
R1
R2
R3
R4
R5
R6
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
F P P F P P
ID Pair
R1
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
F
21
Concurrency Bugs Technique Studies Conclusion
Step 2: Combine Pairs to Patterns
1. Set pairs to patterns for order violation
2. Combine pairs to patterns using sliding window Sort pairs in time order Combine two-pair-based patterns within pattern window
P
TCollect Pairs
From Executions
pairs,outcomes
Step 1:Combine PairsInto Patterns
patterns,outcomes
Step 2:RankPatterns
Step 3:ranked-patterns
22
Concurrency Bugs Technique Studies Conclusion
Step 2: Combine Pairs to Patterns
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2[Run 1] Failing
Pairs Patterns
R0-W0R0-W1R0-W0R0-W1R1-W0R1-W1R1-W2R1-W0R1-W1R1-W2W1-W2W3-W4
R0-W0R0-W1R0-W0R0-W1R1-W0R1-W1R1-W2R1-W0R1-W1R1-W2W1-W2W3-W4
W1-W2-W3-W4
23
Concurrency Bugs Technique Studies Conclusion
Step 2: Combine Pairs to Patterns
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2 ID Pattern
R1
R2
R3
R4
R5
R6
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
F P P F P P
ID Pattern
R1
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
F
24
Concurrency Bugs Technique Studies Conclusion
Step 3: Rank Patterns
Rank patterns Compute suspiciousness of patterns
Jaccard formula [Abreu07]
suspiciousness(P) =failed (P)
totalfailed + passed(P)
P
TCollect Pairs
From Executions
pairs,outcomes
Step 1:Combine PairsInto Patterns
patterns,outcomes
Step 2:RankPatterns
Step 3:ranked-patterns
25
Concurrency Bugs Technique Studies Conclusion
Step 3: Rank Patterns
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2
ID Pattern
R1
R2
R3
R4
R5
R6 Susp
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
F P P F P P
ID Pattern
R1
R2
R3
R4
R5
R6 Susp
1W1 (Tbl)-W2 (Tbl) 0.5
2W3 (LOG)-W4 (LOG) 0.5
3W2 (Tbl)-W1 (Tbl) 0.0
4W4 (LOG)-W3 (LOG) 0.0
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
1.0
F P P F P P
suspiciousness(P) =failed (P)
totalfailed + passed(P)
26
Concurrency Bugs Technique Studies Conclusion
Step 3: Rank Patterns
int delete(…) {
…
Lock(X);
W1: Tbl();
Unlock(X);
…
Lock(Y);
W4: LOG();
Unlock(Y);
…}
int insert(…) {
…
Lock(X);
W2: Tbl();
Unlock(X);
…
Lock(Y);
W3: LOG();
Unlock(Y);
…}
Thread 1 Thread 2
ID Pattern
R1
R2
R3
R4
R5
R6 Susp
1W1 (Tbl)-W2 (Tbl)
2W3 (LOG)-W4 (LOG)
3W2 (Tbl)-W1 (Tbl)
4W4 (LOG)-W3 (LOG)
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
P F P F P P
ID Pattern
R1
R2
R3
R4
R5
R6 Susp
1W1 (Tbl)-W2 (Tbl) 0.5
2W3 (LOG)-W4 (LOG) 0.5
3W2 (Tbl)-W1 (Tbl) 0.0
4W4 (LOG)-W3 (LOG) 0.0
5
W1 (Tbl)-W2 (Tbl)-
W3 (LOG)-W4 (LOG)
1.0
F P P F P P
suspiciousness(P) =failed (P)
totalfailed + passed(P)
SummaryUNICORN • Detects memory access pairs• Combines pairs into patterns• Ranks patterns w.r.t. suspiciousness
27
Concurrency Bugs Technique Studies Conclusion
Empirical Studies
Studies1. Evaluate effectiveness of the technique
2. Measure distance of the pattern window
3. Evaluate efficiency of the technique (see paper)
Empirical Setup Implemented in Java (Soot) and C++ (LLVM) Evaluated on a set of subjects
28
Concurrency Bugs Technique Studies Conclusion
Subjects
Language Program LOC % Failed Bug Type
C++Extracted
TimerThread 68 14.4 Order
LoadScript 110 49.8 Single-variable atomicity
MysqlLog 89 2.8 Single-variable atomicity
JsString 95 1.4 Multi-variable atomicity
MysqlDelete 103 3.8 Multi-variable atomicity
MysqlSlave 94 0.4 Multi-variable atomicity
C++Complete
PBZip2 2k 2.8 Order
Mysql-791 372k 64.0 Single-variable atomicity
Aget 1.2k 49.0 Multi-variable atomicity
Mysql-169 334k 63.0 Multi-variable atomicity
JavaStringBuffer 1.4k 22.3 Multi-variable atomicity
Vector 9.5k 7.6 Multi-variable atomicity
29
Concurrency Bugs Technique Studies Conclusion
Study 1: Effectiveness
Goal To determine how well UNICORN ranks the
pattern of the actual bug
Method Set pair-window size to 5 Set pattern-window size to 100 Ran each subject 100 times and computed
suspiciousness and rankings
30
Concurrency Bugs Technique Studies Conclusion
Study 1: Effectiveness
ProgramNum of Pairs
Num of Patterns
Susp(Pair 1)
Susp(Pair 2)
Susp(Pattern)
Rank(Bug)
Num of Top rank
TimerThread 6 7 1.0 - 1.0 1 3
LoadScript 3 4 0.86 1.0 1.0 1 2
MysqlLog 4 5 0.02 1.0 1.0 1 2
JsString 5 8 0.04 1.0 1.0 1 3
MysqlDelete 7 1 0.04 0.27 1.0 1 1
MysqlSlave 6 11 0.01 0.66 1.0 1 1
PBZip2 59 333 0.42 - 0.42 1 6
Mysql-791 1082 10936 1.0 0.64 1.0 1 4
Aget 37 94 0.51 0.50 0.70 1 1
Mysql-169 894 9051 0.63 0.63 1.0 1 7
StringBuffer 8 18 0.58 1.0 1.0 1 3
Vector 11 25 1.0 1.0 1.0 1 4
1
2
3
31
Concurrency Bugs Technique Studies Conclusion
Study 2: Pattern-window size
Goal To investigate the practicality of the pattern-
window size
Method Computed the distance between the two pairs of
the actual concurrency bugs (using data from Study 1)
Computed the median and maximum distances
32
Concurrency Bugs Technique Studies Conclusion
Study 2: Pattern-window size
ProgramNum of
PairsNum of
PatternsMedian
distanceMaximum distance
TimerThread 6 7 N/A N/A
LoadScript 3 4 2 2
MysqlLog 4 5 2 2
JsString 5 8 3 3
MysqlDelete 7 1 2 2
MysqlSlave 6 11 2 2
PBZip2 59 333 N/A N/A
Mysql-791 1082 10936 18 27
Aget 37 94 2 3
Mysql-169 894 9051 41 41
StringBuffer 8 18 4 5
Vector 11 25 2 2
33
Concurrency Bugs Technique Studies Conclusion
Future Work
Provide information to understand bugs Problem: Several patterns ranked top together Approach: In-depth analysis on the ranking
Provide information on test-suite Problem: No study on effectiveness vs. test suite Approach: Investigation on quality of test suite
Improve overhead Problem: Significant overhead on monitoring Approach: Use techniques such as sampling
34
Concurrency Bugs Technique Studies Conclusion
Contributions
UNICORN is the first unified technique that detects memory-access pairs, combines pairs to
get patterns, and ranks patterns with respect to suspiciousness
handles both single- and multi-variable violations Empirical studies show that UNICORN
is effective in identifying patterns associated with real bugs
is efficient it needs small-/fixed-sized windows to find patterns its runtime overhead comparable to other techniquesQuestions