testing atomicity of composed concurrent operations
DESCRIPTION
Testing Atomicity of Composed Concurrent Operations. Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University Mooly Sagiv Tel Aviv University Martin Vechev ETH & IBM Research Eran Yahav Technion. Concurrent Data Structures. - PowerPoint PPT PresentationTRANSCRIPT
Testing Atomicity of Composed Concurrent Operations
Ohad Shacham Tel Aviv UniversityNathan Bronson Stanford UniversityAlex Aiken Stanford UniversityMooly Sagiv Tel Aviv University
Martin Vechev ETH & IBM Research
Eran Yahav Technion
Concurrent Data Structures
• Writing highly concurrent data structures is complicated
• Modern programming languages provide efficient concurrent collections with atomic semantics
.
.
……
…
…
…
………
Challenge
Testing the atomicity of composed operations
……
…
…
…
………
TOMCAT Motivating Example
attr = new HashMap();…
Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}
TOMCAT 5*.TOMCAT 6*.
Invariant: removeAttribute(name) returns the value it removes from attr or null
attr = new ConcurrentHashMap();…
Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
o
Linearizability removeAttribute(“A”) {
Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
nullnull
o
null
null
o
null
Challenge
Testing the linearizabiliy of composed operations
……
…
…
…
………
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
• Reconstruction in TOMCAT extremely challenging– Large traces– Large number of traces– Bugs occur in rare cases with specific key values
Our Solution
ModularityGenerates simple traces
Base linearizability Restrict generated traces
Influence Restrict generated traces
Enables Env control
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Modular Checking
Challenge
Testing Linearizability of composed operation in a modular fashion
……
…
…
…
………
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ;if (found) {return val;
removeAttribute(“B”) {Attribute val = null;
found = attr.containsKey(“B”) ;if (found) {return val;
removeAttribute(“GGG”) {Attribute val = null;
found = attr.containsKey(“GGG”) if (found) {return val;
attr.put(“A”, o);
removeAttribute(“GGG”) {Attribute val = null;
found = attr.containsKey(“GGG”) if (found) {
val = attr.get(“GGG”) ; attr.remove(“GGG”) ;
}return val;
attr.put(“GGG”, o);
removeAttribute(“K”) {Attribute val = null;
found = attr.containsKey(“K”) if (found) {
val = attr.get(“K”) ; attr.remove(“K”) ;
}return val;
attr.put(“K”, o’);
removeAttribute(“K”) {Attribute val = null;
found = attr.containsKey(“K”)
if (found) {return val;
attr.put(“K”, o’);
removeAttribute(“P”) {Attribute val = null;
found = attr.containsKey(“P”) if (found) {
val = attr.get(“P”) ; attr.remove(“P”) ;
}return val;
attr.put(“P”, o’);
removeAttribute(“P”) {Attribute val = null;
found = attr.containsKey(“P”) if (found) {
val = attr.get(“P”) ; attr.remove(“P”) ;
}return val;
attr.put(“P”, o’);
removeAttribute(“O”) {Attribute val = null;
found = attr.containsKey(“O”) if (found) {
val = attr.get(“O”) ; attr.remove(“O”) ;
}return val;
attr.put(“O”, o’);
removeAttribute(“LL”) {Attribute val = null;
found = attr.containsKey(“LL”) if (found) {
val = attr.get(“LL”) ; attr.remove(“LL”) ;
}return val;
attr.put(“LL”, o’);
removeAttribute(“L”) {Attribute val = null;
found = attr.containsKey(“L”) if (found) {
val = attr.get(“L”) ; attr.remove(“L”) ;
}return val;
attr.put(“L”, o’);
removeAttribute(“G”) {Attribute val = null;
found = attr.containsKey(“G”) if (found) {
val = attr.get(“G”) ; attr.remove(“G”) ;
}return val;
attr.put(“G”, o’);
removeAttribute(“R”) {Attribute val = null;
found = attr.containsKey(“R”) if (found) {
val = attr.get(“R”);
attr.remove(“R”) ;}
return val;
attr.put(“R”, o’);
attr.remove(“R”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Influence Base Environment
Running Example
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
null
null
o
null
null
o
null
COLT
program
CO extractor
candidateCOs
Timeout
instrument linearizability
checking
COkey/value driver
Non-Lin
Execution
library spec
influence driver
Benchmark
• Used simple static analysis to extract composed operations– 19% needed manual modification
• Extracted 112 composed operations from 55 applications– Apache Tomcat, Cassandra, MyFaces – Trinidad, etc…
• Extracted all composed operations per application
• We did not find additional composed operations– Using Google Code and Koders
112Unknown
59Non
Linearizable
53Unknown
53Unknown
42Non
Linearizable
17Open Non
Linearizable
17Open Non
Linearizable
42Non
Linearizable
27Linearizable
26Globals
Globals
…
…
……
…
……
…
…
……
…
…
…
…
……
…
m.put(k,v)
…
27Linearizable
85Non-linearizable
Easy Detection
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
Current Work
• Define a class of data-independent composed operations
• Use small model reduction to prove linearizability of data-independent composed operations
• Empirical study for the ratio of real life data-independent composed operations
Summary• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence
• Sweet spot– Identify many important bugs together with a traces showing and
explaining the violations– Hard to find– Simple and efficient technique