colt: effective testing of concurrent collection usage alex aiken nathan bronson stanford university...
Post on 20-Dec-2015
217 views
TRANSCRIPT
COLT: Effective Testing of Concurrent Collection Usage
Alex AikenNathan Bronson
Stanford University
Martin VechevEran Yahav
IBM Research
Mooly Sagiv
Ohad Shacham
Tel Aviv University
Concurrent Data Structures
• Writing highly concurrent data structures is complicated• Efficient concurrent collections with atomic operations provided
by libraries
• Atomic and efficient composition of several library operations often needed by the client– Efficiency and correctness are client responsibility
atomic V put(K,V) {…} atomic V get(K) {…} atomic V putIfAbsent(K,V) {…} atomic V remove(K) {…} atomic boolean remove (K,V) {…} atomic V replace(K,V) {…} atomic boolean replace (K,V, V’) {…} atomic boolean contains(V) {…} atomic boolean containsKey(K) {…}
V foo (key K) { Object val = m.get(K);
if (val != null){ val = m.remove(K);
} return val;
}
TOMCAT Motivating Example
Attribute removeAttribute(String name) { Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}
Coarse-Grain )TOMCAT 5(*.
Fine-Grain )TOMCAT 6(*.
Attribute removeAttribute(String name) { Attribute val = null;/* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); }/* } */ return val;}
Invariant: removeAttribute(name) returns the value it removes from attr or null
Attribute removeAttribute(String name) { Attribute val = null;
found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
removeAttribute(“A”) { Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
T1 T2 attr
Φ
“<}A”, o{>
Breaking the invariant:
removeAttribute(name) returns the
value it removes from attr or null
T1.val
o
null
TOMCAT Fix
Attribute removeAttribute(String name) { Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }}
Coarse-Grain (Before) Fine-Grain (After)
Attribute removeAttribute(String name){ Attribute val = attr.get(name) ; if (val != null) { val = attr.remove(name); } return val;}
• Very hard to reconstruct in TOMCAT– Specific thread interleaving– Same input key
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
removeAttribute(“A”) { Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
T1 T2 attr
Φ
“<}A”, o{>
T1.val
o
null
TOMCAT Motivating Example
Attribute removeAttribute(String name) {
Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
TOMCAT Motivating Example
• removeAttribute is a new operation of attr composed by attr’s operations
• Should be atomic in any client environment• Resilient for future client changes• Backed up by our experimental results
Attribute removeAttribute(String name) { Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
No operation on attr
attr atomic V put(K,V) {…} atomic V get(K) {…} atomic V putIfAbsent(K,V) {…} atomic V remove(K) {…} atomic boolean remove (K,V) {…} atomic V replace(K,V) {…} atomic boolean replace (K,V, V’) {…} atomic boolean contains(V) {…} atomic boolean containsKey(K) {…}
.
Memoization Example
V compute(K) { synchronized(m) { val = m.get(K); if (val == null) { val = calculateVal(K); // @Pure m.put (K, val); } return val; }}
Coarse-Grain
Fine-Grained Concurrent Clients
• Concurrent collections provide basic operations for fine-grained concurrency (read-check-modify, CAS like)
• V putIfAbsent(K, V)– Store >K, V< and return null or
return the current value if K already exists
• boolean replace(K, V, V’)– Replace >K, V< by >K,V’< if >K,V< already exists – Return true on success and false on failure
• boolean remove(K, V)– Remove >K,V< if its exists– Return true on success and false on failure
Memoization Example
V compute(K) { val = m.get(K); if (val == null) { val = calculateVal(K); // @Pure m.putIfAbsent(K, val); } return m.get(K);}
V compute(K) { synchronized(m) { val = m.get(K); if (val == null) { val = calculateVal(K); // @Pure m.put (K, val); } return val; }}
m.remove(K)
Coarse-Grain (Before) Fine-Grain (After)
Memoization Fix
• What do we mean by atomic?
// atomicV compute(K) {
val = m.get(K); if (val == null) {
val = calculateVal(K); // @Pure tmpVal = m.putIfAbsent(K, val); if (tmpVal != null)
val = tmpVal;}return val;
}
atomic implementation
of compute
Linearizability [Herlihy and Wing, TOPLAS'90]
V compute(K) {val = m.get(K); //@LP val != nullif (val == null) {
val = calculateVal(K); // @Pure tmpVal = m.putIfAbsent(K, val); //@LP if (tmpVal != null)
val = tmpVal;}return val;
}
val=m.get(7)compute(7) { … tmpVal=m.putIfAbsent(7,8) if (tmpVal != null) return val;
<{7,8}>
val=m.get(7)compute(7) { if (tmpVal != null) return val;
Thread Modular Linearizability
ConcurrentHashMap MUT
atomic V put(K,V) {…} atomic V get(K) {…} atomic V putIfAbsent(K,V) {…} atomic V remove(K) {…} atomic boolean remove (K,V) {…} atomic V replace(K,V) {…} atomic boolean replace (K,V, V’) {…} atomic boolean contains(V) {…} atomic boolean containsKey(K) {…}
.
V compute(K){ val = m.get(K) ; if (val == null){
val = calculateVal(K) ; m.putIfAbsent(K, val) ;
} return m.get(K);
}
Thread Modular Linearizability
atomic V put(K,V) {…} atomic V get(K) {…} atomic V putIfAbsent(K,V) {…} atomic V remove(K) {…} atomic boolean remove (K,V) {…} atomic V replace(K,V) {…} atomic boolean replace (K,V, V’) {…} atomic boolean contains(V) {…} atomic boolean containsKey(K) {…}
.
V compute(K){ val = m.get(K) ; if (val == null){
val = calculateVal(K) ; m.putIfAbsent(K, val) ;
} return m.get(K);
}
val=m.get(7)compute(7) { … tmpVal=m.putIfAbsent(7,8) return m.get(7)
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8)m.put(9,10)
val=m.get(12)compute(12) { … tmpVal=m.putIfAbsent(12,8) return m.get(12)m.put(19,12)
val=m.get(5)compute(5) { … tmpVal=m.putIfAbsent(5,13) return m.get(5)m.put(5,12)
val=m.get(20)compute(20) { … return m.get(20)m.put(20,10)
val=m.get(2)compute(2) { … tmpVal=m.putIfAbsent(2,8) return m.get(2)m.put(30,12) m.remove(20)
val=m.get(14)compute(14) { … tmpVal=m.putIfAbsent(14,8) return m.get(14)m.remove(14)
val=m.get(20)compute(20) { … return m.get(20)m.remove(14)m.put(9,10)
.
.
.
Thread Modular Linearizability
val=m.get(7)compute(7) { … tmpVal=m.putIfAbsent(7,8) return m.get(7)
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8)m.put(9,10)
val=m.get(12)compute(12) { … tmpVal=m.putIfAbsent(12,8) return m.get(12)m.put(19,12)
val=m.get(5)compute(5) { … tmpVal=m.putIfAbsent(5,13) return m.get(5)m.put(5,12)
val=m.get(20)compute(20) { … return m.get(20)m.put(20,10)
val=m.get(2)compute(2) { … tmpVal=m.putIfAbsent(2,8) return m.get(2)m.put(30,12) m.remove(20)
val=m.get(14)compute(14) { … tmpVal=m.putIfAbsent(14,8) return m.get(14)m.remove(14)
val=m.get(20)compute(20) { … return m.get(20)m.remove(14)m.put(9,10)
.
.
.
Hard to test even when modular
• Large number of interleavings• Bug exhibited very infrequently
val=m.get(7)compute(7) { … tmpVal=m.putIfAbsent(7,8) return m.get(7)
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8)m.put(9,10)
val=m.get(12)compute(12) { … tmpVal=m.putIfAbsent(12,8) return m.get(12)m.put(19,12)
val=m.get(5)compute(5) { … tmpVal=m.putIfAbsent(5,13) return m.get(5)m.put(5,12)
val=m.get(20)compute(20) { … return m.get(20)m.put(20,10)
val=m.get(2)compute(2) { … tmpVal=m.putIfAbsent(2,8) return m.get(2)m.put(30,12) m.remove(20)
val=m.get(14)compute(14) { … tmpVal=m.putIfAbsent(14,8) return m.get(14)m.remove(14)
val=m.get(20)compute(20) { … return m.get(20)m.remove(14)m.put(9,10)
.
.
.
Hard to test even when modular
Leveraging Commutativity
• Operations on different keys are guaranteed to commute• Cannot lead to linearizability violation• No need to try such interleavings• Use commutatively for partial order reduction at the
library level
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8)m.put(9,10)
m.put(9,10) val=m.get(8) … tmpVal=m.putIfAbsent(8,8) return m.get(8)Compute (8) {
m.put(9,10)val=m.get(8) … tmpVal=m.putIfAbsent(8,8) return m.get(8)compute(8) {
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8)m.put(9,10)
val=m.get(8)compute(8) { … tmpVal=m.putIfAbsent(8,8) return m.get(8) m.put(9,10)
Leveraging Commutativity
• To expose bugs, try operations that do not commute• Execute thread in adversarial environment
– Adversarial environment picks non-commutative operations– Adversarial scheduler uses non-commutative
• No bugs are lost due to the reductionMUT OperationConditionAdversarial Action
get(k)get(k) == nullput(k,*)
get(k)get(k) != nullremove(k),put(k,v) | v!=get(k)
putIfAbsent(k,v)get(k) == nullput(k,*)
putIfAbsent(k,v)get(k) != nullremove(k)
replace(k,oldV,v)get(k) == oldVremove(k)put(k,v’)| v’!= oldV,v
replace(k,oldV,v)get(k) != oldVput(k,oldV)
Implementing Dynamic Modular Analyzer
• Compare concurrent execution to a specific sequential execution• Concurrent execution emulated using an adversary• Check that every concurrent operation returns the same result as its
sequential counterpart
Potential linearizationpoint
operationMUT
Reference Method
compare results
...
Adversary
Potential linearizationpoint
compare results
operation
Running Example
V compute(K) { val = m.get(K); //@LP val != null if (val == null) { val = calculateVal(K); m.putIfAbsent(K, val); //@LP } return m.get(K);}
MUT Adversary Reference Method Map (m) op op Map (refM)
Φ compute(7) { Φ val = m.get(7)
if (val == null) val = calculateVal(7)
{(7,12)} m.put(7,12) != refM.put(7,12) {(7,12)} m.putIfAbsent(7, 14) compute(7) {
val = refM.get(7) if (val == null) return refM.get(7)
Φ m.remove(7) != refM.remove(7) Φ return m.get(7)
MUT OperationConditionAdversarial Action
get(k)get(k) == nullput(k,*)
get(k)get(k) != nullremove(k), put(k,v) | v!=get(k)
putIfAbsent(k,v)get(k) == nullput(k,*)
putIfAbsent(k,v)get(k) != nullremove(k)
replace(k,oldV,v)get(k) == oldVremove(k), put(k,v’)| v’!= oldV,v
replace(k,oldV,v)get(k) != oldVput(k,oldV)
V compute(K) { val = m.get(K); //@LP val != null if (val == null) { val = calculateVal(K); m.putIfAbsent(K, val); //@LP } return m.get(K);}
=!
12null
Benchmark
0
2
4
6
8
10
12
# M
UT
s L
NL
NLO
• Tested 89 MUTs from 48 applications– Apache’s Tomcat, Derby, Cassandra, My Faces – Trinidad,
etc…
Results
• Tested 89 MUTs from 48 applications– Apache’s Tomcat, Derby, Cassandra, My Faces – Trinidad, etc…
• Discovered 56 linearizability violations
• 6 reports were proved as linearizable and the adversary refined accordingly
• Other MUTs were manually proved linearizable
• Filed bug reports; many already fixed
• Random adversary failed to find even a single violation – in 10 hours
Reasons For Success
• Deterministic MUTs• For most MUTS
– A bug exists for a key iff a bug exists for every key– Control influences by collection’s operation result
• Does not value dependant
V compute(K) {val = m.get(K); if (val == null) {
val = calculateVal(K); // @Pure tmpVal = m.putIfAbsent(K, val); if (tmpVal != null)
val = tmpVal;}return val;
}
Future Work
• Verify thread modular linearizability– Small MUT– Use ADT semantics to remove implementation– Small model verification