Verifying Atomicity via Data Independence

Ohad Shacham Yahoo Labs, Israel

Eran Yahav Technion, Israel

Guy Gueta Yahoo Labs, Israel

Alex Aiken Stanford University, USA

Nathan Bronson Stanford University, USA

Mooly Sagiv Tel Aviv University, Israel

Martin Vechev ETH, Zurich

Concurrent Data Structures

• Writing highly concurrent data structures is complicated• Modern programming languages provide efficient

concurrent collections with atomic operations

.

.

…

……

…

…

………

TOMCAT Motivating Example

attr = new HashMap();…

Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}

TOMCAT 5*.TOMCAT 6*.

Invariant: removeAttribute(name) returns the value it removes from attr or null

attr = new ConcurrentHashMap();…

Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

o

Violation Detection

• Testing Atomicity of Composed Concurrent Operation – OOPSLA’12

• Use library influence specification for POR• Many violations we found in real applications• Many were already fixed

Challenge

Verifying the atomicity of composed operations

…

……

…

…

………

Challenges in Verification

• What do we want to check?– Specifying software correctness

• Scalability of static checking– Large programs

• Many sources of unboundedness– Inputs– # threads– # operations

Challenges in Verification

• What do we want to check?– Specifying software correctness

• Scalability of static checking– Large programs

• Many sources of unboundedness– Inputs– # threads– # operations

Linearizability

• Check that composed operations are Linearizable [Herlihy & Wing, TOPLAS’90]– Returns the same result as some sequential run

Linearizability removeAttribute(“A”) {

Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val;

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ; if (found) {

val = attr.get(“A”); attr.remove(“A”);

} return val;

o

o

null

null

o

null

null

o

null

null

o

null

Challenges in Verification

• What do we want to check?– Specifying software correctness

• Scalability of static checking– Large programs

• Many sources of unboundedness– Inputs– # threads– # operations

Modular Checking

ModularityGenerates simple traces

Base linearizability Restrict generated traces

Enables Env control

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Modular Checking

Challenges in Verification

• What do we want to check?– Specifying software correctness

• Scalability of static checking– Large programs

• Many sources of unboundedness– Inputs– # threads– # operations

Modular Checking

RA(“A”)

RA(“F”)

p(“R”)

d(“R”)

d(“R”)

g(“B”)

…

Modular Checking

RA(“A”)

p(“R”)

d(“R”)

d(“R”)

g(“B”)

…

Modular Checking

RA(“A”)p(“A”)

d(“A”)

d(“A”)

Data Independence [Wolper, POPL’86]

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Attribute removeAttribute(String name) { Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Program behaves the same for all Inputs

Data Independence for CCC

• Wolper’s definition is too restrictive for real life CCC

• We defined a data independence of linearizability for CCC

A CCC is linearizable for a single input iff it is linearizable for every input

Data Independence Syntactic Rules

• Data Independence detection is in general undecidable• Syntactic rules that imply a program as data-independent

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Verifying Data Independent Operations

Data independent Verified using single input

Influence CO adds one value

Map elements are bounded

Single Mutation

Verifying data independent operations

• Small model reduction• Decidable when the local state is bounded• Explore all possible executions using:

– One input key and finite number of values– Influenced based environment uses single value

• Employ SPIN

Summary

• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence• Leverage data independence

• Prove the linearizability of several composed operations• Simple and efficient technique