Upload File

testing atomicity of composed concurrent operations ohad shacham tel aviv university nathan bronson...

  • Home
  • Documents
  • Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University
27
Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University Mooly Sagiv Tel Aviv University Martin Vechev ETH & IBM Research Eran Yahav Technion

Post on 21-Dec-2015

221 views

Category:

Documents


0 download

Report

Tags:

TRANSCRIPT

Page 1: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Testing Atomicity of Composed Concurrent Operations

Ohad Shacham Tel Aviv University

Nathan Bronson Stanford University

Alex Aiken Stanford University

Mooly Sagiv Tel Aviv University

Martin Vechev ETH & IBM Research

Eran Yahav Technion

Page 2: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Concurrent Data Structures

• Writing highly concurrent data structures is complicated

• Modern programming languages provide efficient concurrent collections with atomic semantics

.

.

…

……

…

…

………

Page 3: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Challenge

Testing the atomicity of composed operations

…

……

…

…

………

Page 4: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

TOMCAT Motivating Example

attr = new HashMap();…

Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}

TOMCAT 5*.TOMCAT 6*.

Invariant: removeAttribute(name) returns the value it removes from attr or null

attr = new ConcurrentHashMap();…

Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}

Page 5: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

o

Page 6: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Linearizability removeAttribute(“A”) {

Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val;

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ; if (found) {

val = attr.get(“A”); attr.remove(“A”);

} return val;

o

o

null

null

o

null

null

o

null

null

o

null

Page 7: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Challenge

Testing the linearizabiliy of composed operations

…

……

…

…

………

Page 8: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

• Reconstruction in TOMCAT extremely challenging– Large traces– Large number of traces– Bugs occur in rare cases with specific key values

Page 9: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Our Solution

ModularityGenerates simple traces

Base linearizability Restrict generated traces

Influence Restrict generated traces

Enables Env control

Page 10: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Modular Checking

Page 11: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Challenge

Testing Linearizability of composed operation in a modular fashion

…

……

…

…

………

Page 12: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ;if (found) {return val;

removeAttribute(“B”) {Attribute val = null;

found = attr.containsKey(“B”) ;if (found) {return val;

removeAttribute(“GGG”) {Attribute val = null;

found = attr.containsKey(“GGG”) if (found) {return val;

attr.put(“A”, o);

removeAttribute(“GGG”) {Attribute val = null;

found = attr.containsKey(“GGG”) if (found) {

val = attr.get(“GGG”) ; attr.remove(“GGG”) ;

}return val;

attr.put(“GGG”, o);

removeAttribute(“K”) {Attribute val = null;

found = attr.containsKey(“K”) if (found) {

val = attr.get(“K”) ; attr.remove(“K”) ;

}return val;

attr.put(“K”, o’);

removeAttribute(“K”) {Attribute val = null;

found = attr.containsKey(“K”)

if (found) {return val;

attr.put(“K”, o’);

removeAttribute(“P”) {Attribute val = null;

found = attr.containsKey(“P”) if (found) {

val = attr.get(“P”) ; attr.remove(“P”) ;

}return val;

attr.put(“P”, o’);

removeAttribute(“P”) {Attribute val = null;

found = attr.containsKey(“P”) if (found) {

val = attr.get(“P”) ; attr.remove(“P”) ;

}return val;

attr.put(“P”, o’);

removeAttribute(“O”) {Attribute val = null;

found = attr.containsKey(“O”) if (found) {

val = attr.get(“O”) ; attr.remove(“O”) ;

}return val;

attr.put(“O”, o’);

removeAttribute(“LL”) {Attribute val = null;

found = attr.containsKey(“LL”) if (found) {

val = attr.get(“LL”) ; attr.remove(“LL”) ;

}return val;

attr.put(“LL”, o’);

removeAttribute(“L”) {Attribute val = null;

found = attr.containsKey(“L”) if (found) {

val = attr.get(“L”) ; attr.remove(“L”) ;

}return val;

attr.put(“L”, o’);

removeAttribute(“G”) {Attribute val = null;

found = attr.containsKey(“G”) if (found) {

val = attr.get(“G”) ; attr.remove(“G”) ;

}return val;

attr.put(“G”, o’);

removeAttribute(“R”) {Attribute val = null;

found = attr.containsKey(“R”) if (found) {

val = attr.get(“R”);

attr.remove(“R”) ;}

return val;

attr.put(“R”, o’);

attr.remove(“R”);

Page 13: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Influence Base Environment

Page 14: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Running Example

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Page 15: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Page 16: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val;

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ; if (found) {

val = attr.get(“A”); attr.remove(“A”);

} return val;

o

o

null

null

o

null

null

o

null

null

o

null

Page 17: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

COLT

program

CO extractor

candidateCOs

Timeout

instrument linearizability

checking

CO

key/value driver

Non-Lin

Execution

library spec

influence driver

Page 18: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Benchmark

• Used simple static analysis to extract composed operations– 19% needed manual modification

• Extracted 112 composed operations from 55 applications– Apache Tomcat, Cassandra, MyFaces – Trinidad, etc…

• Extracted all composed operations per application

• We did not find additional composed operations– Using Google Code and Koders

Page 19: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

112Unknown

Page 20: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

59Non

Linearizable

53Unknown

Page 21: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

53Unknown

42Non

Linearizable

17Open Non

Linearizable

Page 22: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

17Open Non

Linearizable

42Non

Linearizable

27Linearizable

26Globals

Page 23: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Globals

…

…

……

…

……

…

…

……

…

…

…

…

……

…

m.put(k,v)

…

Page 24: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

27Linearizable

85Non-linearizable

Page 25: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Easy Detection

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Page 26: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Current Work

• Define a class of data-independent composed operations

• Use small model reduction to prove linearizability of data-independent composed operations

• Empirical study for the ratio of real life data-independent composed operations

Page 27: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University

Summary

• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence

• Sweet spot– Identify many important bugs together with a traces showing and

explaining the violations– Hard to find– Simple and efficient technique


Compiler Construction Lexical Analysis Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University

Graduation Program - Stanford Law School - Stanford University

Introduction to Islamic Codicology Stanford University, Stanford

Compiler Construction 0368-3133 Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

Compiler Construction Recap Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University

STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY · STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY STANFORD, CALIFORNIA 94305 SGP-TR-35 SECOND ANNUAL REPORT TO U.S. DEPARTMENT OF

STANFORD UNIVERSITY STANFORD LINEAR ACCELERATOR …

Stanford Universitykendric/PDF/A44.pdf · Stanford University

Digital Forensics at Stanford University Stanford

Compiler Construction Semantic Analysis I Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

Simon Fraser University, Stanford University, Carnegie Mellon ...ryantibs/papers/covariance-aos.pdfSimon Fraser University, Stanford University, Carnegie Mellon University and Stanford

Stanford Universityrehall/Explorations in Gold...Stanford University

stanford continuing studies Stanford University Libraries

Compiler Construction Intermediate Representation I Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

Compiler Construction LR(0) + Visitor Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

Compiler Construction Recap Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

Linda Lim Stanford University Stanford Synchrotron ... · Structure Refinement Linda Lim Stanford University Stanford Synchrotron Radiation Lightsource Stephens’ Law – “A Rietveld

Department of Biochemistry, Stanford University, Stanford ... myosin VI motors.pdfSivaraj Sivaramakrishnan and James A. Spudich Department of Biochemistry, Stanford University, Stanford,

STANFORD GEOTHERiMAL PROGRAiM STANFORD UNIVERSITY

SessionJuggler - Stanford Crypto Group - Stanford University

Routledge - Stanford University · 2017. 4. 24. · Daniel L. Schwartz Stanford University Robb Lindgren Stanford University Sarah Lewis Stanford University Constructivism is a theory

Compiler Construction Parsing Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University

STANFORD UNIVERSITY · 2020-07-02 · STANFORD UNIVERSITY UNIVERSITY ARCHITECT / CAMPUS PLANNING AND DESIGN 340 BONAIR SIDING • STANFORD UNIVERSITY, CA 94305-8442 materially impair

STANFORD UNIVERSITY - Bondholder Informationbondholder-information.stanford.edu/pdf/BudgetBookFY13.pdf · stanford university budget plan 2012/13 stanford university budget plan 2012/13

Stanford Universityweb.stanford.edu/~dlabaree/publications/Public_Goods_Private_Goo… · Stanford University

Stanford BaSketBall - Stanford University

Compiler Construction Parsing I Ran Shaham and Ohad Shacham School of Computer Science Tel-Aviv University

STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY … · STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY Stanford Geothermal Program Interdisciplinary Res ear ch in Engineering and

Columbia University - Stanford University

Stanford University

STANFORD UNIVERSITY Contributory Retirement …collegeretirementjournal.org/wp-content/uploads/2012/03/Stanford...STANFORD UNIVERSITY Contributory Retirement Plan ... the Stanford

Department of Physics, Stanford University, Stanford, CA

Compiler Construction 0368-3133 Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University

Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford

STANFORD UNIVERSITY STANFORD LINEAR ACCELERATOR