concepts, algorithms and open problemssimon/places15-martinvechev.pdf · data race: two unordered...
TRANSCRIPT
![Page 1: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/1.jpg)
Martin Vechev
Commutativity Race Detection Concepts, Algorithms and Open Problems
Department of Computer Science ETH Zurich
![Page 2: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/2.jpg)
Concurrency is Everywhere
![Page 3: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/3.jpg)
Concurrency and Interference
Concurrency and Interference go together Some interference is good (e.g., coordination) Some interference is harmful (e.g., causes errors )
![Page 4: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/4.jpg)
Data Races
A generic notion of interference: data races Data race: two unordered operations from different threads access the same memory location, where one of the accesses is a write
![Page 5: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/5.jpg)
Example of a Data Race
PLACES p = new PLACES(); fork { p.talk = “Martin” } p.talk = “Simon”
1. two writes to p.talk 2. not ordered by program
fork
p.talk = “Martin”
p.talk = “Simon”
![Page 6: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/6.jpg)
2003 Northeast Blackout
concurrency error triggers massive power outage economic effect : ~ 6 billion $USD
![Page 7: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/7.jpg)
Android Data Races Undesirable effects
![Page 8: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/8.jpg)
<html> <head></head> <body> <script> var Gates = “great”;</script> </script> <img src=“img1.png” onload=“Gates=‘poor’;”> <img src=“img2.png” onload=“alert(Gates);”> </body> </html>
Data races: Web
![Page 9: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/9.jpg)
<html> <head></head> <body> <script> var Gates = “great”;</script> </script> <img src=“img1.png” onload=“Gates=‘poor’;”> <img src=“img2.png” onload=“alert(Gates);”> </body> </html>
Data races: Web
![Page 10: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/10.jpg)
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
Data races: Web
![Page 11: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/11.jpg)
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 12: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/12.jpg)
fetch img1.png
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 13: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/13.jpg)
fetch img1.png
fetch img2.png
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 14: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/14.jpg)
fetch img1.png
fetch img2.png
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 15: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/15.jpg)
fetch img1.png
fetch img2.png
Gates = great
great is read from Gates
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 16: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/16.jpg)
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
fetch img1.png
fetch img2.png
Gates = great
great is read from Gates
great
Data races: Web
![Page 17: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/17.jpg)
fetch img1.png
fetch img2.png
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 18: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/18.jpg)
fetch img1.png
fetch img2.png
img1.png loaded
Gates = great
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 19: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/19.jpg)
fetch img1.png
fetch img2.png
img1.png loaded
Gates = great
Gates = poor
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 20: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/20.jpg)
fetch img1.png
fetch img2.png
img1.png loaded
img2.png is loaded
Gates = great
Gates = poor
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 21: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/21.jpg)
fetch img1.png
fetch img2.png
img1.png loaded
img2.png is loaded
Gates = great
Gates = poor
poor is read from Gates
Data races: Web
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
![Page 22: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/22.jpg)
<html>
<head></head>
<body>
<script>
var Gates = “great”;</script>
</script>
<img src=“img1.png” onload=“Gates=‘poor’;”>
<img src=“img2.png” onload=“alert(Gates);”>
</body>
</html>
fetch img1.png
fetch img2.png
img1.png loaded
img2.png is loaded
Gates = great
Gates = poor
poor is read from Gates
poor
Data races: Web
![Page 23: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/23.jpg)
Martin Vechev
Veselin Raychev
Pavol Bielik
Manu Sridharan
Publications Effective Race Detection for Event-Driven Programs, OOPSLA’13
Race Detection for Web Applications, PLDI’12
Android Race Detection, M.Sc. Thesis, ETH
Stateless Model Checking for ER apps, PhD thesis, Aarhus
Tool Features Formal Model of Concurrency Scalable Graph Algorithms Guaranteed Races Bugs found and fixed Works with Chromium/V8
EventRacer: Concurrency Analysis of Event-Driven Apps
Julian Dolby
Anders Moeller
Casper Jensen
Momchil Velikov
Boris Petrov
People
Department of Computer Science More info: http://www.eventracer.org
http://www.eventracer.org/android
![Page 24: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/24.jpg)
Massive amount of work on data race detectors exploit structure for improved asymptotic complexity, parallelization, sampling, hardware acceleration, static optimizations, … Key Theorems • If the analysis reports a race on a given execution, then the race exists • If the analysis does not report a race on a given execution, then there is no race for that
input state (the input state of the execution) Recommended Reading MIT’s Cilk Race Detector for SP-graphs, ACM SPAA’97, M.Feng, C. Leiserson Race Detection for 2D partial orders, ACM SPAA’15, D. Dimitrov, M. Vechev, V. Sarkar FastTrack, ACM PLDI’09, S. Freund, C. Flanagan EventRacer, ACM OOPSLA’13, V. Raychev, M. Sridharan, M. Vechev
Dynamic Race Detection 15+ years of algorithms and optimizations
![Page 25: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/25.jpg)
Increasingly, low-level functionality captured inside high level objects
Trend: Interference shifts to interfaces
util.concurrent
![Page 26: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/26.jpg)
The gap
Interference at the interface
Classic data race detection Uses read-write notion of conflict Many optimizations
Current detectors increasingly ineffective in handling real-world programs
![Page 27: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/27.jpg)
Wanted
Interference at the interface
New kind of race detection Uses higher level notion of conflict Many optimizations
How do we bridge the gap in an elegant and clean manner?
![Page 28: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/28.jpg)
Dimitar Dimitrov
Commutativity Race Detection, ACM PLDI’14
Martin Vechev Veselin Raychev Eric Koskinen
Most of what follows is based on:
![Page 29: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/29.jpg)
Commutativity Race
two high-level operations do not commute
and are not ordered by the program
capture
![Page 30: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/30.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
![Page 31: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/31.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
size()/2
✔
![Page 32: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/32.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
put(‘a’,‘tom’)/nil
size()/1
size()/2
✔ put(‘b’,‘cat’)/nil
![Page 33: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/33.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
put(‘a’,‘tom’)/nil
size()/1
size()/0
put(‘a’,‘tom’)/nil
size()/2
✔ put(‘b’,‘cat’)/nil put(‘b’,‘cat’)/nil
![Page 34: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/34.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
put(‘a’,‘tom’)/nil
size()/1
size()/0
put(‘a’,‘tom’)/nil
size()/2
✔ put(‘b’,‘cat’)/nil put(‘b’,‘cat’)/nil
1. put and size do not commute
2. are not ordered by the program
![Page 35: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/35.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
1. put and size do not commute
2. are not ordered by the program
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
put(‘a’,‘tom’)/nil
size()/1
put(‘b’,‘cat’)/nil
![Page 36: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/36.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
put(‘a’,‘tom’)/nil
size()/1
size()/0
put(‘a’,‘tom’)/nil
size()/2
✔ put(‘b’,‘cat’)/nil put(‘b’,‘cat’)/nil
1. put and size do not commute
2. are not ordered by the program
![Page 37: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/37.jpg)
No Commutativity Races
1. put and size do not commute
2. are ordered by the program
joinAll
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil
size()/2
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } joinAll new Array [data.size()];
![Page 38: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/38.jpg)
To detect interference between concurrent events
Time: Need to efficiently capture event ordering Space: Interference between operations
Principles of Analysis
![Page 39: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/39.jpg)
commutativity specification
structural representation
commutativity race detector happens-before
Commutativity Race Detection
![Page 40: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/40.jpg)
commutativity specification
structural representation
commutativity race detector happens-before
Commutativity Race Detection
![Page 41: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/41.jpg)
Hashmap commutativity spec
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
size()/n1 ⋈ get(k2)/v2
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
get(k1)/v1 ⋈ get(k2)/v2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ v1 = r1
⟺ true
⟺ true
![Page 42: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/42.jpg)
Hashmap commutativity spec
put(k1,v1)/r1 ⋈ size()/n2 ⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
![Page 43: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/43.jpg)
Hashmap commutativity spec
put(k1,v1)/r1 ⋈ size()/n2 ⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
put(‘b’,‘lol’)/cat
⋈
⋈
put(‘a’,‘tom’)/nil size()/1
size()/1
![Page 44: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/44.jpg)
Commutativity Specifications Can be large and complex
“Verification of Semantic Commutativity Conditions and Inverse Operations on Linked Data Structures”, Kim & Rinard, ACM PLDI’11
Example: ArrayList (part of the spec)
![Page 45: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/45.jpg)
Commutativity Race Detection Naïve Algorithm
foreach b Seen { if not(op ⋈ b) and (op || b) report ‘commutativity race’ } Seen = Seen op
Initially Seen = Then during execution, for each operation op do:
![Page 46: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/46.jpg)
Commutativity Race Detection Naïve Algorithm
foreach b Seen { if not(op ⋈ b) and (op || b) report ‘commutativity race’ } Seen = Seen op
Initially Seen = Then during execution, for each operation op do:
Commute?
![Page 47: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/47.jpg)
Commutativity Race Detection Naïve Algorithm
foreach b Seen { if not(op ⋈ b) and (op || b) report ‘commutativity race’ } Seen = Seen op
Initially Seen = Then during execution, for each operation op do:
Commute? Ordered?
![Page 48: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/48.jpg)
Commutativity Race Detection Naïve Algorithm
foreach b Seen { if not(op ⋈ b) and (op || b) report ‘commutativity race’ } Seen = Seen op
Two problems: 1. Extremely inefficient: worst-case O(Seen) time per-operation
2. Does not explore commutativity sharing between operations
Initially Seen = Then during execution, for each operation op do:
Commute? Ordered?
![Page 49: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/49.jpg)
put(k1, v1)/nil
⋈
⋈ ⋈
Commutativity Race Detection Naïve Algorithm
Upon encountering size()/3 , we will perform 3 checks:
Yet, what matters is whether a concurrent resize has occurred
put(k1, v1)/nil put(k2, v2)/nil put(k3, v3)/nil size()/3 For trace:
Wanted: leverage sharing between observed operations
put(k2, v2)/nil
put(k3, v3)/nil
size()/3
![Page 50: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/50.jpg)
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
commutativity specification
structural representation
Commutativity Race Detection
![Page 51: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/51.jpg)
operations
size()/1 put(‘a’,‘tom’)/nil
Micro-ops representation
⋈
![Page 52: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/52.jpg)
resize w:’a’ size
µ µ
operations
size()/1 put(‘a’,‘tom’)/nil
µ-operations
Micro-ops representation
⋈
![Page 53: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/53.jpg)
Micro-ops representation
resize w:’a’ size
µ µ
conflict
operations
size()/1 put(‘a’,‘tom’)/nil ⋈
µ-operations
![Page 54: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/54.jpg)
size()/n1 ⋈ get(k2)/v2
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
get(k1)/v1 ⋈ get(k2)/v2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
⟺ true
⟺ true
resize w:’a’ size
size()/1 put(‘a’,‘tiger’)/nil
µ µ
conflict
vs.
From logic to µ-operations
![Page 55: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/55.jpg)
size()/n1 ⋈ get(k2)/v2
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
get(k1)/v1 ⋈ get(k2)/v2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
⟺ true
⟺ true
resize w:’a’ size
size()/1 put(‘a’,‘tiger’)/nil
µ µ
conflict
vs.
From logic to µ-operations
commutativity compiler
![Page 56: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/56.jpg)
put(k1, v1)/nil
⋈
⋈ ⋈
Commutativity Compiler Benefits
put(k2, v2)/nil
put(k3, v3)/nil
size()/3
w:k1
⋈ w:k2
w:k3
size
resize
conflict on operations
conflict on µ-operations
![Page 57: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/57.jpg)
+ put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
commutativity specification
structural representation
commutativity race detector happens-before
Commutativity Race Detection
![Page 58: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/58.jpg)
+ put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
commutativity specification
structural representation
commutativity race detector happens-before
Commutativity Race Detection
Next: brief review of vector clocks
![Page 59: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/59.jpg)
Vector Clocks Review
A vector clock VC is a map: VC Threads Nat Example: if we have 3 threads, a vector clock can be: 2,0,1 Intuitively, vector clocks are associated with operations and capture ordering (or lack of) between these operations
![Page 60: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/60.jpg)
Vector Clocks Operations
Compare 2,0,1 ⊑ 2,0,2 : operation with clock 2,0,1 happens before operation with clock 2,0,2 2,0,1 ⋢ 1,0,2 : operation with clock 2,0,1 is unordered with operation with clocks 1,0,2
![Page 61: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/61.jpg)
Vector Clocks Operations
Compare 2,0,1 ⊑ 2,0,2 : operation with clock 2,0,1 happens before operation with clock 2,0,2 2,0,1 ⋢ 1,0,2 : operation with clock 2,0,1 is unordered with operation with clocks 1,0,2 Join 2,0,1 1,0,2 = 2,0,2 : used to obtain the most up to date information when combining effects of two operations
![Page 62: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/62.jpg)
Vector Clocks Operations
Compare 2,0,1 ⊑ 2,0,2 : operation with clock 2,0,1 happens before operation with clock 2,0,2 2,0,1 ⋢ 1,0,2 : operation with clock 2,0,1 is unordered with operation with clocks 1,0,2 Join 2,0,1 1,0,2 = 2,0,2 : used to obtain the most up to date information when combining effects of two operations Increment inc ( 2,0,1 , 2 ) results in vector clocks 2,0,2: it advances time for thread 2. Done upon encountering synchronization operations (e.g., fork, lock, etc)
![Page 63: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/63.jpg)
Vector Clocks Operations
Compare 2,0,1 ⊑ 2,0,2 : operation with clock 2,0,1 happens before operation with clock 2,0,2 2,0,1 ⋢ 1,0,2 : operation with clock 2,0,1 is unordered with operation with clocks 1,0,2 Join 2,0,1 1,0,2 = 2,0,2 : used to obtain the most up to date information when combining effects of two operations Increment inc ( 2,0,1 , 2 ) results in vector clocks 2,0,2: it advances time for thread 2. Done upon encountering synchronization operations (e.g., fork, lock, etc) Vector clocks form a lattice
![Page 64: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/64.jpg)
+ put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
commutativity specification
structural representation
commutativity race detector happens-before
Commutativity Race Detection
Key idea: associate vector clocks with µ-operations
![Page 65: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/65.jpg)
Commutativity Races
ConcurrentHashMap data = new ConcurrentHashMap(); for (String key : keys) fork { data.put(key, Value.compute(key)); } new Array [data.size()];
1. put and size do not commute
2. are not ordered by the program
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
![Page 66: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/66.jpg)
put(‘a’,‘tom’)/nil fork put(‘b’,‘cat’)/nil size()/1 fork trace
An example
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
![Page 67: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/67.jpg)
An example
put(‘a’,‘tom’)/nil fork put(‘b’,‘cat’)/nil size()/1 fork
w:’a' resize size w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
![Page 68: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/68.jpg)
An example
put(‘a’,‘tom’)/nil fork put(‘b’,‘cat’)/nil size()/1 fork
w:’a' resize size w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
1,1,0 3,0,0 2,0,1 vc 1,0,0 2,0,0
![Page 69: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/69.jpg)
An example
put(‘a’,‘tom’)/nil fork put(‘b’,‘cat’)/nil size()/1 fork
w:’a' resize size w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
1,1,0 3,0,0 2,0,1 vc 1,0,0 2,0,0
seen vc
![Page 70: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/70.jpg)
An example
µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
vc
seen vc
![Page 71: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/71.jpg)
An example
fork
µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
vc 1,0,0
seen vc
![Page 72: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/72.jpg)
An example
put(‘a’,‘tom’)/nil
w:’a' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
1,1,0 vc
seen vc
![Page 73: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/73.jpg)
An example
put(‘a’,‘tom’)/nil
w:’a' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
1,1,0 vc
seen vc
w:’a'
resize
1,1,0 1,1,0
![Page 74: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/74.jpg)
An example
fork
µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
vc 2,0,0
seen vc
w:’a'
resize
1,1,0 1,1,0
![Page 75: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/75.jpg)
An example
size()/1
size µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
3,0,0 vc
seen vc
w:’a'
resize
1,1,0 1,1,0
![Page 76: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/76.jpg)
An example
size()/1
size µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
3,0,0 vc
seen vc
w:’a'
resize
⋢ 1,1,0 1,1,0
![Page 77: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/77.jpg)
An example
size()/1
size µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
3,0,0 vc
seen vc
w:’a'
resize
⋢ 1,1,0 1,1,0
![Page 78: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/78.jpg)
An example
size()/1
size µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
3,0,0 vc
seen vc
w:’a'
resize
size
⋢ 1,1,0 1,1,0 3,0,0
![Page 79: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/79.jpg)
An example
put(‘b’,‘cat’)/nil
w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
2,0,1 vc
seen vc
w:’a'
resize 3,0,0
size
⋢ 1,1,0 1,1,0
![Page 80: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/80.jpg)
An example
put(‘b’,‘cat’)/nil
w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
2,0,1 vc
seen vc
w:’a'
resize
size
⋢ ⋢ 1,1,0
3,0,0
2,1,1
![Page 81: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/81.jpg)
An example
put(‘b’,‘cat’)/nil
w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
2,0,1 vc
seen vc
w:’a'
resize
size
⋢ ⋢ 1,1,0
3,0,0
2,1,1
![Page 82: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/82.jpg)
An example
put(‘b’,‘cat’)/nil
w:’b' resize µ
trace
fork
fork put(‘a’,‘tom’)/nil
put(‘b’,‘cat’)/nil size()/1
2,0,1 vc
seen vc
w:’a'
resize
size
w:’b'
⋢ ⋢ 1,1,0
3,0,0 2,0,1
2,1,1
![Page 83: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/83.jpg)
Asymptotic complexity
O(n) conflict checks per encountered operation
Can we do better?
![Page 84: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/84.jpg)
Extended Logical Fragment (ECL)
A logical fragment which ensures O(1) checks per operation
![Page 85: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/85.jpg)
S ::= V1 ≠ V2 | S S | true | false
Extended Logical Fragment (ECL) Variables of first method
Variables of second method
![Page 86: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/86.jpg)
S ::= V1 ≠ V2 | S S | true | false
Extended Logical Fragment (ECL) Variables of first method
Variables of second method
put(k1,v1)/r1 ⋈ get(k2)/v2 ⟺ k1 ≠ k2 ∨ v1 = r1
Example:
![Page 87: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/87.jpg)
S ::= V1 ≠ V2 | S S | true | false
Extended Logical Fragment (ECL) Variables of first method
Variables of second method
put(k1,v1)/r1 ⋈ get(k2)/v2 ⟺ k1 ≠ k2 ∨ v1 = r1
Example:
Limited relationship between variables of different methods. For instance, V1 = V2 not allowed
![Page 88: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/88.jpg)
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false
Extended Logical Fragment (ECL)
Any predicate over V1
Any predicate over V2
![Page 89: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/89.jpg)
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false
Extended Logical Fragment (ECL)
Any predicate over V1
Any predicate over V2
put(k1,v1)/r1 ⋈ get(k2)/v2 ⟺ k1 ≠ k2 ∨ v1 = r1
Example:
![Page 90: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/90.jpg)
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false X ::= S | B | X X | X B
Extended Logical Fragment (ECL)
Not X X
![Page 91: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/91.jpg)
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false X ::= S | B | X X | X B
Extended Logical Fragment (ECL)
Not X X
put(k1,v1)/r1 ⋈ get(k2)/v2 ⟺ k1 ≠ k2 ∨ v1 = r1
Example:
Comes from S Comes from B
![Page 92: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/92.jpg)
ECL ensures O(1) time per operation Extends SIMPLE in Kulkarni et al. PLDI ’11
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false X ::= S | B | X X | X B
Extended Logical Fragment (ECL)
![Page 93: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/93.jpg)
Classic R/W Race Detection…
read()/r1 ⋈ read()/r2
read()/r1 ⋈ write(v2)
write(v1) ⋈ write(v2)
⟺ true
⟺ false
⟺ false
Fits inside ECL
…is now a special case:
![Page 94: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/94.jpg)
What about this?
…a more precise spec:
NO!
read()/r1 ⋈ read()/r2
read()/r1 ⋈ write(v2)
write(v1) ⋈ write(v2)
⟺ true
⟺ r1 = v2
⟺ v1 = v2
ECL does not allow = between variables of different operations
![Page 95: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/95.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Commutativity race detection
commutativity specification
structural representation
commutativity race detector happens-before
![Page 96: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/96.jpg)
Experimental Results
Implemented a commutativity detector for Java
Found both correctness and performance bugs in large-scale apps (e.g., H2 Database)
![Page 97: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/97.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Commutativity race detection
commutativity specification
structural representation
commutativity race detector happens-before
Parametric Concurrency Analysis Framework
Generalizes classic read-write race detection
Enables concurrency analysis for clients of high-level abstractions
Logical fragment ensures O(1) complexity
![Page 98: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/98.jpg)
Applications
Software Defined Networks Stateless Model Checking Eventual Consistency Optimistic Concurrency Control
![Page 99: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/99.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
![Page 100: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/100.jpg)
Question 1: Can we learn the commutativity spec?
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
size()/n1 ⋈ get(k2)/v2
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
get(k1)/v1 ⋈ get(k2)/v2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true ⟺ k1 ≠ k2 ∨ v1 = r1
⟺ true
⟺ true
Specification of a Map ADT
Automatic Inference
?
![Page 101: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/101.jpg)
Question 1: Can we learn the commutativity spec?
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
size()/n1 ⋈ get(k2)/v2
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
get(k1)/v1 ⋈ get(k2)/v2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true ⟺ k1 ≠ k2 ∨ v1 = r1
⟺ true
⟺ true
Specification of a Map ADT
Dimitar Dimitrov Timon Gehr
Learning Commutativity Specifications, CAV’15 Black Box Learning of specifications from examples Many open problems: state, quantifiers, minimization
Automatic Inference
?
![Page 102: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/102.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
can we learn it?
![Page 103: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/103.jpg)
Question 2: Logic Expressivity vs. Asymptotic Complexity
What is the richest logical fragment which guarantees O(1)? What about O(logN)?
S ::= V1 ≠ V2 | S S | true | false B ::= PV1 | PV2 | B | B B | B B | true | false X ::= S | B | X X | X B
Recall that ECL allows for O(1) checks per operation:
![Page 104: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/104.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
can we learn it? richest fragment to
obtain O(1) time?
![Page 105: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/105.jpg)
Question 3: Commutativity Compiler Optimizations
Obtaining a succinct micro-operation representation can be tricky New compiler optimizations which depend on the logical fragment and on the property we are checking
![Page 106: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/106.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
can we learn it? richest fragment to
obtain O(1) time?
compiler optimizations exploiting fragment
![Page 107: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/107.jpg)
Question 4 : Impossibility questions
Can a read-write race detector precisely detect commutativity races? At what cost?
What is the “consensus-like” hierarchy of concurrency analyzers?
read-write race detector
some magic implementation + = commutativity
race detector Any
Hashmap
+
Hashmap
?
![Page 108: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/108.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
can we learn it? richest fragment to
obtain O(1) time?
compiler optimizations exploiting fragment
impossibility questions
![Page 109: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/109.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Open Problems
commutativity specification
structural representation
commutativity race detector happens-before
can we learn it? richest fragment to
obtain O(1) time?
compiler optimizations exploiting fragment
impossibility questions
different instantiations (e.g. VCs, SP-graphs)
other analyzers (e.g. atomicity)
![Page 110: Concepts, Algorithms and Open Problemssimon/PLACES15-MartinVechev.pdf · Data race: two unordered operations from different threads access the same memory location, where one of the](https://reader033.vdocument.in/reader033/viewer/2022050611/5fb25e89d879941429477fdb/html5/thumbnails/110.jpg)
new {} fork put
size
put join
+
[0,0,0,0] [1,2,0,0] [0,1,0,3] [4,2,1,0]
put(k1,v1)/r1 ⋈ size()/n2
put(k1,v1)/r1 ⋈ put(k2,v2)/r2
put(k1,v1)/r1 ⋈ get(k2)/v2
size()/n1 ⋈ size()/n2
⟺ (v1 = nil ∧ r1 = nil) ∨ (v1 ≠ nil ∧ r1 ≠ nil)
⟺ true
⟺ k1 ≠ k2 ∨ (v1 = r1 ∧ v2 = r2)
⟺ k1 ≠ k2 ∨ v1 = r1
Commutativity race detection
commutativity specification
structural representation
commutativity race detector happens-before
Parametric Concurrency Analysis Framework
Generalizes classic read-write race detection
Enables concurrency analysis for clients of high-level abstractions
Logical fragment ensures O(1) complexity
Many open problems