testing concurrent programs, a 7-minute jargon-free introduction
DESCRIPTION
Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction. Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010. Program bugs. Slow computers. Computer Annoyances. Unit Testing. Test program parts, not the whole program Test parts individually, then together - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/1.jpg)
Testing Concurrent Programs, A 7-Minute Jargon-Free
IntroductionThesis Writing Seminar
Mathias Ricken
Rice University
February 25, 2010
![Page 2: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/2.jpg)
2
Computer Annoyances
• Program bugs • Slow computers
![Page 3: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/3.jpg)
3
Unit Testing
• Test program parts, not the whole program– Test parts individually, then together– Smaller parts larger parts
• Unit tests are programs– Compare expected result to actual result:
assertEquals(12, multiply(3,4));
![Page 4: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/4.jpg)
4
Moore’s Law
Adopted fromSutter 2009
![Page 5: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/5.jpg)
5
Existing Approaches Fail
• Simple Unit Testing Frameworks– Example: JUnit– Assume program is single-threaded– Many errors go undetected
detected
not detected
Main thread:
Other thread:
![Page 6: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/6.jpg)
6
Problem: Non-Determinism
• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is
non-deterministic
![Page 7: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/7.jpg)
7
Problem: Non-Determinism
• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is
non-deterministic
– Program could succeed in one order but fail in another
![Page 8: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/8.jpg)
8
Comprehensive Tools: Costly
• Model checking or schedule-based execution– Examples: JPF, ExitBlock– Test all arrangements– Too costly
t = # of threads
s = # of sync. points
N = # of arrangements
t s N2 1 62 2 202 3 703 2 16803 4 7.6E+053 8 2.3E+114 8 2.1E+19
Longer than the universe exists!
(if N=1 1 minute)
![Page 9: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/9.jpg)
9
• Insert random delays before synchronization points to modify the order
• Run tests many times with different delays
Probabilistic Approach
![Page 10: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/10.jpg)
10
Probabilistic Approach
• Insert random delays before synchronization points to modify the order
• Run tests many times with different delays
delay
![Page 11: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/11.jpg)
11
(Proposed) Contributions
• Probabilistic Approach– Random delays at synchronization points
• Call Graph Analysis– Determine which parts of the program may interact– Only insert delays where necessary
• Improved JUnit Framework– Exception handler for all threads– Ensures that all errors in all threads are detected
![Page 12: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/12.jpg)
12
Evaluation
• Tested DrJava with Improved JUnit– Number of unit tests: 900– Previously unknown problems: 20– Slowdown: ~1 percent
• Other parts still under development
![Page 13: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/13.jpg)
13
Conclusion
• Unit testing of concurrent programs will become more important
• Concutest helps…– by improving JUnit’s abilities– by using probabilistic techniques– by using call graph analysis to reduce the
number of synchronization points
![Page 14: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/14.jpg)
14
Image Attribution1. Left image on Computer Annoyances:
Adapted from Microsoft
2. Graph on Moore’s Law:Adapted from Herb Sutter 2009
3. Top image on Multi-Cores and Concurrency:Adapted from Brian Goetz et al. 2006, Addison Wesley
4. Bottom image on Multi-Cores and Concurrency:Caption Fridays
![Page 15: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/15.jpg)
15
Extra Slides
![Page 16: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/16.jpg)
16
Multi-Cores and Concurrency
• Recent trends– Clock speed stagnated– Transistors still increasing– Reason: multi-core CPUs
• Implication– Must use concurrency to
benefit from modern CPUs– Concurrent programming is
difficult
![Page 17: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/17.jpg)
17
• Number of schedules (N)– t: # of threads, s: # of slices per thread
– Note: s is # of slices (between synchronization points)
Number of Arrangements
![Page 18: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction](https://reader036.vdocument.in/reader036/viewer/2022062520/568158bd550346895dc60516/html5/thumbnails/18.jpg)
18
Product of s-combinations
For thread 1: choose s out of ts time slicesFor thread 2: choose s out of ts-s time slices…For thread t-1: choose s out of 2s time slicesFor thread t-1: choose s out of s time slices
Writing s-combinations using factorial
Cancel out terms in denominator and next numerator
Left with (ts)! in numerator and t numerators with s!
Number of Arrangements