![Page 1: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/1.jpg)
Leveraging DTrace for runtime verification
Carl Martin Rosenberg 1 Martin Steffen 2 Volker Stolz 2,3
September 28, 20161Simula Research Laboratory
2Inst. for Informatikk, Universitetet i Oslo
3Inst. for Data- og Realfag, Høgskolen i BergenNorway
![Page 2: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/2.jpg)
Context: Runtime Verification
![Page 3: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/3.jpg)
System“Buffers should never overflow”
“Every request gets an answer”
Desired properties
“Variables should never enter an inconsistent state”
1
![Page 4: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/4.jpg)
Monitor
Trace extractor
Systembeing
analyzed
Monitor generator
Specificationformula
ACCEPT REJECT INCONCLUSIVE
Trace
2
![Page 5: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/5.jpg)
Overview
• Goal: Evaluate DTrace’s suitability for RV.• Contribution: graphviz2dtrace, a monitor synthesis tool.• We evaluate the tool on two case studies.
3
![Page 6: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/6.jpg)
DTrace
![Page 7: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/7.jpg)
• DTrace is a system-wide instrumentation framework.• Originally written for the Sun Solaris 10 operating system, nowavailable for for Mac OS X, FreeBSD and othersystems [Gregg and Mauro, 2011].
4
![Page 8: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/8.jpg)
DTrace’s two most compelling features
1. DTrace provides facilities for dynamic tracing.2. DTrace gives a unified view of the whole system.
5
![Page 9: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/9.jpg)
DTrace Architecture
From Solaris Dynamic Tracing Guide, page 28 6
![Page 10: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/10.jpg)
Static and Dynamic Instrumentation
• DTrace allows for both static and dynamic instrumentation.• Dynamic providers: pid and fbt.• All other providers rely on static instrumentation artefacts.
7
![Page 11: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/11.jpg)
Static and Dynamic Instrumentation
• Developers can add their own instrumentation points.• Many prominent projects have static instrumentation points:PostgreSQL, Node.js, Apache, CPython etc.
8
![Page 12: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/12.jpg)
Using DTrace: The D scripting language
• Users interact with DTrace via D, a DSL.• Users specify actions that DTrace should take when an event ofinterest occurs.
9
![Page 13: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/13.jpg)
Using DTrace: The D scripting language
#!/usr/sbin/dtrace -qssyscall::read:entry /* probe *//execname != "dtrace" / /* predicate */{
printf("%s\n", execname);} /* action block */
10
![Page 14: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/14.jpg)
D has all the right building blocks for encodingFinite State Automata.
10
![Page 15: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/15.jpg)
Design and Implementation ofgraphviz2dtrace
![Page 16: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/16.jpg)
Basic idea 1: Associate atomic propositions inLTL specifications with DTrace probes.
10
![Page 17: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/17.jpg)
push → pid$target::push:entrypop → pid$target::pop:return
empty → pid$target::empty:return/arg1 == 1/
11
![Page 18: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/18.jpg)
Basic idea 2: Use standard techniques tocreate automata from specification formulas,
and encode automata in D.
11
![Page 19: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/19.jpg)
graphviz2dtrace
Mapping
D script
12
![Page 20: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/20.jpg)
Specification formalism: LTL3
![Page 21: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/21.jpg)
• LTL3[Bauer et al., 2006] gives a reasonable way of dealing withfinite traces.
• LTL3 is a three-valued variety of Linear Temporal Logic (LTL):Same syntax, different semantics.
• Key idea of LTL3: Identify good and badprefixes [Kupferman and Vardi, 2001].
13
![Page 22: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/22.jpg)
Good prefix
• A trace fragment u is a good prefix with respect to someproperty ϕ if ϕ holds in all possible futures following u.
14
![Page 23: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/23.jpg)
Bad prefix
• A trace fragment u is a bad prefix with respect to some propertyϕ if ϕ holds in no possible futures following u.
15
![Page 24: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/24.jpg)
LTL3 Semantics summarized
We can thus state the truth-value of an LTL3 formula ϕ with respectto a finite trace u as follows:
u|=3ϕ =
⊤ if u is a good prefix wrt. ϕ⊥ if u is a bad prefix wrt. ϕ? otherwise.
16
![Page 25: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/25.jpg)
Creating automata: LamaConv
• Bauer et al. give an algorithm for creatingLTL3-monitors [Bauer et al., 2011, 14:10-14:13]
• This algorithm is implemented in LamaConv1, which we makeuse of.
1http://www.isp.uni-luebeck.de/lamaconv
17
![Page 26: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/26.jpg)
18
![Page 27: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/27.jpg)
graphviz2dtrace
• In essence, graphviz2dtrace is compiles from LTL3-basedautomata to D scripts.
• The automaton’s transition function is encoded in an array, andthe state is stored in a variable.
• When an event occurs, the state of the automaton is updatedaccording to the transition function.
19
![Page 28: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/28.jpg)
Anticipation
• graphviz2dtrace creates anticipatory monitors thatterminate immediately upon finding a good or bad prefix.
• The scripts achieve this by understanding which state it is aboutto enter.
20
![Page 29: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/29.jpg)
Anticipation
pid$target::empty:return/ (arg1 == 1) && (state == 1)/{
trace("REJECTED");HAS_VERDICT = 1;exit(0);
}
21
![Page 30: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/30.jpg)
Monitor
Trace extractor
Systembeing
analyzed
Monitor generator
Specificationformula
ACCEPT REJECT INCONCLUSIVE
Trace
22
![Page 31: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/31.jpg)
ACCEPT REJECT INCONCLUSIVE
Specification formula in LTL3
graphviz2dtrace
Mapping
D script
Dtrace
System being analyzed
23
![Page 32: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/32.jpg)
Evaluation
![Page 33: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/33.jpg)
Case Studies
1. We dynamically instrument a faulty stack implementationwritten in C.
2. We investigate a Node.js web server interacting with aPostgreSQL database.
24
![Page 34: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/34.jpg)
104 105 106 107 108
10−2
10−1
100
101
102
0.003
0.067
0.6020.353 0.414
1.096
8.011
72.363
0.003
0.057
0.398
3.176
30.718
Iterations
Runn
ingtim
ein
seco
nds
Monitor overhead in Case 12
Uninstrumentedwith pid
with printf
2Averaged, measured with time, largest of real or user+sys
25
![Page 35: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/35.jpg)
Case 2
26
![Page 36: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/36.jpg)
Case 2
We want the following properties to hold:
1. The server should never send a response before thecorresponding database query is complete.
2. There should never be an HTTP request for which thecorresponding database query and HTTP response neverhappen.
27
![Page 37: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/37.jpg)
Hack: Use counters to keep track of queries
27
![Page 38: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/38.jpg)
Case 2
The server should never send a response before the correspondingdatabase query is complete:
Approximation: Number of sent responses should never exceednumber of queries:
2¬(nresponses > nqueries)
28
![Page 39: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/39.jpg)
Case 2
There should never be an HTTP request for which the correspondingdatabase query and HTTP response never happen:
Approximation: There should never be more than 100 pendingrequests:
2¬(((nrequests− nresponses) > 100) ∧ ((nrequests− nqueries) > 100))
29
![Page 40: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/40.jpg)
Case 2: Results
1. Monitor with counters detect violations of both properties.2. Screencast: https://vimeo.com/169585739
30
![Page 41: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/41.jpg)
Case 2: Performance Evaluation
0 20 40 60 80 1001,200
1,400
1,600
1,800
2,000
N concurrent connections
Mea
nproc
esse
drequ
ests
Mean processed requests per second at various concurrency levels3
MonitoredUnmonitored
3Averaged, measured with ab31
![Page 42: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/42.jpg)
Gregg’s dictum
Brendan Gregg [Straughan, 2012]
• ‘‘Don’t worry too much about pid provider probe cost at < 1000events/sec.’’
• ‘‘At > 10,000 events/sec, pid provider probe cost will benoticeable.’’
• ‘‘At > 100,000 events/sec, pid provider probe cost may bepainful.’’ [Gregg, 2011]
32
![Page 43: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/43.jpg)
Future Work
• Separate trace-generation from verification: Collect data withDTrace, evaluate with external process.
• Investigate mapping predicates rather than probes.• Steering systems can be created by using the system function.
33
![Page 44: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/44.jpg)
Concluding remarks
• Monitoring overhead is negligible when probe firings are below10 000 per second.
• graphviz2dtrace enables cross-process monitoring.• graphviz2dtrace-generated scripts are susceptible to raceconditions if probe firings may overlap.
34
![Page 45: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/45.jpg)
References I
Bauer, A., Leucker, M., and Schallhart, C. (2006).FSTTCS 2006: Foundations of Software Technology andTheoretical Computer Science: 26th International Conference,Kolkata, India, December 13-15, 2006. Proceedings, chapterMonitoring of Real-Time Properties, pages 260–272.Springer Berlin Heidelberg, Berlin, Heidelberg.
Bauer, A., Leucker, M., and Schallhart, C. (2011).Runtime verification for ltl and tltl.ACM Trans. Softw. Eng. Methodol., 20(4):14:1–14:64.
Gregg, B. (2011).DTrace pid Provider Overhead.http://dtrace.org/blogs/brendan/2011/02/18/dtrace-pid-provider-overhead/.
![Page 46: Leveraging DTrace for runtime verification€¦ · Leveraging DTrace for runtime verification Carl Martin Rosenberg 1 Martin Steffen 2Volker Stolz;3 September 28, 2016 1Simula Research](https://reader034.vdocument.in/reader034/viewer/2022050510/5f9aed3a42c6d01b3278b225/html5/thumbnails/46.jpg)
References II
Gregg, B. and Mauro, J. (2011).DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X, andFreeBSD.Prentice Hall Professional.Kupferman, O. and Vardi, M. Y. (2001).Model checking of safety properties.Formal Methods in System Design, 19(3):291–314.
Straughan, D. (2012).Brendan Gregg speaking at ZFS Day, Oct 2, 2012, San Francisco.(Own work) [CC BY-SA 3.0], via Wikimedia Commons.