Download - This One Time, at PL Camp
![Page 1: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/1.jpg)
This One Time, at PL Camp ...
Summer School on Language-Based Techniques for Integrating with the External World
University of Oregon
Eugene, Oregon
July 2007
![Page 2: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/2.jpg)
Checking Type Safety of Foreign Function Calls
Jeff Foster
University of Maryland
Ensure type safety across languages• OCaml/JNI – C
Multi-lingual type inference system• Representational types
SAFFIRE• Multi-lingual type inference system
![Page 3: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/3.jpg)
Dangers of FFIs
In most FFIs, programmers write “glue code”• Translates data between host and foreign languages
• Typically written in one of the languages
Unfortunately, FFIs are often easy to misuse• Little or no checking done at language boundary
• Mistakes can silently corrupt memory
• One solution: interface generators
![Page 4: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/4.jpg)
Example: “Pattern Matching”
if (Is_long(x)) { if (Int_val(x) == 0) /* B */ ... if (Int_val(x) == 1) /* D */ ...
} else {
if (Tag_val(x) == 0) /* A */ Field(x, 0) = Val_int(0)
if (Tag_val(x) == 1) /* C */ Field(x, 1) = Val_int(0)}
type t = A of int| B| C of int * int| D
![Page 5: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/5.jpg)
Garbage Collection
C FFI functions need to play nice with the GC• Pointers from C to the OCaml heap must be registered
value bar(value list) { CAMLparam1(list); CAMLlocal1(temp); temp = alloc_tuple(2); CAMLreturn(Val_unit);
}
Easy to forget Difficult to find this error with testing
![Page 6: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/6.jpg)
Multi-Lingual Types
Representational Types• Embed OCaml types in C types and vice versa
![Page 7: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/7.jpg)
SAFFIRE
Static Analysis of Foreign Function InteRfacEs
![Page 8: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/8.jpg)
Programming Models for Distributed Computing
Yannis Smaragdakis
University of Oregon
NRMI: Natural programming model for distributed computing.
J-Orchestra: Execute unsuspecting programs over a network, using program rewriting.
Morphing: High-level language facility for safe program transformation.
![Page 9: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/9.jpg)
NRMI
Identify all reachable
4
9 7
1 3
t
alias1
alias2 4
9 7
1 3
tree
Client side Server sideNetwork
![Page 10: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/10.jpg)
NRMI
Execute remote procedure
4
9 7
1 3
t
alias1
alias2 4
0 9
1 8
tree
Client side Server sideNetwork
2
tmp
![Page 11: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/11.jpg)
NRMI
Send back all reachable
4
9 7
1 3
t
alias1
alias2 4
0 9
1 8
tree
Client side Network
2
![Page 12: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/12.jpg)
NRMI
Match reachable maps
4
9 7
1 3
t
alias1
alias2 4
0 9
1 8
tree
Network
2
![Page 13: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/13.jpg)
NRMI
Update original objects
4
0 9
1 8
t
alias1
alias2 4
0 9
1 8
tree
Network
2
![Page 14: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/14.jpg)
NRMI
Adjust links out of original objects
4
0 9
1 8
t
alias1
alias2 4
0 9
1 8
tree
Network
2
![Page 15: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/15.jpg)
NRMI
Adjust links out of new objects
4
0 9
1 8
t
alias1
alias2 4
0 9
1 8
tree
Network
2
![Page 16: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/16.jpg)
NRMI
Garbage collect
4
0 9
1 8
t
alias1
alias2
Network
2
![Page 17: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/17.jpg)
J-Orchestra
Automatic partition system Works as bytecode compiler
• lots of indirection using proxies, interfaces, local and remote objects
Partitioned program equivalent to original
![Page 18: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/18.jpg)
Morphing
Ensure program generators are safe Statically check the generator to determine
the safety of any generated program, under All inputs• ensure that genrated programs compile
Early approach – SafeGen• Using theorem provers
MJ• Using types
![Page 19: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/19.jpg)
Fault Tolerant Computing
David August and David Walker Princeton University
Processors are becoming more susceptible to intermittent faults.• Moore’s Law, radiation• Alter computation or state, resulting in incorrect
program execution. Goal: Build reliable systems from unreliable
components.
![Page 20: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/20.jpg)
Topics
Transient faults and mechanisms designed to protect against them (HW).
The role of languages and compilers may play in creating radiation hardened programs.
New opportunities made possible by languages which embrace potentially incorrect behavior.
![Page 21: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/21.jpg)
Causes
![Page 22: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/22.jpg)
Software/Compiler
Duplicate instructions and check at important locations (store) [SWIFT, EDDI]
![Page 23: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/23.jpg)
λzap
λ calculus with fault tolerance• Intermediate language for compilers• Models single fault• Based on replication
Semantics model type of faults
let x = 2 inlet y = x + x inout y
let x1 = 2 inlet x2 = 2 inlet x3 = 2 inlet y1 = x1 + x1 inlet y2 = x2 + x2 inlet y3 = x3 + x3 inout [y1,y2,y3]
![Page 24: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/24.jpg)
Testing
![Page 25: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/25.jpg)
Typing Ad Hoc Data
Kathleen Fisher
AT&T Labs
PADS project*
• Data Description Language (DDL)• Data Description Calculus (DDC)• Automatic inference of PADS descriptions
*http://padsproj.org
![Page 26: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/26.jpg)
PADS
Declarative description of data source:• Physical format information• Semantic constraints
type responseCode = { x : Int | 99 < x < 600}
Pstruct webRecord { Pip ip; " - - ["; Pdate(’:’) date; ":"; Ptime(’]’) time; "]"; httpMeth meth; " "; Puint8 code; " "; Puint8 size; " "; };Parray webLog { webRecord[] };
![Page 27: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/27.jpg)
Raw Data
ASCII log files Binary Tracesstruct { ........ ...... ...........}
Data Description
XML
CSV
Standard formats & schema;
Visual Information
End-user tools
Learning
Problem: Producing useful tools for ad hoc data takes a lot of time.
Solution: A learning system to generate data descriptions and tools automatically.
![Page 28: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/28.jpg)
Format Inference Engine
Chunked Data
FormatRefinement
Tokenization
StructureDiscovery
ScoringFunction
IR to PADSPrinter
PADSDescription
Input File(s)
![Page 29: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/29.jpg)
Multi-Staged Programming
Walid Taha
Rice University
Writing generic program that do not pay a runtime overhead.• Use program generators• Ensure syntactic well-formed, well-typed
MetaOCaml
![Page 30: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/30.jpg)
The Abstract View
P2P1
I1P
Batch
I2
I2
I2
![Page 31: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/31.jpg)
MetaOCaml
Brackets (.< >.)• delay execution of an expression
Escape (.~ )• Combine smaller delayed values to construct
larger ones
Run (.! )• Compile and execute the dynamically generated
code
![Page 32: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/32.jpg)
Power Example
let rec power (n , x) = match n with 0 → 1 | n → x * (power (n-1, x));;
let power2 (x) = power (2, x);;let power2 = fun x → power (2, x);;
let power2 (x) = 1*x*x;
let rec power (n, x) = match n with 0 → .<1>. | n → .<.~x * ~(power (n-1, x))>.;;let power2 = .! .<fun x → .~(power (2, .<x>.))>.;;
![Page 33: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/33.jpg)
Scalable Defect Detection
Manuvir Das, Daniel Wang,
Zhe Yang, Microsoft Research
Program analysis at Microsoft scale• scalability, accuracy
Combination of weak global analysis and slow local one (for some regions of code)
Programmers are requires to add interface annotations• some automatic inference is available
![Page 34: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/34.jpg)
Web and Database Application Security
Zhendong Su
University of California-Davis
Static analyses for enforcing correctness of dynamically generated database queries.
Runtime checking mechanisms for detecting SQL injection attacks;
Static analyses for detecting SQL injection and cross-site scripting vulnerabilities.
![Page 35: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/35.jpg)
XML and Web Application Programming
Anders Møller
University of Aarhus
Formal models of XML schemas• Expressiveness of DTD, XML Schema, Relax NG
Type checking XML transformation languages• “Assuming that X is valid according to Sin is T(x)
valid according to Sout?”
Web application frameworks• Java Servlets and JSP, JWIG, GWT
![Page 36: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/36.jpg)
Types for Safe C-Level Programming
Dan Grossman
University of Washington
Cyclone, a safe dialect for C• Designed to prevent safety violations (buffer
overflow, memory management, …)
Mostly underlying theory• Types, expression, memory regions
![Page 37: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/37.jpg)
Analyzing and Debugging Software
Understanding Multilingual Software [Foster]• Parlez vous OCaml?
Statistical Debugging [Liblit]• you are my beta tester, and there’s lots of you
Scalable Defect Detection [Das, Wang, Yang]• Microsoft programs have no bugs
![Page 38: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/38.jpg)
Programming Models
Types for Safe C-Level Programming [Grossman]• C without the ick factor
Staged Programming [Taha]• Programs that produce programs that produce programs...
Prog. Modles for Dist. Comp. [Smaragdakis]• We’ve secretly replaced your centralized program with a
distributed application. can you tell the difference?
![Page 39: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/39.jpg)
The Web
Web and Database Application Security [Su]• How not to be pwn3d by 1337 haxxors
XML and Web Application Programming [Møller]• X is worth 8 points in scrabble...let’s use it a lot
![Page 40: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/40.jpg)
Other Really Important Stuff
Fault Tolerant Computing [August, Walker]• Help, I’ve been hit by a cosmic ray!
Typing Ad Hoc Data [Fisher]• Data, data, everywhere, but what does it mean?
![Page 41: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/41.jpg)
Statistical Debugging
Ben Liblit
University Of Wisconsin-Madison
![Page 42: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/42.jpg)
Statistical Debugging & Cooperative Bug Isolation• Observe deployed software in the
hands of real end users• Build statistical models of success &
failure• Guide programmers to the root causes
of bugs• Make software suck less
What’s This All About?
![Page 43: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/43.jpg)
Motivation
“There are no significantbugs in our released software
that any significant numberof users want fixed.”
Bill Gates, quoted in FOCUS Magazine
![Page 44: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/44.jpg)
Software Releases in the Real World
[Disclaimer: this may be a caricature.]
![Page 45: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/45.jpg)
Software Releases in the Real World
1. Coders & testers in tight feedback loop• Detailed monitoring, high repeatability• Testing approximates reality
2. Testers & management declare “Ship it!”• Perfection is not an option• Developers don’t decide when to ship
![Page 46: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/46.jpg)
Software Releases in the Real World
3. Everyone goes on vacation• Congratulate yourselves on a job well done!• What could possibly go wrong?
4. Upon return, hide from tech support• Much can go wrong, and you know it• Users define reality, and it’s not pretty
– Where “not pretty” means “badly approximated by testing”
![Page 47: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/47.jpg)
Testing as Approximation of Reality
Microsoft’s Watson error reporting system• Crash report from 500,000 separate programs• x% of software causes 50% of bugs• Care to guess what x is?
1% of software errors causes 50% of user crashes Small mismatch ➙ big problems (sometime) Big mismatch ➙ small problem? (sometime!)
• Perfection is not an economically viable option
![Page 48: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/48.jpg)
Real Engineers Measure Things;Are Software Engineers Real Engineers?
![Page 49: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/49.jpg)
“The major difference between a thing that might go wrong and a thing that
cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be
impossible to get at or repair.”
Instrumentation Framework
Douglas Adams, Mostly Harmless
![Page 50: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/50.jpg)
Bug Isolation Architecture
ProgramSource
Compiler
ShippingApplicatio
nSampler
Predicates
Counts& /
StatisticalDebugging
Top bugs withlikely causes
![Page 51: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/51.jpg)
Each behavior is expressed as a predicate P on program state at a particular program point.
Count how often “P observed true” and“P observed” using sparse but fair random samples of complete behavior.
Model of Behavior
![Page 52: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/52.jpg)
ProgramSource
Compiler
ShippingApplicatio
nSampler
Predicates
Counts& /
StatisticalDebugging
Top bugs withlikely causes
Predicate Injection:Guessing What’s Interesting
![Page 53: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/53.jpg)
Branch Predicates Are Interesting
if (p)…
else…
![Page 54: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/54.jpg)
if (p)// p was true (nonzero)
else
// p was false (zero)
Syntax yields instrumentation site Site yields predicates on program behavior Exactly one predicate true per visit to site
Branch Predicate Counts
![Page 55: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/55.jpg)
Returned Values Are Interesting
n = fprintf(…);
Did you know that fprintf() returns a value? Do you know what the return value means? Do you remember to check it?
![Page 56: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/56.jpg)
n = fprintf(…);
// return value < 0 ?// return value == 0 ?// return value > 0 ?
Syntax yields instrumentation site Site yields predicates on program behavior Exactly one predicate true per visit to site
Returned Value Predicate Counts
![Page 57: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/57.jpg)
Pair Relationships Are Interesting
int i, j, k;…i = …;
![Page 58: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/58.jpg)
Pair Relationship Predicate Counts
int i, j, k;…i = …;
// compare new value of i with…// other vars: j, k, …// old value of i// “important” constants
![Page 59: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/59.jpg)
Many Other Behaviors of Interest
Assert statements• Perhaps automatically introduced, e.g. by CCured
Unusual floating point values• Did you know there are nine kinds?
Coverage of modules, functions, basic blocks, …
Reference counts: negative, zero, positive, invalid
Kinds of pointer: stack, heap, null, …
Temporal relationships: x before/after y
More ideas? Toss them all into the mix!
![Page 60: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/60.jpg)
Observation stream observation count• How often is each predicate observed true?• Removes time dimension, for good or ill
Bump exactly one counter per observation• Infer additional predicates (e.g. ≤, ≠, ≥) offline
Feedback report is:
1. Vector of predicate counters
2. Success/failure outcome label
Still quite a lot to measure• What about performance?
Summarization and Reporting
![Page 61: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/61.jpg)
ProgramSource
Compiler
ShippingApplicatio
nSampler
Predicates
Counts& /
StatisticalDebugging
Top bugs withlikely causes
Fair Sampling Transformation
![Page 62: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/62.jpg)
Sampling the Bernoulli Way
Decide to examine or ignore each site…• Randomly• Independently• Dynamically
Cannot be periodic: unfair temporal aliasing Cannot toss coin at each site: too slow
![Page 63: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/63.jpg)
Amortized Coin Tossing
Randomized global countdown• Small countdown upcoming sample
Selected from geometric distribution• Inter-arrival time for biased coin toss• How many tails before next head?• Mean sampling rate is tunable parameter
![Page 64: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/64.jpg)
Geometric Distribution
D = mean of distribution= expected sample density
1)1log(
))1,0(log(1
D
randcountdown
![Page 65: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/65.jpg)
Weighing Acyclic Regions
Break CFG into acyclic regions
Each region has:• Finite number of paths
• Finite max number of instrumentation sites
Compute max weight in bottom-up pass
1
2 1
1
1
2
3
4
![Page 66: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/66.jpg)
Weighing Acyclic Regions
Clone acyclic regions• “Fast” variant
• “Slow” variant
Choose at run time
Retain decrements on fast path for now• Stay tuned…
>4?
![Page 67: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/67.jpg)
Path Balancing Optimization
Decrements on fast path are a bummer• Goal: batch them up
• But some paths are shorter than others
Idea: add extra “ghost” instrumentation sites• Pad out shorter paths
• All paths now equal
1
2 1
1
1
2
3
4
![Page 68: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/68.jpg)
Path Balancing Optimization
Fast path is faster• One bulk counter
decrement on entry
• Instrumentation sites have no code at all
Slow path is slower• More decrements
Consume more randomness
1
2 1
1
1
2
3
4
![Page 69: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/69.jpg)
Optimizations
Identify and ignore “weightless” functions / cycles
Cache global countdown in local variable
Avoid cloning
Static branch prediction at region heads
Partition sites among several binaries
Many additional possibilities…
![Page 70: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/70.jpg)
What Does This Give Us?
Absolutely certain of what we do see• Subset of dynamic behavior• Success/failure label for entire run
Uncertain of what we don’t see
Given enough runs, samples ≈ reality• Common events seen most often• Rare events seen at proportionate rate
![Page 71: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/71.jpg)
Playing the Numbers Game
ProgramSource
Compiler
ShippingApplicatio
nSampler
Predicates
Counts& /
StatisticalDebugging
Top bugs withlikely causes
![Page 72: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/72.jpg)
Isolating a Deterministic Bug
Hunt for crashing bug in ccrypt-1.2
Sample function return values• Triple of counters per call site: < 0, == 0, > 0
Use process of elimination• Look for predicates true on some bad runs,
but never true on any good run
![Page 73: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/73.jpg)
Elimination Strategies
Universal Falsehood• Disregard P if |P| = 0 for all runs• Likely a predicate that can never be true
Lack of failing coverage• All predicates for S is |S|=0 for all failed runs• Site not reached in failing executions
Lack of failing example• |P|=0 for all failed executions• Need not be true for a failure to occur
Successful counterexample• |P|>0 on at least one successful run• Can be true without causing failure
![Page 74: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/74.jpg)
Winnowing Down the Culprits
1710 counters• 3 × 570 call sites
1569 zero on all runs• 141 remain
139 nonzero on at least one successful run
Not much left!• file_exists() > 0• xreadline() == 0
0 500 1000 1500 2000 2500 30000
20
40
60
80
100
120
140
Number of successful trials used
Nu
mb
er
of "
go
od
" fe
atu
res
left
![Page 75: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/75.jpg)
Multiple, Non-Deterministic Bugs
Strict process of elimination won’t work• Can’t assume program will crash when it should
• No single common characteristic of all failures
Look for general correlation, not perfect prediction
Warning! Statistics ahead!
![Page 76: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/76.jpg)
Ranked Predicate Selection
Consider each predicate P one at a time• Include inferred predicates (e.g. ≤, ≠, ≥)
How likely is failure when P is true?• (technically, when P is observed to be true)
Multiple bugs yield multiple bad predicates
![Page 77: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/77.jpg)
Some Definitions
)()()(
)(
0 with runs successful # )(
0 with runsfailing # )(
PFPSPF
PBad
PPS
PPF
![Page 78: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/78.jpg)
Are We Done? Not Exactly!
Bad(f = NULL)= 1.0Bad(f = NULL)= 1.0
![Page 79: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/79.jpg)
Are We Done? Not Exactly!
Predicate (x = 0) is innocent bystander• Program is already doomed
Bad(f = NULL)= 1.0Bad(f = NULL)= 1.0
Bad(x = 0) = 1.0Bad(x = 0) = 1.0
![Page 80: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/80.jpg)
Crash Probability
Identify unlucky sites on the doomed path
Background risk of failure for reaching this site, regardless of predicate truth/falsehood
)()()(
)(PPFPPS
PPFPContext
![Page 81: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/81.jpg)
Isolate the Predictive Value of P
Does P being true increase the chance of failure over the background rate?
Formal correspondence to likelihood ratio testing
)()()( PContextPBadPIncrease
![Page 82: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/82.jpg)
Increase Isolates the Predictor
Increase(f = NULL)= 1.0
Increase(f = NULL)= 1.0
Increase(x = 0) = 0.0Increase(x = 0) = 0.0
![Page 83: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/83.jpg)
It Works!
…for programs with just one bug.
Need to deal with multiple bugs• How many? Nobody knows!
Redundant predictors remain a major problem
Goal: isolate a single “best” predictor for each bug, with no prior knowledge
of the number of bugs.
![Page 84: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/84.jpg)
Multiple Bugs: Some Issues
A bug may have many redundant predictors• Only need one, provided it is a good one
Bugs occur on vastly different scales• Predictors for common bugs may dominate, hiding
predictors of less common problems
![Page 85: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/85.jpg)
Bad Idea #1: Rank by Increase(P)
High Increase but very few failing runs These are all sub-bug predictors
• Each covers one special case of a larger bug Redundancy is clearly a problem
![Page 86: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/86.jpg)
Bad Idea #2: Rank by F(P)
Many failing runs but low Increase Tend to be super-bug predictors
• Each covers several bugs, plus lots of junk
![Page 87: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/87.jpg)
A Helpful Analogy
In the language of information retrieval• Increase(P) has high precision, low recall• F(P) has high recall, low precision
Standard solution:• Take the harmonic mean of both• Rewards high scores in both dimensions
![Page 88: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/88.jpg)
Rank by Harmonic Mean
Definite improvement• Large increase, many failures, few or no
successes But redundancy is still a problem
![Page 89: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/89.jpg)
Redundancy Elimination
One predictor for a bug is interesting• Additional predictors are a distraction• Want to explain each failure once
Similar to minimum set-cover problem• Cover all failed runs with subset of predicates• Greedy selection using harmonic ranking
![Page 90: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/90.jpg)
Simulated Iterative Bug Fixing
1. Rank all predicates under consideration
2. Select the top-ranked predicate P
3. Add P to bug predictor list
4. Discard P and all runs where P was true• Simulates fixing the bug predicted by P• Reduces rank of similar predicates
5. Repeat until out of failures or predicates
![Page 91: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/91.jpg)
Not Covered Today
Visualization of Bug Predictors
• Simple visualization may help reveal trends
Increase(P)
S(P)
error bound
log(F(P) + S(P))
Context(P)
![Page 92: This One Time, at PL Camp](https://reader035.vdocument.in/reader035/viewer/2022062802/56814511550346895db1d427/html5/thumbnails/92.jpg)
Not Covered Today
Reconstruction of failing paths.• Bug predictor is often the smoking gun, but not
always.• Want short, feasible path that exhibits bug.
– “Just because it’s undecidable doesn’t mean we don’t need an answer.”