· author: daniel teuchert created date: 2/26/2019 4:37:45 pm
TRANSCRIPT
Fishing for Deep Bugs with Grammars
Daniel Teuchert
Fuzzing
1
...
0
... 0
...
1
...
Daniel Teuchert
Fuzzing1
...
0
... 0
...
1
...
Daniel Teuchert
Fuzzing1
...
0
... 0
...
1
...
Daniel Teuchert
Fuzzing1
...
0
... 0
...
1
...
Daniel Teuchert
Fuzzing
1
...
0
... 0
...
1
...
Daniel Teuchert
Fuzzing1
...
0
... 0
...
1
...
Daniel Teuchert
AFL
0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
AFL0
...
1
...
0
...
1
...0
...
1
...
0
...
1
...
0
...
1
...
0
...
1
...
Daniel Teuchert
if !input.parse() { exit()}
if !input.check() { exit()}
do_stuff()
Daniel Teuchert
if !input.parse() { exit()}
if !input.check() { exit()}
do_stuff()
Daniel Teuchert
if !input.parse() { exit()}
if !input.check() { exit()}
do_stuff()
Daniel Teuchert
if !input.parse() { exit()}
if !input.check() { exit()}
do_stuff()
Daniel Teuchert
if !input.parse() { exit()}
if !input.check() { exit()}
do_stuff()
Daniel Teuchert
Grammars +
Feedback
Context-Free GrammarsPROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Grammars +
Feedback
Context-Free Grammars
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Grammars +
Feedback
Context-Free GrammarsPROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Grammars +
Feedback
Context-Free GrammarsPROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Grammars +
Feedback
Context-Free GrammarsPROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Grammars +
Feedback
Context-Free GrammarsPROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
PROG
STMT
VAR
a
= EXPR
NUM
1
"a=1"
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Design of Nautilus
InstrumentedBinary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queue
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Genera�on:
-Naive Genera�on
PROG → STMTPROG → STMT ; PROGSTMT → return 1STMT → VAR = EXPR
VAR → aEXPR →EXPR → EXPR + EXPR
→ 1 | 2
-Uniform Genera�on
Minimiza�on:
-Subtree Minimiza�on
PROG
STMT
VAR
a
= EXPR
NUM
1
PROG
STMT
Subtree Minimiza�on
return 1
-Recursion Minimiza�on
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
NUM
1
Recursive Minimiza�on
Muta�on:
-Random
-Rules
-Random Recursive
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
EXPR
NUM
1
+ EXPR
NUM
2
Random Recursive Mutation
-Splicing
-AFL
PROG
STMT
VAR
a
= EXPR
EXPR
NUM
1
+ EXPR
NUM
2
PROG
STMT
VAR
a
= EXPR
1xf
AFL Mutation
Daniel Teuchert
Evaluation
Targets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
Evaluation
Targets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?
mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?
mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
EvaluationTargets:
-mruby-PHP-lua-ChackraCore
Baseline AFL IFuzzer Nautilus15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
35.0%
ChakraCore
Baseline AFL Nautilus25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline AFL Nautilus2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline AFL Nautilus40.0%
45.0%
50.0%
55.0%
60.0%
65.0%
70.0%Lua
vs. AFL / IFuzzer
Baseline No feedback Naive gen Uniform gen15.0%
17.5%
20.0%
22.5%
25.0%
27.5%
30.0%
32.5%
ChakraCore
Baseline No feedback Naive gen Uniform gen25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
55.0%
mruby
Baseline No feedback Naive gen Uniform gen2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
PHP
Baseline No feedback Naive gen Uniform gen40.0%
45.0%
50.0%
55.0%
60.0%
Lua
Configurations
0 %
25 %
50 %
75 %
100 %
03m 6m 9m 12m
15m
20m
26m
33m
42m
51m 1h
1h 1
5m1h
30m
1h 4
5m 2h2h
15m
2h 3
0m2h
45m 3h
3h 3
0m 4h4h
30m 5h
5h 3
0m 6h 7h 8h 9h 10h
11h
12h
13h
14h
15h
16h
17h
18h
19h
20h
21h
22h
23h
GenerationSubtree Min.Recursion Min.Rules MutationAFL MutationSplicing MutationRandom MutationRandom Rec. Mut.
ObjectSpace.each do |a| begin a.method(...) rescue end end
Bugs?mruby:CVE-2018-10191: UAFCVE-2018-10199: UAFCVE-2018-11743: Use of Uninitialized PointerCVE-2018-12249: SEGVCVE-2018-12247: SEGVCVE-2018-12248: Heap Buffer OverflowStack Overflow
PHP:Division by ZeroSEGVStack Overflow
lua:UAF
ChakraCore:OOM Crash
Daniel Teuchert
Conclusion
- Grammars & Feedback ++
- Splicing is important!
Daniel Teuchert
Conclusion
- Grammars & Feedback ++
- Splicing is important!
Daniel Teuchert
Conclusion
- Grammars & Feedback ++
- Splicing is important!
Daniel Teuchert
OverviewGeneration
Minimization
MutationsInstrumented
Binary
Parser
InputGeneration
Minimization
Mutation
Scheduler
Queu
e
trigger
trigger
Grammar
𝑆 → 𝑥𝐴 | 𝑦𝑆𝐴 → 𝑦𝐴 | 𝑧𝐵𝐵 → 𝑧
Feedback
NAUTILUS
InstrumentationSource
Daniel Teuchert