Upload File

fairfuzz: a targeted mutation strategy for fuz! increasing ... · fuzzing in one slide university...

  • Home
  • Documents
  • FairFuzz: A Targeted Mutation Strategy for Fuz! Increasing ... · Fuzzing in One Slide University of California, Berkeley FairFuzz, presented by Caroline Lemieux 2 Fuzzer Program
71
FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage Caroline Lemieux, Koushik Sen University of California, Berkeley source: https://github.com/carolemieux/afl-rb Fzuz ?uzr Fair Fuzz Fuz!

Upload: others

Post on 07-Feb-2021

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

  • FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage

    Caroline Lemieux, Koushik SenUniversity of California, Berkeley

    source: https://github.com/carolemieux/afl-rb

    F z u z? u z r

    F a i r F u z zF u z !

  • The Rise of Fuzz Testing

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 1

    • Programs still have bugs.• Fuzz testing has become very popular in practice and theory

  • Fuzzing in One Slide

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 2

    Fuzzer Program

    Feedback

    Input

  • Fuzzing in One Slide

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 3

    Fuzzer Program

    Feedback

    A_VERY_BAD_INPUT

    Input

  • Fuzzing in One Slide

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 4

    Fuzzer Program

    Feedback

    A_VERY_BAD_INPUT

    Input

  • Fuzzing in One Slide

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 5

    Fuzzer Program

    Feedback

    A_VERY_BAD_INPUT

    A_VERY_BAD_INPUTThe input:

    causes a crash.

    Input

  • What Bugs Can Fuzzing Find?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 6

    • Most popular: basic correctness assertions (C/C++)• Segmentation faults• Anything address sanitizer can catch:

    • Buffer overflows• Use-after-frees• Etc…

  • Coverage-Guided (Greybox) Mutational Fuzzing

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 7

    Fuzzer Program

    Feedback(branches covered)

    Input

  • Coverage-Guided (Greybox) Mutational Fuzzing

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 8

    FuzzerSaved inputs

    Program

    Feedback(branches covered)

    Input

  • Coverage-Guided (Greybox) Mutational Fuzzing

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 9

    FuzzerSaved inputs

    Program

    Mutate saved input

    Feedback(branches covered)

    Input

  • Coverage-Guided (Greybox) Mutational Fuzzing

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 10

    FuzzerSaved inputs

    Program

    Mutate saved input

    New coverage? Save. Feedback(branches covered)

    Input

  • What’s Missing? Uneven Fuzzing Coverage

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 11

    Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    }// ...return process_result;

    }

  • What’s Missing? Uneven Fuzzing Coverage

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 12

    Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    }// ...return process_result;

    }

    Hit by 100k+ inputs

  • What’s Missing? Uneven Fuzzing Coverage

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 13

    Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    }// ...return process_result;

    }

    Hit by 100k+ inputsà Code under if well-covered

  • What’s Missing? Uneven Fuzzing Coverage

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 14

    Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    }// ...return process_result;

    }

    Hit by 100k+ inputs

    Hit by 1 input

    à Code under if well-covered

  • What’s Missing? Uneven Fuzzing Coverage

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 15

    Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    }// ...return process_result;

    }

    Hit by 100k+ inputs

    Hit by 1 input

    à Code under if well-covered

    à Code under if barely covered

  • Uneven Fuzzing Coverage à Uncovered Code

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 16

    Observation: some parts of the program easier to cover

    Result: some functionality wholly uncovered by fuzzing

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Hit by 100k+ inputs

    Hit by 1 input

    à Code under if well-covered

    à Code under if barely covered

  • Why So Uneven?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 17

    Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Input satisfying if:AT_LEAST_10_BYTES

  • AT_LEAST_10_BYS

    Why So Uneven?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 18

    Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Input satisfying if:AT_LEAST_10_BYTES

    BT_LEAST_10_BYTESAT???_LEAST_10_BYTES

    ✓✓

    ✓

  • Why So Uneven?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 19

    Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Input satisfying if:

  • Why So Uneven?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 20

    Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Input satisfying if:

  • Our Method: FairFuzz

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 21

    Program

    Feedback

    Input

    F z u z? u z r

    F a i r F u z zF u z !

    Utilize existing greybox infoTo target rarely-exercised code à increase coverage

  • Our Method: FairFuzz

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 22

    Program

    Feedback

    Input

    F z u z? u z r

    F a i r F u z zF u z !

    Utilize existing greybox infoTo target rarely-exercised code à increase coverage

    Identify: branches hit by few inputs (rare branches)

  • Our Method: FairFuzz

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 23

    Program

    Feedback

    Input

    F z u z? u z r

    F a i r F u z zF u z !

    Utilize existing greybox infoTo target rarely-exercised code à increase coverage

    Identify: branches hit by few inputs (rare branches)

    Identify: where input can be mutated and hit branch

  • Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 24

  • Recap: AFL

    • AFL [1]: Popular coverage-guided greybox fuzzer• Fuzzes programs taking in file or stdin• Easy to use (just compile program with afl-gcc or afl-clang)• Has found many bugs in practice

    [1] http://lcamtuf.coredump.cx/afl/

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 25

  • AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 26

    Seeds

  • AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 27

    Parent Input Set

    Seeds

  • AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 28

    Parent Input Set

    Parent Input

    Select parent to mutate

    Seeds

  • mutant

    AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 29

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    Select parent to mutate

    Create mutants

    Seeds

  • mutant

    mutant

    AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 30

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Run on instrumented

    program

    Seeds

  • mutant

    mutant

    AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 31

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Interesting MutantInteresting MutantInteresting Mutant

  • mutant

    mutant

    AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 32

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • mutant

    mutant

    AFL Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 33

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • AFL Mutation Types

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 34

    • Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”

    values, etc.

  • AFL Mutation Types

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 35

    • Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”

    values, etc.

    < ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D

    mutants

    …

  • AFL Mutation Types

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 36

    • Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”

    values, etc.

    • Random-location mutation• Repeat: choose random mutation, apply at random location

    < ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D

    mutants

    …

  • AFL Mutation Types

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 37

    • Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”

    values, etc.

    • Random-location mutation• Repeat: choose random mutation, apply at random location

    < ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D

    mutants

    …

    < ! A T T L I S T B D < ! A T T L 0 0 T B D < ? C A T L 0 0 T B D

    mutant

    …(repeat)

  • mutant

    mutant

    FairFuzz Method

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 38

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Compute mask

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • mutant

    mutant

    FairFuzz Method – Key Differences

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 39

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Compute mask

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • mutant

    mutant

    FairFuzz Method – Selecting Parent Inputs

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 40

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Compute mask

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • FairFuzz Method – Selecting Parent Inputs

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 41

    • Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch

  • FairFuzz Method – Selecting Parent Inputs

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 42

    • Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Hit by 100k+ inputs

    Hit by 1 input

    AT_LEAST_10_BYTES

  • FairFuzz Method – Selecting Parent Inputs

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 43

    • Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    Hit by 100k+ inputs

    Hit by 1 input

    AT_LEAST_10_BYTES

  • mutant

    mutant

    FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 44

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Compute mask

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 45

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 46

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 47

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 48

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D✗

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 49

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D< ? A T T L I S T B D

    ✗✗

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 50

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D< ? A T T L I S T B D...

    ✗✗

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 51

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D< ? A T T L I S T B D...

    < ! A T T L I S T ? D

    ✗✗

    ✓

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 52

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D< ? A T T L I S T B D...

    < ! A T T L I S T ? D

    ✗✗

    < ! A T T L I S T B ?✓✓

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

  • FairFuzz Method – Computing Branch Mask

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 53

    • Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch

    ? ! A T T L I S T B D< ? A T T L I S T B D...

    < ! A T T L I S T ? D

    ✗✗

    < ! A T T L I S T B ?✓✓

    int process_xml(char * fuzzed_data, int fuzzed_data_len) {

    if (fuzzed_data_len >= 10) {// more code

    }// ...if (starts_with(fuzzed_data, “

    if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code

    }}// ...return process_result;

    }

    < ! A T T L I S T B DParent input hits

    Mask: < ! A T T L I S T B D

  • mutant

    mutant

    FairFuzz Method – Targeting Mutations

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 54

    Parent Input Set

    Parent Input

    mutantmutantmutantMutant

    mutantmutantmutant

    Feedback

    Select parent to mutate

    Create mutants

    Compute mask

    Run on instrumented

    program

    Get mutants adding new coverage

    Seeds

    Add to parent set

    Interesting MutantInteresting MutantInteresting Mutant

  • FairFuzz Method – Targeting Mutations

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 55

    • Fixed-location mutation• Don’t produce mutants at locations in mask

    • Random-location mutation• Choose random locations outside mask

    < ! A T T L I S T B D

    < ! A T T L I S T B D

  • FairFuzz Method – Targeting Mutations

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 56

    • Fixed-location mutation• Don’t produce mutants at locations in mask

    • Random-location mutation• Choose random locations outside mask

    < ! A T T L I S T B DCan’t mutate✗ ✗

    …

    < ! A T T L I S T B D

  • FairFuzz Method – Targeting Mutations

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 57

    • Fixed-location mutation• Don’t produce mutants at locations in mask

    • Random-location mutation• Choose random locations outside mask

    < ! A T T L I S T B D

    < ! A T T L I S T 0 0 D < ! A T T L I S T 0 0

    ok to mutateCan’t mutate✗ ✗

    …

    < ! A T T L I S T B D

  • FairFuzz Method – Targeting Mutations

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 58

    • Fixed-location mutation• Don’t produce mutants at locations in mask

    • Random-location mutation• Choose random locations outside mask

    < ! A T T L I S T B D

    < ! A T T L I S T 0 0 D < ! A T T L I S T 0 0

    ok to mutateCan’t mutate✗ ✗

    …

    < ! A T T L I S T B D

    choose random position here

  • Evaluation

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 59

  • Evaluation – Tools Compared

    • FairFuzz: our tool, with highest-performing settings• AFL: vanilla AFL, default settings• FidgetyAFL: AFL with highest-performing settings• AFLFast.new: AFLFast with highest-performing settings

    [1] Zalewski, Michał. http://lcamtuf.coredump.cx/afl/[2] Böhme et al. Coverage-based Greybox Fuzzing as Markov Chain. CCS’16.

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 60

  • Evaluation - Benchmarksdjpegreadpngmutool drawxmllinttcpdumpc++filtobjdumpreadelfnm

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 61

    AFLFast benchmarks

    More complex input structures

    FidgetyAFL benchmarks

  • Evaluation Setup

    For each benchmark:• Run each technique 24hrs• Start with 1 valid seed file• No dictionaries• Repeat runs 20x• Calculated confidence intervals

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 62

  • Summary Results – Coverage Leaders

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 63

  • Summary Results – Coverage Leaders

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 64

    à FairFuzz achieves the highest coverage fast, for nearly all benchmarks

  • Branch Coverage Over Time

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 65

  • Where Does FairFuzz Perform Much Better?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 66

    Both are programs with nested conditional structure• tcpdump: if this packet type, then if

    has this field…• xmllint: byte-by-byte comparisons

  • Where Doesn’t FairFuzz Perform As Well?

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 67

    C++ name demangler: highly recursive-structured program• Covering different branches may

    not be best exploration strategy

  • Conclusion

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 68

    code: github.com/carolemieux/afl-rb slides: carolemieux.com/fairfuzz_ase18_slides.pdf

  • Branch Mask Performance

    For a subset of benchmarks, run a cycle with “shadow run”:• For each selected input, create mutants• (1) without branch mask• (2) without branch mask

    • Compare % of inputs hitting target branch:• Average over all inputs selected for mutation in cycle

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 69

  • Branch Mask Performance

    UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 70

    • Mask substantially increases % of inputs hitting target branch

    Fixed-Location Mutants Random-Location MutantsWith Mask Without Mask With Mask Without Mask

    xmllint 90.3% 22.9% 32.8% 2.9%tcpdump 98.7% 72.8% 36.1% 9.0%c++filt 96.6% 14.8% 34.4% 1.1%readelf 99.7% 78.2% 55.5% 11.4%readpng 97.8% 39.0% 24.0% 2.4%objdump 99.2% 66.7% 46.2% 7.6%


ImmuniWeb® Self-Fuzzer Firefox Extension

ZMT09M X LMTMT09M X-218 - Wing On Traveltoursimagery.wingontravel.com/Site/Upload/GroupTour/danzhang/... · q FuZ;+fB~B`fQ822G?r8rMtK+b+N~brB0fZ;t q FuD3AmXe41em:A~`srbEA~t q FuZ;GccBJ5+J08

Fuzzy Logic - University of Western Australiateaching.csse.uwa.edu.au/units/CITS4404/seminars/seminars17/A1 Fuz.… · References Scientific American. (2017). What is ‘fuzzy logic’?

IMF: Inferred Model-based Fuzzer

1 Amazing Scientific Discoveries Support the Creation Model Dr. Fazale “Fuz” Rana Reasons To Believe

ESBxxKITT01x-Specsheet€uz-HÆnz FUZ-HAm CTBRCB02JM02 SPECIFICATION INSTRUCTIONS Lighting Controller Output Wattage: 600W Max. Universal Voltage (120-277 VAC) 0-10V Luminaire General

Typestate-Guided Fuzzer for Discovering Use-after-Free

DotDotPwn Fuzzer - Black Hat 2011 (Arsenal)

Fuzzing’– WhatTheFuzz - EurOpeneuropen.cz › Proceedings › 38 › fuzzing europen.pdf · 2017-05-04 · – An Introduction to SPIKE, the Fuzzer Creation Kit. "– MSRPC Fuzzing

Hyntrospect: a fuzzer for Hyper-V devices

In the lands of corrupted elves - Breaking ELF software with Melkor fuzzer

Dowsing for over ows: a guided fuzzer to nd bu er boundary violations · Introduction Dowsing Input reduction Code reduction Summary Dowsing for over ows: a guided fuzzer to nd bu

[2012 CodeEngn Conference 06] beist - Everyone has his or her own fuzzer

OWASP WTE: Testing your way. · a collection of web app sec testing tools especially encoding/decoding a web application fuzzer for requests being made over HTTP and/or HTTPS. a fuzzer

PEACH FUZZER FOR DRIVER FUZZING WHITEPAPER · peachtech peach fuzzer for driver fuzzing whitepaper peach.tech

HyperFuzzer: An Efficient Hybrid Fuzzer for Virtual CPUs

FreeDom: Engineering a State-of-the-Art DOM Fuzzer

SNOOZE: Toward a Stateful NetwOrk prOtocol fuzZEr

Fusil the fuzzer -

Building Advanced Coverage-guided Fuzzer ... - Zeronights 2017 · Building Advanced Coverage-guided Fuzzer for Program Binaries NGUYEN Anh Quynh WEI Lei 17/11/2017

FOREST COVER CENTRAL Stanislaw J. Tajchmanl, Hailiang FuZ,

BanditFuzz: A Reinforcement-Learning based Performance Fuzzer …fmora/vstte20.pdf · 2020. 9. 5. · BanditFuzz: A Reinforcement-Learning based Performance Fuzzer for SMT Solvers

Chizpurfle: A Gray-Box Android Fuzzer for Vendor Service

FairFuzz: Targeting Rare Branches to Rapidly Increase

Building Advanced Coverage-guided Fuzzer for Program Binariesgroundx.io/docs/ZeroNights2017-darko-fuzzer.pdf · 2019-01-30 · Building Advanced Coverage-guided Fuzzer for Program

LZfuzz: a fast compression-based fuzzer for poorly documented protocolstrdata/reports/TR2008-634.pdf · 2013-12-23 · LZfuzz: a fast compression-based fuzzer for poorly documented

Amity Report – April 2012 Fuz hou Social Welfare …altrusa.info/trip_reports/Fuzhou-2012-04.pdfspeaking skills. She also has improved her ability to take care of herself. Fuz hou

ClusterFuzz - Nullcon · PDF fileClusterFuzz architect Fuzzer author and facilitator ... Glusterfs 1. Fuzzer data bundles 2. Build Cache 3. Code Coverage Appengine Local Storage 1

Writing a fuzzer - Twistlock...Writing a fuzzer Ariel Zelivansky @ Twistlock Labs with american fuzzy lop Technique for testing software by providing it with random, unexpected or

Automatic Fuzzer Generation FuzzGen

Implementing an USB Host Driver Fuzzer - TROOPERS

PEACH FUZZER FOR FIRMWARE FUZZING WHITEPAPER · peachtech peach fuzzer® for firmware fuzzing whitepaper peach.tech

Aspecte Contabile Si Fiscale Privind Fuz

FairFuzz: A Targeted Mutation Strategy for Increasing ...ksen/papers/fairfuzz.pdfFuzz Testing Coverage Caroline Lemieux University of California, Berkeley, USA [email protected]

FairFuzz: A Targeted Mutation Strategy for Increasing Greybox … · 2020-05-20 · One particular fuzz testing tool, American Fuzzy Lop (AFL), has become popular thanks to its ease-of-use