andrewpetukhov_karimvaliev. you won’t believe that blind can see. benchmarking sql injection...

7/31/2019 AndrewPetukhov_KarimValiev. You wont believe that blind can see. benchmarking SQL injection scanners

1/13

You wont believe that blind can see:Benchmarking SQL injection

scannersAndrew Petukhov, Karim Valiev

Moscow State University

Thursday, May 24,


2/13

Results Our approach showed that theres no such thing as the

best scanner

Questions?

See you!!!

Thursday, May 24,


3/13

Motivation Pentesting and auditing

- p0wn vs fight for completeness

- good idea to collect the low hanging fruits first

- but what if a scanner finds nothing?

- are there really no vulns? this is a point when you want to know aboutlimitations of your tools

- and what about of a superposition of tools? I.e. skipfish then w3af thensqlmap

Look if we could do better than Larry Suto

Look if we could produce a sound testing

Thursday, May 24,


4/13

SQLi detection methods Error-based- a baseline method, should be good in every scanner

- should be no problems with implementation

Blind time-based- main problem: submit a payload which leaves a query syntactically correct

- SELECT id, date, text FROM news LIMIT ?, 1

Blind content-based

- main problem: compare true and false responses

- http://vulnapp/item?id=13 should be the same as http://vulnapp/item?id=13 and1=1 and differ from http://vulnapp/item?id=13 and 1=0

- what is the same for an automated tool?

Thursday, May 24,


5/13

What our research IS NOT It is not sponsored => Acunetix does not win :(((( It does not give an answer on stupid questions like what is

the best scanner?

- suppose you have two equal scanners implementing time-based technique- scanner A: if(substr(field,i,1) = a, sleep(5), 0) for all characters

- scanner B: if(substr(field,i,1) < n, sleep(5), 0) - dichotomy

- one minimizes scan time and the other - number of requests - which one is better?

It does not measure the crawling process- yes, we know that in point-and-shoot scenario it is crucial

Quantity metrics is not a final result

We do not expect tools to test for second order SQLi

Thursday, May 24,


6/13

Wait! Tell me more about metrics!

One could expect to measure scanners using FP/FN rates

Imagine: scanner A detects 80 vulns out of 100 and scannerB detects 20 out of 100

- Scanner A is better? You cannot tell!

- 80 test out of 100 were error-based and 20 out of 100 were blind

- If we generated 100 000 blind tests and 1000 error-based, FP/FN rates would change!

Thursday, May 24,


7/13

Wait! Tell me more about metrics! Now one could expect to measure scanners

using FP/FN rates on test case classes- Like (80%, 20%, 30%) for coverage and (0.1%, 0.9%, 0.2%) for FP

Scanner metrics become incomparable- consider (80%, 20%, 30%) vs (55%, 85%, 0%)

- you can hope that one result would inset the other {(80%, 20%,30%) could include (60%, 10%, 25%)}, but this does not happen

After all, who would establish these classes?- one needs good performance in injection after the LIMIT

statement and the other one - good performance in injection intoDML queries

Thursday, May 24,


8/13

Ideas behind SQLi bench

We wanted our test cases to be representative

We wanted our test cases to be as complete as possible

How would we do that?

Thursday, May 24,


9/13

Our approach

Thursday, May 24,


10/13

Classification! A model of a general workflow with DBMS interaction

- Get user input

- Validate user input

- Construct a query

- Perform a query and handle the result

- Construct and issue an HTTP response

Classification of each step

Test set is a permutation of all classes with each other(44 536 test cases)

Thursday, May 24,


11/13

NumbersVulnerable False positive

Vuln. witherror output

No err. outputand no sleep()

Total 28848 15688 23544 2652

arachni-0.3 15088 0 13876 0sqlmap-0.9 12246 1208 10771 447

sqlmap-r5059 10939 83 10152 102

wapiti-2.2.1 11280 0 10068 0

skipfish-2.03b 18110 1680 16038 1036

skipfish-2.06b 22142 3360 19062 1540

w3af-1.1 21972 450 21576 193

Burp Suite Pro 1.4.07 13459 294 13099 180

Thursday, May 24,


12/13

Some findings Error-based method - approximately equal

results

Blind method - all scanners perform bad Bugs: skipfish and sqlmap does not detect

sql-inj with output in HTTP header Fastest scanner: w3af

Thursday, May 24,


13/13

Contacts

Karim: [email protected] Andrew: [email protected]

Th d M 24
mailto:[email protected]:[email protected]

andrewpetukhov_karimvaliev. you won’t believe that blind can see. benchmarking sql injection...

Documents