Ensuring Security through Continuous Testing

Jeremy Faircloth

Who am I?• Security Guy• Enterprise Applications Admin/Architect• Development/Operations Manager• Author – 16 Published Titles

What’s this about?

• Uh… Ensuring security through continuous testing?

• Let’s break that down…

Background

• What are “vulnerabilities”?– Vulnerabilities are software bugs that allow the

software to behave in an unexpected manner.• So what?

– Vulnerabilities lead to “exploits”, the malicious usage of a vulnerability to disrupt or misuse the software.

• One of the functions of “IT Security” is the prevention of this malicious activity.– This can be through inclusion in the SDLC or

through controls that exist external to the SDLC.

You’re losing me…• What does a vulnerability look like and

how is it exploited?

Gettin’ Funky

I’m lazy.

What’s the impact?

Source:informationisbeautiful.net

Security = Important… Got it!

Whose job is it?

The security guys!!! The developers!!!

What are developers doing today?• The developers are trained on security (a little)

and are told to use “secure development patterns”.

• Does this work?– Security is NOT a developer priority – features are.– Developer skill and knowledge varies – and

security is very specialized.– Development means “figuring out how to make it

work”. When it’s working then security elements are typically added.

What are security guys doing today?

• While the development is happening, sometimes “static code analysis” is done.– But few developers understand the results.

• When development and testing are complete, the application is scanned with “dynamic analysis” and “penetration testing”.

• Security-related bugs are found and quickly patched by development just before the application goes to Production.– Usually by developers who have already moved on to

building the next feature.

This works!• Well defined secure coding patterns help

ensure secure code.• Scanning and remediating issues helps

harden the application.• Having individuals trained in security

performing development and interpreting scan results means more secure code.

…until it doesn’t.

• Good developers with strong security knowledge have a high turnover rate.

• Future application updates tend to introduce new vulnerabilities… and aren’t always subject to full security scanning.– SDLCs move FAST now!

• Security bugs are usually classified as less important than bugs in functionality.

What about all that “security stuff”?

• Security is changing…– The “perimeter” is gone.– The world is moving faster.– The attackers are getting better.

• “With more attacks, more applications, less control, and less time to work, I can’t keep up.” – A Security Guy

What’s the solution?

TESTERS!!!

Why just the security guys?

• Many tools used by the security guys can be used by anyone as long as they can interpret the results.

• This is no different from other forms of application testing; it’s just different use cases.

• Tools + knowledge = Better testing results!

Three steps to paradise!

• Teach security to testers and provide tools.

1

• Develop security-related use cases and test.

2

• Record the defects correctly.3

Teach testers security and provide tools.

• Tools are important, but interpreting results is even more important.– Many false positives

…and assumed false positives

– Code vs. infrastructure– False negatives ?!?

Develop security-related use cases and test.• Negative/misuse cases:

– A misuse case is the negative form of a use case.

– A use case is a countermeasure to a misuse case.

• Examples!!!– Single quote– Field size– Unexpected input

A better exampleRequirement: The field shall accept numbers between 1 and 600

Positive Use Cases:– Enter 1– Enter 600– Enter 10 numbers between 1 and 600

Negative Use Cases:– Enter -1– Enter 0– Enter 601– Enter A– Enter 1.1– Enter @#$– Enter ¿– Enter nothing (blank)– Enter ‘– Enter <script>alert();</script>1

Record the defects correctly

• Security bugs should be HIGH priority (Sev 1)• Developers need bugs presented in a way they

understand– Bad Bug: Field XYZ is vulnerable to SQL injection– Good Bug: Field XYZ is accepting input outside of the

required format/length causing a critical error

• Sending bugs like this back to developers forces them to create more secure code

Why testers are our heroes

• Security guys know security, but not the application.– Testers can test for vulnerabilities in process flows.– Testers know the application better than anyone… and

know how to break it.– Tools only get you so far…

• Testers have “early access” to the application– The earlier a vulnerability is caught, the better the fix will

be.• If testers can test for common vulnerabilities,

security guys can focus on the less common or more complex.

What’s this about?

• Ensuring security through continuous testing.– Ensuring security:

• Identifying vulnerabilities• Recording security defects correctly• Forcing development of secure code

– through continuous testing.• Creating negative use cases• Testing for common vulnerabilities• Including security testing earlier in the SDLC• Testing security use cases constantly

How do we do this?

• If you’re a tester…– Learn more about security!

• Attend security courses• Learn how to hack!• Try to break things

– Encourage your security guys to work with you.• Some security guys don’t like to share… sorry.• Most do! Learn from them!

– Work with your management to change how testing is done.• Include negative use cases• Include use cases that focus on security

How do we do this?

• If you lead a testing team…– Learn more about security!

• Attend security courses• Learn how to hack!• Try to break things

– Encourage your security guys to work with your team• Forge partnerships between security and testing• Include security guys in conversations about testing processes

– Change how testing is done• Train your testers on security• Help them to understand negative use cases from the security

perspective• Support them in including security testing

Summary

• Vulnerabilities = Bad

• Testing = Good

• Testing for Vulnerabilities = Very Good

• Secure Applications = GREAT!

Questions?

THANK YOU!

Negative Use Cases (examples)

• Web-Based Applications– Send request/responses out of order– Modify contents of fields in queries– Modify POST data or cookie values– Send invalid or malformed values– Change source information (HTTP referrer, MAC

address, IP address, username, etc.) in responses– Change parts of the URL that might be “useful” (ID

fields, etc.)– Change XML responses– Modify web service calls

Negative Use Cases (examples)

• Web-Based Applications – SQL Injection Strings– xxx’; SELECT * FROM USERS;--– yyy’ SELECT * FROM USER; --– 9; SELECT * FROM USERS;-- COMMENT– ASC; SELECT * FROM USERS– UNION SELECT ALL 1--– ; IF (LEN(USER)=1) WAITFOR DELAY '00:00:10‘--– 1 and (char(65)%2Bchar(65)%2Bchar(65)%2B(select

name from [master]..[sysdatabases] where dbid=1))>0

Negative Use Cases (examples)

• Web-Based Applications – XSS Strings– <SCRIPT>alert()</SCRIPT>– “><SCRIPT>alert()</SCRIPT>– ‘><SCRIPT>alert()</SCRIPT>– </SCRIPT><SCRIPT>alert()</SCRIPT>– javascript:alert()– vbscript:MsgBox()– “ style “font-family:expression(alert())”x=“– +ADw-SCRIPT+AD4-alert();+ADw-/SCRIPT+AD4-