why security testing is hard
Post on 02-Jul-2015
306 Views
Preview:
TRANSCRIPT
Why Security Testing Why Security Testing Is HardIs Hard
Herbert H. ThompsonHerbert H. Thompson
Presenter: Alicia YoungPresenter: Alicia Young
IntroductionIntroduction
Software Testing good at verifying requirementsSoftware Testing good at verifying requirements UML helps move from specification to test UML helps move from specification to test
casescases Several bugs routinely escape testingSeveral bugs routinely escape testing
Not specification ViolationsNot specification Violations Would escape most automated testingWould escape most automated testing
Examine Security bugs to discover why testing Examine Security bugs to discover why testing can be difficultcan be difficult
Side-Effect BehaviorSide-Effect Behavior
Input A -> result BInput A -> result B What if Input A also resulted in C?What if Input A also resulted in C?
Overt – unexpected dialog box appearsOvert – unexpected dialog box appears Subtle – writing a file or opening a network portSubtle – writing a file or opening a network port
RDISK utility for WindowsRDISK utility for Windows Creates an emergency Repair DiskCreates an emergency Repair Disk Temporary file created with Universal PermissionsTemporary file created with Universal Permissions During testing, product responds as specifiedDuring testing, product responds as specified
Intended vs. ImplementedIntended vs. Implemented
The State of Security TestingThe State of Security Testing
Exploit Libraries (Librarian Method)Exploit Libraries (Librarian Method) New Products tested with only this libraryNew Products tested with only this library Finds old vulnerabilities with no hope of finding Finds old vulnerabilities with no hope of finding
anything newanything new
Problem is…this strategy actually works!Problem is…this strategy actually works! Developers repeatedly make the same mistakesDevelopers repeatedly make the same mistakes Current software is really buggyCurrent software is really buggy
Applications will eventually become immune to Applications will eventually become immune to these test casesthese test cases
The Need for TechniquesThe Need for Techniques
Test like detectivesTest like detectives Past bugs teach us how vulnerabilities get into Past bugs teach us how vulnerabilities get into
our applicationsour applications The key is to learn new techniques of finding bugsThe key is to learn new techniques of finding bugs
Four General Classes of testing techniquesFour General Classes of testing techniques DependenciesDependencies Unanticipated user inputUnanticipated user input Techniques to expose Design VulnerabilitiesTechniques to expose Design Vulnerabilities Techniques to expose implementation vulnerabilitiesTechniques to expose implementation vulnerabilities
Dependency Insecurities and Dependency Insecurities and FailuresFailures
Software resides in co-dependent environmentSoftware resides in co-dependent environment Two Security ConcernsTwo Security Concerns
Application may inherit insecuritiesApplication may inherit insecurities External security service resource may failExternal security service resource may fail
Internet Explorer’s Content AdvisorInternet Explorer’s Content Advisor Content advisor password protects classes of sitesContent advisor password protects classes of sites If the library fails to load, Internet explorer permits If the library fails to load, Internet explorer permits
access to any previously blocked siteaccess to any previously blocked site
Cause of Dependency FailuresCause of Dependency Failures
Severely under-applied inputs to softwareSeverely under-applied inputs to software Error handling code gets little testing scrutinyError handling code gets little testing scrutiny
These types of failures need to be examinedThese types of failures need to be examined
Unanticipated User InputUnanticipated User Input
Inputs that cause undesirable side effects and Inputs that cause undesirable side effects and require special testingrequire special testing Reserved wordsReserved words Escape charactersEscape characters Long stringsLong strings Boundary valuesBoundary values
Most well known side-effect: Buffer OverflowMost well known side-effect: Buffer Overflow Input that can be interpreted as commandsInput that can be interpreted as commands
Design InsecuritiesDesign Insecurities
Many Security Vulnerabilities designed into Many Security Vulnerabilities designed into applicationapplication Seeing high-level impact on an application or host is Seeing high-level impact on an application or host is
difficultdifficult
Test InstrumentationTest Instrumentation Many applications shipped with itMany applications shipped with it Bypassing security controls for ease of testingBypassing security controls for ease of testing
Ports left openPorts left open Insecure default values and configurationsInsecure default values and configurations
Implementation InsecuritiesImplementation Insecurities
Perfect design means nothing if Implementation Perfect design means nothing if Implementation is flawedis flawed
Man-in-the-middle attackMan-in-the-middle attack Attacker gets between time application checks Attacker gets between time application checks
security and when the application uses informationsecurity and when the application uses information Xterm – can be exploited to allow a restricted user Xterm – can be exploited to allow a restricted user
to append data to the password fileto append data to the password file
Standard Bug-Severity RankingsStandard Bug-Severity Rankings UrgentUrgent
System crash, Unrecoverable data loss, jeopardizes personnelSystem crash, Unrecoverable data loss, jeopardizes personnel HighHigh
Impairment of critical system functions and no work-around Impairment of critical system functions and no work-around existsexists
MediumMedium Impairment of critical system functions and work-around Impairment of critical system functions and work-around
existsexists LowLow
Inconvenience, annoyanceInconvenience, annoyance NoneNone
None of the above or an enhancementNone of the above or an enhancement
The Need For ToolsThe Need For Tools
Testers generally rewarded for both quantity and Testers generally rewarded for both quantity and severity of bugsseverity of bugs
Side-effect bugs may not get noticed or even Side-effect bugs may not get noticed or even dismissed by managersdismissed by managers
Equipped with proper tools testers would notice Equipped with proper tools testers would notice odd behaviorodd behavior Writing of a temporary fileWriting of a temporary file Sending of extra network packetsSending of extra network packets
New ToolsNew Tools
Regmon and Filemon – monitor application Regmon and Filemon – monitor application interactions with registry and file systeminteractions with registry and file system www.sysinternals.comwww.sysinternals.com
App-Sight – monitors environmental App-Sight – monitors environmental interactionsinteractions www.identify.comwww.identify.com
Holodeck – Fine grain control over interactions Holodeck – Fine grain control over interactions between application and environmentbetween application and environment www.sisecure.comwww.sisecure.com
Paper AnalysisPaper Analysis
Quality Software is Secure SoftwareQuality Software is Secure Software
Important points madeImportant points made Better testing techniquesBetter testing techniques Better testing toolsBetter testing tools Design concernsDesign concerns
Questions?Questions?
top related