black box testing methodology sans.ppt box testing... · why black box test? ! ... why black box...
TRANSCRIPT
© Copyright Red Tiger Security – Do not print or distribute without consent.
Black Box Testing Methodologies
Joe Cummins, PCIP, OPST Jonathan Pollet, CISSP, CAP, PCIP January 24, 2011 SANS SCADA Webinar, SCADA Summit Series 2011
welcome
© Copyright Red Tiger Security – Do not print or distribute without consent. 2
© Copyright Red Tiger Security – Do not print or distribute without consent.
Outline
� Why Black Box Test?
� Layered approach
� Black Box vs. White Box
� Components of an Assessment
� Process
� Reports and metrics
© Copyright Red Tiger Security – Do not print or distribute without consent.
…Why Black Box testing?
� Know what you are putting out on the network…
� How does a device respond to protocols it does not recognize?
� What happens when it gets a confusing message?
… are you sure?
© Copyright Red Tiger Security – Do not print or distribute without consent.
Phased Approach to Device / Application Testing
� Protocol � RFC’s � Proper communications
� Software � DOS, Overflow, Etc… � Kernel
� Firmware � Assembler
� Hardware � Components � Monitoring
OS
Firmware
Hardware
• Applications • Kernel
• Assembler
© Copyright Red Tiger Security – Do not print or distribute without consent.
Layered Defence
6. Embedded Device
5. Communication Method
4. Servers / Workstations
3. DMZ
2. Infrastructure
© Copyright Red Tiger Security – Do not print or distribute without consent.
Software / Middleware
� Exceptions � Failures � Null Pointers � Access Violations
� Memory Corruption � Buffer Overflow � Stack Overflow
© Copyright Red Tiger Security – Do not print or distribute without consent.
Hardware
� Components � NIC (wired, wireless) � Ports
� Monitoring � CPU � Temperature � Cycles � Processes � Stack
© Copyright Red Tiger Security – Do not print or distribute without consent.
Tools of the Trade
© Copyright Red Tiger Security – Do not print or distribute without consent.
Manual Code Review
� Automated tools � Highlights errors / changes � Known common application faults � Verification of Syntax
� Viewers � Import / Export Source � Render � Analyze
© Copyright Red Tiger Security – Do not print or distribute without consent.
Analysis Engine
� Core Fuzzing Process � Reliance on the Tools and plugins to generate proper data
� Manual Code Review � Line by line review
� Blended Analysis
© Copyright Red Tiger Security – Do not print or distribute without consent.
Blended Analysis
� Device Testing Methodology
� Combination of both aspects � Code review + Fuzzing = closer examination
� Benefits of both forms of Analysis
© Copyright Red Tiger Security – Do not print or distribute without consent.
Anatomy of the Analysis
Model to Mayhem
© Copyright Red Tiger Security – Do not print or distribute without consent.
White Box vs. Black Box Testing
Delivery
Application Implementation Protocol Specification
Function
Design Abstraction Dissection
Analysis
Code Review Input Testing
Testing
Verification Validation
© Copyright Red Tiger Security – Do not print or distribute without consent.
Analysis Engine
Final Deliverable
Input Modules
Protocol Template
Target
Seed File Session
Assembler
Sessions
Collection Method
EKG
Outputs
Core Fuzzing Process
© Copyright Red Tiger Security – Do not print or distribute without consent.
Input Generation Methods (Invalid)
Error Collection
Isolated Element
Invalid Data
© Copyright Red Tiger Security – Do not print or distribute without consent.
Input Generation Methods (Valid)
Valid Output
Isolated Element
Valid Data
© Copyright Red Tiger Security – Do not print or distribute without consent.
Device EKG / ECG
ICMP • Echo • Reply • Config
SNMP • Status • Agent • Manager
TCP • HTTP (S) • SSH (22) • TELNET (23)
© Copyright Red Tiger Security – Do not print or distribute without consent.
Device EKG / ECG
� ICMP � ICMP Echo / Reply � Dropped Config, Delayed Response, etc…
� TCP � Active Session, keep-alive, timeouts � HTTPS, SSH, Telnet,
� SNMP � Monitoring � Statistics
© Copyright Red Tiger Security – Do not print or distribute without consent.
Comparison and Contrast
� What does an error look like? � How do you work with this information? � What can be determined about the program / device? � Can this lead to cascading errors?
� What can you do with an error? � POC? � Weaponization / Exploit Development
© Copyright Red Tiger Security – Do not print or distribute without consent.
Exploit Weaponization (Stages)
Staged Attack Binary
Exploit Payload
Vuln.
Code
Socket
Packaged Exploit
Exploit
© Copyright Red Tiger Security – Do not print or distribute without consent.
Output Collection
� Comparison and contrast � Characteristics of an error � Scale of vulnerability
� “Weaponization” � Malicious code � Payloads � Repeatable
� Hardware EKG � Health of the device � “State” of the device
© Copyright Red Tiger Security – Do not print or distribute without consent.
Reports and Metrics
� Black box testing Report: � Spreadsheet of tests and outputs, � Tools used, � Findings, � Recommendations, � Remediation steps,
� Include: � Packet Captures (in pcap) for replay � Screen captures � Outputs for future analysis
© Copyright Red Tiger Security – Do not print or distribute without consent.
Wrap-up
� Devices need to be tested � Vendors continue to “push” product to market � Consumers need to be aware of the hazards
� Small investment / Resilient Devices
� Testing is CRITICAL
� Does not need to be resource intensive � Complex task, automated and facilitated � Part of the internal Testbed
© Copyright Red Tiger Security – Do not print or distribute without consent.
Contact info:
Jonathan Pollet, CISSP, CAP, PCIP Red Tiger Security - USA office: +1.877.387.7733 web: www.redtigersecurity.com [email protected] Check out our Industry Forum and sign up for RSS feed:
Forum: http://www.redtigersecurity.com/forum/
Joe Cummins, PCIP, OPST Founder, Principal Consultant Red Tiger Security - Canada office: +1.877.387.7733 web: www.redtigersecurity.ca [email protected]
25