ch9-1-emergent.ppt

8/10/2019 Ch9-1-emergent.ppt

1/8

Introduction to Software Testing

Chapter 9.1Challenges in Testing SoftwareTesting for Emergent Properties: Safety and Security

Paul Ammann & Jeff Offutt

http://www.cs.gmu.edu/~offutt/softwaretest/
http://www.cs.gmu.edu/~offutt/softwaretest/http://www.cs.gmu.edu/~offutt/softwaretest/


2/8

Introduction to Software Testing (Ch 9.2) Ammann & Offutt 2

Chapter 9 Outline

1. Testing for Emergent Properties:Safety and Security

2. Software Testability3. Test Criteria and the Future of

Software Testing


3/8


Emergent Property Overview

How do we address such properties?

A property that arises as a result of assembl ing

components together into a system

Emergent properties exist at system level

The key is the interaction of a system with its environment

Emergent properties do not exist at component level

But individual component design can have a profound effect on emergentproperties

Safety and Security are classic emergent properties

General definition:


4/8

Introduction to Software Testing (Ch 9.2) Ammann & Offutt

Example

Sample Secur ity Property: Outsiders only have

access through intended interface

gets (buf)

P

Web Application

Internet

Property Violation: Buffer

overf low vulnerabil i ty leads to

shell access inside component

4


5/8


Why Emergent Properties Are Hard

Fundamentally different than analyzing intended function Trying to show software lacks certain features

Trying to show absence of certain behaviors.

This is really hard!

Alternative approach

Catalogue typical problem areas

Systematically work through catalog.

Not complete!


6/8


High Level Steps

Capture relevant safety/security properties

Often well-understood by system engineers Hazard model for safety domain

Threat model for security domain

Identify high risk areas

Relates system properties to component properties Example: Fault tree analysis for safety

Mitigate risk

Testing is only one possible approach

Often redesign is a better option

It helps to understand the issues as early as possible!


7/8Introduction to Software Testing (Ch 9.2) Ammann & Offutt 7

Test Cases For Emergent Properties

Develop misuse cases

Helps developers think about ways in which system can be misused

Identify assumptions, and devise test cases that violate them

Can a critical object reach an inconsistent state?

What ways beyond the explicit API exist to alter the state?

What happens when objects are deserialized?

What happens when a database file is accessed outside the DBMS?

What normal checks can be easily evaded?

Identify configuration issues, and devise tests to check them

Develop invalid input tests

Often the unsafe or insecure behavior exists outside the expected domainof inputs

See discussion of bypass testing in Chapter 7

Dont forget about static analysis:

Avoidance/removal of unsafe library calls


8/8Introduction to Software Testing (Ch 9.2) Ammann & Offutt 8

Summary

Most real systems have safety and/or security requirements

Emergent properties only exist at the system level Think about the interaction between a system and its environment

Components, by themselves, dont exhibit emergent properties

Emergent property requirements are better understood by

domain experts than by software developers Communication is essential

Successfully addressing emergent properties requires carefulattention at ALL phases of the software development life cycle

Safety and Security cannot be tested in at the end

Testing is only one tool

ch9-1-emergent.ppt

Documents