tdt 4242 torbjørn skramstad requirements specification and testing an introduction tdt 4242...
TRANSCRIPT
TDT 4242
Torbjørn Skramstad
Requirements Specification and TestingAn introduction
TDT 4242
Institutt for datateknikk oginformasjonsvitenskap
TDT 4242
Challenges in Requirements Engineering
What is a requirement?•What a system must do (functional): System requirements•How well the system will perform its functions (non-functional): System quality attributes
definedoperationalcapabilities
businessneeds
satisfy
The RE process:
Ultimately:
TDT 4242 4
How the customer explained it
How the architects and developers understood it
What the customer really needed
Software specification in natural language
TDT 4242
Challenges in Requirements Engineering
Source: Benoy R Nair (IBS software services)
Importance of getting requirements right:1/3 budget to correct errors originate from requirements
TDT 4242
Challenges in Requirements Engineering
Source: Benoy R Nair (IBS software services)
Factors that make a software project challenging:
TDT 4242
Challenges in Requirements Engineering
Source: Benoy R Nair (IBS software services)
Why projects are cancelled:
TDT 4242
Requirements Development - 1Requirements Elicitation:The process of discovering the requirements for a system by communication with customers, system users and others who have a stake in the system development.
Requirements gathering techniques
• Systematic extraction of concrete requirements from high level goals
• Requirements quality metrics
TDT 4242 10
The Ariane 5 accident, 1996 Single root cause failure!
The ”bug”: attitude deviation stored as 2-byte integer (max value 65,535) in stead of 4-byte (max value 4,294,967,295)
SW module was reused from Ariane 4
Insufficient V&V of detailed requriements: larger attitude deviation tolerated in Ariane 5 than in Ariane 4
Ariane 5 production cost 10 years and $7 billion; luckily no victims because it was unmanned. 65,535 = 00000000 00000000 11111111 11111111
65,536 = 00000000 00000001 00000000 00000000
TDT 4242 11
Other software related accidents & incidents Accidents & incidents
Alarm flooding, power distribution failure, BP Grangemouth Scotland, 29th May - 10th June 2000
failure in a data bus + faulty logic in the software => engine power loss, Airbus A340-642, 2005
failed accelerometer + software bug => Faulty air speed metering, Boeing 777-200, 2005
Software bug => shutdown of radio communication between ATC and aircraft, 2004. disrupted 800 flights
Safety related software flaws => recall of 200,000 pacemakers in 1990-2000
radiotherapy machines attacked by computer viruses, 2005
Buggy software in corporate computer + connection between corporate and control systems networks => shutdown of nuclear power plant, USA 2008
In general, most software failures are actually ”bugs” in the detailed requirement specifications, i.e. poor understanding of the very detailed requirements
Korean Air 747 in Guam, 200 deaths (1997): incorrect configuration of ”minimum altitude” warning system
TDT 4242
Requirements Development – 3 Effects of Inadequate Requirements development – Airbus:
Requirement: Reverse thrust may only be used, when the airplane is landed.
Translation: Reverse thrust may only be used while the wheels are rotating.
Implementation: Reverse thrust may only be used while the wheels are rotating fast enough.
Situation: Rainstorm – aquaplaning
Result: Crash due to overshooting the runway!
Problem: erroneous modeling in the requirement phase
TDT 4242
Problem world and machine solution
The problem to be solved is rooted in a complex organizational, technical or physical world.
• The aim of a software project is to improve the world by building some machine (HW/SW) expected to solve the problem.
• Problem world and machine solution each have their own phenomena while sharing others.
• The shared phenomena defines the interface through which the machine interacts with the world.
E-commerce world
Requirements engineering is concerned with the machine’s effect on the surrounding world and the assumption we make about that world.
TDT 4242
Formulation of requirements statements
Statement scope:
• Phenomenon of train physically moving is owned by environment. It cannot be directly observed by software phenomenon
• The phenomenon of train measured speed being non-null is shared by software and environment. It is measured by a speedometer in the environment and observed by the software.
TDT 4242
Two types of requirements statements
• Descriptive statements: state properties about the system that holds regardless of how the system behaves. E.g. if train doors are open, they are not closed.
• Prescriptive statements: States desirable properties about the system that may hold or not depending on how the system behaves
• Need to be enforced by system components• E.g. train doors shall always remain closed when the
train is moving
TDT 4242
Formulation of system requirement
A prescriptive statement enforced by the software-to-be.
• Possibly in cooperation with other system components• Formulated in terms of environment phenomena
Example: All train doors shall always remain closed while the train is moving
In addition to the software-to-be we also require the cooperation of other components:
• Train controller being responsible for the safe control of doors.
• The passenger refraining from opening doors unsafely
• Door actuators working properly
TDT 4242
Formulation of software requirement
A prescriptive statement enforced solely by the software-to-be. Formulated in terms of phenomena shared between the software and the environment.
The software “understands” or “senses” the environment through input data
Example: The doorState output variable shall always have the value ‘closed’ when the measuredSpeed input variable has a non-null value
TDT 4242
Domain properties
A domain property:• Is a descriptive statement about the problem world• Should hold invariably regardless of how the system behaves• Usually corresponds to some physical laws
Example:
A train is moving if and only if its physical speed is non-null.
TDT 4242
Goal orientation in requirements engineering – 1
A goal is an objective that the system under consideration shall achieve.
– Ranges from high-level strategic to low-level technical concerns over a system
– System consist of both the software and its environment. Interaction between active components, i.e. devices, humans, software etc. also called Agents
TDT 4242
Goal orientation in requirements engineering – 2
Goals can be stated at different levels of granularity:– High-level goal: A goal that requires the cooperation of
many agents. They are normally stating strategic objective related to the business, e.g. “the system’s transportation capacity shall be increased by 50%”
– Requirement: A goal under the responsibility of a single agent in the software-to-be.
– Assumption (expectation): A goal under the responsibility of a single agent in the environment of the software-to-be. Assumptions cannot be enforced by the software-to-be
TDT 4242
Goal orientation in requirements engineering – 3
We shall make software for a toll road gate. The system’s transmitter sends a signal to an approaching car which returns the car-owner’s id. The payment is invoiced to the car-owner.
• Assumption: “The car returns the car-owner’s id”. This is the responsibility of a single agent in the environment of the software-to-be• Requirement: “The system’s transmitter sends a signal to an approaching car”. This is the responsibility of a single agent in the software-to-be
TDT 4242
Goal statement typology
TDT 4242
Goal types
TDT 4242
Behavioral goal specialization
TDT 4242
Goal categorization – 1
Goal categories are similar to requirements categories:
TDT 4242
Goal categorization – 2
Functional goal: States the intent underpinning a system service
• Satisfaction: Functional goals concerned with satisfying agent request
• Information: Functional goals concerned with keeping agents informed about important system states
• Stimulus-response: Functional goals concerned with providing appropriate response to specific event
Example: The on-board controller shall update the train’s acceleration to the commanded one immediately on receipt of an acceleration command from the station computer
TDT 4242
Goal categorization – 3
Non-functional goal: States a quality or constraint on service provision or development.
Accuracy goal: Non-functional goals requiring the state of variables controlled by the software to reflect the state of corresponding quantities controlled by environment agent
E.g: The train’s physical speed and commanded speed may never differ by more than X miles per hour
Soft goals are different from non-functional goals. Soft goals are goals with no clear-cut criteria to determine their satisfaction.
E.g: The ATM interface should be more user friendly
TDT 4242
Goal refinement
A mechanism for structuring complex specifications at different levels of concern.
A goal can be refined in a set of sub-goals that jointly contribute to it.
Each sub-goal is refined into finer-grained goals until we reach a requirement on the software and expectation (assumption) on the environment.
NB: Requirements on software are associated with a single agent and they are testable
Assumptions / Expectations example
The notation above shows that we expect the measurement equipment to measure a value corresponding to the reality.
It is not something we, at this level, can do anything about.
It will, however, at some point in time, be e.g. a sensor requirement
Measurement = reality
Measurement equipment
TDT 4242
Goal refinement: Example
TDT 4242
Goal refinement tree – 1
Refinement links are two way links: One showing goal decomposition, the other showing goal contribution
TDT 4242
Goal refinement tree – 2
Goal feature annotation
TDT 4242
Where do the goals come from?
We get goals from:
• Preliminary analysis of the current system.
• Systematically by searching intentional keywords in documents provided, interview transcripts etc. E.g. ‘objective’, ‘purpose’, ‘in order to’.
• Iterative refinement and abstraction of high-level goals: By asking the how and why question. Results in a goal refinement tree
• Approaches: KAOS – Goal driven requirements acquisition. http://www.info.ucl.ac.be/~avl/ReqEng.html
TDT 4242
Goals – Summary
Goals can be defined at different levels of abstraction
There are two types of goals: Behavioral or soft goal
There are several categories of goals, e.g. • Functional and non-functional• Goal refinement provides a natural mechanism
for structuring complex specifications at different levels of concern:
• Goal refinement tree
TDT 4242
Requirements quality metrics – 1
Qualitative Goal-Requirements tracing:An approach to requirements
refinement/abstraction that makes it less likely to generate trace links that are ambiguous, inconsistent, opaque, noisy, incomplete or with forward referencing items
TDT 4242
Requirements quality metrics – 2
TDT 4242
Requirements quality metrics – 3Forward Referencing: Requirement items that make
use of problem world domain features that are not yet defined.E, C and D need to be mapped to a requirement item
TDT 4242
Requirements quality metrics – 4Opacity: Requirement items for which rational or
dependencies are invisible.
Multiple unrelated concept mapping. A is not related to B
TDT 4242
Requirements quality metrics – 5
Noise: Requirement items that yield no information on problem world features. X refers to a concept undefined in the domain
TDT 4242
Requirements quality metrics – 6Completeness: The needs of a prescribed
system are fully covered by requirement items without any undesirable outcome.
No requirement item mentions the goal concept Z
TDT 4242
RQM – Steam boiler example – 1
P
ControlUnit Feed
water
230 V AC
Process Steam
ControlUnit
To air
P
LT
TDT 4242
RQM – Steam boiler example – 2
The steam boiler has three sensors – a temperature senor (T) a pressure sensor(P) and a water level sensor (L)
The water level sensor gives info to the pump control unit
The temperature sensor and the pressure sensor gives info to the heating unit.
TDT 4242
Examples • A requirement that contains a reference to a
sensor is ambiguous since there are three sensors involved.
• The requirement “The pump control unit will read a value from the temperature sensor” is inconsistent since the temperature sensor is not used by the pump control unit.
• The requirement “The collector tray shall have a capacity of at least 500 liters” is opaque since there are no other references to a collector tray and no rational as to why we need one.
TDT 4242
Examples • The requirement “The operator’s chair shall
have leather upholstery” is noise. There is no reason why such a requirement is needed for the control system.
The line between noise and opacity can some times be quite thin. In both cases, the requirement can sometimes be made into a real requirement if it is supplied with the proper rationale.
TDT 4242
Examples • There is a requirement stating that the
pump controller gets the water level from a water level sensor
• There is no requirement specifying that the boiler vessel shall be equipped with a water level sensor
In this case, the requirements set is incomplete.
Quality metrics on a requirements set provides useful understanding, tracking and control of requirements improvement process.
Requirements quality metrics
TDT 4242
Requirements quality metrics in use – 1 In order to use the quality metrics for
requirements elicitation, we will need an ontology.
An ontology is a structured description of the components that we expect to be present for system of a certain type.
TDT 4242
Requirements quality metrics in use – 2
……Water-level……
Boiler-tankFeeding-pumpNon-return valveTank
Max-limit
Pump
infers
has
is controlledby
Min. water levelMax. water levelWater-level indicator
OnOff
Control-system
has-state
infers
is coupledwith
TDT 4242
Requirements quality metrics in use – 3 Using an ontology we can check that e.g.,
• All parts of the system or subsystem are included in the requirements – completeness.
• Terms belonging to one subsystem are not used for other subsystems - consistency.
• Only terms found in the ontology are accepted – no ambiguity.
• If one part of a subsystem is included, the rest of the subsystem is also included – no opacity.
TDT 4242
Requirements quality metrics in use – 4 Using the quality metrics approach for requirements elicitation will in most case require a tool containing the relevant domain ontologies.
E.g., if we have a requirement about the pump but nothing about how it is controlled – control system – or that it can be turned on and off – states – we will get a warning about low completeness.