ETS170 Requirements Engineering

Lecture 6: Validation [Lau:9] Inspections [INSP] Requirements Interdependencies [INTDEP] Agile RE [AGRE] This lecture is input to your current project task: To develop your Validation Checklist for your costumer's validation efforts during next week. Work on this at exercise session.

Björn Regnell Alma Orucevic-Alagic http://www.cs.lth.se/ETS170

How will you do requirements validation in your project?

Requirements validation

Purpose ♦ To make sure that we have elicited

and documented the right requirements in a good way

♦  ”Will we build the right system with these requirements?”

Methods ♦  Inspections [INSP] ♦ Tests, e.g. usability testing,

prototypes, model-based simulations ♦ Mathematical proofs

Requirements Validation through tests

Different types of dynamic validation: ♦  Manual ”simulation” (walk-through) based on

scenarios/use cases/task descriptions ♦  Paper prototypes. “mock-ups” ♦  Executable prototypes ♦  Pilot tests

Important steps: ♦  Choose suitable test approach, environment, etc. ♦  Choose who will do the testing ♦  Create test cases ♦  Run test cases ♦  Document problems ♦  Fix problems ♦  How to avoid problems in the future?

Fig 9. Checking and validation

Guest

Event list

Goals Tasks

E/R

Check that all parts match & everything is included

Validate that stakeholders are happy (customer, user, developer)

Where are the major risks?

Quality product = meeting the spec? From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Discussion

§ What is on your wish list of quality criteria for a requirements specification?

But you can’t get everything…

Fig 9.1 Quality criteria for a specification

Classic: A good requirement spec is: Correct

Each requirement reflects a need. Complete

All necessary requirements included. Unambiguous

All parties agree on meaning. Consistent

All parts match, e.g. E/R and event list. Ranked for importance and stability

Priority and expected changes per requirement. Modifiable

Easy to change, maintaining consistency. Verifiable

Possible to see whether requirement is met. Traceable

To goals/purposes, to design/code. Additional: Traceable from goals to requirements. Understandable by customer and developer.

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Korrekt Fullständig Otvetydig Motsägelsefri Rankad Modifierbar Verifierbar Spårbar bakåt/framåt Begriplig Designoberoende Motiverad Koncis Välorganiserad ...

Fig 9.2A Contents check

Does the spec contain: •  Customer, sponsor, background •  Business goals + evidence of tracing

•  Data requirements (database, i/o formats, comm.state, initialize)

•  System boundaries & interfaces •  Domain-level reqs (events & tasks) •  Product-level reqs (events & features) •  Design-level reqs (prototype or comm. protocol) •  Specification of non-trivial functions •  Stress cases & special events & task failures

•  Quality reqs (performance, usability, security . . .)

•  Other deliverables (documentation, training . . .) •  Glossary (definition of domain terms . . .)

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Fig 9.2B Structure check

Does the spec contain: •  Number or Id for each requirement •  Verifiable requirements •  Purpose of each requirement •  Examples of ways to meet requirement •  Plain-text explanation of diagrams, etc. •  Importance and stability for each requirement •  Cross refs rather than duplicate information •  Index •  An electronic version

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Fig 9.2C Consistency checks

Guest

Event list 1. 2. . . .

Tasks

E/R model

Function list 1. 2. . . .

CRUD Event check

Event check

Support?

Data exists?

Virtual windows

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Book C U O C O U O CheckinBooked RU U O O U O

CheckinNonbkd C U O C O U O

Checkout U U O R U

ChangeRoom R R O U O

RecordService O C R PriceChange C UDO C UDO

Missing? D D C?UD? UD

Fig 9.2D CRUD matrix

Create, Read, Update, Delete + Overview

Gues

t

Stay

Room

Room

Stat

e

Serv

ice

Serv

iceTy

pe

Entity

Task

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

SLUT+Ö Skapa Läsa Uppdatera Ta bort Översikt

Fig 9.3 Checks against surroundings

Reviews

Review: Developers and customer review all parts.

Goal-means analysis: Goals and critical issues covered? Requirements justified?

Risk assessment: Customer assesses his risk. Developers assess their risk. High-risk areas improved.

Tests

Simulation and walk-through Follow task descriptions. Correct? Supported?

Prototype test (experiment with prototypes): Requirements meaningful and realistic? Prototype used as requirement?

Pilot test (install and operate parts of system): Cost/benefit? Requirements meaningful and realistic?

Just before signing?

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Fig 9.4(A) Check list at work

Project: Noise Source Location, NSL vers. X Date, who: 99-03-15, JPV

Contents check Observations - found & missing Problem?Customer & sponsor Missing, OK. . .Data:Database contents

Class model as intermediate workproduct

. . .Initial data & states Missing Seems innocent, but caused many

problems particularly when screenwindows were opened.

Functional reqs:Limits & interfacesProduct-level eventsand functions

Mostly as features

. . .Special cases:Stress casesPower failure, HWfailure, config.

Missing Problem. Front-end caused manyproblems

From: Soren Lauesen: Software Requirements © Pearson / Addison-Wesley 2002

Project: Noise Source Location, NSL vers. X Date, who: 99-03-15, JPV

Contents check (2) Observations - found & missing Problem?Quality reqs:Performance

Missing, also in parts not shownhere.

Problem. Response time becameimportant.

Capacity, accuracy Missing, also in parts not shownhere.

Problem. Data volume, etc.became important.

Usability Missing Would have been usefulInteroperability Missing External dataformats, robot role,

etc. caused problems. . .Other deliverables:Documentation

Missing Unimportant. Company standardsexist.

. . .

Structure check Observations - found & missing Problem?ID for each req. OKPurpose of eachrequirement

Good. Domain described.

Consistency checks Observations - found & missing Problem?CRUD check:Create, read, update,delete all data?

Have been made

Tests Observations - found & missing Problem?Prototype test Not done, nor during development. Should have been done. Caused

many problems later.

[INSP]

Inspections

Described already by M.E. Fagan, IBM, early 70-ies ♦ A systematic assessment technique where

documents are inspected manually by others than the author to detect defects

General objectives of inspection methods: ♦ Defect detection ♦ Knowledge dissemination ♦ Decision-making ♦ Process improvement

The inspection process [INSP]

Overview meeting

Planning Inspection meeting

Root-cause analysis

Request List of defects

Defect summary

Report

Prepa-ration Correction Follow-up

Roles: Moderator Author Reviewer Secretary

The actual individual reviewing is done in this activity

Different methods to detect defects (reading techniques)

Ad hoc ♦  To your best ability (no specific guidelines)

Checklist ♦  A list of questions or check items direct the review

Perspective-based reading ♦  Different reviewers inspect from different perspectives and

their findings are combined: e.g. user, designer, tester – perspectives, or from the perspective of different tasks/use cases

N-fold inspection ♦  N independent groups run inspection process in parallel

[INTDEP] in compendium

An industrial survey of requirements interdependencies in software product release planning

Carlshamre, P., Sandahl, K., Lindvall,

M., Regnell, B., Natt och Dag, J. IEEE Int. Conf. on Requirements

Engineering (RE01), Toronto, Canada, pp. 84–91 (2001)

Research Method

•  survey of five different companies •  a manager of a product/project

was asked to identify and classify interdependencies among 20 high priority requirements.

Data collection

Different types of interdependencies

Examples: AND. A printer requires a driver to function, and the driver requires a printer to function. REQUIRES. Sending an e-mail requires a network connection, but not the opposite. TEMPORAL. The function Add object should be implemented before Delete object. (This type is doubtful, which is discussed in section 3.1) CVALUE. A detailed on-line manual may decrease the customer value of a printed manual. ICOST. A requirement stating that “no response time should be longer than 1 second” will typically increase the cost of implementing many other requirements. OR. In a word processor, the capability to create pictures in a document can either be provided as an integrated drawing module or by means of a link to an external drawing application.

Expressing interdependencies in reqT •  An AND rela6on is equivalent to two mutual requires-rela6ons: Feature("printerX1") requires Feature("driverX") Feature("driverX") requires Feature("printerX1")

•  A requires rela6on can be non-mutual :

Feature("sendEmail") requires Feature("networkAccess") •  Temporal rela6ons regarding a preferred implementa6on order can be expressed using precedes:

Function("add") precedes Function("delete") •  Exclusion (xor) can be expressed by an excludes rela6on (only one is needed as exclusion is mutual):

Design("centralized") excludes Design("distributed") Design("distributed") excludes Design("centralized") •  En66es that support or hinder each other can be modeled using hurts and helps rela6ons :

Goal("secure") helps Goal("safe") Goal("secure") hurts Goal("simple")

[Some examples modified from Carlshamre, P., Sandahl, K., Lindvall, M., Regnell, B., NaP och Dag: "An industrial survey of requirements interdependencies in soRware product release planning", J.: Int. Conf. on Requirements Engineering (RE01), Toronto, Canada, pp. 84–91, 2001]

Expressing CVALUE dependencies as Constraints in reqT

val m = Model( Req("x") has (Order(1), Benefit(100)), Req("y") has Order(1)) // Same release val c = Constraints( Req("y")/Benefit :: {0 to 1000}, Sum(Req("x")/Benefit, Req("y")/Benefit) === Var("SumXY"), Var("SumXY") :: {0 to 2000}, IfThenElse( Req("x")/Order === Req("y")/Order, //If same release Var("SumXY") === 400, //then more valuable Var("SumXY") === 200 //else less valuable )) val m2 = (m + c).satisfy m2: reqT.Model =

Model( Req("y") has (Benefit(300), Order(1)), Req("x") has (Order(1), Benefit(100)), Constraints( Var("SumXY") === 400))

Expressing CVALUE dependencies as Constraints in reqT

val m = Model( Req("x") has (Order(1), Benefit(100)), Req("y") has Order(2)) // Different releases val c = Constraints( Req("y")/Benefit :: {0 to 1000}, Sum(Req("x")/Benefit, Req("y")/Benefit) === Var("SumXY"), Var("SumXY") :: {0 to 2000}, IfThenElse( Req("x")/Order === Req("y")/Order, //If same release Var("SumXY") === 400, //then more valuable Var("SumXY") === 200 //else less valuable )) val m2 = (m + c).satisfy m2: reqT.Model =

Model( Req("y") has (Benefit(100), Order(2)), Req("x") has (Order(1), Benefit(100)), Constraints( Var("SumXY") === 200))

Not always straight forward …

•  “if R2 is completely worthless to the customer without R1, and we would thus never do R2 without R1, do we classify the relationship as REQUIRED or just CVALUE?”

•  REQUIRES sometimes arises from the opposite reasoning: “If we do R2, then we can do R1 too!”, which implies that the direction of the relationship could be the opposite; could e.g. be called “ENABLES” or "HELPS"

Summary of identified interdependencies

1.  10% of the requirements are responsible for roughly 50% of the interdependencies

2.  20% of the requirements are responsible for roughly 75% of all interdependencies

3.  About 20% of the requirements are singular 4.  Customer-specific: more functionality-related ;

Market-driven: more value-related dependencies

Example of dependency structures

Coupling measures

I =#dependencies R =#requriements i = #dep. betw. 2 partitions Release

coupling:

In survey: 10-22%

Paper in compendium [AGRE]

Agile Requirements Engineering Practices: An Empirical Study, by Balasubramaniam Ramesh and Lan Cao, in: IEEE Software, 60-67, January/February 2008

Assumptions underlying agile requirements engineering

§  Changing requirements assumption: §  Requirements always evolve because of changes in technology, customer needs

and business domains. §  Therefore, agile development does not spend much time in initial requirements

elicitation. Instead, requirements emerge during the development process. §  Documentation assumption:

§  Developing extensive documentation and models is counterproductive. §  Therefore, agile development does not document requirements in detail upfront.

§  Customer interaction assumption: §  Customers are available for frequent interaction with the developers. §  Agile development relies on these interactions to elicit and validate requirements.

§  Cost-of-change assumption: §  The cost of making changes to the system does not dramatically increase over

time. §  As evolving requirements do not increase the development cost, RE is carried out

iteratively and incrementally.

Adoption level of agile RE practices in 16 companies

•  Face-to-face communication •  Iterative RE •  Extreme prioritization •  Constant planning •  Prototyping •  Test-driven development •  Reviews & acceptance tests

Face-to-face Communication

§  Agile software processes prefer face-to-face communication over written specifications.

§  The focus of RE in an agile environment is to effectively transfer ideas from the customer to the development team, rather than creating extensive requirements documents.

§  Techniques: User Stories No formal requirements specification; use "backlog" instead

§  Prerequisites: Active involvement of (knowledgeable) customers

Face-to-face Communication

Perceived Benefits §  Customers can steer the

project in unanticipated directions, especially when their requirements evolve owing to changes in the environment or their own understanding of the software solution.

§  Informal communication obviates the need for time-consuming documentation and approval processes, which are perceived as unnecessary, especially with evolving requirements.

Perceived Challenges §  If intensive interaction between

customers and developers cannot be established, this approach poses the risk of wrong or inadequate requirements.

§  Achieving on-site customer representation is difficult (even in the form of a surrogate product manager).

§  When more than one customer group is involved, achieving consensus/compromise in the short development cycle is challenging.

§  Customers used to a traditional development process might not understand or trust the agile RE process, which doesn’t produce detailed requirements.

Iterative RE •  Requirements are not pre-defined; they emerge during development. •  High-level requirement analysis is carried out at beginning of project. •  Initially, the development team acquires a high-level understanding of

the critical features of the application, rather than creating a detailed specification of requirements.

•  Requirements are not intended to be complete or cover all features. •  Requirements serve as a starting point to plan the initial release cycle. •  As more is getting known, more features/user stories are added. •  Techniques:

•  Agile requirements analysis continues at each development cycle. •  At the beginning of each development cycle, the customer sits down with

the development team to provide detailed information on a set of features that need to be implemented. During this process, requirements are discussed at a greater level of detail.

•  Requirements analysis is intertwined with the design activity.

Iterative RE

Perceived Benefits §  Iterative RE creates

a more satisfactory relationship with the customer.

§  Requirements are clearer and more understandable because of the immediate access to customers and their involvement in the project when needed.

Perceived Challenges §  Cost & Schedule Estimation: Because the project

scope is subject to constant change, creating accurate cost and schedule estimates for the entire project is difficult. Obtaining management support for such projects could be challenging.

§  Minimal documentation: When a communication breakdown occurs the lack of documentation might cause a variety of problems (e.g., scalability, evolution, introduction of new team members).

§  Neglect of quality requirements: Especially during early development cycles, customers often focus on core functionality and ignore quality reqts such as scalability, maintainability, portability, safety, or performance.

Extreme Prioritization §  Requirements are categorized from the standpoint of importance into

essential requirements, useful capabilities and desirable capabilities. §  Setting priorities early in the project helps deliver the most valuable features

as early as possible.

§  Techniques: •  Implement the features with the highest priority early in the project so that

the customers can realize most of the potential business value. •  During development, the customer’s as well as the developer’s

understanding of the project improves and new requirements are added, or existing requirements are modified.

•  To keep priorities up to date, prioritization is repeated frequently during the entire development process.

Extreme Prioritization

Perceived Benefits §  Because customers are very

involved in the development process, they can provide business reasons for each requirement at any development cycle. Such a clear understanding of the customer’s priorities helps the development team better meet customer needs.

§  Also, in contrast to traditional development, where achieving reprioritization is difficult, agile RE provides numerous opportunities for reprioritization.

Perceived Challenges §  Using business value

(often focused on time-to-market) as the only or primary criterion for requirements prioritization might cause major problems in the long run (e.g., ‘omitted’ quality reqts).

§  Continuous reprioritization, when not practiced with caution, may lead to instability

Constant Planning §  A core principle in agile software development is to adapt

and react quickly to changes demanded by the environment.

§  Accommodating changes in requirements during the development process is a way of tuning the system to better satisfy customer needs.

Techniques: ♦  Before implementing a feature, the developer engages in detailed

discussions with the customer to roughly understand what the customer needs.

♦  Developers get constant feedback from the customer as the features are implemented.

Constant Planning

Perceived Benefits §  The early and constant

validation of requirements largely minimizes the need for major changes.

§  Thus, the cost of addressing a change request decreases dramatically compared to traditional software development.

Perceived Challenges §  Often, the architecture defined during the

early cycles becomes inadequate as requirements change and redesign of the architecture adds significantly to project cost.

§  Refactoring changes software’s internal structure to make it easier to understand and cheaper to modify without changing its observable behavior. However, the need for refactoring isn’t always obvious, and the ability to refactor depends on developers’ experience and schedule pressure.

§  Refactoring often doesn’t fully address the problem of inadequate/inappropriate architecture.

Prototyping

§  Instead of using formal requirements documents, many agile projects use prototyping as a way to communicate with their customers.

§  Based on customer feedback on the prototypes developed for demonstration and experimentation, requirements are validated and refined.

§  To a certain extent, the production software itself can be a form of an operational prototype, a refinement of the code created for experimentation with required features. ♦  The rush to market encourages a tendency to deploy these prototypes

rather than wait for robust implementations of the desired features. ♦  The ability to quickly deploy newer versions of the product (e.g., enabled

by a web architecture for delivery) promotes this practice.

Prototyping

Perceived Benefits §  Prototyping may serve to

communicate with customers to validate and refine requirements instead of incurring the overhead involved in creating formal requirements documents.

Perceived Challenges §  Deploying prototypes in production mode

has risks. For example, maintaining or evolving prototypes is difficult and may cause problems with features such as scalability, security, and robustness.

§  Also, quick deployment of prototypes in the early stages may create unrealistic expectations among customers. They become unwilling to accept longer development cycles that are necessary to develop more scalable and robust implementations as the product matures.

Test-driven Development

§  TDD is an evolutionary approach where developers create tests before writing new functional code.

§  TDD treats writing tests as part of a requirements/design activity in which a test specifies the code’s behavior.

Test-driven Development

Perceived Benefits §  Tests may be used to

capture complete requirements and design documentation that are linked to production code. This traceability makes incorporating changes easy.

Perceived Challenges §  A major challenge to TDD’s adoption is

that developers aren’t accustomed to writing tests before coding. Also, consistently following the practice demands a lot of discipline.

§  Moreover, TDD requires a thorough understanding of the requirements and extensive collaboration between the developer and the customer, because it involves refining low-level specifications iteratively.

§  Owing to these challenges, most organizations reported that they’re unable to implement this practice.

Reviews and Acceptance Tests §  Agile approaches use frequent review meetings for

requirements validation. §  At the end of each development cycle, a meeting that

involves developers, customers, Quality Assurance (QA), management and other stakeholders is scheduled. ♦  During the meeting, the features delivered are demonstrated;

customers and QA ask questions and provide feedback and comments.

♦  The review meetings show that the project is on target and within schedule, increase customer trust and confidence in the team, and highlight problems early.

Reviews and Acceptance Tests

Perceived Benefits §  Review meetings’ provide

opportunities §  to ascertain whether the

project is on target, §  to increase customer trust

and confidence in the team, and

§  to identify problems early during development.

§  Review meetings often help to obtain management support for the project by providing frequent updates on project status and progress to project sponsors.

Perceived Challenges §  While agile RE practices often focus

more on requirements validation than traditional approaches, they don’t address aspects of formal verification because there’s no formal modeling of detailed requirements.

§  Consistency checking or formal inspections seldom occur.

§  Implementing acceptance testing may be difficult owing due to lacking access to the customers who develop these tests. Thus, organizations often use QA personnel to help customers develop these tests.

Summary of Benefits & Challenges of Agile RE Practices Benefits Challenges Face-to-face communication

•  Customers can steer the project •  No time-consuming documentation

•  If no intensive interaction then bad reqts. •  On-site customer representation is difficult

Iterative RE •  Better relationship with the customer •  More understandable reqts

•  Cost & Schedule Estimation •  Lack of documentation •  Neglect of non-functional requirements

Extreme prioritization

•  Customers provide business reasons •  Opportunities for reprioritization.

•  Business value not enough •  May lead to instability

Constant planning

•  Minimizes the need for major changes •  Cost of addressing a change decreases

•  Early architecture becomes inadequate •  Refactoring isn’t always obvious

Prototyping •  Help communicate with customers to validate and refine requirements

•  Risky to deploy prototypes into production •  Create unrealistic expectations

Test-driven development

•  Gives traceability that make changes easier

•  Developers unused to test before coding •  Requires a thorough understanding of

reqts and extensive collaboration between the developer and the customer

Reviews & acceptance tests

•  Help to know if project is on target •  Increase customer trust and confidence •  Identify problems early •  Obtain management support

•  No formal model or verification of reqts •  Consistency checking or formal

inspections seldom occur. •  Difficult if lacking customer access

Project review of R2 R2 Mon W6, Validation report Fri W6

§  Release R2 for customer review ●  Think hard about how to get the most out of each hour spent on

the review; provide also needed clarifications, project plan etc. ●  Prepare the review by providing the customer group with a

Validation Checklist suitable for your project (Exercise 5!) ●  Validation Report created by the customer group should

contain relevant and useful issues ranked by criticality

§  See course program

Your two roles in validation

•  As developer make a useful checklist

•  As customer make a useful validation report

Project conference Mon W7 come 10:05 latest §  CP – Conference Presentation

♦  Email {.pdf | .ppt | .pptx } -presentation to Supervisor with cc to johan.linaker@cs.lth.se not later than Sunday the day before at 15.00 hrs we will use one computer, but bring backup computer + memory stick if possible

♦  Exactly!! 9 minutes presentation; will be interrupted! ♦  Contents:

●  ~ 1 minute about project mission ●  ~ 4 minutes overview of project results ●  ~ 4 minutes about methods and experiences

♦  Max 1 minute for switching to next group (no Q&A) ♦  One or max 2 presenters (not too much time on switching) ♦  Practice before to keep time and focus on the most important! ♦  If you want to practice English this is a good chance!

(Swedish is also Ok)

Order of examination:Project Mandatory attendance! Intro 10:05 A 10:10 B 10:20 C 10:30 D 10:40 E 10:50 --- 10 min break F 11:10 G 11:20 H 11:30 I 11:40 Outro 11:50

To do … §  Read Lau:9, [INSP], [INTDEP], [AGRE] §  Exercise W5 Validation (project validation preparation, bring

your System Requirements specification + litterature) §  Lab 2: ca 4h mandatory preparations (Labs in E:Venus) §  Project deliverables:

♦  Release R2 Mon W6 ♦  Validation Report Fri W6 based on checklist from other group ♦  Email .pdf (R2+checklist+Validation Report) to other group with Cc

to supervisor §  Second bonus chance Wednesday W6 before 23.59

Exam problems part 2; see guidelines L2.pdf §  Email Conference Presentation CP Sun W6

before 15.00 hrs supervisor AND johan.linaker@cs.lth.se Subject [ETS170] Conference Presentation Team<X>

§  No lecture in Week 6 §  Mon W7 be there 10.05 latest for PROJECT CONFERENCE

Mandatory examination!