4 dr fusani

NATIONALRESEARCHCOUNCIL

Ongoing research activity atISTI-CNR, Pisa, Italy

Mario Fusani, [email protected]


Where

SAFEGUARD Summer School – Odessa, June 2012 2

Pisa


CNR area, Pisa (from the website)



ISTI (from the website)



REQUIREMENTS ENGINEERING

Crucial area, especially in safety-related applications.

Many related themes:elicitation and expressionanalysiscrossing application domainscommunication and managementdirect system behaviour

(reqs. animation)continuous evolutionreqs. changing impact in systems

and environment .........



Why Quality in Requirements ?

Quality in architecture,Quality in code,Quality in documentation,

...



Quality perspective

What is quality ?expectation (stakeholders)wish(good) surprisegoals of useobjective measurement vs. subjectivityquality models (QM)

definable and measurable properties perception among stakeholders

structured attributes/properties set = set + relationships



Why Quality with Safety ?

From Safety Integrity Levels:Quality Management is often an expression of

non-safety-related !

But in our vision Quality is related with objectives and expectations about Safety



A possible view for a Requirements QM



Simpler and straightforward:Meyer’s seven sins (1985)

Noise (no relevant information to any object feature)Silence (object features not covered by any element of the text)Over-specification (elements that correspond not to an object

feature (what) of the problem but to features of a possible solution (how))

Contradiction (elements that define a feature of the system in an incompatible way)

Ambiguity (elements that make it possible to interpret a feature of the problem in at least two different ways)

Forward reference (elements using features of the problem not defined until later in the text)

Wishful thinking (elements that define a feature of the problem in such a way that a candidate solution cannot reasonably be validated)



“sin” interpretation

The important thing is to be aware of the “sin” and decide if it is really a sin:example : including necessary/opportune implementation constraints is no over-specification



Building a (possible) Requirements QM



Building a Requirements QM: Contents side


Why Contents in QM ?Contents still an expectation


Building a Requirements QM: Quality side



RE areas where ISTI is at work



Why Natural Language

Expression of first conceived RequirementsEasy sharing among stakeholders

(implementers, test script editors, user manual authors, marketing staff, product users, customers, ...)

Understandability and ambiguity (possible different meanings in stakeholders) problems

Various research lines in literature to cope with such problems, some related work at ISTI



by the way

the previous quality model suffers itself of non-quality properties such as mixing abstraction levels and including ambiguities

needs explanationsbut helps people thinking and formulating

research questions



RE area for understandability



Lexical & syntactic (L&S) approach

based on a specific quality model against:

optionality ( “this” or “that”, “if needed”, ... ) subjectivity (“simple”, “known”, ... ) vagueness (“adequate”, ”easy”, ... ) weakness (“can”, ... ) implicitly (“the previous task”, “it”, ... ) under-specification (“access to”, ... ) multiplicity (“< sentence> and

<sentence>”, ...)

that is a “defect model”SAFEGUARD Summer School – Odessa, June

2012 25


A L&S analyser

Assumption:sentences having lexical – syntactical

characteristics defined in the previous “defect model” bear the risk of understandability

Approach:QuARS tool (conceived at ISTI in 1996), ised in:

projects and partially in rail industrial environment standard evaluation



L&S: Modcontrol (rail project) experience


Requirements

Analyzed Requireme

nts (Absolute Values)

Defective Requireme

nts (Absolute Values)

Defect Rate (Percentage values)

FREQ 3.209 1.647 51,3

SREQ 2.568 1.279 49,8

TOTAL 5.777 2.926 50,6


L&S: Some lifecycle Standard analysis results ...



(L&S)... with respect to functional SRS analysis


Multiplicity less than in LC Standards, probably for less verbose sentences


L&S limitations

Human judgment in the analysis process (50% false positive from experiments)

needs pre and post-processinguseful in batch requirements processing when

requirements analysis is mandatoryDifficult to express semantics through L&S



Others recent approaches to the ambiguity problem

Pragmatic (contextual) ambiguityAmbiguity from clustering analysis



RE area for ambiguity



“Pragmatic” (contextual and experience-based) ambiguity

Assumption:Text elements are at risk of ambiguous interpretation

when strongly dependent on individual (stakeholder’s) knowledge base (KB)

Approach:simulation of different “personal” (technical) KB’s by

web-mining (using NL processing techniques) letting a Requirements document “interact” by sentences

with the artificial KB’s and checking the results against defined similarity

parameters (a purposely defined metrics for pragmatic ambiguity)



Pragmatic ambiguity: example resultsReq: “The system shall display similar books based on the user

preferences of other users who purchased the same book during previous sessions”

Elaboration (with “stems” in place of words) of paths combining the Req and various KB’s (red):

P1(R5) = {‘system’, ‘shall’, ‘display’, ‘similar’, ‘user’, ‘may’, ‘avail’, ‘book’, ‘movi’, ‘user’, ‘base’, ‘user’, ‘prefer’, ‘user’, ‘purchas’, ‘onlin’, ‘store’, ‘will’, ‘play’, ‘music’, ‘book’, ‘movi’, ‘user’,‘previou’, ‘session’};

P2(R) = {‘system’, ‘shall’, ‘display’, ‘similar’, ‘user’, ‘recommend’, ‘book’, ‘recommend’, ‘base’, ‘user’, ‘prefer’, ‘user’, ‘recommend’, ‘product’, ‘purchas’, ‘store’, ‘user’, ‘recommend’, ‘book’, ‘previou’, ‘session’}

Req gives relatively lower similarity among KB’sReq’: “The system shall display similar books based on content-

based filtering”Req’ gives sensibly higher similarity (“session” is indeed an

ambiguous term)SAFEGUARD Summer School – Odessa, June

2012 34


Pragmatic ambiguity - results

Paper - "Using Collective Intelligence to Detect Pragmatic Ambiguities” (A. Ferrari, S. Gnesi)to appear in RE 2012, September 24th-28th, 2012. Chicago, Illinois, USA

metrics for pragmatic ambiguityexperiencevalidation



Ambiguity detection by cluster analysis

Assumption:there is risk of ambiguity if the frequency of

“isolated concepts” is above a defined threshold

Approach:To find, with NL processing techniques,

aggregates of terms.Defining and evaluating a metrics for

“distance” among termsFinding outliers



RE area for Expressive Requirement Structuring



Expressive Requirement Structuring

Finding relationships among functional requirements

Structuring specific domain Requirements for product line development (rail domain)

One objective of the EU-funded project Trace-IT (about interoperable train protection and control): Communication Based Train Control (CBTC) functions




Approach:Commonalities are found and defined through

an activity of reqs elicitation in CBTC, taking from: Functional Standards in target domain Known proposals and solutions by relevant vendors Typically known scenarios in the operational

environmentVariabilities (not addressed) are typically

defined basing on customer requests and market analysis





Deriving product commonalities and (sub)systems relationships (systems of systems)



Paper “Product Line Engineering Applied to CBTC Systems Development” (A. Ferrari, G. O. Spagnolo, G. Martelli, S. Menabeni) inISOLA 2012, 5th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation, 15-18 October 2012 - Amirandes, Heraclion, Crete



RE areas for Standard requirements consistency



Consistency issues in safety-related Standards

Research questions:Are standard-defined requirements such that

the organizations entitled to adopt them can easily understand what to do to be conformant?

- Are standard-defined requirements such that the entitled independent verification bodies can easily decide on compliance?

- In case of similar but different standards, what is the minimum effort to be sustained by an organization to achieve, when requested, multiple-compliance?



Consistency issues in safety-related standards

Terms are defined in special clausesHow are such terms used in the standard

body ?In order to reduce the ambiguity risk, each

Standard adopts its glossary, but are the definitions of the keywords really useful to disambiguate? And how much?




Assumption:A standard text is at risk of inconsistency when

internally-defined terms/expressions are used in the standard body in a context different than that of the definition

Approach:using NL processing simple technique of

“concordance” (full text is explored by sample text and window width)




we limit Standard list and keywords (only 3: error, fault and failure) with their correlates key expressions


. . . . . . . . . . . . . . . . . . . . . . . .




Expression definedat least in

non definedand used

“Common Cause Failure”

IEC 61508-4ISO 26262

EN 50128

“Failure rate” ISO 26262 EN 50128DO 178B

“Fault detection” ISO 26262 EN 50128DO 178B

•Analysis in progress

•Emerging facts so far:


Conclusions

Even limited to the examined topics:NL UnderstandabilityNL AmbiguityConsistent use of termsExpressive Relationships across requirements

product lines requirements “filtering”

There are still many research questions



Research questions

QMrelationships among QM elementscompletenessabstraction levels

Consistency Syntax active role in Lexical Understandability

and Pragmatic AmbiguityFrom NL requirements to implementation

(modeling)..........


4 dr fusani

Documents