© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Using Natural Language Parsing (NLP) for Automated Requirements Quality Analysis

Chris Ritter, Daniel Hettema, Steven H. Dam, Ph.D., ESEP, SPEC Innovations

April 2014

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

2

Requirement LevelsThe Requirements Hierarchy

Nu

mb

er o

f R

equ

irem

ents

, L

evel

of D

etai

l In

crea

ses

User Needs

Conceptual Requirements

System Requirements

Application/ComponentSpecifications

A capability or feature identified by a User as being needed to perform his mission

A high-level requirement generated during the concept development phase. Contained in ORD, CONOPS

A requirement that describes in technical language the desired capabilities of a system.

Requirement that is at the level of detail needed for actually designing a new capability. Contained in System Requirements Specification (SRD)

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

3

Characteristics of Good Requirements• Each individual requirement should be:

– Correct: Describes the user’s true intent and is legally possible

– Complete: Express a whole idea– Clear: Unambiguous and not confusing– Consistent: Not in conflict with other requirements– Verifiable: Provable (within realistic cost and

schedule) that the system meets the requirement– Traceable: Uniquely identified, and able to be tracked

to predecessor and successor lifecycle items/objects– Feasible: Able to be implemented with existing

technology, and within cost and schedule– Modular: Can be changed without excessive impact

on other requirements– Design: Does not impose a specific solution on

design; says “what”, Independent not “how”

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

4

Avoid these Pitfalls• DON’T use ambiguous language• DON’T use bullet lists; use numbered lists instead • DON’T use jargon• DON’T use language that provides an escape clause

– Ex: “The user shall be able to access the Internet as often as is practicable”

• DON’T write long, rambling sentences• DON’T put two requirements in one sentence; e.g., “The

system shall … and …”• DON’T use vague terms -- Ex: “user-friendly”• DON’T include suggestions or possibilities – Ex: “may”,

“should”, “ought”• DON’T include wishful thinking – Ex: “The system shall be

100% reliable”

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

How Do We Ensure Quality Requirements?

• Follow the “rules” for what makes a good requirement

• Enforce the rule requires significant training and discipline

• Most tools help “manage” the requirements, but do not help you track the quality of the requirements

5

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Solution 1: Identify Requirements on Import

• Innoslate’s one button Import Analyzer identifies requirements vs. statements (context for a requirements) by using a key word search (“will”, “shall”, “should”, or “requirement”)

• Requirements entities have quality attributes associated with them

• The user can transform a statement to a requirement and vice versa

6

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Requirement Entities• Each quality factor

has a Yes/No value to determine if it meets the requirement

• Uncertain what attributes means?– Hover over name

and definitions appear

7

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Solution 2: Innoslate’s Quality Check

8

• Run quality checker to automatically analyze the quality attributes of the requirements

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Quality Check Uses NLP Technology

• Natural Language Processing: – “a field of computer science, artificial

intelligence, and linguistics concerned with the interactions between computers and human (natural) languages”

• Innoslate uses this technology to break down sentences into nouns, verbs and adjectives to identify when conjunctions are used (clear), specific types of hardware/software specified (design), and other parameters that affect the requirement’s quality

9

Wikipedia

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Solution 3: Roll-up Quality Score

• Change values from No to Yes to adjust score upward

10

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Solution 4: Add Comments

• Authors and reviewers can add comments and describe changes in more detail

11

© 2014 Systems and Proposal Engineering Company. All Rights Reserved

Summary

• We need to do more than just “manage” requirements

• Innoslate brings new technology (NLP) into the automation of requirements analysis

• However, it does not substitute for good engineering judgment – it is meant as a way to cue the analyst to potential problems

12