non-functional requirements in software development projects: a systematic review

Download Non-Functional Requirements in Software Development Projects: A Systematic Review

If you can't read please download the document

Upload: adolph

Post on 16-Mar-2016

99 views

Category:

Documents


4 download

DESCRIPTION

Non-Functional Requirements in Software Development Projects: A Systematic Review. Dewi Mairiza [email protected] http://www- staff.it.uts.edu.au /~ mairiza / Faculty of Engineering and Information Technology University of Technology Sydney. Introduction Motivation The Study - PowerPoint PPT Presentation

TRANSCRIPT

  • Non-Functional Requirements in Software Development Projects: A Systematic ReviewDewi [email protected]://www-staff.it.uts.edu.au/~mairiza/

    Faculty of Engineering and Information TechnologyUniversity of Technology Sydney

  • OutlineIntroductionMotivationThe StudyObjectiveQuestionsMethodologyFindingsConclusionPublications*

  • Introduction*

  • Software Projects*

  • Software Projects*

  • Software ProjectsTo develop a software system based on stakeholders requirements, on budget and schedule*

  • Software ProjectsPeople or organizations who will be affected by the system and who have a direct or indirect influence on the system requirements (and those who will be involved in developingthe system).UsersDecision-makersLegislatorsDevelopers*

  • Software ProjectsA software system that suits the specified requirements*

  • Software ProjectsTo develop a software system based on stakeholders requirements, on budget and schedulePeople or organizations who will be affected by the system and who have a direct or indirect influence on the system requirements (and those who will be involved in developingthe system).UsersDecision-makersLegislatorsDevelopersA software system that suits the specified requirements?*

  • Software Requirements*

  • Software Requirements*

  • Software Requirements*

  • Motivation*

  • Non-Functional Requirements*

    NFRs Definition

    NFRs as the set of system properties/characteristics/constraints

    Quality Attributes

    Constraints

    Business Rules

    External Interfaces

    quality requirements, software system attributes

    NFRs as the Quality Attributes

    constraints, non-behavioral requirements

    First Perspective

    Second Perspective

    Similar Terms

    Similar Terms

  • Non-Functional Requirements the requirements that specify the desired quality attributes of the system 1,2,31Chung et al., 2000, 2Alexander & Maiden, 2004; 3Robertson & Robertson, 2006*

  • Non-Functional RequirementsCharacteristicSubjectiveRelativeInteracting Unlike FRs, NFRs are more abstract in natureNot uniform in nature*

  • Why NFRs?The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements.. No other part of the works so cripples the resulting system is done wrong. No other part is more difficult to rectify later 1

    A critical factor to the success of software projectsaddress the important issue of quality of the systemperform as constraints or qualifications of operationsoften more critical than individual FR in determination of systems perceived success or failure1Brooks 1987*

  • What happened in practice?Capturing, specifying, and managing NFRs are still difficult to performNFRs CharacteristicInadequate knowledge about NFRs Little help is available in the literature

    NFRs are often neglected, poorly understood and less consideredNot elicited at the same time and the same level of detail Poorly articulated in the requirements document *

  • The Study*

  • Objectiveto systematically investigate the notion of NFRs in the software engineering literature.*

  • QuestionsQ1. What is the typology of NFRs?Q2. Which types of NFRs are commonly considered or often discussed in the literature?Q3. Which types of NFRs are of concern in various types of systems? Q4. Which types of NFRs are of concern in various application domains? Q5. With respect to NFRs relative characteristic:Q5a. Can we develop a catalogue of conflicts among NFRs?Q5b. How can we create a conceptual model of the conflicts among NFRs?Q5c. Can software developers use this model to perform tradeoffs decision?

    *

  • Questions*

  • MethodologyExtensive systematic literature review

    Articles182 articlesSoftware engineering fieldPublished over the last three decadesAcademic resources and documents from software development industryCover various issues of NFRs and conflicts among them

    *

  • Methodology*

    Non-functional Requirements (NFRs) Research

    Identifying NFRs

    Managing NFRs

    Evaluating NFRs

    Product-Oriented

    Process-Oriented

    Modeling NFRs

  • MethodologyTechniqueContent analysis techniquemethod for summarizing any form of content by counting various aspects of the contentIt enables researchers to identify trends and patterns in the literature through the frequency of key words, coding and categorizing the data into a group of words with similar meaning or connotations It is applicable to all domain contexts

    The starting point for selection of the papers to be reviewed was the study conducted by Chung et al 2000.*

  • MethodologyResearch Framework:Creating a comprehensive catalogue of NFRs, their definition and attributes characterizationIdentifying the interdependencies among NFRs Performing the normalization process to the initial NFRs conflicts catalogue

    *

  • Findings*

  • Types252 types of NFRs114 types correspond to the NFRs definitions that have been discussed specifically in relation to the quality

    *

  • Types*

    1. Accessibility/Access Control2. Accountability3. Accuracy4. Adaptability5. Additivity6. Adjustability7. Affordability8. Agility9. Analyzability10. Anonymity11. Atomicity12. Attractiveness13. Auditability14. Augmentability15. Availability16. Certainty17. Changeability18. Communicativeness19. Compatibility20. Completeness21. Complexity/Interacting Complexity22. Composability23. Comprehensibility24. Comprehensiveness25. Conciseness26. Confidentiality27. Configurability28. Conformance29. Consistency

    30. Controllability31. Correctness32. Customizability33. Debuggability34. Decomposability35. Defensibility36. Demonstrability37. Dependability38. Distributivity39. Durability40. Effectiveness41. Efficiency/Device Efficiency42. Enhanceability43. Evolvability44. Expandability45. Expressiveness46. Extendability47. Extensibility48. Fault/Failure Tolerance49. Feasibility50. Flexibility51. Formality52. Functionality53. Generality54. Immunity55. Installability56. Integratability57. Integrity58. Interoperability59. Learnability

    60. Legibility61. Likeability62. Localizability63. Maintainability64. Manageability65. Maturity66. Measurability67. Mobility68. Modifiability69. Nomadicity70. Observability71. Operability72. Performance/Efficiency/ Time or Space Bounds73. Portability74. Predictability75. Privacy76. Provability77. Quality of Service78. Readability79. Reconfigurability80. Recoverability81. Reliability82. Repeatability83. Replaceability84. Replicability85. Reusability86. Robustness87. Safety

    88. Scalability89. Security/Control and Security90. Self-Descriptiveness91. Simplicity92. Stability93. Standardizability/ Standardization/Standard94. Structuredness95. Suitability96. Supportability97. Survivability98. Susceptibility99. Sustainability100. Tailorability101. Testability 102. Traceability103. Trainability104. Transferability105. Trustability106. Understandability107. Uniformity108. Usability109. Variability110. Verifiability111. Versatility112. Viability113. Visibility114. Wrappability

  • Types*

    have definition

    accuracy; analyzability; attractiveness; changeability; communicativeness; completeness; complexity; composability; confidentiality; consistency; correctness; defensibility; dependability; evolvability; extendability; flexibility; immunity; installability; interoperability; learnability; likeability; localizability; maturity; operability; quality of service; recoverability; replaceability; stability; suitability; survivability

    without definition and attributes

    accountability; additivity; adjustability; affordability; agility; anonymity; atomicity; auditability; augmentability; certainty; compatibility; comprehensibility; comprehensiveness; conciseness; configurability; conformance; controlability; customizability; debuggability; decomposability; demonstrability; distributivity; durability; effectiveness; enhanceability; expandability; expressiveness; extensibility; feasibility; formality; generality; legibility; manageability; measurability; mobility; nomadicity; observability; predictability; provability; reconfigurability; repeatability; replicability; self-descriptiveness; simplicity; standardizability; structuredness; supportability; susceptibility; sustainability; tailorability; traceability; trainability; transferability; trustability; uniformity; variability; verifiability; versatility; viability; visibility; wrappability

    have definition and attributes

    accessibility; adaptability; availability; efficiency; fault tolerance; functionality; integratability; integrity; maintainability; modifiability; performance, portability; privacy; readability; reliability; reusability; robustness; safety; scalability; security; testability; understandability; and usability

  • Types*

    NFRsDefinitionAttributesPerformancerequirements that specify the capability of software product to provide appropriate performance relative to the amount of resources needed to perform full functionality under stated conditions response time, space, capacity, latency, throughput, computation, execution speed, transit delay, workload, resource utilization, memory usage, accuracy, efficiency compliance, modes, delay, miss rates, data loss, concurrent transaction processing

  • Most Frequent NFRsTop five of the most frequent types of NFRs listed in the NFRs catalogue: performance (88.68%) reliability (67.92%) usability (62.26%) security (60.38%) maintainability (54.72%)

    *

  • Most Frequent NFRs*

  • NFRs VS Type of Systems*

    Type of SystemDescriptionExampleReal-time Systemsthose systems that can support the execution of applications with time constraints on that execution (Laplante 2004)reactive system and embedded systemSafety Critical Systemsthose systems whose failure could result in loss of life, significant property damage, or damage to the environment (Knight 2002)nuclear system, medical system, air traffic control systemWeb Systemsthose systems which utilize Web technologies as an integral element of a functionally complex system and which typically incorporates interfaces beyond the organizational boundaries (Yusop, Zowghi & Lowe 2008)e-commerce application, learning management system, online systemsInformation Systemsthose systems which consist of a set of interrelated components that collect (input), manipulate (process), and disseminate (output) data and information and provide a feedback mechanism to meet an objective (Stair & Reynolds 1999)laboratory information system, management information system, purchasing systemProcess-Controlled Systemsthose systems that drive the operation of a hardware process (Easterbrook 2005; Leveson et al. 1994)light control system, fire alarm system, ship command system, space craft

  • NFRs VS Type of System*

    AccuracyAvailabilityCommunicativenessCompatibilityCompletenessConfidentialityConformanceDependabilityExtensibility10.Installability11.Integrity12.Interoperability13.Maintainability14.Performance15.Privacy16.Portability17.Provability18.Reliability19.Reusability20.Safety21.Scalability22.Security23.Standardizability24.Traceability25.Usability26.Verifiability27.Viability

  • NFRs VS Type of SystemPerformance, security, usability the most common NFRs in each software being developed*

  • NFRs VS Application DomainDigital Industry Taxonomy

    *

    Application DomainRelevant NFRsBanking and Financeaccuracy, confidentiality, performance, security, usabilityEducationinteroperability, performance, reliability, scalability, security, usabilityEnergy Resourcesavailability, performance, reliability, safety, usabilityGovernment and Militaryaccuracy, confidentiality, performance, privacy, provability, reusability, security, standardizability, usability, verifiability, viabilityInsuranceaccuracy, confidentiality, integrity, interoperability, security, usabilityMedical/Health Carecommunicativeness, confidentiality, integrity, performance, privacy, reliability, safety, security, traceability, usabilityTelecommunication Servicescompatibility, conformance, dependability, installability, maintainability, performance, portability, reliability, usabilityTransportationaccuracy, availability, compatibility, completeness, confidentiality, dependability, integrity, performance, safety, security, verifiability

  • NFRs VS Application DomainType of System and Application DomanPerformance and UsabilitySecurity *

  • NFRs Conflicts Catalogue*

  • NFRs Conflicts CatalogueThe relativity of conflicts is characterized:Absolute conflictAll pairs of NFRs that are always in conflict. Labeled as XRelative conflictAll pairs of NFRs that are claimed to be in conflict in the certain cases but they are also claimed as not being in conflict in the other cases. Labeled as *Never conflictAll pairs of NFRs that have never been declared as being in conflict with each other. They may contribute either positively or indifferent to one another. Labeled as O*

  • NFRs Conflicts CatalogueAnalysis:Conflicts36 pairs have absolute conflict19 pairs have relative conflict50 pairs have never conflictThe rest of relationships are not known due to there is no information available in the literature about how those pairs of NFRs contribute to each other presented as the blank spaces

    NFRs with the most conflict with other NFRs is Performance *

  • NFRs Conflicts CatalogueAnalysis:Self-conflicting relationshipConflicts between the attributes of a single NFRsE.g. response time VS capacity (performance)increasing the number of concurrent users in the system may diminish the response time of the system

    *

  • NFRs Conflicts Catalogue*

    Conflicting NFRsNature of Conflict%Security and Performanceabsolute33%Security and Usabilityrelative23%Availability and Performanceabsolute20%Performance and Portabilityabsolute17%Reusability and Performanceabsolute17%Interoperability and Performance absolute10%Maintainability and Performanceabsolute10%Reliability and Performance relative10%Usability and Performancerelative10%Usability and Reusability absolute3%

  • ConclusionPurpose of study: to systematically investigate the notion of NFRs in the software engineering community

    Covers four essential issues: (1) definition and terminology; (2) types; (3) NFRs VS types of systems and application domains; (4) conflicts among NFRs

    Limitationthe potential overlaps that exist among definitions and attributes of each NFRs were not investigated; this study does not have the intention to create a structural hierarchy of NFRs types.

    *

  • ConclusionContribution:software engineering research communityto improve understanding about the notion of NFRs; to suggest community reaching a consensus on several NFRs dimensions (e.g. definition, scope, terminology, types, and taxonomy); the top five most considered NFRs presented in this paper (performance, reliability, usability, security, and maintainability) are expected to inform and encourage the research community to perform in-depth studies about these NFRs *

  • ConclusionContribution:software developersthe comprehensive list of NFRs types will let developers know what types of NFRs are there for the system being developed. the matrix of relevant NFRs is expected to help developers to identify the important NFRs for the particular system being developed. the matrix of relevant NFRs can also help the elicitation process by making sure that in the elicitation activity, those relevant NFRs have been discussed with the system stakeholdersCatalogue of NFRs conflicts can be used to identify the NFRs conflicts in various phases of SW development projects*

  • Future WorksWith respect to NFRs relative characteristic:Q5b. How can we create a conceptual model of the conflicts among NFRs?Q5c. Can software developers use this model to perform tradeoffs decision?*

  • Future WorksProgress so far Ontological approach; Helix-Spindle model for ontological engineeringSecurity - Usability conflicts ontology and its knowledge-basesureCM Frameworkto identify the security usability conflictsto characterize the conflictsto perform tradeoffs decision

    MaRK Workshop 18th IEEE International Requirements Engineering Conference (RE10)*

  • PublicationsLNCS CCIS Book Series, Springer-Verlag 2011 (to be appeared)Mairiza, D. and D. Zowghi (2011). Constructing a catalogue of conflicts among non-functional requirements. Evaluation of novel approaches to software engineering, Springer-Verlag.

    CCF SIGRE Workshop 2011, Beijing - PRCMairiza D., The 1st CCF SIGRE Workshop 2011, Beijing, China, 23 February 2011.

    Schlumberger FFTF Forum 2010, Abu Dhabi - UAE Mairiza D., Investigating Conflicts among Software Non-Functional Requirements, Proceedings of the 2010 Faculty For The Future Forum (FFTF 2010), the Schlumberger Foundation, Abu Dhabi, United Arab Emirates, 4-8 December 2010.*

  • PublicationsPeking University 2010, Beijing - PRCMairiza D., Investigating Conflicts among Non-Functional Requirements, Peking University Workshop, Beijing, China, 20 October 2010.

    MaRK 2010 in conjunction with RE 2010, Sydney - AustraliaMairiza D., Zowghi D., An Ontological Framework to Manage the Relative Conflicts between Security and Usability Requirements, Proceedings of the Third International Workshop on Managing Requirements Knowledge (MaRK 2010), in conjunction with the 18th IEEE International Requirements Engineering Conference (RE10), Sydney, Australia, 27th September 2010.

    ENASE 2010, Athens - GreeceMairiza D., Zowghi D., Nurmuliani N., Towards a Catalogue of conflicts among Non-Functional Requirements, Proceeding of the 5th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2010), Athens, Greece, 2010.

    *

  • PublicationsACM SAC 2010, Sierre - SwitzerlandMairiza D., Zowghi D., Nurmuliani N., An Investigation into the Notion of Non-Functional Requirements, Proceeding of the 25th ACM Symposium on Applied Computing (ACM SAC 2010), Sierre, Switzerland, 2010.

    AWRE 2009, Sydney - AustraliaMairiza D., Zowghi D., Nurmuliani N., Managing Conflicts Among Non-Functional Requirements, Proceeding of the 12th Australian Workshop on Requirements Engineering (AWRE 2009), Sydney, Australia, 2009.*

  • Thanks to Faculty for the Future Award, the International Schlumberger Foundation, ParisUTS IRS Award 2009Australia Research Training Scheme, 2010Prof. Didar Zowghi, University of Technology Sydney, AustraliaDr. Vincenzo Gervasi, University of Pisa, ItalyDr. Nurie Nurmuliani, QBE AustraliaProf. Kevin Ryan, Lero (The Irish Software Engineering Research Centre) and University of Limerick, Ireland Prof. Zhi Jin, Chinese Academy of Science, PRCProf. Lin Liu, Tsing Hua University, PRC*

  • Thank You*

  • *---Let we start from a software projects.

    *---When we talk about the software projects, there might be a lot of things come into our mind.

    *For example:Goal; Requirements, Stakeholders, Life Cycle Methodology, etc.**Stakeholders (Ian Sommerville, Kotonya, Ollie Gotel, and Anthony Finkelstein) --- (Newman and Lamming, Interaction Design). - Users - Decision-makers: financial controller, manager of developers and users organizations - Legislators e.g. professional bodies, government, trade union, quality assurance auditors, etc - Developers

    Which each of them has different requirements of the system.

    *Outcome is a new software that suits the specified requirements.

    And maybe there are some other things related to software project such as methodology, etc.*When we look at to Goal, Stakeholders, Outcome, and some other aspects here, we see this keyword, REQUIREMENTS.

    And today we just only focus on software requirements.*What is a software requirement?

    In the simple words, a requirement is .

    And software requirements specification is a complete

    Commonly, requirements can be classified into:Functionaland non-functional requirements*

    Today, well focus on the second type of requirements, which is NFRs.**

    Now, let we zoom in the NFRs.

    **This study considers NFRs as the requirements that

    *Subjective they can be viewed, interpreted, and evaluated differently by different people.

    Relative the interpretation and importance of them may vary depending on many factors, such as the context of the system being developed and the extent of stakeholders involvement.

    Interacting they tend to interfere, conflict or contradict with each other. Achieving a particular type of NFRs can hurt the achievement of the other types of NFRs.

    In software development, customers often state NFRs as general goals such as ease of use, the ability to recover from failure, or good response time.

    There are a large number of different types of NFRs, such as security, usability, and maintainability. Each of them has different characteristics and role during software development life cycle. For instance, security requirements describe various aspects about the security of the system such as authorization, privacy, and authentication while usability requirements describe various aspect of ease of use and user friendliness of the system.

    *The first reason is because the hardest single part . And NFRs in one of those requirements.------------------------------------------------------------In software development, NFRs are recognized as a critical factor to the success of software projects:

    If NFRs are not addressed adequately, a number of potential problems may occur. For instance software which is inconsistent and of poor quality; dissatisfaction of clients, end-users, and developers; causing time and cost overrun for fixing software errors.

    NFRs place restrictions on the product being developed, the development process, and specify external constraints that the product must exhibit.

    Neglecting NFRs has led to a series of software failures, such as systemic failure in London Ambulance System, the system failure in the New Jersey Department of Motor Vehicles Licensing System, etc.

    *Studies to date indicate that capturing, specifying, and managing NFRs are still difficult to perform.

    NFRs CharacteristicThese characteristics make NFRs difficult to deal with. It is more difficult to model, to verify, to test, and to measure NFRs to compare with FRs.

    Most of software developers do not have adequate knowledge about NFRsAnd little help is available in the literature. In the requirements engineering literature, NFRs have received less attention and not as well understood as FRs. Majority of software engineering research, only deal with FRs, i.e. ensuring that the necessary functionality of the system is delivered to the user. -----------------------------------------------------------------As the result, NFRs are often neglected, poorly understood and not considered adequately in developing a software application.

    Empirical studies reveal that in the development of software system:NFRs are not elicited at the same time and the same level of details as the FRs They are also often poorly articulated in the requirements document users naturally focus on specifying their functional requirements software developers commonly do not pay sufficient attention to NFRs

    ** For this study, we did an extensive systematic literature review This study has examined 182 sources of information. All of them are literatures within the discipline of software engineering. They are published over the last three decades They come from academic resources and documents from software development industry They cover various issues of NFRs and conflicts among them

    *Content analysis is a method for summarizing any form of content by counting various aspects of the content.*To develop a catalogue of NFRs conflicts, a research framework was followed. This framework consists of three research stages: ----------------------------------------------------------------------- to create a comprehensive catalogue of NFRs types, their definition and attributes characterization to identify the interdependencies among various NFRs types to perform a normalization process to standardize the NFRs in the conflicts catalogue Normalization is the process of removing the irrelevant NFRs, i.e. the types of NFRs that do not have definition and/or attributes, from the initial catalogue.

    (1) Stage 1:-----------------Since there is no standard catalogue of NFRs types available in the literature and previous studies (Glinz 2005, 2007; Mairiza, Zowghi & Nurmuliani 2010) also claimed that many types of NFRs were introduced without definition or attributes characterization, the first stage of the research was creating a comprehensive catalogue of NFRs types. Each type of NFR discussed in the literature was recorded. The definition and attributes correspond to each of NFR type were also documented. Conflicting terminologies and definitions were handled through the frequency analysis technique and keywords identification.

    Stage 2:------------The second stage of the research was creating an initial catalogue of the conflicts among NFRs. In this stage, NFRs conflict relationships were used as the criteria to develop the catalogue. This stage was initiated by identifying the interdependencies among various types of NFRs. These interdependencies represent the typical interrelationships of a particular type of NFR towards another type of NFR (e.g. positive, negative, or neutral interrelationships). This investigation produced the initial catalogue that presents the conflict relationships among 26 types of NFRs.

    Stage 3:------------The next stage was performing a normalization process against 26 types of NFRs that have been identified in the initial catalogue. This normalization was conducted in order to standardize the data obtained in the previous stage. Normalization is the process of removing the irrelevant NFRs, i.e. the types of NFRs that do not have definition and/or attributes, from the initial catalogue. The objective is to produce a conflicts catalogue of the well-defined NFRs types. In this normalization, the catalogue of NFRs types, their definition, and their attributes are utilized as the basis of removing those irrelevant NFRs. This process has removed six NFRs from the initial catalogue. They are compatibility, expressiveness, legibility, provability, verifiability and analyzability. Therefore, the final conflicts catalogue is a two-dimensional matrix that represents the conflicts interrelationships among 20 types of normalized NFRs.

    *among these 252 types, 114 types correspond to the NFRs definitions that have been discussed specifically in relation to the quality *Among these 114 types:23 types of NFRs (20.18%) have definition and attributes30 types (26.32%) only have definitionand the rest 61 types (53.50%) were introduced without definition or attributes.

    *One of the NFRs types with its definition and attributes characterization.*Five: performance, security, usabilityFour: reliability

    *Adopting a well-known application domain taxonomy from the Digitals Industry Taxonomy (Contemporary Application Domain Taxonomies, Glass 1995, IEEE Software)-------performance and usability in almost all application domains (7 out of 8 domains); security 6 domains; confidentiality 5 domains; accuracy and reliability 4 domains*Performance and Usability are the most commonly considered NFRs in various types of systems and application domains, followed by Security.

    *Two-dimensional matrix represents the typical interrelationships among NFRs in term of the conflicts emerge among them.

    * The investigation also indicates that certain attributes of a particular type of NFR can be in conflicts with each other. These conflicts point to the self-conflicting relationships for a particular type of NFR. Self-conflicting relationship is defined as a case where the attributes of a single type of NFR are in conflict. - - For example, the relative conflict between performance and performance requirements. Performance requirements can be characterized among others by response time and capacity. In many systems, these two attributes are in conflict, because increasing the number of concurrent users in the system may diminish the response time of the system.

    For example in a road traffic pricing system (Brito & A. 2004; Moreira, Araujo & Brito 2002), multi-user attribute has negative contribution to the response time of the system. *Catalogue of conflicts can be used to identify the NFRs conflicts in various phases of SW projectRequirements Phase - Identify the NFRs conflicts early - Discuss the potential conflicts with systems stakeholder when specifying the software requirements

    Design Phase Investigate the potential architecture strategies for the best solution

    **