understanding quality goals
TRANSCRIPT
![Page 1: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/1.jpg)
Understanding Quality Goals
© 2015 ClearSpecs Enterprises 1
David Gelperin, CTOClearSpecs Enterprises
1. Quality crisis and candidate causes (7)2. Requirements risk management (4)3. Tactics for managing quality awareness (10)4. Quality Assumption Reviews (15)5. Resolving the quality crisis (3)
![Page 2: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/2.jpg)
Developers don’t understand quality goals
© 2015 ClearSpecs Enterprises 2
Many developers have a shallow understanding of how to achieve quality goals such as safety, security, and availability.
Developer ignorance endangers project results.
![Page 3: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/3.jpg)
© 2015 ClearSpecs Enterprises 3
Customers want functions
![Page 4: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/4.jpg)
The Quality Crisis
There is a self-inflicted quality crisis in progress. Websites are hacked or fail to handle large loads and we shrug. Failure teaches us little and changes nothing.
© 2015 ClearSpecs Enterprises 4
This may be a “henny penny” (or not)
The resolution: Increased quality awareness and learning
![Page 5: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/5.jpg)
Functions
Qualities
Over-emphasis on functionality
© 2015 ClearSpecs Enterprises 5
![Page 6: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/6.jpg)
FunctionalRequirementsi.e. behaviors
What programmers need to know
© 2015 ClearSpecs Enterprises 6
FunctionalRequirementsi.e. behaviors
Delete reservation & refund payment (for example)
Functional components are crosscut by quality supports e.g. exception handlers
FunctionalRequirementsi.e. behaviors
• Input verification• Request validation• Exception handling• Logging
et. al.
• Safeguards• Security guards• Encryption• Testpoints
![Page 7: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/7.jpg)
© 2015 ClearSpecs Enterprises 7
Becoming Quality-Aware
Understanding that functionality is just the beginning
![Page 8: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/8.jpg)
© 2015 ClearSpecs Enterprises 8
• Goals are numerous (50 +) and crosscutting (pervasive)• Some are software engineering subfields• Attributes (e.g. quality level) are numerous (20 +) and complex• Goals may conflict (e.g. create infeasible pairs) or support• Goals have ad hoc mitigation and support tactics• Tactics are situation-specific• Goals are hard to verify• The nature, scope, and verification of quality goals are not taught• Achieving quality goals is drudgery i.e. no fun• Quality goals are “ignored” by Agile
Candidate causes of shallow understanding
![Page 9: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/9.jpg)
QualityGoalsAhead
© 2015 ClearSpecs Enterprises 9
Qualities are dangerous
![Page 10: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/10.jpg)
ProjectRisk
Mgmt
Requirements Risk Mgmt (RM)
© 2015 ClearSpecs Enterprises 10
![Page 11: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/11.jpg)
11
Searching for the Golden Mean
© 2015 ClearSpecs Enterprises
![Page 12: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/12.jpg)
ProactiveRequirements
Risk Mgmt
Quality Awareness
Mgmt
Waste (Technical Debt)
Mgmt
© 2015 ClearSpecs Enterprises 12
![Page 13: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/13.jpg)
a. Holding quality assumptions reviews (temporary tactic)b. Creating rich project quality models (RPQMs)c. Holding (mini) "quality attribute workshops“ using RPQMsd. Creating "quality scenarios"e. Sketching verification strategies for project quality goalsf. Recording quality learningsg. Holding brown-bag lunch sessions to share quality experiencesh. Running "quality discussion groups" that read book chapters and papers
© 2015 ClearSpecs Enterprises 13
Find resources at quality-aware.com
Tactics for managing quality awareness
![Page 14: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/14.jpg)
© 2015 ClearSpecs Enterprises 14
b. Creating Rich Project Quality Models (1)
ISO 25010 and its peers are meager quality model templates
![Page 15: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/15.jpg)
© 2015 ClearSpecs Enterprises 15
b. Creating Rich Project Quality Models (2)
This is a meagerproject quality model
![Page 16: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/16.jpg)
© 2015 ClearSpecs Enterprises 16
b. Creating Rich Project Quality Models (3)
LiteRM is a rich quality model template
Rich ProjectQuality Model
![Page 17: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/17.jpg)
Rich quality model templates are analogous to
© 2015 ClearSpecs Enterprises 17
b. Creating Rich Project Quality Models (4)
on
Software Quality Goals
![Page 18: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/18.jpg)
Rich quality model templates (and development standards) are natural places to record quality lessons.
Without them, lessons are lost.
© 2015 ClearSpecs Enterprises 18
f. Recording Quality Learnings
![Page 19: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/19.jpg)
Isn’t verification just a fancy word for testing? No
Then, what is it? A composite checking activity that includes analysis e.g. hazard analysis, technical reviews e.g. code inspections, and measurement e.g. mean time between failures, as well as testing.
Why should I care?Quality goals must be verified. Testing alone is inadequate e.g. security testing is not enough, also need threat analysis and security guard code inspections.
© 2015 ClearSpecs Enterprises 19
e. Verification Q & A
![Page 20: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/20.jpg)
© 2015 ClearSpecs Enterprises 20
e. Sketch & Execute Verification Strategies Early
TestThen
Code1986
VerifyThen
Code2015
Preventative Testing (TDD) Preventative Verification (VDD)
![Page 21: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/21.jpg)
“IT projects that applied NFR (Non-Functional Requirement) verification techniques relatively early in development were more successful on average than IT projects that did not apply verification techniques (or applied them relatively late in development)”
- Eltjo Poort, Nick Martens, Inge van de Weerd and Hans van Vliet“How Architects See Non-Functional Requirements:
Beware of Modifiability” REFSQ 2012 [Best Paper]
© 2015 ClearSpecs Enterprises 21
e. Why verify early?
![Page 22: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/22.jpg)
[See sample on last page of handout]
© 2015 ClearSpecs Enterprises 22
e. Sketching a Verification Strategy
![Page 23: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/23.jpg)
• To synchronize understanding of quality goals, how they differ from functional requirements, and how they are achieved.
• To build a culture of quality-awareness
• To prepare for quality goal identification
© 2015 ClearSpecs Enterprises 23
a. Quality Assumptions Review – Why?
![Page 24: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/24.jpg)
1. Create a list of basic assumptions [see handout]2. Distribute assumptions prior to review and
encourage discussions3. During the review, various reviewers:
a. Read assumptionb. Ask for questionsc. Ask for problems with assumption or its statementd. Ask for disagreements and reasons
4. Record open issues and comments during the review5. Distribute resolutions after the review
© 2015 ClearSpecs Enterprises 24
a. Quality Assumptions Review – How?
![Page 25: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/25.jpg)
© 2015 ClearSpecs Enterprises 25
Assumption 3
Quality goals, their relationships, and attributes are invariant across application domains and should be described in a rich quality model template. Relevance, priorities, levels, and implementations must be determined while developing the project quality model.
![Page 26: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/26.jpg)
© 2015 ClearSpecs Enterprises 26
Assumption 4
Most quality goals (e.g. safety, throughput) crosscut functions and thus can’t be implemented in a single increment and have an increasing cost to implement or repair across iterations.
![Page 27: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/27.jpg)
© 2015 ClearSpecs Enterprises 27
Assumption 5
Some quality goals entail levels of achievement e.g. security, privacy, and performance. For example, if security is relevant, should it be minimal using a user name and password, moderate using user name, password, and security questions, or strong using a retinal scanner?
![Page 28: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/28.jpg)
© 2015 ClearSpecs Enterprises 28
Assumption 6
Understanding quality goals entails understanding their feasibility (i.e. achievability). For example, is 99.999% reliability possible is your situation?
![Page 29: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/29.jpg)
© 2015 ClearSpecs Enterprises 29
Assumption 7
Some pairs of qualities conflict e.g. understandability and performance, while others support e.g. security supports privacy.
![Page 30: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/30.jpg)
© 2015 ClearSpecs Enterprises 30
Assumption 8
Some qualities are essential for all applications for example (1) compliance with required rules, (2) domain sufficiency, (3) understandability, and (4) verifiability.
![Page 31: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/31.jpg)
© 2015 ClearSpecs Enterprises 31
Assumption 9
Some qualities are universal in the sense that it is hard to imagine a system where they don’t matter. These include the essential qualities (assumption 8) as well as: ease of use and learning, modularity, reliability, and responsiveness.
![Page 32: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/32.jpg)
© 2015 ClearSpecs Enterprises 32
Assumption 10
Quality goals are achieved using tactics e.g. supports and self-checking results. There are at least four types of supports:
1. other qualities2. system functions e.g. logging & exception handling3. sets of rules e.g. coding standards & design patterns4. warning labels e.g. “are you sure that …”
![Page 33: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/33.jpg)
© 2015 ClearSpecs Enterprises 33
Assumption 11
Many quality goals are harder to understand, express, achieve, and verify than most functional goals.
Even when the quality goal is precise and you know the candidate supports, you must select the specific supports needed to achieve the goal based on the system-specific execution environment.
![Page 34: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/34.jpg)
© 2015 ClearSpecs Enterprises 34
Assumption 12
Some quality goals (e.g. availability, performance, privacy, reliability, safety, security, and usability) are much harder to understand than other quality goals. This is because each is a software engineering subfield with an extensive body of knowledge.
![Page 35: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/35.jpg)
© 2015 ClearSpecs Enterprises 35
Assumption 13
Verifying quality goals is much more difficult than verifying functional goals. Quality goals, levels, and tactics must be verified with composite strategies that include technical review, analysis, measurement, and test. Testing alone is inadequate.
![Page 36: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/36.jpg)
© 2015 ClearSpecs Enterprises 36
Assumption 15
Quality goals should be monitored with built-in tests, exception logging, and service call tracking.
![Page 37: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/37.jpg)
© 2015 ClearSpecs Enterprises 37
Assumption 17 (summary)
Quality goals and functional goals have little in common[See comparison in handout].
![Page 38: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/38.jpg)
Resolving the Quality Crisis
© 2015 ClearSpecs Enterprises 38
The challenge is to raise developer awareness of quality goals to the same level as their awareness of functional goals. This implies early understanding of:
• high-priority quality goals and their attributes
• conflicts between qualities and how they should be resolved
• critical supports for each quality level
• effects of the critical supports on each domain function
• how qualities will be verified
![Page 39: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/39.jpg)
Quality-Aware Development
© 2015 ClearSpecs Enterprises 39
Refers to any development process starting and continuing with activities aimed at helping stakeholders be quality-aware.
For example, Quality-Aware Agile is a hybrid process that begins with the identification of relevant quality goals along with their levels, priorities, and supports.
It should take less than a week to draft a rich, but rough, project quality model.
![Page 40: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/40.jpg)
Quality-Aware Agile
© 2015 ClearSpecs Enterprises 40
The Agile process that follows considers the selected quality goals and supports when designing and coding the architecture and individual modules.
Each iteration includes the identification and verified achievement of incremental quality goals that depend on the functions chosen for the increment and the project’s quality model.
Incremental learning about final quality goals should be recorded in the quality model.
![Page 41: Understanding Quality Goals](https://reader033.vdocument.in/reader033/viewer/2022051303/58a2f8941a28ab722c8ba769/html5/thumbnails/41.jpg)
Wrap Up
Thanks for your participation.
Please help resolve the quality crisis by raising quality awareness in your organization.
Resources available at quality-aware.com
© 2015 ClearSpecs Enterprises 41