The Role of the Quality Analyst The Role of the Quality Analyst in Software Developmentin Software Development
Presented by Presented by Jim Coleman – HJIP QA AnalystJim Coleman – HJIP QA Analyst
A little bit about meA little bit about me
Over 9 years of experienceOver 9 years of experience
Certified in Software TestingCertified in Software Testing
Degrees in IS and ITDegrees in IS and IT
Stereotypical Software TesterStereotypical Software Tester
Isn’t quality everyone’s job?Isn’t quality everyone’s job?
Quality must be an Quality must be an independentindependent processprocess
Quality means more than finding errorsQuality means more than finding errors
Quality requires measurementQuality requires measurement
Quality must be consistent & repeatableQuality must be consistent & repeatable
Quality may be viewed differently by Quality may be viewed differently by business and developmentbusiness and development
Quality as a matter of Quality as a matter of perspectiveperspective
How do you define Quality?How do you define Quality?
TranscendentTranscendent - I know it when I see it - I know it when I see it Product-BasedProduct-Based - Possesses desired features - Possesses desired features User-BasedUser-Based - Fitness for use - Fitness for use Development and Manufacturing BasedDevelopment and Manufacturing Based - Conforms to - Conforms to
requirementsrequirements Value-BasedValue-Based - At an acceptable cost - At an acceptable cost
Quality Control Quality Control Vs Vs
Quality AssuranceQuality Assurance
Cost of QualityCost of Quality
Prevention costsPrevention costs
Appraisal CostsAppraisal Costs
Failure CostsFailure Costs
External failure External failure costscosts
VerificationVerificationVVs s
Validation Validation
Verification answers the question, Verification answers the question, “Did we build the right system?”“Did we build the right system?”
Validations answers the question, Validations answers the question, “Did we build the system right?”“Did we build the system right?”
Focuses on testing the function of the Focuses on testing the function of the program or application against its program or application against its specification. Determining what and how specification. Determining what and how much to test is criticalmuch to test is critical
Equivalence PartitioningEquivalence PartitioningA technique for testing equivalence classes rather than undertaking A technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class. For example, a exhaustive testing of each value of the larger class. For example, a program which edits credit limits within a given range ($10,000 - program which edits credit limits within a given range ($10,000 - $15,000) would have three equivalence classes:$15,000) would have three equivalence classes:
< $10,000 (invalid)< $10,000 (invalid) Between $10,000 and $15,000 (valid)Between $10,000 and $15,000 (valid)
> $15,000 (invalid> $15,000 (invalid))
Boundary AnalysisBoundary AnalysisA technique that consists of developing test cases and data that focus on A technique that consists of developing test cases and data that focus on the input and output boundaries of a given function. In same credit limit the input and output boundaries of a given function. In same credit limit example, boundary analysis would test:example, boundary analysis would test:
Low boundary +/- one ($9,999 and $10,001)Low boundary +/- one ($9,999 and $10,001) On the boundary ($10,000 and $15,000)On the boundary ($10,000 and $15,000) Upper boundary +/- one ($14,999 and $15,001)Upper boundary +/- one ($14,999 and $15,001)
Error GuessingError GuessingTest cases can be developed based upon the intuition and experience of Test cases can be developed based upon the intuition and experience of the tester. For example, in an example where one of the inputs is the the tester. For example, in an example where one of the inputs is the date, a tester may try February 29, 2000.date, a tester may try February 29, 2000.
Black Box Black Box TestingTesting
White Box White Box TestingTesting
Assumes that the path of logic in a unit or Assumes that the path of logic in a unit or program is known. White-box testing program is known. White-box testing consists of testing paths, branch by branch, consists of testing paths, branch by branch, to produce predictable results. to produce predictable results.
Statement CoverageStatement CoverageExecute all statements at least once.Execute all statements at least once.
Decision CoverageDecision CoverageExecute each decision direction at least once.Execute each decision direction at least once.
Condition CoverageCondition CoverageExecute each decision with all possible outcomes at Execute each decision with all possible outcomes at least once.least once.
Decision/Condition CoverageDecision/Condition CoverageExecute all possible combinations of condition Execute all possible combinations of condition outcomes in each decision.outcomes in each decision.
Multiple Condition CoverageMultiple Condition CoverageInvoke each point of entry at least once.Invoke each point of entry at least once.
Structural TestingStructural Testing
Technique Description Example
Stress Determine system performs withexpected volumes
• Sufficient disk space allocated• Communication lines adequate
Execution System achieves desired level ofproficiency
• Transaction turnaround time adequate
• Software/hardware use optimized
Recovery System can be returned to an operational status after a failure
• Induce failure• Evaluate adequacy of backup data
Operations System can be executed in a normal operational status
• Determine systems can run using document
• JCL adequate
Compliance (to process)
System is developed in accordance with standards and procedures
• Standards followed• Documentation complete
Security System is protected in accordance with importance to organization
• Access denied• Procedures in place
Functional TestingFunctional Testing
Technique Description Example
Requirements System performs as specified • Prove system requirements• Compliance to policies, regulations
Regression Verifies that anything unchanged still performs correctly
• Unchanged system segments function• Unchanged manual procedures correct
Error Handling
Errors can be prevented or detected, and then corrected
• Error introduced into test• Errors re-entered
Manual Support
The people-computer interaction works
• Manual procedures developed• People trained
Integration Data is correctly passed from system to system
• Intersystem parameters changed• Intersystem documentation updated
Control Controls reduce system risk to an acceptable level
• File reconciliation procedures work• Manual controls in place
Parallel Old system and new system are run and the results compared to detect unplanned differences
• Old and new systems can reconcile• Operational status of old system maintained
Quality MetricsQuality Metrics
Technical MetricsTechnical MetricsMade throughout the Made throughout the development processdevelopment process
““Is the product ready for Is the product ready for production?”production?”
After-the-fact MetricsAfter-the-fact MetricsMade after product is in Made after product is in developmentdevelopment
““What should we What should we differently next time?”differently next time?”
References and ResourcesReferences and Resources
Quality Assurance Institute (QAI)Quality Assurance Institute (QAI)qaiworldwide.orgqaiworldwide.org
American Society for Quality (ASQ)American Society for Quality (ASQ)asq.orgasq.org
Sticky MindsSticky MindsStickyminds.comStickyminds.com
Benchmark QA – Sheila ConwayBenchmark QA – Sheila Conwaybenchmarkqa.combenchmarkqa.com
Twin Cities Quality Assurance Assc.Twin Cities Quality Assurance Assc.tcqaa.orgtcqaa.org