slide 1 testing workflow purpose l purpose verify the result from implementation by testing each...
TRANSCRIPT
![Page 1: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/1.jpg)
Slide 1
Testing Workflow Purpose
Purpose• Verify the result from implementation by testing
each build• Plan the tests in each iteration• Integration tests for every build within theiteration• System tests for the end of the iteration
• · Design and implement tests by creating• test cases that specify what to test• test procedures that specify how to perform the tests• executable test components to automate the tests
![Page 2: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/2.jpg)
Slide 2
Testing Workflow Flow
![Page 3: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/3.jpg)
Slide 3
Testing Workflow with Phases
![Page 4: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/4.jpg)
Slide 4
Testing Workflow in Phases
Generally, when there is an implementationresult, there is a test --- so Test at each build
Inception phase: initial test planning, prototype testingElaboration phase: test architectural baselineConstruction phase: significant testing at each buildTransition phase: re-test fixes and regression tests
![Page 5: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/5.jpg)
Slide 5
Testing Workflow Other Tasks
Regression tests: in a new build, re-apply tests from old builds to make sure nothing “broke” in new build
Evolve test modelcreate new test cases for next buildrefine prior test cases into regression testsremove obsolete tests and corresponding test procedures and components
![Page 6: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/6.jpg)
Slide 6
Testing Workflow Models
Test model artifactsTest casesTest proceduresTest componentsTest subsystem packages for complex test
Other artifactsTest PlanDefectsTest Evaluation
![Page 7: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/7.jpg)
Slide 7
Testing Workflow Model
TEST MODEL
Describes how executable components (e.g.,builds) in the implementation model are tested byintegration tests and system tests
Describes how specific system aspects are to betested
e.g., usability testing of user interface oruser manual
![Page 8: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/8.jpg)
Slide 8
Testing Workflow – Test Case
TEST CASE
Specifies one way to test the systemWhat to test (requirements)With which inputsExpected resultsTest conditions and constraints
![Page 9: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/9.jpg)
Slide 9
Testing Workflow Test Cases
TEST CASE EXAMPLES Test a use case or a scenario in a use case
(black box)§ actor interaction, pre-conditions,action sequences, post-conditions
Test a use-case realization (white box)
verify interaction between components. Installation tests, configuration tests, negative(abuse) tests, stress tests
Organize similar tests in input/result matrix Family of test for different paths through a use case
![Page 10: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/10.jpg)
Slide 10
Testing Workflow Special TestsInstallation tests verify the system can be installed on the customer platform and that it executes correctly when installed
Configuration tests verify the system works correctly in different configurations
Negative tests (abuse tests) what causes failureTest outside the intended design, e.g., wrong
configuration, wrong input, etc.Reveals weaknesses in design
Stress tests show problems with insufficientor shared resources
![Page 11: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/11.jpg)
Slide 11
Testing Workflow ProceduresSpecify how to perform one or several test cases orparts of them Instructions for tester in manual test case Instructions for interacting with test automation tool to create executable test components, then instructions to integrate and execute those test components
One test procedure may cover several test cases
One test case may require several test procedures Test procedure instructions often reflect flow-of events descriptions, including input values, how to enter input values, and how to verify results
![Page 12: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/12.jpg)
Slide 12
Testing Workflow ComponentsAutomate one or several test procedures or parts
Provide test inputs, control and monitor testexecution, and possibly analyze and report results
Developed with scripting languages, programminglanguages, or test automation tools
test drivers, test harnesses, and test scripts
Complex test cases and procedures may require formal analysis, design, and implementation modeling of the test components
![Page 13: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/13.jpg)
Slide 13
Testing Workflow PlanDescribes testing strategies, resources and schedule· Strategy
What kind of tests to perform for each iterationTest objectivesRequired level of test and code coverageTest acceptance criteriaTrade-off cost vs. improved quality and reduced risk
Test Plan may be defined in conjunction with the model artifacts from other workflows
analysis (black box)design and implementation (white box)
“Tests” may include artifact reviews
![Page 14: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/14.jpg)
Slide 14
Testing Workflow ResultsResults are the output data of the test and/or the behavior of the system during test
Analysis of results: pass or defectDefect: a system anomaly (software fault or adefect found in artifact review) capture symptom insufficient detail to allow problem resolutioncapture defect in problem and resolution tracking
system
Test evaluation: summary of overall test resultstest-case coverage, code coveragedefects and defect status
![Page 15: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/15.jpg)
Slide 15
Testing Workflow Results
![Page 16: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/16.jpg)
Slide 16
Testing Workflow Workers
Test designer: responsible for integrity of the test modelplan tests: test goals and scheduleselect and define test cases, test proceduresevaluate test results
Component engineer: responsible for development of test components
that automate some of thetest proceduresconsider independent test
![Page 17: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/17.jpg)
Slide 17
Testing Workflow Workers
Integration tester: perform integration tests and report defectsmay be same person as system integrator inimplementation workflow
System tester: perform system tests and report defectsfocus on externally observable behaviorconsider users, beta customers, etc. as testers
![Page 18: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/18.jpg)
Slide 18
Testing Workflow – THE PLAN
Plan the Tests Strategy and ResourcesDefine Integration TestsDefine System TestsDesign Regression TestsDefine and Mutate Test CasesImplement TestsEvaluate Tests
![Page 19: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/19.jpg)
Slide 19
Testing Workflow – THE PLANPlan the Tests Strategy and Resources
Describe testing strategy Inputs: use case model, supplemental requirements,
design modelDevelop test cases and procedures that
have a minimum of overlaptest the most important test casestest high risk requirement items
Estimate resources Estimate human and system resources
Schedule the testing effort
![Page 20: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/20.jpg)
Slide 20
Testing Workflow – THE PLAN
Define Integration Tests
Derive test cases from use-case realizations – design (component interaction)
Develop tests with minimal overlap that test aninteresting path or scenario through use case
Primarily consider interaction diagramsCombinations of actor input, output and system
start state that lead to interesting scenarios that employ the classes that participate in the diagrams
![Page 21: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/21.jpg)
Slide 21
Testing Workflow – THE PLAN
Define System TestsTest system as a whole Test combinations of use cases instantiated under different conditions
Hardware configurationsSystem loadsNumber of actorsDatabase size, etc.
Prioritize combinations required or are likely to function in parallel likely to influence each other if in parallelinvolve multiple processes frequently use system resources or use them in complex and unpredictable ways
![Page 22: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/22.jpg)
Slide 22
Testing Workflow – THE PLAN
Design Regression Tests
Modify test cases from earlier builds to be usedas regression tests in subsequent builds
Make more flexible and resilient to changes to the system under test
Specifically address previously found defectsAutomate where possible
![Page 23: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/23.jpg)
Slide 23
Testing Workflow – THE PLAN
Define and Mutate Test Cases
Work test case by test case and suggest testingprocedures for each one
Reuse existing test casesMay need to modify them to be reusable
Align test procedures to subsystems under testMinimize overlapIsolates change to test resulting from changeto subsystem
![Page 24: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/24.jpg)
Slide 24
Testing Workflow – THE PLAN
Implement Tests
Create test components from test procedures With test automation tools
Perform or specify actions Record actions as a test component
When programming explicitlyDesign and implement using test procedures
Provide input and output data generation, visualization, interpretation and management utilities
Use spreadsheets and database applications
![Page 25: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/25.jpg)
Slide 25
Testing Workflow – THE PLAN
Evaluate TestsPerform the integration tests relevant to a build
Manually perform test proceduresCompare test results with the expected test resultsInvestigate test results that deviate from the expectedFaulty system and/or faulty test?
Report defects to the component engineer responsible forthe components likely to contain the fault
Report defects to the test designers for evaluation of overall test results
![Page 26: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/26.jpg)
Slide 26
Testing Workflow – THE PLAN
Evaluate Tests (cont)Test designers prepare metrics of test acceptance withrespect to system quality level
Test completeness and coveragePercentage of test cases executedPercentage of code and paths tested
ReliabilityCategorize defect severityAnalyze trends of discovered defects
Compare with prior experience to predict amount of testing still required to reach an acceptable defect leve
![Page 27: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/27.jpg)
Slide 27
Testing Workflow – THE PLAN
Evaluate Tests (cont)
Suggest actionsSuggest further testsRelax test criteriaDeliver parts of the system that passed; rework and retest parts that failed
Document test completeness, reliability and actions
![Page 28: Slide 1 Testing Workflow Purpose l Purpose Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests](https://reader035.vdocument.in/reader035/viewer/2022070306/5518c290550346881f8b5685/html5/thumbnails/28.jpg)
Slide 28
Testing Workflow – SUMMARY
Test Model Test cases specify what to test Test procedures specify how to perform tests Test components automate test procedures
Test Plan
Test EvaluationDefects to be fed into other core work