automated discovery of faults and fault domains using random testing
DESCRIPTION
Automated Discovery of Faults and Fault Domains using Random Testing. Mian Asbat Ahmad 24-01-2013. Why random testing?. Exhausting testing not possible. Infinite values from 0 and 1. Compatibility across multiple systems. Limited Time and Resources Simple algorithm for data selection - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/1.jpg)
Automated Discovery of Faults and Fault Domains using Random Testing
Mian Asbat Ahmad
24-01-2013
![Page 2: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/2.jpg)
Why random testing?
• Exhausting testing not possible.• Infinite values from 0 and 1.• Compatibility across multiple systems.• Limited Time and Resources
• Simple algorithm for data selection• Easy implementation• Free from human bias• Quick and effective in finding faults• Code Privacy
• Source code not required for Testing
![Page 3: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/3.jpg)
Random Testing
• Black-box testing technique• Dynamic testing process• Input Domain• Random Selection
• Test Input• Test modules
• Test Execution• Test Evaluation
• Specifications• Language exceptions
• Test Output
![Page 4: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/4.jpg)
Automated Random Testing
• Automating the process of random testing that include reading specifications, test data generation, test execution, result analysis and identification of the faulty tests.
• Examples of Automated Random testing tools are YETI, JCrasher, JUnit, Haskel etc…
![Page 5: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/5.jpg)
• Fault lies in Domains [Chan et al]• Input Domain• Fault Domain
• Point Fault Domain• Block Fault Domain• Strip Fault Domain
Fault Domains
![Page 6: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/6.jpg)
Enhancement in Random Testing
• Test Strategy?• Need of test strategy variations?
• High No of Faults• Less time• Low resources
• Famous Variations in Random Strategy• ART• MART• RRT• FD-ART• R+• QRT• …
![Page 7: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/7.jpg)
Dirt Spot Sweeping Random Strategy
• Combination of strategies• Random
• Input and module selection at random• Random Plus
• Random testing with border values• Spot Sweeping
• Move around it when fault found
![Page 8: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/8.jpg)
Working of DSSR
• Start with R and R+• Switch to DSSR,
when Fault found• After Evaluating
block/strip, switch back to R and R+
![Page 9: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/9.jpg)
DSSR in Action
![Page 10: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/10.jpg)
Evaluation of DSSR
• 60 classes selected from 32 different projects• In 43 classes all the strategies found same number of faults• In 17 classes strategies performed different than one another• DSSR found the highest number of faults• No strategy performed better than DSSR
![Page 11: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/11.jpg)
Test Results of 17 classes
• DSSR performed better than R and R+• DSSR up to 33% better than R• DSSR up to 17% better than R+
![Page 12: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/12.jpg)
DSSR Limitations
• Fault Domain• Existence Frequency
• Point, Block and Strip in programs• Domain Discovery
• Find Fault Domain of Program• Automated Method
• Domain Presentation
• Time to Find First Fault• DSSR uses R and R+ to start with• Use of Daikon to make it more targeted• Automated Method
![Page 13: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/13.jpg)
Solutions to Limitations
• Automated Discovery of Failure Domain• Automated Method• Find Fault • Find its Domain• Plot Domain on Graph
• DSSR with Daikon/contracts• Execute class under test by Daikon• Get class invariants• Add data to R+ from invariants• Run DSSR • Reliance on simple R and R+ gone.
![Page 14: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/14.jpg)
What is ADFD
• Automated Technique• Start Test• Find Fault• Generate a program Dynamically• Compile it• Execute it• Generate data• Generate Graphs
![Page 15: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/15.jpg)
Working of ADFD using Example
• Program with multiple gaps• Or we can write program for• Point block or strip pattern• here
![Page 16: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/16.jpg)
Generated Program for finding domain
![Page 17: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/17.jpg)
Executed Program Output
![Page 18: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/18.jpg)
Front End of ADFD
![Page 19: Automated Discovery of Faults and Fault Domains using Random Testing](https://reader036.vdocument.in/reader036/viewer/2022062315/56814e93550346895dbc3c77/html5/thumbnails/19.jpg)
Daikon etc invariants border values…
• Automated Method• Run program by Daikon• Get invariants• Run DSSR with –ea