software tool to assist scenario testing of...
TRANSCRIPT
![Page 1: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/1.jpg)
Software Tool to Assist Scenario Testing of Website
Student: Siqi Fan
Supervisor: Assoc. Prof. Christopher Johnson
Presented on November 4th, 2010
![Page 2: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/2.jpg)
Agenda
• Problem • Analysis • Idea • Result • Conclusion
![Page 3: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/3.jpg)
Large Scale Business Oriented Website
Website More
Contents
More
Features
More
Languages
www.office.com
www.apple.com
www.honda.com
www.dell.com
…
![Page 4: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/4.jpg)
During the production, there are tremendous amount of defects found…
Tester
Manager
Developer
? Which defect should I fix first?
Severity
Priority Defect
Defect Defect Report
… Release Schedule
![Page 5: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/5.jpg)
When the scheduled releasing date is approaching, there are still many defects not fixed…
Manager
? Could the defect be published to the live website?
Severity
Priority Defect
Defect Defect Report
Release Schedule
Business Value
… Developing and Testing Resources
Must-Fix
Bypass
![Page 6: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/6.jpg)
Problem: Ineffective Prioritization
The critical defects do no receive the higher priority because something wrong with the reference factor Severity: • No difference between bugs with same
severity level • Lack of big picture
More distinguishable
More objectively
![Page 7: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/7.jpg)
Severity is an important factor for prioritization, but what is it about?
IEEE standard 1044-2009: “Severity is the highest failure impact that the defect could (or did) cause, as determined by (from the perspective of) the organization responsible for software engineering”
![Page 8: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/8.jpg)
Severity is an important factor for prioritization, but what is it about to a large scale business oriented website?
IEEE standard 1044-1993: “The impact of an anomaly shall be considered at each step of the anomaly process.”
![Page 9: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/9.jpg)
Using a scenario is probably the best way to describe the process.
functional and non-functional requirements
systematic overview of the website
Scenarios
![Page 10: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/10.jpg)
Case Study: Consider impact on scenario
![Page 11: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/11.jpg)
![Page 12: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/12.jpg)
![Page 13: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/13.jpg)
Considering impact on scenario is not enough for measuring severity.
Website Quality
![Page 14: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/14.jpg)
Two aspects of website quality
Website Quality
Customer Website Provider GAP
How to narrow?
![Page 15: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/15.jpg)
The order relationship between quality factors
Finance Page 1. Currency / Timeliness /
Update 2. Completeness /
Comprehensiveness of Information
3. Navigation 4. Accuracy 5. Readability /
Comprehension / Clarity
Entertainment Page 1. Visual design 2. Engaging 3. Navigation 4. Information representation 5. Site accessibility /
Responsiveness
![Page 16: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/16.jpg)
Case Study: Consider importance of quality factors
![Page 17: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/17.jpg)
![Page 18: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/18.jpg)
![Page 19: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/19.jpg)
Shall we also consider the usage?
YES!
Likelihood = Page Trip Rate × Hit Rate Statistical Analysis on the historical data
![Page 20: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/20.jpg)
How to represent Severity in a more distinguishable form? and How to evaluate Severity objectively in a systematic manner?
![Page 21: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/21.jpg)
Framework
![Page 22: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/22.jpg)
Framework
![Page 23: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/23.jpg)
Framework
![Page 24: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/24.jpg)
Framework
![Page 25: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/25.jpg)
Framework
![Page 26: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/26.jpg)
Classification Scheme and Representation
Impact 1~5 Importance 1~5 Likelihood 1~5
𝐒𝐞𝐯𝐞𝐫𝐢𝐭𝐲 = Impact + Importance + Likelihood
ImpactMax + ImportanceMax + LikelihoodMax
![Page 27: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/27.jpg)
Prototype
![Page 28: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/28.jpg)
Prototype
![Page 29: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/29.jpg)
![Page 30: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/30.jpg)
Conclusion
The prototype is built to prove the feasibility to adopt the framework into the practical scenario testing tools
It enables the severity of defects to be set in a systematic manner and be represented in a more distinguishable form so that severity can provides the greatest reference value for effective defect prioritization.
It lights up a direction for website design and website testing research (for software engineering?) for quality delivery and quality assurance.
![Page 31: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/31.jpg)
Future Work
• Extend the content of Quality Dimension , Quality Feature Clusters, Deliverables, and Testing Tasks
• Prove in the practical projects • Refine the severity classification scheme and
representations
𝐒𝐞𝐯𝐞𝐫𝐢𝐭𝐲 = ω1 × Impact + ω2 × Importance + ω3 × Likelihood
ImpactMax + ImportanceMax + LikelihoodMax
• Refine the severity score aggregation method
![Page 32: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/32.jpg)
Acknowledgement First and foremost I would like to offer my sincerest gratitude to my supervisor, Assoc Professor Chris Johnson, who has supported me throughout of my thesis with his patience and knowledge. In the various research works and laboratories along the entire project, I have been also aided in setting the research direction by him. Without Chris Johnson’s kind support and encouragement this thesis would not have been possible. Secondly, I would like to express my thankfulness to all my previous colleagues in the Office International Publishing Group (OIPG), Microsoft Ireland. It was them who created the heuristic work environment and offered me the opportunity to receive the fundamental knowledge and experience of a large scale business oriented website production. Finally, I would like to thank my beloved parents, wife and friends for their understanding and endless love throughout all my studies.
![Page 33: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/33.jpg)
![Page 34: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/34.jpg)
Appendix
![Page 35: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/35.jpg)
Appendix
![Page 36: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/36.jpg)
Appendix
![Page 37: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/37.jpg)
Appendix
![Page 38: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/38.jpg)
Appendix
![Page 39: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/39.jpg)
Appendix
![Page 40: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/40.jpg)
Appendix
![Page 41: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/41.jpg)
Appendix Impact is to measure how the defect affects the scenario. The following list is suggested for ranking impact: • Level 0: Not set • Level 1: The defect impairs at least the third level scenario and can be skipped
without worries in the entire scenario. There is at least one easy work around. The user can go further with no difference in the scenario. The issue is low and cosmetic.
• Level 2: The defect impairs at least the third level scenario and can be skipped with notice by the user. There is at least one work around. The user can go further but it will be better to have the defect fixed. The issue is enhancement.
• Level 3: The defect impairs the dependent (secondary) scenario and can be skipped with worries in the entire scenario. There is at least one work around. The user can go further with mind in the scenario. The issue is minor.
• Level 4: The defect impairs the key (primary) scenario and can be skipped with a lot of troubles in the entire scenario. There is at least one unexpected work around. The user can hardly go further in the scenario. The issue is major.
• Level 5: The defect blocks the entire scenario. There is no work around available. The user cannot go further in the scenario. The issue is fatal.
![Page 42: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/42.jpg)
Appendix Importance is to measure how important the quality feature cluster or quality dimension in the scenario. The following list is suggested for ranking importance: • Level 0: Not set • Level 1: Negligible. The quality factor or the quality feature cluster or the
quality dimension will not affect the overall website quality. • Level 2: Acceptable. The quality factor or the quality feature cluster or the
quality dimension will slightly affect the overall website quality. • Level 3: Remarkable. The quality factor or the quality feature cluster or the
quality dimension will moderately affect the overall website quality. • Level 4: Significant. The quality factor or the quality feature cluster or the
quality dimension will strongly affect the overall website quality. • Level 5: Critical. The quality factor or the quality feature cluster or the quality
dimension will dominantly affect the overall website quality.
![Page 43: Software Tool to Assist Scenario Testing of Websitecourses.cecs.anu.edu.au/courses/CS_PROJECTS/10S2/Final present… · Software Tool to Assist Scenario Testing of Website Student:](https://reader030.vdocument.in/reader030/viewer/2022040307/5ed5862d29851c54136538b2/html5/thumbnails/43.jpg)
Appendix
Likelihood is to measure the product value of page trip rate and hit rate. The following list is suggested for ranking likelihood: • Level 0: Not set • Level 1: 0% - 20% • Level 2: 20% - 40% • Level 3: 40% - 60% • Level 4: 60% - 80% • Level 5: 80% - 100%