cs4723 software validation and quality assurance lecture 01 introduction
TRANSCRIPT
![Page 1: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/1.jpg)
CS4723Software
Validation and Quality Assurance
Lecture 01Introduction
![Page 2: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/2.jpg)
2
Course Instructor
Name:Dr. Xiaoyin Wang (Sean)
Office: NPB 3.208
Email: [email protected]
Experiences Got my PhD from Peking University, China Did my postdoc in UC Berkeley Worked for Microsoft (.net project), and Ensighta (a startup comp
any at Berkeley with 7-8 people)
![Page 3: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/3.jpg)
3
Introduce yourselves!
![Page 4: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/4.jpg)
4
Course Meetings, Web Pages, etc.
Course Meetings: MWF 11:00pm – 11:45pm
MH 3.04.20
Office Hours:
MW 1:00pm - 2:30pm
Course Web Page: http://xywang.100871.net/CS4723.htm
![Page 5: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/5.jpg)
5
Course Textbooks
Choose from the following reference books Software Testing and Quality Assurance: Theory
and Practice, by Kshirasagar Naik and Priyadarshi Tripathy
Software Testing: Principles and Practice, by Srinivasan Desikan and Gopalaswamy Ramesh
Introduction to Software Testing, by Paul Ammann and Jeff Offutt
![Page 6: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/6.jpg)
6
Course Topics
Software Testing Software Debugging Software Verification Anti-pattern Checking and Refactoring Bug Management Research Progress:
Automatic Oracles Automatic Program Repair
![Page 7: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/7.jpg)
7
Grading Scheme
Final Exam: 40% Assignments: 30% Projects: 20% Course participation and presentations: 10%
![Page 8: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/8.jpg)
8
Final Exam
Time: May 7th: 9:45am to 11:45am
Location: MS 3.04.20
Content Selected important topics in the course
We will have a review during the two classes in the last week
![Page 9: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/9.jpg)
9
Assignments
General Post and due date are in the schedule
Both paper and electronic submissions are acceptable is possible, electronic is preferred
Detailed evaluation plan will be introduced at the post time
Assignments Write unit test cases
To be posted
![Page 10: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/10.jpg)
10
Project: Android UI Testing
2-3 people form a small team
Each team is going to do the UI script testing for an Android App
Steps Explore the Android App
Write Testing Reports
Write Scripts for Regression Testing
Deliverable A testing report and the testing scripts
![Page 11: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/11.jpg)
11
Now, let’s go to the real lecture …
![Page 12: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/12.jpg)
12
Why software quality matters?
Prevalent usage Billions of PCs and mobile phones
Chatting, Surfing, Gaming, …
About 1 million lines of code on a new car, MIT Tech Review 2013
Some software on almost all major or small appliances
ATMs
Counters at shopping centers, restaurants, …
Garage Entrances
…
![Page 13: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/13.jpg)
13
Why software quality matters?
We are so dependent on software for critical tasks Traffic Control
Trains, air traffic, …
Transportation Control Ship / plane / car / elevator /…
Financial transactions Bank, stock, online shopping
Power supplies Electricity, Power stations, gas, …
Medical equipments
![Page 14: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/14.jpg)
14
What can caused by software bugs
From inconvenience to disasters Ugly presentation
Not showing some results or output
Unable to do operation
Lost of a hour’s / day’s / week’s work…
Privacy leaks, lost accounts
Money loss
Injuries, Life-threatening dangers
Disasters
![Page 15: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/15.jpg)
15
Some well-known disasters caused by software bugs
Mars Climate Orbiter ($165M, 1998) Sent to Mars to relay signal from the Mars Lander
Smashed to the planet: failing to convert
between English measure to metric values
Shooting down of Airbus A300 (290 death, 1988) US CG-49 shoot down a Airbus A300
Misleading output of the tracking software
THERAC-25 Radiation Therapy (1985) 2 cancer patients at East Texas Cancer
center received fatal overdoses
Miss-handling of race condition of the
software in the equipment
![Page 16: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/16.jpg)
16
Facts about software bugs
On average, 1-5 bugs per KLOC (thousand lines of code) In mature software
More than 10 bugs in prototypes
Windows2000 35MLOC
63K known bugs at the time of release
2 bugs per KLOC
$59.5B loss due to bugs in US 2002 (estimation by NIST)
It is not feasible to remove all bugs But try to reduce critical bugs
![Page 17: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/17.jpg)
17
Why so many bugs in software?
Errors can happen in any engineering discipline
Software is one of the most error-prone products of all engineering areas Requirements are often vague
Software can be really complex, undecidable problems are everywhere
Result Most product are put into the market with no or very
few errors Almost all software in the market has some number of
bugs (we will see that later)
![Page 18: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/18.jpg)
UTSA CS510318
Aspects of software quality
Dependability availability, reliability, safety
Efficiency processing time, memory utilization, responsiveness
Usability appropriate user interface and adequate
documentation
Maintainability ease of change
![Page 19: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/19.jpg)
UTSA CS510319
Dependability (I)
Availability For how large proportion of time the software
system is working
Recover from a crash?
Reliability For how large proportion of inputs the software is
giving correct outputs
Fault tolerance?
![Page 20: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/20.jpg)
UTSA CS510320
Dependability (II)
Safety For critical software, e.g. insulin control
To what extent the software is threatening people?
Runtime fault detection?
Security To what extent the software is able to protect itself
from malicious inputs
Input sanitization?
![Page 21: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/21.jpg)
UTSA CS510321
Efficiency
Processing time for time-consuming tasks Algorithm Complexity
Optimization
Maximal Memory utilization Avoid memory leaks
Release memory at proper time
Responsiveness Interleave UI threads and back-end threads
![Page 22: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/22.jpg)
UTSA CS510322
Usability
User Interface Looks attractive
Easy to understand and convenient to use
Help Documentation Easy to search and read
Cover all the features of the software
![Page 23: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/23.jpg)
UTSA CS510323
Maintainability
Ease to read Code quality
Comments
Ease to extend Design quality
Suitable design patterns
Ease to transplant Avoid hard coded elements
![Page 24: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/24.jpg)
UTSA CS510324
Absolute Software Quality
Impossible Vague Requirements
Free of bugs can never be verified even for some simple programs
Practice on critical software The NASA example
![Page 25: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/25.jpg)
UTSA CS510325
Each execution handles $4Billion equipments, human lives, dreams No prototypes
420k lines of code, 17 errors in 11 versions (is it a large software?)
Commercial equivalent would have at least 1000 bugs
NASA Example
![Page 26: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/26.jpg)
UTSA CS510326
A third of the effort before coding starts
Long specifications, written down, fully discussed 40k pages of specification (longer than the code)
Change to add GPS support (2500 pages more specification)
Specification is almost pseudo code
NASA Example
![Page 27: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/27.jpg)
UTSA CS510327
When fixing bugs, fix what allowed the mistake Unclear API
Insufficient tests
Improper use of tools
Validation and review at all levels 85% of bugs revealed before testing
NASA Example
![Page 28: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/28.jpg)
UTSA CS510328
Cost 260 people
$32 million
A year
TOO EXPENSIVE!!!
Overkill for normal software
NASA Example
![Page 29: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/29.jpg)
29
Quality Economics
Tradeoff of cost and requirements
![Page 30: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/30.jpg)
30
Approaches to Enhance Software Quality
Bug Prevention Prevent bugs during requirement collection,
designing, coding
Bug Removal Remove bugs after coding, but before
release
Bug Tolerance Runtime mechanisms to avoid known or
unknown bugs in the software
![Page 31: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/31.jpg)
31
Approach to prevent bugs
Requirement Engineering Remove ambiguity in requirements
Formal Specification
Design Reduce dependency between modules
Concentrate cross-cut concerns
Coding Proper coding styles
Well written comments
Avoid misleading interfaces
![Page 32: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/32.jpg)
32
Approach to remove bugs
Testing Feed input to software and run it to see whether its
behavior is as expected
Limitations Impossible to cover all cases
Test oracles (what is expected)
Static checking Identify specific problems (e.g., memory leak) in the
software by scanning the code or all possible paths
Limitations Limited problem types
False positives
![Page 33: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/33.jpg)
33
Approach to remove bugs
Formal Proof Formally prove that the program implements the
specification
Limitations Difficult to have a formal specification The proof cost a lot of human efforts
Inspection Manually review the code to detect faults
Limitations: Hard to evaluate Sometime hard to get progress
![Page 34: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/34.jpg)
34
Approach to tolerate bugs
Exception handling Specify what the program should do when
unexpected results are generated
Spend time on writing exception handlers
Consider exceptions in the exception handler
Redundant code Have redundant modules to avoid runtime bugs by
checking consistency of results
Voting for a correct results when bug happens
![Page 35: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/35.jpg)
35
In our course
Bug prevention: something you should learn in software
engineering course, briefly introduce it at the end of our course
Bug removal: The focus of this course
Bug tolerance We will introduce guidance for exception
handling at the end of our course, redundant code will not be introduced because it is rarely used
![Page 36: CS4723 Software Validation and Quality Assurance Lecture 01 Introduction](https://reader034.vdocument.in/reader034/viewer/2022050909/56649e1a5503460f94b073b0/html5/thumbnails/36.jpg)
36
A short review
Software bugs affect every aspect of our life
Software Bugs may cause severe consequences
Software Bugs are prevalent
Aspects of software quality Dependability, Efficiency, Usability, Maintainability
(Bad smells -> bugs)
Approaches to enhance software quality Bug prevention, Bug removal, Bug avoiding