![Page 1: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/1.jpg)
CS 217 Software Verification and
Validation
Week 9, Summer 2014
Instructor: Dong Sihttp://www.cs.odu.edu/~dsi
![Page 2: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/2.jpg)
REVIEW OF THE CLASS
![Page 3: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/3.jpg)
What is (software) testing?
![Page 4: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/4.jpg)
Softeware Testing – definition The process consisting of all life cycle
activities, concerned with planning, preparation and evaluation of software products and related work products to determine:
– that they satisfy specified requirements, – to demonstrate that they are fit for purpose and
– to detect defects
4
![Page 5: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/5.jpg)
Validation & Verification
Validation : Have we built the right software? i.e., do the requirements satisfy the customer? (This is dynamic process for checking and
testing the real product. Software validation always involves with executing the code)
Verification : Have we built the software right? i.e., does it implement the requirements? This is static method for verifying design, code.
Software verification is human based checking of documents and files
Introduction to Software Testing, Edition 2 (Ch 1) © Ammann & Offutt 5
![Page 6: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/6.jpg)
Introduction to Software Testing, Edition 2 (Ch 1) © Ammann & Offutt 6
Software Fault : A static defect in the software
Software Failure : External, incorrect behavior with respect to the requirements or other description of the expected behavior
Software Error : An incorrect internal state that is the manifestation/expression of some fault
Faults in software are equivalent to design mistakes in hardware.
Software does not degrade.
Software Faults, Errors & Failures
![Page 7: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/7.jpg)
Fault and Failure Example The doctor tries to diagnose the root cause, the
disease– Fault
A patient gives a doctor a list of symptoms– Failures
The doctor may look for anomalous internal conditions (high blood pressure, irregular heartbeat, bacteria in the blood stream)– Errors
Introduction to Software Testing, Edition 2 (Ch 1) © Ammann & Offutt 7
Most medical problems result from external attacks (bacteria, viruses) or physical
degradation as we age.They were there at the beginning and do not
“appear” when a part wears out.
![Page 8: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/8.jpg)
Sources of Problems Requirements Definition: Erroneous, incomplete,
inconsistent requirements.
Design: Fundamental design flaws in the software.
Implementation: Mistakes in chip fabrication, wiring, programming faults, malicious code.
Support Systems: Poor programming languages, faulty compilers and debuggers, misleading development tools.
![Page 9: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/9.jpg)
Sources of Problems (Cont’d)
Inadequate Testing of Software: Incomplete testing, poor verification, mistakes in debugging.
Evolution: Sloppy redevelopment or maintenance, introduction of new flaws in attempts to fix old flaws, incremental escalation to inordinate complexity.
![Page 10: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/10.jpg)
Summary: Why Do We Test Software ?
Introduction to Software Testing, Edition 2 (Ch 1) © Ammann & Offutt 10
A tester’s goal is to eliminate faults as early as possible
• Improve quality• Reduce cost• Preserve customer
satisfaction
![Page 11: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/11.jpg)
Testing main principles
![Page 12: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/12.jpg)
Testing Principles (1) Testing can demonstrate only the presence of
defects and not their absence– Testing can show that defects are present, but cannot
prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.
Exhaustive testing is impossible– Exhaustive testing (all combinations of inputs and
preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.
![Page 13: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/13.jpg)
Testing Principles (2)
Early testing is important– Testing activities should start as early as possible in
the software or system development life cycle and should be focused on defined objectives.
Defects are clustering– A small number of modules contain most of the
defects discovered during pre-release testing, or are responsible for the most operational failures.
![Page 14: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/14.jpg)
Testing Principles (3)
Testing is context dependent– Testing is done differently in different contexts. For
example, military software is tested differently from an business site.
![Page 15: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/15.jpg)
Software Testing Process
15
Unit test
Integrationtest
Systemtest
System engineering
Software Design
Code & Implementation
V&V Targets
![Page 16: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/16.jpg)
Software Development Lifecycles
Code and Fix Waterfall Cycle
![Page 17: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/17.jpg)
BASIC OF LOGICS
![Page 18: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/18.jpg)
Motivation
LOGIC enabled mathematicians to point out WHY a proof is wrong, or WHERE in the proof, the reasoning has been faulty.
Faults (bugs) have been detected in proofs (programs)
Is such a tool that by symbolizing arguments rather than writing them out in some natural language (which is fraught with ambiguity), checking the correctness of a proof becomes a much more viable task.
18
![Page 19: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/19.jpg)
Introduction to Logic
CS areas where we use LOGIC Architecture (logic gates) Software Engineering (Validation & Verification) Programming Languages (Semantics & Logic
Programming) AI (Automatic theorem proving) Algorithms (Complexity) Databases (SQL)
19
![Page 20: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/20.jpg)
Fundamental of Logic
Declarative statements
Examples of declarative statements– “A is older than B”– “There is ice in the glass”
– In CIS, describing the data (variables, functions, etc.)
20
![Page 21: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/21.jpg)
Propositions - a statement that is either true or false.
For every proposition p, either p is T or p is F For every proposition p, it is not the case that
p is both T and F
21
![Page 22: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/22.jpg)
Fundamental of Logic
We not only want to specify such statements, but also want to check whether a given program or system fulfills specifications that user needs. (Validation)
We are interested in precise declarative statements about computer systems and programs. (Verification)
22
![Page 23: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/23.jpg)
Propositional Logic: Basics Propositional logic describes ways to combine
some true statements to produce other true statements.
If it is proposed that `Jack is taller than John' and
`John can run faster than Jack' are both T =`Jack is taller than John and John can run faster than Jack'.
Propositional logic allows us to formalize such statements.
In concise form: A ^ B23
![Page 24: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/24.jpg)
Propositional Logic
Composition of atomic sentencesp: I won the lottery yesterdayq: I will purchase a lottery ticket todayr: I played a football game yesterday
~ p: Negation. “I did not win the lottery last week”
p v r: Disjunction. The statement is true if at least one of them is true. “I won the lottery or played a football game yesterday.”
24
![Page 25: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/25.jpg)
Propositional Logic
p ^ r: Conjunction. “Yesterday I won the lottery and played a football game.”
p q: Implication. “If I won the lottery last week, then I will purchase a lottery ticket today.” p is called the assumption and q is called conclusion.
– p implies q– If p then q
25
![Page 26: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/26.jpg)
Natural Deduction Proof
Set of rules which allow us to draw a conclusion by given a set of preconditions
Constructing a proof is much like a programming!
It is not obvious which rules to apply and in what order to obtain the desired conclusion, be careful to choose proof rules!
26
![Page 27: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/27.jpg)
Rules of Natural Deduction Fundamental rule 1 (rule of detachment)
pp q
. . . q
The rule is a valid inference because [p ^ (p q)] q is a tautology!
27
![Page 28: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/28.jpg)
Rules of Natural Deduction
Example: if it is 11:00 o’ clock in Norfolk
if it is 11:00 o’ clock in Norfolk, then it is 11:00 o’ clock in DC
then by rule of detachment, we must conclude:
it is 11:00 o’ clock in DC
28
![Page 29: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/29.jpg)
Rules of Natural Deduction
Fundamental rule 2 (transitive rule)p qq r
. . . p r This is a valid rule of inference because the
implication (p q) ^ (q r) (p r) is a tautology!
29
![Page 30: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/30.jpg)
Rules of Natural Deduction FR 3 (De Morgan’s law)
~(p v q) = (~p) ^ (~q)~(p ^ q) = (~p) v (~q)
FR 4 (Law of contrapositive)p q = (~q ~p)
FR 5 (Double Negation)~(~p) = p
30
![Page 31: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/31.jpg)
Examples of Arguments If a baby is hungry, then the baby cries. If the
baby is not mad, then he does not cry. If a baby is mad, then he has a red face. Therefore, if a baby is hungry, then he has a red face.
Model this problem!!
h: a baby is hungryc: a baby criesm: a baby is madr: a baby has a red face
31
h c~m ~cm r
. . . h r
h cc mm r
. . . h r
![Page 32: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/32.jpg)
Logic is the Skeleton
What remains when arguments are symbolized is the bare logical skeleton
It is this form that enables us to analyze the program / code / software.
Software V&V = Logical proof & Logic error detection
32
![Page 33: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/33.jpg)
Q4. Problem:
“Tom is a math major but not computer science major”M: Tom is a math majorC: Tom is a computer science major
Tasks:
Use De Morgan's Law to write the negation of the above statement as logic expression
![Page 34: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/34.jpg)
Answer:
Original: M Λ ¬ C (Tom is a math major but not
computer science major)
Negation: ¬ (M Λ ¬ C) = ¬ M V ¬ (¬ C) (De Morgan's
Laws) = ¬ M V C (Double negation rule)
34
![Page 35: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/35.jpg)
BLACK-BOX TESTING
&
WHITE-BOX TESTING
![Page 36: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/36.jpg)
Differences Between BB and WB
36
![Page 37: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/37.jpg)
Black-Box Testing
37
![Page 38: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/38.jpg)
White-Box testing
38
![Page 39: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/39.jpg)
REVIEW OF BB-testing
![Page 40: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/40.jpg)
Black-box Testing
1. Input Space Partitioning
2. Boundary Value Analysis
40
![Page 41: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/41.jpg)
Example 1: compare two numbers – p50 of week3
Function ‘Compare (x, y)’
Inputs: Two numbers – x and y Outputs: A larger number between x and y
41
Compare (x, y) = z
(x, y) z
![Page 42: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/42.jpg)
– p51 of week3
42
• Equivalence Classes:
{ (x, y) | x < y }
{ (x, y) | x > y }
{ (x, y) | x = y }
{ input other than a pair of numbers,
“as&%dfget^$(&w” }
Valid inputs
Invalid inputs
![Page 43: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/43.jpg)
43
Valid (x, y) Input Space
x = y
x < y
x > y
Three test cases:
(1, 2) --- 2
(8, 8) --- 8
(100, 30) --- 100
Plus one test cases:
(^&%*) --- ERROR
– p52 of week3
![Page 44: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/44.jpg)
Example 2: Loan application- p53 of week3
44
Customer Name
Account number
Loan amount requested
Term of loan
Monthly repayment
Term:
Repayment:
Interest rate:
Total paid back:
6 digits, 1stnon-zero
$500 to $9000
1 to 30 years
Minimum $10
2-64 chars.
Choosing (or defining) partitions seems easy, but is easy to get wrong…
![Page 45: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/45.jpg)
45
Customer name
Number of characters:
2 64 65invalid valid invalid
1
Conditions ValidPartitions
InvalidPartitions
ValidBoundaries
InvalidBoundaries
Customername
2 to 64 charsvalid chars
< 2 chars> 64 charsinvalid chars
2 chars64 chars
1 chars65 chars0 chars
Valid characters:Anyother
A-Za-z-’
space
![Page 46: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/46.jpg)
46
Loan amount
500 9000 9001
invalid valid invalid
499
Conditions ValidPartitions
InvalidPartitions
ValidBoundaries
InvalidBoundaries
Loanamount
500 - 9000 < 500>90000non-numericnull
5009000
4999001
![Page 47: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/47.jpg)
WB Testing Techniques
Logic coverage: (learned in previous classes) Statement coverage Branch coverage Condition coverage …
Dataflow based testing / Path coverage: all program paths have been traversed at least once
47
![Page 48: CS 217 Software Verification and Validation Week 9, Summer 2014 Instructor: Dong Si dsi](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e615503460f94b5c096/html5/thumbnails/48.jpg)
Pseudo code & Control/Dataflow Graphs
48
“nodes”
“edges”Input
output
Absolute (x){ IF (x>=0) THEN y = x;
ELSE IF (x<0) THEN y = -x;
Output y;}
x
IF (x>=0) ELSE IF (x<0)
y = x; y = -x;
y