cooperative verification [3ex] blackcpachecker-workshop ... · cooperativeverification...
TRANSCRIPT
![Page 1: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/1.jpg)
Cooperative Verification
CPAchecker-Workshop 2019, Chiemsee, 2019-10-01
Dirk Beyer
LMU Munich, Germany
Dirk Beyer LMU Munich, Germany 1 / 30
![Page 2: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/2.jpg)
Automatic Software Verification
Program
SpecificationVerifier
Witness
ProgramSpecificationResult (True/False)
Dirk Beyer LMU Munich, Germany 2 / 30
![Page 3: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/3.jpg)
Competitions in Software Verification and Testing
I SV-COMP: off-site, automatic tools, controlled [1]I Test-Comp: off-site, automatic tools, controlled [3]
Dirk Beyer LMU Munich, Germany 3 / 30
![Page 4: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/4.jpg)
SV-COMP (Automatic Tools 2012)
Dirk Beyer LMU Munich, Germany 4 / 30
![Page 5: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/5.jpg)
SV-COMP (Automatic Tools 2013, cumulative)
Dirk Beyer LMU Munich, Germany 5 / 30
![Page 6: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/6.jpg)
SV-COMP (Automatic Tools 2014, cumulative)
Dirk Beyer LMU Munich, Germany 6 / 30
![Page 7: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/7.jpg)
SV-COMP (Automatic Tools 2015, cumulative)
Dirk Beyer LMU Munich, Germany 7 / 30
![Page 8: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/8.jpg)
SV-COMP (Automatic Tools 2016, cumulative)
Dirk Beyer LMU Munich, Germany 8 / 30
![Page 9: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/9.jpg)
SV-COMP (Automatic Tools 2017, cumulative)
Dirk Beyer LMU Munich, Germany 9 / 30
![Page 10: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/10.jpg)
SV-COMP (Automatic Tools 2018, cumulative)
Dirk Beyer LMU Munich, Germany 10 / 30
![Page 11: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/11.jpg)
SV-COMP (Automatic Tools 2019, cumulative)
Dirk Beyer LMU Munich, Germany 11 / 30
![Page 12: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/12.jpg)
What is the best verifier?
I Many different kinds of programs seem to require manydifferent good tools with different strengths
Dirk Beyer LMU Munich, Germany 12 / 30
![Page 13: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/13.jpg)
SV-COMP (Automatic Tools)
https:/
/sv-comp.sosy-la
b.org/2019/results
Dirk Beyer LMU Munich, Germany 13 / 30
![Page 14: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/14.jpg)
Which techniques are used?
CompetitionRe
port
[2]
https:/
/doi.org/10.1007/978-3-030-17502-3_9
Dirk Beyer LMU Munich, Germany 14 / 30
![Page 15: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/15.jpg)
Algorithms
17 Bounded Model Checking13 CEGAR8 Predicate Abstraction5 k-Induction4 Symbolic Execution3 Automata-Based Analysis2 Property-Directed Reachability (IC3)
Dirk Beyer LMU Munich, Germany 15 / 30
![Page 16: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/16.jpg)
Abstract Domains
24 Bit-Precise Analysis10 Explicit-Value Analysis9 Numerical Interval Analysis4 Shape Analysis1 Separation Logic
Dirk Beyer LMU Munich, Germany 16 / 30
![Page 17: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/17.jpg)
Testing
I Fuzzing (VeriFuzz [12], based on AFL)I Symbolic execution (KLEE [11])I Software model checking (CoVeriTest [9], → Poster)
Dirk Beyer LMU Munich, Germany 17 / 30
![Page 18: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/18.jpg)
Cooperative Verification
Dirk Beyer LMU Munich, Germany 18 / 30
![Page 19: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/19.jpg)
Approaches for Combinations
Verification Approach
Basic Combination
Black Box White Box
Portfolio Selection Cooperative Conceptual Integration
Dirk Beyer LMU Munich, Germany 19 / 30
![Page 20: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/20.jpg)
Conditional Model Checking
Conditional
Verifier A
P |= ϕ?
Conditional
Verifier B
P |= ϕ?
TRUE
FALSE
Program P
TRUE under condition ψ
Condition ψ
Proc. FSE 2012 [8]
Dirk Beyer LMU Munich, Germany 20 / 30
![Page 21: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/21.jpg)
Reducer-Based ConstructionVerifier B Conditional Verifier B
?
Reducer
Condition
Input Program
Verifier B
Construction
Residual Program
Reducer (preprocessor)I Builds standard input (C program)I Representing a subset of pathsI Contains at least all non-verified paths
+ Verifier-unspecific approach+ Many conditional verifiers possible
Proc. ICSE 2018 [10]Dirk Beyer LMU Munich, Germany 21 / 30
![Page 22: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/22.jpg)
Reducer-Based ConstructionVerifier B Conditional Verifier B
?
Reducer
Condition
Input Program
Verifier B
Construction
Residual Program
Reducer (preprocessor)I Builds standard input (C program)I Representing a subset of pathsI Contains at least all non-verified paths
+ Verifier-unspecific approach+ Many conditional verifiers possible
Proc. ICSE 2018 [10]Dirk Beyer LMU Munich, Germany 21 / 30
![Page 23: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/23.jpg)
Reducer-Based ConstructionVerifier B Conditional Verifier B
?
Reducer
Condition
Input Program
Verifier B
Construction
Residual Program
Reducer (preprocessor)I Builds standard input (C program)I Representing a subset of pathsI Contains at least all non-verified paths
+ Verifier-unspecific approach+ Many conditional verifiers possible
Proc. ICSE 2018 [10]Dirk Beyer LMU Munich, Germany 21 / 30
![Page 24: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/24.jpg)
Reducer-Based ConstructionVerifier B Conditional Verifier B
?
Reducer
Condition
Input Program
Verifier B
Construction
Residual Program
Reducer (preprocessor)I Builds standard input (C program)I Representing a subset of pathsI Contains at least all non-verified paths
+ Verifier-unspecific approach+ Many conditional verifiers possible
Proc. ICSE 2018 [10]Dirk Beyer LMU Munich, Germany 21 / 30
![Page 25: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/25.jpg)
Software Verification with Witnesses
Program
SpecificationVerifier
Witness
ProgramSpecificationResult (True/False)
Proc. FSE 2015, 2016 [6, 5]
Dirk Beyer LMU Munich, Germany 22 / 30
![Page 26: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/26.jpg)
Witness-Based Result Validation
Witness
ProgramSpecificationResult (True/False) Validator Result (True/False)
I Validate untrusted resultsI Easier than full verification
Dirk Beyer LMU Munich, Germany 23 / 30
![Page 27: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/27.jpg)
Stepwise Refinement
Witness
ProgramSpecificationResult (True/False)
WitnessTestifier
Witness
ProgramSpecificationResult (True/False)
Dirk Beyer LMU Munich, Germany 24 / 30
![Page 28: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/28.jpg)
Execution-based Witness Validation
CProgram
Specification
VerificationTask
Verifier
Proof found
Witness
CPA-w2t FShell-w2t
witness2testC
TestHarness
gcc
Executable
runexec
Witness Spurious Witness Confirmed
FalseBug foundTrue
Bug foundNo bug
found
Proc. TAP 2018 [7]Made “Generating Tests from Counterexamples” more practical(Proc. ICSE 2004, [4])
Dirk Beyer LMU Munich, Germany 25 / 30
![Page 29: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/29.jpg)
Witness Creation
Program
Specification
VerificationTask
Verifier
Blast Cbmc CPAchecker Esbmc
Smack UltimateAutomizer · · ·
ViolationWitness
CorrectnessWitness
False
Bug foundTrue
Proof found
Dirk Beyer LMU Munich, Germany 26 / 30
![Page 30: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/30.jpg)
Graphical Visualization of the Coop Framework
Analyze
Conditional Verifier
p
ϕb
ψ
r
ω
ψ
Analyze
Verifier
p
ϕb
r
ω
Analyze
Validator
p
ϕb
ω
r
ω
Transform
Reducer
p
ψ
p’ Transform
Test Extractor
p
ϕb
ω
t Present
Test Executor
p
t
Transform Analyze
Conditional Verifier constructed fromReducer andVerifier
p
ψ
p’
ϕb
r
ω
Analyze Transform Present
Execution-BasedValidation constructed fromVerifier, Test Extractor, andTest Executor
p
ϕb
r
ω
t
Dirk Beyer LMU Munich, Germany 27 / 30
![Page 31: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/31.jpg)
Conclusion
I Software verification: successful past, bright futureI Competitions solve several problemsI Cooperating combinations are the future
Dirk Beyer LMU Munich, Germany 28 / 30
![Page 32: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/32.jpg)
References IBeyer, D.: Competition on software verification (SV-COMP). In: Proc. TACAS.pp. 504–524. LNCS 7214, Springer (2012).https://doi.org/10.1007/978-3-642-28756-5_38
Beyer, D.: Automatic verification of C and Java programs: SV-COMP 2019. In:Proc. TACAS, Part 3. pp. 133–155. LNCS 11429, Springer (2019).https://doi.org/10.1007/978-3-030-17502-3_9
Beyer, D.: Competition on software testing (Test-Comp). In: Proc. TACAS,Part 3. pp. 167–175. LNCS 11429, Springer (2019).https://doi.org/10.1007/978-3-030-17502-3_11
Beyer, D., Chlipala, A.J., Henzinger, T.A., Jhala, R., Majumdar, R.: Generatingtests from counterexamples. In: Proc. ICSE. pp. 326–335. IEEE (2004).https://doi.org/10.1109/ICSE.2004.1317455
Beyer, D., Dangl, M., Dietsch, D., Heizmann, M.: Correctness witnesses:Exchanging verification results between verifiers. In: Proc. FSE. pp. 326–337.ACM (2016). https://doi.org/10.1145/2950290.2950351
Beyer, D., Dangl, M., Dietsch, D., Heizmann, M., Stahlbauer, A.: Witnessvalidation and stepwise testification across software verifiers. In: Proc. FSE. pp.721–733. ACM (2015). https://doi.org/10.1145/2786805.2786867
Dirk Beyer LMU Munich, Germany 29 / 30
![Page 33: Cooperative Verification [3ex] blackCPAchecker-Workshop ... · CooperativeVerification CPAchecker-Workshop2019,Chiemsee,2019-10-01 DirkBeyer LMUMunich,Germany DirkBeyerLMUMunich,Germany1/30](https://reader033.vdocument.in/reader033/viewer/2022043012/5fa8065f338cfe6fd255250a/html5/thumbnails/33.jpg)
References IIBeyer, D., Dangl, M., Lemberger, T., Tautschnig, M.: Tests from witnesses:Execution-based validation of verification results. In: Proc. TAP. pp. 3–23.LNCS 10889, Springer (2018).https://doi.org/10.1007/978-3-319-92994-1_1
Beyer, D., Henzinger, T.A., Keremoglu, M.E., Wendler, P.: Conditional modelchecking: A technique to pass information between verifiers. In: Proc. FSE.ACM (2012). https://doi.org/10.1145/2393596.2393664
Beyer, D., Jakobs, M.C.: Coveritest: Cooperative verifier-based testing. In: Proc.FASE. pp. 389–408. LNCS 11424, Springer (2019).https://doi.org/10.1007/978-3-030-16722-6_23
Beyer, D., Jakobs, M.C., Lemberger, T., Wehrheim, H.: Reducer-basedconstruction of conditional verifiers. In: Proc. ICSE. pp. 1182–1193. ACM(2018). https://doi.org/10.1145/3180155.3180259
Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automaticgeneration of high-coverage tests for complex systems programs. In: Proc. OSDI.pp. 209–224. USENIX Association (2008)
Chowdhury, A.B., Medicherla, R.K., Venkatesh, R.: VeriFuzz: Program awarefuzzing (competition contribution). In: Proc. TACAS. LNCS 11429, Springer(2019)
Dirk Beyer LMU Munich, Germany 30 / 30