cs 267: automated verification lecture 18, part 2: data model...
TRANSCRIPT
CS 267: Automated Verification Lecture 18, Part 2: Data Model Analysis for Web Applications Instructor: Tevfik Bultan
WebApplicationDependability
2
WebApplicationDependability
3
WebApplicationDependability
President Obama: “I want to go in and fix myself, but I don't write code"
• TRACKS:Atodolistapplica2on
WebApplicationDependability
5
Context Recurring Todo
Feed the Dog EDIT
WebApplicationArchitecture
• ModelViewController(MVC)pa;ern:RubyonRails,ZendforPHP,CakePHP,StrutsforJava,DjangoforPython,…• ObjectRela2onalMapping(ORM)Ac2veRecord,Hibernate,…
6
RESTful Controller
View
OOP Rel Db ORM
Data Model
AnExampleRailsDataModel
7
classUser<ActiveRecord::Base
has_many:todos
has_many:projects
end
classProject<ActiveRecord::Base
belongs_to:user
has_many:todos
has_many:notes
end
classTodo<ActiveRecord::Base
belongs_to:user
belongs_to:project
end
classNote<ActiveRecord::Base
belongs_to:project
end
classProjectsController<ApplicationController
defdestroy
@project=Project.find(params[:project_id])
@project.notes.eachdo|note|
note.delete
end
@project.delete
respond_to(...)
end
end
Static Data Model
Data Model Updates: Actions
StaticDataModel
• Ac2veRecordclassdeclara2ons• setsofobjects
• Ac2veRecordassocia2ondeclara2ons• has_one,has_many,belongs_to,has_and_belongs_to_many
• Associa2ondeclara2onscanbeusedtodeclarethethreebasictypesofrela2onsbetweenclasses• one-to-one• one-to-many• many-to-many 8
ExtensionstoStaticDataModel• :throughOp2on
• Toexpressrela2onswhicharecomposi2onofotherrela2ons
• :condi2onsOp2on• Torelateasubsetofobjectstoanotherclass
• :polymorphicOp2on• Toexpresspolymorphicrela2onships
• :dependentOp2on• Ondelete,thisop2onexpresseswhethertodeletetheassociatedobjectsornot
9
The:throughOptionclass User < ActiveRecord::Base
has_one :profile
has_many :photos, :through => :profile end
class Profile < ActiveRecord::Base
belongs_to :user
has_many :photos
end
class Photo < ActiveRecord::Base belongs_to :profile
end Profile
User Photo
*
*
1 1
1
1
10
The:dependentOption
• :delete directlydeletetheassociatedobjectswithoutlookingatitsdependencies
• :destroy firstcheckswhethertheassociatedobjectsthemselveshaveassocia2onswiththe:dependentop2onset
class User < ActiveRecord::Base has_one :profile, :dependent => :destroy end class Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroy end
PhotoProfileUser * 1 11
11
DataModelVeri@ication
• Formalizethesta2cdatamodelas• Asetofclasses• Asetofrela2onsbetweenthoseclasses• Asetofconstraintsontherela2onsthatareimposedbytheassocia2ondeclara2ons
• Givenaformaldatamodelwecanautoma2callycheckifagivenpropertyholdsforthedatamodel• Automatedverifica2ondetermines:Dotheconstraintsofthedatamodelimplytheproperty?
12
DataModelVeri@ication
AlloyEncoder
instanceorunsat
formula
formaldatamodel+property
AlloyAnalyzer
Property
Ac+veRecord
SMTSolver
instanceorunsatorunknown
formulaSMT-LIBEncoder
PropertyFailed+Counterexample
PropertyVerified
Unknown
ModelExtrac2on
ResultsInterpreter
ResultsInterpreter
BOUNDEDVERIFICATION UNBOUNDEDVERIFICATION
n Bound
bound
13
HowAutomatedisAutomatedVeri@ication?• Allexceptonestep:Propertyspecifica2on• Example:ItispossibletohaveaUserwhodoesnothaveanyPhotos.• InAlloy:
predprop{alls:PreState|someu:User|allp:Photo|
• InSMT-LIB:
• Canwemakeiteasier?
14
(pnotin(s.photo_user).u)}
(assert(exists((aPolymorphicClass))(forall((pPhoto))(and(isUsera)(not(=p(auser_photop)))))))
PropertyTemplates
• Propertytemplatesforpropertyspecifica2on• Language-neutral• DonotrequirefamiliaritywithSMT-LIBandAlloy
• Examplepropertytemplate:• noOrphans[classA,classB]
• Tocheckthatdele2nganobjectfromclassAdoesnotcauserelatedobjectsinclassBtobeorphaned
• Easilyreruntoolandswitchtheverifica2ontechnique,withouthavingtorewritetheproperty
• Wedevelopedsevenpropertytemplatesforthemostcommondatamodelproper2es
15
CanWeDoMore?
AutomaticPropertyInference
• Automa2callyinferproper2esbasedondatamodelschema• Datamodelschema:Adirected,annotatedgraphthatrepresentstherela2ons
• Lookforpa;ernsinthedatamodelschemaandinferapropertyifapa;ernthatcorrespondstoapropertyappears
• Forexample,orphanpreven2on
17
0 1 n. . .
. .
.
CanWeDoEvenMore?
• noOrphans(X,Y)propertyfailingmeansdele2nganobjectfromclassXcreatesanorphanchainthatstartswithassociatedobjectinclassY
• Repair:Set:dependent op2onto:destroy onassocia2ondeclara2oninclassXandonremainingrela2onsinthechainthatstartswithclassY
Set :dependent => :destroy on all relations in chain
X Y N. . .
. .
.
18
AutomatedDataModelRepair
Summary
ModelExtrac2on Verifica2on
Verifica2onResults
Ac2veRecords
FormalDataModel+Proper2es
PropertyInference
FormalDataModel
19
DataModelRepair
forfailingproper2es
ExperimentResultsApplica+on PropertyType #Inferred #Timeout #Failed
LovdByLessdeletePropagates 13 0 10
noOrphans 0 0 0
transi2ve 1 0 1
SubstructdeletePropagates 27 0 16
noOrphans 2 0 1
transi2ve 4 0 4
TracksdeletePropagates 15 0 6
noOrphans 1 0 1
transi2ve 12 0 12
FatFreeCRMdeletePropagates 32 1 19
noOrphans 5 0 0
transi2ve 6 2 6
OSRdeletePropagates 19 0 12
noOrphans 1 0 1
transi2ve 7 0 7
TOTAL 145 3 96
20
PropertyType#DataModel&Applica+on
Errors
#DataModelErrors
#FailuresDuetoRails
Limita+ons
#FalsePosi+ves
deletePropagates 1 9 0 0
noOrphans 0 0 0 0
transi2ve 0 0 0 1
deletePropagates 1 3 5 7
noOrphans 0 1 0 0
transi2ve 0 1 0 3
deletePropagates 1 1 3 1
noOrphans 0 0 0 1
transi2ve 0 7 0 5
deletePropagates 0 18 1 0
noOrphans 0 0 0 0
transi2ve 0 0 0 6
deletePropagates 0 12 0 0
noOrphans 0 1 0 0
transi2ve 0 7 0 0
TOTAL 3 60 9 28
21
WhatAboutDataModelActions?
22
classUser<ActiveRecord::Base
has_many:todos
has_many:projects
end
classProject<ActiveRecord::Base
belongs_to:user
has_many:todos
has_many:notes
end
classTodo<ActiveRecord::Base
belongs_to:user
belongs_to:project
end
classNote<ActiveRecord::Base
belongs_to:project
end
classProjectsController<ApplicationController
defdestroy
@project=Project.find(params[:project_id])
@project.notes.eachdo|note|
note.delete
end
@project.delete
respond_to(...)
end
end
Static Data Model
Data Model Updates: Actions
Veri@icationofDataModelActions
23
AbstractDataStores
24
classUserhas_many:todoshas_many:projectsendclassProjectbelongs_to:userhas_many:todoshas_many:notesendclassTodobelongs_to:userbelongs_to:projectendclassNotebelongs_to:projectend
classUser{0+Todotodosinverseofuser0+Projectprojectsinverseof
user}classProject{0..1Useruser0+Todotodosinverseofproject0+Notenotesinverseofproject}classTodo{0..1Useruser0..1Projectproject}classNote{0..1Projectproject}
Rails Abstract Data Store
AbstractDataStores
25
defproject_destroy@project=Project.find(
params[:project_id])@project.notes.eachdo|note|[email protected]_to(...)end
actionproject_destroy(){at_project=
oneof(allof(Project))foreachnote:at_project.notes{deletenote}deleteat_project}
invariant(forall{|project|!project.user.empty?})
invariant(forall{|user|user.projects.todos.include?(user)})
forall(Projectproject:notempty(project.user))
forall(Useruser:userinuser.projects.todos.users)
Our library allows developers to specify invariants in native Ruby
Extraction
Extrac2onishardforac2ons• Dynamictypesystem• Metaprogramming• Eval• GhostMethodssuchas:User.find_by_name(‘Rob’)Observa2ons• Theschemaissta2c• Ac2ondeclara2onsaresta2c• ORMclassesandmethodsdonotchangetheirseman2c
duringexecu2on• eveniftheimplementa2oncodeisgenerateddynamically
26
ExtractionviaInstrumentedExecution
• Boot-uptheRailsrun2meinasimulatedenvironment• Withoutopeningsocketsorconnec2ngtothedatabase
• Prepareac2onmethodsforextrac2on• ORMopera2onswillrecordtheirinvoca2oninsteadof
communica2ngwiththedatabase• Methodcallspropagateinstrumenta2onjustbefore
execu2on• Extrac2onispathinsensi2ve,execu2ngbothbranches
subsequently
• TriggeranHTTPrequestthattriggersanac2on27
Veri@icationviaTranslationtoFOL
• Apredicateisgeneratedforeachclassandassocia2onUser(o)meansthatoisaninstanceofUserProject_user(t)meansthattrepresentsanassocia2onbetweenaProjectobjectandUserobject
• Typesystemconstraintsbecomeaxioms∀u:User(u)→¬(Project(u)∨Todo(u)...)
• Cardinalityofassocia2onsisexpressedthroughaxiomseg.0..1:∀t1,t2:(Project_user(t1)∧Project_user(t2)∧Project_user_lhs(t1)=Project_user_lhs(t2))
→Project_user_rhs(t1)=Project_user_rhs(t2)28
TranslationofStatementstoFOL
• Anac2onisasequen2alcomposi2onofstatements.• Statements
• Astateisrepresentedwithapredicatedeno2ngallen22esthatexistinastate
• Astatementisamigra2onbetweenstatese.g.,acreateNotestatement:¬pre_state(newly_created())¬∃t:post_state(t)∧Note_project_lhs(t)=newly_created()∀o:(post_state(o)↔(pre_state(o)∨o=newly_created()))
29
TranslationofLoopstoFOL
• WeonlysupportForEachloops(fornow)• Theycorrespondtouniversalquan2fica2on
• Statementscanexecutemul2ple2mesinloops• Contextstodifferen2ateitera2ons
• Orderingofitera2ons• Itera2oninterdependence
30
InductiveVeri@ication
• Inv(s)isaformuladeno2ngthatallinvariantsholdinstates
• Ac3on(s,s’)isaformuladeno2ngthattheac2onmaytransi2onfromstatestostates’
Checkif: ∀s,s’:Inv(s)∧Ac2on(s,s’)→Inv(s’)
31
Experiments
Experimentedon3opensourceRailsapplica2ons• FatFreeCRM,Tracks,Kandan• 272 actions, 23 invariants Iden2fied4bugs• Reportedtooriginaldevelopers• Allimmediatelyconfirmedand,since,fixed• Missedbypreviousverifica2oneffortsontheseapplica2ons
32
Experiments
33
Publications• JaideepNijjarandTevfikBultan.BoundedVerifica+onofRubyonRailsDataModels.InProc.Interna3onalSymposiumonSo>wareTes3ngandAnalysis(ISSTA),pages67–77,2011.• JaideepNijjarandTevfikBultan.UnboundedDataModelVerifica+onUsingSMTSolvers.InProc.27thIEEE/ACMInt.Conf.AutomatedSo>wareEngineering(ASE),pages210–219,2012.• JaideepNijjar,IvanBocicandTevfikBultan.AnIntegratedDataModelVerifierwithPropertyTemplates.InProc.1stFMEWorkshoponFormalMethodsinSo>wareEngineering(FormaliSE2013).• JaideepNijjarandTevfikBultan.DataModelPropertyInferenceandRepair.InProc.Interna3onalSymposiumonSo>wareTes3ngandAnalysis(ISSTA),pages202—212,2013.• IvanBocic,andTevfikBultan.Induc+veVerifica+onofDataModelInvariantsforWebApplica+ons.InProc.Interna3onalConferenceonSo>wareEngineering(ICSE),2014• JaideepNijjar,IvanBocic,andTevfikBultan.DataModelPropertyInference,Verifica+onandRepairforWebApplica+ons.(Submi;edtoACMTransla3onsonSo>wareEngineeringandMethodology).
34