the future of correct software george necula. 2 software correctness is important ► where there is...
Post on 19-Dec-2015
217 views
TRANSCRIPT
The Future of The Future of Correct SoftwareCorrect Software
George NeculaGeorge Necula
2
Software Correctness is Software Correctness is ImportantImportant
►Where there is software, there are bugsWhere there is software, there are bugs
► It is estimated that software bugs cost the It is estimated that software bugs cost the economy over $60B a year (1% of GDP)economy over $60B a year (1% of GDP) Average cost of downtime can be $1M/hourAverage cost of downtime can be $1M/hour
►Software bugs are responsible for over Software bugs are responsible for over 50% of known security vulnerabilities50% of known security vulnerabilities
3
Software Correctness is HardSoftware Correctness is Hard►Social challengesSocial challenges
Customers still favor features and performanceCustomers still favor features and performance Programmers notoriously overconfidentProgrammers notoriously overconfident
►Economic challengesEconomic challenges Correctness costs more than extra featuresCorrectness costs more than extra features
►Technical challengesTechnical challenges Impossible to build perfect software quality tools Impossible to build perfect software quality tools False alarms and missed errors are facts of lifeFalse alarms and missed errors are facts of life
4
The Open Source Quality The Open Source Quality GroupGroup
►MembersMembers Rastislav Bodik, George Necula, Sanjit SeshiaRastislav Bodik, George Necula, Sanjit Seshia Collaborators at Stanford, Microsoft, IBM, IntelCollaborators at Stanford, Microsoft, IBM, Intel And 15 graduate studentsAnd 15 graduate students
► Develop techniques and tools for Develop techniques and tools for buildingbuilding, , deployingdeploying and and monitoringmonitoring quality software quality software
►Use Open Source software as a test bedUse Open Source software as a test bed
5
1. Building Correct Software1. Building Correct Software
►Tools can help only if we bring more Tools can help only if we bring more information in the software process information in the software process
►Find unobtrusive ways to get programmer Find unobtrusive ways to get programmer assistance with correctness reasoningassistance with correctness reasoning Programs express “how” things must be donePrograms express “how” things must be done Programmers know the “what” and “why” ! Programmers know the “what” and “why” !
►Example: Programming by sketchingExample: Programming by sketching by Rastislav Bodik, Sanjit Seshiaby Rastislav Bodik, Sanjit Seshia
6
The Sketching ExperienceThe Sketching Experience
sketch implementation (completed sketch)
spec
specification
+
7
Promising PropertiesPromising Properties
Sketched programs are developed …Sketched programs are developed …
rapidly:rapidly: the low-level details are the low-level details are synthesized automaticallysynthesized automatically
correctly: correctly: implementation guaranteed to implementation guaranteed to behave like the specificationbehave like the specification
8
Example: Sorting by handExample: Sorting by hand
int[] merge (int[] a, int b[], int n) {int[] merge (int[] a, int b[], int n) {for (int i = 0; i < n; i++)for (int i = 0; i < n; i++)
if ( if ( j<n && ( !(k<n) || a[j] < b[k])j<n && ( !(k<n) || a[j] < b[k]) ) { ) { result[i] = a[j]; j++; result[i] = a[j]; j++;
} else { } else { result[i] = b[k]; k++; result[i] = b[k]; k++;
}}}}return result;return result;
}}
► The devil is in the detailsThe devil is in the details
9
int[] merge (int[] a, int b[], int n) {int[] merge (int[] a, int b[], int n) {for (int i = 0; i < n; i++)for (int i = 0; i < n; i++)
if ( if ( synthesize( ||, &&, <, !, [] )synthesize( ||, &&, <, !, [] ) ) { ) { result[i] = a[j]; j++; result[i] = a[j]; j++;
} else { } else { result[i] = b[k]; k++; result[i] = b[k]; k++;
}}}}return result;return result;
}}► Sketch compiler fills in the details correctlySketch compiler fills in the details correctly
► Sketches are programs with Sketches are programs with missing detailsmissing details► Specifications can be slow/simple programsSpecifications can be slow/simple programs
Sorting Sorting sketchedsketched
hole
10
Experience with Sketching Experience with Sketching CiphersCiphers
User experiment: User experiment: goal:goal: implement a mini-cipher implement a mini-cipher how:how: C programmer C programmer vs.vs. sketching programmer sketching programmer
Results:Results: sketching programmer was twice as fastsketching programmer was twice as fast sketched cipher ran 50% fastersketched cipher ran 50% faster
Next: sketching for general purpose Next: sketching for general purpose programsprograms
11
2. Deploying Correct Software2. Deploying Correct Software
►Today’s view of software: Today’s view of software: Software is executableSoftware is executable
►Future view of software: Future view of software: Software is checkable and executableSoftware is checkable and executable
►We need to We need to redefine what software isredefine what software is:: Software = Executable content Software = Executable content
+ Assurance support+ Assurance support
12
Today: Digital SignaturesToday: Digital Signatures
CPU
Code
Consumer
Signature
SignatureCheckingTrust the code producer
• Not a behavioral assurance
• Dangerous !• Does not scale well
Good but not enough
13
Future: Semantic AssuranceFuture: Semantic Assurance
CPU
Code
Consumer
Safety Proof
ProofChecking
• Proof-carrying code• Provides semantic
assurance• Producer does the hard
work
Code producer “helps” the consumer to check the code
14
ChallengesChallenges
►How small can you make the proofs? How small can you make the proofs? Today about 25% of the code and shrinkingToday about 25% of the code and shrinking
►How do you generate proofs ?How do you generate proofs ? Certifying software synthesis tools (compilers)Certifying software synthesis tools (compilers) Automatic today for memory safety, resource Automatic today for memory safety, resource
usage constraintsusage constraints
►Next: make more software tools certifyingNext: make more software tools certifying
15
3. When Everything Else Fails3. When Everything Else Fails
► The future of correct software must The future of correct software must include incorrect softwareinclude incorrect software
► We must deal with execution errorsWe must deal with execution errors Monitoring, recovery, restarting, …Monitoring, recovery, restarting, …
► Example: Cooperative Bug IsolationExample: Cooperative Bug Isolation
16
Post-Deployment MonitoringPost-Deployment Monitoring
Cooperative Bug Isolation
17
Idea: Measure RealityIdea: Measure Reality
►Go beyond measuring crashesGo beyond measuring crashes►Monitor good and bad executionsMonitor good and bad executions
Spread cost of monitoring over many usersSpread cost of monitoring over many users Collect feedback data & mine for bug Collect feedback data & mine for bug
causescauses
►Actual user runs are a vast resourceActual user runs are a vast resource Number of real runs >> number of testing Number of real runs >> number of testing
runsruns Real-world executions are most importantReal-world executions are most important
18
Bug Isolation ArchitectureBug Isolation Architecture
ProgramSource Compiler
SamplerShipping
Application
Pro le/
StatisticalDebugging
Top bugs withlikely causes
19
Public Deployment in Public Deployment in ProgressProgress
0%
2%
4%
6%
8%
10%
Evolu
tion
Gaim
The GIM
P
Gnum
eric
Nautil
us
Rhythm
box
SPIM
success runs
failure runs
► Applications do have bugsApplications do have bugs► Attract more users for statistical analysisAttract more users for statistical analysis
20
ConclusionConclusion► Social factors will work in favor of Social factors will work in favor of
software correctness software correctness
► Technology must provide affordable Technology must provide affordable solutions for correctnesssolutions for correctness Bring more information into software processBring more information into software process Software synthesis from high-level Software synthesis from high-level
specificationsspecifications Software distributions with assurance supportSoftware distributions with assurance support
► Good error handling always important Good error handling always important