Mobility, Security, andProof-Carrying Code

Peter LeeCarnegie Mellon University

Lecture 1

Course Overview

July 10, 2001

Lipari School on Foundations of Wide Area Network Programming

Opportunities and Challenges

Arianne 5

On June 4, 1996, the Arianne 5 took off on its maiden flight.

40 seconds into its flight it veered off course and exploded.

It was later found to be an error in reuse of a software component.

For the next two years, virtually every research presentation used this picture.

“Better, Faster, Cheaper”

In 1999, NASA lost both the Mars Polar Lander and the Climate Orbiter.

Later investigations determined software errors were to blame.

Orbiter: Component reuse error.

Lander: Precondition violation.

USS Yorktown

“After a crew member mistakenly entered a zero into the data field of an application, the computer system proceeded to divide another quantity by that zero. The operation caused a buffer overflow, in which data leaked from a temporary storage space in memory, and the error eventually brought down the ship's propulsion system. The result: the Yorktown was dead in the water for more than two hours.”

Programmable mobile devices

By 2003, one in five people will own a mobile communications device.

Nokia expects to sell 500M Java-enabled phones in 2003.

Most of these devices will be power and memory limited.

Observations

Failures often due to simple problems “in the details.”

Reuse is critical but perilous.

Performance still matters a lot.

Safety Engineering

Small theorems about large programs would be useful.

Need clearly specified interfaces and checking of interface compliance.

Must not sacrifice performance.

But in the Real World?

Security Attacks

According to CERT, the majority of security attacks exploit

input validation failure

buffer overflow

VBShttp://www.cert.org/summaries/CS-2000-04.html

BSOD embarrassments

Warrantees?

LIMITED WARRANTY. Microsoft warrants that (a) the SOFTWARE PRODUCT will perform substantially in accordance with the accompanying written materials for a period of ninety (90) days from the date of receipt, …

LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENTPERMITTED BY APPLICABLE LAW, IN NO EVENT SHALLMICROSOFT OR ITS SUPPLIERS BE LIABLE FOR ANYSPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIALDAMAGES WHATSOEVER (INCLUDING, …) ARISING OUT OF THE USE OF … THE SOFTWARE PRODUCT…MICROSOFT’S ENTIRE LIABILITY … SHALL BE LIMITED TO THE GREATER OF THE AMOUNT ACTUALLY PAID BY YOU FOR THE SOFTWARE PRODUCT OR U.S. $5.00; PROVIDED...

Automotive Analogy

“If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and ...

Automotive Analogy

“If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year killing everyone inside."

- Robert Cringely

Mobile/Wireless Devices

In ‘97, 101M mobile phones vs 82M PCs. (40% vs 14%.)

95% phones will be WAP enabled by ‘04.

64Mbits of RAM in 2002.

Battery life a primary factor.

Efficiency and bandwidth will still be precious.

Bluetooth

670M Bluetooth-enabled devices by ‘03.

70% of mobile phones Bluetooth-enabled by ‘04.

Priceline.com’s grocery-store scenario.

Commercial world creates demand for “push” technologies.

Networked Appliances

By far the largest-growing segment.

Enormous diversity of platforms.

Reliability and longevity are expected.

Major challenges for OS and language standards.

Commercial Demands

Performance.

Mobility/extensibility.

Reliability/quality.

Well-defined languages.

Scalable security.

Opportunities

High assurance depends fundamentally on our ability to reason about programs.

The opportunities for computational logic, type theory, and formal semantics are great.

Challenges

The impact and cost of software failures will increase, as will the demand for extensibility.

The distinction between “safety-critical” and “consumer electronics” software will fade away.

Somebody will provide technology for “safe” systems. Will it be us?

Is the World Ready?

Is the World Ready?

What we start with:

What we want:

What we get along the way:

Is the World Ready?

What we start with:

What we want:

What we get along the way:

                                                                        

Cheese and the Sum Total of Human Knowledge

The Code Safety Problem

The Code Safety Problem

Please install and execute this.

Code Safety

CPU

Code

Trusted Host

Is this safe to execute?

Approach 1Trust the Code Producer

CPU

Code

Trusted Host

sig

Trusted 3rd Party

PK1

PK1

PK2

PK2

Trust is based on personal authority, not program properties

Scaling problems?

Approach 2Baby-sit the Program

CPU

Code

Trusted Host

Execution monitor

Expensive

Limited in expressive power(Why?)

E.g., Software Fault Isolation [Wahbe & Lucco], Inline Reference Monitors [Schneider]

Approach 3Java

CPU

Code

Trusted Host

Interp/ JIT

Expensive and/or big

Limited in expressive power

Verifier

TheoremProver

Approach 4Formal Verification

CPU

Code

Flexible andpowerful.

Trusted Host

But really reallyreally hard andmust be correct.

A Key Idea: Explicit Proofs

CertifyingProver

CPU

ProofChecker

Code

Proof

Trusted Host

A Key Idea: Explicit Proofs

CertifyingProver

CPU

Code

Proof

No longer need totrust this component.

ProofChecker

Proof-Carrying Code[Necula & Lee, OSDI’96]

A

B

Formal proof or“explanation” of safety

Typically nativeor VM code

rlrrllrrllrlrlrllrlrrllrrll…

Proof-Carrying Code

CertifyingProver

CPU

Code

Proof

Simple,small (<52KB),and fast.

No longer need totrust this component.

ProofChecker

Reasonable in size (0-10%).

But...

...How to generate the proofs?

Proving theorems about real programs is hard.

Most useful safety properties of low-level programs are undecidable.

Theorem-proving systems are unfamiliar to programmers and hard to use even for experts.

The Role ofProgramming Languages

Civilized programming languages can provide “safety for free”.

Well-formed/well-typed safe.

Idea: Arrange for the compiler to “explain” why the target code it generates preserves the safety properties of the source program.

The Role ofJava in this Short Course

Java will be the main focus of the PCC examples in this course.

Java is just barely a civilized programming language.

We can and should do better.

Java

Java is a worthwhile subject of research.

However, it contains many outrageous and mostly inexcusable design errors.

As researchers, we should not forget that we have already done much better, and must continue to do better in the future.

Certifying Compilers[Necula & Lee, PLDI’98]

Intuition:

Compiler “knows” why each translation step is semantics-preserving.

So, have it generate a proof that safety is preserved.

“Small theorems about big programs.”

Don’t try to verify the whole compiler, but only each output it generates.

Automation viaCertifying Compilation

CertifyingCompiler

CPULooks and smells like a compiler.

% spjc foo.java bar.class baz.c -ljdk1.2.2

Sourcecode

Proof

Objectcode

CertifyingProver

ProofChecker

Overview of the Necula/Lee Approach to PCC

Note

Our current approach seems to work for many problems.

But it is the only one we have tried — there are many others.

PCC is a general concept and we have just barely scratched the surface.

Overview of Our Approach

Please install and execute this.

OK, but let me quickly look over the instructions first.

Code producer Host

Overview of Our Approach

Code producer Host

Overview of Our Approach

This store instruction is dangerous!

Code producer Host

Overview of Our Approach

Can you prove that it is always safe?

Code producer Host

Overview of Our Approach

Can you prove that it is always safe?

Yes! Here’s the proof I got from my certifying Java compiler!

Code producer Host

Overview of Our Approach

Your proof checks out. I believe you because I believe in logic.

Code producer Host

Course Overview

This Short Course

This short course will focus on the concept of proof-carrying code.

PCC addresses code safety issues.

Reducing the trusted computing base.

Introducing a concept of “proof engineering”.

Exploiting modern ideas in compiling, theorem-proving, and logic programming.

Proof Engineering

This course will spend much of its time on engineering matters.

In particular, the problems of “scaling up” ideas to handle realistic problems.

A completely formal or systematic understanding of many of the concepts has not yet been attained.

Outline

In four parts:

0) Introduction and informal overview.

1) Safety infrastructure: proof representation and checking.

2) Verification and programming tools.

3) System engineering and related work.

Summary

Summary

The code safety problem presents great opportunities and challenges for applied logic and programming language design.

Proof-carrying code may be an example of how current knowledge can be applied to practical problems.

Homework Exercise 1

CertifyingCompiler

CPU

Sourcecode

Proof

Objectcode

CertifyingProver

ProofChecker

The architecture shown in this lecture has the compiler and prover as separate communicating components. An alternative would be to have a single component that compiles and proves simultaneously.

What are some advantages and disadvantages of the separate-component approach?